AI Explainedby Milad Fahmy

AI Explained

Synthetic data: training AI on data a model wrote

Synthetic data is training and test data generated by a model. How Self-Instruct, Alpaca and phi-1 used it, and why model collapse means you still need real data.

Milad FahmyNov 11, 20262 min read

On this page
  1. Why use it
  2. The classic recipe: Self-Instruct
  3. Alpaca: cheap instruction data
  4. phi-1: synthetic textbooks
  5. Synthetic labels
  6. The catch: model collapse
  7. Check the license
  8. Verify what you generate

Some of the best training data was never written by a person. A model wrote it. That's synthetic data: examples generated by a model and used to train, fine-tune or test another model.

Why use it

Real data is slow to collect, expensive to label, and often private. Synthetic data is fast, cheap, and made to order: you can ask for exactly the kind of examples you're missing.

The classic recipe: Self-Instruct

The Self-Instruct paper (Wang et al.) set out the basic loop:

  1. Start from a small set of seed tasks written by people.
  2. Have the model write new instructions, inputs and outputs.
  3. Filter out invalid examples and near-duplicates.
  4. Fine-tune on what's left.

Applied to GPT-3, that gave a 33% absolute improvement on the Super-NaturalInstructions benchmark, with almost no new human labels.

Alpaca: cheap instruction data

Stanford's Alpaca used the same idea. It started from 175 human-written seed tasks and generated 52,000 examples, then fine-tuned LLaMA 7B on them. Generating the data cost under $500.

phi-1: synthetic textbooks

Microsoft's phi-1 went further, training a code model on textbooks and exercises written by GPT-3.5: about 1 billion synthetic tokens, plus 6 billion tokens of filtered "textbook quality" web data. The model has only 1.3 billion parameters and scored 50.6% on HumanEval.

Synthetic labels

Generated data can be labels as well as examples. Constitutional AI trains a harmless assistant without human labels for harmful outputs. It uses AI feedback instead, guided by a written list of principles.

The catch: model collapse

Train on model output again and again, and the rare cases (the tails of the real distribution) disappear. Researchers call this model collapse and say it causes irreversible defects. That makes real human data more valuable, not less: mix it in, and don't train only on generated text.

Check the license

Some providers restrict what you can do with their output. OpenAI's terms, for example, forbid using output to develop models that compete with OpenAI.

Verify what you generate

  • Run the code to check that generated programs actually work.
  • Check the answers, with tests, a stronger model or spot checks by a person.
  • Deduplicate before you train.
Step Why
Generate Fast, cheap, made to order
Filter Drop invalid and duplicate examples
Keep it real Mix in human data to avoid collapse

Generate it, filter it, keep it real.

Sources

#synthetic data#llm#fine-tuning#model collapse#machine learning

Watch