合成訓練數據需要血緣圖，而非單純增加數量

TL;DR — Teams scale synthetic data generation to fix data scarcity but almost never track generation lineage — which teacher model, prompt template, and filter produced each example. Without that lineage, curation becomes guesswork and distillation loops quietly entrench the teacher model's blind spots. The fix is treating synthetic data like a software supply chain artifact: versioned, manifested, and auditable.

Every team running short on labeled data eventually reaches for the same lever: generate more with a model. Use a strong teacher LLM to produce instruction-response pairs, synthetic rationales, or classification labels, then train a smaller model on the output. It works. It's fast. It's also accumulating a debt that most teams can't even see, because almost nobody tracks where a synthetic example actually came from.

That's the real failure mode in synthetic data pipelines today. Not quality, not scale — lineage. Teams treat synthetic data generation as a volume problem: generate more, filter harder, mix ratios, done. They rarely treat it as a provenance problem: which generator model, which prompt template, which filter, which sampling run produced this specific row that's now sitting in your training set next to a hundred thousand others.

The Distillation Loop Nobody Audits

When you generate training data with a teacher model, you aren't just transferring its knowledge. You're transferring its blind spots, stylistic tics, and failure modes, and you're doing it silently. If the teacher consistently botches multi-step arithmetic, hedges on ambiguous causal questions, or has a subtle bias in how it frames a particular class of entity, every synthetic example inherits that pattern. The student model doesn't just learn the task — it learns the teacher's specific shape of being wrong.

This would be manageable if teams could trace a bad eval result back to the generation batch that caused it. In practice they can't, because the synthetic data sits in the training set indistinguishable from human-labeled data, with no record of which teacher checkpoint, which prompt version, or which decoding parameters produced it. A regression shows up three fine-tuning runs later, and the only diagnostic tool available is re-reading thousands of examples by hand.

Compare this to how mature data engineering treats ETL: every transformation has a DAG, a run id, an input hash. Synthetic data generation is a transformation too — arguably the riskiest one in the pipeline, since it's injecting model-generated content as ground truth — and it's the one stage most teams run with zero lineage tracking.

Filters Compound the Problem, They Don't Fix It

The standard mitigation is a filtering step: a reward model, a classifier, or a rule-based heuristic that scores synthetic examples and drops the bad ones. This feels like quality control. Often it's actually variance reduction around the same bias, not correction of it.

If the filter model was trained on preferences elicited from outputs of the same teacher family, or fine-tuned on a dataset that already encodes the teacher's stylistic preferences, it will systematically favor examples that look like what the teacher would produce — fluent, confident, well-formatted — regardless of correctness. Errors that are stated smoothly survive the filter. Errors that are stated bluntly get cut. You end up with a cleaner-looking dataset that is not more correct, just more self-consistent with its own generation process.

This is the synthetic-data equivalent of a feedback loop in a recommender system: the filter and the generator are drawing from overlapping distributions, so the filter can't see outside the box the generator put it in. Real correction requires a signal external to the generation process — human review, a different model family, execution-based verification, or ground truth from the real world. Teams that skip this and trust a same-family filter are laundering bias, not removing it.

Human Labelers Anchor on Synthetic Fluency

The labeling layer makes this worse in a specific, under-discussed way. When human annotators review a mix of real and synthetic examples without knowing which is which, they don't review them with equal skepticism. Fluent, well-structured synthetic text reads as more trustworthy than messy real-world text, even when the synthetic example is factually wrong. Labelers anchor on surface polish. A rubric that asks "is this response good?" will systematically favor synthetic examples over real ones that are correct but awkwardly phrased.

The practical consequence: synthetic data doesn't just get generated with bias, it gets validated with bias, by the same human pipeline you're relying on to catch errors. If your annotation guidelines don't explicitly account for this — blind review, origin randomization, separate rubrics for synthetic versus organic sources — your labeling step is adding a false stamp of approval rather than independent verification.

What a Lineage-First Pipeline Actually Looks Like

The fix isn't more filtering. It's treating every synthetic example as a build artifact with a manifest, the same way software supply chains now require a bill of materials for every dependency. At minimum, that manifest should carry:

• Generator model identity and checkpoint hash, not just a model family name

• Prompt template version and the exact sampling parameters used

• Filter or reward model identity, version, and the threshold applied

• Seed example or source document the generation was conditioned on

• Acceptance and rejection rates for that batch, not just the surviving examples

With that manifest attached, curation stops being a single global quality bar and becomes something closer to a query. Eval regressed on a specific capability — pull every training example whose lineage traces to the teacher checkpoint and prompt version used before the regression, and check whether that batch is overrepresented. A labeling rubric turns out to systematically favor fluent text — pull every example that batch approved and cross-reference against a lineage-tagged origin field. None of this is possible if synthetic rows are indistinguishable from organic ones once they land in the training set.

Volume Was Never the Scarce Resource

Synthetic data generation solved the wrong scarcity. Raw example count was never the bottleneck once LLMs got good enough to generate plausible-looking text at scale. The bottleneck is knowing, after the fact, which examples are responsible for a model's specific behaviors — good or bad. A ten-million-row synthetic dataset with no lineage is strictly worse than a hundred-thousand-row dataset where every example can be traced to its generator, its filter, and its acceptance decision, because only the second one is debuggable.

Teams that keep scaling synthetic generation without building the lineage layer underneath it aren't building a bigger dataset. They're building a bigger black box, and they'll find out how big when the next eval regression takes three weeks to localize instead of three hours.