Training capable AI models depends on a resource that is quietly running short: high-quality, specialized data. The open internet supplies effectively unlimited text and images for today’s generalist models, but the next wave of progress — in areas such as cybersecurity, legal reasoning, and healthcare — needs data that either does not exist in sufficient quantity or cannot be used freely because of privacy constraints. Simula, a framework introduced by researchers from Google and EPFL, addresses this with a reasoning-first approach to generating and evaluating synthetic datasets, emphasizing transparency, fine-grained control, and scalability. Rather than depending on seed data from the target distribution, hand-crafted prompts, or evolutionary search, it builds each dataset from first principles, treating data generation as a mechanism-design problem. The approach is detailed in the team’s research paper on reasoning-driven synthetic data.
Why synthetic data generation is harder than it looks
Anyone who has built fine-tuning pipelines knows that simply prompting a large model to “produce more examples” tends to yield narrow, repetitive data. The hard parts are coverage (spanning the full range of a domain), diversity (avoiding near-duplicates), complexity (including genuinely difficult cases, not just easy ones), and quality (ensuring the labels are actually correct). Simula’s central design choice is to treat quality, diversity, and complexity as independent axes that can be controlled separately rather than tuned all at once.
How Simula works: taxonomy, meta-prompts, and dual critics
The first stage handles global diversity. A capable multimodal teacher model enumerates the main factors of variation in a target domain and expands them into a hierarchical taxonomy using a best-of-N proposal step refined by a critic. This produces a structured map of what a domain actually contains, instead of relying on whatever a single prompt happens to surface.
The second stage handles local diversity. Combinations of taxonomy nodes — referred to as “mixes” — are passed to the teacher model to generate “meta-prompts,” scenarios that anchor a specific concept.
{house cat, poem, travel enthusiast}
To prevent mode collapse when several prompts are generated from the same set of nodes, Simula produces multiple meta-prompts at once and encourages them to differ, for example by varying the framing of an otherwise similar request.
c
A configurable fraction of meta-prompts then passes through a complexification step, in which the teacher model deliberately increases difficulty while preserving the other requirements. Separating this step is what lets complexity be dialed up or down independently of coverage.
Finally, quality is checked with a dual-critic design. Instead of asking the model once whether an answer is correct, Simula separately asks whether it is correct and whether it is incorrect. Requiring the two judgments to agree helps counter sycophancy bias — the tendency of language models to endorse plausible-sounding but wrong outputs.
What the experiments show
The researchers report testing Simula with Gemini 2.5 Flash (non-thinking) as the teacher and Gemma 3 4B as the student, running ten LoRA fine-tuning runs with different seeds per configuration and reporting average accuracy with 95% confidence intervals. They generated datasets of up to 512K examples across five domains: two cybersecurity tasks (CTI-MCQ and CTI-RCM), a legal exam set (LEXam, covering Swiss, EU, and international law in English and German), grade-school math (GSM8k), and a multilingual Global MMLU subset. Across datasets and sizes, the full system — combining global and local diversification with complexity control and dual-critic filtering — generally outperformed reduced versions of itself, and the team notes that the two diversification stages work best together rather than alone.
The results also expose where the method depends on the teacher. Critic rejection rates rose to 61% on LEXam versus around 2-9% on the other tasks, reflecting the teacher’s relative weakness on law; on that domain, lower-complexity data actually beat higher-complexity data because the teacher could not reliably label the harder cases. The team also describes student-teacher scaling effects: on one cybersecurity task the student closed most of the gap to the teacher and then saturated around 128K examples, while on GSM8k performance kept climbing without obvious saturation.
Rethinking how synthetic data is evaluated
Beyond generation, the researchers propose evaluation measures aimed at the properties that matter. Taxonomic coverage records what fraction of taxonomy nodes appear at each level, giving a structured view of breadth. By this measure, they report that real-world reference datasets often cover less of a target domain than Simula’s generated variants — even in cases where standard embedding-distance metrics suggest the opposite. The broader takeaway is that scaling behavior is driven by the properties of the data, not by sheer size alone.
Limitations and what to watch
Simula is a research framework, and its results come from the teams that built it, so independent replication will be important before treating the numbers as settled. Its quality ceiling is bound to the teacher model: when the teacher is weak in a domain, high-complexity generation and even label correctness suffer, as the legal-exam results show. Running multiple teacher-model passes for taxonomy building, meta-prompting, complexification, and dual-critic checking also carries real compute cost, and synthetic data can still encode the teacher’s blind spots and biases. For practitioners, the sensible reading is that structured, reasoning-driven generation is promising for data-scarce domains, but it should be validated against trusted real-world benchmarks and paired with careful human review in high-stakes settings. Teams building on such models may also find this overview of Python libraries for LLM applications useful.
