🧪 Frontier Agent Data Engineering

Synthetic Data for Agents: Trajectory Synthesis & Distillation

Real human agent trajectories are scarce, costly, and riddled with privacy risks. Learn how frontier labs use teacher models to synthesize millions of multi-turn tool calling traces with Evol-Instruct and rejection sampling.

Methodology: Teacher-Student Model Distillation

Verification: Unit-Test Sandbox Rejection Sampling

Cost Savings: 98% Cheaper than Human Annotation

Scenario Seed Gen

Generate diverse seed scenarios across 50+ enterprise tool schemas (Git operations, SQL queries, payment processing, Kubernetes clusters).

Evol-Instruct Stress

Mutate simple tasks into complex multi-hop challenges by injecting network failures, corrupted API responses, ambiguous parameters, and permission errors.

Execution Filter

Discard hallucinated runs: only trajectories where code executes successfully inside isolated microVM sandboxes pass into the training corpus.

1. Multi-Turn Trajectory Synthesis Loop

Python 3.11+

This pipeline orchestrates a frontier Teacher model (Claude 3.7 or GPT-4o) running inside a mock execution environment to generate validated Thought-Action-Observation trajectories:

import json
from openai import OpenAI

client = OpenAI()

def generate_agent_trajectory(tool_schema: dict, scenario: str) -> dict:
    system_prompt = f"""You are generating gold-standard agent training trajectories.
Target Tool Schema:
{json.dumps(tool_schema, indent=2)}

Generate a complete multi-turn trajectory for the user scenario: "{scenario}".
Rules:
1. Include explicit <thought> reasoning tokens before every tool call.
2. If tool returns an error, the agent must diagnose the error and retry with corrected parameters.
3. Output valid JSON formatted with keys: 'system', 'turns', 'is_verified_success'."""

    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[
            {"role": "system", "content": system_prompt},
            {"role": "user", "content": f"Generate trajectory for: {scenario}"}
        ],
        response_format={"type": "json_object"},
        temperature=0.7
    )
    
    trajectory = json.loads(response.choices[0].message.content)
    return trajectory

# Example Seed Scenario Generation
sample_tool = {
    "name": "manage_dns_records",
    "parameters": {
        "domain": "aiagent.org",
        "action": "CREATE",
        "record_type": "CNAME",
        "target": "cname.vercel-dns.com"
    }
}

trajectory = generate_agent_trajectory(sample_tool, "Migrate aiagent.org subdomain to Vercel with verification")
print(f"Synthesized {len(trajectory['turns'])} agent execution turns!")

2. Evol-Instruct Mutation Operators for Agents

In-Depth Evolution (Deepening Complexity)

Add Constraints: "Ensure database migration completes in under 2 seconds without locking the table."
Simulate Partial Outages: Inject HTTP 429 Rate Limit responses to train the agent on exponential backoff tool re-calls.
Multi-Tool Dependencies: Force the agent to chain 4 distinct tools (Fetch credentials -> Decrypt -> Query DB -> Generate PDF).

In-Breadth Evolution (Domain Expansion)

Cross-Domain Transfer: Convert DevOps container debugging scripts into AWS IAM permission auditing scripts.
Negative & Adversarial Scenarios: Inject indirect prompt injections into mock documents to train the model to safely reject malicious tool calls.
Non-Tool Plain Text Turns: Ensure model knows when NOT to call tools when simple conversational clarifications suffice.

3. Rejection Sampling & Sandbox Verification

The single most critical failure in synthetic data generation is unverified hallucinations: if a Teacher LLM hallucinates a fictitious CLI flag or invalid JSON syntax, your Student model will learn to reproduce those exact hallucinations.

1. AST & Schema Linter

Strict Pydantic / JSON Schema validation rejecting syntactically invalid tool calls.

2. Sandbox Run Check

Runs generated Python / Bash in an ephemeral Firecracker sandbox to verify exit code 0.

3. Critic Model Audit

A secondary Critic LLM audits the reasoning chain for logical consistency before export.

Frequently Asked Questions

Does training on synthetic data lead to model collapse? →

Model collapse only occurs with unverified, recursive plain-text generation. In AI agent trajectories, execution verification (unit tests, compilers, sandboxes, and database assertions) acts as an external ground-truth anchor, preventing degradation and frequently outperforming human annotator consistency.

How can I format synthetic trajectories for Direct Preference Optimization (DPO)? →

Pair successful sandbox-verified trajectories as the chosen response, and failed trajectories (those that hallucinated parameters or failed tests) as the rejected response. This teaches the student model both how to succeed and what common tool-calling pitfalls to avoid.