⚡ Specialized Model Training & LoRA

Agent Fine-Tuning: Tool Calling & Function Loops

Learn how to fine-tune compact open-weights models (Llama 3.1 8B, Qwen 2.5 7B) for deterministic tool execution, complex JSON schema adherence, and multi-turn agent reasoning loops using Unsloth, LoRA, and QLoRA.

Speedup: 5x Faster Training with Unsloth

Quantization: 4-bit QLoRA on a single 24GB GPU

Target Output: 99.8% Valid JSON Tool Calls

Sub-50ms Latency

General frontier models (GPT-4o, Claude 3.5 Sonnet) suffer from 400-800ms TTFT. Fine-tuning an 8B model cuts routing latency by 10x, enabling true real-time agent reflexes.

Zero Schema Hallucination

General models frequently add non-existent parameters or fail JSON punctuation. Fine-tuning encodes your private API schemas and tool schemas directly into model weights.

Self-Hosted Privacy

Deploy fine-tuned LoRA adapters directly onto air-gapped vLLM or Ollama clusters. Sensitive enterprise database queries and customer records never leave your VPC.

1. Tool-Calling Chat Template Format

Hermes / Llama 3 Style

To train models on multi-turn function calling, training data must use consistent special tokens separating system instructions, tool definitions, thoughts, and execution observations:

<|im_start|>system
You are an autonomous operations agent. You have access to the following tools:
<tools>
[
  {
    "name": "query_database",
    "description": "Execute a read-only SQL query on the Postgres cluster",
    "parameters": {
      "type": "object",
      "properties": {
        "query": {"type": "string", "description": "The SQL select query"}
      },
      "required": ["query"]
    }
  }
]
</tools>
If you need to call a tool, reply in the format: <tool_call>{"name": "...", "arguments": {...}}</tool_call>
<|im_end|>
<|im_start|>user
Check active connections count in the billing database.
<|im_end|>
<|im_start|>assistant
<thought>I need to query pg_stat_activity to find current connections.</thought>
<tool_call>{"name": "query_database", "arguments": {"query": "SELECT count(*) FROM pg_stat_activity;"}}</tool_call>
<|im_end|>
<|im_start|>tool
{"name": "query_database", "content": {"count": 42}}
<|im_end|>
<|im_start|>assistant
There are currently 42 active connections on the billing database cluster.
<|im_end|>

2. High-Speed Fine-Tuning with Unsloth & QLoRA

Python 3.11+

The script below trains a Qwen 2.5 7B or Llama 3.1 8B model using 4-bit QLoRA. Unsloth patches attention layers for 2x to 5x faster training while fitting in 16GB VRAM.

import torch
from unsloth import FastLanguageModel
from datasets import load_dataset
from trl import SFTTrainer
from transformers import TrainingArguments

# 1. Load 4-bit Base Model
max_seq_length = 4096
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="unsloth/Qwen2.5-7B-Instruct-bnb-4bit",
    max_seq_length=max_seq_length,
    load_in_4bit=True,
)

# 2. Add LoRA Adapters to all projection layers
model = FastLanguageModel.get_peft_model(
    model,
    r=32,                         # LoRA Rank
    target_modules=[
        "q_proj", "k_proj", "v_proj", "o_proj",
        "gate_proj", "up_proj", "down_proj"
    ],
    lora_alpha=64,                # Alpha scaling factor
    lora_dropout=0.05,
    bias="none",
    use_gradient_checkpointing="unsloth",
    random_state=3407,
)

# 3. Load Synthetic Agent Trajectory Dataset
dataset = load_dataset("json", data_files="agent_tool_trajectories.jsonl", split="train")

# 4. Supervised Fine-Tuning Trainer
trainer = SFTTrainer(
    model=model,
    tokenizer=tokenizer,
    train_dataset=dataset,
    dataset_text_field="text",
    max_seq_length=max_seq_length,
    dataset_num_proc=4,
    args=TrainingArguments(
        per_device_train_batch_size=2,
        gradient_accumulation_steps=4,
        warmup_steps=10,
        max_steps=200,
        learning_rate=2e-4,
        fp16=not torch.cuda.is_bf16_supported(),
        bf16=torch.cuda.is_bf16_supported(),
        logging_steps=5,
        optim="adamw_8bit",
        weight_decay=0.01,
        lr_scheduler_type="cosine",
        seed=3407,
        output_dir="qwen-agent-lora",
    ),
)

# 5. Train & Export LoRA / GGUF
trainer.train()
model.save_pretrained_merged("qwen-agent-vllm", tokenizer, save_method="merged_16bit")
print("Model fine-tuned and merged successfully for vLLM deployment!")

3. Evaluation on BFCL & Real-World Testing

Key Benchmark Metrics

AST Match Score: Validates exact parameter types (numbers as integers, not stringified floats).
Execution Correctness: Executes tool calls against live mock sandboxes to verify valid outputs.
Irrelevance Rejection: Confirms the agent avoids invoking tools when the user's prompt requires plain text conversation.

Production vLLM Serving

                        vllm serve qwen-agent-vllm \

  --tensor-parallel-size 1 \

  --gpu-memory-utilization 0.90 \

  --max-model-len 4096 \

  --enable-auto-tool-choice

Serves 150+ requests per second with sub-40ms P50 latency on an NVIDIA L4 or RTX 4090.

Frequently Asked Questions

How many agent trajectories are required for high accuracy? →

For targeted tool domains (10 to 30 enterprise APIs), 500 to 2,000 diverse synthetic multi-turn trajectories are typically sufficient to achieve >99% function calling schema accuracy when starting from an instruction-tuned base model (Qwen 2.5 or Llama 3.1).

Should I fine-tune with LoRA or Full Fine-Tuning? →

LoRA (or QLoRA) is strongly recommended for tool calling. LoRA preserves the foundational conversational reasoning and knowledge of the base model while adapting only projection layers to follow your tool calling syntax, preventing catastrophic forgetting.