Fine-Tune a Model

Use Xilos orchestration data to fine-tune an open-source model and import it back into Xilos.

Prerequisites

A Xilos account with admin access
Sufficient query history (1,000+ queries recommended)
An understanding of which query patterns you want to optimize

The Fine-Tune Pipeline

The fine-tune pipeline consists of five steps:

Review suggestions — Identify which routing rules are expensive enough to justify fine-tuning
Export training data — Filter the query log into a training dataset
Choose a base model — Select an open-source model to fine-tune
Create a fine-tune job — Submit the job to a fine-tune provider
Import the model — Register the fine-tuned model in Xilos and update routing rules

Step 1: Review Suggestions

Xilos analyzes your query patterns and suggests fine-tune targets:

curl https://api.xilos.ai/api/v1/suggestions \
  -H "Authorization: Bearer YOUR_XILOS_API_KEY"

Look for suggestions of type fine_tune_target. Each suggestion includes:

The routing rule name and ID
Current monthly cost for that rule
Estimated savings if fine-tuned
Recommended base model

Step 2: Export Training Data

Export the query log as a filtered training dataset:

curl -X POST https://api.xilos.ai/api/v1/training-data/export \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_XILOS_API_KEY" \
  -d '{
    "format": "openai",
    "routing_rule_id": "uuid-from-suggestion",
    "min_faithfulness": 0.7,
    "min_relevance": 0.7,
    "limit": 10000
  }'

See Export Training Data for details.

Step 3: Choose a Base Model

View available base models:

curl https://api.xilos.ai/api/v1/fine-tune/models \
  -H "Authorization: Bearer YOUR_XILOS_API_KEY"

Available base models typically include:

Model	Provider	Context Window	Best For
Llama-3.3-70B-Instruct	Together AI	128k	General purpose, strong reasoning
Llama-3.2-1B-Instruct	Together AI	128k	Fast, cheap, simple tasks
Phi-4 Mini	Together AI	16k	Compact, efficient
Mistral-7B	Fireworks	32k	Balanced performance

Info: Choose a base model that matches the complexity of your queries. A 70B model will give better quality but cost more to fine-tune and run. A 1B model is cheaper but may not match the quality of a premium API model.

Step 4: Create a Fine-Tune Job

curl -X POST https://api.xilos.ai/api/v1/fine-tune/create \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_XILOS_API_KEY" \
  -d '{
    "base_model": "meta-llama/Llama-3.3-70B-Instruct",
    "provider": "together",
    "training_data_export_id": "uuid-from-step-2",
    "name": "Customer Support Fine-Tune"
  }'

The fine-tune job is submitted to the provider. Monitor the job status:

curl https://api.xilos.ai/api/v1/fine-tune/jobs \
  -H "Authorization: Bearer YOUR_XILOS_API_KEY"

Job statuses: pending → training → completed (or failed).

Step 5: Import the Model

Once the fine-tune job is completed:

The fine-tuned model is registered in Xilos.
Navigate to SuperAdmin > Model Config.
The new model appears in the available models list.
Update your routing rules to use the fine-tuned model instead of the premium API model.

Step 6: Verify and Monitor

Send test queries through the updated routing rule.
Compare response quality before and after the switch.
Monitor cost savings in Cost Controls — the fine-tuned model runs on your own infrastructure at a fraction of the API cost.
Check quality scores in the Query Log to ensure the fine-tuned model maintains acceptable quality.

Info: The fine-tuned model captures your organization's specific query patterns, terminology, and desired response style. It often outperforms generic premium models on domain-specific queries while costing significantly less.

Cost Savings

Fine-tuning typically pays for itself within the first month:

Fine-tune cost: $5-50 (one-time, depending on dataset size and model)
Monthly savings: $100-2,000+ (depending on query volume)
ROI: 10-40x in the first year

Export Training Data Set Up Webhooks