Orchestrate

Xilos orchestrates AI traffic with intelligent routing, semantic caching, multi-model workflows, context compression, and a context engine that compounds knowledge across every interaction.

Orchestration Features

• Routing Rules — Route every query to the right model based on cost, latency, or capability. Define rules in natural language or with keywords.
• Semantic Caching — Cache LLM responses and reuse them for semantically similar queries. Reduces cost and latency by 50-90%.
• Multi-Model Workflows — Chain LLMs sequentially, run them in parallel, and merge outputs. Build pipelines with the visual workflow builder.
• Context Compression — Compress prompts before they reach the LLM. 50-90% token reduction with three strategies.
• Context Engine — Every answered query becomes a searchable knowledge asset. Extracts atomic facts and enriches future queries.
• Smart Routing — The internal SLM classifies query intent and selects the optimal LLM, tools, and context automatically.
• Routines — Schedule agent queries on a recurring basis using cron expressions.
• Webhooks — Receive event-driven notifications when queries complete, are blocked, cached, or compressed.

The Orchestration Loop

Every query that passes through Xilos follows an orchestration loop:

1. Compress — Reduce token count using prefix stabilization, conversation summarization, and BM25 content extraction.
2. Enrich — Add relevant context from the Context Engine's atomic fact store.
3. Check compliance — Apply guardrails and restriction rules.
4. Select tools — Auto-select tools based on query intent (when configured).
5. Route to best model — Smart Routing uses the SLM to classify intent and select the optimal LLM.
6. Execute — The AI Kernel processes the query through the selected model or workflow.
7. Score quality — Faithfulness, answer relevance, and context relevance are scored automatically.
8. Store as atomic fact — The response is extracted into searchable facts for future enrichment.
9. Fire webhooks — Event-driven notifications are sent to subscribed endpoints.
10. Schedule routine analysis — Routines analyze patterns and generate suggestions on a schedule.

Info: The orchestration loop is what makes Xilos "auto-magic" — the more queries flow through it, the smarter the routing, caching, and context enrichment become.