Xilos is live — connect your agents today →
Integrations
OpenAI SDK

OpenAI SDK

The official OpenAI SDKs for Python and JavaScript work with Xilos out of the box. Only the base_url and api_key change — no other code modifications are required. Xilos accepts the full OpenAI Chat Completions API surface, including streaming and tool calls.

Prerequisites

  • A Xilos account with an active organization.
  • An API key from Settings in the Xilos dashboard, or a Virtual Key with scoped limits.
  • The OpenAI SDK installed for your language.
  • The base URL: https://api.xilos.ai/api/v1.

Install the SDK

Python

    pip install openai

JavaScript

    npm install openai

Configuration

Create an OpenAI client with the Xilos base URL and your Xilos API key.

Python

    from openai import OpenAI
 
    client = OpenAI(
        api_key="your-xilos-api-key",
        base_url="https://api.xilos.ai/api/v1"
    )

JavaScript

    import OpenAI from "openai";
 
    const client = new OpenAI({
      apiKey: "your-xilos-api-key",
      baseURL: "https://api.xilos.ai/api/v1",
    });

cURL

    curl https://api.xilos.ai/api/v1/chat/completions \
      -H "Content-Type: application/json" \
      -H "Authorization: Bearer your-xilos-api-key" \
      -d '{
        "model": "xilos",
        "messages": [{"role": "user", "content": "Hello, Xilos!"}]
      }'

Info: When model is set to xilos, the Xilos routing engine selects the underlying LLM for each query based on your routing rules. To target a specific model, pass a concrete model name (for example, claude-sonnet-4).

Basic Usage

Send a chat completion request through the Xilos gateway:

Python

    response = client.chat.completions.create(
        model="xilos",
        messages=[
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "What is the capital of France?"}
        ]
    )
 
    print(response.choices[0].message.content)

JavaScript

    const response = await client.chat.completions.create({
      model: "xilos",
      messages: [
        { role: "system", content: "You are a helpful assistant." },
        { role: "user", content: "What is the capital of France?" },
      ],
    });
 
    console.log(response.choices[0].message.content);

Streaming

Set stream: true to receive a stream of server-sent events (SSE). This is useful for chat interfaces and real-time applications where you want to display tokens as they arrive.

Python

    stream = client.chat.completions.create(
        model="xilos",
        messages=[{"role": "user", "content": "Tell me a story."}],
        stream=True
    )
 
    for chunk in stream:
        if chunk.choices[0].delta.content:
            print(chunk.choices[0].delta.content, end="")

JavaScript

    const stream = await client.chat.completions.create({
      model: "xilos",
      messages: [{ role: "user", content: "Tell me a story." }],
      stream: true,
    });
 
    for await (const chunk of stream) {
      const content = chunk.choices[0]?.delta?.content;
      if (content) process.stdout.write(content);
    }

Tool Calls

Xilos supports OpenAI-compatible function calling. Define tools, let the model decide when to call them, and return results for the model to act on.

Python

    tools = [
        {
            "type": "function",
            "function": {
                "name": "get_weather",
                "description": "Get the current weather for a city",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "city": {"type": "string", "description": "City name"}
                    },
                    "required": ["city"]
                }
            }
        }
    ]
 
    response = client.chat.completions.create(
        model="xilos",
        messages=[{"role": "user", "content": "What is the weather in Paris?"}],
        tools=tools
    )
 
    tool_call = response.choices[0].message.tool_calls[0]
    print(tool_call.function.name)   # "get_weather"
    print(tool_call.function.arguments)  # '{"city": "Paris"}'

JavaScript

    const tools = [
      {
        type: "function",
        function: {
          name: "get_weather",
          description: "Get the current weather for a city",
          parameters: {
            type: "object",
            properties: {
              city: { type: "string", description: "City name" },
            },
            required: ["city"],
          },
        },
      },
    ];
 
    const response = await client.chat.completions.create({
      model: "xilos",
      messages: [{ role: "user", content: "What is the weather in Paris?" }],
      tools,
    });
 
    const toolCall = response.choices[0].message.tool_calls[0];
    console.log(toolCall.function.name);       // "get_weather"
    console.log(toolCall.function.arguments);  // '{"city": "Paris"}'

Verify the Connection

After sending your first request:

  1. Open your Xilos dashboard.
  2. Navigate to Query Log.
  3. Confirm the query appears with the routing decision, governance actions, and cost breakdown.

Troubleshooting

ProblemCauseFix
401 UnauthorizedThe API key is invalid or expired.Regenerate the key in the Xilos dashboard under Settings and update the client configuration.
403 ForbiddenA Xilos restriction rule blocked the query.Check the Query Log for the triggered rule and adjust the rule or the query content.
429 Rate limit exceededYou exceeded your organization or Virtual Key rate limit.Reduce request frequency or create a Virtual Key with a higher limit.
Routing rules do not applyThe model parameter is set to a concrete model name.Set model to xilos so the Xilos routing engine selects the model.
Streaming stops mid-responseA network interruption or proxy timeout.Implement a retry with exponential backoff. The OpenAI SDK retries automatically for most transient errors.
Tool calls not returnedThe model selected by routing does not support function calling.Verify the target model supports tools, or set model to a model known to support function calling.

Next Steps

Claude CodeLangChain