Xilos is live — connect your agents today →
Integrations
LangChain

LangChain

LangChain (opens in a new tab) supports custom base URLs through the ChatOpenAI class. Point it at Xilos to route all chain and agent calls through the Xilos gateway for routing, guardrails, caching, and cost control.

Prerequisites

  • A Xilos account with an active organization.
  • An API key from Settings in the Xilos dashboard, or a Virtual Key with scoped limits.
  • LangChain and the OpenAI integration package installed.
  • The base URL: https://api.xilos.ai/api/v1.

Install LangChain

Python

    pip install langchain langchain-openai

JavaScript

    npm install @langchain/core @langchain/openai

Configuration

Create a ChatOpenAI instance with the Xilos base URL and your Xilos API key.

Python

    from langchain_openai import ChatOpenAI
 
    llm = ChatOpenAI(
        model="xilos",
        api_key="your-xilos-api-key",
        base_url="https://api.xilos.ai/api/v1",
    )
 
    response = llm.invoke("Hello, Xilos!")
    print(response.content)

JavaScript

    import { ChatOpenAI } from "@langchain/openai";
 
    const llm = new ChatOpenAI({
      model: "xilos",
      apiKey: "your-xilos-api-key",
      configuration: {
        baseURL: "https://api.xilos.ai/api/v1",
      },
    });
 
    const response = await llm.invoke("Hello, Xilos!");
    console.log(response.content);

Info: When using LangChain agents or chains, the model value xilos lets your routing rules decide which underlying LLM handles the request. You can also pass a specific model name (for example, claude-sonnet-4) if you want to bypass routing.

Streaming

LangChain supports streaming through the .stream() method. Xilos returns server-sent events (SSE) that LangChain consumes token by token.

Python

    for chunk in llm.stream("Tell me a story."):
        print(chunk.content, end="", flush=True)

JavaScript

    const stream = await llm.stream("Tell me a story.");
    for await (const chunk of stream) {
      process.stdout.write(chunk.content);
    }

Using with Agents and Chains

You can pass the Xilos-backed ChatOpenAI instance to any LangChain agent, chain, or tool. The LLM calls are transparent — LangChain sends requests to Xilos, which applies routing and governance.

Python

    from langchain_openai import ChatOpenAI
    from langchain.agents import create_tool_calling_agent, AgentExecutor
    from langchain_core.prompts import ChatPromptTemplate
 
    llm = ChatOpenAI(
        model="xilos",
        api_key="your-xilos-api-key",
        base_url="https://api.xilos.ai/api/v1",
    )
 
    prompt = ChatPromptTemplate.from_messages([
        ("system", "You are a helpful assistant."),
        ("human", "{input}"),
        ("placeholder", "{agent_scratchpad}"),
    ])
 
    agent = create_tool_calling_agent(llm, tools=[], prompt=prompt)
    executor = AgentExecutor(agent=agent, tools=[], verbose=True)
 
    result = executor.invoke({"input": "What is 2 + 2?"})
    print(result["output"])

JavaScript

    import { ChatOpenAI } from "@langchain/openai";
    import { createReactAgent } from "@langchain/langgraph/prebuilt";
 
    const llm = new ChatOpenAI({
      model: "xilos",
      apiKey: "your-xilos-api-key",
      configuration: { baseURL: "https://api.xilos.ai/api/v1" },
    });
 
    const agent = createReactAgent({ llm, tools: [] });
    const result = await agent.invoke({
      messages: [{ role: "user", content: "What is 2 + 2?" }],
    });
    console.log(result.messages[result.messages.length - 1].content);

Verify the Connection

After running your first LangChain query:

  1. Open your Xilos dashboard.
  2. Navigate to Query Log.
  3. Confirm the query appears with the routing decision, governance actions, and cost breakdown.

Troubleshooting

ProblemCauseFix
401 UnauthorizedThe API key is invalid or expired.Regenerate the key in the Xilos dashboard under Settings and update the ChatOpenAI configuration.
403 ForbiddenA Xilos restriction rule blocked the query.Check the Query Log for the triggered rule and adjust the rule or the query content.
Routing rules do not applyThe model parameter is set to a concrete model name.Set model to xilos so the Xilos routing engine selects the model.
base_url not respected (JS)The baseURL was passed at the top level instead of inside configuration.In LangChain.js, pass baseURL inside the configuration object, not at the top level of the ChatOpenAI constructor.
Streaming not workingThe chain or agent does not call .stream() or .astream().Use .stream() (Python) or .stream() / .astream() (JavaScript) instead of .invoke().
Tool calls not returnedThe model selected by routing does not support function calling.Verify the target model supports tools, or set model to a model known to support function calling.

Next Steps

OpenAI SDKGitHub Copilot