Documentation

The API works like OpenAI's Chat Completions API. Change the base URL and API key and your existing code keeps working.

Quickstart

1. Create an account and add at least $1 credit in Billing.
2. Create an API key in API Keys.
3. Make your first request:

• Python
• JavaScript
• cURL

from openai import OpenAI

client = OpenAI(base_url="BASE_URL", api_key="YOUR_API_KEY")

reply = client.chat.completions.create(
    model="smart",
    messages=[{"role": "user", "content": "Hello!"}],
)
print(reply.choices[0].message.content)

import OpenAI from "openai";

const client = new OpenAI({ baseURL: "BASE_URL", apiKey: "YOUR_API_KEY" });

const reply = await client.chat.completions.create({
  model: "smart",
  messages: [{ role: "user", content: "Hello!" }],
});
console.log(reply.choices[0].message.content);

curl BASE_URL/chat/completions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"smart","messages":[{"role":"user","content":"Hello!"}]}'

API keys

Send your key in the Authorization: Bearer header. The full key is shown only once when you create it, so copy it right away. Treat it like a password and never put it in public code. You can revoke a key at any time and set a monthly spending limit per key.

Chat completions

POST /v1/chat/completions

Use "model": "smart" to let us choose, or a model name from the Models page to choose yourself.

{
  "model": "smart",
  "messages": [{"role": "user", "content": "Explain DNS in one paragraph."}],
  "routing": {"quality": "mid", "speed": "high", "cost": "low"}
}

Images (image_url parts), tools and JSON output (response_format) are supported. We automatically pick only models that support what your request needs.

Streaming

Add "stream": true and the answer arrives piece by piece as server-sent events, ending with data: [DONE]. Add "stream_options": {"include_usage": true} to receive token usage and cost in the final chunk.

stream = client.chat.completions.create(
    model="smart",
    messages=[{"role": "user", "content": "Write a short poem."}],
    stream=True,
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")

If a model fails before it starts writing, we move to the next model. Once writing has started we do not switch; if the stream breaks, you receive an error and are charged only for what was generated. The answering model is sent in the response headers X-Model, X-Server and X-Used-Fallback.

Smart Routing & fallback

If you send nothing, your default setting from the dashboard (Smart Routing page) is used. At most 3 models are tried, within the total time limit of 60 seconds. You are charged only for the model that answered.

Not every request problem triggers a fallback: if your own request is invalid, too long or blocked, you get the error right away, because another model would fail the same way.

Parameters

An option a model does not understand is quietly ignored. Requests with tools, images or JSON output only go to models that support them. Send an Idempotency-Key header to make sure a retried request is never charged twice.

max_tokens matters for cost: before each request we hold the maximum possible cost, then charge the real cost and release the rest. Default is 4096 if you do not set it.

Response details

The response matches OpenAI's format, plus:

"usage": {"prompt_tokens": 12, "completion_tokens": 40, "total_tokens": 52, "cost": 0.000021},
"unified": {
  "request_id": "...", "model": "llama-3.3-70b", "server": "Groq",
  "used_fallback": false, "tokens_per_sec": 540.2, "latency_ms": 310
}

cost is in USD and is what was deducted from your balance.

Other endpoints

Errors

Every error has the same shape: {"error": {"message": "...", "type": "...", "code": "..."}}. You are never charged for a failed request.

Limits

Each key has a requests-per-minute limit (60 by default; contact us if you need more). You can also set a monthly spending limit per key; when it is reached, that key stops working.

Pricing & billing

You add credit first and usage is deducted from it. Minimum top-up is $1, balances are in USD, and there is no free credit. See Pricing. Cost per request is: input tokens × input price + output tokens × output price, at the server that answered. Your Usage page lists every request with its cost.

Moving from OpenAI

1. Change base_url to BASE_URL.
2. Use your API key from this site.
3. Keep your code. Use a model name from the Models page, or smart.

LangChain & LlamaIndex

# LangChain
from langchain_openai import ChatOpenAI
llm = ChatOpenAI(model="smart", base_url="BASE_URL", api_key="YOUR_API_KEY")

# LlamaIndex
from llama_index.llms.openai_like import OpenAILike
llm = OpenAILike(model="smart", api_base="BASE_URL", api_key="YOUR_API_KEY", is_chat_model=True)