Chat completions

Runs a chat completion against your allocation's model and meters it. The request and response follow the OpenAI chat completions format, so existing OpenAI SDK code works once you change the base URL, the key and the model id.

Requires the inference scope.

Example

curl https://cairva.xyz/v1/chat/completions \
  -H "Authorization: Bearer $CAIRVA_API_KEY" \
  -H "Content-Type: application/json" \
  -H "x-request-id: ticket-4812" \
  -d '{
    "model": "gemma4:e2b",
    "messages": [
      {"role": "system", "content": "Answer in one sentence."},
      {"role": "user", "content": "What does a metered allowance include?"}
    ],
    "max_tokens": 200
  }'

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.CAIRVA_API_KEY,
  baseURL: "https://cairva.xyz/v1",
});

const completion = await client.chat.completions.create({
  model: "gemma4:e2b",
  messages: [
    { role: "system", content: "Answer in one sentence." },
    { role: "user", content: "What does a metered allowance include?" },
  ],
  max_tokens: 200,
});

console.log(completion.choices[0].message.content);
console.log(completion.cairva);

import os
from openai import OpenAI

client = OpenAI(api_key=os.environ["CAIRVA_API_KEY"], base_url="https://cairva.xyz/v1")

completion = client.chat.completions.create(
    model="gemma4:e2b",
    messages=[
        {"role": "system", "content": "Answer in one sentence."},
        {"role": "user", "content": "What does a metered allowance include?"},
    ],
    max_tokens=200,
)

print(completion.choices[0].message.content)
print(completion.model_extra["cairva"])

Request body

Any other field is dropped before the request reaches the provider. That includes tools, n, response_format, logprobs and seed. A request with an image part, or a tool, function or developer role, fails with 400 validation_failed. Details in OpenAI compatibility.

Headers

Response

{
  "id": "chatcmpl-7d0c2f4e-6a51-4c43-9f0e-2b8f5a1c9e30",
  "object": "chat.completion",
  "created": 1793520000,
  "model": "gemma4:e2b",
  "choices": [
    {
      "index": 0,
      "message": { "role": "assistant", "content": "..." },
      "finish_reason": "stop"
    }
  ],
  "usage": { "prompt_tokens": 31, "completion_tokens": 18, "total_tokens": 49 },
  "cairva": {
    "request_id": "7d0c2f4e-6a51-4c43-9f0e-2b8f5a1c9e30",
    "allocation_id": "c1a9e1d2-0b7f-4f9e-a3c4-58e2d6b0f114",
    "service_mode": "sandbox",
    "usage_source": "provider_reported",
    "remaining": { "input": 999969, "output": 499982 }
  }
}

There's always exactly one choice. finish_reason is whatever the provider returned, usually stop or length.

The cairva object

When usage_source is pending, usage is null. Cairva never fills in an estimate. The request's hold stays in place until Cairva operations reconcile it from provider metering. See Rate limits and holds.

Output tokens you don't see

Some models reason before they answer, and that reasoning counts as output tokens. The gateway returns only the message content, so reasoning isn't included in the response. Ollama, which serves the sandbox route, doesn't return reasoning as content either.

If max_tokens runs out while the model is still reasoning, you can get an empty or very short content with finish_reason: "length". The provider still reports the output tokens it generated, and they're metered. Give reasoning models enough max_tokens for both the reasoning and the answer.

Errors

Admission errors such as model_not_permitted, allocation_exhausted and rate_limited come back as an HTTP error before the provider is called. If the provider fails, you get 502 upstream_error, the hold is released, and nothing is charged. All codes are in Errors.