Streaming

Set "stream": true on a chat completion and the response arrives as server-sent events. Usage and the cairva metering object come in the last chunk, so you don't need stream_options.

Example

curl -N https://cairva.xyz/v1/chat/completions \
  -H "Authorization: Bearer $CAIRVA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "gemma4:e2b", "stream": true,
       "messages": [{"role": "user", "content": "Count to five."}]}'

const stream = await client.chat.completions.create({
  model: "gemma4:e2b",
  messages: [{ role: "user", content: "Count to five." }],
  stream: true,
});

for await (const chunk of stream) {
  const text = chunk.choices[0]?.delta?.content;
  if (text) process.stdout.write(text);
  // The last chunk has no choices. It carries usage and the cairva object.
  if (chunk.usage) console.log("\n", chunk.usage, chunk.cairva);
}

stream = client.chat.completions.create(
    model="gemma4:e2b",
    messages=[{"role": "user", "content": "Count to five."}],
    stream=True,
)

for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")
    # The last chunk has no choices. It carries usage and the cairva object.
    if chunk.usage:
        print("\n", chunk.usage, chunk.model_extra["cairva"])

Event sequence

The response has Content-Type: text/event-stream. Each event is one data: line holding a chat.completion.chunk:

1. Content deltas: choices[0].delta.content with the next piece of text and finish_reason: null.
2. One chunk with an empty delta and the finish_reason.
3. One chunk with an empty choices array, usage, and the cairva object.
4. data: [DONE].

data: {"id":"chatcmpl-…","object":"chat.completion.chunk","created":1793520000,"model":"gemma4:e2b","choices":[{"index":0,"delta":{"content":"One"},"finish_reason":null}]}

data: {"id":"chatcmpl-…","object":"chat.completion.chunk","created":1793520000,"model":"gemma4:e2b","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: {"id":"chatcmpl-…","object":"chat.completion.chunk","created":1793520000,"model":"gemma4:e2b","choices":[],"usage":{"prompt_tokens":12,"completion_tokens":9,"total_tokens":21},"cairva":{"request_id":"…","allocation_id":"…","service_mode":"sandbox","usage_source":"provider_reported","remaining":{"input":999976,"output":499991}}}

data: [DONE]

Deltas carry content only. There's no separate first chunk with role. The cairva object has the same fields as on a non-streaming response. If the provider didn't report usage, usage is null and usage_source is pending.

Errors before the stream starts

Authentication, validation and admission errors, such as rate_limited or allocation_exhausted, come back as a normal JSON error with an HTTP status. No stream is opened. See Errors.

Errors during the stream

Once admission passes, the response is already 200 with an open stream, and the provider is called from inside it. So a provider failure, even one before the first token, arrives as one event and then the stream closes, without [DONE]:

{"error": {"code": "upstream_error", "message": "The inference provider stopped responding"}}

The request's hold is released and nothing is charged for it.

Disconnecting early

If your client disconnects, Cairva keeps reading the provider's stream to the end so the provider's usage report still arrives. The tokens the model generated after you left are still metered. To spend less, set max_tokens lower instead of disconnecting.