Streaming responses — when you actually need stream: true

stream: true gets added to requests almost reflexively — "models stream tokens, so I should stream too." But a streaming response adds real complexity on the client side: parsing SSE events, accumulating partial chunks, handling a mid-stream drop. If the frontend does not show text as it generates and just waits for the full response, streaming only adds code with no payoff.

What streaming actually does

Without stream: true, the server responds with a single JSON object after the model has fully generated the answer. With stream: true, the server sends the response in pieces via Server-Sent Events (SSE) — as the model generates tokens, not after it finishes.

A regular request:

curl https://api.zukko.pro/v1/chat/completions \
  -H "Authorization: Bearer fetch_your_key" \
  -H "Content-Type: application/json" \
  -d '{"model":"claude-sonnet-5","messages":[{"role":"user","content":"Write a short poem"}]}'

A streaming request — same body plus "stream": true:

curl https://api.zukko.pro/v1/chat/completions \
  -H "Authorization: Bearer fetch_your_key" \
  -H "Content-Type: application/json" \
  -d '{"model":"claude-sonnet-5","stream":true,"messages":[{"role":"user","content":"Write a short poem"}]}' \
  --no-buffer

The --no-buffer flag in curl matters for watching the stream in real time — otherwise the terminal can buffer output and dump it all at once.

Event format: OpenAI style

In /v1/chat/completions, each event is a data: {...} line, ending with data: [DONE]:

The client needs to accumulate delta.content from each event into a single text:

import requests

resp = requests.post(
    "https://api.zukko.pro/v1/chat/completions",
    headers={"Authorization": "Bearer fetch_your_key"},
    json={"model": "claude-sonnet-5", "stream": True,
          "messages": [{"role": "user", "content": "Write a short poem"}]},
    stream=True,
)

full_text = ""
for line in resp.iter_lines():
    if not line or not line.startswith(b"data: "):
        continue
    payload = line[len(b"data: "):]
    if payload == b"[DONE]":
        break
    import json
    chunk = json.loads(payload)
    delta = chunk["choices"][0]["delta"].get("content", "")
    full_text += delta
    print(delta, end="", flush=True)

Event format: Anthropic style

In /v1/messages, events are structured differently — with types like message_start, content_block_delta, message_stop, and others:

The required anthropic-version: 2023-06-01 header is needed in streaming mode too — without it the request returns an error before the stream even starts:

curl https://api.zukko.pro/v1/messages \
  -H "Authorization: Bearer fetch_your_key" \
  -H "anthropic-version: 2023-06-01" \
  -H "Content-Type: application/json" \
  -d '{"model":"claude-sonnet-5","max_tokens":300,"stream":true,"messages":[{"role":"user","content":"Write a short poem"}]}' \
  --no-buffer

Client-side code should filter by type inside data, not just check for content_block_delta — what actually matters is text_delta; other event types carry housekeeping information (block start, token usage stats, message end).

When streaming is actually worth it

A chat UI with visible typing. If the user sees the answer being typed out as it generates — the classic chatbot UX — streaming is essential, or the user stares at a blank screen for several seconds on a long response.

Long responses with a high max_tokens. The longer the expected response, the more time the model spends on full generation. Without streaming, the client just waits for the entire HTTP response — with max_tokens in the thousands, that can be a noticeable pause with no feedback.

The ability to cut off early in the UI. Streaming lets the client stop reading early if the user changes their mind or the answer already contains what's needed — the interface is freed up without waiting for full generation. But this does not affect spend: a dropped stream does not stop billing — the gateway reads the provider's response to the end and charges for the full generation. Cap spend with max_tokens, not by closing the connection.

When streaming is not worth it

Server-side processing with no UI. If the model's response is used programmatically — parsed, stored in a database, passed further down a pipeline — no one watches the generation happen, and streaming only complicates the parsing code with no benefit.

Short responses. A one-word classification, a 1-to-10 score, a small structured JSON object — generation takes a fraction of a second, and the difference between streaming and a regular response is not noticeable.

Strict JSON parsing. If the response needs to be valid JSON parsed as a whole once received, accumulating it piece by piece gives no advantage — the parser still waits for the complete object before it can process it.

Mid-stream drops — a common mix-up with limits

If a stream cuts off midway with no explicit error, it is almost always a client or network timeout, not a token limit — check max_tokens in the request and the HTTP client's timeout setting (many HTTP libraries default to a 30–60 second timeout, which is not enough for long generation). Keep in mind that a client-side drop does not cancel the charge: the gateway still reads the provider's response to the end and charges for the full generation. You can check what a cut-off request cost in the dashboard or via GET /v1/usage — an entry with model, input_tokens, output_tokens, and cost_cents will appear there.

Headers in a streaming response

Streaming responses do not carry the X-Zukko-Cost-Micros and X-Fetch-Balance-Cents cost headers — they only come with regular JSON responses (stream: false). The exact cost is only finalized after generation completes, while the stream's headers are sent earlier. For streaming requests, check spend in the dashboard (Usage), via GET /v1/usage (account-wide history), or by the balance after the request:

resp = requests.post(url, headers=headers, json=body, stream=True)
for line in resp.iter_lines():
    ...  # read the stream to the end

balance = requests.get("https://api.zukko.pro/v1/balance", headers=headers).json()
print("balance after, cents:", balance["balance_cents"])

If you need the exact cost of each request right in the response, use regular (non-streaming) mode and read X-Zukko-Cost-Micros (micro-dollars, 1,000,000 = $1).

Bottom line

Turn on stream: true where there is a live interface showing text as it generates, or where the response is long enough that a pause with no feedback would be noticeable. For server-side processing, short responses, and strict JSON, a regular request is simpler to implement and loses nothing functionally.