Error 499: the client closed the connection before the response arrived

Code 499 is the status nginx and most proxies in front of an API use to mark a situation where the client closed the connection before the server finished writing its response. From the API's point of view this isn't really an error: the server didn't deliberately return anything wrong, the conversation just got cut from the other end of the wire.

Why it's almost always a timeout, not a token limit

The most common cause is an HTTP client timeout set shorter than the model actually needs to finish generating. This shows up mostly on long responses: a large max_tokens or extended thinking turned on both take noticeably longer than a short chat reply.

The difference from hitting a token limit matters. If the model had run into the context limit or max_tokens, the response would still arrive in full, just truncated with a finish_reason. With 499 no response arrives at all — the connection breaks before the data reaches the client, regardless of how many tokens the model had already produced.

Where the default timeout usually comes from

Plain curl without an explicit --max-time won't cut the connection on its own, but the libraries inside most tools almost always set one. Python's requests has no default timeout either, but many HTTP clients bundled into IDE plugins and proxy wrappers do, often around 30-60 seconds — not enough for extended thinking on a hard request.

You can check this by raising the timeout explicitly:

curl https://api.zukko.pro/v1/messages \
  --max-time 180 \
  -H "Authorization: Bearer fetch_your_key" \
  -H "Content-Type: application/json" \
  -H "anthropic-version: 2023-06-01" \
  -d '{"model":"claude-opus-5-5","max_tokens":4096,"thinking":{"type":"enabled","budget_tokens":8000},"messages":[{"role":"user","content":"Outline a migration plan to a new message queue"}]}'

import requests

resp = requests.post(
    "https://api.zukko.pro/v1/messages",
    timeout=180,  # seconds, not the library default
    headers={
        "Authorization": "Bearer fetch_your_key",
        "anthropic-version": "2023-06-01",
    },
    json={
        "model": "claude-opus-5-5",
        "max_tokens": 4096,
        "thinking": {"type": "enabled", "budget_tokens": 8000},
        "messages": [{"role": "user", "content": "Outline a migration plan"}],
    },
)

const controller = new AbortController();
const timeout = setTimeout(() => controller.abort(), 180_000);

const resp = await fetch("https://api.zukko.pro/v1/chat/completions", {
  method: "POST",
  signal: controller.signal,
  headers: {
    Authorization: "Bearer fetch_your_key",
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    model: "gpt-5.6-sol",
    max_tokens: 4096,
    messages: [{ role: "user", content: "Outline a migration plan" }],
  }),
});
clearTimeout(timeout);

Streaming cuts the odds of a drop

When you wait for the whole answer to arrive as one block, any proxy or load balancer in between can decide the connection looks stalled and close it itself — even with a generous timeout on your own end. With streaming ("stream": true on /v1/chat/completions or /v1/responses, same idea on /v1/messages) bytes keep arriving continuously, and that's usually enough for intermediate hops to treat the connection as alive.

Image generation and longer timeouts

Image generation through Zukko is temporarily unavailable — this section is kept for reference. Current model list: GET https://api.zukko.pro/v1/models.

Models with kind=image — nano-banana-2-text-to-image, openai-gpt-image-2-edit, and the rest of the catalog — usually take longer to process a single request than a text answer spanning a couple of paragraphs, and they're billed separately, per item rather than per token. A client timeout tuned for chat is often too short for /v1/images/generations. If 499 shows up consistently on image generation rather than on text, that's the first thing worth raising separately from the timeout used for /v1/messages and /v1/chat/completions.

Idempotency-Key: retrying without paying twice

If a request drops, the obvious move is to retry it. The catch is that the client dropping the connection doesn't stop the charge: the gateway reads the provider's response to the end and bills the whole generation, so a plain retry without protection means paying twice. To avoid that, attach an Idempotency-Key header with a random value on the first attempt — unique to this request and the same on every retry of it (don't regenerate it when retrying):

curl https://api.zukko.pro/v1/messages \
  -H "Authorization: Bearer fetch_your_key" \
  -H "Idempotency-Key: retry-20260919-01" \
  -H "anthropic-version: 2023-06-01" \
  -d '{"model":"claude-opus-5-5","max_tokens":4096,"messages":[...]}'

Repeating the same key with the same request body within 24 hours returns the same result without a second charge. If the body changes while the key stays the same, you'll get 422 idempotency_conflict — which is itself a useful signal that you're actually retrying, not sending a new request under an old name.

Checking what actually happened

Since no response reached the client, the way to find out whether the generation completed is the history. GET /v1/usage returns a cursor-paginated list of charges — if the dropped request shows up there, the model did finish and a result exists on the provider's side, it just never reached you. Keep in mind a drop doesn't cancel the generation: the gateway reads the provider's response and bills it in full, so the entry may show up not immediately but once the generation completes. If there's still no entry after a few minutes, the request genuinely didn't go through; to be safe, retry with the same Idempotency-Key, and the retry won't double anything.

The X-Zukko-Cost-Micros and X-Fetch-Balance-Cents headers only arrive with a successful regular (non-streaming) response, so you won't see them at all on a 499 — another way to tell "it finished but didn't arrive" from "it never happened": compare balance via GET /v1/balance before and after (bearing in mind the dropped generation still gets charged).

What to do about it

If 499 keeps happening on the same kind of requests, it's almost always a large max_tokens or extended thinking combined with a client timeout inherited from a library or proxy default. Raise the timeout explicitly, switch to streaming wherever the tool supports it, and attach Idempotency-Key to any request that could plausibly be retried — that turns a dropped connection from a double-charge risk into an ordinary network hiccup you can just retry.