503 errors — the model provider is unavailable

A 503 means the service temporarily cannot handle your request. For a model API it usually means the gateway itself is fine but the model provider behind it is not.

A separate case is a 503 with the code model_disabled: the model is disabled in Zukko, and retries will not help. Switch to an enabled model — the current list is GET https://api.zukko.pro/v1/models, and model status is available from the public GET https://api.zukko.pro/api/status.

How it differs from its neighbours

503 — temporary unavailability. The request is valid; retrying makes sense.

500 — an internal error. Also not your fault, but retries help less often: a request that reproducibly breaks processing will break it again.

429 — rate limit exceeded at the model provider; retry with a delay. The Zukko gateway has no rate limit of its own: a 429 from Zukko itself means the key's spending limit (key_daily_limit_reached, reset at 00:00 UTC, or key_monthly_limit_reached), and retrying will not help.

502 — a failure talking to an upstream service.

The practical difference is how long to wait. For a provider 429, follow the Retry-After header. For 503, intervals from a few seconds up to a minute are sensible: if the provider is down, one-second retries only add load.

An important trap: a streamed 503 arrives with a 200 status

This is where hand-written error handling breaks.

On a streaming request the connection opens immediately, headers go out with 200, and only then does an error event arrive inside the stream:

data: {"error":{"message":"The model provider is temporarily unavailable","type":"service_unavailable"}} data: [DONE]

The status code is 200, so a client that checks only the status treats the request as a success and returns an empty result.

The fix: parse the stream contents, not just the status. An event containing an error field is an error regardless of the response code. And if you keep your own accounting, do not bill such requests — they carry no tokens.

Surviving an outage

Retry with growing delays. Three or four attempts at 5, 15 and 45 seconds cover most short incidents.

Cap the attempts. An unbounded loop during a long outage becomes a hung process.

Persist state before the call. In agent workloads the expensive part of a failure is losing the working context. With state saved, you resume instead of starting over.

Have a fallback model. Outages usually affect one family rather than everything at once. Switching to a comparable model is often cheaper than waiting.

When it is planned maintenance

If the response is instant and consistent, with a clear message about maintenance and a Retry-After header, this is a planned stop rather than an incident. Retrying will not help until it ends — but such requests are not billed either.