Why the model responds slowly and what you can speed up

A request that used to finish in a couple of seconds now hangs for half a minute. Before blaming "the server is slow," it's worth checking a few specific things — half of them are entirely in your control.

Context size is the most common culprit

The Claude and GPT models in the catalog support up to 1,000,000 tokens of context (GLM up to 128K), which is convenient right up until you forget that every extra input token adds processing time before generation even starts. If your prompt habitually carries the entire chat history, a whole file instead of the relevant snippet, or a system prompt that's grown to several thousand words, response time climbs along with it — even when the actual answer is short.

You can check input size for free before sending, with no call to the model:

curl https://api.zukko.pro/v1/messages/count_tokens \
  -H "Authorization: Bearer fetch_your_key" \
  -H "Content-Type: application/json" \
  -H "anthropic-version: 2023-06-01" \
  -d '{"model":"claude-sonnet-5","messages":[{"role":"user","content":"..."}]}'

If the number is surprisingly large, that's the first thing worth trimming before digging further.

Extended thinking makes responses longer on purpose

Extended thinking is a mode where the model spends extra tokens reasoning before producing the final answer (check whether your chosen model and client support it). That's a deliberate trade-off: for tasks where accuracy matters (math, multi-step logic, reviewing code with several conditions), extended thinking usually gives a more reliable result, but the reasoning process itself takes time and tokens that aren't directly visible in the final text. For simple tasks — a short question, reformatting text — having extended thinking on can be overkill time-wise.

max_tokens and actual generation length

Generation time scales roughly linearly with output tokens: the more you ask the model to generate, the longer the response takes, regardless of how many tokens the substance of the answer would actually need. If max_tokens is set with a generous "just in case" margin, and the model ends up generating close to that ceiling because of a verbose style, response time grows proportionally. Ask for brevity explicitly in the system prompt when the task doesn't call for a long answer, and set max_tokens to the actual expected length rather than padding it.

A response cutting off midway is almost never a token limit

A separate symptom: the response breaks off partway through, and the first suspicion lands on a token limit. In practice this is almost always a client or network timeout, not a server-side cutoff. Check your HTTP client's timeout (many libraries default to 30-60 seconds, which is thin for a long generation with extended thinking enabled), and make sure max_tokens isn't artificially truncating the response early. Keep the money side in mind too: a client-side disconnect doesn't stop the charge — the gateway reads the provider's response to the end and bills the full generation.

import requests

response = requests.post(
    "https://api.zukko.pro/v1/messages",
    headers={"Authorization": "Bearer fetch_your_key", "anthropic-version": "2023-06-01"},
    json={"model": "claude-opus-5-5", "max_tokens": 4096, "messages": [...]},
    timeout=180,  # not the default 30-60 seconds
)

Response caching — free speedup for repeats

If the same request from the same user gets sent more than once — say, on a UI re-render or a retry after a network hiccup — turn on caching. With the X-Zukko-Cache: true header (temperature: 0 also enables it, but only on models that accept it — Claude models may reject a temperature other than 1), fully identical requests (the whole body matches) are cached for up to 7 days for chat, and a repeat call returns instantly and for free. Check whether you hit the cache via the X-Zukko-Cache: hit response header.

curl https://api.zukko.pro/v1/messages \
  -H "Authorization: Bearer fetch_your_key" \
  -H "anthropic-version: 2023-06-01" \
  -H "X-Zukko-Cache: true" \
  -H "Content-Type: application/json" \
  -d '{"model":"claude-sonnet-5","max_tokens":256,"messages":[{"role":"user","content":"A frequently repeated question"}]}'

This doesn't speed up the first request, but it removes the delay entirely on every identical one after it.

Images on input

The Claude and GPT models accept images (vision; glm-5.3 and glm-5.3-flash don't), and processing visual input takes time before text generation starts — comparable to, or longer than, processing an equivalent amount of text. If a task can be described in text instead of a picture, that's often faster.

When it isn't you: 503 and provider unavailability

If a specific model is temporarily overloaded on the provider's side, the gateway may return 503 provider_unavailable instead of just answering slowly. In that case there's nothing to speed up on your end — either wait and retry, or switch temporarily to another model from the catalog. A 503 model_disabled, on the other hand, means the model is disabled in the catalog — only switching to another model helps (current list: GET https://api.zukko.pro/v1/models).

What you can do right now

A practical checklist, roughly in order of impact: trim context down to what's actually needed using count_tokens, set max_tokens to the expected response length, turn off extended thinking for simple tasks if your client exposes that toggle, enable caching for repeated requests with X-Zukko-Cache: true, and raise your HTTP client's timeout if the task genuinely needs a long generation.

Bottom line

A slow response almost never means "the server is broken" — more often it's the sum of specific factors: bloated context, extended thinking left on for a simple task, an oversized max_tokens, or simply a short client timeout that cuts the connection before the model finished. Every one of these is checkable and fixable in a few minutes.