A million-token context window: what it's actually for

The Claude and GPT models in the catalog — from claude-opus-5-5 to gpt-5.6-luna — ship with a 1,000,000-token context (glm-5.3 and glm-5.3-flash have 128K; current list: GET https://api.zukko.pro/v1/models). That's far more than a normal chat needs, so the real question isn't "is it available" but "when is it actually worth using."

What actually fits in a million tokens

Rough rule of thumb: prose runs around 1.3-1.5 tokens per word, code is usually closer to one token per word or a bit more because of brackets and indentation. A million tokens is, loosely, several thousand pages of text or a mid-sized monorepo's source without node_modules. The exact number depends on language and content type, so instead of estimating, count it:

curl https://api.zukko.pro/v1/messages/count_tokens \
  -H "Authorization: Bearer fetch_your_key" \
  -H "Content-Type: application/json" \
  -H "anthropic-version: 2023-06-01" \
  -d '{
    "model": "claude-opus-5-5",
    "messages": [{"role": "user", "content": "<your file or repository content>"}]
  }'

The call is free, so you can check input size before paying for the actual response.

Where it actually solves a problem

Reading a whole log file or error dump at once. Instead of chunking a log and losing context between pieces, you hand over the whole file and ask for the root cause — the model sees what happened before and after the moment in question as one piece.

Reviewing changes against the whole repo, not just the diff. A diff without surrounding context is often misleading: a function looks suspicious until you see that the calling code already validates everything. Loading several full files removes that class of false positives.

Long conversation history without losing detail. In support, or in an assistant that runs a conversation for hours, trimming history to the last N messages loses agreements made earlier on. A million tokens lets you keep the whole session intact.

Working with a long document as a whole — a contract, a spec, a two-hour meeting transcript — without pre-chunking it and stitching the answer back together from pieces.

Where it's not needed and just costs more

For classification, extracting a single field, a short answer to a question — a 1M context gives you nothing, but input tokens are still billed for what you actually sent. If the task is about one paragraph, sending the entire document "just to be safe" isn't safer, it's just paying for every token the model read and didn't use.

A cut-off response is not a token limit

Working with large input, it's easy to mix up the symptoms. If a response cuts off mid-way, that's almost always an HTTP client or network timeout, not the context running out — a 1M window is rarely filled completely. Worth checking:

• max_tokens in the request — is it set too low for the length of answer you expect;
• the HTTP client timeout — large input takes longer to process, and a 30-second client timeout sized for short requests can fire before the model finishes.

Caching and re-sending a large context

The gateway cache (the X-Zukko-Cache: true header; temperature: 0 enables it too, but only on models that accept that parameter — Claude models may reject a temperature other than 1 with a 400) only works on a fully identical request body: if the same large document is sent with the same system prompt and the same messages, unchanged, the repeat counts as a cache hit (up to 7 days for chat), isn't billed again and isn't written to /v1/usage. In a long conversation where new messages get added on top of the same spec file, that doesn't apply: a shared prefix or an "almost identical" request doesn't hit the cache, and every turn pays for the whole context it sends. Keep the difference from vendors in mind too: they charge ~10% of the input price for prompt-cache reads, while Zukko bills cached tokens at the regular price — for agents and conversations with a large repeated context, the real price gap is smaller than a head-to-head price-list comparison suggests.

A key limit guards against accidental overspend

A request with a context of hundreds of thousands of tokens costs noticeably more than a short one, and if such requests fire from code stuck in a retry loop by mistake, a monthly budget can disappear fast. Set a daily or monthly dollar limit right on the key, on its page in the dashboard — once the limit is hit, requests start returning 429 with key_daily_limit_reached or key_monthly_limit_reached instead of quietly continuing to charge.

Text and images in one large request

The Claude and GPT models accept images (GLM has no vision), and that can be combined with a large context in a single request: send screenshots of several UI screens together with the full source of the components that render them, and ask the model to find the mismatch between what's drawn and what's written in the code. Sending text and images in separate requests loses exactly that link between "what's visible" and "what's written."

The practical takeaway

Ask not "does it fit" but "does the model need all of this at once to answer." If yes — count tokens ahead of time with /v1/messages/count_tokens, cache fully repeated requests, and check client timeouts. If no — trim the input to what's actually needed and stop paying for context that never gets read.