The context window in neural networks: what it is and why size is not everything

The question "what is a context window in neural networks" comes up when a model suddenly "forgets" the start of a conversation or the API returns a length error. In short, it is the maximum number of tokens a model processes in a single request, and everything counts together: what you sent and what it answered. Below we cover what fills the window, which sizes models have as of October 18, 2026, and why window size directly affects price.

What goes into the context

A model does not remember earlier requests. Every API call is independent, and the "memory" of a chat is created by the client, which sends the whole history again. The context window of one request contains:

• the system prompt with the role and rules;
• the whole conversation history: your messages and the model's replies;
• attachments: pasted files, document text, search results, images (they are converted to tokens too);
• tool descriptions, if you use function calling;
• the model's answer: its tokens also occupy the window, so the max_tokens parameter must be taken into account.

The unit is the token, not the word or character. Russian text usually takes more tokens than English text of the same length, so a "million-token" window holds fewer Russian words than English ones. A general explanation of tokens is in what tokens are in neural networks↗.

LLM context window sizes at Zukko

Window sizes in the catalog as of October 18, 2026:

If you wonder which neural network has the largest context window, in our catalog it is 1M tokens, and most models have that size, so there is little to choose by this criterion. Models differ in price and strength index, so choose by those and by your task. Check the current window of each model through GET https://api.zukko.pro/v1/models or on the "Models" page in the dashboard.

Why size is not everything

A large window is a ceiling, not a guarantee. Three practical points:

1. You pay for every context token on every request. A chat with 200,000 tokens of history costs, each time, as much as sending those 200,000 tokens again.
2. A long context does not guarantee that the model uses all of it equally well. Check on your own data: place an important fact at the start, middle and end, and see whether the model finds it.
3. The window can overflow. Then you get an error; how to deal with it is described in the article on context_length_exceeded↗.

What a full window costs

Request price grows with context. One request with a completely full window, input only (formula: window size × input price / 1,000,000, prices as of October 18, 2026):

Now a conversation. Say you have a 200,000-token document and ask 10 questions about it in a row, getting 1000 tokens of answer each time. Simplification: the document stays the only large chunk of context, and history growth is ignored. Model claude-sonnet-5-5, input 0.4, output 2:

• Without cache: input 10 × 200,000 × 0.4 / 1M = $0.80, output 10 × 1000 × 2 / 1M = $0.02, total $0.82.
• With cache in the Anthropic format: the first request writes the document to cache (write price 0.5): 200,000 × 0.5 / 1M = $0.10. The other nine read it from cache (0.05): 9 × 200,000 × 0.05 / 1M = $0.09. Output is the same $0.02. Total 0.10 + 0.09 + 0.02 = $0.21.

For Claude, the cache discount on reads applies if you call the /v1/messages format (Claude Code and the Anthropic SDK). In the OpenAI format /v1/chat/completions, all input for Claude is billed at the input price. For GPT models, cache reads are visible in both formats. Details are on the page about key limits and cache↗.

Counting context before a request

You can learn the token count in advance without paying for an answer. The count_tokens method does not run generation (see the reference↗):

import anthropic

client = anthropic.Anthropic(
    base_url="https://api.zukko.pro",
    api_key="fetch_your_key",
)
MODEL = "claude-sonnet-5-5"

history = [{"role": "user", "content": open("report.txt", encoding="utf-8").read()}]
n = client.messages.count_tokens(model=MODEL, messages=history).input_tokens
print("input tokens:", n)

The address for the Anthropic SDK has no /v1, and the fetch_... key for the examples is created in the dashboard↗. The same call with curl:

curl https://api.zukko.pro/v1/messages/count_tokens \
  -H "x-api-key: $ZUKKO_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d '{"model":"claude-sonnet-5-5","messages":[{"role":"user","content":"Hello"}]}'

What to do when context keeps growing

A few techniques that work with any model:

• Trim history to a budget. Keep the system prompt and the latest messages; drop the old ones or fold them into a short summary.
• Do not send the whole file when you need a piece. Pick the relevant sections or use knowledge-base search (RAG), which puts only the matching fragments into the request.
• Put a repeated large block at the start of the request. Cache hits depend on an unchanged prefix.
• Limit max_tokens so the answer does not occupy the window or raise the cost.

The simplest trimming to a budget:

def trim(messages, budget):
    # drop the oldest question-answer pairs until the input fits the budget
    while len(messages) > 2:
        used = client.messages.count_tokens(model=MODEL, messages=messages).input_tokens
        if used <= budget:
            break
        messages = messages[2:]
    return messages

Messages are removed two at a time so that roles keep alternating and the first message stays a user message.

FAQ

What is a context window in simple terms?

It is the amount of text, measured in tokens, that a model sees in one request. Anything that did not fit does not exist for it. It includes both your request with the history and the answer.

Which neural network has the largest context window?

Among Zukko models as of October 18, 2026, most have the maximum window of 1M tokens. Grok has 256K and GLM has 128K. Check current values in GET /v1/models.

Does a model forget the start of a conversation?

If the whole history fits in the window and you send it in full, the model sees it. "Forgetting" is more often caused by the client trimming the history. How well a model uses facts from different parts of a long text is something to test on your own data.

Is a large context expensive?

You pay for the tokens you actually send, not for the window size. But a long history is sent with every request, so use the cache in the Anthropic format and trim what you do not need.