Temperature and top_p: what they do and where you can no longer set them

Temperature and top_p are two parameters that used to tune how "random" a language model's answers are. Today they need more care: some newer models do not accept them and return an error. Here is what temperature and top_p do, how to check whether a model supports them, and how to steer answers when they are unavailable.

What is temperature

A model generates text one token at a time: at each step it gets a probability distribution over the next token and picks from it. temperature reshapes that distribution before the pick.

• Low temperature (close to 0): the distribution gets "sharper" and the most likely tokens are chosen almost every time. Answers are more uniform.
• temperature = 1: the distribution is unchanged.
• High temperature: less likely tokens get a bigger chance, answers vary more, and so do mistakes.

Even temperature = 0 does not guarantee the same answer to the same request; the Anthropic Messages API reference↗ states this directly. If you need reproducibility, store the answers instead of relying on the parameter.

What is top_p

top_p (nucleus sampling) limits the candidate set. Tokens are sorted by probability, and the model picks only among the top ones whose cumulative probability reaches top_p.

• top_p = 1: all tokens are considered.
• top_p = 0.9: the "tail" of unlikely options holding 10% of the probability is cut off.
• top_p = 0.1: only the most likely tokens, together worth 10%, remain.

Note: this is a share of probability, not "the top 10% of words".

The key limitation: newer Claude models

According to the Anthropic Messages API reference↗, models released after Claude Opus 4.6 do not support setting these parameters:

• temperature is accepted only as 1.0; any other value returns a 400 error;
• top_p is accepted only at 0.99 or above; other values return a 400 error.

In the Zukko catalog this applies to claude-sonnet-5 and claude-opus-5-5. In practice: do not send these parameters to Claude models; set the behavior you need through instructions. If a request with them returns 400, remove temperature and top_p first.

For GPT and GLM models, parameter support and allowed ranges may differ. It is safer to check than to guess.

How to check support on any model

Send a short request with the value you need and see whether the model accepts it. Such a request costs a fraction of a cent.

import openai
from openai import OpenAI

client = OpenAI(base_url="https://api.zukko.pro/v1", api_key="fetch_your_key")

def accepts(model: str, **params) -> bool:
    try:
        client.chat.completions.create(
            model=model,
            max_tokens=5,
            messages=[{"role": "user", "content": "ok"}],
            **params,
        )
        return True
    except openai.BadRequestError as e:
        print(model, params, "→", e.message)
        return False

for model in ["gpt-5.6-sol", "gpt-5.6-luna", "glm-5.3", "glm-5.3-flash", "claude-sonnet-5"]:
    print(model, "temperature=0.2:", accepts(model, temperature=0.2))

Run this check before a rollout and after switching models. The current model list is at GET https://api.zukko.pro/v1/models or on the "Models" page in the dashboard.

Tuning order when the parameters are supported

1. Start without parameters, with the defaults.
2. Collect 10–20 real requests and a criterion for a good answer.
3. Change one parameter, usually temperature: for example 0.2, 0.5, 0.8. Run the whole set for each value.
4. Compare the share of good answers and pick a value.
5. Touch top_p only if temperature alone was not enough, and do not change both at once, or you will not know which one had the effect.

Common starting points: low temperature for data extraction, SQL and JSON; medium for prose; higher for brainstorming. These are only starting points: test on your own tasks.

Steering answers without temperature

For models that reject the parameter, and as a more reliable approach in general:

• Strict format. Ask for a single word, a fixed set of values or JSON matching a schema. A narrow format reduces variation by itself.
• Examples in the prompt. Two or three "input → output" examples set the style more precisely than a number.
• Explicit rules. "Do not add explanations", "use only data from the text".
• Variety on request. If you need different options, ask for "5 clearly different options" in one request instead of five requests at high temperature.

An example for Claude via the Anthropic SDK, with no temperature and strict instructions:

import anthropic

client = anthropic.Anthropic(base_url="https://api.zukko.pro", api_key="fetch_your_key")

response = client.messages.create(
    model="claude-sonnet-5",
    max_tokens=256,
    system="Extract the email from the text. Reply with the address only, no explanations. If there is none, reply NONE.",
    messages=[{"role": "user", "content": "Write to me at ivan@example.com, I'll reply tonight"}],
)
print("".join(b.text for b in response.content if b.type == "text"))

And a brainstorming example with options in one request:

response = client.messages.create(
    model="claude-opus-5-5",
    max_tokens=1024,
    messages=[{
        "role": "user",
        "content": "Suggest 5 names for a flower delivery service. "
                   "They should differ noticeably in style. List only.",
    }],
)
print("".join(b.text for b in response.content if b.type == "text"))

Temperature via the OpenAI SDK

If the check showed the model accepts the parameter:

response = client_openai.chat.completions.create(
    model="glm-5.3-flash",
    temperature=0.2,
    messages=[{"role": "user", "content": "Translate to Russian: 'The invoice has been paid'"}],
)
print(response.choices[0].message.content)

Here client_openai is an OpenAI SDK client with the address https://api.zukko.pro/v1. More on parameters in the Chat Completions↗ and Messages↗ references.

FAQ

What is the default temperature?

For Anthropic, 1.0 per the reference. For other vendors, see their docs. If you do not send the parameter, the model uses its own default, which is the safest option.

Why did my Claude request return a 400 error?

A common cause when porting old code is temperature or top_p with a disallowed value. Newer Claude models accept only temperature 1.0 and top_p of 0.99 or above. Remove these parameters from the request.

Does temperature affect cost?

The per-token price does not change. But parameters and instructions affect answer length, and therefore the number of output tokens. Compare usage in responses across settings.

What if a tool (chat UI, plugin) sends temperature on its own?

Many tools pass generation parameters from their settings. If you get a 400 error with Claude models, find the generation parameters in the tool's settings and disable them or reset them to defaults; check the tool's documentation for the exact setting name.