Which model writes the best Python code

Any fixed answer to "which model writes the best Python" goes stale the moment the catalog updates — models change, and everyone's tasks differ. The working approach isn't hunting for someone else's ranking, it's running your actual task through a few candidates and comparing by real output and price. Here's how to do that in one evening instead of a week.

Candidates worth comparing

For code it's worth comparing at least three or four models with different profiles: claude-opus-5-5 (Anthropic's flagship) or claude-sonnet-5, gpt-5.6-sol/gpt-5.6-luna (the GPT-5.6 lineup), glm-5.3 or glm-5.3-flash (current list: GET https://api.zukko.pro/v1/models). Claude and GPT have vision and a 1M context, GLM has 128K and no vision, so keep that in mind for a large module or screenshots; beyond that, the difference shows up on your specific task and in price per token.

Extended thinking isn't equally useful for every task

On Claude, extended thinking is turned on explicitly, through the thinking parameter on a /v1/messages request:

curl https://api.zukko.pro/v1/messages \
  -H "Authorization: Bearer fetch_your_key" \
  -H "Content-Type: application/json" \
  -H "anthropic-version: 2023-06-01" \
  -d '{
    "model": "claude-opus-5-5",
    "max_tokens": 4096,
    "thinking": {"type": "enabled", "budget_tokens": 2048},
    "messages": [{"role": "user", "content": "Why is this test failing: ..."}]
  }'

On tasks like "write a function from this description," turning thinking on usually makes a small difference. On "why does this test fail after the refactor" or "find the race condition in this async code," extended thinking tends to matter more, because the task needs several steps of reasoning rather than generating code from a blank page.

A comparison method that uses your own data

Take 5-10 real tasks from your project — not abstract "write a sort function" prompts, actual tickets or PR reviews. Run the same prompt through each candidate model with identical parameters (max_tokens, plus temperature: 0 for reproducibility only where the model accepts it: Claude models may reject a temperature other than 1 with a 400, so don't pass that parameter to them), then run the output through your own test suite — pytest, mypy, whatever you use. Eyeballing "I like this one" isn't a method, it's a guess; running it through your project's real tests is.

Track price alongside quality

Every regular (non-streaming) response carries an X-Zukko-Cost-Micros header — the exact cost of that specific request; responses with stream: true don't have it. Keep a running table of "model → task → tests passed (yes/no) → cost" as you go, instead of reconstructing it from memory afterward. What you get isn't an abstract "model X is better," it's a concrete "on my 8 tickets, model X solved 7 against model Y's 5, at 40% higher cost" — and from there you can decide whether the gap is worth it for your project.

Check context size ahead of time

If the task is a refactor across a whole module or reading through a large stack trace, count tokens first, for free:

curl https://api.zukko.pro/v1/messages/count_tokens \
  -H "Authorization: Bearer fetch_your_key" \
  -H "Content-Type: application/json" \
  -H "anthropic-version: 2023-06-01" \
  -d '{"model": "claude-opus-5-5", "messages": [{"role": "user", "content": "<module code>"}]}'

That way you don't hit a model's limit after you've already sent a paid request — the input is counted ahead of time, and for free.

Same model, two formats — don't mistake that for two candidates

If you're comparing through /v1/chat/completions and /v1/messages at the same time, remember that's two request formats for potentially the same model, not two different candidates. Don't count "the same claude-opus-5-5, called a different way" as a separate line in your comparison — it'll skew the conclusions.

Don't test against a switched-off model

If one of your candidates suddenly answers 503 with the model_disabled code, the model has been switched off; find a replacement in the current GET https://api.zukko.pro/v1/models list, update the model name in your test set and keep comparing against the current version instead of trying to hang onto the old identifier.

Show the model more than just code

If a bug is visual — broken layout, a chart rendering wrong — don't describe it in words. The Claude and GPT candidates on this list accept images (GLM has no vision), so you can send a screenshot together with the component's code in one request and ask what's inconsistent between what's rendered and what's written. For a "why does this test fail" question that's unnecessary, but for frontend bugs it often saves several rounds of clarification.

Don't burn budget on repeated comparison runs

If a five-to-ten-task comparison gets rerun several times while you tune the prompt, add an Idempotency-Key per unique task-plus-model request — that way an accidental script rerun after a crash doesn't charge twice for the same calls. And set a daily limit on the test key in the dashboard, so a ten-model comparison doesn't turn into a surprise bill if one candidate starts answering much longer than expected.

The bottom line

There's no single answer to "model X writes the best Python" independent of the task and codebase. There is a fast way to get a precise answer for your specific case: the same prompt, real tests instead of eyeballing the code, cost measured through X-Zukko-Cost-Micros, and free token counting wherever the input is large.