Routing between models by task and price

A common mistake in multi-provider model integrations is hardcoding one model for everything, usually the most powerful and most expensive one. It works, but the overpayment is not always justified — classifying text, extracting entities from JSON, or rephrasing a single sentence do not need the same reasoning depth as an architectural decision or a complex refactor. Task-based routing fixes this without losing quality where it actually matters.

Why use multiple models at all

A single fetch_... key gives access to models from different families: Claude (claude-opus-5-5 and claude-sonnet-5, $0.30/$0.30 per 1M tokens each), GPT-5.6 (gpt-5.6-sol at $0.30/$0.30, gpt-5.6-luna at $0.10/$0.10), and GLM (glm-5.3 at $0.10/$0.10, glm-5.3-flash at $0.04/$0.04). Claude and GPT have a 1,000,000-token context; GLM has 128K and no vision. Beyond that, the difference is price per token and response character. The current list is at GET https://api.zukko.pro/v1/models.

That means switching between them in code is just changing the model value in the request body; the protocol and response shape stay the same within a given API (Anthropic Messages or OpenAI Chat Completions).

Simple strategy: cheap model by default, expensive one on demand

The basic pattern is a router that looks at task type and picks a model:

import requests

ROUTES = {
    "classify": "glm-5.3-flash",
    "extract": "glm-5.3",
    "rewrite": "gpt-5.6-sol",
    "reasoning": "claude-opus-5-5",
    "code_review": "claude-sonnet-5",
}

def call_model(task_type, messages, max_tokens=1024):
    model = ROUTES.get(task_type, "claude-sonnet-5")
    resp = requests.post(
        "https://api.zukko.pro/v1/chat/completions",
        headers={"Authorization": "Bearer fetch_your_key"},
        json={"model": model, "messages": messages, "max_tokens": max_tokens},
    )
    resp.raise_for_status()
    return resp.json()

Sentiment classification or tag labeling goes to glm-5.3-flash, the cheapest model in the catalog, while an architectural decision goes to claude-opus-5-5. Their price per million tokens is $0.04 versus $0.30, and at high classification volume that shows up clearly on the monthly bill.

Routing by context length

You do not always need a difference in model "intelligence" — sometimes what matters is just having a large context window. Claude and GPT offer a 1,000,000-token context, while GLM has 128K and no vision, so the router does need an input-length criterion: send long documents and requests with images to Claude or GPT, and hand short text tasks to the cheaper GLM models.

Fallback on model failure

A model can be temporarily unavailable (503 provider_unavailable) or switched off (503 model_disabled). The router should be able to fall back to a backup model at the same tier:

FALLBACK = {
    "claude-opus-5-5": "claude-sonnet-5",
    "gpt-5.6-sol": "claude-sonnet-5",
}

def call_with_fallback(model, messages, max_tokens=1024):
    for attempt_model in [model, FALLBACK.get(model)]:
        if not attempt_model:
            break
        resp = requests.post(
            "https://api.zukko.pro/v1/chat/completions",
            headers={"Authorization": "Bearer fetch_your_key"},
            json={"model": attempt_model, "messages": messages, "max_tokens": max_tokens},
        )
        if resp.status_code == 503:
            continue
        resp.raise_for_status()
        return resp.json()
    raise RuntimeError("All models in the chain are unavailable")

If the response carries the model_disabled code, the model is switched off rather than temporarily unavailable — log it separately and update ROUTES against the current GET /v1/models, rather than hardcoding the old id forever.

Routing by cost via X-Zukko-Cost-Micros

Every regular (non-streaming) response carries an X-Zukko-Cost-Micros header — the actual request cost in micro-dollars (1,000,000 = $1); streaming responses (stream: true) do not include it. This is useful not for picking a model upfront but for calibrating the router afterward: if accumulated statistics for extract show glm-5.3-flash delivers acceptable quality for less than glm-5.3, you can shift the threshold without touching business logic, just the ROUTES config.

cost_micros = int(resp.headers.get("X-Zukko-Cost-Micros", 0))
balance_micros = int(resp.headers.get("X-Fetch-Balance-Cents", 0))  # despite the name — micro-dollars: 1_000_000 = $1

A week or two of cost metrics gathered by task type usually surfaces quickly where you are overpaying, and where, conversely, a cheap model fails often enough to erase the savings.

Saving through cache instead of routing

A separate case is not different tasks but the same request repeated. If the X-Zukko-Cache: true header is present (temperature: 0 also enables the cache, but only for models that accept it — Claude models may reject a temperature other than 1 with a 400), requests from the same user with an exactly matching body are cached (for chat, up to 7 days), and a repeat call returns instantly and for free — the response carries X-Zukko-Cache: hit. For a router, this means that before reaching for an expensive model on a repeated request (say, classifying the same text again), it can be cheaper to turn on the cache with the X-Zukko-Cache: true header first — the saving comes from the cache, not from switching models.

Idempotency-Key on retries

If the router retries after a network error (not a 503, but a dropped connection), it is important not to double-charge. The Idempotency-Key header with the same value within 24 hours guarantees that a repeat request with the same body returns the same response without being billed again. The key is set per logical request — unique to it and identical across all its retries; do not regenerate it on retry and do not put it in the client's default headers:

curl https://api.zukko.pro/v1/chat/completions \
  -H "Authorization: Bearer fetch_your_key" \
  -H "Idempotency-Key: retry-task-4821" \
  -H "Content-Type: application/json" \
  -d '{"model":"claude-sonnet-5","messages":[{"role":"user","content":"..."}]}'

This matters especially for a router with automatic retries — without an idempotency key, a request that failed on the client side but actually succeeded on the server can end up billed twice.

Bottom line

Routing by task and price is not complicated infrastructure — it is a lookup table, a fallback chain, and monitoring via X-Zukko-Cost-Micros. The model catalog and current prices are available without a key through GET /v1/models/info; keep ROUTES synced against real ids instead of relying on memory of which models still exist.