Saving on repeated requests: caching via X-Zukko-Cache and temperature:0

If an application regularly sends the same request over and over — a template status check, classifying repeated text, a test run of a prompt — there is no need to pay full price every time. Response caching on the gateway handles exactly this, and it does not turn on automatically by default; it triggers on one of two conditions.

When caching kicks in

There are two ways to get a cached response instead of a fresh model call:

The X-Zukko-Cache: true header. The primary method: it works with any model and regardless of temperature — the explicit header forces caching even if you never touch temperature.

temperature: 0 in the request body. This also enables caching: at zero temperature the model should already produce a deterministic (or near-deterministic) answer for the same input. But it only works for models that accept the parameter. claude-sonnet-5 and claude-opus-5-5 may reject a temperature other than 1 (and top_p below 0.99) with a 400 — with Claude models, don't pass temperature or top_p, and turn on caching with the header.

curl https://api.zukko.pro/v1/chat/completions \
  -H "Authorization: Bearer fetch_your_key" \
  -H "X-Zukko-Cache: true" \
  -H "Content-Type: application/json" \
  -d '{"model":"claude-sonnet-5","messages":[{"role":"user","content":"Translate the word table to Russian"}]}'

The cache is scoped per user — identical requests are cached within one account, not globally across different gateway users. The retention period for chat requests is up to 7 days, after which the cache expires and the next identical request goes back to the model.

How to tell a response came from cache

The response carries an X-Zukko-Cache header, with a value of hit on a cache hit:

curl -i https://api.zukko.pro/v1/chat/completions \
  -H "Authorization: Bearer fetch_your_key" \
  -H "X-Zukko-Cache: true" \
  -H "Content-Type: application/json" \
  -d '{"model":"claude-sonnet-5","messages":[{"role":"user","content":"What is HTTP 404?"}]}' \
  | grep -i x-zukko-cache

A cached response returns instantly and for free — the X-Zukko-Cost-Micros header in this case will be zero or absent, since no charge occurs. Cache hits do not show up in the /v1/usage history either.

Where this actually saves money

Repeated prompt testing. While debugging a system prompt, a developer typically runs the same input text dozens of times, changing only the model's system-role instruction. If the test input stays fixed and caching is on via the X-Zukko-Cache: true header, every repeat run with an identical request body costs nothing after the first.

Classifying repeated values. If a service classifies a data stream where values repeat (say, the same product category names, the same statuses), caching removes duplicate model calls for values already seen — as long as the surrounding prompt also stays unchanged.

Demos and CI tests. In automated tests where the scenario is fixed and runs on every push, the X-Zukko-Cache: true header gives you free repeat runs until the cache expires, and for models that accept temperature: 0 it adds determinism too (important for catching regressions).

What caching does not do

Caching only triggers on an exact match of the request — model, temperature, messages, and other body parameters. The smallest change (an extra space in the prompt, a different max_tokens, an added system message) makes it a different request, and caching does not apply. This is not semantic caching by meaning — it is an exact match on the request body.

Caching also does not replace Idempotency-Key. These are different mechanisms: Idempotency-Key protects against double billing on a retry of the same request within 24 hours (useful for network drops), while response caching is a separate feature about reusing a result for up to 7 days when the body matches. You can use both at once, but do not confuse them — idempotency is about reliability, caching is about repeatability and savings.

curl https://api.zukko.pro/v1/chat/completions \
  -H "Authorization: Bearer fetch_your_key" \
  -H "Idempotency-Key: batch-42-item-7" \
  -H "X-Zukko-Cache: true" \
  -H "Content-Type: application/json" \
  -d '{"model":"glm-5.3-flash","messages":[{"role":"user","content":"Classify: iPhone 15 Pro"}]}'

In this example both mechanisms work together: X-Zukko-Cache: true opens the door to caching on a repeated request, and Idempotency-Key protects this specific call from being billed twice on a network failure.

Checking cost before and after caching

To confirm caching is actually saving money, it helps to compare the headers of a first and second identical call:

import requests

url = "https://api.zukko.pro/v1/chat/completions"
headers = {"Authorization": "Bearer fetch_your_key", "Content-Type": "application/json", "X-Zukko-Cache": "true"}
body = {
    "model": "claude-sonnet-5",  # no temperature for Claude
    "messages": [{"role": "user", "content": "What is 17 * 24?"}],
}

first = requests.post(url, headers=headers, json=body)
second = requests.post(url, headers=headers, json=body)

print("first cache:", first.headers.get("X-Zukko-Cache"), "cost:", first.headers.get("X-Zukko-Cost-Micros"))
print("second cache:", second.headers.get("X-Zukko-Cache"), "cost:", second.headers.get("X-Zukko-Cost-Micros"))

The first call hits the model and gets billed; the second one — with an identical body and the X-Zukko-Cache: true header — should come back with X-Zukko-Cache: hit and zero cost.

When caching is the wrong tool

If the task needs variety in the answer to the same input — generating text variants, creative work where a repeat request should sound different each time — caching and temperature: 0 work against the actual goal. In these scenarios, do not enable caching (do not set X-Zukko-Cache: true and do not zero out temperature), or the user will keep getting the same variant instead of a new one.

Bottom line

Response caching is a free way to remove repeated spend on identical requests, but it requires a deliberate choice. The primary method is the explicit X-Zukko-Cache: true header: it works with any model. temperature: 0 is an alternative where determinism is already needed, but only for models that accept that parameter (not Claude). Checking for X-Zukko-Cache: hit in the response is the fastest way to confirm the saving is actually happening, not just assumed.