How to compare API gateways honestly — six checks in half an hour

Choosing an intermediary for API access usually comes down to comparing price lists. That's the worst available method: price per million tokens says nothing about how many tokens you'll be charged for, or what happens when the provider goes down.

Here are six checks, each a few minutes.

1. Do charges match the response?

The most important one. Send a single non-streaming request and compare two numbers: usage in the model's reply and the charge in your transaction history.

curl -s https://api.zukko.pro/v1/chat/completions \
  -H "Authorization: Bearer fetch_your_key" \
  -H "Content-Type: application/json" \
  -d '{"model":"glm-5.3","max_tokens":50,
       "messages":[{"role":"user","content":"Say one word"}]}' \
  | python3 -c "import sys,json; print(json.load(sys.stdin)['usage'])"

Then open your charge history and find that request. The numbers should match to the token. If more was charged, ask why — a coherent answer is rare.

2. Is there a request history at all?

It sounds trivial, but some intermediaries simply don't have one: the balance drops and what for is anyone's guess. Without a per-model, per-request breakdown you can neither find overspending nor dispute a charge.

Check that history is available through an API call, not only by eye in a dashboard — otherwise you can't automate cost control.

3. What happens during an outage?

The most revealing point and the least often checked. Ask directly: what will I see if the model's provider goes down?

A bad answer is "nothing, everything always works". Providers go down for everyone; the only question is how it looks from your side. A good answer is specific: the model leaves the list, the refusal carries a clear message naming a working alternative, and the state is publicly visible.

You can check like this:

curl https://api.zukko.pro/api/status

If there's no public state at all, during the next outage you'll be guessing whose problem it is.

4. Does tool calling work?

Critical for anyone using agents, and almost never checked in advance. Send a request with a tool definition and confirm the reply contains a call, not prose suggesting you write the code by hand.

Check in the format you actually use. Anthropic clients hit /v1/messages, OpenAI-compatible ones /v1/chat/completions, and it happens that one works while the other doesn't.

5. Are both formats supported?

An Anthropic-compatible address and an OpenAI-compatible one are different things, and which tools you can connect depends on it. If only one is supported, half the clients are out.

While you're at it, check that /v1/models exists: without it your agent won't show you a list and you'll be typing names by hand.

6. Can you get your money back?

Prosaic but important. Clarify the refund terms for an unspent balance before topping up, not after. And don't load much the first time — a small amount is enough to run all six checks.

What's not on this list

Don't choose on advertised speed: it depends on the model and the provider's load, and a number on a reseller's site guarantees you nothing. Measure it yourself on your own requests if it matters.

Don't choose on model count either. Three working models beat thirty where half return model not found.

The upshot

Compare prices last, once the first five points check out. The price gap between providers can be severalfold, but the gap in billing honesty can be larger — and you discover that one after you've already built production on top of them.

Zukko↗ passes all six, and that claim is better verified than believed: top up the minimum ($1) and run the list yourself.