Common errors when sending images to vision models

A broken text-only request usually fails loudly and clearly — a 401, or a 400 with a readable message. A request with an image also tends to come back as 400, but the actual cause is almost never spelled out: the server sees "unrecognized field" or "invalid content," and you're left reconstructing where that field came from. Here's where images actually get lost, and how to debug it in a couple of minutes instead of an hour.

Anthropic and OpenAI encode images differently

This is the most common cause by far. On /v1/messages (Anthropic schema), an image is a content block with an explicit media_type:

curl https://api.zukko.pro/v1/messages \
  -H "Authorization: Bearer fetch_your_key" \
  -H "Content-Type: application/json" \
  -H "anthropic-version: 2023-06-01" \
  -d '{
    "model": "claude-sonnet-5",
    "max_tokens": 512,
    "messages": [{
      "role": "user",
      "content": [
        {"type": "image", "source": {"type": "base64", "media_type": "image/png", "data": "iVBORw0KG..."}},
        {"type": "text", "text": "What is in this image?"}
      ]
    }]
  }'

On /v1/chat/completions (OpenAI schema), it's an image_url with a data URI packed into a string, no separate media_type field:

curl https://api.zukko.pro/v1/chat/completions \
  -H "Authorization: Bearer fetch_your_key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-5.6-sol",
    "messages": [{
      "role": "user",
      "content": [
        {"type": "image_url", "image_url": {"url": "data:image/png;base64,iVBORw0KG..."}},
        {"type": "text", "text": "What is in this image?"}
      ]
    }]
  }'

Paste an image_url block into a /v1/messages body, or the reverse, and the server won't recognize the structure — it returns 400 with no pointer to a specific field, because from its side it's just garbage inside content. The first thing worth checking on an image error: is the body actually built for the endpoint you're hitting.

A missing data-URI prefix

The OpenAI-compatible image_url.url needs the full data:image/png;base64, string before the actual data. Send raw base64 without the prefix, and some client SDKs won't add it for you — the model receives a string that doesn't parse as a URL at all. The Anthropic schema needs no prefix whatsoever: media_type and data are separate fields, and stuffing data:image/png;base64, into the data field breaks the request just as reliably, because the model expects clean base64 there.

media_type doesn't match the actual file format

image/jpeg on a file that's actually PNG (often after an on-the-fly conversion) is a common one — the base64 is clearly valid, the key is fine, and yet the model either can't read the image or describes something that doesn't match it. It's trivial to check: look at the file's first bytes, or just re-save it in the declared format before encoding rather than trusting the filename's extension.

Wrong endpoint: vision input vs image generation

The confusion here is structural. Images as input through /v1/messages or /v1/chat/completions are read by the Claude and GPT models — claude-opus-5-5, claude-sonnet-5, gpt-5.6-sol, gpt-5.6-luna. glm-5.3 and glm-5.3-flash have no vision — an image request to them needs to be redirected to Claude or GPT. The current model list is at GET https://api.zukko.pro/v1/models. And reading images is not the same thing as generating them. Image generation through Zukko is temporarily unavailable; when it was running, models like nano-banana-2-text-to-image were a separate kind (kind=image), billed per image, and lived only on /v1/images/generations — sending such an id to /v1/chat/completions returned model_not_found. For reference, this is what the request looked like:

curl https://api.zukko.pro/v1/images/generations \
  -H "Authorization: Bearer fetch_your_key" \
  -H "Content-Type: application/json" \
  -d '{"model": "nano-banana-2-text-to-image", "prompt": "a city at sunset, watercolor"}'

Estimate the cost before sending a large image

Images consume noticeably more tokens than they look like they should, especially at high resolution. Before looping a request with a large file back and forth, it's cheaper to run the body through the free token count first:

curl https://api.zukko.pro/v1/messages/count_tokens \
  -H "Authorization: Bearer fetch_your_key" \
  -H "Content-Type: application/json" \
  -H "anthropic-version: 2023-06-01" \
  -d '{"model": "claude-sonnet-5", "messages": [{"role":"user","content":[{"type":"image","source":{"type":"base64","media_type":"image/png","data":"iVBORw0KG..."}}]}]}'

This costs nothing and never reaches the model — it only counts tokens.

Debug fast instead of guessing from the error code

Build a body with a 1x1 pixel image — a string small enough to hold in an environment variable. If that request goes through and your real file fails, the problem is in the encoding or size of that specific file, not the request shape. If the 1x1 version also fails, the body's shape itself is wrong: wrong endpoint, wrong field name, a missing media_type. That single comparison splits two whole categories of bug apart instead of guessing blind.

The general errors — 401 for a bad key, 402 for an empty balance, 429 when a key's limit is hit — apply to image requests exactly like text ones: the auth and balance checks run before the server even looks at what's inside content.

What looks like a vision error but isn't

Sometimes the image has nothing to do with it. A 503 provider_unavailable means the model's provider is temporarily down, and 503 model_disabled means the model is switched off in the catalogue. That's about model availability, not image format, and it's fixed by switching the model in the request, not by re-encoding base64. A separate case is a model without vision: glm-5.3 and glm-5.3-flash don't accept images, and the issue there is the model choice, not the encoding. Worth running the same model on a text-only request first — if it fails with the same code with no image attached, the problem is the model or the key, not vision, and there's no point debugging base64 encoding at that point.

Another source of confusion: a stream that cuts off mid-answer when an image is part of the input almost always means a client or network timeout, not the model choking on the picture. Images add processing time, and if the HTTP client is configured with a short timeout, running the request through count_tokens beforehand helps decide whether to raise that timeout ahead of time rather than after a real user's connection drops.