OpenAI Batch API: what it is, when it pays off and what to use via a gateway

The OpenAI Batch API is a way to send many requests as a file, not wait for the answer, and collect the results later at a reduced price. Anthropic has a similar mechanism. Below we explain how it works, when it pays off, and how to get a comparable result through a gateway that has no batch endpoints. The description is based on the OpenAI and Anthropic documentation at the time of checking: terms change there, so verify the figures against the providers' current pages.

What a Batch API is

A normal request is synchronous: you send it, wait, get an answer. Batch mode works differently: you collect requests into one file, upload it, start a job, and later fetch a file with the results. The price of giving up an instant answer is a discount and higher limits.

How it works at OpenAI

According to OpenAI's Batch API guide:

1. A .jsonl file is prepared in which each line is a separate request with a required unique custom_id, a method, an endpoint address and a request body. The body is the same as for a regular request to that endpoint.
2. Only one model is allowed per file.
3. The file is uploaded through the Files API with the purpose batch.
4. A job is created; the completion window can currently only be 24h.
5. The status goes through validating, in_progress, finalizing and ends in completed, or in failed (the file failed validation), expired (missed the deadline) or cancelled after cancelling.
6. Results are fetched by the output file identifier, and errors of individual requests are in an error file.

The endpoints named in the guide include /v1/responses, /v1/chat/completions, /v1/embeddings, /v1/completions, /v1/moderations and image requests. The stated advantages: a 50% discount against synchronous prices, separate and substantially higher limits, and completion within 24 hours, often faster. There are per-batch limits: according to the guide, up to 50,000 requests and up to 200 MB per input file.

An example input line:

{"custom_id": "doc-1", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "<model>", "messages": [{"role": "user", "content": "Classify the text..."}], "max_tokens": 200}}

How it works at Anthropic

At Anthropic this is the Message Batches API. According to the docs: a batch is limited to 100,000 requests or 256 MB, most batches finish within an hour, but a batch expires if it does not complete within 24 hours, results are kept for 29 days, and the price is 50% of the standard one. Streaming (stream: true) is not supported in batches. For batches with shared context the docs advise using one-hour cache duration, because processing can take longer than five minutes.

When batch mode pays off

• The result is not needed right now: nightly reports, dataset labeling, classifying an archive.
• There are many requests and you hit the rate limits of the regular API.
• The task is judged in money, not latency: evaluating models on a test set, mass rework of texts.

It does not fit where a user waits for an answer in a chat, where the next request depends on the previous answer, and where streaming output is needed.

What to use instead through a gateway

The Zukko gateway's endpoint list (chat requests in OpenAI and Anthropic formats, /v1/responses, the model list, token counting) has no batch endpoint, so plan processing with regular requests. The "lots and cheap" effect comes from three techniques.

1. Parallel requests with a cap. Run several requests at once, but not unlimited: keep the number of simultaneous requests within what the model can handle. Retry 429 and 503 errors with a pause.
2. A cheap model. For labeling and classification, start with a model from the lower part of the table and test it on a sample of your data; move to a pricier one only if the cheap one falls short.
3. Caching a shared prefix. If every request starts with the same long instruction, reading it from cache costs less than input. For Claude this works in the Anthropic format; for GPT the cache is visible in both formats. More in the limits and cache docs↗.

A Python skeleton with a concurrency cap and a retry on overload:

import asyncio
from openai import AsyncOpenAI, APIStatusError

client = AsyncOpenAI(base_url="https://api.zukko.pro/v1", api_key="fetch_your_key")
sem = asyncio.Semaphore(8)  # at most 8 requests at once

async def classify(doc_id: str, text: str):
    async with sem:
        for attempt in range(4):
            try:
                r = await client.chat.completions.create(
                    model="deepseek-v4.1-flash",
                    messages=[
                        {"role": "system", "content": "Answer in one word: the topic of the text."},

The dictionary key plays the role of custom_id here: a result is always tied to its source document. A fuller version with retries and spend tracking is in the article async requests in Python↗, and error handling is covered in retry, 429 and 503↗.

Comparison in money

Suppose you need to process 10,000 documents, with 1000 input tokens and 200 output tokens each. That is 10M input tokens and 2M output tokens. Prices as of October 18, 2026, cache not included.

Official batch mode takes 50% of the provider's regular price (per the providers' docs). For claude-sonnet-5-5 the regular official price in our table is 2 / 10 per million:

• regular mode: 10 × 2 + 2 × 10 = 20 + 20 = $40;
• batch mode at 50%: $20.

Through the gateway with regular requests, Zukko prices:

• claude-sonnet-5-5 (0.4 / 2): 10 × 0.4 + 2 × 2 = 4 + 4 = $8;
• gpt-6.1-sol (0.27 / 1.32): 10 × 0.27 + 2 × 1.32 = 2.7 + 2.64 = $5.34;
• deepseek-v4.1-flash (0.013 / 0.038): 10 × 0.013 + 2 × 0.038 = 0.13 + 0.076 = $0.206.

Draw conclusions carefully. These numbers compare different things: the batch discount applies to official prices, while Zukko prices are lower than official on their own. You also pay a payment page fee when topping up (SBP adds 8.5% on top), and different models perform differently on your task. Choose a model by a sample of your data, not by the table.

The current list and prices: GET https://api.zukko.pro/v1/models or the "Models" page in your account.

FAQ

Is a Batch API cheaper than a regular one?

At OpenAI and Anthropic, yes: per their docs the discount is 50% off regular prices, in exchange for results arriving within a day. The gateway has no separate batch discount: its advantage comes from a low token price and caching.

Does the Zukko gateway have a batch mode?

The gateway documentation has no batch endpoint. Use parallel requests with a cap, as in the example above.

Can I get the result immediately?

Not in batch mode. If you need a quick answer, send regular requests.

How do I avoid hitting limits in bulk processing?

Cap the parallelism, retry 429 and 503 with a pause, and split the work across several keys with daily limits. Details are in bulk processing without hitting the rate limit↗. Create a key in your account↗; access from Russia goes through the gateway, with payment by SBP, MasterCard or crypto.