OpenAI Python SDK: timeouts, retries, errors and raw responses

Most examples with the openai library for Python end at client.chat.completions.create(...). In production the questions begin: how long to wait for an answer, what to retry automatically, how to tell a bad key from an empty balance, where to get response headers. Below are the client settings of the openai python sdk that are rarely explained, shown on an example of working through an OpenAI-compatible gateway.

The description is checked against the openai-python README on 18 October 2026. SDK versions change quickly (the README of that date, for example, already mentions the httpx2 HTTP client), so check your installed version with python -c "import openai; print(openai.__version__)" and verify details against the README and changelog.

The client and environment variables

A minimal client for the Zukko gateway:

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.zukko.pro/v1",
    api_key=os.environ["ZUKKO_API_KEY"],
)

The address for the OpenAI format ends in /v1. Instead of parameters you can set the environment variables OPENAI_BASE_URL and OPENAI_API_KEY, and then plain OpenAI() is enough in code. Create the fetch_... key in the dashboard↗. The library's README names the Responses API as the primary way to generate text, while Chat Completions remains supported. Both are available on the gateway: /v1/chat/completions and /v1/responses. The examples below use chat.completions, but the client settings are the same for both.

Timeouts

By default a request waits up to 10 minutes. For an interactive app that is a lot. The timeout is set as a number of seconds:

client = OpenAI(
    base_url="https://api.zukko.pro/v1",
    api_key=os.environ["ZUKKO_API_KEY"],
    timeout=30.0,
)

long_client = client.with_options(timeout=180.0)  # for one heavy request

The with_options method returns a copy of the client with changed settings; the original client stays as it was. Instead of a number you can pass an object with separate timeouts for connect, read and write. But the type of that object depends on the SDK version: in the README at the time of checking it is httpx2.Timeout, in older versions httpx.Timeout. If you do not want to depend on the version, stay with a number.

On expiry openai.APITimeoutError is raised. Note that a request that timed out is retried twice more by default. Three 30-second timeouts mean a minute and a half of waiting before you see the error.

For long generations it is more reliable to turn on streaming: the read timeout applies to pauses between pieces, not to the whole answer. More: timeouts on long generations↗.

Retries: what is repeated and what is not worth repeating

By default the SDK automatically retries requests 2 times with exponential backoff. Connection errors, responses 408, 409, 429 and all 5xx are retried, and only if the request body can be safely resent. It is configured through max_retries:

client = OpenAI(
    base_url="https://api.zukko.pro/v1",
    api_key=os.environ["ZUKKO_API_KEY"],
    max_retries=2,          # default is 2, 0 disables retries
)

r = client.with_options(max_retries=5).chat.completions.create(
    model="gpt-6.1-sol",
    messages=[{"role": "user", "content": "Hello"}],
)

For a gateway this rule has nuances worth knowing:

• 429. It may be a limit at the model provider, in which case a retry after a pause helps. But if the key's daily or monthly limit is exhausted (key_daily_limit_reached, key_monthly_limit_reached), retrying is pointless until the next period or until you raise the limit.
• 503. The provider is temporarily unavailable: a retry is fine. The model_disabled code means the model is switched off. Then you need to switch to another from GET /v1/models rather than retry.
• 401, 402, 403, 404. Not retried: wrong key, not enough balance, the key is not allowed the model, unknown model name.

If a request must be retried safely after a disconnect without paying twice, read about the Idempotency-Key header in Idempotency-Key: how to retry requests↗.

Error classes

The SDK turns responses into exceptions. All of them inherit from openai.APIError. Status codes have their own classes: 400 BadRequestError, 401 AuthenticationError, 403 PermissionDeniedError, 404 NotFoundError, 422 UnprocessableEntityError, 429 RateLimitError, 5xx InternalServerError. If a connection could not be made, it is APIConnectionError, and its subclass APITimeoutError means a timeout.

Mapping to gateway codes:

The order of except blocks matters: narrow classes first, then APIStatusError.

import openai

try:
    r = client.chat.completions.create(
        model="gpt-6.1-sol",
        messages=[{"role": "user", "content": "Hello"}],
    )
except openai.AuthenticationError:
    print("401: check the fetch_... key")
except openai.PermissionDeniedError as e:
    print("403: model not allowed for this key", e.body)
except openai.NotFoundError as e:
    print("404: wrong model name, compare with GET /v1/models", e.body)
except openai.RateLimitError as e:

e.body holds the parsed error body, and e.response.headers the headers. Which field (code or type) holds strings like model_not_found on the gateway is most reliably checked on a real response by printing e.body, and then branching on it. A general breakdown of codes: 401, 402, model not found↗.

Raw responses and headers

To get headers and status, add with_raw_response before the method:

raw = client.chat.completions.with_raw_response.create(
    model="deepseek-v4.1-flash",
    messages=[{"role": "user", "content": "Say 'test'"}],
)
print(raw.status_code)
print(dict(raw.headers))
completion = raw.parse()
print(completion.choices[0].message.content)

parse() returns the regular response object. For streaming there is with_streaming_response as a context manager. The request id lands in response._request_id if the server returned the corresponding header, and for failed requests it is in e.request_id. Which headers the gateway returns is best seen by printing dict(raw.headers): do not build logic on headers you have not seen in a live response.

Streaming and token accounting

In OpenAI-format streams Zukko always sends a final chunk with usage and an empty choices, so check if chunk.choices: before chunk.choices[0]:

stream = client.chat.completions.create(
    model="gpt-6.1-sol",
    messages=[{"role": "user", "content": "Explain retries in two sentences"}],
    stream=True,
)

usage = None
for chunk in stream:
    if chunk.choices:
        print(chunk.choices[0].delta.content or "", end="", flush=True)
    if chunk.usage:
        usage = chunk.usage

An error in the middle of a stream surfaces as APITimeoutError (read timeout) or APIConnectionError. A stream is not retried automatically: otherwise text already delivered could be duplicated. Decide yourself: restart the request entirely or return to the user what you managed to get.

From usage you can compute the cost right away. Prices as of 18 October 2026, dollars per 1M tokens: gpt-6.1-sol 0.27 for input and 1.32 for output, deepseek-v4.1-flash 0.013 and 0.038.

PRICES = {"gpt-6.1-sol": (0.27, 1.32), "deepseek-v4.1-flash": (0.013, 0.038)}

def cost(model, usage):
    p_in, p_out = PRICES[model]
    return usage.prompt_tokens / 1e6 * p_in + usage.completion_tokens / 1e6 * p_out

An example: 1,200 input tokens and 300 output tokens on gpt-6.1-sol give 0.0012 × 0.27 + 0.0003 × 1.32 = 0.000324 + 0.000396 = $0.00072. This is an upper estimate: for GPT part of the input may come from the cache (usage.prompt_tokens_details.cached_tokens) and is priced at the cache-read price, 0.013. Reconcile the total with the charges in the dashboard.

The async client and debugging

AsyncOpenAI takes the same parameters (base_url, timeout, max_retries), and calls are written with await. For bulk requests limit concurrency with asyncio.Semaphore, otherwise hundreds of simultaneous requests will hit 429 at the provider. To see what the SDK actually sends, turn on logging: export OPENAI_LOG=info (or debug for verbose output).

Access from Russia goes through the gateway, with payment by SBP (the payment page adds 8.5% on top), MasterCard, or crypto. More material: the Python reference↗.

FAQ

How do I change the timeout for just one request?

client.with_options(timeout=120.0).chat.completions.create(...). The original client does not change.

Why does a request with a 30-second timeout hang longer?

After a timeout the SDK retries up to two more times. Set max_retries=0 or a smaller timeout if you need a fast failure.

Do I need to catch 402 separately?

The SDK's table has no separate class for it, so catch APIStatusError and check status_code == 402.

How do I turn retries off completely?

OpenAI(..., max_retries=0). Turn retries off where you implement your own failure logic or duplicate protection.