Microsoft AutoGen: pointing it at your own key instead of OpenAI

AutoGen builds its client on top of the plain openai Python package, so switching providers comes down to base_url and api_key inside config_list. There is no separate adapter needed for Anthropic or GLM models — every model on the gateway answers in the /v1/chat/completions shape that AutoGen already understands.

That's convenient precisely because AutoGen doesn't try to guess the provider from the model name the way some LiteLLM-based wrappers do. It simply builds openai.OpenAI(base_url=..., api_key=...) and then calls chat.completions.create() with whatever model you set in the config. So the setup experience is identical for GPT, Claude, and GLM — the only difference is the model name string, not the connection architecture.

Basic config_list

import autogen

config_list = [
    {
        "model": "gpt-5.6-sol",
        "base_url": "https://api.zukko.pro/v1",
        "api_key": "fetch_your_key",
        "default_headers": {"X-Zukko-Cache": "true"},
    }
]

assistant = autogen.AssistantAgent(
    name="assistant",
    llm_config={"config_list": config_list},
)

user_proxy = autogen.UserProxyAgent(name="user_proxy", code_execution_config=False)
user_proxy.initiate_chat(assistant, message="Draft a database migration plan")

The X-Zukko-Cache: true header in default_headers turns on the gateway's response cache — that's the primary way, and it works for any model (unlike Idempotency-Key, this header is fine to set client-wide). temperature: 0 also enables the cache, but only for models that accept that value: newer Claude models (claude-sonnet-5, claude-opus-5-5) may reject a temperature other than 1 with a 400, so we don't set temperature in llm_config for them. A request whose body exactly matches an earlier one returns instantly and for free on repeat, with an X-Zukko-Cache: hit header. "Nearly identical" requests (say, with one more turn appended to the history) no longer match; cache hits are not written to /v1/usage. Chat responses stay cached for up to 7 days, which helps when you rerun the same AutoGen prompt ten times in a row while debugging.

Different agents, different models

config_list expects the field api_key, not apiKey — a casing typo silently breaks the connection, and AutoGen fails with an authorization error that looks like a bad key even though the key itself is correct. A second common pattern is giving each agent in a GroupChat its own model:

critic = autogen.AssistantAgent(
    name="critic",
    llm_config={
        "config_list": [{
            "model": "claude-opus-5-5",
            "base_url": "https://api.zukko.pro/v1",
            "api_key": "fetch_your_key",
        }],
    },
)

coder = autogen.AssistantAgent(
    name="coder",
    llm_config={

Claude Opus 5.5, like the other Claude and GPT models in the catalogue, carries a 1,000,000-token context and accepts images (GLM has 128K and no vision, so a coder on glm-5.3 can't read pictures) — not needed for code review, but useful the moment you add an agent that reads UI screenshots:

message = {
    "role": "user",
    "content": [
        {"type": "text", "text": "What's wrong with this layout?"},
        {"type": "image_url", "image_url": {"url": screenshot_data_url}},
    ],
}
user_proxy.initiate_chat(ui_reviewer, message=message)

The image message format is the standard OpenAI image_url shape, and it works the same for any vision-capable model (Claude, GPT) behind config_list.

A model gets switched off mid-run

The catalogue changes over time, and a model can be switched off while a long multi-round scenario is already running. In that case the gateway answers 503 with the code model_disabled (that's what happened to gpt-5.4, for instance — its replacement is gpt-5.6-sol):

{"error": {
  "message": "...",
  "code": "model_disabled"
}}

In long-lived AutoGen scenarios it's worth catching model_disabled separately from other errors and swapping in a fallback model you picked in advance, once, instead of letting the whole GroupChat crash. The current model list is at GET https://api.zukko.pro/v1/models:

try:
    user_proxy.initiate_chat(assistant, message=task)
except Exception as e:
    if "model_disabled" in str(e):
        assistant.llm_config["config_list"][0]["model"] = "gpt-5.6-sol"
        user_proxy.initiate_chat(assistant, message=task)
    else:
        raise

Retries on top of openai-python's own retries

The openai.OpenAI client that AutoGen is built on has its own retry mechanism — up to two extra attempts on network errors and 5xx responses by default, before your own code even finds out. The Idempotency-Key is set per logical request — through extra_headers on the specific chat.completions.create() call. Then all internal retries of that call go out with the same header value — which is exactly right: the gateway sees the repeat and doesn't charge twice. What you must not do is put the key in the client's default_headers: it then goes out with every request, and the second request with a different body already gets 422 idempotency_conflict. The other mistake is generating a fresh key on every retry — then each attempt gets a new key, and idempotency stops working precisely where it matters most: on a flaky connection.

A 401 with a key that looks fine

If config_list is assembled from environment variables via autogen.config_list_from_json, it is easy to leave a stale api_type field. AutoGen expects "openai" by default, and if the JSON still says "azure" from an old Azure OpenAI config, the request goes out with a different header structure and the gateway answers 401 with Invalid API key, even though the key itself was copied correctly.

429 inside a group chat

In practice, a group chat with four or five agents and max_round around twenty easily runs into dozens of calls per debugging session, and running such sessions in a loop while iterating on prompts pushes that into the hundreds per hour.

GroupChatManager can fire off a dozen consecutive calls within seconds as agents pass the turn around. If a fetch_... key has a daily or monthly spend limit set on its page in the dashboard, a group chat with a high max_round hits key_daily_limit_reached faster than a single agent would. The daily limit resets at 00:00 UTC; the gateway has no rate limit of its own, so a 429 without key_daily_limit_reached / key_monthly_limit_reached comes from the model provider. While debugging, it helps to lift the limit on a test key or keep a separate unrestricted key just for development.

Idempotency on retries

A UserProxyAgent with code_execution_config enabled sometimes resends the same request when a code container times out. Don't try to guard against that through default_headers in config_list: that field goes into the openai.OpenAI constructor, so every request the agent makes would carry the same Idempotency-Key, and the very next round with a new body fails with 422 idempotency_conflict. The key belongs to each logical request — unique to that request and identical across its retries. For calls that must not be doubled, make them directly through the client and pass the key on the specific .create():

import uuid
from openai import OpenAI

client = OpenAI(base_url="https://api.zukko.pro/v1", api_key="fetch_your_key")
idem_key = str(uuid.uuid4())  # one per logical request; the same on retry

resp = client.chat.completions.create(
    model="gpt-5.6-sol",
    messages=[{"role": "user", "content": task}],
    extra_headers={"Idempotency-Key": idem_key},
)

Repeating the same Idempotency-Key value within 24 hours returns the same answer without charging again.

Check before running the scenario

curl https://api.zukko.pro/v1/chat/completions \
  -H "Authorization: Bearer fetch_your_key" \
  -H "Content-Type: application/json" \
  -H "X-Zukko-Cache: true" \
  -d '{"model":"gpt-5.6-sol","messages":[{"role":"user","content":"ping"}]}'

If the response carries X-Zukko-Cost-Micros, the call reached the model and was charged; the same request repeated will come back with X-Zukko-Cache: hit and charge nothing. Balance and spend history are available through GET /v1/balance and GET /v1/usage, and keys are issued in the dashboard at https://zukko.pro/app/api-keys↗. If a config is assembled dynamically from several JSON files and it's unclear where a field got dropped, it's faster to ask support on Telegram (https://t.me/zukkopro?direct↗) than to keep guessing from openai.OpenAI logs alone.