Automating customer support with Claude and GPT

A support bot in a messenger or on a website is one loop: customer question, system prompt with a knowledge base, answer. What separates a working bot from a demo mockup comes down to three things: what happens when a model is unavailable, who can spend how much, and where the bot pulls current information from when it isn't in the prompt.

The basic request

For a chat bot, the OpenAI-compatible mode is more convenient — most bot frameworks already speak this format:

curl https://api.zukko.pro/v1/chat/completions \
  -H "Authorization: Bearer fetch_your_key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-sonnet-5",
    "messages": [
      {"role": "system", "content": "You are a support agent for an online store. Answer from the knowledge base below; if there is no answer, suggest escalating to a human agent."},
      {"role": "user", "content": "Where is my order 48213?"}
    ]
  }'

claude-sonnet-5 is the workhorse here: reasonable price, a 1,000,000-token context with headroom for a long knowledge base in the system prompt, and vision for when a customer sends a screenshot of an error.

Fallback across providers

Support can't afford downtime — if a model is unreachable at night with no human agents on duty, the bot still needs to answer. The gateway's model catalog covers both Anthropic-compatible and OpenAI-compatible models at once, so a fallback chain can cross families instead of staying within one:

import requests

MODELS = ["claude-sonnet-5", "gpt-5.6-sol", "glm-5.3"]

def ask_support(question, kb):
    for model in MODELS:
        resp = requests.post(
            "https://api.zukko.pro/v1/chat/completions",
            headers={"Authorization": "Bearer fetch_your_key"},
            json={"model": model, "messages": [
                {"role": "system", "content": kb},
                {"role": "user", "content": question},
            ]},
        )
        if resp.status_code == 503:
            continue
        return resp.json()
    raise RuntimeError("All fallback models are unavailable")

503 provider_unavailable or model_disabled is the signal to move to the next model in the list rather than surface an error to the customer.

Idempotency on the messenger side

Telegram, the WhatsApp Business API, and similar platforms occasionally redeliver the same user message twice over an unstable connection. Without protection, the bot calls the model twice and gets billed twice for one customer question. An Idempotency-Key built from the platform's message ID closes that gap:

curl https://api.zukko.pro/v1/chat/completions \
  -H "Authorization: Bearer fetch_your_key" \
  -H "Idempotency-Key: tg-update-$UPDATE_ID" \
  -H "Content-Type: application/json" \
  -d @payload.json

Restricting operator key permissions

If several departments use the bot — say, sales support and technical support — each should get its own key with its own list of allowed models, set on the key's page in the dashboard. That guards against a scenario where a technical-support operator accidentally, or deliberately, switches the bot to the most expensive model for routine questions. If a key tries to call a model outside its list, the server returns:

{"error": {"message": "Model not allowed for this key", "code": "model_not_allowed_for_key"}}

A daily or monthly dollar cap on the same key protects against abnormal traffic — for instance, if the bot gets caught in a spam blast and receives thousands of identical messages within an hour.

Web search for questions outside the knowledge base

A system prompt with a knowledge base covers the routine questions but not everything — say, questions about a third-party carrier's current delivery status or recent company news. For models without built-in search, the gateway exposes a dedicated endpoint:

curl https://api.zukko.pro/v1/web-search \
  -H "Authorization: Bearer fetch_your_key" \
  -H "Content-Type: application/json" \
  -d '{"query": "carrier delivery delays September 2026"}'

The result can be folded into the next model call as an extra system block, rather than letting the model hallucinate an answer to something outside the knowledge base.

Monitoring conversation cost

Every regular (non-streaming) response carries X-Zukko-Cost-Micros (the cost of that specific call, in micro-dollars) and X-Fetch-Balance-Cents (the remaining balance — despite the name, also in micro-dollars, 1,000,000 = $1); responses with stream: true don't have these headers. Logging the first per conversation makes it easy to compute average cost per support ticket and compare it to an operator's salary for the same volume. GET /v1/balance is handy for a dashboard showing the current balance without opening the account panel:

curl https://api.zukko.pro/v1/balance -H "Authorization: Bearer fetch_your_key"

A referral setup for support agencies

If an agency runs support for several clients on separate keys, each client can have its own key under its own account, and the agency earns 10% on referred top-ups through the referral program — a convenient setup when the agency bills infrastructure separately from development work.

Common production errors

A 401 with an invalid-key message usually means the key was revoked or a new one was issued without updating the bot's environment variable. 402 means the balance is empty, and it's worth catching ahead of time with an hourly background check against /v1/balance rather than waiting for the first live customer to hit it. A response cutting off mid-answer almost always means the bot's HTTP client timeout is shorter than the time a long answer takes to generate — for detailed explanations, raise the timeout to 30-60 seconds instead of cutting the connection after 10.

Bottom line: several models in a fallback chain, separate capped keys per department, an Idempotency-Key on the platform's message ID, and web search for anything outside the knowledge base — with that, a support bot holds up under load without downtime or billing surprises.