CrewAI: connecting agents to Claude and GPT with one key

CrewAI groups agents into a crew: each has a role, memory, and its own llm. By default the framework talks to OpenAI through LiteLLM, but the LLM class accepts base_url and api_key, so instead of juggling ten provider keys you can use one fetch_... key and switch models right inside the agent definition.

In practice, almost all of the early pain with CrewAI has nothing to do with agent logic — it comes from LiteLLM assembling the request with official provider addresses in mind. The moment you switch to a third-party compatible gateway, three things surface: the right prefix on the model name, the right base_url (with or without /v1, depending on the protocol), and the fact that LiteLLM's automatic retries on network failures can double-charge a call unless you guard it with idempotency. Here's the order to work through them.

Basic setup

CrewAI follows LiteLLM's notation: the provider is a prefix on model, and the actual address comes from base_url. For the OpenAI-compatible mode it looks like this:

from crewai import Agent, Task, Crew, LLM

llm = LLM(
    model="openai/gpt-5.6-sol",
    base_url="https://api.zukko.pro/v1",
    api_key="fetch_your_key",
)

researcher = Agent(
    role="Researcher",
    goal="Gather facts on a topic and summarize them as a short list",
    backstory="Writes in Russian, checks sources, never invents numbers",
    llm=llm,
)

The openai/ prefix here is not about the model — it is about the protocol shape. LiteLLM sends the request to POST /v1/chat/completions, and which model actually answers is decided by the gateway based on the model field. That is why Claude and GLM are wired up the exact same way:

claude_llm = LLM(
    model="openai/claude-sonnet-5",
    base_url="https://api.zukko.pro/v1",
    api_key="fetch_your_key",
)

If you write model="claude-sonnet-5" without the openai/ prefix, LiteLLM tries to guess the provider itself and reaches for the native Anthropic SDK with its own default addresses — your base_url gets ignored. The prefix matters precisely because you are going through the OpenAI-compatible path /v1/chat/completions, not /v1/messages.

Different models for different agents

A crew rarely needs the same model tier for every role. A cheap GLM model for drafts, Opus 5.5 for the final check:

drafter = Agent(
    role="Drafter",
    goal="Sketch a fast draft from the brief",
    llm=LLM(model="openai/glm-5.3", base_url="https://api.zukko.pro/v1", api_key=key),
)

reviewer = Agent(
    role="Reviewer",
    goal="Check the draft for factual errors",
    llm=LLM(model="openai/claude-opus-5-5", base_url="https://api.zukko.pro/v1", api_key=key),
)

crew = Crew(agents=[drafter, reviewer], tasks=[draft_task, review_task])
crew.kickoff()

If your keys carry different daily limits (set on the key's page in the dashboard), it is worth issuing a separate fetch_... key per agent — that way the expensive Opus call can't eat the budget meant for cheap drafts.

An agent with web access

None of GPT, Claude, or GLM in this catalogue have built-in web search — that lives behind a separate POST /v1/web-search. In CrewAI it's easy to wrap as a plain tool and hand it to a research agent:

from crewai.tools import tool
import requests

@tool("web_search")
def web_search(query: str) -> str:
    """Looks up current information online."""
    resp = requests.post(
        "https://api.zukko.pro/v1/web-search",
        headers={"Authorization": "Bearer fetch_your_key"},
        json={"query": query},
    )
    return resp.json()

researcher = Agent(
    role="Researcher",
    goal="Find current data the model's base knowledge doesn't cover",
    llm=llm,
    tools=[web_search],
)

It's a separate call with its own billing line, not part of the chat request, so its cost shows up as its own entry in the spend history. For an agent that searches first and then summarizes the findings through the LLM, the task's total cost adds up from two separate lines in GET /v1/usage, worth accounting for when estimating the budget for a full crew run.

What a specific run actually cost

CrewAI exposes the raw response through llm.call() for manual calls outside an Agent, and the headers of the gateway's regular (non-streaming) response carry X-Zukko-Cost-Micros (cost in micro-dollars, 1,000,000 = $1) and X-Fetch-Balance-Cents (remaining balance after the charge — despite the name, also in micro-dollars; the exact balance in cents comes from GET /v1/balance). Streamed responses don't carry these headers. If you wrap LLM.call with your own logging, it's worth writing both headers to the log next to the agent's name — that shows which role in the crew spends the most, without a trip to the dashboard for every number.

That kind of logging also helps with billing: if a team invoices clients for agent usage, your own log of X-Zukko-Cost-Micros per key or agent is what gives the cost of the expensive role separately from the drafting one. GET /v1/usage can't provide that split: its history covers the whole account (all keys together, no per-key filter), and the balance belongs to the account. If you need a hard split per client, use a separate account for that client.

A 404 instead of an answer

The most common complaint in CrewAI threads is an agent that silently hangs or throws a connection error. Nine times out of ten it is a base_url missing /v1: the gateway's OpenAI-compatible path is https://api.zukko.pro/v1, not https://api.zukko.pro. The bare address is only for Anthropic-compatible clients that append /v1/messages themselves — LiteLLM's openai/ mode does not do that.

Model not found

If leftover code from an older CrewAI tutorial still references gpt-4 or claude-3-opus, you'll get:

{"error": {"message": "The model 'gpt-4' does not exist", "code": "model_not_found"}}

Neither exists in the current catalogue. Check the live list without a key:

curl https://api.zukko.pro/v1/models

Retries and double charges

LiteLLM inside CrewAI retries a few times on network failures by default. If the failure happened after the request already reached the model, a retry with the same body can charge twice. Ruling that out takes your own Idempotency-Key — but one per logical request, not one per LLM. Pass it to the LLM(..., extra_headers=...) constructor and it goes out with every call the agent makes, so the very next request with a different body gets 422 idempotency_conflict. For calls you make yourself, set the key on the specific call — once per request, never regenerated on retries:

import uuid, litellm

idem_key = str(uuid.uuid4())  # one key per logical request
resp = litellm.completion(
    model="openai/gpt-5.6-sol",
    api_base="https://api.zukko.pro/v1",
    api_key=key,
    messages=[{"role": "user", "content": prompt}],
    extra_headers={"Idempotency-Key": idem_key},
    num_retries=2,  # LiteLLM's retries reuse the same key
)

Repeating the same idempotency key within 24 hours returns the same response without charging again. For the calls CrewAI agents build themselves, don't set a shared key in the LLM constructor.

Check before you run the crew

Before assembling the full Crew, confirm the key and model work on their own:

curl https://api.zukko.pro/v1/chat/completions \
  -H "Authorization: Bearer fetch_your_key" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-5.6-sol","messages":[{"role":"user","content":"ping"}]}'

If that answers, whatever is left to debug lives in your agent logic, not the connection. Balance and per-call spend history live in the dashboard (GET /v1/balance, GET /v1/usage), and a one-time $0.50 test credit comes from subscribing to @zukkopro↗. If you deploy CrewAI for clients and point new users at the same gateway, the referral program pays 10% of what they top up — for an agency running several crews across different projects, that adds up to a meaningful discount on a shared tool rather than a one-off perk.