Rust and async-openai: connect Claude through an OpenAI-compatible API

In Rust you don't have to wait for a dedicated SDK for every API: Zukko accepts OpenAI-format requests at https://api.zukko.pro/v1, so Claude connects through the async-openai crate or directly through reqwest. Below are both options on the current async-openai 0.42, streaming with correct handling of the last chunk, bounded parallel requests and error handling.

Dependencies

[package]
name = "zukko-rust"
version = "0.1.0"
edition = "2021"

[dependencies]
async-openai = { version = "0.42", features = ["chat-completion"] }
futures = "0.3"
reqwest = { version = "0.13", features = ["json"] }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
tokio = { version = "1", features = ["full"] }

The main trap in recent async-openai versions: APIs are split into feature flags, and only TLS is enabled by default. Without features = ["chat-completion"] the client has no chat() method, and code from older examples won't compile. Chat types now live in the async_openai::types::chat module. The list of flags is on the crate's docs.rs↗ page.

reqwest and serde are only needed for the first option; if you only use async-openai, you can drop them.

Create a fetch_... key in the Zukko dashboard↗ and put it in an environment variable: export ZUKKO_API_KEY=fetch_....

Option 1: reqwest and serde

Minimal abstraction: a plain POST and JSON parsed into your own structs.

use serde::Deserialize;
use serde_json::json;

#[derive(Deserialize)]
struct ChatResponse {
    choices: Vec<Choice>,
    usage: Option<Usage>,
}

#[derive(Deserialize)]
struct Choice {
    message: Message,
}

content is declared as Option<String>: in the OpenAI format the field can be null (for example, when the model returns a tool call), and a plain String would break deserialization.

Option 2: async-openai with Zukko

The API address is overridden with with_api_base on OpenAIConfig, the key with with_api_key. The request is built with CreateChatCompletionRequestArgs, as in the crate's official examples.

use async_openai::{
    config::OpenAIConfig,
    types::chat::{
        ChatCompletionRequestSystemMessage, ChatCompletionRequestUserMessage,
        CreateChatCompletionRequestArgs,
    },
    Client,
};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let config = OpenAIConfig::new()
        .with_api_base("https://api.zukko.pro/v1")
        .with_api_key(std::env::var("ZUKKO_API_KEY")?);

The model's answer is in choice.message.content, an Option<String>. Older articles parse the answer through ChatCompletionRequestMessage::Assistant, which is a request type, not a response type; such code doesn't compile.

Streaming the answer

create_stream returns a stream of Result<CreateChatCompletionStreamResponse, OpenAIError>. Zukko's last chunk contains only usage, with an empty choices array. So we take choices.first() via if let rather than indexing with [0], which would panic on that chunk.

use std::io::Write;

use async_openai::{
    config::OpenAIConfig,
    types::chat::{ChatCompletionRequestUserMessage, CreateChatCompletionRequestArgs},
    Client,
};
use futures::StreamExt;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let config = OpenAIConfig::new()
        .with_api_base("https://api.zukko.pro/v1")
        .with_api_key(std::env::var("ZUKKO_API_KEY")?);

flush() is needed because stdout is line-buffered, and without it the text would appear in bursts.

Parallel requests with a limit

Client is cheap to clone, so you can hand it to tasks. Don't fire a hundred requests at once, or you'll get 429. buffer_unordered(3) keeps at most three requests in flight:

use async_openai::{
    config::OpenAIConfig,
    error::OpenAIError,
    types::chat::{ChatCompletionRequestUserMessage, CreateChatCompletionRequestArgs},
    Client,
};
use futures::StreamExt;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let config = OpenAIConfig::new()
        .with_api_base("https://api.zukko.pro/v1")
        .with_api_key(std::env::var("ZUKKO_API_KEY")?);
    let client = Client::with_config(config);

Results arrive in completion order, not question order, which is why the question is returned together with the answer.

Error handling

API errors arrive as OpenAIError::ApiError(ApiErrorResponse): it contains status_code (the HTTP status) and api_error with a message field. Network problems are OpenAIError::Reqwest; a response in an unexpected format is OpenAIError::JSONDeserialize together with the raw text.

match client.chat().create(request).await {
    Ok(response) => {
        if let Some(choice) = response.choices.first() {
            println!("{}", choice.message.content.as_deref().unwrap_or(""));
        }
    }
    Err(OpenAIError::ApiError(e)) => match e.status_code.as_u16() {
        401 => eprintln!("Invalid key"),
        402 => eprintln!("Insufficient balance"),
        403 => eprintln!("Model not allowed for this key"),
        429 => eprintln!("Too many requests, retry later"),
        503 => eprintln!("Provider unavailable: {}", e.api_error.message),
        code => eprintln!("API error {code}: {}", e.api_error.message),
    },
    Err(e) => eprintln!("Error: {e}"),
}

Retrying with a pause makes sense for 429 and 503, but not for 401–403: a retry won't change anything.

Models and cost

The model is set by the string in .model(...). Zukko prices per 1M tokens (input/output): claude-sonnet-5 and claude-opus-5-5 $0.30 / $0.30, gpt-5.6-sol $0.30 / $0.30, gpt-5.6-luna $0.10 / $0.10, glm-5.3-flash $0.04 / $0.04. The current list is at GET https://api.zukko.pro/v1/models or on the "Models" page in the dashboard.

Example: a CLI tool makes 5,000 requests a month at 1,500 input and 500 output tokens each. On claude-sonnet-5: 5,000 × (1,500 + 500) × $0.30 / 1,000,000 = $3.00. On glm-5.3-flash: 5,000 × 2,000 × $0.04 / 1,000,000 = $0.40. Test which model handles your task on your own data: run the same set of requests through two models and compare the results.

FAQ

Error "no method named chat found"

The feature flag isn't enabled. Add features = ["chat-completion"] to the async-openai dependency. If you need all of the crate's APIs, there is a full flag, but it noticeably increases build time.

I get 404 Not Found

Check with_api_base: the address must end with /v1, as in https://api.zukko.pro/v1. The crate appends /chat/completions itself.

Can I pass parameters that the crate's types don't have?

Yes. With the byot flag, methods get _byot variants, for example client.chat().create_byot(json!({...})), which accept arbitrary JSON. This is described in the crate documentation under "Bring Your Own Types".

How do I keep a conversation with history?

Keep a Vec<ChatCompletionRequestMessage>, push user turns and model answers (ChatCompletionRequestAssistantMessage::from(...)) into it and pass the whole vector to .messages(...). The API remembers nothing between requests, and history is billed as input tokens, so trim old turns. More on the request format in the chat completions reference↗.