OpenAI API in Ollama: one code path for a local and a cloud model

Ollama can answer requests in the OpenAI format, and that is very convenient: an application written for the openai SDK can be pointed at a model running on your machine and then at a cloud gateway without rewriting the logic. Below we look at the address and port Ollama uses, what of the OpenAI API is supported and what isn't, and how to organize code so a local and a cloud model can be switched by a setting.

Ollama OpenAI API: address, port and key

According to the Ollama documentation, the server listens on 127.0.0.1 at port 11434 by default, and the OpenAI-compatible routes sit under the /v1 prefix. So the base_url for an OpenAI client is:

The client library requires a key, but Ollama ignores it; the documentation uses ollama as the value. To expose the server on a local network you change the bind address with the OLLAMA_HOST environment variable (for example 0.0.0.0:11434). That requires extra protection: in local mode Ollama itself doesn't check authorization.

The simplest curl request:

curl http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "qwen3:8b", "messages": [{"role": "user", "content": "Say hello"}]}'

The model must be downloaded first (ollama pull qwen3:8b or ollama run). Names like qwen3:8b and gpt-oss:20b come from the documentation examples; look for available models in the Ollama catalog.

What is supported

The documentation lists the endpoints /v1/chat/completions, /v1/responses, /v1/completions and /v1/models. For chat requests it marks:

• streaming, JSON mode, reproducible results (seed);
• vision: images are passed as base64, image URLs are not supported;
• tools (tools);
• reasoning control via reasoning_effort for models that have it;
• the parameters temperature, top_p, max_tokens, stop, frequency_penalty, presence_penalty, response_format.

Not supported, among others: tool_choice, logprobs, logit_bias, n and user. Ollama supports the Responses API only in its stateless form: previous_response_id and conversation do not work. This matters for portable code: anything relying on tool_choice or n has to be worked around.

One more nuance: the context window. In the Ollama documentation the default depends on VRAM: below 24 GiB it is 4k tokens, from 24 to 48 GiB it is 32k, from 48 GiB it is 256k, and for agent and tool tasks at least 64,000 tokens is recommended. Cloud models use the maximum length by default. If a local model "forgets" the start of a conversation, this is often the reason. You can change the length with the OLLAMA_CONTEXT_LENGTH variable or in the Ollama app.

A shared client for two modes

The idea is simple: the address, key and model come from settings, and the rest of the code doesn't know where it's calling. For the cloud option we take the Zukko gateway: the OpenAI-compatible address https://api.zukko.pro/v1 and a fetch_... key.

import os
from openai import OpenAI

PROFILES = {
    "local": {
        "base_url": "http://localhost:11434/v1/",
        "api_key": "ollama",
        "model": os.environ.get("LOCAL_MODEL", "qwen3:8b"),
    },
    "cloud": {
        "base_url": "https://api.zukko.pro/v1",
        "api_key": os.environ.get("ZUKKO_API_KEY", ""),
        "model": os.environ.get("CLOUD_MODEL", "deepseek-v4.1-flash"),
    },

To run on the local model: LLM_PROFILE=local python app.py; on the cloud: LLM_PROFILE=cloud python app.py. Nothing else needs to change. Cloud models change with the same CLOUD_MODEL variable: for example glm-5.3-flash, gpt-6.1-sol or claude-sonnet-5-5. The current list comes from GET https://api.zukko.pro/v1/models.

An automatic fallback

Often you want to "try local, and go to the cloud if Ollama is unavailable". Catching the connection error is enough:

import openai


def ask_with_fallback(prompt: str) -> str:
    try:
        return ask(prompt, "local")
    except openai.APIConnectionError:
        return ask(prompt, "cloud")

This scheme helps on a laptop during development and on a server with no local model. But remember that different models answer differently, and tests on the answer format should run in both modes.

Streaming and quirks

Streaming works with the same code in both modes. On the cloud gateway the stream ends with a chunk carrying usage and an empty choices list, so always check the array is non-empty before chunk.choices[0]:

client, model = make_client("cloud")
stream = client.chat.completions.create(
    model=model,
    messages=[{"role": "user", "content": "Explain TCP in three sentences"}],
    stream=True,
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)

This check also protects against other servers whose service chunks can be empty too. More on streams: streaming with stream=true↗.

Two practical portability rules:

1. Pass images as base64: URLs don't work locally.
2. Don't rely on tool_choice and n: local Ollama doesn't support them.

What about spending

Local mode is not billed by tokens: you pay in hardware and electricity. In cloud mode Zukko counts tokens at the catalog price. Suppose 50,000 requests a month, each with 800 input and 200 output tokens: 40M input and 10M output. Prices as of October 18, 2026, $ per 1M tokens:

• deepseek-v4.1-flash (0.013 / 0.038): 40 × 0.013 + 10 × 0.038 = $0.52 + $0.38 = $0.90;
• gpt-6.1-sol (0.27 / 1.32): 40 × 0.27 + 10 × 1.32 = $10.8 + $13.2 = $24.

A practical order: develop and test locally, then run production traffic through the gateway with a daily limit on the key. How to decide what to keep local is covered in the note local model or API↗, and the format's general principles are in the article on OpenAI-compatible APIs↗ and in the Chat Completions reference↗.

FAQ

What is the Ollama API URL?

For OpenAI clients http://localhost:11434/v1/, for Ollama's own API http://localhost:11434/api/.... The port is 11434, and it changes through OLLAMA_HOST.

Do I need a key for local Ollama?

The client library needs any non-empty value, such as ollama, but the server doesn't check it.

Why does the cloud base_url have /v1 while Claude Code's doesn't?

The OpenAI SDK and compatible clients don't add /v1 themselves, so it is part of the address. Claude Code and the official Anthropic SDK add it themselves, so for them the address is https://api.zukko.pro without /v1.

Does this work from Russia?

Payment from Russia: SBP, MasterCard or cryptocurrency. Local Ollama doesn't depend on payment.

Create a key in the dashboard↗, set ZUKKO_API_KEY and switch LLM_PROFILE between local and cloud.