LangChain Ollama or a cloud model: one codebase, two models

LangChain Ollama and cloud models in LangChain are two ways to get the same "chat model" object: ChatOllama runs a model on your computer, while ChatOpenAI with a different base_url calls a cloud one. The rest of the code (prompts, chains, agents) does not depend on the choice. Below we compare both options using the LangChain and Ollama documentation and assemble code where a local and a cloud model switch with one setting. Check package versions against the documentation: they change.

ChatOllama: a local model

The integration lives in the langchain-ollama package. According to the LangChain documentation the order is: install Ollama and start the local server, download a model with ollama pull, then create the object.

pip install -qU langchain-ollama
ollama pull gpt-oss:20b

from langchain_ollama import ChatOllama

llm = ChatOllama(model="gpt-oss:20b", temperature=0)

ai_msg = llm.invoke([
    ("system", "You translate English into French. Translate the user's sentence."),
    ("human", "I love programming."),
])
print(ai_msg.content)

The model name in ChatOllama(model=...) must match the tag you downloaded with ollama pull (ollama list shows the downloaded models). Among the ChatOllama features the documentation lists tool calling, structured output, image input (for multimodal models), token-level streaming and token usage.

ChatOpenAI with a gateway base_url: a cloud model

For the cloud we take ChatOpenAI from langchain-openai and set the gateway address. For OpenAI-format clients the Zukko base ends with /v1:

pip install -qU langchain-openai
export ZUKKO_API_KEY="fetch_your_key"

import os
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="deepseek-v4.1-flash",
    base_url="https://api.zukko.pro/v1",
    api_key=os.environ["ZUKKO_API_KEY"],
)

ai_msg = llm.invoke([
    ("system", "You translate English into French. Translate the user's sentence."),
    ("human", "I love programming."),
])
print(ai_msg.content)

Note that the invoke call, the message format and the response object are the same as with ChatOllama. This is the common chat model interface in LangChain. Connection is covered in more detail in LangChain: how to set our base_url↗, and Python client examples are collected in the reference↗.

One codebase, two models: switching with one setting

There are two ways to make the code independent of where the model runs.

Option 1: a factory over two classes

import os
from langchain_ollama import ChatOllama
from langchain_openai import ChatOpenAI


def make_llm(backend: str | None = None):
    backend = backend or os.getenv("LLM_BACKEND", "cloud")
    if backend == "local":
        return ChatOllama(model=os.getenv("LOCAL_MODEL", "gpt-oss:20b"), temperature=0)
    return ChatOpenAI(
        model=os.getenv("CLOUD_MODEL", "deepseek-v4.1-flash"),
        base_url="https://api.zukko.pro/v1",
        api_key=os.environ["ZUKKO_API_KEY"],
    )


llm = make_llm()

Running LLM_BACKEND=local python app.py uses Ollama; without the variable the cloud is used. Both branches return a chat model with the same invoke, stream, bind_tools and with_structured_output methods.

Option 2: one class and Ollama's OpenAI-compatible address

Ollama can serve an OpenAI-compatible API at http://localhost:11434/v1 (the OpenAI compatibility section of the Ollama docs). Then you need only ChatOpenAI, and three variables switch:

import os
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model=os.environ["LLM_MODEL"],
    base_url=os.environ["LLM_BASE_URL"],
    api_key=os.environ["LLM_API_KEY"],
)

The cloud profile:

export LLM_BASE_URL="https://api.zukko.pro/v1"
export LLM_API_KEY="fetch_your_key"
export LLM_MODEL="deepseek-v4.1-flash"

The local Ollama profile:

export LLM_BASE_URL="http://localhost:11434/v1"
export LLM_API_KEY="ollama"
export LLM_MODEL="gpt-oss:20b"

For the local address the key is needed only because the client requires a non-empty value and Ollama does not check it (if your setup differs, see the Ollama documentation). The benefit of this option is one class and one set of parameters. The price is that you only get what the Ollama compatibility layer exposes. Native Ollama parameters, such as the num_ctx context window size, are easier to set through ChatOllama.

Comparing the three options

A cloud model frees you from choosing hardware, but data leaves your machine. If requests contain personal data, weigh this in advance. A general discussion is in the article local model or API↗.

What the cloud option costs

Prices as of October 18, 2026 (per 1M tokens). Say a month sends 1M input and 0.2M output tokens. Formula: input × input price + output × output price.

For Claude through the OpenAI format the whole input is billed at the input price. A cache discount for Claude is possible only in the Anthropic format. A local model has no per-token fee but has hardware and electricity costs; calculate them separately and compare with the rows above for your volume. Model quality differs, so first run your own 20 to 30 examples on both options.

A fallback: local model plus a cloud backup

If the local server does not respond, you can switch to the cloud automatically. In LangChain every runnable object has a with_fallbacks method:

primary = ChatOllama(model="gpt-oss:20b", temperature=0)
backup = ChatOpenAI(
    model="deepseek-v4.1-flash",
    base_url="https://api.zukko.pro/v1",
    api_key=os.environ["ZUKKO_API_KEY"],
)
llm = primary.with_fallbacks([backup])

Remember that when the fallback fires, the request and its data go to the cloud. General principles of retries and a backup model are covered in the article LLM API error handling↗.

FAQ

Do I need Ollama to call cloud models?

No: for cloud models with Zukko, ChatOpenAI with base_url is enough. Ollama is needed only when the model runs locally.

Will a local and a cloud model give the same answers?

No, they are different models. The code will be the same, while the quality and style of answers must be checked on your own examples.

Which base_url do I use for the gateway?

https://api.zukko.pro/v1 for OpenAI-compatible clients, which ChatOpenAI is.

How do I find out which models are available in the cloud?

Request GET https://api.zukko.pro/v1/models or open the "Models" page in the dashboard.

To test the cloud option, create a key in the dashboard↗.