Ollama: what it is, what it's for and how it differs from a cloud API

Ollama is a program that runs large language models on your computer and gives you access to them through a terminal chat and a local API. Put simply, it is a "model manager and server": it downloads a model, loads it into memory and answers requests. Below we look at what Ollama is and what it is for, which models you can run, what hardware you need, and how local running differs from working through a cloud API.

Ollama: what it is, in plain words

A language model is a large file of weights. To get text out of it you need a program that loads the file into GPU or CPU memory and accepts requests. Ollama does that work. You enter one command, it downloads the model, starts it and opens an interface.

Key properties named in the project documentation:

• it runs on macOS, Windows and Linux, and there is an official Docker image ollama/ollama;
• it provides a REST API; by default the server listens on 127.0.0.1:11434;
• it supports OpenAI-compatible routes under /v1, so many ready-made clients connect without changes;
• it can run open models from its catalog and, more recently, cloud models, covered below;
• with local running your requests stay on your machine: the project FAQ says that when you run locally Ollama doesn't see your prompts and data.

Installation and first run

The project README gives an install-script command for macOS and Linux, a PowerShell equivalent for Windows, and a manual installer download:

curl -fsSL https://ollama.com/install.sh | sh

The script runs with your privileges, so read it before running or install the program manually from the download page on ollama.com. After installation, running a model looks like this:

ollama run gemma4

The name gemma4 is taken from the README; the model catalog updates constantly, so check current names at ollama.com/library. On first run the model is downloaded (that can be several gigabytes), then a chat opens. ollama ps shows which models are loaded and where they run: the Processor column reports whether a model is entirely on GPU, entirely on CPU or split between them.

Which models you can run

Ollama works with open models: those whose weights are published. The list is large and changes, so we don't name specific models here: see the catalog. Each model's page lists variant sizes (parameter counts), file size and capabilities, such as image input or tool calling.

Closed models such as Claude or GPT from Anthropic and OpenAI do not run locally: their weights are unavailable. You reach them through an API, for example via the Zukko gateway↗.

What hardware you need

The main constraint is memory. A model is loaded whole, and its weight plus memory for context must fit in video memory (the fast option) or in system RAM (slower). The file size is on the model's page. We can't give exact numbers for your machine: they depend on the model and its quantization.

Useful points from the Ollama documentation:

• NVIDIA GPUs are supported (minimum compute capability 5.0 and a suitable driver), along with other families; see the Hardware support section;
• if a model doesn't fit in video memory, some layers go to the CPU and speed drops;
• Docker on macOS gets no GPU access, so install the app directly there;
• the default context length depends on VRAM: under 24 GiB it is 4k tokens, from 24 to 48 GiB it is 32k, from 48 GiB it is 256k. For agents and tools at least 64,000 tokens is recommended, and a larger window needs more memory.

How to call a model via API

Ollama's native API:

curl http://localhost:11434/api/chat -d '{
  "model": "gemma4",
  "messages": [{"role": "user", "content": "Why is the sky blue?"}],
  "stream": false
}'

For Python there is the ollama package, and for any code written for the openai SDK it is enough to change the address:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:11434/v1/", api_key="ollama")  # the key is not checked
r = client.chat.completions.create(
    model="gemma4",
    messages=[{"role": "user", "content": "Explain what a token is"}],
)
print(r.choices[0].message.content)

The same trick lets you later point the application at a cloud gateway by changing only the address, key and model name. The choice between running locally and using an API has its own note: local model or API↗.

What Ollama is for in practice

• Local experiments and development. You can write LLM code with no token costs and no internet.
• Sensitive data. Requests don't leave the machine.
• Learning and getting to know models. It is easy to try different models and sizes.
• Automation on your own server. Scripts, bots and integrations that get by with a small model.
• Tests. A local server works well as a stand-in for checking code that will later go to the cloud.

Local running or a cloud API

There is also a middle option: Ollama's own cloud mode, where models run on the project's servers. See the Ollama documentation for details; the "neural network via API" approach itself is explained in the note what a neural network API is↗.

A worked example

Suppose a developer makes 200 requests a day for 30 days, each with 1,000 input and 300 output tokens: 6,000 requests, 6M input and 1.8M output tokens a month. Zukko prices as of October 18, 2026, $ per 1M tokens:

• glm-5.3-flash (0.04 / 0.04): 6 × 0.04 + 1.8 × 0.04 = $0.24 + $0.072 = $0.312;
• deepseek-v4.1-flash (0.013 / 0.038): 6 × 0.013 + 1.8 × 0.038 = $0.078 + $0.0684 = $0.1464;
• claude-sonnet-5-5 (0.4 / 2): 6 × 0.4 + 1.8 × 2 = $2.4 + $3.6 = $6.

These sums should be compared not with zero but with the cost of hardware able to run the model you need, and with your setup time. For occasional requests the cloud is often simpler; for constant load on a small model, local running can pay off. Current prices: GET https://api.zukko.pro/v1/models.

FAQ

Is Ollama free?

The program for local running is distributed openly. You pay for hardware and electricity. See Ollama's site for cloud mode terms and plans.

Can I run Claude or ChatGPT in Ollama?

No: they are closed models. Ollama runs open models from its catalog. Claude and GPT are reached through an API.

Do I need the internet?

Only to download a model. After that a local model works offline.

How do I use Ollama together with a gateway?

Write your code with the OpenAI SDK and take the address, key and model from settings: locally http://localhost:11434/v1/, in the cloud https://api.zukko.pro/v1. Access to the gateway from Russia: payment by SBP, MasterCard or crypto.

If you want to try cloud models next to local ones, create a key in the dashboard↗ and plug it into the same code by changing base_url.