Local LLM or API: hardware, quality, cost, and privacy

Local LLM or API — a question developers face as soon as a model has a regular job: answering emails, processing documents, helping with code. Your own model seems free and private; an API seems more powerful and simpler. Below: an honest comparison by hardware, context, quality, cost, and privacy, an API cost calculation at October 7, 2026 prices, and code that works with both options.

How people run a local LLM

The most popular route is Ollama: it installs with one command on macOS, Linux, and Windows, downloads open models, and runs them on your computer. An example from the official README:

ollama run gemma4

Once running, Ollama starts a local server at http://localhost:11434. It has an OpenAI-compatible /v1 endpoint, so the same OpenAI SDK code works with both the local model and the API — only the address, key, and model name change:

from openai import OpenAI

local = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")  # required, but Ollama ignores it
cloud = OpenAI(base_url="https://api.zukko.pro/v1", api_key="fetch_your_key")


def ask(client: OpenAI, model: str, prompt: str) -> str:
    resp = client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": prompt}],
    )
    return resp.choices[0].message.content


question = "Explain in two sentences what HTTP 503 means"
print(ask(local, "gemma4", question))
print(ask(cloud, "deepseek-v4.1-flash", question))

One privacy caveat. Besides local models, Ollama has cloud models — they run on Ollama's servers, not on your machine. If you specifically need local processing, make sure you're using a local model. According to Ollama's docs, cloud features can be turned off with the disable_ollama_cloud setting in ~/.ollama/server.json.

Local LLM or API: comparison

Hardware

According to its docs, Ollama supports Nvidia GPUs (compute capability 5.0 or newer, driver 550 or newer), AMD Radeon, and on Macs, acceleration via Metal. The main constraint is memory: the model has to fit in VRAM, otherwise part of it spills over to the CPU and things slow down. You can check how the model was split with ollama ps — the PROCESSOR column should read 100% GPU.

Context

This is where local models surprise people most. According to Ollama's docs, the default context depends on VRAM:

For agents, web search, and coding tools, the same docs recommend at least 64,000 tokens. You can raise the context (for example, with the OLLAMA_CONTEXT_LENGTH variable), but it takes more memory. Models in the Zukko catalog have a 1M-token context; glm-5.3 and glm-5.3-flash have 128K.

Quality

We won't quote figures like "local model X is N% worse": the comparison depends on the specific model and your tasks. Two facts we can state with certainty:

• Only open-weight models run locally, and only those that fit in your memory. The Claude and GPT models from the catalog can't run locally: their weights aren't published.
• API models have a reference point — the intelligence index in the Zukko catalog (the Artificial Analysis Intelligence Index): Claude Opus 5.5 — 58, GPT-6.1 Sol — 52, Gemini 3.8 Flash — 41, DeepSeek V4.1 Flash — 39.

How to compare in practice: collect 30–100 real examples with reference answers and run them through the local model and 2–3 API models with the same code as above. The method is described in detail in the article on the intelligence index and mini-evals↗.

What the API costs

Take a load of 1,000 requests a day at 2,000 input and 500 output tokens. Prices as of October 7, 2026; daily formula: 1000 × (2000 × input + 500 × output) / 1,000,000, i.e. 2 × input + 0.5 × output; monthly is × 30.

Checking a row: for GPT-6.1 Sol, 2 × 0.27 + 0.5 × 1.32 = 0.54 + 0.66 = $1.20 a day, × 30 = $36. If your requests share a long common prefix (system prompt, instructions), part of the input may be billed at the cache-read price, and the total will be lower. For the current list and prices, call GET https://api.zukko.pro/v1/models or open the Models page in the dashboard.

How to calculate the local option

Hardware and electricity prices differ for everyone, so instead of our numbers, here's a formula for yours:

If you already have the hardware sitting idle, only electricity counts. If you'd need to buy it, compare against the table above: at volumes where a cheap API model costs a few dollars a month, buying a GPU to save money is unlikely to pay off. The picture changes with large, steady volumes on expensive models — so calculate your own scenario. A selection of inexpensive models for bulk tasks is in the article on the cheapest models↗, and how many requests the $0.50 bonus covers is in the article free AI model API↗.

Privacy

This is the strongest argument for a local model. According to Ollama's docs, when you run locally the service doesn't see your prompts or data: everything stays on the machine. With an API, the request goes through the gateway and on to the model provider.

If you work with personal data or trade secrets, first find out which data-handling requirements apply in your company. A common compromise is a hybrid: a local model anonymizes or classifies documents, and only the cleaned text goes to the API.

When to choose which

A local model, if:

• data must not leave the computer or the network;
• you need to work offline;
• the tasks are simple and repetitive, and a small model handles them — verified on your examples;
• you already have suitable hardware.

An API, if:

• you need strong models or a long context — agents, coding, large documents;
• the load is uneven, and you'd rather not pay for idle hardware;
• you don't have time for drivers, updates, and choosing models;
• a team works with the models and needs shared access with per-key limits.

A hybrid, if part of the data is sensitive and part of the tasks needs a strong model. The code at the start of this article lets you switch between the two clients by task type.

To try the API side of the comparison, create a key in the dashboard and run the same examples you used on the local model; setup is covered in the quick start↗.

FAQ

Can I connect Claude Code or Codex to a local model?

Ollama's README lists Claude Code and Codex among its supported integrations. Mind the context requirement: for coding tools Ollama's docs recommend at least 64,000 tokens, while on a GPU with under 24 GiB of VRAM the default context is 4k.

How much VRAM does a local LLM need?

It depends on the model and the context size. Go by the model size in the Ollama library and use ollama ps to check whether it fit entirely in VRAM.

What's cheaper at a small load?

At 1,000 requests a day with our profile, the cheapest model in the table costs $1.35 a month. If you'd have to buy hardware, the API is likely to be cheaper at such volumes — but check with the formula above using your own prices.

Can I combine both options?

Yes. Both Ollama and Zukko accept OpenAI-compatible requests, so all your code needs is two clients with different base_url values and a rule for which task goes where.