A neural-network chat on Gradio: a streaming UI in an hour

Need a web UI to talk to a model but do not want to write a frontend? Gradio for Python solves it with one class: gr.ChatInterface draws the chat window, history and a stop button. We will plug an OpenAI-compatible gateway into it, enable streaming, add a model picker and show the token count and price under every reply. The result is a chatbot based on GPT, Claude or DeepSeek that you can run within an hour.

What you need first

You need Python (check the Gradio docs for the supported version) and a fetch_... key from the dashboard↗. Install:

pip install --upgrade gradio openai
export ZUKKO_API_KEY="fetch_your_key"

Use a fresh Gradio version. The examples follow the gradio.app documentation: the chat function receives message and history, and history is a list of dicts with role and content keys.

The shortest option: gr.load_chat

The Gradio documentation has a one-liner for any OpenAI-compatible server (it shows Ollama as the example). With the Zukko gateway it looks like this:

import os
import gradio as gr

gr.load_chat(
    "https://api.zukko.pro/v1/",
    model="deepseek-v4.1-flash",
    token=os.environ["ZUKKO_API_KEY"],
).launch()

This is enough to check the key and the model. But you cannot configure a model picker, a system prompt or token accounting here, so next we write our own chat function. The same trick works with a local LM Studio server: put in the address from the program's settings (the LM Studio local server also exposes an OpenAI-compatible API) and you get LM Studio in Python with a ready UI.

Gradio in Python: a streaming chat

Streaming in Gradio is done with yield: each yield replaces the previous version of the reply. Parameters such as the model and the system prompt go through additional_inputs: Gradio renders them under the chat and passes them to the function after message and history.

import os
import gradio as gr
from openai import OpenAI

client = OpenAI(
    base_url="https://api.zukko.pro/v1",
    api_key=os.environ["ZUKKO_API_KEY"],
)

PRICES = {  # $ per 1M tokens (input, output), as of October 18, 2026
    "claude-sonnet-5-5": (0.4, 2.0),
    "gpt-6.1-sol": (0.27, 1.32),
    "deepseek-v4.1-flash": (0.013, 0.038),
    "glm-5.3-flash": (0.04, 0.04),

Run python app.py and open the address printed in the console. Let us go through the important parts.

Why the if chunk.choices check

In OpenAI-format streaming, Zukko always sends a final chunk with usage and an empty choices list. Without the check, chunk.choices[0] fails with IndexError. We reuse that same chunk to show tokens and price.

Why stream.close()

While a reply is running, the Submit button turns into a Stop button. If the user stops generation, Gradio interrupts the generator function and finally closes the connection. That way the connection to the gateway does not stay open after the stop, and the loop does not keep reading a reply nobody will see.

History and trimming

The model does not remember earlier turns, so the history is sent again with every request and the input grows each turn. The MAX_HISTORY constant caps it. We cut the statistics line out of the history using the marker so it is not sent back to the model.

What one reply costs

Formula: (input tokens × input price + output tokens × output price) / 1,000,000. Say a request has 2,000 input tokens including history and the reply is 500 tokens. Prices as of October 18, 2026:

The math column is in millionths of a dollar. When you use Claude through the OpenAI format (/v1/chat/completions), the whole input is billed at the input price and no cache discount applies. For a chat with short history this is not critical. You can always get the list of models and prices from GET /v1/models.

Choosing a model and quality

We do not claim which model in this list answers best. Reference points from the Zukko table: the Artificial Analysis strength index is 48 for claude-sonnet-5-5, 52 for gpt-6.1-sol, 42 for glm-5.3-flash and 39 for deepseek-v4.1-flash. The dropdown in the UI exists precisely so you can compare models on your own questions with the price next to them.

Security before you publish

• Do not hard-code the key: use an environment variable, as above.
• Create a separate key for the chat with a daily dollar limit and a list of allowed models (allowed_models). If strangers see the chat link, they can spend only that limit. More: key limits and cache↗.
• For private access, demo.launch() has an auth parameter taking a login/password pair, and it is better not to enable the public share=True link on a key with a large limit.

A similar chat on another framework is covered in the Streamlit↗ article.

FAQ

Does Gradio count tokens itself?

No. We take tokens from the last streaming chunk: usage.prompt_tokens and usage.completion_tokens. Spending is also visible in the "Usage" section of the dashboard.

Why is content sometimes a list in history?

In recent Gradio versions a message's content can be a list of parts (text, files). The as_text function joins the text parts and ignores the rest.

Can I add images?

Enable multimodal=True and pass the images to a model that accepts image input. Of the four models above, only claude-sonnet-5-5 and gpt-6.1-sol accept images in the Zukko table. Check the attachment format against the Gradio docs and the chat/completions reference↗.

Can I run the same thing with a local model?

Yes, just change base_url and the model to the address of a local server, for example Ollama or LM Studio. The chat code stays the same.

To run the chat yourself, create a key in the dashboard↗, set a daily limit on it and put it in the ZUKKO_API_KEY variable. More Python examples are in the reference↗.