AI chat on Streamlit: streaming, history and settings

Streamlit lets you build an AI chat in plain Python in one evening: the built-in st.chat_message and st.chat_input elements draw the interface, st.session_state keeps the history, and st.write_stream prints the answer as it is generated. Below is a complete app.py with the OpenAI SDK, model selection, error handling and cost tracking, plus a breakdown of where such prototypes usually break.

Installation and secrets

pip install streamlit openai

Create .streamlit/secrets.toml in the project root:

ZUKKO_API_KEY = "fetch_your_key"

Streamlit reads it via st.secrets. Add the file to .gitignore (.streamlit/secrets.toml) and don't commit the key. When deploying to Streamlit Community Cloud, paste the same lines into the secrets section of the app settings. Create the key on the API keys↗ page; for a public demo, use a separate key with a daily limit and a list of allowed models.

AI chat on Streamlit: app.py

import streamlit as st
from openai import OpenAI, APIStatusError

PRICES = {
    "claude-sonnet-5": 0.3,
    "gpt-5.6-sol": 0.3,
    "glm-5.3-flash": 0.04,
}
MAX_HISTORY = 20
ERRORS = {
    401: "Invalid API key",
    402: "Insufficient balance",
    403: "Model not allowed for this key",
    429: "Too many requests, try again later",

Run it with streamlit run app.py; the app opens at http://localhost:8501.

How it works

The script reruns on every action. Streamlit executes app.py top to bottom after every input or button click. That's why history lives in st.session_state — a dictionary that survives reruns within one browser tab. The for msg in ... loop redraws all previous messages, and the new answer is appended at the end.

The client is created once. @st.cache_resource caches the function result globally — for all sessions and users of the app, not per user. For an OpenAI client that's exactly right: connections are reused. Never store per-user data in cache_resource — that's what session_state is for.

Streaming with st.write_stream. According to the Streamlit documentation, the function accepts a generator, an iterable or an OpenAI stream, renders text with a typewriter effect and returns the full response text. We pass our own generator rather than the raw SDK stream for two reasons:

• the last chunk of the stream carries only usage and an empty choices list; the if chunk.choices: check prevents an IndexError and lets us capture usage;
• from the generator it's easy to store spend in the usage dict and show the cost under the answer.

The manual approach with st.empty() and placeholder.markdown(text) works too, but st.write_stream is shorter and returns the final string for the history right away.

History is capped. Only the last MAX_HISTORY messages are sent. Each new request is billed together with the whole history you send, so without a cap a long conversation gets more expensive every turn.

Errors. If the API returns an error, we show a clear message and remove the last question from history, so the next attempt doesn't contain two user messages in a row. Zukko codes: 401 — invalid key, 402 — insufficient balance, 403 — model not allowed for the key, 429 — too many requests, 503 — provider unavailable.

Models and parameters

You can switch models in the sidebar. The model list changes; the current one is at GET https://api.zukko.pro/v1/models or on the "Models" page in your dashboard. Test which model answers best in your scenario using the same questions — that's what the sidebar switch is for.

The temperature slider is shown only for GPT and GLM models. Newer Claude models (claude-sonnet-5, claude-opus-5-5) may reject a temperature other than 1 and a top_p below 0.99 with a 400 error, so the parameter isn't passed to them.

What such a chat costs

The caption under each answer computes the cost from the usage in the last chunk. Example for claude-sonnet-5 ($0.3 per 1M input and output tokens): a turn with 3,000 input tokens (system prompt plus history) and 500 output tokens costs (3,000 + 500) × 0.3 / 1,000,000 = $0.00105. On glm-5.3-flash the same turn costs (3,000 + 500) × 0.04 / 1,000,000 = $0.00014. Total spend per key is visible in the "Usage" section of the dashboard, and key limits are described in the limits documentation↗.

Images in the chat

Claude and GPT models understand images; GLM 5.3 and GLM 5.3 Flash are text-only. To send an image, upload it with st.file_uploader and pass it in the user message as a data URL:

import base64

file = st.file_uploader("Image", type=["png", "jpg", "jpeg"])
if file and prompt:
    b64 = base64.b64encode(file.getvalue()).decode("utf-8")
    content = [
        {"type": "text", "text": prompt},
        {"type": "image_url", "image_url": {"url": f"data:{file.type};base64,{b64}"}},
    ]

This content replaces the string in the last user message. Keep in mind the image is resent with every subsequent request while it stays in the history. The request format is described in the chat completions↗ reference, and Python client setup in the Python guide↗.

FAQ

Why does history disappear after a page refresh?

st.session_state lives as long as the tab is open. To keep conversations between visits, write messages to a database or file and load them on start.

Can I pass an OpenAI stream to st.write_stream directly?

Yes, the Streamlit documentation explicitly supports OpenAI streams. A custom generator is useful when you want to handle empty choices yourself, capture usage or modify text on the way.

Why does user B see user A's answer?

Most likely history is stored in a global variable or in a function decorated with @st.cache_resource/@st.cache_data. User data must live only in st.session_state.

How do I hide the key in a public deployment?

The key is stored in the app's secrets and used only on the Streamlit server — it never reaches the browser. To protect the balance, give the key a daily limit and allow only the models you need.