Streaming LLM responses in Node.js Express with the OpenAI SDK

A long model response takes tens of seconds to generate, and showing an empty screen all that time is a bad idea. Streaming in Express solves this: the server receives a token stream from the model through the OpenAI SDK and immediately forwards it to the browser as Server-Sent Events (SSE). Below is a ready server.js, a plain JavaScript page, and a breakdown of where such endpoints usually break: empty choices, client disconnects and API errors.

Project setup

mkdir llm-stream && cd llm-stream
npm init -y
npm pkg set type=module
npm install express openai dotenv express-rate-limit

type=module is needed because the code below uses import. Keep the key in .env next to the server and don't commit it:

Create the key on the API keys↗ page. For a server application it's convenient to have a separate key with a daily limit, so a bug or abuse can't drain the balance.

Express streaming response: server.js

import 'dotenv/config';
import express from 'express';
import rateLimit from 'express-rate-limit';
import OpenAI from 'openai';

const app = express();
app.use(express.json({ limit: '100kb' }));
app.use(express.static('public'));

const client = new OpenAI({
  apiKey: process.env.ZUKKO_API_KEY,
  baseURL: 'https://api.zukko.pro/v1',
});

What matters in this code

• Base URL is https://api.zukko.pro/v1: the OpenAI SDK appends /chat/completions to it.
• Empty choices. The last chunk of the stream carries only usage, and its choices array is empty. Accessing chunk.choices[0].delta without a check throws a TypeError at the very end of the response. So check the array length first and handle usage separately, outside that check.
• Client disconnect is tracked with res.on('close'), not req.on('close'): in modern Node.js versions the request's close event fires as soon as the body has been read, i.e. right after express.json(). When the response closes, AbortController cancels the model request — generation that nobody reads doesn't continue.
• The X-Accel-Buffering: no header disables buffering in nginx if the server sits behind it. Without it the stream may reach the browser in one piece.
• Errors arrive as OpenAI.APIError with a status field. Zukko codes: 401 — invalid key, 402 — insufficient balance, 403 — model not allowed for the key, 413 — request too large, 429 — too many requests, 503 — provider unavailable. Show the user a clear message and log the details.
• Rate limiting with express-rate-limit and a message length check protect your balance from abuse. A separate cors() isn't needed: the page is served from the same server. Open CORS would let any website spend your balance through this endpoint.

Frontend: reading the stream with fetch

EventSource supports only GET requests, and we need to send text in a POST body. So we read the stream with fetch and ReadableStream. File public/index.html:

<!DOCTYPE html>
<html lang="en">
<head><meta charset="UTF-8"><title>Streaming response</title></head>
<body>
  <textarea id="prompt" rows="4" cols="80"></textarea><br>
  <button id="send">Send</button>
  <button id="stop" disabled>Stop</button>
  <pre id="out" style="white-space: pre-wrap"></pre>
  <script>
    const out = document.getElementById('out');
    const sendBtn = document.getElementById('send');
    const stopBtn = document.getElementById('stop');
    let controller;

SSE events are separated by a blank line, so the buffer is split on \n\n and the unfinished tail waits for the next piece. The Stop button calls abort(): the browser closes the connection, res.on('close') fires on the server, and the model request is cancelled too.

Run it with node server.js, then open http://localhost:3000.

Other models and conversation history

Switching models is a one-line change: gpt-5.6-sol, claude-opus-5-5, glm-5.3-flash and others from the price table. The model list changes; the current one is at GET https://api.zukko.pro/v1/models or on the "Models" page in your dashboard. Test which model suits your product on your own requests.

To make the bot remember the conversation, accept a messages array (roles user and assistant) from the frontend and pass it to create in full. Cap the history length on the server: every new request is billed together with the whole history you send.

Spend is visible in the usage of the last chunk. For example, with 1,500 input and 600 output tokens on claude-sonnet-5, a request costs (1,500 × 0.3 + 600 × 0.3) / 1,000,000 = $0.00063. The request and response format is described in the chat completions↗ reference, and basic setup in the quick start↗.

FAQ

Why does the response arrive in one piece instead of streaming?

Most often a proxy (nginx, a load balancer) buffers the stream, or response compression is enabled. Check the X-Accel-Buffering: no header, disable compression for this route, and make sure the model request has stream: true.

Where does "Cannot read properties of undefined (reading 'delta')" come from?

It's accessing chunk.choices[0] in the last chunk, where choices is empty. Add an array length check as in the code above.

Can I call Zukko directly from the browser without Express?

Technically yes, but then the key ends up in the page code and any visitor can take it. The key must stay on the server.

Why does generation continue after the tab is closed?

Most likely you listen to req.on('close') and don't pass a signal to the SDK. Use res.on('close') and AbortController as in the example.