Next.js Route Handler for AI: streaming responses in the App Router

In the App Router, server endpoints are written as Route Handlers — GET, POST and other functions in a route.ts file. If you connect an AI model with streaming to such a handler, the user sees the answer as it is generated, and the API key never leaves the server. Below is a working route.ts with the OpenAI SDK, a client chat component and a breakdown of the errors such routes usually trip over.

Installation and environment variables

npm install openai

The .env.local file in the project root:

The variable name has no NEXT_PUBLIC_ prefix, so Next.js won't inline it into the client bundle — only server code can read it. Don't commit .env.local. Create the key on the API keys↗ page.

A Next.js Route Handler with streaming

File app/api/chat/route.ts:

import OpenAI from "openai";

export const runtime = "nodejs";

const client = new OpenAI({
  apiKey: process.env.ZUKKO_API_KEY,
  baseURL: "https://api.zukko.pro/v1",
});

type ChatMessage = { role: "user" | "assistant"; content: string };

const ERRORS: Record<number, string> = {
  401: "Invalid API key on the server",
  402: "Insufficient balance",

What matters in this code

• Base URL is https://api.zukko.pro/v1, and the key is read from process.env.ZUKKO_API_KEY. The client is created once at module level.
• Bytes, not strings. A Response body must consist of Uint8Array chunks, so every piece of text is encoded with TextEncoder. If you enqueue a string, Node.js throws an error about a non-Uint8Array chunk. The streaming example in the Next.js documentation works the same way.
• Empty choices. The last chunk of the stream carries only usage and an empty choices array. The line if (chunk.choices.length === 0) continue; skips it — without it, accessing chunk.choices[0].delta fails at the very end of the answer.
• close() and error() are called on separate branches. Calling close() after error() throws by itself.
• Cancellation. If the client closes the tab or aborts fetch, the stream is cancelled and cancel() runs; it aborts the model request through AbortController.
• Errors before the stream starts (invalid key, no balance, model not allowed) are returned as regular JSON with the same status — the client checks response.ok. A 503 with the code model_disabled means the model is turned off: pick an available one from GET https://api.zukko.pro/v1/models.
• messages.slice(-20) caps the history: each request is billed together with the whole history you send.
• content-type: text/plain. We return raw text, not SSE events, so text/event-stream would be wrong here.

Client component

File app/chat.tsx:

"use client";

import { useState } from "react";

type Message = { role: "user" | "assistant"; content: string };

export default function Chat() {
  const [messages, setMessages] = useState<Message[]>([]);
  const [input, setInput] = useState("");
  const [loading, setLoading] = useState(false);

  async function send(e: React.FormEvent) {
    e.preventDefault();
    const text = input.trim();

And render the component in app/page.tsx:

import Chat from "./chat";

export default function Page() {
  return <Chat />;
}

The component immediately adds an empty assistant message and appends every incoming piece to it. State is updated without mutation — via map with a new object for the last message; otherwise, in React strict mode, text can get duplicated.

Models and parameters

Switching models is a one-line change: claude-opus-5-5, gpt-5.6-sol, glm-5.3-flash and others. The model list changes; the current one is at GET https://api.zukko.pro/v1/models or on the "Models" page in your dashboard. Test which model is best for your product on your own conversations.

Be careful with temperature and top_p: newer Claude models (claude-sonnet-5, claude-opus-5-5) may reject a temperature other than 1 and a top_p below 0.99 with a 400 error. Better not to pass these parameters to them. For GPT and GLM you can set them as usual:

const stream = await client.chat.completions.create({
  model: "gpt-5.6-sol",
  messages,
  stream: true,
  temperature: 0.7,
  max_tokens: 2048,
});

You can estimate spend like this: with 2,000 input and 500 output tokens on claude-sonnet-5, a request costs (2,000 × 0.3 + 500 × 0.3) / 1,000,000 = $0.00075. See the full picture per key in the "Usage" section of the dashboard, and per-key limits in the limits documentation↗. The request format is described in the chat completions↗ reference.

FAQ

Do I need the Anthropic SDK for Claude models?

No. All models — Claude, GPT, GLM — are available through the OpenAI-compatible format; the gateway converts the request. If you do need the Anthropic SDK, its base URL is https://api.zukko.pro without /v1; see the article on choosing a format↗.

Can I use the Vercel AI SDK?

Yes, the Next.js documentation mentions it as a ready-made streaming wrapper. Connect it through an OpenAI-compatible provider with the base URL https://api.zukko.pro/v1, and check the exact option names in the AI SDK documentation. The code above shows that streaming takes only a few dozen lines without a library.

Why does the answer arrive all at once instead of streaming?

Make sure the request has stream: true and that there's no buffering proxy between client and server. When self-hosting behind nginx, disable buffering for this route.

Can the key leak to the browser?

No, as long as the variable doesn't start with NEXT_PUBLIC_ and is used only in route.ts. The browser sees only the request to /api/chat. To be safe, use a separate key with a daily limit for the site.