The model's answer stops mid-sentence — why

A truncated answer looks the same in every case, but there are four causes and they are fixed differently. One field tells them apart: finish_reason in the OpenAI-compatible format, stop_reason in the Anthropic-compatible one.

You hit max_tokens

Sign: finish_reason is length.

The model did not finish its thought — it ran out of allowed output length. The response is valid and fully billed.

The fix is obvious: raise max_tokens. Many SDKs default to a small value, and some clients set only a few hundred. Be generous for long answers, but remember the reserved space is subtracted from the context window.

A client-side timeout

Sign: the response always cuts off at roughly the same number of seconds, and you never see a finish_reason because no complete response arrived.

HTTP clients wait a bounded time by default — often 30 or 60 seconds. A long generation does not fit.

Two fixes. First, raise the client timeout to several minutes. Second, and better: enable streaming. With a stream, data arrives continuously and the connection never looks hung.

A broken stream

Sign: you used streaming, some text arrived, then the stream simply ended without a terminating event.

Usually a network drop, or a proxy closing a long-lived connection. Clients often report no error at all — they received valid data, just less than expected.

Check that the stream terminated properly: data: [DONE] in the OpenAI format, a message_stop event in the Anthropic one. Without it the answer is incomplete and should be treated as an error, not a result. Keep in mind when retrying: a dropped stream does not stop billing — the gateway reads the provider's response to the end and charges for the whole generation, so a retry is paid for again.

A buffering proxy

Sign: streaming is on, but the text arrives in one lump at the end — or not at all on long generations.

An intermediate or reverse proxy accumulates the response instead of passing it through. For nginx, set proxy_buffering off on that location. If a corporate proxy sits between you and the API, it may be the culprit.

What to check first

Look at finish_reason. If it says length, the question is settled — raise the limit. If the field is missing entirely, the response never arrived in full, and the investigation belongs to timeouts and streaming rather than request parameters.