LangChain with_structured_output: a typed model response

LangChain with_structured_output solves a problem everyone faces who parses model replies with regexes: instead of text you get a typed object, and instead of json.loads and hope you get schema validation. In this article we show how to define a schema with Pydantic, which method to pick when working through an OpenAI-compatible gateway, what to do on parsing errors and how to test several models on your own data. The code follows the LangChain documentation (the Models and Structured output sections); check package versions against it.

LangChain with_structured_output: a basic example with Pydantic

The schema is an ordinary Pydantic class. The LangChain documentation notes that Pydantic offers the richest feature set: field validation, descriptions, nested structures. Field descriptions (description) go into the model request, so write them clearly.

pip install langchain langchain-openai pydantic
export ZUKKO_API_KEY="fetch_your_key"

import os
from typing import Literal

from pydantic import BaseModel, Field
from langchain_openai import ChatOpenAI


class Ticket(BaseModel):
    """Analysis of a customer ticket."""

    category: Literal["payment", "access", "error", "other"] = Field(
        description="Ticket category"
    )
    urgency: int = Field(ge=1, le=5, description="Urgency from 1 (low) to 5 (high)")

The result is a Ticket instance, not a dict. For ChatOpenAI the gateway address must end with /v1. If you do not have Pydantic, with_structured_output also accepts TypedDict and JSON Schema, but according to the documentation you would then write value checks yourself.

Three methods: json_schema, function_calling, json_mode

The method parameter decides how LangChain asks the model to return a structure. According to the documentation:

• json_schema uses the provider's native structured output features;
• function_calling gets the structure through a forced tool call that follows the schema;
• json_mode generates valid JSON, but the schema must be described in the prompt.

Which method is the default for ChatOpenAI, check in the integration documentation for your version. There is a nuance here that is a reason to choose the method explicitly. To LangChain, ChatOpenAI means "OpenAI", so it may use OpenAI's native mode. But behind the gateway stand different models (Claude, GPT, DeepSeek, GLM and others), and we do not guarantee that each of them accepts that mode equally. The most portable option is function_calling: it relies on tool calling, which many models in the catalog support.

structured = llm.with_structured_output(Ticket, method="function_calling")

If a model does not support the required mode, you get a 400 error from the provider or None instead of an object. So do not guess, test on your data (a script is below). A comparison of model behavior in function calls is in the article function calling in Claude, GPT and Qwen↗.

What to do on parsing errors

A model can return a value outside the Literal, urgency 7 instead of the 1 to 5 range, or not call the required tool at all. To avoid an exception in the middle of a pipeline, enable include_raw=True. The method then returns a dict with three keys: raw (the model message), parsed (the object or None) and parsing_error.

structured = llm.with_structured_output(Ticket, method="function_calling", include_raw=True)


def classify(text: str, retries: int = 2) -> Ticket:
    last_error = None
    for _ in range(retries + 1):
        out = structured.invoke(text)
        if out["parsed"] is not None:
            return out["parsed"]
        last_error = out["parsing_error"]
    raise ValueError(f"could not get a structure: {last_error}")


print(classify("I cannot log in to my account, it says wrong password"))

A retry is billed like a normal request, so keep the number of attempts small. If errors come in a row, the causes are usually these:

• A truncated reply. With too small a max_tokens the JSON is never closed. Raise the output token limit.
• A schema that is too strict. A Literal with ten rare values or several levels of nesting. Simplify it or split into two steps.
• Vague field descriptions. The model relies on description, and a fuzzy one yields random values.
• The model or method does not fit. Change the method or the model.

A structured response inside an agent

If you have an agent on create_agent, you can get the structure at the end of the loop through response_format. According to the LangChain documentation you can pass a schema directly (the strategy is then chosen automatically) or wrap it in ToolStrategy. ToolStrategy has validation error handling on by default: the model receives the error message and tries again.

from langchain.agents import create_agent
from langchain.agents.structured_output import ToolStrategy

agent = create_agent(model=llm, tools=[], response_format=ToolStrategy(Ticket))
result = agent.invoke({"messages": [{"role": "user", "content": "Error 402 on a request, what should I do?"}]})
print(result["structured_response"])

When combined with tools, the model must be able to call tools and return a structure at the same time. Test that combination on your chosen model separately.

Without LangChain: the OpenAI SDK

The query "python openai structured output" often means the same result without LangChain. The openai SDK has a helper method parse that accepts a Pydantic class (the Ticket class from the first example is reused below):

import os
from openai import OpenAI

client = OpenAI(base_url="https://api.zukko.pro/v1", api_key=os.environ["ZUKKO_API_KEY"])

completion = client.chat.completions.parse(
    model="gpt-6.1-sol",
    messages=[{"role": "user", "content": "I paid yesterday, the balance did not update"}],
    response_format=Ticket,
)
print(completion.choices[0].message.parsed)

Support for this mode depends on the model and the provider behind the gateway: test on yours. A fallback for any model is to ask for JSON in the prompt and validate it by hand with Ticket.model_validate_json(text). That approach is covered in the article strictly structured JSON from a model↗.

Testing models on your data, and cost

Collect 20 to 30 real examples and run them through each model, counting the share of parsed is None:

models = ["gpt-6.1-sol", "claude-sonnet-5-5", "deepseek-v4.1-flash"]
samples = ["I paid, the balance did not update", "The email never arrives", "I want to change my e-mail"]

for name in models:
    model = ChatOpenAI(model=name, base_url="https://api.zukko.pro/v1", api_key=os.environ["ZUKKO_API_KEY"])
    chain = model.with_structured_output(Ticket, method="function_calling", include_raw=True)
    failed = sum(chain.invoke(s)["parsed"] is None for s in samples)
    print(name, "failures:", failed, "of", len(samples))

What it costs. Say a request with the schema definition takes 300 input tokens and the answer 60 output tokens. Formula: input × input price + output × output price, prices as of October 18, 2026, in millionths of a dollar:

The schema definition is added to every request, so long field descriptions increase input tokens. For Claude through the OpenAI format the whole input is billed at the input price. The list of models and prices: GET https://api.zukko.pro/v1/models.

FAQ

Which models support structured output?

We do not give a universal list: support depends on the model and the chosen method. The safest approach is to start with function_calling and test your 20 to 30 examples on several catalog models.

What is returned if the model could not fill the schema?

With include_raw=True you get parsed=None and the error description in parsing_error. Without this parameter an exception is possible.

Can I use nested structures?

Yes, Pydantic models can contain lists and other models, as the documentation shows. But the more complex the schema, the higher the risk of errors, so start with a flat one.

Does the schema affect the price?

Yes: the schema description is sent with the input of every request and billed as input tokens.

Create a key for tests in the dashboard↗ and put a daily limit on it. Python client reference: Python↗.