Answering in a Fixed Form
Structured output means having an LLM produce its answer in a fixed form such as JSON (a data format that lists field names and values in a set way). When you put answers into a table, read them with a program, or pass them to another system, it is a problem if the form differs from one answer to the next. An example is extracting three fields from an inquiry: its type, whether it is urgent, and the product name.
Just asking for an answer in JSON in the instructions makes the form fairly consistent. According to Anthropic, however, that alone can still lead to syntax errors, missing required fields, and mismatched types.
How the Form Is Specified
Many APIs have a feature for specifying the form of the answer. You pass the field names, types (number or text), and lists of allowed values as a schema (a definition of the form). The model is then restricted to writing only answers that fit that form.
OpenAI says this feature makes answers always follow the supplied schema, preventing missing required fields and values outside the defined options. Anthropic and Google have features with the same purpose. Each service limits which ways of writing a schema can be used to specify the form.
Check the Content Even When the Form Is Right
Even if the form is kept, the content is not necessarily correct. Google's guide recommends always checking the values in your own program, even when the output is syntactically correct JSON. It also asks you to prepare for answers that fit the form but mean the wrong thing. Check as follows.
- Check that values fall within the defined range or list
- Compare extracted values against examples whose correct answers are known
- Send the request again when the form or values are off
Shorter field names and dropping unneeded fields also reduce output tokens.