Summary

Model output is untrusted input. Define it as a Pydantic model, generate the schema you send from the same class that validates what comes back, and make fields optional when the answer may genuinely not exist, since a required field the model cannot fill produces invention rather than data. Close vocabularies with Literal and enums, bound numbers and list lengths, and remember that schema size costs tokens on every call and accuracy past a certain point.

Raw output arrives with fences, preambles, trailing commas, and truncation. Extract the JSON region conservatively rather than rewriting text with regular expressions, and detect truncation from the stop reason rather than from the parse failure, because truncation is not repairable by retrying. When validation fails, send the failed output and the specific validation error back to the model with a low attempt ceiling. Native structured output modes reduce how often that loop runs and do not remove the need to validate. Shape validation is not semantic validation, and a batch failure should salvage the records that succeeded rather than losing all of them.

Tool calling requires an explicit registry mapping names to callables, never dynamic lookup. Validate arguments before execution so handlers receive typed objects, and return failures the model could act on as messages rather than raising, since the model usually corrects itself. The agent loop needs three guards: an iteration limit, repeated-call detection, and no-progress detection, checked before each model call, plus a cost ceiling for anything unattended. Parallel tool calls must preserve their call identifiers, and every tool needs a timeout.

MCP standardises this interface so tools can be shared between applications. It is changing quickly, having recently moved to a stateless protocol with the SSE transport deprecated in favour of streamable HTTP and stdio. Adapt remote tools into your own registry so your validation, timeout, and approval policy still apply.

Security follows from one observation: the model holds your permissions and can be given instructions by anything in its context, including retrieved documents and tool results. You cannot reliably stop it being fooled, so the question is what it can do when it is. Scope permissions per tool, watch for the combination of a read capability with any outbound capability, enforce boundaries below your code with sandboxing, gate irreversible actions behind a person who sees the full action, and treat output filtering as signal rather than as a defence.

Key takeaways

• A required field the model cannot fill produces fabrication, not data
• Generate the schema you send and the validation you run from one definition
• Every coercion is a decision to accept something wrong-looking, so coerce format and never meaning
• Truncation is not a parsing problem and retrying will not fix it
• Send the model the exact validation error, not your summary of it
• A native structured output mode reduces repairs and does not replace validation
• Shape validation says the answer has the right form, not that it is true
• Never map a model-supplied name to a callable dynamically
• Return tool errors to the model when a different action could succeed
• Every agent loop needs an iteration ceiling, and stopping is not failing
• A model that is fooled acts with your permissions, so limit what it can do
• Read capability plus any outbound capability is an exfiltration path
• Output filtering catches naive attempts and nothing else

Common mistakes to remember

• Making every schema field required
• Repairing JSON with regular expressions that change meaning
• Treating truncation as malformed output and retrying identically
• Omitting an attempt ceiling on the repair loop
• Trusting a native structured output mode and skipping validation
• Losing ten records because two failed validation
• Using eval or globals() to dispatch a tool name
• Executing tool arguments before validating them
• Returning internal exception detail to the model
• Raising on a tool error the model could have corrected
• Running an agent loop with no iteration or cost limit
• Losing the call id when executing tools in parallel
• Giving one credential to every tool
• Checking a file path before resolving it
• Building a shell command by interpolating model output
• Asking for approval so often that nobody reads the prompt
• Relying on output filtering as the primary defence