Structured Output Reliability: What Validation Can and Cannot Guarantee

A validator can tell you the shape of an answer. It cannot tell you whether the answer is true.

You already run a validate-reject-retry loop. The schema passes, the payload parses, the downstream action fires — and then someone notices the value was wrong. Nothing threw an exception. Nothing logged a warning. The system did exactly what you told it to do, and the result was still incorrect.

That gap is not a bug in your validator. It is the boundary of what structural validation can ever guarantee. This article builds a small formal model of that boundary so you can reason about it instead of hoping the next schema tweak closes it. If you have not yet built a validation loop, the mechanics of reject-repair-retry are assumed background here; the question now is what that loop actually buys you.

Why a Green Validator Is Not a Correct Answer

Start with the symptom. A payload arrives, every required field is present, every type matches, every enum value is legal, and the values are wrong. A risk score of 72 when the evidence supported 31. A "moderate" category when the case was high-risk. The JSON is impeccable. The answer is not.

Grant the narrow case first, because it is real: schema enforcement genuinely eliminates parse failures, missing fields, and type errors. Provider-side strict decoding and local validators both do this well. That work is worth having, and it is not nothing.

The limitation is structural. A required field cannot express "I don't know." When the schema demands a value and the model lacks the information to produce the right one, it produces a plausible wrong one. Confidence is no indicator of accuracy here — the model has no mechanism to signal uncertainty through a field that only accepts a number.

This is why I think of validation as a filter with a measurable pass-through rate, not a truth oracle. It decides what gets through. It says nothing about whether what got through is right. Both provider-side strict decoding and local validators operate on shape, not meaning.

Three Layers: Structural Validity, Correctness, Task Suitability

To reason about this precisely, separate three properties that beginners tend to collapse into one.

Structural validity. The payload parses and satisfies the schema. This is decidable, cheap, and local — you can check it without any external reference.

Correctness. The values match ground truth. This requires an external reference the validator does not have. The validator sees the payload; it does not see the world.

Task suitability. The correct values are the right ones for this decision. A risk score can be accurate and still be the wrong input for a threshold that triggers an irreversible action. Suitability is a property of the task, not the payload.

The containment relation is not guaranteed in either direction. Valid does not imply correct. Correct does not imply suitable. Each layer can fail while the others pass. Picture three overlapping regions: accepted-but-wrong outputs, correct-but-unsuitable outputs, and rejected-but-correct outputs. Your validator only sees the first boundary.

Semantic validation narrows the correctness gap — but only for rules you can actually write down. Cross-field business rules, range checks tied to source evidence, and consistency constraints catch some wrong-but-valid values. They cannot catch a value that is wrong in a way no rule anticipated.

A Toy Model You Can Calculate With

Now make it formal. Define three per-attempt quantities:

• $p_v$ — the probability an attempt is structurally valid
• $p_c$ — the probability an attempt is correct
• $p_{cv}$ — the probability an attempt is both valid and correct

Define the acceptance rule: the system accepts the first structurally valid attempt and stops.

Two assumptions make the arithmetic valid, and both are idealizations:

1. Independence. Attempts are treated as independent draws. This is not a fact about real models — it is a simplifying assumption.
2. Validator-only acceptance. Acceptance is decided by the validator alone, with no semantic or human check.

One more flag before we compute: $p_{cv}$ is not generally $p_v \times p_c$. Validity and correctness are correlated in practice. A model that produces well-formed output is often also more likely to be right, and a confused model often fails both at once. Treat $p_{cv}$ as its own quantity.

Deriving the Two Numbers That Matter

Two questions matter, and they have different answers.

Question 1: What is the chance of at least one accepted correct result across $n$ attempts?

Work from the complement. The probability that a single attempt is not an accepted correct result is $1 - p_{cv}$. Across $n$ independent attempts:

$P(\text{at least one accepted correct}) = 1 - (1 - p_{cv})^n$

Question 2: What is the correctness rate conditional on structural acceptance?

This is the number your users actually experience. By definition of conditional probability:

$P(\text{correct} \mid \text{accepted}) = \frac{p_{cv}}{p_v}$

Here is the consequence that matters. The conditional rate does not improve with more attempts. Retries change the chance of getting something accepted; they do not change the quality of what gets accepted. Raising $p_v$ alone — a stricter schema, provider-side enforcement — can raise the acceptance rate while leaving the conditional correctness rate flat or worse.

Retries buy coverage, not accuracy. The conditional rate is the ceiling your acceptance rule imposes.

Worked Example: Three Attempts, One Acceptance Rule

Plug in illustrative numbers. Suppose $p_v = 0.90$, $p_{cv} = 0.72$, and $n = 3$.

Coverage first:

$1 - (1 - 0.72)^3 = 1 - 0.28^3 = 1 - 0.021952 \approx 0.978$

So roughly a 97.8% chance of at least one accepted correct result across three attempts. That looks generous.

Now the conditional rate:

$\frac{0.72}{0.90} = 0.80$

Eighty percent. One in five accepted outputs is structurally valid and wrong. The retry budget looks comfortable on the coverage number and thin on the conditional number — and it is the conditional number that governs what your users see.

Recompute with a stricter schema. Suppose $p_v$ rises to $0.97$ but $p_{cv}$ falls to $0.70$ — tighter constraints suppress reasoning quality on harder tasks. Coverage barely moves. The conditional rate drops to $0.70 / 0.97 \approx 0.72$. The acceptance rate rose; accepted-output quality fell.

These numbers are illustrative inputs to a model, not measured benchmarks for any provider or model.

Where the Toy Model Breaks

The arithmetic is only as good as its assumptions. Both fail in practice.

Attempts are not independent. The same prompt, context, and model bias produce correlated failures. Retries repeat the same mistake rather than sampling around it. This is the single most important violation: it means the coverage formula overstates your real chance of escaping a systematic error.

Error feedback changes the distribution. A repair loop that feeds validation errors back to the model shifts the distribution between attempts. The fixed-$p$ model does not represent this.

Acceptance is often not validator-only. Refusals, token-limit truncation, and content-filter interruptions surface outside the schema path entirely. Your validator never sees them.

Strict decoding can fail at the provider boundary. Even the structural layer is not a guaranteed constant. Intermittent schema-validation failures on strict structured outputs are a real operational phenomenon, not a theoretical one.

Schema tightening has a cost. Overly strict constraints can impair reasoning on complex tasks, trading one failure mode for another.

Treat the model as a way to reason about limits, not as a predictor of your system's behavior.

What Validation Can and Cannot Guarantee

Consolidate the boundary into a decision rule.

What schema validation guarantees: parseable, typed, complete-by-construction payloads that downstream code can consume without defensive parsing.

What it cannot guarantee: that any value is true, that the chosen enum is the right one, or that the output fits the decision it triggers.

What retries guarantee: more chances to accept something. Not a higher quality bar for what is accepted.

What actually narrows the correctness gap: cross-field business rules, source-evidence checks, ground-truth comparison on a sample, and human review for high-stakes branches.

Decision rule: If a wrong-but-valid value would trigger an irreversible action, the schema is not your safety net — the review gate is.

The Monitoring Habit That Closes the Gap

Turn the derivation into a habit: track the conditional correctness rate on accepted outputs, not just the acceptance rate. The acceptance rate tells you the loop is working. The conditional rate tells you whether the loop is producing anything worth accepting.

Pick one high-stakes field where a valid-but-wrong value would cause real damage. Measure that field against ground truth on a sample. If the conditional rate is lower than you assumed, the schema is not the problem — the acceptance rule is.

The next useful step is designing the semantic and review checks that the schema cannot perform: the cross-field rules, the evidence comparisons, and the human gates that catch what structural validation was never built to see.