Designing decisions and workflows that route uncertain, consequential, or otherwise unsuitable LLM outputs to a person for review.
Tagged articles
12 articles in this tag.
The moment you give an agent a tool, you stop deciding what it does and start deciding what it is allowed to do. The real design question is not "can the…
Read tutorial
A model returns clean JSON. The pipeline writes it to a table. One wrong field quietly poisons everything downstream.
Read tutorial
The most common mistake I see when teams first put an LLM into a real workflow is also the most understandable one: they bolt a human approval step onto…
Read tutorial
A review queue is a purchase order. You are buying a reduction in expected loss, and the only real question is whether the reduction is worth the price.
Read tutorial
Most developers don't fail with AI coding assistants because the code is wrong. They fail because the code looks right.
Read tutorial
An LLM that reads a document and returns JSON is not performing a database import. It is performing an act of reading—and reading, even by a very capable…
Read tutorial
The demo worked. The pilot looked fine. Then the invoice arrived, the review queue grew, and the savings quietly disappeared.
Read tutorial
Your judge reports a 92% pass rate. The dashboard is green. Then a reviewer reads ten of those passes and disagrees with four of them.
Read tutorial
The output is sitting in front of you. It reads well. It answers the question. And you still cannot say whether it should ship.
Read tutorial
You can recite that untrusted content can carry instructions. Then someone shows you a proposed action and asks, "Should this run?" and the recitation…
Read tutorial
A proposed tool call sits on your screen: send the summary to the client. You have a policy document open in another tab. Is this allowed, does it need…
Read tutorial
A citation chip proves a chunk was retrieved. It does not prove the chunk was ever authorized or trustworthy.
Read tutorial