Human-in-the-Loop Review for AI-Generated Content Pipelines
A publishing client wanted AI-generated product descriptions for a catalog of around 15,000 items, but couldn't accept the risk of an AI hallucinating a product feature that didn't exist. Reviewing all 15,000 by hand wasn't realistic given their editorial team's size, and fully automated publishing wasn't acceptable given the accuracy stakes. We built a routing layer that decides which generated items need human eyes before publishing.
The routing score combines a few signals: the model's own token-level confidence on factual claims, whether the generated description references specific numeric specs that can be cross-checked against structured product data, and a similarity check against past items where human reviewers had previously made corrections. Items scoring low risk on all three publish automatically; anything else queues for review with the specific flagged claim highlighted, not just the whole description.
We tuned the auto-publish threshold conservatively at launch — around 40% of items auto-published in the first month — and loosened it gradually as we tracked post-publish correction rates on the automated batch. By month three, with no meaningful increase in error rate, auto-publish covered about 68% of new items, and the editorial team's review time shifted almost entirely to the genuinely uncertain cases instead of skimming everything uniformly.
One mistake we made early was routing based only on the model's confidence score, without the numeric cross-check. Confidence scores turned out to be poorly calibrated for factual correctness specifically — the model was often just as 'confident' about a fabricated spec as a correct one, since confidence reflected fluency more than accuracy. Adding the structured-data cross-check as a separate, independent signal caught a category of errors confidence alone missed entirely.
The review queue itself got instrumented too — we track which flagged claims reviewers actually correct versus approve as-is, and feed that back into the routing model periodically. Reviewer corrections are the highest-quality training signal we have, and using them to retrain the risk classifier every few weeks kept the auto-publish rate climbing without the error rate following it up.
