Migrating a Legacy Rules Engine to an LLM-Based System
The rules engine in question decided eligibility for a client's insurance-adjacent product, built up over 15 years into roughly 4,000 nested conditional branches across multiple files, maintained by a rotating cast of developers, several of whom had left the company. Nobody could confidently say what the full rule set actually did anymore, and every change risked breaking an edge case buried three layers deep.
Our first instinct was to have an LLM read the rule set and reimplement its logic directly. That approach was risky in a way that mattered: an LLM confidently misreading one branch and silently dropping an eligibility condition could deny or approve coverage incorrectly at scale, with no obvious signal that anything had gone wrong. We couldn't ship a black-box replacement for logic this consequential.
Instead we used the LLM as an extraction and documentation tool first, not a replacement engine. It walked the codebase and produced a structured, human-readable description of every branch and condition, which the client's domain experts then reviewed and corrected — catching several rules that were themselves outdated or contradictory, artifacts of old policy changes nobody had cleaned up. That review process alone surfaced value independent of any migration.
Only after the rule set was fully documented and validated by humans did we rebuild it — as ordinary, deterministic code, not an LLM at runtime. We used the LLM during development to help translate the documented rules into test cases and to flag inconsistencies between the old implementation and the new one, but the production eligibility decision remained fully deterministic, auditable code with 100% test coverage against the documented rules.
The AI's actual role ended up being migration tooling, not the runtime system — which wasn't what the client originally asked for, but was the right call given the stakes. We now use this same pattern (LLM-assisted extraction and documentation, human validation, deterministic reimplementation) whenever a client wants to retire aging business logic they no longer fully understand.
