Evaluating AI Vendor Lock-In and Building Portable Abstractions
Midway through a client engagement, their primary LLM provider changed pricing in a way that made continuing on that model meaningfully more expensive at their volume. Because we'd built the integration behind a thin internal abstraction layer rather than calling the provider's SDK directly throughout the codebase, swapping the underlying model took about three days of testing and prompt adjustment instead of the multi-week rewrite it would have otherwise been.
The abstraction wasn't elaborate — a single internal interface with methods like `complete()` and `stream()` that every feature called, with provider-specific request formatting and response parsing isolated behind it. The cost was a small amount of upfront design work and slightly more boilerplate than calling a provider SDK directly. The payoff only showed up eighteen months later, which is exactly the kind of investment that's easy to skip under deadline pressure and easy to regret skipping later.
Portability wasn't free, though, and we're honest with clients about where it breaks down. Provider-specific features — a particular function-calling format, a caching mechanism tied to one API, a fine-tuned model that only exists on one platform — don't abstract cleanly, and forcing them into a generic interface either loses the benefit or adds complexity that isn't worth it for features we're confident we'll keep using long-term regardless of provider.
Our rule of thumb: abstract the request/response path for standard completion and chat calls, since that's genuinely portable across providers with minor prompt adjustments. Don't abstract provider-specific capabilities we're deliberately choosing to depend on — that's a real trade-off, not an oversight, and pretending otherwise just adds indirection without buying real portability.
We also keep a small evaluation set specifically for provider-swap testing — the same eval set used for prompt regression testing, run against a candidate replacement model before switching. When the pricing change happened, that eval set is what let us confirm the new model met the accuracy bar within a day, rather than discovering problems after the switch was already live.
