Designing an Internal API Gateway for a Growing Set of Services
The trigger for building an internal gateway wasn't scale, it was duplication — six services, each with its own JWT validation logic, its own rate limiting middleware, and its own request logging format, all subtly inconsistent because they'd been built by different engineers at different times. A bug in one service's auth middleware didn't get automatically fixed in the other five, and we'd already had one incident where a token validation edge case was fixed in one service and forgotten in another.
We put a gateway (Kong, in this case, though the specific product mattered less than the pattern) in front of all internal service traffic, centralizing authentication, rate limiting, and request/response logging at that single layer. Services behind the gateway trust a validated identity header rather than each independently verifying tokens, which removed an entire category of auth-related inconsistency across the fleet.
The risk we had to actively manage was the gateway becoming a single point of failure for every service at once, which is a much bigger blast radius than any one service going down individually. We ran the gateway in a highly available configuration across multiple availability zones from day one, and set conservative timeouts so a slow downstream service couldn't cascade into gateway-level saturation affecting unrelated services.
Routing and service discovery were the other real value beyond auth — as new services came online, adding a route to the gateway config was a five-minute change instead of every calling service needing to know a new internal hostname and update its own configuration. This mattered more than we initially expected once the service count passed ten; without centralized routing, every new service meant N separate configuration changes across every existing caller.
We deliberately kept business logic out of the gateway layer entirely — no request transformation beyond header injection, no orchestration across multiple backend calls. Gateways that accumulate business logic become a shared bottleneck that every team has to coordinate through for even minor changes, which defeats the purpose of having independently deployable services behind it in the first place.
