How Spotify Deployed Kong's AI Gateway to Power Generative AI at Scale
Spotify built an AI Gateway using Kong's AI Gateway to power generative AI at scale and unified AI adoption across the company. The AI Gateway provides a shared infrastructure for AI workloads, solving issues with authentication, rate limiting, cost tracking, and failure over. It enables teams to talk to large language models (LLMs) behind a single API, providing centralized observability and consistent error handling. The AI Gateway uses Kong's enterprise version, which is self-hosted and provides features like hybrid mode for deployment, declarative configuration for GitOps workflows, and a plugin ecosystem for extensibility. The plugin chain includes multi-provider routing, AI metrics, and end-to-end tracing for observability. As a result, over 1,000 internal services and individuals have been onboarded, and the AI Gateway has become the shared foundation for the company's AI-transformation strategy, enabling "supercharging product innovation" and "reimagining business operations" with AI.