Envoy AI Gateway is a self-hosted, open-source gateway that routes application traffic to 17 language model providers through one OpenAI-compatible API. Built on Envoy Gateway with Kubernetes custom resources such as AIGatewayRoute, plus a standalone aigw laptop binary, it adds provider fallback, token quotas, usage-based rate limits, model name virtualization, a built-in MCP gateway, and OpenTelemetry GenAI tracing. In September 2026 the project was renamed Agent Router under the Agentic AI Foundation; the code, APIs, and Helm charts kept their names.[1][2]
Quick verdict
Envoy AI Gateway, renamed Agent Router in September 2026, is a self-hosted LLM gateway that puts 17 providers behind one OpenAI-compatible API with fallback, token quotas, usage-based rate limits, MCP routing, and GenAI tracing. Kubernetes and Envoy Gateway shops plus agent platform teams get the most value, while teams without platform operations should weigh the cluster burden against hosted products built on the same core.[1][2][3]
Capabilities
One OpenAI-compatible API Presents a single OpenAI-style endpoint that fans out to 17 providers including OpenAI, Anthropic, AWS Bedrock, Azure OpenAI, Google Vertex AI and Gemini, and self-hosted vLLM, so clients integrate once instead of once per provider.[1][2]
Provider fallback Retries a failed request on another provider or model according to declared fallback rules, lifting effective availability without client-side retry logic.[1]
Token quotas Enforces per-consumer token budgets, for example 100 tokens per 60 seconds, so one team cannot exhaust shared provider spend.[1]
Usage-based rate limiting Limits request rates by token usage rather than raw request counts, matching cost control to how providers actually bill.[1]
Model name virtualization Lets operators rename and remap models centrally, so applications keep calling a stable model name while the backing provider or version changes underneath.[1]
Inference-aware routing Routes to self-hosted model endpoints with awareness of inference load, fitting Kubernetes InferencePool style deployments.[1]
Built-in MCP gateway Fronts agent tool servers through the same gateway, so Model Context Protocol tool calls get routing and policy alongside chat completions.[1]
GenAI observability Emits OpenTelemetry GenAI semantic traces recording which model answered, token counts, and where fallbacks fired, fitting standard OTel collectors.[1]
Who will love it
Multi-provider LLM routing Platform teams that want one self-hosted endpoint across many LLM providers with central fallback and spend control.[1][2]
Envoy and Kubernetes shops Operators already running Envoy Gateway or Gateway API resources who want AI traffic policy expressed as Kubernetes configuration.[1][2]
Agent and MCP governance Teams governing agent tool calls that need MCP traffic routed and observed through the same control plane as completions.[1]
Who should think twice
Kubernetes operations burden Production use needs a Kubernetes cluster with Envoy Gateway plus gateway configuration know-how, which is heavier than a hosted proxy for teams without platform operations.[1][2]
Rename in progress The September 2026 rename to Agent Router is still rolling through docs, CRDs, images, and charts, so teams should expect documentation and URL churn while identifiers catch up.[1]
No project-run managed tier There is no paid or free managed edition from the project itself; teams that want zero infrastructure must buy a separately priced hosted product from Tetrate or Paper Compute.[1][2]
How people experience it
One inspected independent September 2026 comparison of 9 LLM gateways, verified against vendor documentation and repositories, with a direct Agent Router entry.
1 cited community source. Single-report sample from a competing gateway vendor (VDF AI sells its own router); it supports an attributed evaluation, not prevalence or satisfaction scores. No Reddit or G2 end-user reports were found for this infrastructure gateway.
Foundation stewardship The project moved from the CNCF into the Agentic AI Foundation as Agent Router in September 2026 with the same code, maintainers, and APIs, signalling neutral multi-vendor governance.[1]
Named production adopters Bloomberg, Tencent Cloud, and Nutanix are named as users and contributors, and commercial solutions from Tetrate, Nutanix, and Paper Compute are built on the same core.[1][2]
Independent evaluation An independent comparison verified the gateway's documented features against vendor docs and judged it the most familiar option for teams already operating Envoy, with provider fallback, model name virtualization, and per-team token limits confirmed.[1]
Fit and compatibility
Provider API breadth Covers OpenAI, Anthropic, AWS Bedrock, Azure OpenAI, Google Vertex AI and Gemini, and self-hosted vLLM behind one interface, rather than replacing any single provider console.[1][2]
Telemetry compatibility Fits OpenTelemetry-based observability stacks through GenAI semantic conventions, and Kubernetes inference serving through inference-aware endpoint selection.[1]
Questions worth asking
What is a less technical alternative to Envoy AI Gateway?
If operating a Kubernetes gateway feels too heavy, use a hosted product built on the same core: Tetrate sells a hosted and an enterprise edition, and Paper Compute offers one-click GPU plus gateway hosting. Choose hosted when you want zero infrastructure; choose the open-source gateway when you need self-hosted control under Apache 2.0.[1][2]
Can I use Envoy AI Gateway without writing code?
Partly. Routes, backends, fallbacks, and rate limits are declared in configuration files and Kubernetes resources rather than application code, and the standalone laptop binary starts a local router without a cluster. You still need API and infrastructure concepts such as routes, providers, tokens, and Kubernetes, so it is low-code for operators rather than a general no-code builder.[1][2]
How much does Envoy AI Gateway cost per month?
The gateway software itself is free under Apache 2.0, so there is no per-month license fee for the open-source project. Your monthly cost is infrastructure for the Kubernetes cluster or laptop runtime plus operations time; hosted and enterprise editions from Tetrate and Paper Compute are separately priced commercial products.[1][2]
Can I self-host Envoy AI Gateway?
Yes. The project is designed for self-hosting: production setups run as a Kubernetes control plane on Envoy Gateway with Gateway API custom resources, and the standalone aigw command starts a local router on Linux or macOS for development. No managed free tier comes from the project itself; managed hosting comes from commercial vendors.[1][2]
Sources and editorial review
Source research. Reviewed on October 9, 2026. Written by Sanjib Sah. AI assisted; reviewed by a human editor.
We have not run or tested this product.
One inspected independent September 2026 comparison of 9 LLM gateways, verified against vendor documentation and repositories, with a direct Agent Router entry. Single-report sample from a competing gateway vendor (VDF AI sells its own router); it supports an attributed evaluation, not prevalence or satisfaction scores. No Reddit or G2 end-user reports were found for this infrastructure gateway.
- Envoy AI Gateway Documentation: Unified AI Gateway
official · Observed October 9, 2026
Official Envoy AI Gateway documentation home: built on Envoy Gateway, AIGatewayRoute CRDs, standalone aigw binary, token quotas, usage-based rate limiting, fallback, model virtualization, inference-aware routing, MCP gateway, OpenTelemetry GenAI tracing, deploy with uv or Docker.
- envoyproxy/ai-gateway: Connect Any Model in Minutes
official · Observed October 9, 2026
Official envoyproxy/ai-gateway repository README: unified cross-provider API, connect any model in minutes, Kubernetes CRDs and standalone mode, MCP gateway, GenAI tracing, adopters Bloomberg Tencent Cloud Nutanix, Apache 2.0 license with no paid edition, 2,182 stars.
- Tetrate and Bloomberg Unveil Open Source Envoy AI Gateway
official · Observed October 9, 2026
Official launch announcement: Tetrate and Bloomberg created Envoy AI Gateway as a CNCF project to unify fragmented LLM provider APIs, with cost and access controls, semantic caching goals, and adoption by Bloomberg, Tencent Cloud, and Nutanix.
- Agent Router: The Control Plane for Enterprise AI Traffic
official · Observed October 9, 2026
Official Agent Router site: Envoy AI Gateway renamed Agent Router under the Agentic AI Foundation on 10 September with same code maintainers and APIs; 1.0 stable control-plane API; laptop-to-cluster quickstart; commercial solutions from Tetrate hosted and enterprise, Nutanix, and Paper Compute.
- AI Gateway Comparison 2026: 9 LLM Gateways Compared
community · Observed October 9, 2026
Independent September 2026 comparison of 9 LLM gateways verified against vendor documentation and repositories, with a direct Agent Router (formerly Envoy AI Gateway) entry covering hosting, licence, MCP, routing, and limits. Publisher VDF AI sells its own gateway router, a competing product.
Compared with these products
No explicit proprietary baseline is recorded.