Higress

Higress is an open-source, AI-native API gateway based on Istio and Envoy, developed under the Higress group and described as a CNCF Sandbox project. It unifies management of LLM API traffic, MCP (Model Context Protocol) servers and Kubernetes ingress, and can be extended with Wasm plugins written in Go, Rust or JS. The latest release listed on GitHub is v2.2.5.

What It Is

Higress is a gateway that sits between AI applications or agents and the model APIs, tools and services they call. It also works as a general cloud-native API gateway, providing an out-of-the-box console and dozens of ready-to-use plugins. Higress originated at Alibaba and is now developed as a vendor-neutral CNCF project, according to its README.

AI Gateway and MCP Hosting

The AI gateway connects to mainstream model providers through a unified protocol, with AI observability, multi-model load balancing, token rate limiting, caching and model-level fallback. The site describes semantic caching, token quota management and content-safety protection for model inputs and outputs. Higress also hosts MCP servers through its plugin mechanism, and the openapi-to-mcp tool can convert OpenAPI specifications into remote MCP servers. Hosted MCP servers get unified authentication, fine-grained rate limiting, audit logs for tool calls and observability.

Gateway Roles Beyond AI

Higress can act as a Kubernetes ingress controller compatible with many NGINX Ingress annotations, and it supports the Gateway API standard. It is listed as a Gateway API Inference Extension implementation for inference-aware routing. It can also serve as a microservice gateway that discovers services from registries such as Nacos, ZooKeeper, Consul and Eureka, and as a security gateway with WAF and authentication such as key-auth, JWT, OIDC and basic-auth.

Setup Path

A single Docker command starts an all-in-one instance exposing the console on port 8001 and HTTP/HTTPS gateway entries on ports 8080 and 8443. Kubernetes deployments use Helm, and regional image registry mirrors can be configured through the global.hub value. A hosted online demo is also available.