Gateway API in Practice: The Nginx→Envoy Migration

The Retirement Announcement

November 2025: Kubernetes SIG Network and the Security Response Committee announced the retirement of Ingress NGINX. Best-effort maintenance would continue until March 2026. After that: no releases, no bugfixes, no security updates.

The reasons were clear: keeping ingress-nginx aligned across Kubernetes versions, NGINX releases, and Helm charts had become unsustainable. Recent high-severity vulnerabilities (including one allowing complete cluster takeover) exposed how heavy the maintenance burden had become.

We'd been running Nginx Ingress for years. It worked fine. But retirement meant migration was now urgent—continuing on unmaintained software wasn't acceptable for production.

Why Gateway API Made Migration Straightforward

Gateway API is Kubernetes' successor to Ingress. The key architectural improvement: separation of intent from implementation.

With Ingress (the old way): Routing configuration was tightly coupled to the ingress controller implementation. Switching from Nginx to Traefik or HAProxy meant rewriting configurations because each controller used different annotations.

Example:

# Nginx Ingress
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
spec:
  ingressClassName: nginx
  rules:
  - host: api.example.com
    http:
      paths:
      - path: /
        backend:
          service:
            name: api
            port: 8080

These annotations are Nginx-specific. Moving to different ingress controller requires rewriting.

With Gateway API (the new way): Routing intent is expressed in standard resources (HTTPRoute). Implementation details live in separate Gateway resources. Changing implementations doesn't require touching routing configuration.

Example:

# HTTPRoute (controller-agnostic)
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: api-route
spec:
  parentRefs:
  - name: main-gateway
  hostnames:
  - "api.example.com"
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /
    backendRefs:
    - name: api
      port: 8080
---
# Gateway (implementation-specific)
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: main-gateway
spec:
  gatewayClassName: envoy  # Could be cilium, istio, etc.
  listeners:
  - name: https
    protocol: HTTPS
    port: 443
    tls:
      mode: Terminate
      certificateRefs:
      - name: api-tls

The HTTPRoute is implementation-agnostic. It works with Envoy, Cilium, Istio, or any Gateway API-compliant controller. Only the Gateway resource knows which implementation you're using.

Our Migration Process

Week 1: Deploy Envoy Gateway alongside Nginx

Both controllers running. Nginx handling all traffic. Envoy Gateway deployed but not routing anything yet.

# Install Envoy Gateway
helm install envoy-gateway oci://docker.io/envoyproxy/gateway-helm \
  --version v1.2.8 \
  --namespace envoy-gateway-system \
  --create-namespace

# Verify both controllers running
kubectl get pods -n ingress-nginx
kubectl get pods -n envoy-gateway-system

Week 2: Create Gateway and HTTPRoutes

Convert Nginx Ingress resources to Gateway API:

# Gateway (infrastructure layer)
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: production-gateway
  namespace: infrastructure
spec:
  gatewayClassName: envoy
  listeners:
  - name: https
    protocol: HTTPS
    port: 443
    hostname: "*.clouds-of-europe.eu"
    tls:
      mode: Terminate
      certificateRefs:
      - name: wildcard-tls
        namespace: cert-manager
---
# ReferenceGrant (allows Gateway to reference cert in different namespace)
apiVersion: gateway.networking.k8s.io/v1beta1
kind: ReferenceGrant
metadata:
  name: allow-gateway-cert-access
  namespace: cert-manager
spec:
  from:
  - group: gateway.networking.k8s.io
    kind: Gateway
    namespace: infrastructure
  to:
  - group: ""
    kind: Secret

# HTTPRoute (application layer)
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: app-route
  namespace: app
spec:
  parentRefs:
  - name: production-gateway
    namespace: infrastructure
  hostnames:
  - "clouds-of-europe.eu"
  - "www.clouds-of-europe.eu"
  rules:
  - backendRefs:
    - name: app
      port: 3000

Week 3: Gradual traffic shift

Update DNS to point to Envoy Gateway load balancer. Monitor closely:

Error rates (should be unchanged)
Latency (should be comparable)
TLS handshake success
Certificate validation

If issues appear, DNS can revert to Nginx immediately.

Week 4: Monitor new system

Envoy handling 100% traffic. Nginx still deployed but receiving nothing. Watch for:

Any edge cases not handled by Gateway API
Performance characteristics
Resource usage of new controller

Week 5: Remove Nginx

After week of stable Envoy operation, remove Nginx Ingress completely:

helm uninstall ingress-nginx -n ingress-nginx
kubectl delete namespace ingress-nginx

Total migration: 5 weeks. Zero incidents. Zero downtime. That's the goal.

The Boring Change Principle

If changing a core component requires heroism, your architecture has a problem.

Good architecture makes change boring. Routine. Unremarkable. We don't celebrate impressive migrations—we celebrate migrations so smooth nobody notices.

Gateway API enabled this. By separating routing intent (HTTPRoute) from implementation (Gateway), we could swap the proxy layer without touching application configuration.

What We Learned

Gateway API works: The abstraction is sound. HTTPRoute is expressive enough for complex routing while remaining implementation-agnostic.

ReferenceGrants are essential: Without ReferenceGrants, you're forced to copy TLS certificates between namespaces. This creates synchronization problems and duplicates sensitive data. ReferenceGrants solve this cleanly.

Not all features are standardized yet: We needed some Envoy-specific features (BackendTrafficPolicy for circuit breaking). These required Envoy Gateway CRDs—not portable. But 90% of our routing is standard HTTPRoute.

Separation of concerns works: Platform team owns Gateways (infrastructure). Application teams own HTTPRoutes (routing). Neither needs elevated permissions for the other's layer.

Implementation matters: Gateway API is a standard. Implementations (Envoy, Cilium, Istio) have different performance characteristics, feature sets, and maturity levels. The standard enables choice; you still need to choose wisely.

Looking Forward

Gateway API is now GA (generally available) in Kubernetes. Ingress is effectively deprecated with Nginx's retirement.

This is the pattern for infrastructure evolution: new standard emerges, provides better abstractions, gradual migration from old to new.

Organizations building on Gateway API today are building on the future of Kubernetes networking. When the next generation of proxy technology emerges—and it will—Gateway API will enable migration without touching application routing.

The pattern: Own the interface, not the implementation. Standards provide interfaces. Products provide implementations. Build on standards.

Sources:

#gatewayapi #envoy #kubernetes #migration #standards