Scalable reverse proxy for the whole cluster

Problem

A cluster with tens of services cannot expose a host port for each one. Teams copy TLS into every compose file, pick conflicting ports, and forget to renew certificates. HTTP stays open next to HTTPS. When a replica moves to another node, the URL still points at yesterday’s host.

Without a shared edge, every stack invents its own routing. Development and production disagree on paths. A new service means a firewall change, a certificate, and a hope that the old one still works.

The platform needed one door for the cluster: HTTP redirected or served as required, HTTPS terminated once, and traffic sent to the right service by name and path — at a scale that grows with the cluster, not with a spreadsheet of ports.

System Design

This project is a cluster-wide reverse proxy. It sits in front of the overlay network. Clients hit one or a few published ports. The proxy reads service labels or config, matches host and path, and forwards to healthy replicas.

TLS lives at the edge: certificates are issued and renewed there, not inside each app. HTTP either redirects to HTTPS or serves internal-only routes by policy. When a service scales or a node dies, the proxy follows the cluster membership and stops sending traffic to dead tasks.

Deploy is a stack like any other. Applications do not publish 80/443 themselves. They declare how they want to be reached. The proxy is the only public listener.

Architecture Decisions

• One ingress for the cluster instead of a port per service
• HTTPS terminated at the edge; apps speak HTTP on the overlay
• Host and path routing so many apps share a domain or a set of domains
• Load balancing across replicas; unhealthy tasks leave the pool
• Certificates issued and rotated at the proxy, not copied into every repo
• HTTP kept only where policy allows; otherwise redirect to HTTPS
• Config from service labels so a new stack is routed without editing the proxy by hand

Tradeoffs

• The proxy is a critical path: if the edge is down, the cluster is dark from outside
• Label-driven routing is convenient; a bad label can steal a host
• TLS at the edge means apps must not assume they terminate HTTPS themselves
• Shared ingress needs rate limits and timeouts or one noisy service hurts the rest
• Horizontal scale of the proxy needs shared state or DNS that can follow more than one replica

Production Behavior

Point DNS at the proxy. Deploy a service with a host or path rule. HTTP and HTTPS land on the edge; the matching replicas on the overlay receive the request. Scale the service and the proxy spreads connections. Replace a node and routing follows.

Watch certificates, backend health, and 502s after each deploy. Built for a cluster that should look like one platform from the outside, not a list of ports.