Skip to main content
PreparationZone Logo
PrepZone

Load Balancers

Spread traffic across servers with L4/L7 balancers, health checks and sticky sessions.

Pass checkpoint below

Read these first

Why StreamHub needed a load balancer

At ~20K DAU, StreamHub's single API instance hit connection limits during peak evening viewing. Adding an Application Load Balancer (ALB) in front of four identical pods removed the bottleneck without code changes.

AWS ALB → EKS traffic path

TLSCLIENT
UsersHTTPS requests
NETWORK
Route 53alias → ALB
NETWORK
AWS ALBtarget group
COMPUTE
EKS pod 1streamhub-api
COMPUTE
EKS pod 2streamhub-api
COMPUTE
EKS pod NHPA scaled
L7 ALB terminates TLS, health-checks targets, and fans out to stateless pods.

StreamHub production architecture (AWS)

HTTPSstaticmissAPICLIENT
Mobile / WebStreamHub cli…
NETWORK
Route 53GeoDNS routing
NETWORK
CloudFrontCDN + WAF edge
NETWORK
AWS ALBTLS terminati…
NETWORK
API GatewayJWT · rate li…
STORAGE
Amazon S3media origin
COMPUTE
Amazon EKSAPI · auth · …
DATABASE
ElastiCachesessions · ho…
DATABASE
RDS Postgresprimary + rep…
INTEGRATION
Amazon MSKdomain events
ANALYTICS
OpenSearchstream discov…
OPS
CloudWatchmetrics · X-R…
End-to-end path from user to data — reference this when placing any new service.

Load balancer responsibilities

  • Distribute requests — round-robin, least connections, weighted, or hash-based.
  • Health checks — remove unhealthy backends automatically.
  • TLS termination — decrypt HTTPS at the edge, forward HTTP internally.
  • Sticky sessions — route same client to same backend when needed.
  • Path-based routing — /api/* to API pool, /admin/* to admin pool.

Layer 4 vs Layer 7

AspectL4 (Transport)L7 (Application)
Routes onIP + portURL path, headers, cookies
AwarenessTCP/UDP onlyHTTP headers, WebSocket upgrade
ExamplesAWS NLB, HAProxy TCP modeAWS ALB, NGINX, Envoy
StreamHubInternal DB proxy (rare)Public API entry point
  • Routes on

    L4 (Transport)IP + port
    L7 (Application)URL path, headers, cookies
  • Awareness

    L4 (Transport)TCP/UDP only
    L7 (Application)HTTP headers, WebSocket upgrade
  • Examples

    L4 (Transport)AWS NLB, HAProxy TCP mode
    L7 (Application)AWS ALB, NGINX, Envoy
  • StreamHub

    L4 (Transport)Internal DB proxy (rare)
    L7 (Application)Public API entry point

Most HTTP APIs use L7 balancers for path routing and header inspection.

The rest of this guide is part of Pro

Free covers every fresher guide in full. Pro unlocks Senior, Staff and Architect guides across every track.

Still to come · 5 sections

  • Load balancing algorithms
  • Health checks and graceful shutdown
  • Sticky sessions
  • TLS termination
  • Scaling the load balancer itself

Test yourself

Answer these before moving on — recall is what makes it stick.