Expert · 4–5 weeks
AI API Gateway with Cost Controls & Routing
Central gateway for all LLM calls: auth, per-team quotas, prompt versioning, model routing (cheap vs premium), PII scrubbing, and token usage analytics.
Why this project
Platform-level AI governance is staff engineer territory. Every company needs this as AI adoption grows.
Tech stack
Java 17Spring Cloud GatewayRedisPostgreSQLSpring AIOpenTelemetry
Skills demonstrated
- AI platform design
- Cost governance
- Multi-model routing
- Compliance
Interview talking points
- Why centralize LLM calls
- Model routing criteria
- PII scrubbing approach
- Handling provider outage with fallback
Problem statement
Central gateway for all LLM calls: auth, per-team quotas, prompt versioning, model routing (cheap vs premium), PII scrubbing, and token usage analytics.