Rate Limiting & Throttling
API gateway mechanism controlling client ingestion limits to prevent DDoS attacks, abuse, and database resource starvation.
1. System Requirements
- Limit client API calls per time window (e.g., 100 requests per minute).
- Differentiate rate limits by API key, IP address, or subscription tier.
- Return RFC-standard headers (X-RateLimit-Limit, X-RateLimit-Remaining).
- Sub-millisecond latency lookup (< 2ms) to avoid bottlenecking requests.
- Distributed accuracy: Rate limiting metrics must be synchronized across multiple API gateway nodes.
2. Scale & Resource Estimates
Throughput: Can handle hundreds of thousands of key increment lookups per second using Redis caching.
Storage & RAM: Minimal: Redis key structures storing `API_KEY:timestamp` (8 bytes per client key).
3. System API Endpoints
// Header Response Signature
HTTP/1.1 200 OK
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 98
X-RateLimit-Reset: 17837089004. Database Schema Design
Rate Limit Counter (Redis Hash):
Key: "rate:user-4412:min"
Value: { "count": 2, "expire": 58 }5. Interactive High-Level Architecture
6. Scalability Deep Dive & Bottlenecks
Rate limiting is commonly implemented using Token Bucket or Leaky Bucket algorithms. In Token Bucket, tokens are added to a bucket at a constant rate. When a request arrives, we try to draw a token; if none are available, the request is blocked. In sliding window log algorithms, we store timestamp sets in a Redis Sorted Set (ZSET) per user and remove older records to maintain an exact running rate calculation.
7. Production Scaling Strategies
Redis Lua Scripts: Combining read, write, and check operations in a single atomic script to avoid race conditions.
Local Memory Buffers: Gateway nodes buffer count updates locally for 1 second before flushing to global Redis to decrease load.
Fallback to Local: If global Redis cache fails, gateway falls back to local memory rate limiting to ensure system availability.
8. Runnable Production Starter Code
Scaffold