Design Slack / Teams
High-volume group communication hub with instant message fanouts, channels, and read markers.
1. System Requirements
- Real-time group chat communication inside channels.
- Sync thread reading offset markers across multiple devices.
- Search channel history logs.
- Sub-100ms message broadcast latency.
- High availability and durability for messaging histories.
2. Scale & Resource Estimates
Throughput: 10 Million DAU. Average messages sent/day = 100 Million = ~1,150 messages/sec writes. Active WebSocket connections = 1 Million.
Storage & RAM: 100M messages/day * 200 bytes = 20 GB/day. Replicated partition stores.
3. System API Endpoints
POST /api/v1/channels/message
Request: { channel_id: "chn-110", content: "Code builds successfully." }
Response: { message_id: "msg-9923", created_at: 1783708872 }
POST /api/v1/channels/read_marker
Request: { channel_id: "chn-110", last_read_message_id: "msg-9923" }
Response: { status: "updated" }4. Database Schema Design
Table: channel_messages (Cassandra - sharded by channel_id)
- channel_id (UUID - Partition Key)
- message_id (TimeUUID - Clustering Key)
- sender_id (UUID)
- content (TEXT)
Table: channel_read_markers (Cassandra)
- user_id (UUID - Partition Key)
- channel_id (UUID - Clustering Key)
- last_read_message_id (TimeUUID)5. Interactive High-Level Architecture
6. Scalability Deep Dive & Bottlenecks
Stores chat history in Cassandra partitioned by `channel_id` to scale writes. When a message is sent via WebSockets, the WebSocket Handler writes it directly to Cassandra and publishes the event to a Redis Pub/Sub backplane. All WebSocket Manager nodes subscribing to that channel's stream intercept the message and push it to connected channel members instantly.
7. Production Scaling Strategies
Channel Partitioning: Partition Cassandra by workspace_id + channel_id. Each workspace's data is co-located. At 750K+ organizations, shard workspaces across 200+ Cassandra nodes. Large channels (10K+ members) get their own dedicated partition to prevent hot spots.
WebSocket Backplane: Use Redis Pub/Sub (or NATS) as a cross-server message bus. When user A on Gateway-1 sends a message to a channel, all gateways subscribed to that channel receive the event and forward to their connected clients. Each gateway handles 50K connections.
Message Search (Elasticsearch): All messages are indexed in Elasticsearch via Kafka for full-text search. Indices sharded by workspace_id + time (monthly). Search queries are scoped to workspace and channel ACLs. Handles 100K search queries/sec across all workspaces.
File Storage: Uploaded files stored in S3, metadata in PostgreSQL. Generate thumbnails/previews async via worker pool. Serve files via CDN with signed URLs (expire in 1 hour). Enterprise customers get dedicated S3 buckets for data isolation.
Presence and Typing Indicators: 'User is typing' events are ephemeral — broadcast via WebSocket only, never persisted. Presence (online/away/DND) stored in Redis with 30s heartbeat TTL. Presence fan-out limited to channel members currently viewing the channel.
Rate Limiting per Workspace: Each workspace has API rate limits (Tier 1: 1 msg/sec per channel, Tier 4: 100 msg/sec for bots). Enforced via Redis sliding window counters. Prevents runaway bot integrations from overwhelming the message pipeline.
8. Runnable Production Starter Code
Scaffold