System Design Interview Questions: Practice for Every Level

The root problem is that the app nodes are stateful: each holds session data only it can see. Externalising state to a shared store makes the nodes interchangeable, which is the precondition for horizontal scaling and for free rolling deploys and failover. Sticky sessions are a workaround, not a fix — they pin users to a node, so a deploy or crash still drops their session and they defeat even load distribution. Vertical scaling (more RAM) raises the ceiling but keeps the state trapped on one box, so the cross-node inconsistency remains. A CDN caches public, cacheable responses; per-user session data is neither, so it does nothing here.

Write-through updates the cache and the database in the same write path, so the just-written value is already in cache for the next read — you get fresh reads immediately at the cost of a slower write, exactly the trade the team accepted. Plain cache-aside only populates the cache on a read miss, so the entry written can be stale until the next miss (and a delete-on-write is needed to avoid serving the old value). Write-behind acknowledges before the DB is durable, trading consistency and durability for write speed — the opposite of the stated priority. TTL-only caching guarantees staleness for up to the TTL window, which violates the "fresh immediately" requirement.

CAP says that when a partition occurs, a system can preserve either linearizable Consistency or Availability, not both. Partition tolerance is not optional in a real distributed system — networks drop packets and nodes fail — so P is assumed, and the live decision is C-vs-A. The other framings misstate this: you cannot "choose" to forgo partition tolerance and still be distributed, and consistency is precisely the property at risk, not the one that is guaranteed. Latency/durability describes PACELC's else-clause and write tuning, which is a separate axis from the partition-time CAP choice.

A monotonic timestamp routes all current writes to whichever shard owns the latest range, creating a hot partition no matter how many shards exist. Choosing a high-cardinality key (a hash of tenant or entity id) distributes writes uniformly, which is the actual fix. Adding replicas helps read load but not the write hotspot — replicas still funnel writes through one leader. Raising the shard count without changing the key leaves the newest range concentrated on a single shard. Vertically scaling the hot shard only raises its ceiling and reintroduces a single point of contention; it does not balance the distribution.

The symptom is replication lag: an async replica hasn't applied the user's write yet. Pinning that specific user's reads to the primary for a brief post-write window gives them read-your-writes consistency while everyone else still benefits from replica offload — minimal global cost. Sending all reads to the primary throws away the scaling benefit the replicas existed to provide. Forcing fully synchronous replication makes every write wait on every replica, hurting write latency and availability far beyond what this problem warrants. A fixed client-side delay is a guess — lag is variable, so it is both unreliable and a poor user experience.

Token bucket refills tokens at the sustained rate and lets unused tokens accumulate up to the bucket size, so an idle client banks capacity and can spend it in a burst — exactly the behaviour required. A leaky bucket smooths output to a fixed drain rate, deliberately flattening bursts, which is the opposite of what we want. A fixed window counter is bursty in the wrong way: it permits double the limit across a window boundary while still rejecting legitimate bursts inside a window. A concurrency semaphore of 1 limits parallelism, not request rate, and would serialise the client rather than rate-limit it.

A circuit breaker detects a run of failures, trips to the open state, and short-circuits subsequent calls — failing fast instead of letting requests stack up on a dead dependency — then probes for recovery via a half-open state. That directly stops the thread exhaustion and cascade. Raising the timeout makes things worse: each call holds a thread longer, accelerating exhaustion. Unbounded fixed-delay retries amplify load on an already-struggling dependency (a retry storm) and prevent it from recovering. Adding instances just provides more threads to be consumed by the same stuck calls, scaling the failure rather than containing it.

Distributed tracing stitches together the spans of one request as it crosses service boundaries via a propagated trace context, so you can see exactly which hop consumed the time — the precise need when each service looks fine on its own. Aggregate metrics tell you that p95 is high but average away the individual slow request and cannot attribute latency across the chain. Logs are per-service and unordered relative to each other; without a shared trace id you cannot reliably reconstruct one request's journey across all six. A health check only answers up/down liveness and says nothing about per-request latency distribution. Metrics, logs, and traces are complementary, but tracing is the one built for cross-service request attribution.

A queue decouples producers from consumers: checkout returns as soon as the event is enqueued (a), the queue acts as a buffer that smooths spikes so consumers process at a sustainable rate (b), and durable queues retain messages so a consumer that was down can drain the backlog on recovery (c). The two wrong options reflect common misconceptions. Async processing does not lower per-request end-to-end latency (d) — the downstream work still happens, just later; you trade latency-to-completion for responsiveness and resilience. And most queues offer at-least-once delivery, so consumers must be idempotent to tolerate duplicates (e); the queue does not remove that obligation.

Idempotency keys work because the client mints one key per logical operation and reuses it across retries (a), and the server records that key alongside the result so a repeat presents the same outcome instead of executing twice (b). They matter most for non-idempotent methods like POST (c) — GET/PUT/DELETE are already idempotent by definition, so a blind retry of those is safe. The wrong options break the mechanism: minting a new key per attempt (d) defeats the whole point — the server sees each retry as a distinct operation and double-charges. And TCP guarantees reliable, ordered bytes within a connection, not application-level exactly-once semantics across reconnects and timeouts (e); that is exactly the gap idempotency keys fill.

NoSQL shines when access is key-based and predictable at scale (a), when the schema is fluid and write volume is huge with native horizontal partitioning (b), and when you can denormalise to avoid joins and accept eventual consistency (c). The remaining two are signals to stay relational. Multi-row ACID transactions across related tables (d) are exactly what a relational engine guarantees cleanly, whereas many NoSQL stores limit transactions to a single partition or item. Ad-hoc analytical queries with complex joins and aggregations (e) are the relational/SQL sweet spot; forcing them onto a key-value or document model leads to expensive scatter-gather or duplicated denormalised data. The choice is driven by access patterns and consistency needs, not by one store being universally "better."

Availability comes from redundancy with automatic failover and removing single points of failure. Spreading stateless instances across multiple AZs behind a health-checking balancer (a) survives an instance or whole-zone outage; a replicated DB with automated leader promotion (b) removes the database as a single point of failure; and health checks that eject unhealthy nodes (c) keep traffic flowing only to working capacity. The two wrong options reduce availability. Putting everything in one AZ (d) trades resilience for latency — a single zone outage takes the whole service down. A single oversized instance (e) is the textbook single point of failure no matter how reliable the box: when it dies, you have zero capacity, which is why N+1 redundancy beats one big node.

A CDN caches content at edge points of presence near users (a), so it lowers latency and offloads repeat requests from the origin, cutting load and egress (b), with freshness controlled by Cache-Control/TTL and revalidation (c). The wrong options are classic edge-caching traps. Personalized, per-user dynamic responses are generally not cacheable at a shared edge (d) — caching them risks leaking one user's data to another, so they need private/no-store handling or edge-compute personalization, not naive caching. And edges do not instantly reflect origin changes (e): cached objects live until their TTL or an explicit purge/invalidation, which is precisely why a bad asset deploy can keep serving stale content until you invalidate the cache or bust the URL.

The two layers are often combined: an L4 balancer fronts a fleet of L7 proxies for scale, while the L7 tier handles routing logic. The cost of L7 is the per-request parsing and connection termination overhead, which is why latency-critical or non-HTTP paths sometimes stay at L4.

Always measure before acting so you scale the actual bottleneck, not a guess. Caching is the cheapest lever because it removes work entirely; adding stateless app instances is straightforward when the tier holds no session state; resharding the data tier is last because it is the most disruptive and hardest to reverse. The ordering reflects rising blast radius and engineering cost.