System Design Interview Questions: Microservices Architecture

Microservices buy independent deployability and scaling per team/domain — value that materialises when many teams own separate domains with divergent scaling needs. A 6-person single-product team has none of that yet, but would immediately pay the tax: network failure modes, distributed transactions/saga complexity, deploy and versioning coordination, and cross-cutting changes (their stated norm) now touching 15 repos at once. A modular monolith gives clean module boundaries and a path to extract services later, without that tax. The distractors are false absolutes: in-process speed isn't a blanket win that decides architecture (a), microservices use relational databases routinely (b), and they add rather than remove integration-testing surface (c).

In a multi-region active-active architecture, the core challenge with eventual consistency is handling concurrent writes to the same record from different regions. Vector clocks (or similar mechanisms like CRDTs) track causality across nodes so that conflicts can be detected and resolved rather than silently overwriting data. Last-Write-Wins (LWW) based solely on wall-clock time is unsafe because clocks can skew, making it possible to overwrite a newer write with an older one. Two-phase commit provides strong consistency but introduces cross-region coordination latency and is an availability risk — the opposite of what active-active targets. Read repair is a read-path technique and does not prevent write conflicts. Vector clocks remain the standard causality-tracking answer for active-active replication conflict detection.

The Saga pattern decomposes a cross-service transaction into a series of local service transactions, each publishing an event (choreography) or receiving an orchestrator command. If a step fails, previously completed steps are rolled back via compensating transactions (e.g. release the reserved inventory, void the payment). What you give up is ACID atomicity across services: the system is eventually consistent — during execution, partial state is visible, and compensations can fail, requiring careful idempotency and retry design. This is the accepted trade for autonomy and fault isolation. Long polling (b) is a transport mechanism that provides no consistency semantics. Re-merging into a monolith (c) defeats the whole point of the microservices architecture. Event sourcing (d) is a data-modelling pattern that offers durability and audit, not cross-service atomicity — and it is orthogonal to, not a replacement for, saga coordination.

They operate at different traffic boundaries. The API gateway is the north-south edge: it handles the public API surface — authentication, TLS termination, routing, rate limiting, and request aggregation for external callers. The service mesh is east-west: it intercepts every inter-service call via sidecar proxies (e.g. Envoy) to provide mTLS between pods, circuit breaking, retries, load balancing, and per-call telemetry without any changes to application code. These two layers complement each other — the gateway cannot see internal pod-to-pod calls, and the mesh is not designed as an external ingress. A gateway does not subsume mesh features (b); they have different scopes. Sidecars handle inter-service calls, not external ingress (c). A service mesh has nothing to do with database connection pooling (d) — that is managed by the application or a separate proxy like PgBouncer.

The question tests deep understanding of the CAP theorem applied to a concrete split-brain scenario. First, the system must detect the partition (A) — without detection, no decision can be made. Second, nodes must identify majority vs. minority (B) — this quorum assessment is a prerequisite to knowing which side to suppress or allow. Third, the architect must consult the system's SLA to decide between CP and AP behaviour (C). Fourth, the system enforces that decision (D): an AP system (e.g., Cassandra, DynamoDB) accepts writes on both sides and logs divergence; a true CP system (e.g., etcd, ZooKeeper — both Raft/Paxos-based) outright rejects writes that cannot achieve quorum on the minority side — queuing writes would defer the consistency decision and is an AP-leaning strategy, not CP. Finally, after healing (E), AP systems perform conflict resolution (LWW, vector clocks, CRDTs); CP systems simply re-sync minority nodes from the authoritative majority log, because those minority nodes accepted no writes and have nothing to replay.

The root problem is that the app nodes are stateful: each holds session data only it can see. Externalising state to a shared store makes the nodes interchangeable, which is the precondition for horizontal scaling and for free rolling deploys and failover. Sticky sessions are a workaround, not a fix — they pin users to a node, so a deploy or crash still drops their session and they defeat even load distribution. Vertical scaling (more RAM) raises the ceiling but keeps the state trapped on one box, so the cross-node inconsistency remains. A CDN caches public, cacheable responses; per-user session data is neither, so it does nothing here.

Write-through updates the cache and the database in the same write path, so the just-written value is already in cache for the next read — you get fresh reads immediately at the cost of a slower write, exactly the trade the team accepted. Plain cache-aside only populates the cache on a read miss, so the entry written can be stale until the next miss (and a delete-on-write is needed to avoid serving the old value). Write-behind acknowledges before the DB is durable, trading consistency and durability for write speed — the opposite of the stated priority. TTL-only caching guarantees staleness for up to the TTL window, which violates the "fresh immediately" requirement.

CAP says that when a partition occurs, a system can preserve either linearizable Consistency or Availability, not both. Partition tolerance is not optional in a real distributed system — networks drop packets and nodes fail — so P is assumed, and the live decision is C-vs-A. The other framings misstate this: you cannot "choose" to forgo partition tolerance and still be distributed, and consistency is precisely the property at risk, not the one that is guaranteed. Latency/durability describes PACELC's else-clause and write tuning, which is a separate axis from the partition-time CAP choice.

A monotonic timestamp routes all current writes to whichever shard owns the latest range, creating a hot partition no matter how many shards exist. Choosing a high-cardinality key (a hash of tenant or entity id) distributes writes uniformly, which is the actual fix. Adding replicas helps read load but not the write hotspot — replicas still funnel writes through one leader. Raising the shard count without changing the key leaves the newest range concentrated on a single shard. Vertically scaling the hot shard only raises its ceiling and reintroduces a single point of contention; it does not balance the distribution.

A queue decouples producers from consumers: checkout returns as soon as the event is enqueued (a), the queue acts as a buffer that smooths spikes so consumers process at a sustainable rate (b), and durable queues retain messages so a consumer that was down can drain the backlog on recovery (c). The two wrong options reflect common misconceptions. Async processing does not lower per-request end-to-end latency (d) — the downstream work still happens, just later; you trade latency-to-completion for responsiveness and resilience. And most queues offer at-least-once delivery, so consumers must be idempotent to tolerate duplicates (e); the queue does not remove that obligation.

Idempotency keys work because the client mints one key per logical operation and reuses it across retries (a), and the server records that key alongside the result so a repeat presents the same outcome instead of executing twice (b). They matter most for non-idempotent methods like POST (c) — GET/PUT/DELETE are already idempotent by definition, so a blind retry of those is safe. The wrong options break the mechanism: minting a new key per attempt (d) defeats the whole point — the server sees each retry as a distinct operation and double-charges. And TCP guarantees reliable, ordered bytes within a connection, not application-level exactly-once semantics across reconnects and timeouts (e); that is exactly the gap idempotency keys fill.

NoSQL shines when access is key-based and predictable at scale (a), when the schema is fluid and write volume is huge with native horizontal partitioning (b), and when you can denormalise to avoid joins and accept eventual consistency (c). The remaining two are signals to stay relational. Multi-row ACID transactions across related tables (d) are exactly what a relational engine guarantees cleanly, whereas many NoSQL stores limit transactions to a single partition or item. Ad-hoc analytical queries with complex joins and aggregations (e) are the relational/SQL sweet spot; forcing them onto a key-value or document model leads to expensive scatter-gather or duplicated denormalised data. The choice is driven by access patterns and consistency needs, not by one store being universally "better."

Availability comes from redundancy with automatic failover and removing single points of failure. Spreading stateless instances across multiple AZs behind a health-checking balancer (a) survives an instance or whole-zone outage; a replicated DB with automated leader promotion (b) removes the database as a single point of failure; and health checks that eject unhealthy nodes (c) keep traffic flowing only to working capacity. The two wrong options reduce availability. Putting everything in one AZ (d) trades resilience for latency — a single zone outage takes the whole service down. A single oversized instance (e) is the textbook single point of failure no matter how reliable the box: when it dies, you have zero capacity, which is why N+1 redundancy beats one big node.

A CDN caches content at edge points of presence near users (a), so it lowers latency and offloads repeat requests from the origin, cutting load and egress (b), with freshness controlled by Cache-Control/TTL and revalidation (c). The wrong options are classic edge-caching traps. Personalized, per-user dynamic responses are generally not cacheable at a shared edge (d) — caching them risks leaking one user's data to another, so they need private/no-store handling or edge-compute personalization, not naive caching. And edges do not instantly reflect origin changes (e): cached objects live until their TTL or an explicit purge/invalidation, which is precisely why a bad asset deploy can keep serving stale content until you invalidate the cache or bust the URL.

Asynchronous replication acknowledges the write on the primary before the change has propagated, so a read routed to a lagging replica can miss the just-written value — the classic read-your-writes anomaly. Mitigations include routing a user's reads to the primary for a short window after a write, using a synchronous/quorum write, or tracking replication lag and pinning sticky reads.