Definition:
Cell-Based Architecture (also called cellular architecture, cell-based design, or deployment stamps in Azure’s vocabulary) is a fault-isolation pattern in which a workload is split into multiple complete, independent copies of itself — cells — where each cell serves a slice of traffic chosen by a partition key, and a thin cell router in front maps every request to exactly one cell. Cells share nothing: their own compute, their own database, their own cache, their own queues, their own secrets. A crash, an overload, a bad deploy, or a poison-pill request inside one cell cannot escape it.
The failure modes cells exist for are the ones redundancy does not fix. “A machine died” is not the target — that is what load balancers and replicas have handled for twenty years. Cells target the correlated failures: the ones where every copy of your stack falls over at the same instant because they are all running the same bad thing:
- a bad deployment or a bad schema migration (every instance gets the same bug at the same time)
- a poison-pill request or tenant — one query shape, one customer, that saturates a shared resource everyone else depends on
- hitting a hard quota ceiling (connection pool, partition throughput, API rate limit) that no amount of horizontal scaling raises
- a black-swan / metastable failure — a retry storm or a cache stampede that feeds on itself fleet-wide
In a single shared pool all of these are fleet-wide events. AWS’s Well-Architected guidance frames the design question better than any paraphrase: “Is it better for 100% of customers to experience a 5% failure rate, or 5% of customers to experience a 100% failure rate?” Cell-based architecture answers the second, on purpose — it trades a small, bounded, known group of users being fully degraded for the guarantee that nobody else notices. That bounded number is called the blast radius, and shrinking it is the entire point.
SINGLE SHARED STACK vs CELL-BASED — where the blast radius is measured:
═══════════════════════════════════════════════════════════════════════
WITHOUT CELLS — one stack, every tenant in the same pool:
all tenants, all traffic
│
▼
┌─────────────────────────────────────────┐
│ ONE SHARED STACK │
│ ┌────────┐ ┌────────┐ ┌───────────┐ │
│ │ API │ │ Redis │ │ workers / │ │
│ │ hosts │ │ cache │ │ queues │ │
│ └───┬────┘ └───┬────┘ └─────┬─────┘ │
│ └───────────┴─────────────┘ │
│ │ │
│ ┌────────▼─────────┐ │
│ │ ONE DATABASE │ │
│ │ one conn pool │ │
│ │ one quota │ │
│ └──────────────────┘ │
└─────────────────────────────────────────┘
ONE bad deploy · ONE poison-pill tenant ·
ONE quota ceiling · ONE hot partition
│
▼
EVERY tenant is hit at once
BLAST RADIUS = 100% of tenants
WITH CELLS — traffic sliced, the whole stack replicated per slice:
all tenants
│
┌─────────▼─────────┐
│ CELL ROUTER │ stateless, thin,
│ hash(tenant_id)%N │ statically stable
│ (deterministic!) │ the ONLY shared part
└──┬─────┬─────┬────┘
┌─────────────┘ │ └─────────────┐
▼ ▼ ▼
┌────────────┐ ┌────────────┐ ┌────────────┐
│ CELL 0 │ │ CELL 1 │ ... │ CELL N-1 │
│ ┌────────┐ │ │ ┌────────┐ │ │ ┌────────┐ │
│ │ API │ │ │ │ API │ │ │ │ API │ │
│ │ Redis │ │ │ │ Redis │ │ │ │ Redis │ │
│ │ queue │ │ │ │ queue │ │ │ │ queue │ │
│ │ DB │ │ │ │ DB │ │ │ │ DB │ │
│ └────────┘ │ │ └────────┘ │ │ └────────┘ │
│ tenants │ │ tenants │ │ tenants │
│ 0, 8, 16… │ │ 1, 9, 17… │ │ 7, 15, 23… │
└────────────┘ └────────────┘ └────────────┘
no state shared between cells — ever, by design
ONE bad tenant / ONE bad deploy is CONTAINED
BLAST RADIUS = 1 cell = 1/N of tenants
(N = 4 → 25%, N = 8 → 12.5%)
and it is bounded BY CONSTRUCTION, not by luck
┌────────────────────────────────────────────────────────────────┐
│ KEY IDEA: redundancy protects you from things that BREAK (a │
│ disk, an AZ). Isolation protects you from things that │
│ BREAK EVERYONE — a bad deploy, a poison pill, a quota wall. │
│ Cells are not "more servers"; they are a smaller number of │
│ users who can be sacrificed to keep the rest correct. │
└────────────────────────────────────────────────────────────────┘The three components. Every cell-based system decomposes into exactly these, and getting the boundaries between them right is the whole design:
- The cells — a template instantiated with a cell identifier. A cell is a complete stack: API/compute, data store, cache, queues, and whatever else the request path touches. The critical property is that it owns its state. The most under-appreciated line in the AWS whitepaper, and the one that gets this pattern past a budget holder: “Building a cell-based architecture doesn’t necessarily mean having to double, triple, or more your application’s infrastructure. It might be that your application has 30 hosts, and in a cell-based architecture it has the same 30 hosts, but with a cell router and with tasks that are distributed or grouped between cells.” You are not buying N times the fleet — you are regrouping the fleet you have and paying extra only for the stateful parts (N databases instead of one).
- The cell router — the hardest component, because it is the one thing that cannot be cellularized. It is the only element holding shared state about all cells, so it must be brutally thin, stateless, deterministic, and unavailable-free: ideally a hash of the partition key against a mapping that is cached in-process and replicated (S3/CDN/Route 53/API-Gateway routing rules/edge worker), never a call to a live control plane on the request path. Two rules matter more than any implementation detail: the mapping must be deterministic (same tenant → same cell, always; hash the tenant, never the request, or you lose data locality and cache hits and start splitting a tenant’s writes across cells), and the router must be statically stable — it must keep serving correctly with the control plane completely down.
- The control plane — everything that manages the fleet rather than serving traffic: provisioning cells, maintaining the tenant→cell mapping, deploying code, migrating tenants between cells, running migrations. It is deliberately off the request path. Cells move the deployment problem from “shipping to production” to “shipping to N productions”, which is exactly why the control plane has to be fully automated: if creating cell 51 involves a manual step, you do not have a cell-based architecture, you have N snowflakes.
CONTROL PLANE vs DATA PLANE — static stability is the requirement
═══════════════════════════════════════════════════════════════════
CONTROL PLANE (off the request path, may be down) DATA PLANE (always on)
┌────────────────────────────────────────────┐ ┌───────────────────┐
│ • provision / de-provision cells │ │ cell router │
│ • own the tenant → cell mapping │ │ + │
│ • deploy code, run migrations │ │ cells 0..N-1 │
│ • migrate tenants between cells │ └─────────┬─────────┘
└───────────────────┬────────────────────────┘ │
│ pushes mapping + config (cached, │
│ replicated, versioned) │
└────────────────────────────────────────►│
requests never wait
on the control plane
If the control plane dies, the data plane must KEEP SERVING with the
mapping it already has. Otherwise your "isolation" has a single point
of failure sitting right in front of it.Shuffle sharding: isolation inside a cell. Cells and shuffle sharding are constantly confused, and the distinction is precise. Shuffle sharding (AWS, 2014 — published by Colm MacCarthaigh, built for Route 53’s Infima library, framed as a deck of cards) gives each tenant its own unique combination of the fungible workers: with M workers and a shard size r, a tenant is assigned a distinct r-sized subset, so two tenants overlap only if they happened to draw the identical combination. With 8 workers in shards of 2 there are C(8,2) = 28 distinct shards, so the chance two random tenants collide is ~3.6%; scale to 100 workers in shards of 2 and there are 4,950 shards — a collision probability of 0.02%. In shards of 3: 161,700 shards, 0.0006%.
FAN-OUT (naive) SHUFFLE SHARDING (M=8, r=2 → 28 shards)
════════════════════════════ ═══════════════════════════════════════
every tenant hits every worker each tenant gets a UNIQUE pair
tenant A ──►┌───────────┐ tenant A ──► {w0, w1}
│ w0 .. w7 │ tenant B ──► {w4, w5}
tenant B ──►│ ALL │ tenant C ──► {w2, w6}
└───────────┘ ...all 28 combinations in use...
one bad tenant poisons the two tenants share a fate only if they
WHOLE pool → drew the IDENTICAL shard: 1/28 = 3.6%
(M=100, r=2 → 4,950 shards → 0.02%)
SPREAD THE RISK vs GIVE EVERY ONE OF THEM A DIFFERENT RISKThe boundary rule AWS states outright: no overlap between cells, shuffle sharding inside a cell. Cells bound the impact of a bad deployment or a poison pill on stateful things; shuffle sharding bounds the impact of a noisy tenant on a cell’s fungible resources — worker pools, queues, rate limiters, sender slots. Letting a shuffle shard span cells would destroy the cell’s independence, which is the one thing the pattern cannot give up: “in a cell-based architecture, a cell should be self-contained, not share its state.” A cell is defined by sharing nothing; a shuffle shard is defined by deliberately overlapping.
What cells are not — the four mix-ups, and why each one matters:
- Not a failover domain. This is the most consequential misunderstanding. A cell is a fault-isolation boundary, not a spare that tenants get moved onto. If your disaster-recovery plan is “fail over to another cell”, you do not have a cell-based architecture — you have an active-passive pair with extra steps, and your tenants’ state does not live where you are sending them. When a cell fails, its tenants are degraded or down; the win is that everyone else is untouched. Cells do not make you highly available, they make you fault-isolated.
- Not the same as multi-AZ. Multi-AZ gives you redundant copies inside a provider-defined failure domain. Cells create isolation boundaries you define with a partition key — and they compose, because the boundary is yours to place: a cell can be zonal, regional, or global. Slack’s implementation (aligned to Availability Zones after a 2021 AZ networking incident) puts one complete, siloed backend deployment in each AZ, assigns workspaces to cells by workspace ID, and leaves only the low-traffic admin plane (workspace creation, billing) shared.
- Not the same as the Bulkhead Pattern (already in this journal). A bulkhead partitions resource pools inside one shared stack — a separate thread pool and connection pool per downstream dependency, so one slow dependency cannot starve the others. It still shares the app, the database, and the schema with everyone. A cell partitions the entire stack by traffic slice, all the way down to the database. Bulkhead: “don’t let dependency X starve dependency Y.” Cell: “don’t let tenant A’s bad day ever be tenant B’s bad day.”
- Not free. Running N copies of the stateful layer costs real money. In practice the published rule of thumb is ~40% added infrastructure cost, not N× — compute scales sub-linearly because each cell holds fewer tenants — plus a permanent complexity tax. Every operational procedure grows a “which cell?” step: migrations, schema changes, config rollouts, secrets, and dashboards all become cell-aware; deprovisioning or rebalancing eventually demands online tenant migration between cells; and global features (cross-tenant search, analytics, billing, admin) need a separate aggregation layer that reads from all cells — which must be thin, read-only from the cell’s perspective, and highly available, or it becomes the shared failure domain you just spent all that money removing.
Operating a cell fleet has three disciplines that teams learn the hard way. First, deploy in waves with a canary cell: one cell always receives code and migrations first, is watched for a defined window, and is rolled back alone if a signal appears — turning a fleet-wide 45-minute outage into a 5-minute degradation for 1/N of users. Second, observability must be cell-scoped: tag every metric with a cell label and publish the worst cell’s health, not the fleet average, because the aggregate dashboard lies — a fleet-wide error rate of 0.4% looks healthy and can be one cell sitting at 100% with the rest at 0%. Third, audit capacity and drift per cell on a schedule: quota headroom against service limits, configuration drift between cells, and whether each cell can absorb the load placement might send it. A cell has a published, tested maximum — that is the number that makes all of this tractable.
The honest decision rule, which is also where to start: use cells when a single failure domain affecting all users is an existential risk to the business, and your workload is multi-tenant with a stable natural partition key. Below that bar, circuit breakers, bulkheads, load shedding and multi-AZ are cheaper and enough. And when you do build it: start with 4 cells, not 40. Each cell is a full copy of the stack; more cells means finer isolation and exponentially more operational overhead. Teams that start at 8 and shrink to 4 once the monitoring burden becomes real are following the standard path.
Example:
Kyomel’s fasting-bot outgrew the friend group. What started as one WhatsApp group became “Komunitas” — 12,000 community workspaces (gyms, mosques, offices, alumni groups) with ~1.4M members, each community with its own leaderboard, streaks, and reminder scheduler. Architecturally it was the obvious v1: one Go service, one Postgres, one Redis, one worker pool, and one notification scheduler, all shared. Traffic is brutally rhythmic — a sahur burst at 04:00–05:00 WIB and a buka-puasa burst at 17:30–18:15 WIB — when every community’s membership wants the same “jangan lupa sahur” ping within the same twenty minutes.
Then KomunitasGymBesar joined. One workspace, 40,000 members, and a leaderboard feature that runs SELECT member, COUNT(*) FROM fasting_logs ... GROUP BY member ORDER BY count DESC LIMIT 100 plus an “export my streak” CSV for every member. At 04:12 WIB it saturated the shared connection pool, and 12,000 unrelated communities learned about it instantly.
BEFORE — 04:12 WIB, ONE POISON PILL, EVERY COMMUNITY HIT
═══════════════════════════════════════════════════════════════════
KomunitasGymBesar ──►┌──────────────────────────────────────────┐
(40k members: │ ONE SHARED STACK │
leaderboard ×40k, │ │
CSV export, GROUP │ ┌──────────────┐ │
BY over 900 days of │ │ Postgres │ max_connections = 200 │
fasting logs) │ │ 200/200 BUSY │ 8s waits, then 500s │
│ │ └──────┬───────┘ │
│ │ │ │
│ │ ┌──────▼───────┐ │
│ │ │ Redis + │ cache misses pile up, │
│ │ │ worker pool │ 1 shared queue │
│ │ └──────┬───────┘ │
└─────────────►└──────────┼───────────────────────────────┘
│
┌─────────────────────────┴──────────────────────────┐
▼ ▼
04:00 sahur reminder job streak / leaderboard API
→ queued behind 40k exports → 30s timeouts, HTTP 500
→ delivered 41 MINUTES LATE → members think streaks broke
ALL 12,000 KOMUNITAS ◄──────────────────────┘
missed or got a useless sahur ping
BLAST RADIUS: 100% of tenants · DURATION: 41 min
A week later the same shape repeated with a BAD DEPLOY: an index
migration on fasting_logs took a table lock and held it for 45 min —
every instance ran the same migration, so every instance was down.
ROOT CAUSE: no isolation boundary anywhere between a tenant
and the shared state every tenant depends on.They rebuilt it as a cell-based architecture — but with the same 12 app hosts they already ran. The stack was regrouped into 8 cells, each cell being 1–2 app hosts plus its own small Postgres, its own Redis, and its own worker pool, with a stateless cell router at the WhatsApp gateway assigning every workspace deterministically via hash(workspace_id) % 8.
AFTER — same poison pill, same bad deploy: 1/8 of the fleet
═══════════════════════════════════════════════════════════════════
WhatsApp inbound ──►┌─────────────────────────────────────────────┐
(wa-gateway) │ CELL ROUTER — stateless, statically stable │
│ mapping: workspace_id → cell │
│ cached in-process + replicated (versioned) │
│ hash the WORKSPACE, never the message │
└──┬────┬────┬────┬────┬────┬────┬────┬──────┘
┌────────────────┘ │ │ │ │ │ │ └────────┐
▼ ▼ ▼ ▼ ▼ ▼ ▼ ▼
┌───────────┐ ┌───────────┐ ... ... ... ... ┌───────────┐
│ CELL 0 │ │ CELL 1 │ │ CELL 7 │
│ ◄ canary │ │ │ │ │
│ API │ │ API │ │ API │
│ Postgres │ │ Postgres │ │ Postgres │
│ Redis │ │ Redis │ │ Redis │
│ workers │ │ workers │ │ workers │
│ ~1,500 │ │ ~1,500 │ │ ~1,500 │
│ tenants │ │ tenants │ │ tenants │
└─────┬─────┘ └───────────┘ └───────────┘
│
│ GymBesar lives HERE — it is just one of ~1,500 tenants
▼
04:12 its own conn pool saturates (200/200 busy, 8s waits)
• its own members' reminders late, its own API 500s
──────────────────────────────────────────────────────────────
CELLS 1–7: sahur reminders delivered ON TIME, no 500s,
no idea anything happened. p99 unchanged.
The support channel gets messages from ONE
community, not twelve thousand.
BLAST RADIUS: 12.5% of tenants · 7/8 of the fleet unaffected
DEPLOY WAVES (fixes the bad-migration failure mode too):
cell 0 (canary) ──► watch 5 min ──► cell 1 ──► … ──► cell 7
bad migration on cell 0? ROLLBACK CELL 0 ALONE.
5 minutes, 12.5% of tenants, other 87.5% never saw it.Shuffle sharding came in one layer inside each cell, for the piece that is genuinely fungible: the WhatsApp sender slots. Each cell has 64 sender slots against the WA gateway, and a community that bursts (or gets rate-limited) must not clog the slots everyone shares — so each community was assigned a unique pair of senders out of the cell’s 64: C(64,2) = 2,016 distinct shards, meaning two random communities collide with probability ~0.05%. GymBesar flooding its own two senders costs GymBesar latency, not the cell.
THE SHAPE THEY ENDED UP WITH
────────────────────────────────────────────────────────────────
cells 8 cells (1–2 app hosts each) on the SAME 12 hosts
that already existed — only the stateful layer
multiplied (8 small Postgres + 8 Redis instead of 1)
partitioning hash(workspace_id) % 8, deterministic, cached at the
router; a workspace NEVER spans two cells
canary cell 0 permanently receives deploys + migrations first
inside a cell 64 WA sender slots, shuffle-sharded r=2 per community
global view cross-community leaderboard = nightly batch job over
all cells → thin read-only API (NOT on the request path)
observability every metric tagged cell="0..7"; the dashboard shows
the WORST cell's availability, not the fleet average
cost +38% infrastructure (extra databases, caches, per-cell
dashboards + tooling) for blast radius ÷ 8
────────────────────────────────────────────────────────────────Three notes from their migration that generalise. They cellularized the template before they cellularized anything real — cell 0 was stood up from IaC as a throwaway and rebuilt from scratch four times until provisioning a cell was one boring command. Without that step, 8 cells would have become 8 hand-tuned snowflakes and the whole exercise would have made the fleet fragile. They moved the global leaderboard off the request path before opening the second cell, because a cross-cell feature is the one place where the isolation quietly leaks back in: an on-demand query that reads all 8 cells would have made the aggregation layer the new single point of failure. And they resisted going wider: the plan started at 16 cells, they shipped 8, and after a month of running per-cell dashboards they were glad they had not — the monitoring and migration work was already the dominant cost, and 12.5% was a blast radius they could live with.
The punchline: cell-based architecture is the admission that not all failures can be made rare, so we make them small instead. For two decades the answer to “a bad thing happened to everyone” was stronger engineering — safer migrations, better reviews, more capacity, more retries. Cells accept that some of these will get through anyway and change the shape of the damage: from a vertical cliff where the whole product is down to a horizontal slice where 1/N of users have a bad ten minutes and everyone else keeps doing what they were doing. It is not free, it is not for everyone, and it does not make you highly available — it makes you isolated. But for any multi-tenant system where a table lock or one customer’s runaway query can reach every user at once, the question stops being “should we do cells?” and becomes the one AWS asks: whose bad day are you willing to let it be?