Field Note 20 current
The simplest-looking system is often the most complex to live with
TL;DR
Don’t confuse easy to stand up with simple. The classic starter stack — a backend service, Postgres, and a Redis cache — looks simpler than a more deliberately-architected design, and it is: simpler to start. But the hard parts of the problem (consistency, invalidation, concurrency, partial failure, ordering, multi-tenancy) don’t go away because you didn’t build for them — they’re still there, now surfacing as customer-visible bugs and data inconsistency. A design with more moving parts that actually addresses those cases has fewer surprises, and in data architecture it very often turns out to be the simpler system to live with over time. Pay the essential complexity on purpose, up front, rather than accreting it as patches later.
This is not a license to over-engineer. It’s the narrower claim that when you know the hard cases exist, the design that handles them — even if it has more components — usually beats the one that pretends they don’t.
Context
“Simple” is two different things, and conflating them is the trap:
- Easy — familiar, fast to assemble, few boxes on the diagram on day one.
- Simple — few entanglements; few edge cases and failure modes you have to hold in your head to reason about correctness. (Rich Hickey’s Simple Made EasyRich Hickey — Simple Made Easy (Strange Loop 2011)Rich Hickey's talk separating simple (one role or concept, un-entangled — an objective property of a thing) from easy (familiar, near-at-hand — a subjective relationship to it). He argues we habitually optimise for easy and pay in incidental complexity, and that choosing the simple option is what keeps systems reliable and changeable over the long run.infoq.com ↗ draws exactly this line.)
The backend-plus-Postgres-plus-cache version is easy. It is not necessarily simple, because the genuinely hard parts of a stateful product are still present, just unhandled:
- The cache and the database disagree, and there’s no correct invalidation, so reads go stale and users see inconsistency (ZFN-21).
- Concurrent writes race because nothing was designed for it; retries double-apply because nothing is idempotent (ZFN-19).
- A partial failure leaves two stores out of sync with no path back to consistency.
- It can’t be sharded later because everything assumed one database (ZFN-15).
So the “simple” system spends its second year accreting special cases — a lock here, a reconciliation cron there, a cache-busting hack, a nightly repair job. That’s accidental complexity: the worst kind, because no one chose it and no one understands all of it. The system that looked complex up front — a real event/state model, projections, partitioning, idempotency, a control/data-plane split — paid its complexity as essential complexity, deliberately, in places you can name and reason about. Over the life of the system, fewer surprises wins.
Recommendation
Choose the design that minimises total, whole-life complexity, not day-one part count — and in stateful/data-heavy systems, that’s frequently the more deliberate design.
- Distinguish essential from accidental complexity. Essential complexity is inherent to the problem (you have multiple tenants; writes race; failures are partial) — it must be handled somewhere. The only choice is whether you handle it deliberately or accrete it as patches. Accidental complexity is the patches. Spend on the first to avoid the second.
- Don’t price the edge cases at zero. When estimating “the simple option,” include the cost of the consistency bugs, the data-repair work, the migration you’ve foreclosed, and the on-call. Those are real and they’re usually what makes the easy option expensive.
- Still apply YAGNI to the speculative. This is about complexity you know is essential, not about building for imagined futures. If a hard case genuinely won’t exist, don’t design for it. The judgement is “essential vs. speculative,” not “complex is always better.”
- Get the data model and contracts right early, because those are the expensive things to change later (ZFN-1, ZFN-14, ZFN-15, ZFN-17). A clean data architecture is what lets the implementation behind it stay genuinely simple.
- When you did take the easy path, plan to dig out. Quarantine the patched part behind an interface and replace it (ZFN-22, ZFN-23) rather than patching the patch.
Consequences
Easier:
- Fewer correctness incidents and less data-inconsistency firefighting, because the hard cases were designed for instead of discovered in production.
- The system stays reason-about-able and extendable; new work composes with a real model instead of threading through special cases.
- You’re not foreclosing scale, sharding, or evolution by baking in a too-simple assumption.
Harder:
- More to build and understand up front, and a real risk of talking yourself into over-engineering — the essential-vs-speculative call takes judgement and honesty.
- It’s a harder sell early (“why is this so involved?”) precisely because the cost it avoids is invisible until later — you’re trading a visible up-front cost for an invisible avoided one.
- Some genuinely simple problems are well served by the easy stack; this note is about stateful, correctness-sensitive data architecture, not every CRUD app.
References
- ZFN-21 — the canonical “easy but not simple” patch: reaching for a cache instead of fixing the access pattern.
- ZFN-2 — the easy option often quietly trades correctness for performance/convenience.
- ZFN-15, ZFN-17, ZFN-16 — the deliberate data-architecture choices that pay off over the system’s life.
- ZFN-22 / ZFN-23 — how to dig out when you took the easy path.
- Rich Hickey, Simple Made Easy — simple (un-entangled) vs easy (familiar/quick).
Changelog
- 2026-06-12: First published as a Field Note.