Deletion is a feature: design it on day one

Context

Systems are built to remember. Every pattern in the modern data stack — replication, projections, event journals, caches, immutable logs, append-only analytics — is a machine for making copies, and each copy is made by a different subsystem with a different owner. Deletion has to swim against all of it, and it’s the only common data operation that regularly isn’t designed at all. It gets a column.

The deleted_at column is fine for what it actually is: an undo feature. Users delete things by accident; a grace window is good product. The trouble starts when the same mechanism is allowed to stand in for erasure:

• The row is still there, so every query must remember to filter it — and the one that forgets resurrects data its owner was told is gone.
• Nothing happened downstream: the search index still matches, the cache still serves (ZFN-21Field Note · currentZFN-21 — Cache only immutable objects; treat caches as tech debtUse caches sparingly, only for immutable addressed objects — never for mutable DB results, where invalidation bugs and stale reads live; use projections instead. A cache in the data path is usually a patch over an architectural gap that trades correctness for performance.Open ZFN-21 →), the analytics table still joins, the read replica of a projection nobody owns still answers.
• The backups hold every copy ever made, indefinitely, and nobody can say which ones.

Then the forcing event arrives — an Article 17 request with a statutory clock on it, an enterprise contract with “certified deletion within 30 days” in the DPA, a breach where the question is “why did we still have that?” — and the team discovers that deletion is a cross-team, cross-store project being invented under a deadline. The cost asymmetry is the same one as tenancyField Note · currentZFN-15 — Partition customer data by tenant from day oneMake customer data tenant-partitioned from day one: tenant-scope every query, never join across tenants, route through a tenant→location directory. Run one physical database at first — but keep the model shardable. Retrofitting isolation onto a shared DB is brutal.Why it's cited here: Partitioning by tenant is what makes tenant deletion an operation instead of a query festival — data organised by owner can be dropped by owner.Open ZFN-15 →: a day-one design decision, or a brutal retrofit.

There’s a deeper point under the compliance one: data you were supposed to delete and didn’t is pure liability. It serves no product purpose, the customer believes it’s gone, and it’s still in scope for your next breach. Minimising what you retain is the same security posture as minimising what you expose.

Recommendation

Treat deletion as a first-class workflow with an owner, a deadline, and a proof.

• Two named operations, never conflated. Soft-delete: user-visible, reversible, a grace window — product owns it. Erasure: irreversible, propagating, deadline-bound — the platform owns it, and soft-deleted data flows into it when the window lapses.

• Organise data so deletion is an operation, not a search. Owner-partitioned stores (ZFN-15Field Note · currentZFN-15 — Partition customer data by tenant from day oneMake customer data tenant-partitioned from day one: tenant-scope every query, never join across tenants, route through a tenant→location directory. Run one physical database at first — but keep the model shardable. Retrofitting isolation onto a shared DB is brutal.Why it's cited here: Partitioning by tenant is what makes tenant deletion an operation instead of a query festival — data organised by owner can be dropped by owner.Open ZFN-15 →) turn “erase tenant” into dropping a partition, a keyspace, a prefix. The interesting question for any new store in a design review: “how does data leave this?” If the answer is a full scan with a WHERE clause someone must write later, the design isn’t done.

• Propagate on the rails you already trust. Erasure is a write like any other: committed in the owning store, emitted as a tombstone event in the same transaction (ZFN-48Field Note · currentZFN-48 — Emit async work into the WAL, not a job tableWhen a DB write should trigger async work, ride the WAL instead of dual-writing or polling a job table. pg_logical_emit_message emits the event transactionally — outbox semantics, no table. A WAL listener consumes it statefully and fans out, keeping load off the primary.Open ZFN-48 →), consumed by every projection, index, and cache — which each must treat “delete” as a mandatory part of their contract, not an optional message type. A derived store that can’t process a tombstone is a derived store that can’t legally exist.

• Crypto-shred the immutable tail. Backups, WAL archives, cold object storage — you can’t rewrite them, and restoring-scrubbing-rearchiving every backup is fantasy. Encrypt per tenant (or per user, where individual erasure matters) with keys held in one place; erasure of the key is erasure of every copy at once (NIST SP 800-88NIST SP 800-88 Rev. 1 — Guidelines for Media SanitizationThe reference for what 'actually gone' means at the storage layer, including cryptographic erasure: destroying the key that encrypts the data as a sanitisation technique — the mechanism that makes deletion tractable in backups and archives you can't rewrite.csrc.nist.gov ↗ calls this cryptographic erasure). This only works if it’s designed in — key-per-owner from the start, and a key store whose own deletion is real. Where old data predates the scheme, document the honest fallback: backup expiry windows, stated in the retention policy the customer sees.

• Starve the stores you can’t clean. Logs, traces, and metrics are effectively immutable and widely replicated — so personal data must not enter them. IDs, not emails (ZFN-56Field Note · currentZFN-56 — IDs are an interface: prefix the type, randomise the bodyAn ID is read by more than your database: humans in logs, machines at boundaries, adversaries probing. Serve all three — a type prefix so IDs self-describe and misuse fails at parse time, a random body so nothing leaks or enumerates, time-ordered only when the index needs it.Open ZFN-56 → helps: an opaque ID in a log is a pointer to data you can erase, not a copy of it). Enforce at the emission layer; retrofit-scrubbing a log archive is the same fantasy as rewriting backups.

• Track third parties as deletion targets. Every vendor you sync data into — support desk, analytics, email — is part of your erasure surface. The inventory of “where does customer data go?” is the deletion runbook’s first page.

• Record the tombstone, not the corpse. Proof of deletion is a durable record that erasure of X ran to completion on date D across systems S — retained after the data is gone (ZFN-49Field Note · currentZFN-49 — Verify by computation, not lookup; store revocations, not issuancesVerification should be a computation, not a query: HMACs, signatures, hashes, and pass-by-value claims let any node verify locally. When revocation is rarer than issuance, invert the state — keep the few revocations for the lifetime of what they revoke, not a row per grant.Open ZFN-49 →: store the small set of revocations, not the world they revoke). The tombstone also prevents resurrection when a nine-day-old backup is restored: replaying tombstones after restore is part of the restore rehearsalField Note · currentZFN-36 — An untested backup is not a backup — test it by restoringAn untested backup is a hope, not a backup — the only thing that counts is a restore. Rehearse restores regularly (game days), measure and meet your RTO/RPO, automate them, and cover the whole recovery path — data, schema, config, secrets, cutover — not just the dump.Why it's cited here: The same discipline pointed at the opposite guarantee: a deletion you haven't tested by verifying absence is a hope, exactly like an untested backup.Open ZFN-36 →.

• Rehearse it. Pick a real (test) tenant, run erasure, then go look for the data — primaries, projections, indexes, caches, a restored backup. An erasure workflow you’ve never verified by absence is exactly as trustworthy as a backup you’ve never restored.

Consequences

Easier:

• The compliance clock stops being frightening. An Article 17 request or an offboarding DPA clause is a runbook execution, not a project.
• Breach blast radius shrinks to data you actually meant to have.
• Restores stop resurrecting the dead — the tombstone log replays over whatever comes back.
• “Where does customer data live?” has an answer, which pays off in every audit, incident, and architecture review, not just deletion.

Harder:

• Every derived store now carries a contract obligation — consume tombstones, converge to absence — and that’s a real tax on the “just add a projection” reflex.
• Key-per-owner encryption is operational machinery: a key store with real availability needs, key lifecycle, and the sharp edge that losing keys is deletion, of the involuntary kind.
• Analytics and ML pipelines feel it most — aggregates are fine, but row-level copies of personal data in a warehouse are deletion targets, and honouring that shapes what you’re allowed to materialise.
• It’s genuinely at odds with “keep everything forever” instincts — event journals with personal data in the payload now need payload erasure or key-shredding designs of their own (ZFN-12Field Note · currentZFN-12 — Queues, topics, and journals are different tools — don't conflate themQueues (competing consumers), topics (fan-out), and journals (ordered, replayable logs) give different guarantees. Don't conflate them; a pipeline often uses several. Prefer journals over topics, but not where head-of-line blocking hurts. With queues, bound the concurrency.Open ZFN-12 →); “we might need it someday” stops being a free argument.

References

• ZFN-15Field Note · currentZFN-15 — Partition customer data by tenant from day oneMake customer data tenant-partitioned from day one: tenant-scope every query, never join across tenants, route through a tenant→location directory. Run one physical database at first — but keep the model shardable. Retrofitting isolation onto a shared DB is brutal.Why it's cited here: Partitioning by tenant is what makes tenant deletion an operation instead of a query festival — data organised by owner can be dropped by owner.Open ZFN-15 → — owner-partitioned data is what makes erasure droppable rather than searchable.
• ZFN-24Field Note · currentZFN-24 — One transactional store per write; propagate changes asynchronouslyCommit each logical write to exactly one transactional store; update other systems via reliable ordered async events — never a synchronous write across two stores, and never 2PC. With a relational primary the WAL is your replayable journal; write events into the same transaction.Why it's cited here: Deletion rides the same rails as every other write: committed once in the owning store, propagated asynchronously and reliably to every derived copy.Open ZFN-24 → and ZFN-48Field Note · currentZFN-48 — Emit async work into the WAL, not a job tableWhen a DB write should trigger async work, ride the WAL instead of dual-writing or polling a job table. pg_logical_emit_message emits the event transactionally — outbox semantics, no table. A WAL listener consumes it statefully and fans out, keeping load off the primary.Open ZFN-48 → — the reliable propagation rails a tombstone rides.
• ZFN-36Field Note · currentZFN-36 — An untested backup is not a backup — test it by restoringAn untested backup is a hope, not a backup — the only thing that counts is a restore. Rehearse restores regularly (game days), measure and meet your RTO/RPO, automate them, and cover the whole recovery path — data, schema, config, secrets, cutover — not just the dump.Why it's cited here: The same discipline pointed at the opposite guarantee: a deletion you haven't tested by verifying absence is a hope, exactly like an untested backup.Open ZFN-36 → — the sibling discipline: prove the guarantee by exercising it; here, prove absence.
• ZFN-21Field Note · currentZFN-21 — Cache only immutable objects; treat caches as tech debtUse caches sparingly, only for immutable addressed objects — never for mutable DB results, where invalidation bugs and stale reads live; use projections instead. A cache in the data path is usually a patch over an architectural gap that trades correctness for performance.Open ZFN-21 → — every cache you didn’t add is a place deletion doesn’t have to reach.
• ZFN-49Field Note · currentZFN-49 — Verify by computation, not lookup; store revocations, not issuancesVerification should be a computation, not a query: HMACs, signatures, hashes, and pass-by-value claims let any node verify locally. When revocation is rarer than issuance, invert the state — keep the few revocations for the lifetime of what they revoke, not a row per grant.Open ZFN-49 → — store the tombstones, not the corpses.
• GDPR Article 17GDPR Article 17 — Right to erasure ('right to be forgotten')The legal floor under this note in much of the world: data subjects can require erasure of their personal data 'without undue delay', including data that has propagated to processors. A deletion capability is not optional product surface; in many markets it's a compliance requirement with deadlines.gdpr-info.eu ↗ and NIST SP 800-88NIST SP 800-88 Rev. 1 — Guidelines for Media SanitizationThe reference for what 'actually gone' means at the storage layer, including cryptographic erasure: destroying the key that encrypts the data as a sanitisation technique — the mechanism that makes deletion tractable in backups and archives you can't rewrite.csrc.nist.gov ↗ — the legal floor and the storage-layer definition of gone.

Changelog

• 2026-08-12: First published as a Field Note.