# zrz.io

> Personal site of Theo Zourzouvillys. Two machine-readable collections are published here for
> import: **Blueprints** (ZBP-N) — versioned, normative implementation specifications written to
> be pulled into a build and implemented directly; and **Engineering Field Notes** (ZFN-N) —
> short, opinionated positions on how to build software well.

A Blueprint specifies; a Field Note argues. If you are building something, you want a Blueprint,
and its `Use when` line tells you whether it applies before you spend the context on it. Each
blueprint is self-contained: every normative requirement is stated in the document, not behind a
link. Requirements are individually addressable as ZBP-N-Rk, so an implementation can cite them
and a review can check them one at a time.

## Bundles

Several blueprints behind one URL, in dependency order. If you are building a whole subsystem,
start here — a bundle already includes everything its parts require.

- [Multi-tenant SaaS: identity and isolation](https://zrz.io/zbp/bundle/multi-tenant-saas.md)
  - Everything needed to stand up authentication and tenant isolation for a B2B SaaS product: the isolation boundary, the identity and membership model on top of it, and sender-constrained credentials for its machine callers.
  - Includes: ZBP-1 → ZBP-3 → ZBP-4
- [Workload identity: an STS and the credentials it issues](https://zrz.io/zbp/bundle/workload-identity.md)
  - Service-to-service authentication with no long-lived secrets: a token service bootstrapped from platform attestation, issuing short-lived tokens bound to a holder key.
  - Includes: ZBP-1 → ZBP-2

## Blueprints

- [ZBP-0: Blueprints](https://zrz.io/zbp/0-blueprints/v1.md) (v1.0.0, stable)
  - Use when: You are writing a Blueprint, or you want to understand the contract a Blueprint offers before pulling one into a build.
  - Not for: Deciding whether a practice is a good idea — that argument lives in a Field Note, not here.
  - Not for: Learning a technology from scratch; a blueprint assumes you know the domain and want the specification.
  - Provides: blueprint-format
- [ZBP-1: DPoP-bound requests across an API surface](https://zrz.io/zbp/1-dpop-bound-requests/v1.md) (v1.0.0, stable)
  - Use when: You are issuing or accepting access tokens over HTTP and want a stolen token to be useless without the holder's private key.
  - Not for: Service-to-service calls where mutual TLS is already terminated end-to-end — mTLS-bound tokens (RFC 8705) are a better fit.
  - Not for: Protecting the request body's integrity end-to-end; DPoP binds the request line, not the payload.
  - Not for: Authorising *what* a principal may do; this is proof-of-possession, not authorisation.
  - Provides: dpop-bound-requests, sender-constrained-tokens
- [ZBP-2: A Security Token Service for workload identity](https://zrz.io/zbp/2-workload-security-token-service/v1.md) (v1.0.0, stable)
  - Use when: Services in your platform need to authenticate to each other and you want every call attributable to a named workload holding no long-lived secret.
  - Not for: End-user authentication, sessions, or consent — this issues machine credentials, not user logins.
  - Not for: Authorisation policy itself; the STS decides what a workload may be *issued*, not what it may *do* once inside a service.
  - Not for: A platform with one runtime and one service, where the token exchange costs more than it returns.
  - Requires: ZBP-1
  - Provides: workload-identity, token-exchange, short-lived-service-credentials
- [ZBP-3: Tenant isolation in a multi-tenant system](https://zrz.io/zbp/3-tenant-isolation/v1.md) (v1.0.0, stable)
  - Use when: One system stores or serves data belonging to more than one customer, and one customer seeing another's data would be a serious incident.
  - Not for: Authorisation within a tenant — who may see what once you know the tenant is right is a separate problem.
  - Not for: Single-tenant deployments, where the isolation boundary is the deployment itself.
  - Not for: Deciding your pricing or packaging tiers; the isolation tier is an engineering choice, not a plan name.
  - Provides: tenant-isolation, tenant-context
- [ZBP-4: Authentication for a multi-tenant SaaS product](https://zrz.io/zbp/4-multi-tenant-saas-authentication/v1.md) (v1.0.0, stable)
  - Use when: You are building sign-in for a product where people belong to customer organisations, and one person may belong to several.
  - Not for: Service-to-service authentication — workloads should get tokens from an STS, not sign in.
  - Not for: Authorisation inside an organisation; this establishes who the caller is and which org they act in, not what they may do to a given record.
  - Not for: Consumer products with no organisation concept, where the org model is pure overhead.
  - Requires: ZBP-1, ZBP-3
  - Provides: saas-authentication, organization-membership, session-management

## Interfaces

- [blueprints.json](https://zrz.io/zbp/blueprints.json): every blueprint with version, requirement ids, content hash, and pinned URL
- [notes.json](https://zrz.io/zfn/notes.json): every field note with status, relations, and raw markdown URL
- Raw markdown: `/zbp/<slug>/vN.md` (pinned), `/zbp/<slug>.md` (latest), `/zfn/<slug>.md`

## Engineering Field Notes

50 notes. Positions and reasoning, not specifications — read one when you want to know *why*.

- [ZFN-49: Verify by computation, not lookup; store revocations, not issuances](https://zrz.io/zfn/49-verify-by-computation-not-lookup.md) — Verification should be a computation, not a query: HMACs, signatures, hashes, and pass-by-value claims let any node verify locally. When revocation is rarer than issuance, invert the state — keep the few revocations for the lifetime of what they revoke, not a row per grant.
- [ZFN-48: Emit async work into the WAL, not a job table](https://zrz.io/zfn/48-emit-async-work-into-the-wal.md) — When a DB write should trigger async work, ride the WAL instead of dual-writing or polling a job table. pg_logical_emit_message emits the event transactionally — outbox semantics, no table. A WAL listener consumes it statefully and fans out, keeping load off the primary.
- [ZFN-47: Govern the contract between teams, not the code inside them](https://zrz.io/zfn/47-govern-the-contract-between-teams.md) — Teams own services end to end; one team owns the gateway that dispatches to them. Govern exactly one thing centrally — the contract at the boundary (schema, identity, errors, idempotency) — and enforce it at runtime. Don't mandate libraries; ship them as an opt-in blueprint.
- [ZFN-46: Extract the reusable component from the product — and open-source it](https://zrz.io/zfn/46-extract-and-open-source-components.md) — Product work keeps producing general infrastructure — a sync engine, a bus bridge. When it isn't your moat, pull it out behind a clean, zero-opinion boundary; better yet, open-source it. Extraction forces honest design; don't do it before it's earned its generality.
- [ZFN-45: Read the standards; better yet, help write them](https://zrz.io/zfn/45-read-the-standards.md) — Learn to read standards docs — RFCs, W3C recs — fluently; they're the primary source, not a last resort. Even better, get involved: reading them well makes you a sharper builder, and helping write them is the best protocol education there is.
- [ZFN-44: Burnout is a conditions problem, not a workload problem](https://zrz.io/zfn/44-burnout-is-conditions-not-workload.md) — Burnout was never about the hours. I've woken up excited on 100-hour weeks and drowned in thirty-hour ones; the variable is the conditions, not the workload. The red flags I watch for, the six pillars that make intensity sustainable, and the levers I pull at the first tremor.
- [ZFN-43: Name your systems as archetypes, not machinery](https://zrz.io/zfn/43-name-systems-as-archetypes.md) — Names shape thought: a good name compresses a system's purpose, boundaries, and temperament into a word. Name the long-lived cast as archetypes — a Sentinel, a Seer — not Rule Engine and Decision Service. Bureaucratic names flatten judgment; a character must live up to its role.
- [ZFN-42: My one cloud is AWS](https://zrz.io/zfn/42-my-one-cloud-is-aws.md) — Applying the one-cloud principle (ZFN-32), my pick is AWS — 100%, a league of its own. Go native: skip Kubernetes, use ECS; lean into SQS, SNS, Kinesis, IAM, ALB, RDS. The one exception to native — provision with OpenTofu, not CloudFormation, now that LLMs write .tf so well.
- [ZFN-41: Make the technical choice you can put your name behind](https://zrz.io/zfn/41-choices-you-can-put-your-name-behind.md) — When two options would both work, it's fine to choose the one your experience lets you put your name behind, and to resist pressure that isn't about the merits — vendor incentives, politics, top-down preference. Owning the call costs capital up front; being right repays it.
- [ZFN-40: No anonymous "system" actor](https://zrz.io/zfn/40-no-anonymous-system-actor.md) — If "system" appears as an actor in your audit log, attribution is already broken. Every automated action — cron job, cleanup task, migration, agent — runs as a named identity with its own credentials and scope, so "who did this?" has an answer and revocation is surgical.
- [ZFN-39: Break loops, not spirals](https://zrz.io/zfn/39-break-loops-not-spirals.md) — Event-driven and agentic systems echo. A loop — a lineage recurring with the same data — produces nothing new; detect it with runtime-stamped provenance and break it loudly. A spiral — recurring with new data — is legitimate work; bound it with budgets, never loop-breakers.
- [ZFN-38: Agents are principals: delegate, never impersonate](https://zrz.io/zfn/38-agents-are-principals.md) — An agent acting with a copied user credential is impersonation — untraceable by design. Give agents their own identities and keys; let them act for a human only through explicit, scoped, time-bounded, revocable delegation; and record both actor and principal on every action.
- [ZFN-37: Every lock is a lease](https://zrz.io/zfn/37-every-lock-is-a-lease.md) — A lock that can outlive its holder is a deadlock scheduled for later. Give every lock — including informal ones like claimed_by columns — a TTL, a named owner, and a heartbeat; make expiry automatic and server-side; fence the side effects so a stale holder can't corrupt anything.
- [ZFN-36: An untested backup is not a backup — test it by restoring](https://zrz.io/zfn/36-test-backups-by-restoring.md) — An untested backup is a hope, not a backup — the only thing that counts is a restore. Rehearse restores regularly (game days), measure and meet your RTO/RPO, automate them, and cover the whole recovery path — data, schema, config, secrets, cutover — not just the dump.
- [ZFN-35: Reference secrets in config; dereference, refresh, and re-fetch](https://zrz.io/zfn/35-dereference-secrets-not-store-in-config.md) — Don't put secret values in config — store a reference (a path in a secret store) and dereference it at runtime via your workload identity. Refresh on a signal or expiry so rotation needs no redeploy; re-fetch on auth failure so a rotated secret self-heals.
- [ZFN-34: A resource-free 'bouncer' account: the single gateway to customer resources](https://zrz.io/zfn/34-resource-free-bouncer-account.md) — Funnel access to customer resources through one dedicated account that holds no resources. Customer trust names only its role; the role is denied from your own org (aws:ResourceOrgID). Fenced both ways, it shrinks the confused-deputy surface to one audited gateway.
- [ZFN-33: Are LLMs swinging us away from prepackaged services?](https://zrz.io/zfn/33-moving-away-from-prepackaged-services.md) — A signal: LLMs may be swinging build-vs-buy away from prepackaged services. A generic complex service used to beat building until real scale; now building what you need is cheap. Maybe open source becomes shared architectures an LLM implements per user — like C replaced ASM.
- [ZFN-32: Commit to one cloud, and go all-in native](https://zrz.io/zfn/32-commit-to-one-cloud.md) — Cloud-independence is a false benefit: portability is the least common denominator, costing you the native services that are the point. For end products you run: commit to one cloud, go all-in native. Libraries, software others run, LLM-embeddables, and edge code are exempt.
- [ZFN-31: Own your components — when you deeply understand the domain](https://zrz.io/zfn/31-own-your-components.md) — Owning your own components rather than generic off-the-shelf services is often the better path as you grow: own what's core, lean on small vetted libraries for the hard parts. LLMs make it attainable at smaller scale — but only when you truly understand the domain, or it hurts.
- [ZFN-30: Use the standard; don't reinvent the protocol](https://zrz.io/zfn/30-use-standards-dont-reinvent.md) — When a standard exists for a common or complex problem, use it — don't reinvent the protocol. Standards encode huge adversarial expertise, especially in auth and crypto; a partial implementation beats rolling your own. You're not that special, and your problem isn't either.
- [ZFN-29: Blameless culture, taken seriously — and its one hard line](https://zrz.io/zfn/29-blameless-culture-with-a-hard-line.md) — When something breaks, support the person, don't blame. Run post-mortems with ceremony and learn at every level — software, org, culture, process, even solo. Blameless protects honest mistakes, not dishonesty: evading or blaming gets coached; hiding evidence is a firing offense.
- [ZFN-28: Capability without understanding: brute-force LLM PRs](https://zrz.io/zfn/28-llm-brute-force-prs-understanding.md) — An open problem: people brute-force PRs with LLMs in domains they don't understand, taking on more than their knowledge supports — and the struggle that used to teach them is smoothed away. How do we stop un-understood code without killing learning or banning a good tool?
- [ZFN-27: Don't tolerate assholes — but be strict about what one is](https://zrz.io/zfn/27-no-assholes-strict-definition.md) — Don't tolerate assholes — people who demean, belittle, punch down. But filter hard on the word: disagreeing, raising ideas, or opening a competing PR isn't being an asshole, it's the work. Assholes attack people; colleagues attack problems. Don't let the label silence dissent.
- [ZFN-26: AI-assisted content needs no disclaimer, only a human who can back it](https://zrz.io/zfn/26-ai-assisted-content-cosign.md) — Using an LLM to draft engineering content — chat, commits, PRs, docs, comments — is fine and needs no disclaimer. The obligation is human co-signing: every word under your name is one you drove, reviewed, and can defend. Disclose when the ideas are the model's, not yours.
- [ZFN-25: Track the version a client has seen for read-your-writes](https://zrz.io/zfn/25-read-your-writes-version-token.md) — For read-your-writes across backends, track the latest version a client has seen — a token or vector clock. Return it on write; reads then go to a backend at least that fresh. Hold it client-side (a token they present) or server-side (a gateway tracks the session and routes).
- [ZFN-24: One transactional store per write; propagate changes asynchronously](https://zrz.io/zfn/24-one-transactional-store-per-write.md) — Commit each logical write to exactly one transactional store; update other systems via reliable ordered async events — never a synchronous write across two stores, and never 2PC. With a relational primary the WAL is your replayable journal; write events into the same transaction.
- [ZFN-23: Rewriting an implementation is fine — refactoring isn't always the answer](https://zrz.io/zfn/23-iterate-and-rewrite-implementations.md) — Refactoring isn't always right. When the structure is wrong at the root, it's fine — often better — to rewrite an implementation from scratch. Clean interfaces and data models make the implementation disposable: stable contract, swappable internals. LLMs make it cheaper still.
- [ZFN-22: Quarantine bad architecture behind an interface, then replace it](https://zrz.io/zfn/22-extract-complexity-at-the-seam.md) — When a subsystem is complex and badly architected, quarantine it at its seam: write a clean adapter interface over the mess so the rest of the system depends on the contract, then build a better implementation behind it and expose the new interface directly.
- [ZFN-21: Cache only immutable objects; treat caches as tech debt](https://zrz.io/zfn/21-caches-sparingly-immutable-only.md) — Use caches sparingly, only for immutable addressed objects — never for mutable DB results, where invalidation bugs and stale reads live; use projections instead. A cache in the data path is usually a patch over an architectural gap that trades correctness for performance.
- [ZFN-20: The simplest-looking system is often the most complex to live with](https://zrz.io/zfn/20-deliberate-complexity-is-often-simpler.md) — The system that's simplest to stand up often isn't simplest to live with — it skips the correctness edge cases, so bugs and inconsistency surface fast. A more deliberate design has more parts but fewer surprises, and is often the simpler one over time.
- [ZFN-19: Annotate read-only and idempotent endpoints; make every mutation idempotent](https://zrz.io/zfn/19-annotate-readonly-idempotent-endpoints.md) — Annotate every endpoint as read-only (safe) or idempotent, in the schema, so infrastructure can retry, route to replicas, and cache safely. Make every state-changing endpoint idempotent (idempotency keys for create/charge/send); a non-idempotent retry double-applies.
- [ZFN-18: Enforce a quota at ingress on every endpoint — even unabused ones](https://zrz.io/zfn/18-enforce-quotas-at-ingress.md) — Put a quota on every endpoint and enforce it at ingress from day one — per tenant, principal, IP — even for endpoints nobody abuses yet. Unlimited-by-default means the first runaway client or compromised key is an outage. Return 429 + Retry-After; retrofitting limits is painful.
- [ZFN-17: Separate configuration, state, and ephemeral data](https://zrz.io/zfn/17-separate-config-state-ephemeral.md) — Customer data splits into mostly-static config, durable state, and ephemeral sessions — different access, durability, and change rates. Model and store each separately. For bounded static config, prefer loading one validated snapshot held in memory over fetching on demand.
- [ZFN-16: Separate the data plane from the control plane](https://zrz.io/zfn/16-separate-data-plane-control-plane.md) — Split the serving path (data plane) from the management path (control plane). The data plane keeps serving on last-known-good config when the control plane is down — never call it on the hot path. Coupling them turns a control-plane bug into a serving outage.
- [ZFN-15: Partition customer data by tenant from day one](https://zrz.io/zfn/15-partition-customer-data-by-tenant.md) — Make customer data tenant-partitioned from day one: tenant-scope every query, never join across tenants, route through a tenant→location directory. Run one physical database at first — but keep the model shardable. Retrofitting isolation onto a shared DB is brutal.
- [ZFN-14: Define every API with a schema, and generate the clients](https://zrz.io/zfn/14-schema-first-apis-generate-clients.md) — Define every API with a machine-readable schema (OpenAPI, Protobuf, GraphQL) as the source of truth, and generate clients and server stubs from it — never hand-roll request-building and JSON parsing. Hand-written clients drift and break silently; check schema compatibility in CI.
- [ZFN-13: Fail fast and push back: retries, load shedding, and flow control](https://zrz.io/zfn/13-load-shedding-and-flow-control.md) — Build client retries (backoff, jitter, Retry-After) from day one. Under overload, shed fast and push the failure back to the source to retry — don't retry internally and amplify it. Flow-control everywhere, bound every queue, and don't take more work than you can finish in time.
- [ZFN-12: Queues, topics, and journals are different tools — don't conflate them](https://zrz.io/zfn/12-queues-topics-journals.md) — Queues (competing consumers), topics (fan-out), and journals (ordered, replayable logs) give different guarantees. Don't conflate them; a pipeline often uses several. Prefer journals over topics, but not where head-of-line blocking hurts. With queues, bound the concurrency.
- [ZFN-11: Route outbound HTTP through an isolated egress proxy](https://zrz.io/zfn/11-outbound-http-egress-proxy.md) — Application compute shouldn't make arbitrary outbound HTTP — it's an SSRF pivot to internal services and the cloud metadata endpoint. Route all egress through a proxy (SOCKS, or a gRPC egress service) on isolated compute with no route inward. The proxy's network is the boundary.
- [ZFN-10: Pin the expected owner on cross-account resource calls (confused-deputy defense)](https://zrz.io/zfn/10-verify-resource-owner.md) — Authority to call a resource isn't proof it's the one you meant. Any call crossing an account boundary must assert the expected owner: ExpectedBucketOwner on S3, aws:ResourceAccount conditions, validation of untrusted ARNs, plus inbound trust pinned with SourceArn/ExternalId.
- [ZFN-9: No long-lived cloud keys; workloads authenticate by federated identity](https://zrz.io/zfn/9-no-long-lived-cloud-keys.md) — No static AWS or GCP keys anywhere — not in code, secret stores, or env. Workloads use their runtime's own identity and cross clouds by exchanging it (OIDC) for short-lived credentials via federation. Static keys are a documented carve-out only.
- [ZFN-8: Don't hide behind anonymous 'people'](https://zrz.io/zfn/8-dont-speak-for-anonymous-people.md) — Never invoke unnamed 'people' to carry weight — 'a few people are concerned', 'some think'. It launders one view as phantom consensus and makes the listener argue a crowd they can't see. Name them and bring them in, or own it. If they can't speak up, fix the culture.
- [ZFN-7: Sign the message, not just the session (HTTP Message Signatures)](https://zrz.io/zfn/7-sign-the-message.md) — A bearer token proves nothing about the request it rides on. Sign the message itself (HTTP Message Signatures, RFC 9421) — request, and ideally response — so the recipient can prove who sent this exact message and not a byte changed. Shared keys first; asymmetric better.
- [ZFN-6: Bind tokens to a key: sender-constrained tokens (DPoP)](https://zrz.io/zfn/6-sender-constrained-tokens-dpop.md) — A bearer token grants access to whoever holds it — steal it, replay it. Bind the token to a holder key (DPoP, RFC 9449) so using it requires proving possession of a private key the token names. A stolen token alone becomes useless.
- [ZFN-5: Make workload identity a platform-owned service](https://zrz.io/zfn/5-platform-workload-identity-service.md) — Workload identity belongs in shared platform infrastructure, not reimplemented per service. A small token service mints short-lived tokens any service verifies. Shared keys are a fine first step; asymmetric signing the better end-state — don't let 'no PKI' block it.
- [ZFN-4: Incident tooling must not depend on what it recovers](https://zrz.io/zfn/4-incident-tooling-independence.md) — Anything you need to respond to an incident — deploy/rollback, kill switches, observability, break-glass access — must not depend, directly or transitively, on the systems likely to be down during it. Never gate incident tooling behind a system it might need to recover.
- [ZFN-3: Default-encrypt internal service traffic](https://zrz.io/zfn/3-default-encrypt-internal-traffic.md) — All external traffic is TLS, no exceptions. Internal traffic is encrypted by default; an internal call site may skip transport encryption (never authentication) only under a documented, audited carve-out anchored to a network-perimeter guarantee.
- [ZFN-2: Engineering priority ordering](https://zrz.io/zfn/2-engineering-priority-ordering.md) — When concerns conflict, prioritize security > correctness > availability > performance — and never trade a higher-ranked concern for a lower one. The rule binds the moment you must choose. Cite it instead of re-arguing it.
- [ZFN-1: Keep engineering decision records](https://zrz.io/zfn/1-engineering-decision-records.md) — Record significant engineering decisions as short, versioned markdown files — context, decision, consequences. Write one for cross-team contracts, directional principles, hard-to-reverse choices, and conventions others must follow. Cite them instead of re-arguing.
- [ZFN-0: Engineering Field Notes](https://zrz.io/zfn/0-engineering-field-notes.md) — What Field Notes are, why I write them, and how they work — numbered, status-tracked notes on building software well, published openly and importable as JSON and raw markdown.