Tenant isolation in a multi-tenant system

https://zrz.io/zbp/3-tenant-isolation/v1.md

TL;DR

This specifies how to keep one customer’s data away from another’s in a system that serves many. The shape of the answer: establish the tenant context once, from authenticated material; make the tenant id the partition key of every record; enforce isolation in a layer the application cannot bypass; and verify ownership of every identifier that arrives from outside.

Everything else — cache keys, search indexes, background jobs, error messages, exports — is the same rule applied to a place people forget. This document enumerates those places, because a single forgotten one is the whole breach.

The controlling insight is that isolation must not depend on application code being written correctly every time. Any design whose safety rests on every future query carrying the right WHERE clause has already failed; it is a matter of when.

Applicability

Use this when a system stores or serves data belonging to more than one customer and a cross-tenant disclosure would be a serious incident — which is to say, nearly any B2B SaaS, and any consumer product where “tenant” means “user account.”

Do not use this when:

• You are working out authorisation inside a tenant. Roles, permissions, and record-level sharing among colleagues at the same customer are a different problem. This document is only about the boundary between customers, which is categorically more severe and deserves its own mechanism.
• Each customer has their own deployment. If the isolation boundary is a separate stack, that boundary is infrastructure, and most of this is already answered.
• You are choosing packaging tiers. The isolation tier is an engineering decision about blast radius. It sometimes correlates with what you sell, but “Enterprise” is not an isolation model.

Scope and non-goals

In scope: tenant identity and context propagation; storage layout and the enforcement point; ownership verification at boundaries; caches, search indexes, and derived state; asynchronous work; observability and error surfaces; isolation tiers and blast radius; per-tenant limits; and the testing regime that keeps all of it true over time.

Out of scope: intra-tenant authorisation; authentication and session management (see ZBP-4Blueprint · v1.0.0ZBP-4 — Authentication for a multi-tenant SaaS productYou are building sign-in for a product where people belong to customer organisations, and one person may belong to several.Why it's cited here: The authentication layer that produces the tenant context this blueprint enforces. Kept separate on purpose: establishing who the caller is and constraining what they can reach are different problems with different failure modes.Open ZBP-4 →); encryption at rest and per-tenant key management; data residency and regulatory placement; and billing.

Vocabulary

• Tenant — the unit of isolation: a customer organisation, an account, a workspace. One tenant’s data must never be visible to another.
• Tenant id — the stable, opaque identifier for a tenant.
• Tenant context — the tenant a unit of work is executing for. Established once, propagated, never re-derived.
• Tenant-scoped — a table, record, cache entry, index, or job belonging to exactly one tenant.
• Shared — deliberately visible to all tenants (a currency table, a plan catalogue). Rare, and always explicit.
• Cross-tenant operation — work that legitimately spans tenants (platform analytics, a support action, a migration). Always an explicit, separate, audited path.
• Enforcement point — the layer that actually prevents a cross-tenant read or write, and that application code cannot bypass by writing a different query.
• Isolation tier — how much infrastructure a tenant shares: shared row, shared schema, separate database, separate account.

The key words MUST, MUST NOT, SHOULD, SHOULD NOT, and MAY are to be interpreted as in RFC 2119 and RFC 8174.

Architecture

Two properties do the work, and both are structural rather than behavioural:

1. The context comes from the credential. Not from a path segment, not from a header, not from a body field — even when the value there happens to be correct. The moment a client can influence which tenant it is, every downstream control is decoration.
2. The enforcement point is below the application. The application asks for data; something the application cannot argue with decides what it may see. If the only thing standing between tenants is a WHERE clause a developer must remember, isolation is a code-review property, and code review has a known defect rate.

Normative requirements

Tenant identity and context

• ZBP-3-R1 Every tenant MUST have a stable, opaque identifier. It MUST NOT be sequential or otherwise enumerable, and MUST NOT be a customer-supplied string such as a company name or subdomain, which can change and can collide.
• ZBP-3-R2 The tenant context MUST be established exactly once, at ingress, from authenticated material — the session, token, or certificate that identified the caller.
• ZBP-3-R3 The tenant context MUST NOT be derived from, or influenced by, any client-controlled part of the request: path, query string, body, or header. A tenant id appearing in a request is an input to be verified, never a value to be trusted.
• ZBP-3-R4 The tenant context MUST be propagated explicitly to every layer that touches data, and MUST NOT be recomputed downstream. Recomputation is how two layers come to disagree.
• ZBP-3-R5 A unit of work MUST have exactly one tenant context, or MUST be explicitly marked as a cross-tenant operation. There MUST NOT be a third state.
• ZBP-3-R6 Absence of tenant context MUST deny. A missing context MUST NOT be interpreted as “all tenants”, “no filter”, or a default tenant.
• ZBP-3-R7 Asynchronous work — queued jobs, scheduled tasks, retries, webhooks — MUST carry the tenant context of the work that created it, and MUST execute under it. A job that iterates all tenants MUST be a cross-tenant operation under ZBP-3-R5, running as a named principal.

Storage and the enforcement point

• ZBP-3-R8 The tenant id MUST be the leading column of the primary key of every tenant-scoped table, and the partition key where the store partitions. It is the record’s address, not a column to filter on.
• ZBP-3-R9 Isolation MUST be enforced at a layer that application code cannot bypass by issuing a different query — a row security policy, a per-tenant credential, a per-tenant schema, or a separate database. A WHERE tenant_id = ? written by hand MUST NOT be the only control.
• ZBP-3-R10 The enforcement point MUST fail closed: with no tenant context, a query MUST return zero rows rather than all rows.
• ZBP-3-R11 Writes MUST be constrained as well as reads. It MUST be impossible to insert a record into, or move a record to, a tenant other than the current context.
• ZBP-3-R12 The application’s database role MUST NOT be a superuser and MUST NOT hold a bypass-row-security attribute. Both bypass row policies unconditionally.
• ZBP-3-R13 The application’s database role MUST NOT own the tables it queries, unless row security is explicitly forced for table owners. A table owner is exempt from its own policies by default.
• ZBP-3-R14 Every role that can bypass the enforcement point MUST be enumerated, owned, audited, and unavailable to application code paths.
• ZBP-3-R15 Cross-tenant operations MUST use a distinct, separately credentialed path, MUST be audited per invocation with the reason and the operator, and MUST NOT share a connection pool or credential with tenant-scoped serving.
• ZBP-3-R16 Foreign keys between tenant-scoped tables MUST include the tenant id in the constraint, so a record cannot reference another tenant’s record even if an identifier is known.
• ZBP-3-R17 Migrations, backfills, and repair scripts MUST run tenant-scoped, or MUST be explicitly authorised and audited as cross-tenant operations under ZBP-3-R15.

Verifying ownership at boundaries

• ZBP-3-R18 Any identifier arriving from a client MUST be resolved within the tenant context, so that a record belonging to another tenant is not found rather than fetched and then checked.
• ZBP-3-R19 Object identifiers SHOULD be unguessable, but unguessability MUST NOT be relied upon as an isolation control. Identifiers leak — into URLs, referrers, exports, support tickets.
• ZBP-3-R20 Responses MUST NOT distinguish “does not exist” from “exists but belongs to another tenant”, in status code, error body, or response timing.
• ZBP-3-R21 Data deliberately shared across tenants MUST be modelled as shared explicitly, in its own tables or namespace. A tenant-scoped table MUST NOT be made readable across tenants by exception.

Caches, indexes, and derived state

• ZBP-3-R22 Every cache key for tenant-scoped data MUST include the tenant id, at every layer: in-process, shared cache, HTTP cache, and CDN.
• ZBP-3-R23 A cache entry populated under one tenant context MUST NOT be readable under another, including after a context switch within the same process or connection.
• ZBP-3-R24 Search indexes, materialised views, read models, and analytics stores MUST carry the tenant id and MUST enforce it at query time. A derived store is a second copy of the data and needs the same boundary as the first.
• ZBP-3-R25 Exports, reports, and backups MUST be tenant-scoped, and MUST record which tenant’s data they contain. A restore MUST NOT be able to reintroduce one tenant’s rows into another.
• ZBP-3-R26 Logs, traces, and metrics MUST carry the tenant id as a structured field so that cross-tenant activity is detectable, and MUST NOT contain another tenant’s data in the record for one tenant’s request.
• ZBP-3-R27 Error responses, stack traces, and debug output returned to a client MUST NOT contain identifiers, counts, or data belonging to another tenant.

Isolation tiers and blast radius

• ZBP-3-R28 The isolation tier MUST be a deliberate, documented decision: shared row, shared schema, separate database, or separate account. “Whatever the ORM did” is not a tier.
• ZBP-3-R29 The design MUST allow an individual tenant to be moved to a stronger tier without application code changes, because the requirement to do so arrives from a customer, not a roadmap.
• ZBP-3-R30 Per-tenant resource limits MUST exist for request rate, concurrency, storage, and the cost of a single operation.
• ZBP-3-R31 One tenant MUST NOT be able to exhaust a shared resource to the point of denying service to others. Where a shared resource cannot be fairly divided, it MUST be a documented risk with a mitigation, not an assumption.

Testing and continuous assurance

• ZBP-3-R32 The test suite MUST include a canary tenant, populated with recognisable data, that no test executing as another tenant may ever return.
• ZBP-3-R33 Isolation tests MUST be generated from the schema rather than written per table, so that coverage cannot silently lag behind new tables.
• ZBP-3-R34 Continuous integration MUST fail when a new tenant-scoped table lacks the tenant id in its primary key, lacks an enforcement policy, or is reachable by a role that bypasses enforcement.
• ZBP-3-R35 The absence-of-context case MUST be tested explicitly for every tenant-scoped table, asserting zero rows rather than an error.
• ZBP-3-R36 The application’s runtime database role MUST be asserted, by an automated test against a production-like environment, to lack superuser and bypass attributes.
• ZBP-3-R37 Leak detection MUST run continuously in production — sampling responses, comparing the tenant of returned records against the tenant context — not only in CI.
• ZBP-3-R38 A confirmed cross-tenant disclosure MUST be handled as a security incident of the same severity as a credential compromise, including customer notification obligations.

Wire format and schema

There is no wire protocol here; the interface is the schema and the enforcement policy. The reference form, using PostgreSQL row security, is the one exercised by the test vectors below.

-- Tenant id is the leading column of the primary key (ZBP-3-R8): the record's
-- address, not a filter. Every tenant-scoped table has this shape.
CREATE TABLE invoice (
  tenant_id    uuid   NOT NULL,
  id           uuid   NOT NULL,
  amount_cents bigint NOT NULL,
  PRIMARY KEY (tenant_id, id)
);

-- ENABLE turns the policy on. FORCE also applies it to the table's owner, which
-- ENABLE alone does not (ZBP-3-R13) — see vectors I9 and I10.
ALTER TABLE invoice ENABLE ROW LEVEL SECURITY;
ALTER TABLE invoice FORCE  ROW LEVEL SECURITY;

-- USING governs what is visible; WITH CHECK governs what may be written
-- (ZBP-3-R11). Omitting WITH CHECK leaves writes unconstrained.
-- nullif(...,'') keeps an empty setting from raising instead of denying.
CREATE POLICY tenant_isolation ON invoice
  USING      (tenant_id = nullif(current_setting('app.tenant_id', true), '')::uuid)
  WITH CHECK (tenant_id = nullif(current_setting('app.tenant_id', true), '')::uuid);

A foreign key between tenant-scoped tables carries the tenant id (ZBP-3-R16):

CREATE TABLE invoice_line (
  tenant_id  uuid NOT NULL,
  id         uuid NOT NULL,
  invoice_id uuid NOT NULL,
  PRIMARY KEY (tenant_id, id),
  -- Composite: a line cannot reference another tenant's invoice even if its id is known.
  FOREIGN KEY (tenant_id, invoice_id) REFERENCES invoice (tenant_id, id)
);

The context is set per request by the connection owner, never by handler code:

-- At checkout from the pool, before any application query runs. Parameterised,
-- because a tenant id interpolated into SQL is an injection point.
SELECT set_config('app.tenant_id', $1, true);   -- true = transaction-local

Transaction-local (true) matters: a session-local setting survives the connection’s return to the pool and becomes the next request’s context (ZBP-3-R23).

Test vectors

These vectors were executed against PostgreSQL 18.3. The outputs below are the observed results, not expected ones. Reproduce with the schema above; two tenants:

Isolation under a tenant context

I4 is the vector that distinguishes a real enforcement point from a convention: the query explicitly asks for another tenant’s rows and gets nothing. If your isolation lives in application WHERE clauses, this case returns data, because the developer who wrote this query simply wrote a different one.

I7 is a deliberate design choice worth noticing. Deleting another tenant’s row is not an error — it affects zero rows, exactly as if the row did not exist. An error here would confirm the row exists (ZBP-3-R20).

Who bypasses the enforcement point

This is the set of results most often assumed rather than checked. All five were run with no tenant context set, so the correct answer for a constrained role is 0.

Three of these five see everything. That is the practical content of ZBP-3-R12, ZBP-3-R13, and ZBP-3-R14, and it is why those are requirements rather than advice:

• I9 — enabling row security without forcing it leaves the table’s owner exempt. If the application connects as the role that owns its tables, which is the default in almost every framework’s setup instructions, the policy you wrote does nothing at all. The schema looks correct; a code review passes; the isolation is absent.
• I12 — BYPASSRLS is a role attribute, not a grant, so it does not appear in a table’s permissions and will not show up in a review of who was granted what.
• I13 — superusers are never subject to row security, under any configuration. There is no policy you can write that constrains one.

An implementation that has not asserted its runtime role against I11 has not verified isolation; it has verified that isolation would work if the role were what it assumed (ZBP-3-R36).

The canary

A conforming test suite additionally seeds a canary tenant (ZBP-3-R32) whose records carry a recognisable marker, then asserts that no response produced under any other tenant context contains that marker — across API responses, search results, exports, logs, and cache reads. This catches the leaks that schema-level tests cannot, because they occur above the database: in a cache key (ZBP-3-R22), a search index (ZBP-3-R24), or an error message (ZBP-3-R27).

Threat model and failure modes

The residual risks, stated plainly:

• The enforcement point itself. A bug in the row security implementation, or in whatever plays that role, is a total failure. This is the price of concentrating the control: it is one thing to get right and one thing to lose. Concentrated-and-verified still beats diffuse-and-assumed.
• The cross-tenant path (ZBP-3-R15) is a legitimate, deliberate hole. It should be small, loud, separately credentialed, and audited per invocation. Most cross-tenant breaches in mature systems come through this path, not around the enforcement point — usually via a support tool that grew features.
• Application-layer leaks above the database. Caches, search, exports, and error messages sit above the enforcement point and are not protected by it. They need their own discipline, which is what the canary exists to test.

Anti-patterns

• WHERE tenant_id = ? as the isolation mechanism. Correct in every query written so far. Vector I4 is the demonstration (ZBP-3-R9).
• Connecting as the table owner, or as a superuser. The single highest-impact mistake in this document, and the default in most getting-started guides. Vectors I9 and I13 (ZBP-3-R12, ZBP-3-R13).
• Session-local tenant context with a connection pool. The setting outlives the request and the next request inherits it — a leak that appears only under concurrency, in production (ZBP-3-R23).
• “Ids are UUIDs, so they’re unguessable.” Identifiers leak through URLs, referrers, exports, screenshots, and support tickets. Unguessability is a speed bump (ZBP-3-R19).
• Returning 403 for another tenant’s record and 404 for a missing one. A working enumeration oracle (ZBP-3-R20).
• The admin endpoint that skips the context. It is a cross-tenant operation; give it a separate credential, an audit record, and a reason field (ZBP-3-R15).
• Global caches keyed by object id. Correct while ids are globally unique, catastrophic the moment one is not, and invisible until then (ZBP-3-R22).
• Sharing a tenant-scoped table across tenants “just for this feature”. Model shared data as shared (ZBP-3-R21).
• Testing isolation once, by hand, at launch. New tables arrive weekly. Generate the tests (ZBP-3-R33, ZBP-3-R34).

Rollout

Retrofitting isolation onto a system that filters by hand:

1. Add the tenant id everywhere it is missing, as a nullable column, and backfill. This is the long part and it is pure data work.
2. Make it non-nullable and move it into the primary key (ZBP-3-R8). Composite keys will surface every place that assumed a globally unique id — which is the point, and is best discovered here.
3. Introduce the context plumbing (ZBP-3-R2, ZBP-3-R4): established at ingress, transaction-local, propagated. No behaviour changes yet.
4. Turn on enforcement in report-only mode. Run policies against a shadow role, or compare each query’s result set against the policy-constrained one, and log divergence. Divergence is a list of the places that were relying on the missing filter.
5. Fix the divergences, which are of two kinds: genuine bugs, and legitimate cross-tenant operations that need the explicit path (ZBP-3-R15).
6. Enforce, table by table, highest-value first. Then fix the role: not owner, not superuser, no bypass (ZBP-3-R12, ZBP-3-R13), and assert it in a test (ZBP-3-R36).
7. Add the canary and the generated tests (ZBP-3-R32, ZBP-3-R33) so that step 6 cannot regress.

Step 4 is the one to insist on. It converts an unbounded audit — “find every query that forgot” — into a measured list, and it is the only step that tells you the size of the problem before you commit to the schedule.

Conformance checklist

Context

• Tenant ids are opaque and non-enumerable (ZBP-3-R1)
• Context established once at ingress from authenticated material (ZBP-3-R2)
• No client-controlled value influences the context (ZBP-3-R3)
• Context propagated explicitly, never recomputed (ZBP-3-R4)
• Exactly one context, or an explicit cross-tenant marking (ZBP-3-R5)
• Missing context denies (ZBP-3-R6)
• Async work carries and executes under its originating context (ZBP-3-R7)

Storage and enforcement

• Tenant id leads the primary key of every tenant-scoped table (ZBP-3-R8)
• Enforcement is below the application and unbypassable by query (ZBP-3-R9)
• No context returns zero rows, not all rows (ZBP-3-R10)
• Writes constrained as well as reads (ZBP-3-R11)
• Runtime role is not a superuser and has no bypass attribute (ZBP-3-R12)
• Runtime role does not own its tables, or row security is forced (ZBP-3-R13)
• All bypass-capable roles enumerated, owned, and audited (ZBP-3-R14)
• Cross-tenant path is separate, separately credentialed, and audited (ZBP-3-R15)
• Foreign keys include the tenant id (ZBP-3-R16)
• Migrations and backfills are scoped or explicitly authorised (ZBP-3-R17)

Boundaries

• Client identifiers resolved within the context (ZBP-3-R18)
• Unguessability not relied on (ZBP-3-R19)
• Missing and forbidden are indistinguishable, including in timing (ZBP-3-R20)
• Shared data modelled explicitly as shared (ZBP-3-R21)

Derived state

• Tenant id in every cache key at every layer (ZBP-3-R22)
• No cache entry survives a context switch (ZBP-3-R23)
• Search, read models, and analytics carry and enforce the tenant id (ZBP-3-R24)
• Exports and backups scoped and labelled (ZBP-3-R25)
• Telemetry carries the tenant id and no other tenant’s data (ZBP-3-R26)
• Client-visible errors leak nothing cross-tenant (ZBP-3-R27)

Tiers and limits

• Isolation tier chosen and documented (ZBP-3-R28)
• A tenant can be promoted to a stronger tier without code changes (ZBP-3-R29)
• Per-tenant limits on rate, concurrency, storage, and operation cost (ZBP-3-R30)
• No tenant can starve another of a shared resource (ZBP-3-R31)

Assurance

• Canary tenant exists and is asserted against every response surface (ZBP-3-R32)
• Isolation tests generated from the schema (ZBP-3-R33)
• CI fails on a non-conforming new table (ZBP-3-R34)
• Absence-of-context asserted per table, expecting zero rows (ZBP-3-R35)
• Runtime role attributes asserted automatically (ZBP-3-R36)
• Continuous production leak detection (ZBP-3-R37)
• Confirmed leak handled as a top-severity security incident (ZBP-3-R38)

Test suite

• I1–I8 reproduce against the real store
• I9–I13 reproduced, and the runtime role’s row asserted to be I11
• The canary is asserted above the database, not only in SQL

Rationale

Partitioning customer data by tenant rather than filtering it is ZFN-15Field Note · currentZFN-15 — Partition customer data by tenant from day oneMake customer data tenant-partitioned from day one: tenant-scope every query, never join across tenants, route through a tenant→location directory. Run one physical database at first — but keep the model shardable. Retrofitting isolation onto a shared DB is brutal.Why it's cited here: The position this blueprint implements. That note argues for partitioning rather than filtering; this specifies the schema, the enforcement point, and how to prove it holds.Open ZFN-15 →. That authority to reach a resource is not proof it is the right resource — the argument behind verifying ownership at every boundary — is ZFN-10Field Note · currentZFN-10 — Pin the expected owner on cross-account resource calls (confused-deputy defense)Authority to call a resource isn't proof it's the one you meant. Any call crossing an account boundary must assert the expected owner: ExpectedBucketOwner on S3, aws:ResourceAccount conditions, validation of untrusted ARNs, plus inbound trust pinned with SourceArn/ExternalId.Why it's cited here: The argument behind ZBP-3-R18: an identifier from a client is resolved within the tenant context, never fetched and then checked.Open ZFN-10 →. Per-tenant limits at the edge are ZFN-18Field Note · currentZFN-18 — Enforce a quota at ingress on every endpoint — even unabused onesPut a quota on every endpoint and enforce it at ingress from day one — per tenant, principal, IP — even for endpoints nobody abuses yet. Unlimited-by-default means the first runaway client or compromised key is an outage. Return 429 + Retry-After; retrofitting limits is painful.Why it's cited here: Behind the per-tenant limits in ZBP-3-R30 and the noisy-neighbour requirement that follows it.Open ZFN-18 →; the caution about caches as a second, quieter copy of your data is ZFN-21Field Note · currentZFN-21 — Cache only immutable objects; treat caches as tech debtUse caches sparingly, only for immutable addressed objects — never for mutable DB results, where invalidation bugs and stale reads live; use projections instead. A cache in the data path is usually a patch over an architectural gap that trades correctness for performance.Why it's cited here: Cited because a cache is a second copy of tenant data sitting above the enforcement point, which is why every cache key must carry the tenant id.Open ZFN-21 →; and the rule that no automated actor is anonymous — which is why a background job has a tenant and a name rather than running as the system — is ZFN-40Field Note · currentZFN-40 — No anonymous "system" actorIf "system" appears as an actor in your audit log, attribution is already broken. Every automated action — cron job, cleanup task, migration, agent — runs as a named identity with its own credentials and scope, so "who did this?" has an answer and revocation is surgical.Why it's cited here: Why a background job carries a tenant context and a named principal instead of running as the system across all tenants.Open ZFN-40 →.

The position specific to this document: isolation must be structural, not behavioural. Every alternative reduces to “the code will be written correctly each time,” and over a long enough period with a large enough team that is simply false. The value of moving enforcement below the application is not that it catches today’s bug; it is that it makes an entire category of future bug impossible to write, including by people who have never read this document. That is also why the requirements about which database role you connect as are stated as forcefully as they are: they are the difference between a policy that enforces and a policy that merely exists, and vectors I9 and I13 show that the failure is completely silent.

Changelog

• 2026-07-27 (1.0.0): First published. Specifies tenant identity and context propagation, storage layout and the enforcement point, ownership verification at boundaries, derived-state requirements, isolation tiers and per-tenant limits, the assurance regime, and test vectors I1–I13 executed against PostgreSQL 18.3.