Design the request envelope before the first endpoint

Context

Start a new API and the first cross-cutting concern arrives immediately: authentication. That one gets handled properly, because it arrives before there’s anything to bolt it onto — it goes in a header, the client library attaches it, middleware validates it, handlers never see it. Nobody has ever proposed putting the access token in the request body.

Then the rest arrive one at a time, each in its own quarter, each solved on its own terms:

• Idempotency keys show up when the first retry double-charges someone. They get a header — but only on the three endpoints that hurt, and the “generate a fresh key per attempt” bug lives in two of the four SDKs (ZFN-19Field Note · currentZFN-19 — Annotate read-only and idempotent endpoints; make every mutation idempotentAnnotate every endpoint as read-only (safe) or idempotent, in the schema, so infrastructure can retry, route to replicas, and cache safely. Make every state-changing endpoint idempotent (idempotency keys for create/charge/send); a non-idempotent retry double-applies.Open ZFN-19 →, ZFN-13Field Note · currentZFN-13 — Fail fast and push back: retries, load shedding, and flow controlBuild client retries (backoff, jitter, Retry-After) from day one. Under overload, shed fast and push the failure back to the source to retry — don't retry internally and amplify it. Flow-control everywhere, bound every queue, and don't take more work than you can finish in time.Open ZFN-13 →).
• Trace ids show up when an incident takes four hours to attribute. Whatever the tracing agent auto-instruments becomes the de facto contract, which means the trace survives HTTP hops and dies at every queue and every hand-rolled client.
• Vector clocks show up when a customer reports the “I saved it and it’s gone” bug (ZFN-25Field Note · currentZFN-25 — Track the version a client has seen for read-your-writesFor read-your-writes across backends, track the latest version a client has seen — a token or vector clock. Return it on write; reads then go to a backend at least that fresh. Hold it client-side (a token they present) or server-side (a gateway tracks the session and routes).Open ZFN-25 →). And this is where it goes wrong, because by now the envelope has no owner and no obvious place for a new field — so the version token goes into the request body, since the payload is the only part of the contract the team building the feature controls.

That last decision is the expensive one, and it’s worth being precise about why. Once the vector clock is a field on CreateOrderRequest, it must also be on UpdateOrderRequest, and ListOrders, and every message type added afterwards, forever. It is now versioned with your domain model, so changing its shape is a schema migration across every message. It can’t be read by anything that doesn’t have your schema — no ingress routing on it, no gateway holding the session’s clock (ZFN-16Field Note · currentZFN-16 — Separate the data plane from the control planeSplit the serving path (data plane) from the management path (control plane). The data plane keeps serving on last-known-good config when the control plane is down — never call it on the hot path. Coupling them turns a control-plane bug into a serving outage.Open ZFN-16 →). It doesn’t survive a hop into a queue or a WAL record (ZFN-48Field Note · currentZFN-48 — Emit async work into the WAL, not a job tableWhen a DB write should trigger async work, ride the WAL instead of dual-writing or polling a job table. pg_logical_emit_message emits the event transactionally — outbox semantics, no table. A WAL listener consumes it statefully and fans out, keeping load off the primary.Open ZFN-48 →, ZFN-12Field Note · currentZFN-12 — Queues, topics, and journals are different tools — don't conflate themQueues (competing consumers), topics (fan-out), and journals (ordered, replayable logs) give different guarantees. Don't conflate them; a pipeline often uses several. Prefer journals over topics, but not where head-of-line blocking hurts. With queues, bound the concurrency.Open ZFN-12 →) unless someone remembers to copy it by hand into a different shape. And the business logic now touches it, because it arrived in the same object as the domain data.

The failure mode isn’t that it doesn’t work. It’s that it works on most paths. Some endpoint added in a hurry omits the field; some async hop drops it; some SDK never learned to echo it back. Read-your-writes then holds usually, which is precisely the intermittent, unreproducible bug the vector clock existed to eliminate. Partial propagation is worse than no propagation, because no propagation is at least honest about what it doesn’t give you.

Recommendation

Define the envelope as a first-class part of the API contract, before the first endpoint exists — and route every cross-cutting concern into it rather than into the payload.

Separate the three layers, explicitly

• Payload — the domain object. What the operation is about. Owned by the team that owns the endpoint.
• Envelope — metadata about this call, this hop: credential and proof of possession, idempotency key, deadline/timeout budget, quota key, content negotiation, client identity and version. Consumed by infrastructure and middleware; handlers rarely read it directly.
• Baggage — context that must survive the whole causal chain, including async hops, and that intermediate services carry without needing to understand: trace contextW3C Trace Context (W3C Recommendation)Standardises `traceparent` and `tracestate` as the interoperable way to propagate a trace identifier and vendor-specific trace state across service boundaries, so a request can be correlated end to end even when hops are instrumented by different vendors.w3.org ↗, the vector clock, the originating principal, the tenant, a sampling or debug flag.

The envelope/baggage split is the one people skip, and it’s the one that decides whether a value survives the first queue. Write it down per field, in the contract: per-hop or propagated, and whether it’s client-settable or server-only. Getting it backwards produces real bugs in both directions — a propagated idempotency key makes two genuinely different downstream writes collide on one key, and a per-hop trace id makes every async boundary the end of a trace.

Put it in the schema, and let the transport carry it

If the API is defined by a schema and the clients are generated (ZFN-14Field Note · currentZFN-14 — Define every API with a schema, and generate the clientsDefine every API with a machine-readable schema (OpenAPI, Protobuf, GraphQL) as the source of truth, and generate clients and server stubs from it — never hand-roll request-building and JSON parsing. Hand-written clients drift and break silently; check schema compatibility in CI.Open ZFN-14 →), the envelope is defined there, as a type distinct from any request message — so it lands in every generated client and every server stub without anyone remembering it. On the wire it rides the metadata layer the transport already has: HTTP headers, gRPC metadata, message attributes on a queue, a header block on a WAL record. Every mainstream transport has one; that is what it is for.

This matters beyond tidiness, because the envelope is the part intermediaries need. Ingress has to rate-limit and reject on it without parsing a domain object (ZFN-18Field Note · currentZFN-18 — Enforce a quota at ingress on every endpoint — even unabused onesPut a quota on every endpoint and enforce it at ingress from day one — per tenant, principal, IP — even for endpoints nobody abuses yet. Unlimited-by-default means the first runaway client or compromised key is an outage. Return 429 + Retry-After; retrofitting limits is painful.Open ZFN-18 →); a gateway routing reads to a caught-up replica needs the version token before it knows which service it’s calling (ZFN-25Field Note · currentZFN-25 — Track the version a client has seen for read-your-writesFor read-your-writes across backends, track the latest version a client has seen — a token or vector clock. Return it on write; reads then go to a backend at least that fresh. Hold it client-side (a token they present) or server-side (a gateway tracks the session and routes).Open ZFN-25 →); a proxy has to forward baggage for services whose schemas it has never seen. Anything in the body is invisible to all of them.

Make it the SDK’s job and the middleware’s job

Application code should not mention any of this. The generated client:

• attaches the credential and mints a fresh DPoP proofRFC 9449 — OAuth 2.0 Demonstrating Proof of Possession (DPoP)Defines a mechanism to bind an access token to a client-held key: the client sends a signed DPoP proof header on each request, so a stolen token cannot be replayed by a party that doesn't hold the corresponding private key.rfc-editor.org ↗ per request (ZFN-6Field Note · currentZFN-6 — Bind tokens to a key: sender-constrained tokens (DPoP)A bearer token grants access to whoever holds it — steal it, replay it. Bind the token to a holder key (DPoP, RFC 9449) so using it requires proving possession of a private key the token names. A stolen token alone becomes useless.Open ZFN-6 →);
• generates one idempotency key per logical operation and reuses it across every retry of that operation — the retry policy and the key are one feature, not two (ZFN-13Field Note · currentZFN-13 — Fail fast and push back: retries, load shedding, and flow controlBuild client retries (backoff, jitter, Retry-After) from day one. Under overload, shed fast and push the failure back to the source to retry — don't retry internally and amplify it. Flow-control everywhere, bound every queue, and don't take more work than you can finish in time.Open ZFN-13 →);
• starts or continues the trace span and emits traceparent;
• stores the version token returned by the last write and presents it on subsequent reads automatically, so read-your-writes becomes a property of using the SDK rather than something each caller remembers.

On the server, the envelope is parsed, validated, and turned into a request context by middleware. Handlers receive a domain object and an ambient context. If a handler is reading a header, the layering has already leaked.

Make it bidirectional

Responses need an envelope too: the version token or vector clock the write produced, the trace id (so a support ticket can carry it), Retry-After and quota state, deprecation and sunset signals, an idempotent-replay indicator. Each of these otherwise becomes a field on every response type, with the same sediment problem. A response envelope also gives you somewhere to put the next cross-cutting concern without touching a single response schema.

Bound it, and don’t trust it

A general propagation mechanism is an attractive nuisance. First a vector clock, then a feature flag, then someone’s serialised session. Constrain it:

• Enumerate the fields in the schema. “Arbitrary key-values” is a transport capability, not a licence; unknown keys get dropped or rejected at ingress.
• Cap the total size. Proxies and servers enforce header limits, and the failure mode is a 431 or a silently truncated header, not a clean error you’ll notice in testing.
• Propagated context is an input, never an authority. A tenant id in baggage is a hint for logging and routing; the authorization decision comes from the authenticated credential, always (ZFN-38Field Note · currentZFN-38 — Agents are principals: delegate, never impersonateAn agent acting with a copied user credential is impersonation — untraceable by design. Give agents their own identities and keys; let them act for a human only through explicit, scoped, time-bounded, revocable delegation; and record both actor and principal on every action.Open ZFN-38 →, ZFN-40Field Note · currentZFN-40 — No anonymous "system" actorIf "system" appears as an actor in your audit log, attribution is already broken. Every automated action — cron job, cleanup task, migration, agent — runs as a named identity with its own credentials and scope, so "who did this?" has an answer and revocation is surgical.Open ZFN-40 →). Strip client-supplied values for any key the server treats as trusted, at the trust boundary, rather than hoping nobody sets them.
• Define absent behaviour per field. Someone will always call you with curl. Every field needs a defined meaning when missing — usually “you don’t get that guarantee,” never “undefined.”

Do it on day one

The cost asymmetry is the whole argument. Defining an empty envelope with two fields and a propagation rule costs an afternoon when there are no endpoints yet. Adding one once there are two hundred endpoints, four SDKs, and a dozen internal services means touching every handler, versioning every message type that already carries these as body fields, and living through a long migration in which some paths propagate and some don’t. That intermediate state is the one that hurts, because the guarantees are partial and silent — you cannot tell from a green test suite which paths lost the clock.

You don’t need to know on day one that you’ll eventually want a vector clock. You only need the place to put it.

Consequences

Easier:

• A new cross-cutting concern becomes one envelope field plus one change in the client and one in the middleware — instead of a change to every endpoint, every message type, and every SDK.
• Infrastructure can act on request context without understanding the payload: quota enforcement at ingress, freshness-aware read routing, trace stitching, deadline propagation.
• Async hops stop dropping context. You copy an envelope across the boundary, which is one thing to remember, rather than N fields, which is N things to forget.
• Domain schemas stay about the domain, and handlers stay testable without constructing a request context.
• Because the fields are standard ones, off-the-shelf proxies, tracing backends, and other people’s clients already speak them (ZFN-30Field Note · currentZFN-30 — Use the standard; don't reinvent the protocolWhen a standard exists for a common or complex problem, use it — don't reinvent the protocol. Standards encode huge adversarial expertise, especially in auth and crypto; a partial implementation beats rolling your own. You're not that special, and your problem isn't either.Open ZFN-30 →, ZFN-45Field Note · currentZFN-45 — Read the standards; better yet, help write themLearn to read standards docs — RFCs, W3C recs — fluently; they're the primary source, not a last resort. Even better, get involved: reading them well makes you a sharper builder, and helping write them is the best protocol education there is.Open ZFN-45 →).

Harder:

• You now have a second contract, and it’s a global one: adding or changing an envelope field touches every caller and every service at once. It needs an owner and a change process (ZFN-47Field Note · currentZFN-47 — Govern the contract between teams, not the code inside themTeams own services end to end; one team owns the gateway that dispatches to them. Govern exactly one thing centrally — the contract at the boundary (schema, identity, errors, idempotency) — and enforce it at runtime. Don't mandate libraries; ship them as an opt-in blueprint.Open ZFN-47 →) — precisely the kind of boundary contract worth governing centrally, and precisely the kind of thing that rots without one.
• Baggage is unbounded by default and crosses trust boundaries. Size caps, allowlists, and a hard rule that propagated values are never authorization inputs are all now your problem — and a real attack surface if you skip them.
• It puts machinery into the client. A thin generated wrapper acquires a small runtime (key state, trace context, retry-scoped idempotency keys), and that runtime has to behave identically in every language you ship.
• Header-based metadata has real limits: size caps, proxies that strip unknown headers, and transports whose metadata layer is weaker than HTTP’s. Some hops need an explicit mapping rather than a straight copy.

New obligations:

• Every new transport or hop you introduce has to carry the envelope, or the chain breaks silently there. That’s a checklist item for any new queue, cache, proxy, or protocol boundary.
• Every envelope field needs documented behaviour when absent, so hand-rolled clients degrade predictably instead of mysteriously.
• The SDK becomes where these guarantees actually live, which makes SDK bugs protocol bugs. Test the envelope behaviour — a retry reuses the key, the clock round-trips, the trace survives a queue hop — in the contract test suite, not as an SDK implementation detail.

References

• ZFN-14Field Note · currentZFN-14 — Define every API with a schema, and generate the clientsDefine every API with a machine-readable schema (OpenAPI, Protobuf, GraphQL) as the source of truth, and generate clients and server stubs from it — never hand-roll request-building and JSON parsing. Hand-written clients drift and break silently; check schema compatibility in CI.Open ZFN-14 → — the envelope belongs in the schema and gets generated into clients, exactly like the payload does.
• ZFN-19Field Note · currentZFN-19 — Annotate read-only and idempotent endpoints; make every mutation idempotentAnnotate every endpoint as read-only (safe) or idempotent, in the schema, so infrastructure can retry, route to replicas, and cache safely. Make every state-changing endpoint idempotent (idempotency keys for create/charge/send); a non-idempotent retry double-applies.Open ZFN-19 → — idempotency keys are an envelope concern; annotating endpoints is what lets infrastructure act on the envelope safely.
• ZFN-25Field Note · currentZFN-25 — Track the version a client has seen for read-your-writesFor read-your-writes across backends, track the latest version a client has seen — a token or vector clock. Return it on write; reads then go to a backend at least that fresh. Hold it client-side (a token they present) or server-side (a gateway tracks the session and routes).Open ZFN-25 → — the version token / vector clock that this note says must travel as baggage rather than as a payload field.
• ZFN-13Field Note · currentZFN-13 — Fail fast and push back: retries, load shedding, and flow controlBuild client retries (backoff, jitter, Retry-After) from day one. Under overload, shed fast and push the failure back to the source to retry — don't retry internally and amplify it. Flow-control everywhere, bound every queue, and don't take more work than you can finish in time.Open ZFN-13 → — retries and idempotency keys are one feature, and the client owns both.
• ZFN-47Field Note · currentZFN-47 — Govern the contract between teams, not the code inside themTeams own services end to end; one team owns the gateway that dispatches to them. Govern exactly one thing centrally — the contract at the boundary (schema, identity, errors, idempotency) — and enforce it at runtime. Don't mandate libraries; ship them as an opt-in blueprint.Open ZFN-47 → — the envelope is the boundary contract worth governing centrally.
• ZFN-30Field Note · currentZFN-30 — Use the standard; don't reinvent the protocolWhen a standard exists for a common or complex problem, use it — don't reinvent the protocol. Standards encode huge adversarial expertise, especially in auth and crypto; a partial implementation beats rolling your own. You're not that special, and your problem isn't either.Open ZFN-30 →, ZFN-45Field Note · currentZFN-45 — Read the standards; better yet, help write themLearn to read standards docs — RFCs, W3C recs — fluently; they're the primary source, not a last resort. Even better, get involved: reading them well makes you a sharper builder, and helping write them is the best protocol education there is.Open ZFN-45 → — every field here already has a standard; use it.
• W3C Trace ContextW3C Trace Context (W3C Recommendation)Standardises `traceparent` and `tracestate` as the interoperable way to propagate a trace identifier and vendor-specific trace state across service boundaries, so a request can be correlated end to end even when hops are instrumented by different vendors.w3.org ↗ and W3C BaggageW3C BaggageDefines a `baggage` header carrying arbitrary key-value context that propagates along the causal chain of a distributed request, so a value set at the edge is available to every downstream hop without any intermediary having to understand it.w3.org ↗ — the propagation layer, already specified.
• Idempotency-Key HTTP headerThe Idempotency-Key HTTP Header Field (IETF draft)Specifies an `Idempotency-Key` request header letting a client mark a mutating request with a unique key so servers can deduplicate retried requests, together with the expected server behaviour for concurrent, replayed, and mismatched-payload cases.datatracker.ietf.org ↗ and RFC 9449 DPoPRFC 9449 — OAuth 2.0 Demonstrating Proof of Possession (DPoP)Defines a mechanism to bind an access token to a client-held key: the client sends a signed DPoP proof header on each request, so a stolen token cannot be replayed by a party that doesn't hold the corresponding private key.rfc-editor.org ↗ — per-hop envelope fields with existing specifications.

Changelog

• 2026-07-30: First published as a Field Note.