Field Note 48current

Emit async work into the WAL, not a job table

When a DB write should trigger async work, ride the WAL instead of dual-writing or polling a job table. pg_logical_emit_message emits the event transactionally — outbox semantics, no table. A WAL listener consumes it statefully and fans out, keeping load off the primary.

By: Theo Zourzouvillys
Published: June 29, 2026
Est. reading time: 8 min
Tags: architectureinfradatamessagingreliability

My Personal LLM Policy: Extract, not generate

The thinking is mine, whether it’s years old or from this week. What a model does is get it out of my head and onto the page — writing time I’d otherwise never spend, not substance I didn’t have. I read every line, I can defend any sentence, and the errors are mine: the same bar I hold everything here to, model or no model.

What that distinction means →The standard I hold to →

TL;DR

When the thing that should trigger async work already happens inside your database — a row is written, a state flips — don’t dual-write to a queue and don’t poll a job table. Ride the write-ahead log. Postgres’s pg_logical_emit_message(transactional => true, prefix, payload)PostgreSQL — pg_logical_emit_message()Writes a logical-decoding message straight into the WAL: a routing prefix plus an opaque text/bytea payload, returning the LSN. With transactional => true it is decoded only if the surrounding transaction commits; with false it is emitted immediately regardless of outcome. No table, no heap tuple — the message lives only in the WAL stream.postgresql.org ↗ writes an event straight into the WAL inside your transaction — no table, no row to insert, no dead tuple to vacuum — and a logical-replication consumer (a “WAL listener”) reads the decoded stream, processes it statefully, and pushes the work onward (into a queue, a search index, another datastore).

You get the transactional outbox guarantee — the event is emitted atomically with the write, or not at all — without the outbox table. And because the expensive processing lives in the consumer, the load stays off your primary: no polling loop, no job-table churn, no vacuum tail.

The catch you must respect: a replication slot pins WAL until the consumer confirms it, so a dead or lagging listener can fill the primary’s disk and take it down. Bound it (max_slot_wal_keep_sizePostgreSQL — max_slot_wal_keep_sizeCaps how much WAL a replication slot may pin. Past the limit Postgres invalidates the lagging slot so it can recycle WAL — sacrificing the slot to protect the primary's disk, rather than letting an unconfirmed slot fill it. The essential safety valve for any logical-decoding consumer.postgresql.org ↗), alert on slot lag, and make the consumer idempotent — delivery is at-least-once.

Context

This is the concrete mechanism behind ZFN-24Field Note · currentZFN-24 — One transactional store per write; propagate changes asynchronouslyCommit each logical write to exactly one transactional store; update other systems via reliable ordered async events — never a synchronous write across two stores, and never 2PC. With a relational primary the WAL is your replayable journal; write events into the same transaction.Open ZFN-24 →: one transactional store per write, changes propagated asynchronously. The whole difficulty of “propagate asynchronously” is the seam between the commit and the propagation. Two common ways to cross it are both bad:

Dual write. Commit the row, then publish to a queue as a second operation. The two aren’t atomic — the commit lands and the publish fails (the process dies, the broker is down), and your systems silently diverge. This is the canonical distributed-systems footgun, and no amount of retry-after-commit fully closes it.
Poll a table. Write a jobs row, run a worker that loops SELECT … FOR UPDATE SKIP LOCKED. Correct, but it dumps the entire async workload — the inserts, the dead rows, the constant polling, the vacuum — onto your primary, the one database you least want to load.

The transactional outboxTransactional Outbox patternWrite an 'outbox' record in the same database transaction as the business change so the two commit atomically, then a separate relay publishes the record. It solves the dual-write problem (you must not commit a row and publish to a queue as two non-atomic steps). CDC off the WAL is the clean, poll-free relay.microservices.io ↗ fixes the atomicity: write an outbox row in the same transaction as the business change so they commit together, then a separate relay publishes it. Reading that outbox via change-data-capture off the WAL — rather than polling it — is the clean version.

pg_logical_emit_messagePostgreSQL — pg_logical_emit_message()Writes a logical-decoding message straight into the WAL: a routing prefix plus an opaque text/bytea payload, returning the LSN. With transactional => true it is decoded only if the surrounding transaction commits; with false it is emitted immediately regardless of outcome. No table, no heap tuple — the message lives only in the WAL stream.postgresql.org ↗ removes the last piece of overhead: the table itself. Every durable write in Postgres already flows through the WAL — that’s how durability and replication work. The function writes a message directly into that log: a prefix for routing and an opaque content payload, surfaced by logical decodingPostgreSQL — Logical DecodingHow Postgres streams a logically-decoded view of the WAL to consumers through a replication slot. The slot is a durable cursor: it tracks the consumer's confirmed LSN so a crashed listener resumes exactly where it left off — and, crucially, the server retains WAL until the slot confirms it, which is both the durability guarantee and the operational hazard.postgresql.org ↗ and delivered to your consumer in commit order. With transactional => true the message is decoded only if the surrounding transaction commits — so you keep the outbox’s atomicity — but it leaves no heap tuple, no index entry, nothing to vacuum. It’s the outbox without the box.

Recommendation

When the source of truth and the trigger both live in your database, emit the work into the WAL and process it off-box.

Emit transactionally. In the same transaction as the business write, call pg_logical_emit_message(true, 'order.placed', payload). It commits atomically with your change or not at all — outbox semantics, zero table. Reach for transactional => false only when you deliberately want a fire-immediately signal that ignores commit/rollback, and accept that you’ve given up the atomicity that was the whole point.
Make the payload an explicit, versioned event — not a row diff. This is a real advantage over table-based CDC: capturing row changes couples the consumer to your physical schema, so renaming a column breaks it downstream. An emitted message is an intentional event with its own contract and a version field (ZFN-14Field Note · currentZFN-14 — Define every API with a schema, and generate the clientsDefine every API with a machine-readable schema (OpenAPI, Protobuf, GraphQL) as the source of truth, and generate clients and server stubs from it — never hand-roll request-building and JSON parsing. Hand-written clients drift and break silently; check schema compatibility in CI.Open ZFN-14 →). You publish what you mean, not whatever shape your tables happen to have this quarter.
Consume through a durable logical replication slot. The slotPostgreSQL — Logical DecodingHow Postgres streams a logically-decoded view of the WAL to consumers through a replication slot. The slot is a durable cursor: it tracks the consumer's confirmed LSN so a crashed listener resumes exactly where it left off — and, crucially, the server retains WAL until the slot confirms it, which is both the durability guarantee and the operational hazard.postgresql.org ↗ is the consumer’s cursor: Postgres tracks its confirmed LSN, so the listener can crash and resume exactly where it left off — that’s the “stateful” part, and you get it for free. Decode with whatever fits (pgoutput, wal2json, Debezium), do the work, then advance the slot.
Treat the listener as a bridge, then fan out. A slot is a single ordered stream with one consumer — you do not get a queue’s competing-consumers parallelism for free. So the listener’s job is usually to validate, dedupe, and push the work into the right downstream tool: a queue for work distribution, a topic for fan-out, a journal for history, an OLAP store or search index for derived state. Choose that downstream deliberately — they are not interchangeable (ZFN-12Field Note · currentZFN-12 — Queues, topics, and journals are different tools — don't conflate themQueues (competing consumers), topics (fan-out), and journals (ordered, replayable logs) give different guarantees. Don't conflate them; a pipeline often uses several. Prefer journals over topics, but not where head-of-line blocking hurts. With queues, bound the concurrency.Open ZFN-12 →).
Process idempotently. Delivery is at-least-once: a consumer that does its work and then dies before confirming the LSN will re-see the message on restart. Key every effect on the message’s LSN or your own idempotency key (ZFN-19Field Note · currentZFN-19 — Annotate read-only and idempotent endpoints; make every mutation idempotentAnnotate every endpoint as read-only (safe) or idempotent, in the schema, so infrastructure can retry, route to replicas, and cache safely. Make every state-changing endpoint idempotent (idempotency keys for create/charge/send); a non-idempotent retry double-applies.Open ZFN-19 →) so a redelivery is a no-op rather than a double-charge.
Keep the heavy work off the primary. Run the consumer as its own process and do the expensive transformation and fan-out there, not in database queries against the primary. This is data-plane / control-plane hygiene (ZFN-16Field Note · currentZFN-16 — Separate the data plane from the control planeSplit the serving path (data plane) from the management path (control plane). The data plane keeps serving on last-known-good config when the control plane is down — never call it on the hot path. Coupling them turns a control-plane bug into a serving outage.Open ZFN-16 →) applied to async work: the primary’s extra job is writing a WAL message it was effectively writing anyway.

From the field

The first time I reached for this, the goal was a steady firehose of state changes into a search index and an analytics store without the primary feeling a thing. The polling worker it replaced had quietly become the heaviest thing on the box — thousands of SKIP LOCKED probes an hour, most returning nothing, plus the vacuum tail of the job table. Emitting WAL messages and running one listener that fanned out into a queue dropped that load to near-zero on the primary and moved it onto a consumer I could scale and restart on its own. The part that surprised me was operational, not architectural: the day the consumer silently wedged, the slot kept pinning WAL and the primary’s disk began marching toward full. That’s the trade you’re signing up for — and exactly why max_slot_wal_keep_size and slot-lag alerting are not optional extras.

Consequences

Easier:

Atomic event emission with no outbox table — no inserts, no deletes, no dead rows, no vacuum on the work-trigger path.
The async load moves off the primary: no polling, no job-table churn; the expensive processing runs in a consumer you scale and restart independently.
The consumer is resumable by construction — the slot’s confirmed LSN is durable state, so a crash resumes precisely where it stopped, with no checkpoint table of your own.
You emit an explicit event contract, decoupled from physical schema, instead of leaking row shapes to every downstream system.

Harder:

A replication slot pins WAL until it’s confirmed — a dead or lagging consumer grows WAL without bound and can fill the primary’s disk and take it down. You must cap it (max_slot_wal_keep_sizePostgreSQL — max_slot_wal_keep_sizeCaps how much WAL a replication slot may pin. Past the limit Postgres invalidates the lagging slot so it can recycle WAL — sacrificing the slot to protect the primary's disk, rather than letting an unconfirmed slot fill it. The essential safety valve for any logical-decoding consumer.postgresql.org ↗, which invalidates the slot rather than filling the disk) and alert on slot lag. This is a sharper footgun than a bloated job table: the failure mode is the primary, not the worker.
A slow consumer is a backpressure problem you now own end-to-end — there’s no broker absorbing the burst, so flow control and bounded retention matter (ZFN-13Field Note · currentZFN-13 — Fail fast and push back: retries, load shedding, and flow controlBuild client retries (backoff, jitter, Retry-After) from day one. Under overload, shed fast and push the failure back to the source to retry — don't retry internally and amplify it. Flow-control everywhere, bound every queue, and don't take more work than you can finish in time.Open ZFN-13 →).
At-least-once, single ordered stream. No exactly-once and no free parallelism — you need idempotent consumers and you fan out downstream for throughput.
Messages are ephemeral. Once the slot advances past a WAL message and the WAL recycles, it’s gone: no SELECT over history, no replay from the source. If you need durable history or audit, write a journal too (ZFN-12Field Note · currentZFN-12 — Queues, topics, and journals are different tools — don't conflate themQueues (competing consumers), topics (fan-out), and journals (ordered, replayable logs) give different guarantees. Don't conflate them; a pipeline often uses several. Prefer journals over topics, but not where head-of-line blocking hurts. With queues, bound the concurrency.Open ZFN-12 →) — the WAL message is a trigger, not a system of record.
More operating surface. wal_level = logical (a modest WAL increase), a decoding plugin, slot monitoring, and failover handling — on older Postgres or some managed platforms a failover can drop a slot’s position and force a reconcile (PG 16+ failover slots help). Decoding itself also costs the primary some walsender CPU, so the win is avoiding the polling, the table, and the processing — not that the primary does literally nothing.

New obligations:

Monitor every slot’s lag and bound its WAL retention from day one. Treat an unconfirmed, growing slot as a page-worthy condition, not a dashboard curiosity.
Own the event payload as a versioned contract, and define the dead-letter / poison-message path for messages the consumer genuinely cannot process — otherwise one bad message stalls the whole ordered stream behind it.

References

pg_logical_emit_message() — emit a message into the WAL, transactionally or not; the table-free outbox primitive.
PostgreSQL Logical Decoding — replication slots as durable cursors, and the WAL-retention behaviour you must operate.
Transactional Outbox pattern — why you emit atomically with the write instead of dual-writing to a queue.
max_slot_wal_keep_size — the safety valve that protects the primary from a lagging slot.
ZFN-24Field Note · currentZFN-24 — One transactional store per write; propagate changes asynchronouslyCommit each logical write to exactly one transactional store; update other systems via reliable ordered async events — never a synchronous write across two stores, and never 2PC. With a relational primary the WAL is your replayable journal; write events into the same transaction.Open ZFN-24 → — the principle this implements: one transactional store, propagate changes asynchronously.
ZFN-12Field Note · currentZFN-12 — Queues, topics, and journals are different tools — don't conflate themQueues (competing consumers), topics (fan-out), and journals (ordered, replayable logs) give different guarantees. Don't conflate them; a pipeline often uses several. Prefer journals over topics, but not where head-of-line blocking hurts. With queues, bound the concurrency.Open ZFN-12 → — the WAL stream is journal-shaped; choose the downstream queue / topic / journal deliberately.
ZFN-19Field Note · currentZFN-19 — Annotate read-only and idempotent endpoints; make every mutation idempotentAnnotate every endpoint as read-only (safe) or idempotent, in the schema, so infrastructure can retry, route to replicas, and cache safely. Make every state-changing endpoint idempotent (idempotency keys for create/charge/send); a non-idempotent retry double-applies.Open ZFN-19 → — at-least-once delivery demands idempotent processing.
ZFN-16Field Note · currentZFN-16 — Separate the data plane from the control planeSplit the serving path (data plane) from the management path (control plane). The data plane keeps serving on last-known-good config when the control plane is down — never call it on the hot path. Coupling them turns a control-plane bug into a serving outage.Open ZFN-16 → and ZFN-13Field Note · currentZFN-13 — Fail fast and push back: retries, load shedding, and flow controlBuild client retries (backoff, jitter, Retry-After) from day one. Under overload, shed fast and push the failure back to the source to retry — don't retry internally and amplify it. Flow-control everywhere, bound every queue, and don't take more work than you can finish in time.Open ZFN-13 → — keep processing off the primary, and handle the backpressure the slot exposes.
ZFN-14Field Note · currentZFN-14 — Define every API with a schema, and generate the clientsDefine every API with a machine-readable schema (OpenAPI, Protobuf, GraphQL) as the source of truth, and generate clients and server stubs from it — never hand-roll request-building and JSON parsing. Hand-written clients drift and break silently; check schema compatibility in CI.Open ZFN-14 → — the emitted message is an event contract; version it rather than leaking your schema.

Changelog

2026-06-29: First published as a Field Note.