---
id: 48
title: "Emit async work into the WAL, not a job table"
status: current
kind: note
date: 2026-06-29
authors:
  - "Theo Zourzouvillys"
tags: [architecture, infra, data, messaging, reliability]
references:
  - id: emit-message
    title: "PostgreSQL — pg_logical_emit_message()"
    url: https://www.postgresql.org/docs/current/functions-admin.html#FUNCTIONS-REPLICATION
    abstract: "Writes a logical-decoding message straight into the WAL: a routing prefix plus an opaque text/bytea payload, returning the LSN. With transactional => true it is decoded only if the surrounding transaction commits; with false it is emitted immediately regardless of outcome. No table, no heap tuple — the message lives only in the WAL stream."
  - id: logical-decoding
    title: "PostgreSQL — Logical Decoding"
    url: https://www.postgresql.org/docs/current/logicaldecoding.html
    abstract: "How Postgres streams a logically-decoded view of the WAL to consumers through a replication slot. The slot is a durable cursor: it tracks the consumer's confirmed LSN so a crashed listener resumes exactly where it left off — and, crucially, the server retains WAL until the slot confirms it, which is both the durability guarantee and the operational hazard."
  - id: outbox
    title: "Transactional Outbox pattern"
    url: https://microservices.io/patterns/data/transactional-outbox.html
    abstract: "Write an 'outbox' record in the same database transaction as the business change so the two commit atomically, then a separate relay publishes the record. It solves the dual-write problem (you must not commit a row and publish to a queue as two non-atomic steps). CDC off the WAL is the clean, poll-free relay."
  - id: slot-keep
    title: "PostgreSQL — max_slot_wal_keep_size"
    url: https://www.postgresql.org/docs/current/runtime-config-replication.html#GUC-MAX-SLOT-WAL-KEEP-SIZE
    abstract: "Caps how much WAL a replication slot may pin. Past the limit Postgres invalidates the lagging slot so it can recycle WAL — sacrificing the slot to protect the primary's disk, rather than letting an unconfirmed slot fill it. The essential safety valve for any logical-decoding consumer."
summary: "When a DB write should trigger async work, ride the WAL instead of dual-writing or polling a job table. pg_logical_emit_message emits the event transactionally — outbox semantics, no table. A WAL listener consumes it statefully and fans out, keeping load off the primary."
supersedes: null
superseded_by: null
aliases: []
---

## TL;DR

When the *thing that should trigger async work* already happens inside your database — a row is
written, a state flips — don't dual-write to a queue and don't poll a job table. **Ride the
write-ahead log.** Postgres's [`pg_logical_emit_message(transactional => true, prefix, payload)`](ref:emit-message)
writes an event **straight into the WAL inside your transaction** — no table, no row to insert, no
dead tuple to vacuum — and a logical-replication consumer (a "WAL listener") reads the decoded
stream, processes it statefully, and pushes the work onward (into a queue, a search index, another
datastore).

You get the **transactional outbox** guarantee — the event is emitted atomically with the write, or
not at all — without the outbox *table*. And because the expensive processing lives in the consumer,
the load stays **off your primary**: no polling loop, no job-table churn, no vacuum tail.

The catch you must respect: a replication slot **pins WAL until the consumer confirms it**, so a dead
or lagging listener can fill the primary's disk and take it down. Bound it
([`max_slot_wal_keep_size`](ref:slot-keep)), alert on slot lag, and make the consumer idempotent —
delivery is at-least-once.

## Context

This is the concrete mechanism behind [ZFN-24](/zfn/24-one-transactional-store-per-write/): one
transactional store per write, changes propagated asynchronously. The whole difficulty of "propagate
asynchronously" is the *seam* between the commit and the propagation. Two common ways to cross it are
both bad:

- **Dual write.** Commit the row, then publish to a queue as a second operation. The two aren't
  atomic — the commit lands and the publish fails (the process dies, the broker is down), and your
  systems silently diverge. This is the canonical distributed-systems footgun, and no amount of
  retry-after-commit fully closes it.
- **Poll a table.** Write a `jobs` row, run a worker that loops `SELECT … FOR UPDATE SKIP LOCKED`.
  Correct, but it dumps the *entire* async workload — the inserts, the dead rows, the constant
  polling, the vacuum — onto your primary, the one database you least want to load.

The **[transactional outbox](ref:outbox)** fixes the atomicity: write an `outbox` row *in the same
transaction* as the business change so they commit together, then a separate relay publishes it.
Reading that outbox via change-data-capture off the WAL — rather than polling it — is the clean
version.

[`pg_logical_emit_message`](ref:emit-message) removes the last piece of overhead: the table itself.
Every durable write in Postgres already flows through the WAL — that's how durability and replication
work. The function writes a **message directly into that log**: a `prefix` for routing and an opaque
`content` payload, surfaced by [logical decoding](ref:logical-decoding) and delivered to your consumer
in commit order. With `transactional => true` the message is decoded **only if the surrounding
transaction commits** — so you keep the outbox's atomicity — but it leaves **no heap tuple, no index
entry, nothing to vacuum**. It's the outbox without the box.

## Recommendation

**When the source of truth and the trigger both live in your database, emit the work into the WAL and
process it off-box.**

- **Emit transactionally.** In the same transaction as the business write, call
  `pg_logical_emit_message(true, 'order.placed', payload)`. It commits atomically with your change or
  not at all — outbox semantics, zero table. Reach for `transactional => false` only when you
  deliberately want a fire-immediately signal that ignores commit/rollback, and accept that you've
  given up the atomicity that was the whole point.

- **Make the payload an explicit, versioned event — not a row diff.** This is a real advantage over
  table-based CDC: capturing row changes couples the consumer to your physical schema, so renaming a
  column breaks it downstream. An emitted message is an *intentional* event with its own contract and
  a version field ([ZFN-14](/zfn/14-schema-first-apis-generate-clients/)). You publish what you
  *mean*, not whatever shape your tables happen to have this quarter.

- **Consume through a durable logical replication slot.** The [slot](ref:logical-decoding) is the
  consumer's cursor: Postgres tracks its confirmed LSN, so the listener can crash and resume exactly
  where it left off — that's the "stateful" part, and you get it for free. Decode with whatever fits
  (`pgoutput`, `wal2json`, Debezium), do the work, then advance the slot.

- **Treat the listener as a bridge, then fan out.** A slot is a *single ordered stream* with *one*
  consumer — you do **not** get a queue's competing-consumers parallelism for free. So the listener's
  job is usually to validate, dedupe, and **push the work into the right downstream tool**: a queue
  for work distribution, a topic for fan-out, a journal for history, an OLAP store or search index for
  derived state. Choose that downstream deliberately — they are not interchangeable
  ([ZFN-12](/zfn/12-queues-topics-journals/)).

- **Process idempotently.** Delivery is **at-least-once**: a consumer that does its work and then dies
  before confirming the LSN will re-see the message on restart. Key every effect on the message's LSN
  or your own idempotency key ([ZFN-19](/zfn/19-annotate-readonly-idempotent-endpoints/)) so a
  redelivery is a no-op rather than a double-charge.

- **Keep the heavy work off the primary.** Run the consumer as its own process and do the expensive
  transformation and fan-out *there*, not in database queries against the primary. This is
  data-plane / control-plane hygiene ([ZFN-16](/zfn/16-separate-data-plane-control-plane/)) applied to
  async work: the primary's extra job is writing a WAL message it was effectively writing anyway.

> [!aside]
>
> The first time I reached for this, the goal was a steady firehose of state changes into a search
> index and an analytics store without the primary feeling a thing. The polling worker it replaced
> had quietly become the heaviest thing on the box — thousands of `SKIP LOCKED` probes an hour, most
> returning nothing, plus the vacuum tail of the job table. Emitting WAL messages and running one
> listener that fanned out into a queue dropped that load to near-zero on the primary and moved it
> onto a consumer I could scale and restart on its own. The part that surprised me was operational,
> not architectural: the day the consumer silently wedged, the slot kept pinning WAL and the primary's
> disk began marching toward full. That's the trade you're signing up for — and exactly why
> `max_slot_wal_keep_size` and slot-lag alerting are not optional extras.

## Consequences

**Easier:**

- Atomic event emission with **no outbox table** — no inserts, no deletes, no dead rows, no vacuum on
  the work-trigger path.
- The async load moves **off the primary**: no polling, no job-table churn; the expensive processing
  runs in a consumer you scale and restart independently.
- The consumer is **resumable by construction** — the slot's confirmed LSN is durable state, so a
  crash resumes precisely where it stopped, with no checkpoint table of your own.
- You emit an **explicit event contract**, decoupled from physical schema, instead of leaking row
  shapes to every downstream system.

**Harder:**

- A replication slot **pins WAL until it's confirmed** — a dead or lagging consumer grows WAL without
  bound and can **fill the primary's disk and take it down**. You must cap it
  ([`max_slot_wal_keep_size`](ref:slot-keep), which invalidates the slot rather than filling the disk)
  and alert on slot lag. This is a *sharper* footgun than a bloated job table: the failure mode is the
  primary, not the worker.
- A slow consumer is a backpressure problem you now own end-to-end — there's no broker absorbing the
  burst, so flow control and bounded retention matter ([ZFN-13](/zfn/13-load-shedding-and-flow-control/)).
- **At-least-once, single ordered stream.** No exactly-once and no free parallelism — you need
  idempotent consumers and you fan out downstream for throughput.
- **Messages are ephemeral.** Once the slot advances past a WAL message and the WAL recycles, it's
  gone: no `SELECT` over history, no replay from the source. If you need durable history or audit,
  write a journal too ([ZFN-12](/zfn/12-queues-topics-journals/)) — the WAL message is a *trigger*,
  not a system of record.
- **More operating surface.** `wal_level = logical` (a modest WAL increase), a decoding plugin, slot
  monitoring, and failover handling — on older Postgres or some managed platforms a failover can drop
  a slot's position and force a reconcile (PG 16+ failover slots help). Decoding itself also costs the
  primary some walsender CPU, so the win is avoiding the polling, the table, and the processing — not
  that the primary does literally nothing.

**New obligations:**

- Monitor every slot's lag and bound its WAL retention from day one. Treat an unconfirmed, growing
  slot as a page-worthy condition, not a dashboard curiosity.
- Own the event payload as a versioned contract, and define the dead-letter / poison-message path for
  messages the consumer genuinely cannot process — otherwise one bad message stalls the whole ordered
  stream behind it.

## References

- [`pg_logical_emit_message()`](https://www.postgresql.org/docs/current/functions-admin.html#FUNCTIONS-REPLICATION)
  — emit a message into the WAL, transactionally or not; the table-free outbox primitive.
- [PostgreSQL Logical Decoding](https://www.postgresql.org/docs/current/logicaldecoding.html) —
  replication slots as durable cursors, and the WAL-retention behaviour you must operate.
- [Transactional Outbox pattern](https://microservices.io/patterns/data/transactional-outbox.html) —
  why you emit atomically with the write instead of dual-writing to a queue.
- [`max_slot_wal_keep_size`](https://www.postgresql.org/docs/current/runtime-config-replication.html#GUC-MAX-SLOT-WAL-KEEP-SIZE)
  — the safety valve that protects the primary from a lagging slot.
- [ZFN-24](/zfn/24-one-transactional-store-per-write/) — the principle this implements: one
  transactional store, propagate changes asynchronously.
- [ZFN-12](/zfn/12-queues-topics-journals/) — the WAL stream is journal-shaped; choose the downstream
  queue / topic / journal deliberately.
- [ZFN-19](/zfn/19-annotate-readonly-idempotent-endpoints/) — at-least-once delivery demands
  idempotent processing.
- [ZFN-16](/zfn/16-separate-data-plane-control-plane/) and
  [ZFN-13](/zfn/13-load-shedding-and-flow-control/) — keep processing off the primary, and handle the
  backpressure the slot exposes.
- [ZFN-14](/zfn/14-schema-first-apis-generate-clients/) — the emitted message is an event contract;
  version it rather than leaking your schema.

## Changelog

- **2026-06-29**: First published as a Field Note.