---
id: 65
title: "Journal the write, apply it in micro-batches"
kind: note
status: current
date: 2026-08-19
authors:
  - "Theo Zourzouvillys"
tags: [architecture, data, messaging, reliability, consistency]
references:
  - id: ddia
    title: "Martin Kleppmann — Designing Data-Intensive Applications, ch. 11"
    url: https://dataintensive.net/
    abstract: "The chapter on stream processing: logs as ordered, replayable records; consumer offsets as durable cursors; and why ordering guarantees are per-partition rather than global. The reference text for why a buffered write pipeline behaves the way it does."
  - id: pgpopulate
    title: "PostgreSQL — Populating a Database"
    url: https://www.postgresql.org/docs/current/populate.html
    abstract: "The manual's own guidance on bulk writes: batch many inserts into a single transaction (or COPY) rather than committing each one, because per-transaction commit overhead (round trip, WAL flush, fsync) dominates the actual cost of the row. The mechanical reason micro-batching wins."
summary: "If nothing the caller does next depends on a write, it does not belong in the request path: append it to a durable ordered journal and apply it in micro-batches. The bill is ordering. Sequence comes from the journal, never the clock, and never two paths to one row."
supersedes: null
superseded_by: null
aliases: []
crossrefs:
  ZFN-48: "The mirror image of this note: there the database is the source of async work and the WAL carries it outward; here the database is the sink and the journal sits in front of it."
  ZFN-12: "The buffer is specifically a journal and not a queue. The ordering and replay it gives you are the whole reason a batched apply can be made safe."
  ZFN-59: "The rule that makes buffered writes orderable at all: sequence comes from one writer's log position, never from each producer's opinion of what time it is."
  ZFN-24: "Buffering is how 'propagate changes asynchronously' actually gets implemented on the write path, without giving up the single transactional store."
---
## TL;DR

**If nothing the caller does next depends on a write having already happened, that write does not
belong in the request path.** Append it to a durable, ordered journal, return, and let a consumer
apply it.

- The consumer applies in micro-batches (*N* records or *T* milliseconds, whichever comes first,
  one transaction per batch). A hundred single-row inserts become one multi-row statement: one
  round trip, one commit, [one flush](ref:pgpopulate).
- You also buy load-levelling. A traffic spike becomes a deeper backlog instead of a write storm,
  and a bad minute on the store stops being a bad minute on your API.
- **The bill is ordering**, and it is paid in full or not at all. A buffer is a reordering machine
  unless you stop it being one: order comes from the journal's own sequence, never from a
  producer's wall clock, and you partition by entity key so you parallelise *across* keys and never
  within one.
- The sharp edge is mixing paths. If one write to a row is buffered and another is synchronous, the
  synchronous one overtakes. A `DELETE` applied immediately, while the `INSERT` it was meant to
  remove is still sitting in the buffer, leaves you a resurrected row that nothing will ever clean
  up. Every write to an entity goes through the buffer, or none of them do.
- Do not buffer a write the caller reads back immediately, one a constraint must reject at request
  time, or one whose failure the caller needs to hear about (you have already returned `200`).

## Context

A large share of the writes in a typical service are not things anyone is waiting on:
`last_seen_at` touches, usage counters, telemetry and audit events, search-index updates,
denormalised read models, syncs into a third-party system. They sit in the request path because
that is the default shape of code, not because anything requires it. You pay for that three times
over: latency on every request, the availability of your endpoint welded to the availability of the
store, and a traffic burst converted directly into a write burst.

The store dislikes it as much as you should. Row-at-a-time is a horrible way to write to anything,
and the reason is arithmetic rather than taste: per-statement round trip, per-transaction commit,
per-commit durable flush, and
[the overhead per transaction dominates the cost of the row itself](ref:pgpopulate). Put numbers on
it. A hundred inserts at a millisecond of commit apiece is a tenth of a second of pure overhead
carrying maybe a few kilobytes of actual data. The same hundred rows as one statement is one round
trip and one flush. The same arithmetic holds for an external API, where each call is a TLS round
trip and a slice of somebody's rate limit.

This is the mirror image of [ZFN-48](/zfn/48-emit-async-work-into-the-wal/). There the write has
already happened and the database is the *source* of async work, emitted outward through the WAL.
here the write has not happened yet and does not need to, and the database is the *sink*, so the
journal sits in front of it, absorbing writes and feeding them in at a rate and shape the store
actually likes. Same primitive, pointed the other way.

The reason this is not simply "put it on a queue" is [ZFN-12](/zfn/12-queues-topics-journals/). A
queue gives you independent per-item progress and no ordering, which is exactly wrong when the
items are mutations of the same rows. What you want is a journal: ordered, durable, with a consumer
that tracks its own offset and can be replayed after a bad deploy ([DDIA ch. 11](ref:ddia)).

## Recommendation

**Buffer writes that nobody is waiting on into an ordered journal, and apply them in
micro-batches.** Treat ordering as a design obligation and not as a property you hope survives.

**Decide per write, with one test.** Does anything the caller does next depend on this having
already happened? If yes, it is synchronous. If no, buffer it. That test, and not "is it
important?", is the one that partitions correctly. An audit event can be extremely important and
still not need to be written before the response goes out.

**Make the buffer as durable as the data deserves, and say which you chose.** An in-process ring
buffer is a decision to lose the last few seconds of writes on a crash. That is a perfectly good
trade for view counters and a terrible one for audit records. Name the loss budget out loud,
because the failure mode of not naming it is discovering it during an incident. Whatever you
choose, the buffer has to be flushed on shutdown, which makes it part of the drain sequence in
[ZFN-60](/zfn/60-graceful-shutdown-is-a-protocol/) rather than something the process can exit
without.

**Apply in micro-batches, and commit the offset last.** Take up to *N* records or wait up to *T*
milliseconds, whichever comes first (the timer stops a quiet period stranding records, the count
stops a busy one building an unbounded transaction). Apply the batch in a single transaction, then
advance the offset *after* that transaction commits. That ordering is what makes the pipeline
at-least-once rather than at-most-once, and at-least-once means the apply has to be idempotent
([ZFN-19](/zfn/19-annotate-readonly-idempotent-endpoints/)), because a crash between commit and
offset advance will re-apply the batch.

**Take order from the journal, never from the clock.** This is the part that quietly goes wrong.
Once records from many producers are sitting in a buffer, something has to decide what order they
apply in, and the tempting field is the `created_at` each producer stamped on the way in. That is
not an ordering. It is several machines' disagreeing opinions of *now*, and the skew is small
enough that it always works in testing ([ZFN-59](/zfn/59-two-clocks/)). The sequence that means
something is the journal's own: the offset assigned by the single writer that accepted the append
([ZFN-24](/zfn/24-one-transactional-store-per-write/)). Coalescing inside a batch (collapsing five
updates to one row into a last-write-wins) has to resolve by that offset too, or you have built a
machine that reliably applies the oldest update last.

**Partition by entity key. Parallelise across keys, never within one.** One consumer per journal is
usually not enough throughput, and the safe way to scale is to shard by the key whose order you
care about: user id, tenant id, document id. Round-robin sharding of a batch across workers is the
classic version of this bug. Two mutations of the same row land in different workers and race, and
the winner is whichever one the scheduler felt like. Sharding by key keeps operations on a given
entity in a single ordered lane, and lets unrelated entities get on with it in parallel.

**Never run two paths to the same entity.** This is where `DELETE` overtakes `INSERT`. If the
create is buffered and the delete is synchronous (because deletes felt urgent, or because they were
written by someone else, or because one call arrives through the API and the other through an admin
tool) then the delete finds nothing to delete, the buffered insert lands afterwards, and the row is
back from the dead with nobody watching. The fix is not more care, it is structural: route every
mutation of an entity through the same lane, or make the apply conditional on a version the
producer supplied so a stale operation is discarded rather than applied
([ZFN-25](/zfn/25-read-your-writes-version-token/)). Two paths and hope is not a design.

**Bound the buffer, and decide what "full" means.** A buffer that grows without limit relocates the
outage rather than preventing it. Cap it, monitor consumer lag as a first-class signal, and choose
deliberately whether a full buffer sheds the write or pushes back on the producer
([ZFN-13](/zfn/13-load-shedding-and-flow-control/)). For counters, dropping is often correct. For
audit it never is, which usually means audit gets a durable journal rather than an in-memory one.

## Consequences

**Easier:**

- Request latency drops by whatever the write cost, and endpoints stop failing because a downstream
  store or third-party API is having a bad minute.
- The store sees far fewer, far larger transactions, which is the shape it is efficient at, instead
  of a storm of single-row commits.
- Bursts turn into backlog. The journal absorbs the spike and the consumer drains at a rate you
  choose, rather than passing the spike straight through.
- Coalescing becomes possible at all. A thousand touches of one counter inside a window can become
  one write, which is simply not available when each is applied inline.
- A bad consumer deploy is recoverable by replay, because a journal keeps history and a queue does
  not.

**Harder:**

- **You have moved the error contract.** The caller got `200` before the write succeeded, so a
  failure at apply time has nobody to tell. It needs somewhere to go (a dead-letter path and an
  alert), and the API documentation needs to stop implying the write already happened
  ([ZFN-58](/zfn/58-errors-are-part-of-the-contract/)).
- **Read-your-writes is gone** for anything buffered. A caller that writes and immediately reads
  sees the old value, and that surprise is usually worse than the latency you saved
  ([ZFN-25](/zfn/25-read-your-writes-version-token/)).
- Constraints no longer reject at request time. A uniqueness violation surfaces in the consumer,
  minutes later, on a request that has already been answered successfully.
- Ordering is now something you own explicitly, and its failures are the quiet kind: a resurrected
  row, a counter that drifts, a stale value that wins. none of them raise an error.
- One more moving part to run, with its own lag, its own backlog, and its own way of being down.

**New obligations:**

- Monitor consumer lag and buffer depth, and page on them. An unnoticed stalled consumer is a
  silent, growing pile of writes that have been acknowledged and not performed.
- Define the dead-letter path before launch, and make sure a poison record cannot wedge the lane
  behind it forever.
- Write down the durability choice, meaning what a crash loses, where the next person will find it.

If that list of obligations reads as long, that is the honest price and it is worth checking you
want to pay it. Buffering is not free, it is a trade: you are buying latency and load-levelling
with ordering work and a worse error contract. For a `last_seen_at` touch that is an obvious yes.
For anything a human will be shown as confirmation, think a lot harder!

## References

- [Designing Data-Intensive Applications, ch. 11](https://dataintensive.net/) on logs, consumer
  offsets, and why ordering guarantees are per-partition rather than global.
- [PostgreSQL: Populating a Database](https://www.postgresql.org/docs/current/populate.html), the
  mechanical case for batching, where per-transaction overhead dominates per-row cost.
- [ZFN-48](/zfn/48-emit-async-work-into-the-wal/), the same journal pointed the other way: the
  database as the source of async work rather than its sink.
- [ZFN-12](/zfn/12-queues-topics-journals/) on why the buffer is a journal and not a queue.
- [ZFN-59](/zfn/59-two-clocks/) on never ordering cross-machine events by wall-clock timestamps.
- [ZFN-24](/zfn/24-one-transactional-store-per-write/), the single writer whose sequence is the only
  real ordering you have.
- [ZFN-19](/zfn/19-annotate-readonly-idempotent-endpoints/) on at-least-once delivery demanding an
  idempotent apply.
- [ZFN-13](/zfn/13-load-shedding-and-flow-control/) on a bounded buffer, and what happens when it
  fills.
- [ZFN-60](/zfn/60-graceful-shutdown-is-a-protocol/), where the buffer is part of the drain.
- [ZFN-58](/zfn/58-errors-are-part-of-the-contract/) and
  [ZFN-25](/zfn/25-read-your-writes-version-token/), the two contracts buffering quietly changes.

## Changelog

- **2026-08-19**: First published as a Field Note.