Field Note 46current

Extract the reusable component from the product — and open-source it

Product work keeps producing general infrastructure — a sync engine, a bus bridge. When it isn't your moat, pull it out behind a clean, zero-opinion boundary; better yet, open-source it. Extraction forces honest design; don't do it before it's earned its generality.

By
Theo Zourzouvillys
Published
Est. reading time
Tags
architectureopen-sourcedesignprocess
My Personal LLM Policy: Extract, not generate

The thinking is mine, whether it’s years old or from this week. What a model does is get it out of my head and onto the page — writing time I’d otherwise never spend, not substance I didn’t have. I read every line, I can defend any sentence, and the errors are mine: the same bar I hold everything here to, model or no model.

TL;DR

In the course of building a product you keep producing things that aren’t the product at all — a sync engine, a bus bridge, an egress proxy, a token service. Some of them are genuinely general infrastructure: just as useful to a stranger solving a different problem. When you spot one, pull it out of the product and give it a boundary of its own — a real component with a stable interface and no reach back into product state, not a folder that happens to be importable. Then take the stronger step: open-source it.

The extraction is the discipline. Forcing a component to stand alone — no opinions about your config format or your logger, no hidden dependency on a product table, an interface a stranger could read — is what makes it good infrastructure instead of a tangle you can only run in one place. Open-sourcing is that same discipline turned up: you cannot smuggle an internal shortcut into something the public can read, and you give a useful thing back to the commons you took so much from (ZFN-45Field Note · currentZFN-45 — Read the standards; better yet, help write themLearn to read standards docs — RFCs, W3C recs — fluently; they're the primary source, not a last resort. Even better, get involved: reading them well makes you a sharper builder, and helping write them is the best protocol education there is.Open ZFN-45 →).

The gate: extract the general part, never the moat, and only once generality is earned — proven by more than one real use, not guessed at on day one. ZFN-31Field Note · currentZFN-31 — Own your components — when you deeply understand the domainOwning your own components rather than generic off-the-shelf services is often the better path as you grow: own what's core, lean on small vetted libraries for the hard parts. LLMs make it attainable at smaller scale — but only when you truly understand the domain, or it hurts.Open ZFN-31 → says build the component when you understand the domain; this note is the next move — when what you built turns out to be general, set it free.

Context

Product code and infrastructure code get written in the same repo, by the same people, on the same afternoon — so by default they fuse. The sync mechanism grows a hard reference to a product table. The bus bridge learns the shape of your domain objects. The “library” reads a global config and logs through your house logger. None of that is wrong while it’s one product; it’s just welded in place. And welded infrastructure is infrastructure you can run in exactly one place, evolve at exactly one cadence, and reason about only with the whole product loaded in your head.

But a surprising amount of what you build is not specific to your product. Streaming Postgres changes to somewhere else, bridging a hardware bus to a web transport, proxying outbound HTTP, minting short-lived tokens — these are general problems wearing a thin coat of your domain. The domain part is yours and should stay yours. The general part underneath wants to be its own thing, and keeping it welded in costs you twice:

  1. The component is worse than it could be. Fused to the product, it never has to justify its interface to anyone, so it doesn’t. It accretes shortcuts, implicit coupling, and “well, in our case…” assumptions until only the original authors can safely touch it.
  2. The product is worse than it could be. It now carries infrastructure complexity inline, indistinguishable from business logic, untestable without spinning up the world, and impossible to reuse the next time you need the same thing — so you build it badly a second time.

Pulling the general part out behind a clean seam (ZFN-22Field Note · currentZFN-22 — Quarantine bad architecture behind an interface, then replace itWhen a subsystem is complex and badly architected, quarantine it at its seam: write a clean adapter interface over the mess so the rest of the system depends on the contract, then build a better implementation behind it and expose the new interface directly.Open ZFN-22 →) fixes both. The component gets an interface it has to stand behind; the product gets a dependency it can reason about, test in isolation, and swap (ZFN-23Field Note · currentZFN-23 — Rewriting an implementation is fine — refactoring isn't always the answerRefactoring isn't always right. When the structure is wrong at the root, it's fine — often better — to rewrite an implementation from scratch. Clean interfaces and data models make the implementation disposable: stable contract, swappable internals. LLMs make it cheaper still.Open ZFN-23 →).

So why go the extra distance and open-source it, rather than just extracting it into an internal package? Because the open version is held to a higher standard, and that standard flows back into your own use of it. A public component cannot quietly depend on a private detail — the boundary has to be real or it doesn’t build for anyone else. It gets read by people who don’t share your assumptions, which surfaces the assumptions. It earns issues, patches, and ports you’d never have written. It decouples from your product’s release cadence and lives on its own. And — not least — it’s how the craft compounds: nearly everything you build sits on a mountain of other people’s open source, and contributing a genuinely useful piece back is the same instinct as participating in standards work rather than only consuming it (ZFN-45Field Note · currentZFN-45 — Read the standards; better yet, help write themLearn to read standards docs — RFCs, W3C recs — fluently; they're the primary source, not a last resort. Even better, get involved: reading them well makes you a sharper builder, and helping write them is the best protocol education there is.Open ZFN-45 →, ZFN-33Signal · currentZFN-33 — Are LLMs swinging us away from prepackaged services?A signal: LLMs may be swinging build-vs-buy away from prepackaged services. A generic complex service used to beat building until real scale; now building what you need is cheap. Maybe open source becomes shared architectures an LLM implements per user — like C replaced ASM.Open ZFN-33 →).

Recommendation

Pull the general infrastructure out behind a clean boundary, prove it on more than one real use, and open-source the part that isn’t your moat.

  • Find the seam, then promote it. Look for the place where a chunk of code stops being about your product and starts being about a general problem. Put it behind an interface first (ZFN-22Field Note · currentZFN-22 — Quarantine bad architecture behind an interface, then replace itWhen a subsystem is complex and badly architected, quarantine it at its seam: write a clean adapter interface over the mess so the rest of the system depends on the contract, then build a better implementation behind it and expose the new interface directly.Open ZFN-22 →); once it has proven it can stand there, promote that seam to a real published boundary — its own package, its own repo, its own versioning.
  • Extract to a zero-opinion core. The extracted core should assume nothing about your world: no global config, no house logger, no product types, no network it didn’t ask for. Take laredo’s shape — a core that holds the engine and the interfaces and nothing else, optional modules for the concrete wiring, and a ready-to-run service on top. Opinions belong at the edges, never in the core.
  • Make it embeddable first, a service second. Ship the library other people (including future you) can call in-process, and then a batteries-included binary on top for the people who just want to run it. A service-only component is a component nobody can compose. Client libraries for the common languages are part of “embeddable,” not a nicety.
  • Wait for earned generality. Don’t extract on day one. A component pulled out before it’s been used in anger more than once is a generic abstraction shaped by a single example — it fits nothing well, and you’ll fight the wrong-abstraction tax for years. Two genuinely different real uses is the signal that the generality is real and not imagined.
  • Keep the moat welded in. Open-source the generic mechanism — the sync engine, the bus bridge — not the domain logic, the customer-specific assumptions, the data, or anything carrying embedded secrets (ZFN-35Field Note · currentZFN-35 — Reference secrets in config; dereference, refresh, and re-fetchDon't put secret values in config — store a reference (a path in a secret store) and dereference it at runtime via your workload identity. Refresh on a signal or expiry so rotation needs no redeploy; re-fetch on auth failure so a rotated secret self-heals.Open ZFN-35 →). If a piece is your actual competitive advantage, keep it; the test for “open-source it” is general and not differentiating.
  • Use the standard inside it. Extracting doesn’t mean inventing — the component should speak the standard protocols and formats, not a homegrown dialect (ZFN-30Field Note · currentZFN-30 — Use the standard; don't reinvent the protocolWhen a standard exists for a common or complex problem, use it — don't reinvent the protocol. Standards encode huge adversarial expertise, especially in auth and crypto; a partial implementation beats rolling your own. You're not that special, and your problem isn't either.Open ZFN-30 →). You’re sharing a good implementation, not a new wheel.
  • Treat it as a real project, not a code dump. A README that says what it is and isn’t, a license, documented interfaces (ZFN-1Field Note · currentZFN-1 — Keep engineering decision recordsRecord significant engineering decisions as short, versioned markdown files — context, decision, consequences. Write one for cross-team contracts, directional principles, hard-to-reverse choices, and conventions others must follow. Cite them instead of re-arguing.Open ZFN-1 →, ZFN-14Field Note · currentZFN-14 — Define every API with a schema, and generate the clientsDefine every API with a machine-readable schema (OpenAPI, Protobuf, GraphQL) as the source of truth, and generate clients and server stubs from it — never hand-roll request-building and JSON parsing. Hand-written clients drift and break silently; check schema compatibility in CI.Open ZFN-14 →), versioned releases, and a way to report security issues. Open-sourcing a thing nobody can understand or trust helps no one — including you.

Consequences

Easier:

  • The component gets better the moment it has to justify its boundary to a stranger: cleaner interface, no smuggled coupling, a surface you fully understand because it has nowhere to hide.
  • The product gets simpler — infrastructure complexity moves behind a dependency you can test in isolation, evolve on its own cadence, and replace (ZFN-23Field Note · currentZFN-23 — Rewriting an implementation is fine — refactoring isn't always the answerRefactoring isn't always right. When the structure is wrong at the root, it's fine — often better — to rewrite an implementation from scratch. Clean interfaces and data models make the implementation disposable: stable contract, swappable internals. LLMs make it cheaper still.Open ZFN-23 →).
  • You stop rebuilding the same general thing badly each time it comes up; there’s now one good version.
  • You get external eyes, bug reports, and contributions you’d never have produced alone — and you give a genuinely useful thing back to the ecosystem you build on top of every day.

Harder:

  • A public component is a public commitment. You now own an API other people depend on, an issue tracker, releases, docs, and — if you’re lucky enough to get users — a community and a support expectation. “Just extract it internally” carries a fraction of that weight.
  • The boundary discipline is real work. Severing every reach-back into product state, every implicit config and logger, is more effort than leaving the code welded in — that effort is the point, but it is effort.
  • Extract too early and you ship the wrong abstraction; extract the wrong thing and you give away your moat. Both are real mistakes, and the judgement of which part is general and non-differentiating is yours to get right each time.
  • Not everything deserves this. Plenty of code is genuinely product-specific, or general-but-trivial, and dragging it into its own open repo is ceremony with no payoff.

New obligations:

  • Once it’s public and someone depends on it, you owe them basic stewardship: don’t break the interface carelessly, respond to security reports, and be honest in the README about what’s maintained and what isn’t. If you can’t carry that, keep it internal — an abandoned public dependency is worse than no public dependency.

References

  • laredo — real-time PostgreSQL data sync; the core / optional-modules / service split this note argues for.
  • lplex — an NMEA 2000 bus bridge with an embeddable Go core; the reusable heart pulled out of a product and shared.
  • ZFN-31Field Note · currentZFN-31 — Own your components — when you deeply understand the domainOwning your own components rather than generic off-the-shelf services is often the better path as you grow: own what's core, lean on small vetted libraries for the hard parts. LLMs make it attainable at smaller scale — but only when you truly understand the domain, or it hurts.Open ZFN-31 → — build the component when you understand the domain; this note is what to do after you’ve built one that turns out to be general.
  • ZFN-22Field Note · currentZFN-22 — Quarantine bad architecture behind an interface, then replace itWhen a subsystem is complex and badly architected, quarantine it at its seam: write a clean adapter interface over the mess so the rest of the system depends on the contract, then build a better implementation behind it and expose the new interface directly.Open ZFN-22 → — the mechanics: put it behind a clean interface first; this note is about promoting that seam to a published boundary.
  • ZFN-30Field Note · currentZFN-30 — Use the standard; don't reinvent the protocolWhen a standard exists for a common or complex problem, use it — don't reinvent the protocol. Standards encode huge adversarial expertise, especially in auth and crypto; a partial implementation beats rolling your own. You're not that special, and your problem isn't either.Open ZFN-30 → — extract a good implementation; don’t invent the protocol it speaks.
  • ZFN-45Field Note · currentZFN-45 — Read the standards; better yet, help write themLearn to read standards docs — RFCs, W3C recs — fluently; they're the primary source, not a last resort. Even better, get involved: reading them well makes you a sharper builder, and helping write them is the best protocol education there is.Open ZFN-45 → — the same instinct, one level up: stop being a pure consumer of your field and contribute back to it.
  • ZFN-33Signal · currentZFN-33 — Are LLMs swinging us away from prepackaged services?A signal: LLMs may be swinging build-vs-buy away from prepackaged services. A generic complex service used to beat building until real scale; now building what you need is cheap. Maybe open source becomes shared architectures an LLM implements per user — like C replaced ASM.Open ZFN-33 → — where open source may be heading: shared architectures and reference cores, not just monolithic implementations.

Changelog

  • 2026-06-13: First published as a Field Note.