How eterDB works

One undo, end to end.

Writes are copied out of the database. A sidecar reads the WAL through logical decoding and records the row before and the row after, for every write to a tracked table, into a metadata store outside your database.
Reads are captured inside it. A read leaves no row behind for a sidecar to find, so a patched engine records each one as (table, primary key) while the query is still running, and streams the set out after commit.
An undo starts as a question. eter preview <txid> builds the dependency graph for that one transaction out of the store: who overwrote the rows it wrote, and who read them.
Clean means nothing else touched those rows. eterDB inverts the recorded before-images into compensating DML and, under --apply, runs all of it against your live database in one transaction.
Dependent means something did. eterDB names each dependent transaction and says whether it provably read the row or only looks like it did. You choose: refuse, reverse them too, or reverse the target alone and let them diverge.
A dropped table has no before-image. That case is a restore instead: a base backup plus WAL replay into a throwaway Postgres, with the object extracted from the copy. The live database stays up throughout.

1. Transactions, and what gets undone

Postgres wraps a bare UPDATE in a transaction for you:

What you write

UPDATE accounts
  SET balance = balance - 100
WHERE id = 7;

What Postgres runs

BEGIN;     ← added
  UPDATE accounts
    SET balance = balance - 100
  WHERE id = 7;
COMMIT;    ← added

The wrapper matters when one action is several writes that have to stand or fall together. A transfer is a debit and a credit. An ORM's transaction block lowers to the same shape, a BEGIN, the writes, a COMMIT.

Postgres discards the whole unit if it fails partway, and keeps all of it through a crash once it commits. eterDB reverses that unit rather than an individual row.

A transfer staged statement by statement, then committed or rolled back.

Running one backwards takes a record of what it wrote and what it read. Two halves of the system capture those separately.

2. The shape of the system

Recording what each query read is the one job that has to happen inside the Postgres process. A read leaves nothing behind once the query finishes, and it is visible only to the backend that ran it (§4). Everything else runs outside the engine: write history in one sidecar, storage snapshots in another, the dependency graph and undo logic in extensions over a separate metadata store.

Reverts are rare and don't have to be fast, so anything not required at commit time stays off the hot path.

Writes (blue) and the read-set (amber) stream out of the engine after commit. The revert path (green) runs on demand. Amber is the only in-engine piece. Hover for detail.

In-engine there is a read-capture hook, an append-only file the reads are written to, per-backend bookkeeping of what has been read, and the functions that apply compensating DML. The dependency graph is derived in the metadata store, at the point an undo is requested, from reads forwarded there after commit.

Revert latency therefore depends on the transaction rather than on the database.

revert time   scales with     candidate read-edges + the txn's own ops
              independent of  database size, history length, txn age

A months-old target on a million-row database costs about what a fresh one with the same fan-out costs.

3. Capturing writes

Undoing a write takes the row as it was before, and Postgres already keeps one. An UPDATE doesn't overwrite the row where it sits. It writes a new version and marks the old one dead, so a row that has just been debited physically holds two versions.

accounts, row id = 7

balance = 900 xmin 42, xmax 57 dead, the before-image

balance = 800 xmin 57, xmax ∞ live

Each version carries two stamps, xmin for the transaction that created it and xmax for the transaction that retired it (∞ while it's live). A reader sees whichever version was live when its own transaction began, so readers and writers never block each other. That is MVCC, and the dead version it leaves behind is what an undo restores.

Each UPDATE appends a version and stamps the old one dead. The reader still sees the old value. VACUUM cleans up.

VACUUM, Postgres's background cleanup, reclaims those dead versions, so a before-image is not still in the heap an hour later. eterDB copies one out at commit time instead. The before/after pairs come from logical decoding, the same stream that drives logical replication, which reads the WAL, the log Postgres writes every change to before it touches the table itself.

The WAL records just enough to redo a change rather than the whole prior row. A true before-image takes REPLICA IDENTITY FULL on the tables you want reversible, one statement per table:

ALTER TABLE orders REPLICA IDENTITY FULL;   -- WAL now carries the whole old row

The replica identity alternates between FULL and the default. Under the default the before-image never reaches the history; FULL carries the whole old row, at the write cost the meter shows.

All of this happens after commit. The transaction commits at normal speed and the history record follows from the WAL, read out of process by a sidecar that appends to the metadata store.

The replication slot needs operational attention. Postgres retains WAL on the sidecar's behalf while it is down, which is what keeps capture gap-free across restarts (exactly-once, tested) and also disk that grows until capture catches up. Watch slot lag when you run the stack yourself.

REPLICA IDENTITY FULL logs the entire old row on every update rather than the key alone. On wide rows or large TEXT/JSONB values that is write amplification, metered in the figure above. A TOASTed value is logged in full every time, so undo restores it even when the update never touched it. The setting is per-table, so a table you never need to undo keeps the default identity.

4. The dependency a backup can't see

B reads a value A wrote, then writes something derived from it:

A:  UPDATE config SET max_qty = 500;         -- writes 500
B:  SELECT max_qty FROM config;            -- reads A's 500
    INSERT INTO orders(qty) VALUES (500);   -- derives an order from it

undo A  →  config reverts, but B's order still says 500   ✗ silently wrong

Nothing outside the engine can reconstruct that link. B holds a result computed from a value that no longer exists, and nothing flags it. eterDB records these read-dependencies so it can undo B as well, or refuse and report it. Reads are the hard part. A write leaves a row behind; a read leaves nothing, so triggers and CDC never see one.

The same undo of A from both viewpoints: what a backup knows about B, and what eterDB knows.

Postgres already does the tracking, in the machinery behind SERIALIZABLE. The default isolation level, READ COMMITTED, is loose and fast. SERIALIZABLE guarantees a result that some serial order of the transactions would have produced, and aborts a transaction with a 40001 error rather than let it break that guarantee.

Deciding what to abort takes knowing who read what, so SSI (Serializable Snapshot Isolation), Postgres's algorithm for SERIALIZABLE, records each read as a predicate lock, a note that this transaction read the rows matching some condition, which blocks nobody. When another transaction writes a version of that data, SSI draws an rw-antidependency edge between the two: A read some data, and B wrote a version of it A didn't see. That edge records the relationship that makes undoing A alone unsafe, so eterDB harvests it.

The patch reads the same tracking in observe mode, under ordinary READ COMMITTED. Queries return what they returned before, with no 40001s and no serialization failures. Read overhead is a few percent and does not grow with concurrency, with no shared-memory tuning (§8). Full SERIALIZABLE, with the usual 40001 retries, is still available.

Predicate locks coarsen under memory pressure, from tuples to pages to whole relations, which can make the graph over-approximate. Coarsening only ever adds candidate edges and never drops a real one, so it costs a review rather than a missed dependency. §5 shows how each edge is labelled.

Naming the row that was read

An edge is only useful if it names a row that can still be found later. Postgres's own name for a row is its physical address, the ctid. A table on disk is a stack of fixed-size 8 KB pages; each page keeps an array of line pointers at the top, one per row version stored in it, and the ctid is which page plus which slot:

ctid = (0, 2)
        │   └── slot 2   line pointer inside the page
        └────── page 0   which 8 KB block

That address is not stable. When a version dies and vacuum frees its slot, an unrelated row can move in, and the same address comes to mean a different row:

An update, a vacuum, and a slot being reused, one step at a time.

Record "row R is at (0,2)", look it up later, and you may read row S. eterDB therefore identifies a read by the row it touched, as (table, primary key), and resolves that key while the read is still happening.

Why the key is resolved at read time

Sidecar-mode writes carry no stable ctid to match against, so a read has to become (table, primary key) at some point, and the timing decides whether the edge survives. Resolving at drain time, by reading the physical tuple slot, is too late: vacuum or HOT-pruning can have reused that line pointer for a different row by then, so the key resolves to the wrong row, the read-edge is lost, and a dependent transaction is reported clean.

The hook instead copies the tuple into a per-transaction stash the moment it is read, and resolves the key from that copy at PRE_COMMIT, where vacuum cannot reach it. Copies are kept for index-path reads, typically few rows; a bulk seqscan is captured coarsely at table level, so the stash stays small.

An index-only scan over an all-visible page takes a second rule. It answers from the index and never touches the heap (Heap Fetches: 0), so there is no tuple to stash and the read's only trace is a predicate lock on the index. The index scan is hooked as well, and rows an index returns are stashed and resolved the same way, so a read through a covering index is captured as precisely as a heap read. An index that returned nothing falls back to a table-level edge.

test/false-clean.sh is the adversarial harness that holds both rules. rw-edge recall across its seeds is 100%.

5. Computing an undo

eter preview <txid> builds the graph from the store and classifies the target. If nothing depends on it, the transaction is clean: eterDB generates the inverse DML from the captured before-images and, with --apply, runs it against your tables. If later transactions read or overwrote those rows, it's dependent, and a blind revert would corrupt them, so eterDB reports the affected set and gives you the choice.

History is in a separate store, so an undo spans two databases. The part that has to be atomic stays in one: eterDB reads the plan and before-images from the store, then applies all the compensating DML to your database in a single transaction. A crash mid-undo therefore either fully commits or fully rolls back, and inverse DML that cannot apply, say because a unique constraint added since would reject the restored row, aborts the transaction and leaves your tables untouched.

--apply re-runs classification inside the applying transaction rather than trusting the preview, so a dependent that committed since you previewed blocks a clean_only undo. Concurrent writers to the same rows queue behind ordinary row locks. A concurrent reader does not block, and can read a pre-undo value mid-revert; that read is captured like any other and surfaces as a dependent afterwards.

The three undo modes in turn: which transactions get reverted, skipped, or left to diverge.

A cohort undo (eter undo-cohort) reverses every write since an incident marker. One transaction with a live dependent doesn't abort the batch: eterDB reverts the clean ones and returns reverted_txns, skipped_dependent and skipped_txids for you or the agent to decide about the rest. Every command supports --json and returns stable exit codes.

Reading a blocked undo

A dependent classification comes with the set behind it. A config write (txid 4821) was read by later transactions before you caught it, and preview returns each of them:

$ eter preview 4821 --json{
  "txid": 4821,
  "classification": "dependent",
  "op_count": 1,
  "ops": [
    { "table": "public.app_config", "original_op": "U",
      "compensating_op": "UPDATE", "pk": { "key": "max_order_qty" } }
  ],
  "conflicts": [4822, 4825, 4830],
  "conflict_edges": [
    { "txid": 4822, "kinds": ["rw"],       "precision": "exact",       "rw_granularity": "tuple" },
    { "txid": 4825, "kinds": ["rw"],       "precision": "over-approx", "rw_granularity": "relation" },
    { "txid": 4830, "kinds": ["ww","rw"],  "precision": "exact",       "rw_granularity": "tuple" }
  ],
  "precision": {
    "exact_dependents": 2,
    "over_approx_dependents": 1,
    "coarse_tables": ["public.order_audit"]
  },
  "external_refs": { "count": 0, "kinds": {}, "samples": [] },
  "dependency_basis": "write-write exact + read-write from persisted SSI graph"
}

On a busy database, one write to a hot row such as a config or a feature flag is read by many later transactions, and the dependent set grows fast. Two things keep it from collapsing into "everything depends on everything".

Every dependent is labelled. exact means it provably read the reverted row. over-approx means it only looks dependent because a writer seq-scanned the table and SSI coarsened the lock to relation level (the dashed edge above), so those are candidates to review. coarse_tables names the tables driving them; index the columns those writers scan and the reads become tuple-precise.
You choose how far to reverse. clean_only, the default, refuses and reports when dependents exist. cascade reverses the dependents too, newest-first. targeted reverses this transaction alone and leaves the dependents to diverge.

eterDB reverses database state. A reverted row that already charged a card or sent an email surfaces under external_refs in the same preview, so the external effects are in front of you before you decide.

6. Schema changes and time travel

Row-level undo cannot restore a dropped table. A DROP is not row data, the heap is gone, and no before-image in eter.history covers it, so destructive DDL takes a storage answer.

The storage sidecar is a point-in-time-recovery engine: WAL archiving, replay to an exact position, standing up a throwaway Postgres, extracting one object. Its base image is a pg_basebackup physical backup, taken alongside the WAL archive. Recovery copies the newest backup from before the change, replays archived WAL forward to just before the drop, and pulls the object out of the copy.

The deploy drops the table. The database is read as of a moment before it. The table comes back from a base backup plus WAL replay.

Every recovery reads from a copy, and the live database stays up and is never restored over. An event trigger records destructive DDL in eter.ddl_log with the WAL position it happened at, so eterDB knows how far to replay: one instant before the drop.

The substrate is standard Postgres PITR. There is no page store, and destructive DDL is never rewritten inside the database as a rename-to-tombstone plus a view. Either would make the production database harder to reason about.
The backup is only a base. WAL replay covers everything after it, so backup cadence bounds how long a recovery replays rather than how much it recovers. Writes made after the last backup and before the drop come back too.
As-of reads restore nothing. Read a table as it was before a bad deploy, then revert the deploy's writes; unrelated writes in the same window survive.
Recovery costs minutes on a large database, and backups are full physical copies rather than shared blocks. In exchange the storage tier runs anywhere containers run.
History is never pruned. eter.history is append-only, and base backups and WAL segments are retained by default, so a months-old restore still works. History grows by one row image per write to a tracked table; backups and WAL can be bounded with ETER_BACKUP_RETAIN_COUNT or ETER_BACKUP_HORIZON_DAYS.

7. Deploying it

eterDB is the Postgres, a patched engine build with observe mode and the eter_ssi background worker. What it takes to run:

Any container runtime: Kubernetes, Cloud Run, Fargate, Docker Desktop on a Mac.
One storage volume, shared by the engine (WAL archive) and the storage tier (base backups).
Ordinary unprivileged containers. No root, no kernel module, no privileged mode.

Two containers sharing one volume. The engine has the patched build baked in, so you never compile Postgres. It runs with no history trigger on your write path, and pre-creates the capture slot on first boot, which keeps change capture gap-free even if the rest of the stack starts late.

The control plane holds everything else behind one supervisor: the capture sidecar from §3, which the stack requires rather than offers, reading the slot out of process; the orchestrator, carrying the storage machinery; and a separate metadata store for history, the dependency graph and the backup catalog. That metadata has to survive the tenant failing, so it is a database of its own even inside the one container. The CLI and agents talk to a single URL, the orchestrator's API, and recoveries run there as managed background jobs, queued one at a time.

The patch is nine files and about 390 lines, tracked against upstream. The isolation and regression suites pass on the patched binary, and thirteen extensions, pgvector included, build against it and pass their own suites. It targets one major at a time, today 18.

You run it yourself, on anything that runs containers or plain processes: the engine with wal_level=logical and a replication entry in pg_hba.conf (base backups arrive over the replication protocol), plus the control plane next to it. Apps connect over the normal Postgres wire protocol with any driver.

8. Trade-offs and overhead

Capturing writes is nearly free and off the commit path (§3). Observe mode costs about 3% of read throughput, flat from 16 to 100 concurrent clients, and about 13% on writes.

It is that cheap because none of it is shared between backends. Postgres keeps predicate locks in shared memory so that another backend's write can detect a conflict against your reads. Observe mode never needs that: nobody reads another backend's locks, and dependency matching happens later, out of process, by (table, primary key). The read-set stays in per-backend memory, with the same coarsening and promotion logic, and is streamed out at commit.

Throughput overhead by capture path, at 100 concurrent clients.
Path (@100 clients)	Overhead
Write capture (logical decoding)	~10%, off the commit path
Observe, writes	~13%, flat
Observe, reads	~3%, flat

The workload is pgbench point-select traffic (-S) at 16 to 100 clients against the production-shape (assertions-off, -O2) build, observe on vs off on the same binary, interleaved and order-controlled on one warm cluster. The harness is test/observe-read-scaling.sh. The residual on writes is the work of streaming the read-set out at commit: a read-heavy workload with observe on writes tens of thousands of read-set records to disk.

The completeness harness holds 100% read-edge recall across its adversarial seeds, and the patch passes the isolation (119/119) and regression (231/231) suites on PG 18 with observe off.

Observe mode serializes parallel query. Capture is in one backend and a parallel worker's reads would bypass it, so with observe on the planner runs every query without parallel workers. Point reads and OLTP writes never notice. A big analytical aggregate that used four workers does, and for that workload the lost parallelism dominates the 2-3%. It is settable per role, so a BI tool or a read-only analyst whose reads need no capture turns observe off for itself and keeps full parallelism. Capturing from parallel workers is future engine work.

The first implementation, and what made it slow

Observe mode originally reused Postgres's own machinery whole. SSI already tracks who-read-what to enforce SERIALIZABLE, so the first version registered a serializable transaction under each query, let stock SSI take the predicate locks, and harvested them at commit. The graph it produced was correct, at ~27% read overhead at 16 clients and ~53% at 100.

Profiling a busy backend under a pure read workload put taking the locks at 0.2%. Two-thirds of the backend's wall-clock sat in semop at commit, asleep on one cluster-wide lock. The teardown was the cost: registering that hidden serializable transaction meant unlinking it at commit, which took SerializableXactHashLock exclusively every time. At tens of thousands of commits a second every backend queued behind every other, and that wait is what grows with concurrency. The registration, the shared lock table and the commit-time teardown all came out, which is how the read-set came to be backend-local.

Read overhead against client count, shared-SSI capture and backend-local capture. Same binary, observe on vs off, interleaved and order-controlled on one warm cluster.

9. What's next

More profiling: mixed read/write workloads, larger datasets, longer runs, and the write path's remaining lever, moving commit-time read-set emission off the backend entirely.
Testing at realistic parameters. The recovery suite already runs against real application schemas, an inventory system at ~1M rows with months of backdated history and a CRM with cross-table cohorts. Next are bigger databases, longer histories, higher churn and more concurrent writers.
Per-source scoping of the dependency graph, keeping reads from a read-only analytics role out of it, so the dashboard that scans every table doesn't surface as a dependent of every write. Today's dial is §5's exact/over-approx split plus the index hint.
Storage-compute separation. A restore is a replay today (§6), so recovery time grows with how far back you reach. A page server in the Neon mold, retaining every version of every page, would make "this table as of 13:59" a lookup and drop recovery from minutes to seconds. It is a large piece of engineering and comes after the launch.

Try it.

Get Started

Star