Styxis
Company
Knowledge

Airbyte Postgres Xmin bookmarks: retain rows that commit late

The largest issued transaction number differs from the boundary imposed by older transactions that are not visible yet.

A successful incremental run can leave a gap

An incremental boundary must distinguish the largest issued transaction number from the oldest transaction still not visible. A Truthound check that only verifies increasing bookmark numbers can miss rows that will commit later behind the saved boundary. [1, 2, 3]

Late arrival from an open transaction

The public Airbyte Postgres report describes an Xmin bookmark on the primary advancing beyond a transaction that remains open. A lower-xmin row committed afterwards can be skipped again on the next run while synchronization reports success. This is an incremental-boundary problem, distinct from CTID page movement. [1]

Record the bookmark and safe boundary together

Preserve the saved bookmark and the oldest-active boundary observed at the time. Use a late-commit control to verify key coverage in the next run. Confirm transaction epochs and wraparound handling; plain integer comparison across different epochs is not a safety proof. [1, 3]

An illustrative same-epoch bound

If an illustrative policy sets the safe bookmark ceiling to 990 within one epoch, bookmark 990 passes and 1000 fails. The reference comes from a policy accounting for not-yet-visible transactions, not from the largest observed number. This numerical example does not replace every inclusion-boundary rule of the actual connector. [1, 3]

The Truthound comparison and operational decision

Truthound Core's ColumnPairValidator can inspect the ordering relationship between a prepared bookmark and its approved safe boundary. Snapshot acquisition and boundary-function selection must follow the supported connector version's policy. Truthound does not acquire Postgres snapshots or amend Airbyte state. After choosing a corrected path, verify that late-commit keys actually enter the following synchronization. [1, 2, 3]

Application conditions

Record saved_bookmark and oldest_active_xid for the same epoch and sync_id, comparing later arrival of late-commit keys with the independent source.

Alternatives

Confirm a supported safe Xmin-bookmark policy and run a late-commit control.

Limitations

  • Truthound does not acquire Postgres snapshots or automatically amend Airbyte state. Check supported versions and connector implementation before selecting a boundary function.

Evidence

Event checked
Subject checked
Modified