Styxis
Company
Knowledge

PostgreSQL decoderbufs resume: a BEGIN can consume a row skip

Count control messages separately from rows sharing an LSN.

What a repeated row means

Count control messages separately from rows sharing an LSN. A re-emitted record is not a new source transaction, so separate delivery attempts from committed business effects. Keeping both in a Truthound reconciliation prevents a retry from becoming a second sale or update. [1, 2, 3]

A control message spends a skip

Debezium #2894 is a code analysis of decoderbufs BEGIN messages consuming lsn_events_processed skips intended for rows. It distinguishes duplicates from loss and the older count=1 behavior from the behavior after the referenced change. No end-to-end reproduction was performed. The BEGIN and first row sharing an LSN are the relevant boundary. [1]

Separate delivery and effects

Build a resume ledger showing BEGIN and three rows in order, and hold the interval if an already processed row creates an additional business effect. Record decoderbufs configuration, offset count and control-message order. A redelivery rejected by the sink does not belong in the committed-effects table. [1, 3]

C is the new effect after A and B

In the illustrative LSN, A,B,C exist and A,B are already processed. Only C may create a new effect after resume, so expected duplicate_business_effects is 0. Reapplying B gives 1 and fails. If the sink rejects the repeated B, the additional-effect count stays 0 and meets this condition. Reconcile all source rows separately. [1, 3]

Compare committed effects

Truthound Core Python SDK 3.3.2 can compare prepared duplicate_business_effects and allowed_extra_effects through ColumnPairValidator with relationship="==". The operator classifies control messages and committed effects, retaining unknown effects separately instead of guessing 0. The SDK does not repair skip logic or WAL processing. [1, 2, 3]

The boundary of duplicate handling

Test control-message handling on a supported target version and retain idempotent consumption based on stable source keys. Delivery duplication can occur without an additional effect. Conversely, an ordinary-looking message count does not justify recovery completion if a business effect was applied again. [1, 3]

Application conditions

Record decoderbufs and offset count, separating control messages, delivery attempts and committed effects.

Alternatives

Test the corrected skip handling and configure sink idempotency using stable source keys.

Limitations

  • The SDK does not repair skips or WAL; the upstream evidence is code analysis rather than a full connector test.

Evidence

Event checked
Subject checked
Modified