Styxis
Company
Knowledge

Debezium IBM i snapshots: did the initial load cover every member that CDC can read?

Reconcile source and snapshot data by member. Matching row counts still require matching keys, duplicate checks and a common reference time.

New changes can hide an incomplete history

A new change arriving from a second IBM i member does not establish that its historical rows were loaded. Reconcile the member scope of streaming separately from the initial snapshot. That distinction also determines where Truthound belongs: compare source and landed data per member instead of treating a healthy total event count as evidence of complete history. [1, 2, 3]

Two different read paths

Debezium #2994 describes an asymmetry in the incubating IBM i connector: SQL name resolution selects the first member for a snapshot, while journal-based streaming covers all members. The public report does not attach complete execution logs. Its explanation therefore does not establish a failure in every IBM i installation or a fix in a particular release. [1]

Keep member identity in the reconciliation

Inventory the files and members, then fix the reference time for the source comparison. Retain member identity and source keys on snapshot rows, and count subsequent CDC records separately. Reconcile source and landed snapshot counts per member; withhold initial-load completion until an approved extraction covers any missing member. [1, 3]

What four rows establish

In an illustrative file, place 3 rows in M1 and 4 in M2, then stop changes. The independent source fixture sets the expected M2 count to 4. A snapshot count of 4 satisfies the count condition, but the same 4 keys and the absence of duplicates still need checking. If M2 has 0 rows, neither a successful M1 load nor later CDC arrivals repair that gap. This defines a reconciliation example, not a connector execution result. [1, 3]

The check Truthound can perform

Truthound Core Python SDK 3.3.2 can compare prepared source_member_rows and snapshot_member_rows using ColumnPairValidator with relationship="==". Expose absent aggregate rows with an external join, and record 0 only when an empty result has actually been established. Guessing that a null observation means 0 turns an unknown state into a misleading result. Calculate and inspect key-set differences and duplicates separately. [1, 2, 3]

Recover the missing scope

A recovery option is a DBA-verified extraction using aliases for individual members and an explicit UNION ALL. Before repeating the entire snapshot, test the missing-member path and reconcile keys at the same reference time. A single-member file or an approved custom query already covering every member requires a different assessment. [1, 3]

The boundary of load acceptance

Truthound does not enumerate members or create SQL aliases. This design passes prepared tables to the Core SDK; it does not claim that Local MCP executes arbitrary rules. Accept the initial scope only after counts, source keys, duplicates and reference time agree. If the necessary source evidence is unavailable, retain the unresolved scope instead of declaring completion. [1, 3]

Application conditions

Fix the member inventory and reference time, counting snapshot rows separately from later CDC records.

Alternatives

Test a DBA-verified extraction with member-specific aliases and explicit UNION ALL.

Limitations

  • Truthound does not enumerate IBM i members or create SQL aliases, and does not certify the incubating connector for production.

Evidence

Event checked
Subject checked
Modified