Styxis
Company
Knowledge

Validate Iceberg DynamicSink key changes at checkpoint boundaries

Similar field names do not establish a shared key contract when types, order or optionality change.

Key meaning can change within a checkpoint

One checkpoint needs compatible key semantics. Do not treat a key change as an ordinary column change. A Truthound composite-key check also needs to know the key-contract epoch governing the result. [1, 2, 3]

A follow-up beyond the proposed base fix

The public Iceberg DynamicIcebergSink analysis describes remaining cases after a proposed base fix. Changing key optionality, types, field sets or field order within one checkpoint can prevent tracker sharing and leave duplicate versions. The source investigates remaining correction boundaries; it does not establish that every deployed version is fixed. [1]

Separate transitions at completed checkpoints

Record a key-contract epoch and move changes to a completed checkpoint boundary. Use the same business-key controls before and after the change to verify one live version. First approve whether different field sets denote the same business entity; matching or concatenating names does not establish that equivalence. [1, 3]

An illustrative live-version count

If an illustrative current-state contract requires 1 live version per business key, 1 passes and 2 fails. That result does not decide which key schema is correct. Preserve the checkpoint and key epoch associated with the duplication to investigate the transition. [1, 3]

Truthound and writer reconfiguration

Truthound Core's MultiColumnUniqueValidator can inspect the current-state composite key under the approved contract. Business equivalence across key sets and tracker-sharing feasibility remain external design decisions. Truthound does not stop checkpoints or reconfigure writers, so an operator separately defines the boundary transition and replay scope. [1, 2, 3]

Application conditions

Link checkpoint_id, key_contract_epoch, key_fields and live_business_key to the same completed boundary, separately approving business equivalence of the keys.

Alternatives

Separate key-contract changes at completed checkpoint boundaries.

Limitations

  • Business equivalence across key sets and tracker-sharing feasibility are external design decisions. Truthound neither stops checkpoints nor reconfigures writers.

Evidence

Event checked
Subject checked
Modified