Styxis
Company
Knowledge

Two active SCD rows in Trino? Inspect non-key statistics in the delete file

Pruning an equality-delete file using payload-field statistics can leave an old version active beside its replacement.

Active-version uniqueness includes the time point

Confusing equality keys with additional payload statistics can violate the single-active-version contract. The key inspected by Truthound should include the as-of time and person identity. Multiple versions across the complete history are not inherently erroneous. [1, 2, 3]

The reported delete-planning omission

The public Trino reproduction adds non-key fields to an equality-delete file. Their statistics appear not to match the query predicate, so the file is excluded from the scan. An SCD row that should be closed remains alongside a new active row. Investigate why delete keys and auxiliary fields were treated as equivalent pruning evidence. [1]

An illustrative as-of acceptance condition

If an illustrative as-of contract permits 1 active version for a person_id, 1 passes and 2 fails. First define inclusive and exclusive interval boundaries in the business model. Do not combine rows from different as-of times and call them duplicates, or remove legitimate historical versions. [1, 3]

Inspect the interpreted current state with Truthound

Check active versions by person and as-of time. On violation, inspect separation of equality IDs and additional delete-file fields. Truthound Core's MultiColumnUniqueValidator can verify the materialized time–person composite key. It does not repair delete-file pruning. Compare an isolated non-key-field control with actual queries while preserving the original history. [1, 2, 3]

Application conditions

Materialize as_of_date, person_id and active_version_id from the same snapshot after approving inclusive and exclusive validity boundaries.

Alternatives

Inspect reader applicability with a control delete file whose non-key fields are isolated.

Limitations

  • The business model must define validity boundaries. Truthound does not change Iceberg delete-file pruning.

Evidence

Event checked
Subject checked
Modified