Two active SCD rows in Trino? Inspect non-key statistics in the delete file
Pruning an equality-delete file using payload-field statistics can leave an old version active beside its replacement.
Active-version uniqueness includes the time point
Confusing equality keys with additional payload statistics can violate the single-active-version contract. The key inspected by Truthound should include the as-of time and person identity. Multiple versions across the complete history are not inherently erroneous. [1, 2, 3]
The reported delete-planning omission
The public Trino reproduction adds non-key fields to an equality-delete file. Their statistics appear not to match the query predicate, so the file is excluded from the scan. An SCD row that should be closed remains alongside a new active row. Investigate why delete keys and auxiliary fields were treated as equivalent pruning evidence. [1]
An illustrative as-of acceptance condition
If an illustrative as-of contract permits 1 active version for a person_id, 1 passes and 2 fails. First define inclusive and exclusive interval boundaries in the business model. Do not combine rows from different as-of times and call them duplicates, or remove legitimate historical versions. [1, 3]
Inspect the interpreted current state with Truthound
Check active versions by person and as-of time. On violation, inspect separation of equality IDs and additional delete-file fields. Truthound Core's MultiColumnUniqueValidator can verify the materialized time–person composite key. It does not repair delete-file pruning. Compare an isolated non-key-field control with actual queries while preserving the original history. [1, 2, 3]
Application conditions
Materialize as_of_date, person_id and active_version_id from the same snapshot after approving inclusive and exclusive validity boundaries.
Alternatives
Inspect reader applicability with a control delete file whose non-key fields are isolated.
Limitations
The business model must define validity boundaries. Truthound does not change Iceberg delete-file pruning.
Evidence
- Event checked
- Subject checked
- Modified
