Checking Delta deletion-vector references when encoded paths disagree
Separate object identity from deletion application when only escaped partition paths return deleted rows.
Narrow the path that returns deleted rows
Validate deletion metadata by the identity of the object it references, rather than by superficial path spelling. A successful delete request does not establish that a reader attached its deletion vector. First establish the vector-to-file link, then separately compare whether the deleted record is excluded from the read result. [1, 2, 3]
Encoded and decoded paths
The Polars report returns deleted id=2 for a partition containing a space, while its unescaped control returns only id=1. It attributes the discrepancy to joining decoded file paths against percent-encoded paths retained in the Delta log. Other readers are controls described by the reporter, not executions performed for this article. [1]
Preserve both representations
Record each original URI and its normalized object key, then verify the data-file and deletion-vector links. Do not approve the path using only partitions without escaped characters. Retaining both forms distinguishes an absent file from a representation mismatch. Also check whether normalization collapses two genuinely different objects onto one key. [1, 3]
A link check is not proof of deletion
For an illustrative contract, define the normalized inventory key as k=a b/part and require its deletion vector to resolve to that object. 0 unresolved links passes this reference condition. Leaving the log spelling k=a%20b/part unmatched produces 1 unresolved link and fails. The expected result comes from the approved inventory; a resolved reference still does not prove that deleted rows were removed. [1, 3]
Where Truthound fits
Truthound Core's ReferentialIntegrityValidator can check references between prepared file inventories and deletion-vector records. Establish the storage system's URI rules before supplying those tables. Repeated decoding can accidentally select another object. After resolving references, compare row identities across readers to check deletion application; the validator does not repair the Polars reader. [1, 2, 3]
Application conditions
Prepare a link table containing data_file_uri, normalized_object_key and deletion_vector_key from the original data or an independent control. Align it with the same run, file and key scope as the observed data.
Alternatives
Rebuild path links using verified storage URI normalization rules.
Limitations
Repeatedly decoding a URI can change the actual key. Establish the storage system's path rules first.
Evidence
- Event checked
- Subject checked
- Modified
