Styxis
Company
Knowledge

When Iceberg fixed-binary delete keys collapse, inspect buffer ownership

Compare the distinct keys written to the file with those retained in the reader's in-memory set.

Inspection data needs independent ownership

Delete-key cardinality must survive both writing the file and loading the in-memory structure. If the values supplied to Truthound still reference the same mutable buffer, the check inherits the defect. First prepare data that independently owns each key's bytes. [1, 2, 3]

The mutable-array reuse report

The public Iceberg reproduction describes a fixed-binary equality-delete reader reusing a mutable byte array, causing different keys to behave like one set member. Most intended deletes then remain unapplied. Counting keys in the file alone does not establish the reader's actual key set. [1]

Compare the file and loader boundaries

Create independently copied byte keys and compare distinct counts and key sets at both boundaries. If an illustrative approved set contains 143 distinct keys, the loader must retain 143. Collapsing to 1 fails. Matching counts can still hide substituted values, so compare the hexadecimal key set as well. [1, 3]

The Truthound cardinality check

Truthound Core's DistinctCountValidator can compare the distinct count in an independently copied key column with the approved reference. Align observation timing and delete scope between the file and loader. The check does not fix reader memory ownership. A successful independent-copy control is not proof that the operational reader was repaired; verify actual deletion results again. [1, 2, 3]

Application conditions

Prepare delete_file_key_hex and loaded_key_hex as independently owned byte copies for the same delete scope, aligning observation points.

Alternatives

Create independently owned copies of each byte key at the loader boundary.

Limitations

  • Inspection values referencing the same mutable buffer contaminate the check. Truthound does not fix reader memory ownership.

Evidence

Event checked
Subject checked
Modified