Styxis
Company
Knowledge

Uppercase Doris rowset hex IDs: check collisions, not just parsing success

An identifier defect in empty rowsets warrants a contract check without being overstated as observed data loss.

Preserve one-to-one identity

Identifier parsing needs checks of one-to-one correspondence and round-trip meaning, not just output format. Truthound's uniqueness check can reveal distinct source IDs collapsing into one parsed value. Discarding the original strings makes it harder to distinguish harmless case changes from actual collisions. [1, 2, 3]

The observation concerns empty initial rowsets

The public Doris cloud report describes the FE generating initial rowset IDs in uppercase hex while the BE parser applies lowercase rules, collapsing different IDs. The reporter explicitly states that no data impact was observed because these initial rowsets are empty. Removing that qualification would incorrectly turn the report into an observed data-loss incident. [1]

Separate collisions from semantic preservation

Preserve original, parsed and reserialized ID pairs. Check collisions and semantic equality separately, without describing this case as actual data loss. If an illustrative input contains 2 distinct original IDs, it requires 2 distinct parsed IDs. Collapsing to 1 fails the collision condition, but equal cardinality still requires a per-ID round-trip comparison. [1, 3]

Detection is not recovery

Truthound Core's UniqueValidator can inspect the parsed-ID column. If the same source ID legitimately repeats, prepare one row per original or define duplicate semantics first. The check does not establish historical data impact or successful recovery. Reparse the preserved originals through an approved implementation and inspect their actual references separately. [1, 2, 3]

Application conditions

Link original_rowset_id and parsed_rowset_id within the same tablet_id, comparing collisions on independently identified source-ID rows.

Alternatives

Preserve original hex IDs and compare round trips through an approved parser.

Limitations

  • The source observes an empty initial-rowset ID defect. A Truthound check does not establish data loss or successful recovery.

Evidence

Event checked
Subject checked
Modified