Validate a ClickHouse patch recovery through the merge after ATTACH
Seeing the updated value immediately after reattachment is not enough to close a lightweight-update recovery.
Choose the right recovery endpoint
Do not approve patch recovery solely because the first read succeeds. Compare values after the subsequent merge as well. Truthound can serve as a boundary comparing approved updates with stored state; it is not the tool performing the recovery procedure. [1, 2, 3]
The name and index carry different versions
The public ClickHouse reproduction describes ATTACH resetting the mutation version in a patch part's name while its internal patch index retains the earlier version. Reads apply the update, but a later merge involving a pending DROP COLUMN can lose it. Immediate readability and post-maintenance persistence are therefore separate observations. [1]
Take expectations from the update record
Record keys and updated values across DETACH, ATTACH, mutation and merge. Close recovery only after the approved last value still survives the completed merge. Derive expectations from the original update record rather than copying a single post-incident read. Multiple updates to the same key also require an approved ordering. [1, 3]
A value that silently reverts
For an illustrative key whose final approved value is 5, the post-merge observation must remain 5. Reverting to 0 fails the recovery contract even if the query itself succeeds. Apply the comparison to the same key after confirmed merge completion, without mixing in a read from an earlier snapshot. [1, 3]
Truthound and the operator's responsibilities
Truthound Core's ColumnPairValidator can compare non-null approved and observed values joined by key. A column supporting NULL transitions needs a separate Boolean comparison of missingness. Truthound does not repair part names or mutation versions. An operator must select a corrected recovery path after testing subsequent mutation and merge in isolation. [1, 2, 3]
Application conditions
Link each key’s approved update record to its value after merge completion, preserving the execution sequence of DETACH, ATTACH, mutation and merge.
Alternatives
After identifying a corrected part-handling path, test both mutation and merge in an isolated environment.
Limitations
Truthound does not change part names or mutation versions. Obtain expectations from the update log, not a copied post-incident read.
Evidence
- Event checked
- Subject checked
- Modified
