Classify missing Hudi clean instants by the sync window
Classify an absent archived clean by whether it belongs to the window required for the current sync. If required evidence is missing, reconcile a verified full partition inventory at the same boundary before declaring completion.
Different absences need different policy
Missing old history and missing input required by the current sync need different error policies. Truthound Core can inspect a prepared required-window classification without treating every archived clean instant as permanently necessary. [1, 2, 3]
A sync and archiver report
Hudi issue #20379 reports meta sync racing with the archiver while reading cleans already covered by earlier syncs. It distinguishes those obsolete records from missing cleans inside the needed window. A follow-up links a correction PR but does not establish the installed version. [1]
Keep listing and opening times
Fix last synced commit and the clean window needed for the next partition update. Record list and open times plus instant identity. If required evidence is missing, obtain a verified full-partition-list comparison rather than silently declaring the sync complete. [1, 3]
One currently required clean
Designate one illustrative clean after last sync as necessary to explain a current partition deletion. Its expected availability=true. true passes this input prerequisite. false fails if ignored like an old 404, because the current deletion calculation then lacks its required evidence. [1, 3]
Classification belongs to the operator
ConditionalExpressionValidator in Truthound Core Python SDK 3.3.2 can require availability for rows classified as currently required. The operator defines that classification and collects the records. The SDK does not stop archiving or synchronize Hive or Glue. [1, 2, 3]
Align the fallback boundary
Validate a supported fallback comparing a full partition list and catalog list at the same boundary. The shape of a wrapped S3 exception cannot identify whether its deletion was already applied. [1, 3]
Permanent history is not implied
Retain enough provenance to explain why an absent instant was irrelevant or essential. Permanent retention of all old instants is not implied by this contract; correct classification of the current input window is. [1, 3]
Application conditions
Fix last sync and the clean window required for the current deletion calculation.
Alternatives
Verify a full-partition-list fallback at the same boundary.
Limitations
Do not infer prior application from exception wrapping alone.
Evidence
- Event checked
- Subject checked
- Modified
