Inspect nullable and comment metadata when Delta inventory schemas are rejected
Investigate a VACUUM inventory rejection by separating dtype mismatches from field metadata differences such as nullability and comments. Preserve the original schema and keep DRY RUN review separate from deletion approval.
Documented and full-schema contracts
Separate the documented inventory columns from the full schema equality an implementation uses. Truthound Core can inspect a prepared name-and-type contract while nullable and comment metadata remain visible diagnostic evidence. [1, 2, 3]
Nullable and comment rejection
Delta issue #7735 reports VACUUM USING INVENTORY rejecting correctly named and typed path, length, isDir, and modificationTime columns because of nullable or comment metadata. Its version fields are blank. Follow-up discussion also notes order sensitivity and a NULL-first UNION control. [1]
Review through DRY RUN
Record all four fields’ name, dtype, nullable flag, and comment. Classify a genuine type mismatch separately from metadata-only rejection. Review compatibility through DRY RUN and preserve the original inventory; this diagnosis is not permission to delete files. [1, 3]
The four required pairs
Declare required pairs path:string,length:long,isDir:boolean,modificationTime:long. A complete match yields true and passes only that documented contract. length:string yields false and fails before metadata is considered. A true result does not promise VACUUM accepts the full schema. [1, 3]
Inspect an extracted schema summary
RowLevelValidator in Truthound Core Python SDK 3.3.2 can apply a supplied predicate to an extracted schema summary. The operator supplies field metadata and documented expectations. The SDK does not strip Spark StructField metadata or perform deletion. [1, 2, 3]
Check the transformed schema
Inspect the installed version’s actual comparison and compare original versus transformed inventory with DRY RUN. A no-op cast can retain comments; an operator should verify the transformed schema instead of assuming the expression erased metadata. [1, 3]
Explain the right mismatch
A clearer rejection explanation prevents pointless column renaming when the names already match. Keep file eligibility and deletion safety as later, independent questions after schema compatibility is understood. [1, 3]
Application conditions
Prepare names, dtypes, nullability, comments, and order for the four inventory fields.
Alternatives
Inspect the actual comparison through DRY RUN with original/transformed schemas retained.
Limitations
Passing the documented columns does not guarantee VACUUM acceptance or deletion safety.
Evidence
- Event checked
- Subject checked
- Modified
