Styxis
Company
Knowledge

Inspect nullable and comment metadata when Delta inventory schemas are rejected

Investigate a VACUUM inventory rejection by separating dtype mismatches from field metadata differences such as nullability and comments. Preserve the original schema and keep DRY RUN review separate from deletion approval.

Documented and full-schema contracts

Separate the documented inventory columns from the full schema equality an implementation uses. Truthound Core can inspect a prepared name-and-type contract while nullable and comment metadata remain visible diagnostic evidence. [1, 2, 3]

Nullable and comment rejection

Delta issue #7735 reports VACUUM USING INVENTORY rejecting correctly named and typed path, length, isDir, and modificationTime columns because of nullable or comment metadata. Its version fields are blank. Follow-up discussion also notes order sensitivity and a NULL-first UNION control. [1]

Review through DRY RUN

Record all four fields’ name, dtype, nullable flag, and comment. Classify a genuine type mismatch separately from metadata-only rejection. Review compatibility through DRY RUN and preserve the original inventory; this diagnosis is not permission to delete files. [1, 3]

The four required pairs

Declare required pairs path:string,length:long,isDir:boolean,modificationTime:long. A complete match yields true and passes only that documented contract. length:string yields false and fails before metadata is considered. A true result does not promise VACUUM accepts the full schema. [1, 3]

Inspect an extracted schema summary

RowLevelValidator in Truthound Core Python SDK 3.3.2 can apply a supplied predicate to an extracted schema summary. The operator supplies field metadata and documented expectations. The SDK does not strip Spark StructField metadata or perform deletion. [1, 2, 3]

Check the transformed schema

Inspect the installed version’s actual comparison and compare original versus transformed inventory with DRY RUN. A no-op cast can retain comments; an operator should verify the transformed schema instead of assuming the expression erased metadata. [1, 3]

Explain the right mismatch

A clearer rejection explanation prevents pointless column renaming when the names already match. Keep file eligibility and deletion safety as later, independent questions after schema compatibility is understood. [1, 3]

Application conditions

Prepare names, dtypes, nullability, comments, and order for the four inventory fields.

Alternatives

Inspect the actual comparison through DRY RUN with original/transformed schemas retained.

Limitations

  • Passing the documented columns does not guarantee VACUUM acceptance or deletion safety.

Evidence

Event checked
Subject checked
Modified