Styxis
Company
Knowledge

Polars strict struct casts: check field sets and order instead of success alone

Do not assume strict rejects the same schema changes on every path. Compare the actual nested structure.

An option name is not a schema contract

Maintain an explicit approved field list and order rather than relying on the name strict. A workflow allowing added fields differs from one requiring an identical structure. A successful conversion cannot distinguish those policies by itself. [1, 2, 3]

Different Series and expression boundaries

Polars issue 30471 reports a follow-up case where Series.cast rejects strict struct casts adding or reordering fields while expressions allow them. It distinguishes an earlier fix for dropped and renamed fields. The current scope is the remaining addition and ordering conditions, not a new event for every cast issue in the project. [1]

Flatten nested paths and positions for handoff

Compare pre- and post-conversion nested fields by path and order. Do not call an unapproved field addition successful schema evolution just because its value is NULL. Record allowed changes under an approved revision and retain type and ordering requirements separately. [1, 3]

A 2-field contract receiving 3 fields

Define an illustrative approved struct with 2 fields, a and b. Retaining those 2 passes cardinality. Adding c produces 3 and fails; a NULL value for c does not authorize the addition. Even with 2 fields, a renamed field or reversed ordering can escape a count check, so compare the path set and positions as well. [1, 3]

Truthound set equality is separate from ordering

Truthound Core's DistinctValuesEqualSetValidator can compare distinct values in an externally flattened field-path column with an approved set. This is one part of schema acceptance; count, repetition and order need separate checks. Do not assume a top-level table check automatically preserves nested Arrow metadata. Test an explicitly reconstructed struct with approved fields and ordering at the same boundary. [1, 2, 3]

Application conditions

Flatten struct_path, field_position, field_name and declared_type for the same struct, preserving approved paths and order from the source contract.

Alternatives

Use explicit struct reconstruction with an approved field list and order.

Limitations

  • Do not assume a top-level Truthound schema check automatically preserves all nested Arrow metadata.

Evidence

Event checked
Subject checked
Modified