Styxis
Company
Knowledge

Distinguish whole-column format inference failure under strict=False

One invalid row differs from a column with no inferable format. Keep an explicit format beside actual completion status.

Inference precedes per-row failure

Separate per-row parse failures from failure to infer any format for the column. Test completion and missing representation under an approved explicit format. Teams interpreting strict=False at different stages can otherwise record an interrupted run as a successful empty result. Connect Truthound to a missing-flag comparison table prepared after parsing has completed. [1, 2, 3]

The reporter narrows the all-invalid case

Polars #29420 reports an error rather than NULL when no format is supplied and the column contains only bad date. The reporter later corrects an initial misunderstanding: if at least one date supplies an inferable format, other bad rows become NULL. Do not repeat a claim that any invalid date fails the whole column. [1]

Do not relabel failure as successful NULL output

Define allowed formats and record completion, original text, and missing flags. Do not portray an inference exception as normal NULL output; test an explicit-format path. Preserve original text and format, checking completion before inspecting row-level flags. [1, 3]

An independent missing-value expectation

The illustrative explicit format is %Y-%m-%d, which bad date does not match. After successful nonstrict conversion, required missing flag true is the independently defined state. Actual true passes; false fails. If the run raises an exception, do not manufacture a successful flag for comparison. [1, 3]

Compare flags after completion

ColumnPairValidator in Truthound Python SDK 3.3.2 compares actual_null_flag_after_success and required_null_flag prepared by the operator after completed parsing. Operators run the parser and classify errors. Core does not catch parsing exceptions, infer all formats, or guarantee date validity. [1, 2, 3]

Classify allowed formats explicitly

Use an explicit format or classify allowed formats before parsing them separately, retaining failed source text. A regression already covering the all-invalid case can reduce another audit’s value. Keep execution errors distinct from a legitimate column of NULL values. [1, 3]

Application conditions

Check parsing completion first, retaining original text, explicit format, and row-level missing flags.

Alternatives

Test an explicit format or prior classification by allowed formats, retaining failed source text.

Limitations

  • Flag comparison on completed output does not handle parser exceptions or infer every format.

Evidence

Event checked
Subject checked
Modified