Styxis
Company
Knowledge

Polars projection after left join_where: a correct column name can hide the wrong source

Distinct field markers and a complete schema check distinguish a wrong column value under a plausible name.

A name alone does not approve field lineage

Join projection must preserve source-field identity even when names and dtypes look plausible. Multiple integer columns can have similar ranges while one supplies another's value. The contract needs the relationship between the requested field and the observed value. [1, 2, 3]

Full collection differs from selecting a subset

Polars issue 30519 reports a lazy left join_where reading the wrong right-side field or retaining an unnecessary column after projection. It distinguishes full collection, selected optimization-disabled controls, eager execution and inner joins. These controls narrow the report's scope; they do not certify every join configuration. [1]

Make fields distinguishable in the fixture

Use different marker values for right-side columns and inspect selected-field lineage and output schema. Coincidentally equal real values can hide substitution, so deliberately distinguish fixture fields. Production reconciliation should also retain source keys and selected-field lineage. [1, 3]

What requesting right_b should return

Define an illustrative requested field with marker right_b. Observing right_b passes that field correspondence; observing right_a fails despite a matching dtype. Separately check for unrequested extra columns. One passing marker does not approve every join key or duplicate correspondence. [1, 3]

Check allowed field-value pairs with Truthound

Truthound Core's ColumnPairInSetValidator can check a prepared result table against approved requested-field and marker pairs. Derive the allowed pairs from source definitions, not by registering the observed wrong output. The validator does not repair a projection optimizer. Recheck both schema and value lineage after choosing an alternative execution path. [1, 2, 3]

Application conditions

Link independent expected_marker and observed_marker to the same joined_key and requested_field, preparing distinguishable markers for different fields.

Alternatives

Materialize the complete join, verify field provenance, then project.

Limitations

  • Coincidentally equal real values can hide column substitution. Truthound does not change the projection optimizer.

Evidence

Event checked
Subject checked
Modified