Styxis
Company
Knowledge

Dask isin with string and integer codes: identical display text can select different entities

Compare mixed-object filtering against the original combination of type and value.

Dask isin with string and integer codes: identical display text can select different entities

A filter's reference is the original type and value, not only displayed text. When string and numeric codes identify different entities, treating them as equivalent can reverse selection. Do not discard source types because the same digit appears on a screen. [1, 2, 3]

Scope of the source report

Dask issue 12612 reports mixed-object isin selecting an integer row with a string needle differently from pandas. The author suggests possible string coercion but does not establish the internal conversion path. A difference in this example is not a confirmed root cause for every object column. [1]

What to inspect in the input

Build a membership truth table preserving original types and compare selected row IDs for string and integer needles separately. Save type tags and values before transformation to locate whether the distinction was already lost upstream. Record the reference dtype as well. [1, 3]

A small acceptance example

An illustrative contract distinguishes string '1' from integer 1. A string needle must not select the integer row, so false is the expected result. A true selection mixes different entities and fails. If the business explicitly normalizes both codes to one identity, define the expectation under that separate policy. [1, 3]

The boundary passed to Truthound

ColumnPairValidator in Truthound Core 3.3.2 can compare independently expected type-aware membership with actual selected values. Preserve type distinctions while preparing the table; the product does not establish Dask's conversion root cause. After explicit typing or type-tagging changes, compare the same pair of needles again. [1, 2, 3]

Application conditions

Retain raw values together with their type tags and define membership independently of the output.

Alternatives

Declare types during ingestion and retain both type tags and values for codes that must remain distinct.

Limitations

  • The report does not establish behavior of every object column or confirm the internal coercion mechanism.

Evidence

Event checked
Subject checked
Modified