Styxis
Company
Knowledge

171 versus 13 in Dask loc: do not force an index mask into positional alignment

Check partition correspondence of the data and index mask at the execution boundary.

The risk of matching counts alone

Check partition correspondence of the data and index mask at the execution boundary. An index expression can acquire a different block count after optimization. Retain boundaries and source row IDs in Truthound inputs; merely reshaping a mask to match counts can silently select different rows. [1, 2, 3]

The scope of a version comparison

Dask #12146 reports index-condition masks with block counts differing from the DataFrame, causing loc failure in a version comparison. Its 171 versus 13 is an observed condition, not a universal required partition count for Parquet or proof of a newly validated alignment repair. [1]

Collect boundaries from the same snapshot

Record actual block identities and counts for data, index and mask. Do not force an unmatched mask into a positional array; choose a verified alignment or partition path. Confirm the same snapshot and retain block boundaries and row IDs, checking selected rows even when using a workaround. [1, 3]

The limit of a count of 171

The illustrative data inventory contains 171 partitions with one mask block per partition. Expected mask count is therefore 171; observed 171 meets only that count condition, while 13 fails. A different set of 171 boundaries still needs separate one-to-one boundary and row-membership checks. [1, 3]

Compare measured tables

Truthound Core Python SDK 3.3.2 can compare prepared data_partition_count and mask_block_count using ColumnPairValidator with relationship="==". The operator measures them at the same snapshot and execution boundary. The SDK does not repair loc or optimization, and equal counts do not guarantee alignment or row identity. [1, 2, 3]

Reconcile rows after removing the exception

Test a verified Series condition, explicit partition correspondence or a small materialized control, then recheck selected rows. Reuse existing evidence for identical conditions. Removing the exception and selecting the correct rows are different completion items. [1, 3]

Application conditions

Obtain block boundaries and row IDs externally and confirm masks and data use the same snapshot.

Alternatives

Test a verified Series condition, explicit partition correspondence or a small materialized control and reconcile selected rows.

Limitations

  • Equal counts do not prove block boundaries or identity; Core compares measurements and does not repair loc or the optimizer.

Evidence

Event checked
Subject checked
Modified