Styxis
Company
Knowledge

When a filter moved before shift changes transition statistics

Derive previous states from the original sequence. A test expecting only row counts is insufficient to establish operation order.

Previous state depends on order

Previous states must be calculated from the original sequence. Recomputing shift after filtering defines different events, so verify that a later state filter has not crossed that boundary. Operation order belongs in the definition of the transition statistic. Truthound can reconcile a prepared table of raw-sequence transition references and actual aggregates. [1, 2, 3]

The optimizer control narrows the boundary

Polars #29456 reports group_by, shift, and lazy filtering producing an empty transition result while a post-collect filter retains it. Follow-up triage identifies a correct predicate_pushdown-disabled control and later agrees to closure. Closure alone does not establish a fix in every release. [1]

Build an independent transition table

Build a reference transition table from stable row IDs in the original order, with before and after values. Hold differing lazy output while testing a materialized boundary. Define meaningful order, tie-breaking, and the first row’s previous-state policy before comparison. [1, 3]

A reference with two 1→0 transitions

The illustrative sequence [1,1,0,0,1,0] contains 1→0 transitions at positions three and six, giving independent expected count 2. An observed 2 passes; 0 fails. Matching the count still leaves the identity of both transitions to row-level reconciliation. [1, 3]

Counts versus transition identity

ColumnPairValidator in Truthound Python SDK 3.3.2 compares the observed count with independent_transition_count calculated from the raw sequence by the operator. Operators establish before and after values and tie-breaking. Core does not force optimizer order, and counts do not establish which rows transitioned. [1, 2, 3]

Choose a materialization boundary

Materialize and verify the shifted table or compare an approved optimizer configuration using the same row-level transitions. Existing regressions for that operation boundary reduce another audit’s benefit. Do not redefine the reference by running the same suspect optimization. [1, 3]

Application conditions

Build independent before and after states using original order, row IDs, tie-breaking, and a first-row policy.

Alternatives

Compare materialization of a verified shifted table or an approved optimizer configuration.

Limitations

  • Transition counts do not establish row identity or force optimizer order.

Evidence

Event checked
Subject checked
Modified