Styxis
Company
Knowledge

Checking input ownership when IterativeImputer changes the original missing values

If imputation is allowed to change only the output, the source missingness mask must survive. Check that shared-data contract across the call.

Protect the source that another analyst will use

Preserve shared-input ownership and missingness so colleagues who did not choose imputation still access the same source. A plausible imputed result is insufficient if the original array changes too: the next analyst may be unable to distinguish observations from replacements. Checking only the returned value does not establish the team's source-preservation contract. [1, 2, 3]

A report dependent on memory layout

scikit-learn issue 35157 reports that IterativeImputer with keep_empty_features=True can modify a supplied Fortran-ordered float64 array. An entirely missing column becomes 0 in the original, while a pandas read-only view produces a write error. The reporter distinguishes the copying path for a C-ordered array. These are reported input conditions, not evidence that every imputation call modifies its source. [1]

An acceptance record for shared-data reuse

Record an input copy, memory order and missingness mask. Check for unauthorized source changes after the call before approving reuse. Keep source and imputed results under separate references and fix row and column identities for comparison. If the supposed copy shares source memory, the expected baseline may change with it; obtain an independent reference first. [1, 3]

A small missingness-preservation fixture

Suppose an illustrative source cell is missing and only the transformed output may be filled. Expected source missingness is true. Retaining true after the call satisfies preservation. A false mask accompanied by 0 writes an unobserved value back into the source and fails. This mask check does not decide whether imputing 0 is statistically appropriate. A separately authorized operation that changes the source needs a different contract. [1, 3]

Compare extracted states with Truthound

ColumnPairValidator in Truthound Core 3.3.2 can compare pre-call and post-call masks by row and column. Extract actual boolean masks and resolve missing comparison inputs first. The validator does not protect NumPy memory or repair the imputer. Retest a failing path using an appropriate copying or version alternative, while assessing statistical validity and downstream fairness separately. [1, 2, 3]

Application conditions

Prepare row_id, feature_id, original_missing, after_call_missing and caller_owned from raw input and approved references, preserving row identity.

Alternatives

Use a copying path with separate input ownership and retain original data with its missingness.

Limitations

  • Source preservation does not establish the statistical validity of imputed values or downstream fairness.

Evidence

Event checked
Subject checked
Modified