Styxis
Company
Knowledge

Reconcile Welch ANOVA significance after a change of scale

Retain the original data and common scale. Do not selectively report a decision flipped at a numerical boundary.

A decision that units should preserve

A test invariant to a common nonzero scale should retain its decision when only the unit changes. Check against an independent reference rather than selecting a preferred outcome. The mathematical invariance and the assumptions needed for statistical interpretation both belong in the record. Supply Truthound with original-versus-transformed decision comparisons prepared at the same significance level. [1, 2, 3]

The artificial downscaling boundary

SciPy #26146 reports Welch ANOVA overflow for artificial inputs scaled by 2^-511, with a decision flip at 0.05. Follow-ups provide high-precision controls and a weight-normalization fix proposal. They do not establish how frequently this occurs in ordinary data or guarantee a correction in every installed release. [1]

Hold the test conditions fixed

Retain the original data, common scale, and test settings. Reconcile decisions at the approved significance level, holding discrepant results while examining numerical paths and errors. Operators prepare actual and independently defined decisions under the same analysis ID; keep calculation failures separate. [1, 3]

Compare 49/1849 with 0.05

The illustrative reference p is 49/1849, which is less than 0.05 and therefore gives required significance true. An observed true passes. A false derived from p=0.05611184474 fails. Agreement of the decision still leaves accuracy of the p-value itself to be checked. [1, 3]

Decision comparison and statistical validity

ColumnPairValidator in Truthound Python SDK 3.3.2 compares observed_significance and reference_significance prepared at the same significance level. Operators supply the scale, test settings, and reference calculation. Core does not run Welch ANOVA or establish the research design; decision agreement does not prove p-value accuracy. [1, 2, 3]

Disclose rescaling alternatives

Compare verified common rescaling with high-precision references and disclose the original unit and every decision change. Do not silently favor the result that crosses the threshold. Existing independent checks under the same scale conditions can make another comparison redundant. [1, 3]

Application conditions

Retain original data, common scale, test settings, significance level, and the independent decision.

Alternatives

Compare verified rescaling with high-precision references and report every decision change.

Limitations

  • Decision agreement does not establish p-value accuracy, study design, or test assumptions.

Evidence

Event checked
Subject checked
Modified