Truthound · Machine learning data validation
Check the alignment of features, labels and evaluation inputs.
20 articles
Before handing off sparse unpenalized regression with a constant feature
Hand off the intercept, constant-feature, and penalty settings together. Distinguish path comparisons from an independently known simple solution.
Trace disappearing LOF outliers through the shared distance buffer
Do not treat a worker-count change as model improvement. Distinguish the retracted pickle explanation from the later BallTree reproducer.
Hand off residuals alongside the rank cutoff in large float32 regression
Record sample size, precision, and cutoff in the same run. Evaluate residual quality alongside weight consistency.
Before using a Crammer–Singer iteration count to trigger retraining
Reconcile joint-solver measurements with reported counts. Do not repeatedly raise max_iter from a warning alone.
Separate category-code magnitude from tree category cardinality
Before changing original labels to satisfy an error message, establish whether the real limit concerns values or their count.
DecisionTree check_input=False: a validation flag cannot replace the missing-value mask
Skipping input checks requires knowing which information the skipped stage normally creates for the algorithm.
scikit-learn class_weight string keys: preserve the type of numeric-looking labels
Parsing a string label as an integer can create a different dictionary key even when the printed characters look identical.
An all-NaN partial_fit batch is not permission to erase accumulated statistics
Treating a feature's lack of new observations as invalidation of earlier statistics can turn later finite inputs into NaN.
Interpreting RFECV support: selected count is not the original feature axis
A support result for one subset is still a vector over all original input features. Do not use an incorrect shape annotation as an aggregation contract.
KBinsDiscretizer on near-constant features: detect a final bin count of 0
Include features with a tiny positive range, as well as truly constant features, in discretization tests.
NCA rejects string labels: preserve class meaning through an encoding workaround
Check supported inputs and a reversible mapping before changing label representation.
Checking input ownership when IterativeImputer changes the original missing values
If imputation is allowed to change only the output, the source missingness mask must survive. Check that shared-data contract across the call.
HGB errors on all-missing features: check observation semantics before filling with 0
Replacing unobserved values with arbitrary numbers to avoid a training failure changes the data's meaning.
Checking that Pipeline.fit_predict delivers X_val in the same preprocessing space
A validation value can remain in its original units while training data is standardized, without an exception. Inspect the final estimator boundary.
FeatureAgglomeration inverse transforms need both container and feature-width contracts
A correctly shaped value can still arrive in a container the inverse transform rejects. Separate execution success from output-column validation.
Routing Bagging X_val with the same feature selection as each sub-estimator
Selecting training columns while forwarding validation metadata unchanged gives the two inputs different feature spaces.
RandomProjection n_components=auto: verify resolved dimensions and output names together
auto describes the request; fitting resolves the actual dimension. Feature names and output containers must use that resolved value.
scikit-learn auc with decreasing uint8 coordinates: use a triangle to check direction
Changing the numeric representation of the same curve should preserve its area.
Adjusted balanced accuracy on a single-class fold: keep undefined states separate from scores
Replacing an undefined adjusted score with an arbitrary finite value changes both metric meaning and the population being evaluated.
DSPy OutputField constraints: checking the generated schema and returned score separately
Structured-output experiments should separate schema generation from post-response business-rule validation.
