Trace disappearing LOF outliers through the shared distance buffer
Do not treat a worker-count change as model improvement. Distinguish the retracted pickle explanation from the later BallTree reproducer.
Observation decisions that parallelism should preserve
Hold data, distances, and thresholds fixed when checking whether a worker-count change preserves row-level LOF decisions. Similar total outlier counts can conceal different affected observations. Hand off original row identity and distance evidence so another team can continue the diagnosis. Provide Truthound with parallel and reference labels matched by stable observation IDs to inspect handoff differences. [1, 2, 3]
The follow-up beyond the pickle hypothesis
Scikit-learn #32753 reports Mahalanobis LOF labels changing with n_jobs. Follow-ups retract the initial pickle explanation and provide a smaller BallTree reproducer implicating shared self.buffer, with draft #34671. A serialization cast is therefore not an established solution, and a draft is not evidence of a released fix. [1]
Fix VI and row identity
Fix observation IDs, feature order, VI, neighbor count, and threshold. Reconcile parallel labels with a separately reviewed reference path, withholding discrepancies while examining distance transmission and computation. Do not replace row-level reconciliation with an aggregate outlier count. [1, 3]
A bounded reviewed-label fixture at -1
The illustrative reviewed reference for observation O1 is -1. An actual parallel label -1 passes; 1 differs and fails. This is a defined comparison fixture, not a truth label for the original dataset. Agreement can still be wrong if the reference itself has not been validated. [1, 3]
The row-label comparison boundary
ColumnPairValidator in Truthound Python SDK 3.3.2 reconciles parallel_label and approved_reference_label prepared by the operator for matching observation_id values. Distance computation, VI validation, and evaluation of the reference path are external. Core does not run LOF or Mahalanobis distance or establish statistical validity. [1, 2, 3]
Memory costs of distance alternatives
Compare a verified single-worker or separately computed-distance path after evaluating memory cost. Do not call pickle changes or contiguous-array conversion proven repairs. Reconcile actual distances and row labels after any workaround; existing independent coverage can make a separate table unnecessary. [1, 3]
Application conditions
Fix data, VI, observation IDs, and thresholds, separately validating the reference path.
Alternatives
Compare single-worker or separate-distance paths using their memory costs and actual distances and labels.
Limitations
Label agreement does not establish statistical validity or truth labels; the check does not calculate LOF or distances.
Evidence
- Event checked
- Subject checked
- Modified
