Styxis
Company
Knowledge

Approve integer missing sentinels and physical-value decoding per variable

A global mask_and_scale setting should not silently decide distinct representation contracts for different variables.

Different meanings changed by one option

Separate preservation of an integer sentinel from scaling of physical observations per variable. This avoids retaining one dtype while losing another variable’s units. A policy that represents missingness with NaN can also be valid; establish the promised representation before judging the change. A table joining approved per-variable sentinel representations with decoded results can supply the Truthound comparison. [1, 2, 3]

A policy question without an adopted change

Xarray #11089 asks about integer values becoming floating values and NaN through missing_value decoding, while globally disabling mask_and_scale also disables scaling elsewhere. A maintainer closed the issue because incorrectness of existing behavior was unclear. This is a policy question, not evidence of an adopted parser change. [1]

A decoding contract per variable

Record each variable’s raw dtype, sentinel, scale, offset, and decoding policy. Compare the required and actual representations and test physical values before globally disabling decoding. Define the policy per variable and retain raw values and metadata for a separate scale check. [1, 3]

An illustrative policy retaining -128

The illustrative integer policy retains -128 as an int8 missing sentinel. The required representation sentinel:-128 comes from that approved policy. An actual sentinel:-128 passes; NaN differs and requires separate approval. This counterexample does not apply unchanged to a policy that explicitly allows NaN. [1, 3]

A sentinel check is not physical-value validation

ColumnPairValidator in Truthound Python SDK 3.3.2 compares operator-normalized actual_missing_representation and approved_missing_representation values. Collect raw integers and decoded results externally. Matching sentinel representations does not establish scale or offset accuracy; Core neither decides the single correct CF policy nor controls parsing. [1, 2, 3]

Responsibilities of explicit conversion

Consider per-variable decoding settings or raw reading followed by explicit conversion, testing both integer sentinels and floating physical values. If the task already approves conversion to NaN, that conversion is not a breach of this contract. Record it as a distinct policy rather than changing expectations after seeing the result. [1, 3]

Application conditions

Retain each variable’s raw dtype, sentinel, scale, offset, and approved decoding policy.

Alternatives

Test per-variable decoding or raw reading followed by explicit conversion.

Limitations

  • Matching sentinel representations does not establish physical scaling accuracy or a uniquely correct CF policy.

Evidence

Event checked
Subject checked
Modified