Styxis
Company
Knowledge

Check the returned drop_encoding object when NetCDF coordinates truncate

Approve in-memory concatenation and final-file coordinates separately. Do not reject a workaround tested without retaining its returned object.

A write contract after concatenation

Reconcile in-memory concatenated values and final NetCDF reread values separately to detect effects of file order on stored string width. A representation inherited from a shorter-coordinate file can conflict with the current text. The displayed dtype alone cannot establish preservation of the original string. Supply Truthound with a table joining coordinates actually reread from the final file to their originals. [1, 2, 3]

The corrected reassignment

Xarray #11250 gives A and BB coordinates that truncate after files are read, concatenated, and written again. Follow-up testing established that drop_encoding must have its returned object reassigned. The reporter confirmed the corrected path worked; do not repeat the earlier unsuccessful test while omitting that correction. [1]

Separate the file boundaries

Retain each file’s original coordinates and input order, comparing values immediately after concatenation and after the final reread. Hold the consolidated archive until reversed-file-order tests pass too. Preserve source files and read the actual output file rather than a cached Dataset. [1, 3]

The original text distinguishes BB from B

The illustrative files contain coordinates A and BB. The second coordinate’s required original text is BB regardless of file order. A reread BB passes; B fails because a character was lost. Matching coordinate counts and plausible dtype labels do not replace comparison of the entire string. [1, 3]

The bounded coordinate comparison

ColumnPairValidator in Truthound Python SDK 3.3.2 compares reread_text extracted by the operator from the final NetCDF file with separately retained original_text. Operators define the encoding requirements first. Core does not update NetCDF encoding; the follow-up confirms the reassignment workaround without proving every internal cause. [1, 2, 3]

Recheck after removing encoding

Test a separate write using da = da.drop_encoding(), explicitly restoring any required encoding policy. Reread and reconcile every coordinate from the final file. Where all files have an explicit compatible string representation and writer roundtrip tests already cover order, an additional order test may add less value. [1, 3]

Application conditions

Preserve the source files and reconcile coordinates reread from the final output file.

Alternatives

Run a separate write and reread test after reassigning the returned drop_encoding object.

Limitations

  • String agreement does not modify NetCDF encoding or establish the semantics of the entire file.

Evidence

Event checked
Subject checked
Modified