Check the returned drop_encoding object when NetCDF coordinates truncate
Approve in-memory concatenation and final-file coordinates separately. Do not reject a workaround tested without retaining its returned object.
A write contract after concatenation
Reconcile in-memory concatenated values and final NetCDF reread values separately to detect effects of file order on stored string width. A representation inherited from a shorter-coordinate file can conflict with the current text. The displayed dtype alone cannot establish preservation of the original string. Supply Truthound with a table joining coordinates actually reread from the final file to their originals. [1, 2, 3]
The corrected reassignment
Xarray #11250 gives A and BB coordinates that truncate after files are read, concatenated, and written again. Follow-up testing established that drop_encoding must have its returned object reassigned. The reporter confirmed the corrected path worked; do not repeat the earlier unsuccessful test while omitting that correction. [1]
Separate the file boundaries
Retain each file’s original coordinates and input order, comparing values immediately after concatenation and after the final reread. Hold the consolidated archive until reversed-file-order tests pass too. Preserve source files and read the actual output file rather than a cached Dataset. [1, 3]
The original text distinguishes BB from B
The illustrative files contain coordinates A and BB. The second coordinate’s required original text is BB regardless of file order. A reread BB passes; B fails because a character was lost. Matching coordinate counts and plausible dtype labels do not replace comparison of the entire string. [1, 3]
The bounded coordinate comparison
ColumnPairValidator in Truthound Python SDK 3.3.2 compares reread_text extracted by the operator from the final NetCDF file with separately retained original_text. Operators define the encoding requirements first. Core does not update NetCDF encoding; the follow-up confirms the reassignment workaround without proving every internal cause. [1, 2, 3]
Recheck after removing encoding
Test a separate write using da = da.drop_encoding(), explicitly restoring any required encoding policy. Reread and reconcile every coordinate from the final file. Where all files have an explicit compatible string representation and writer roundtrip tests already cover order, an additional order test may add less value. [1, 3]
Application conditions
Preserve the source files and reconcile coordinates reread from the final output file.
Alternatives
Run a separate write and reread test after reassigning the returned drop_encoding object.
Limitations
String agreement does not modify NetCDF encoding or establish the semantics of the entire file.
Evidence
- Event checked
- Subject checked
- Modified
