Styxis
Company
Knowledge

Beam Go INT16 wire changes: inspect the bytes of 999 and existing stored rows

Check both the new cross-SDK wire contract and migration of existing Go bytes.

New compatibility and existing data

Check both the new cross-SDK wire contract and migration of existing Go bytes. A self-roundtrip cannot establish compatibility between different coders. Supply Truthound with writer version, raw bytes and reader results so existing stored rows are included in the migration decision. [1, 2, 3]

Different bytes of the same length

Beam #40151 reports the Go reflection row coder using varint for INT16 while Java and Python use two-byte big endian. For 999, both encodings occupy two bytes but differ in content. A correction changes the persisted Go wire format, so compatibility with existing rows needs an explicit migration assessment. [1]

Compare readers by writer generation

Compare the raw bytes of 999 and readback through Go, Java and Python. Separate existing rows by writer generation before switching. Isolate row headers from value bytes and retain signed or unsigned declarations; equal byte counts are not sufficient evidence of equivalent encoding. [1, 3]

Decompose 999 directly

For illustrative INT16 value 999, 3×256+231 gives canonical 2-byte big-endian hex 03e7. That value passes; e707 fails this value-byte contract. Exclude row headers from the comparison. A matching value encoding still does not establish compatibility of the complete row schema. [1, 3]

Compare after decoding framing

Truthound Core Python SDK 3.3.2 can compare prepared canonical_int16_hex and encoded_value_hex through ColumnPairValidator with relationship="==". The operator decodes framing and normalizes hex for the same value. The SDK does not patch Beam coders or automatically migrate persisted bytes. [1, 2, 3]

Other migration obligations

Pair a migration reader for the old format with cross-SDK tests of the new writer. Reconcile the complete schema, null representation and subsequent fields as well as value bytes. No persisted data may reduce migration scope, but actual exchange tests between SDKs are still necessary. [1, 3]

Application conditions

Separate framing from value bytes and retain signedness and writer generation.

Alternatives

Use a separate migration reader for old Go bytes and cross-test the corrected writer with multiple SDKs.

Limitations

  • The SDK does not patch coders or migrate persisted bytes; value-byte equality does not prove complete-schema compatibility.

Evidence

Event checked
Subject checked
Modified