Styxis
Company
Knowledge

Wait for Doris load visibility across a schema-change barrier

Link COMMITTED loads to visible keys before announcing completion during a schema change. Investigate transaction and version traces instead of treating a visibility timeout as an automatic reason to reload.

Commit differs from visible completion

Commit acknowledgment and reader-visible completion are different states during schema change. Truthound Core can compare expected business keys with an actual visible snapshot before the consumer receives a completion signal. [1, 2, 3]

A shadow-version wait cycle

Doris issue #69410 reports MoW schema change and overlapping loads forming a wait cycle around shadow-tablet version gaps and WAITING_TXN. Follow-up source review identifies a liveness concern rather than demonstrated permanent loss. A restart is not required for the described cycle. [1]

Join transaction and query boundaries

Connect the ALTER watershed transaction with each load’s COMMITTED and VISIBLE states, partition versions, and query snapshot. Check expected keys at the approved observation time. Avoid blindly retrying a COMMITTED load merely because visibility timed out. [1, 3]

The 102 and 202 key set

The selected reported load keys are 102 and 202. Their expected visible set is 102|202. Returning both passes this two-load visibility condition. empty fails even when a COMMITTED response exists; it is not consumer-visible completion. [1, 3]

Prepare the operational reference

DistinctValuesEqualSetValidator in Truthound Core Python SDK 3.3.2 compares an exported key column with the supplied expected set. The operator chooses the observation boundary and records transaction traces. The SDK does not cancel ALTER or disable FE protection settings. [1, 2, 3]

Recheck the same transactions

An operator should review supported recovery procedures and the acceptable schema-change scope, then verify the same transaction IDs become visible. Temporary recovery settings described in a report must not become a permanent unverified remedy. [1, 3]

Lag needs further evidence

Keep elapsed time and version traces to distinguish ordinary lag from a dependency cycle. A later visible snapshot can resolve the acceptance check without establishing the exact underlying cause of an earlier delay. [1, 3]

Application conditions

Join load transactions, ALTER watershed, and the approved query snapshot.

Alternatives

After a supported recovery procedure, check visibility of the same transactions.

Limitations

  • COMMITTED means neither abort nor permanent loss; time/version traces remain necessary.

Evidence

Event checked
Subject checked
Modified