Beam OrderedListState: retain both A and B at the same timestamp
A sorting timestamp does not replace event identity; check bucket counts and the actual members.
A sorting key is not an identity
A timestamp sorts events; it does not identify them. If distinct events may share a timestamp, both must survive state readback. Preserve key, bucket and event id in the observations supplied to Truthound instead of collapsing them by timestamp and calling them duplicates. [1, 2, 3]
The limit of a single-value map
Beam #39782 reports overwritten OrderedListState elements with equal timestamps in Java heap state. Its environment is Beam 2.61.0, Flink 1.19.1 and HashMapStateBackend. A comment also identifies the single-value timestamp map limitation. Neither that explanation nor the report establishes a shipped repair or the same loss rate on other backends. [1]
Reduce the test to two appends
Append distinct event ids consecutively at the same timestamp, then compare readRange counts and membership before accepting the backend. Use matching keys and time ranges in the append ledger and readback. Record the count and id-set comparison separately so two copies of A cannot hide a missing B. [1, 3]
Two is not enough on its own
The illustrative key K receives A and B once each at timestamp 1000, giving an expected bucket count of 2. A readback count of 2 meets only the count condition; both A and B must also be present. B alone gives 1 and fails. Two copies of A also fail the id-set check. This is a defined fixture, not a backend execution result. [1, 3]
Compare prepared state observations
Truthound Core Python SDK 3.3.2 can compare readback_bucket_members and appended_bucket_members with ColumnPairValidator using relationship="==". The external test extracts the element lists and id-set differences. The SDK does not modify Flink state or restore elements missing from a checkpoint. [1, 2, 3]
Choose a state representation deliberately
Test a runner change or alternative state representation that explicitly preserves multiple values. A business map keeping only the last value per timestamp has a different contract and is not a valid control for OrderedListState. Acceptance requires counts, identities and a recovery test together. [1, 3]
Application conditions
Match key and time ranges in append and readRange observations, checking event-id sets as well as counts.
Alternatives
Verify a corrected runner or test a different state representation explicitly preserving multiple elements.
Limitations
Truthound does not change the Flink state backend or recover elements lost from a checkpoint.
Evidence
- Event checked
- Subject checked
- Modified
