Styxis
Company
Knowledge

Pulsar replay after partial batch ACK: check duplicate business effects

An ACK completion signal and one-time business effects after recovery are separate contracts.

Redelivery is not message loss

ACK completion and one-time business effects after recovery are different guarantees. Detecting repeated messages with Truthound does not alone establish that the broker violated its delivery policy. Systems permitting redelivery need a separate check of effects produced by the same message. [1, 2, 3]

A bitmap change without persistence scheduling

The public Pulsar report describes a partial batch ACK updating an in-memory bitmap without marking the state dirty for persistence. An already processed message can then be redelivered after a crash. The report distinguishes that risk from message loss. Preserving this distinction matters when selecting recovery actions. [1]

Retain a stable message identity

Record processing using stable message identity including batch index. After partial-ACK recovery, verify that no duplicate business effect occurred. A delivery-attempt identity alone can make replay look like a new business event. Keep the connection between the original message and effect identity in the ledger. [1, 3]

An illustrative effect contract

For a synthetic operation allowing only one effect, duplicate_business_effects must be 0. A count of 0 additional business effects preserves the effect contract despite redelivery. A count of 1 means processing the same message affected the business operation again. A redelivered message can still pass if idempotent handling prevents another effect. Do not combine delivery count and effect count into one metric. [1, 3]

Separate Truthound from idempotent execution

Apply Truthound Core's MultiColumnUniqueValidator to a ledger containing only business effects that were actually applied. Under a contract allowing one effect per message, message identity and batch index form the composite key. Keep delivery attempts and deduplication rejections in separate logs; including them would misclassify legitimate retries as duplicate effects. Workloads allowing repeated effects require another key contract. The feature does not persist ACKs or provide exactly-once delivery. Validate consumer idempotence and the broker correction separately, then repeat the recovery control. [1, 2, 3]

Application conditions

Prepare message_id, batch_index and business_effect_id as a ledger of actually applied effects within the same recovery scope, keeping delivery-attempt logs separate.

Alternatives

Make business effects idempotent using stable consumer message identities.

Limitations

  • Redelivery can be valid under the delivery policy. Truthound neither persists broker ACKs nor provides exactly-once delivery.

Evidence

Event checked
Subject checked
Modified