Styxis
Company
Knowledge

Align the Doris ANALYZE count and load-counter baseline

Align the COUNT snapshot and load-counter generation before adding a load delta to ANALYZE statistics. A race in the estimate does not establish duplicated stored rows.

Keep count and baseline boundaries

A count and its incremental baseline need the same visibility boundary. Saving only their values can double-count a load already included in COUNT. Truthound Core can check a prepared fallback estimate against a snapshot-aligned arithmetic reference. [1, 2, 3]

An estimate rather than stored duplicates

Doris issue #69281 and its follow-up source review describe an ANALYZE construction-time baseline combined with a later count. The concern is optimizer fallback cardinality, not demonstrated duplicate stored rows. The follow-up also distinguishes an existing statistics record from an unestablished first post-TRUNCATE record. [1]

Identify loads already counted

Record job construction, count observation, application time, and their load counters. Determine which loads were visible to COUNT before computing a delta. A post-query counter sample is only an approximation if another load becomes visible between the query snapshot and that sample. [1, 3]

Do not turn 150 into 200

Start from 100 rows and add 50 before COUNT sees 150. An aligned baseline of 150 yields 150+(150-150)=150, which passes. Reusing the old baseline 100 yields 200, double-counting the same 50 rows and failing. These values illustrate the aligned contract, not a cluster execution here. [1, 3]

Align counter generations externally

ColumnPairValidator in Truthound Core Python SDK 3.3.2 can compare the exported fallback estimate with an independently aligned reference. The operator supplies version and counter-generation alignment. The SDK cannot synchronize Doris snapshots or tablet statistics reporting. [1, 2, 3]

Collection modes need separate evidence

Compare after direct tablet counts are available and retain the concurrent-load sequence in a focused test. Sampled and partition analysis use different collection paths; this arithmetic witness alone does not certify them. [1, 3]

Describe the statistic accurately

Rejecting the inflated estimate changes planner evidence, not the stored data. Keep that distinction in operational reporting so a statistics race does not become an unsupported claim of duplicate records or query-result corruption. [1, 3]

Application conditions

Align the COUNT snapshot and load baseline by version and counter generation.

Alternatives

Obtain direct tablet counts and controls for each actual collection mode.

Limitations

  • Do not report a fallback-estimate error as duplicate stored rows.

Evidence

Event checked
Subject checked
Modified