Styxis
Company
Knowledge

Before comparing Iceberg Evaluator and SQL NULL filters, choose the policy

If APIs intentionally use different NULL semantics, a result difference alone does not establish an error.

Equivalence requires an approved policy

Evaluate cross-engine filter equivalence only after explicitly selecting the NULL policy. Truthound does not choose the correct logic on the user's behalf. Build the chosen policy's truth table, then check whether API results follow it. [1, 2, 3]

A proposal to align semantics

The public Iceberg proposal describes Evaluator notEq and notIn retaining NULL values differently from filters under SQL three-valued logic. A SQL filter does not treat unknown as true. The source proposes aligning these meanings; it is not evidence of an agreed and deployed fix. [1]

An illustrative NULL decision table

If the illustrative policy selects SQL filtering semantics, keep_row for a NULL comparison must be false. For a NULL input, keep_row=false agrees with the selected SQL filter policy. A true value retains a row whose predicate is unknown and therefore violates that policy. Applying the same verdict to an API explicitly intended to retain NULL could misclassify a legitimate result. Complete the table with NULL, matching and nonmatching inputs separately. [1, 3]

Give Truthound explicit Boolean states

Build the input-state decision table and compare both API and SQL outputs with the selected policy. Do not describe the proposal as an already confirmed bug fix. Truthound Core's ExpressionValidator can inspect the relationship between prepared input states and keep_row. Preserve NULL as an explicit non-null category in the audit data, and write policy-specific predicates when needed. [1, 2, 3]

Application conditions

Record input_is_null, comparison_state and keep_row as explicit non-null states for the same input, using an approved NULL truth table as the independent reference.

Alternatives

Write policy-specific predicates that handle NULL explicitly.

Limitations

  • Truthound does not choose which API semantics are correct. Do not count valid results under a different policy as errors.

Evidence

Event checked
Subject checked
Modified