Passing IATI format validation can still produce a different quality rating
Maintain format conformity separately from timeliness, coverage and comprehensiveness of the information.
Two checks can disagree legitimately
An IATI file may follow its format while omitting activities or carrying information too old for a user's question. A lower quality rating does not, conversely, establish an invalid file format. A Truthound integration should preserve these two conclusions rather than compress them into one pass field.
What the draft changes
IATI opened the third Reimagining Data Quality consultation on October 23, 2026. The draft covers Availability, Timeliness, Coverage and Comprehensiveness and will underpin a dashboard planned for the 2nd quarter of 2027. Indicators, thresholds and weights remain subject to consultation.
Define the expected population first
Define what should exist before calculating a score. The expected activities of a publisher and required fields within already published activities are different populations. Check whether an indicator applies equally to every publisher. Automatically using all rows as the denominator can collapse coverage and comprehensiveness into the same misleading statistic.
Separate absence from non-observation
No activity in a period, data not yet published and a collector unable to read data need separate states. Filling all three with 0 activities makes the rating difficult to explain. Preserve observability alongside business absence, then record which rows were eligible for each calculation.
The narrow Truthound contribution
Truthound Core Python SDK 3.3.4, operating on a prepared Polars table, provides CompletenessRatioValidator for a supplied minimum non-null ratio and ConditionalExpressionValidator for a caller-defined applicable population. These do not automatically implement every draft dimension. Use them for the part of the contract whose population, null meaning and denominator the organisation has explicitly defined.
Where pandas and the official Validator remain
An existing pandas isna path may remain appropriate. Align the indicator with its handling of empty strings and infinity, which are not default NA values. Keep official IATI XML format validation with the IATI Validator. A generic table check and a specialised format validator answer different questions; neither should be presented as a replacement for the other.
Compare without lowering the threshold
Do not lower a threshold merely to turn a disappointing score into a pass. Apply the proposed indicator version and the internal operating criterion to the same input, then explain the difference. An internal result is not an official rating, and a consultation draft should not be labelled an adopted rule.
Explain a daily score change
When a daily score changes, determine whether the data improved, observation became possible, or the indicator definition changed. Preserve file receipt time, the activity's own update time and the calculation version. Reproducibility requires this history; it does not require deleting records when their evidence becomes older.
Keep two linked assessments
The useful output is two linked assessments: format conformity and the information dimensions relevant to users. Show which activities can be read and what remains missing. Explaining the gap between those assessments produces a more actionable improvement request than a single impressive grade.
Conditions
Define the population and observation time for each indicator in a table extracted from IATI data.
Alternatives
Keep existing pandas isna checks and the official IATI Validator within their different scopes.
Limitations
Draft indicators and thresholds are not final official ratings; one missing-value ratio cannot establish overall usefulness.
Evidence
- Event checked
- Subject checked
- Modified
