Declare timestamp precision before a DuckDB Parquet glob chooses it
Compare actual timestamp values when changing the first file in a glob. An output dtype alone can make already-rounded timestamps appear acceptable.
Declare the mixed-file precision
A first filename is a weak place to store a timestamp precision policy. Declare the precision for the mixed file set, then use Truthound Core to compare exported subsecond integers with an independent source reference. [1, 2, 3]
A first-file schema report
DuckDB issue #28070 reports that the first file in a mixed-unit Parquet glob changes schema selection and can round nanosecond timestamps to microseconds in 2.0. The follow-up report distinguishes union_by_name and a different truncation behavior in 1.5.5. [1]
Preserve order and units
Preserve file order, physical timestamp units, output dtype, and source values. Add a different-unit file before the original first file and compare both dtype and value preservation. A changed dtype alone does not reveal whether previously read values were already rounded. [1, 3]
A 211-nanosecond difference
The source fraction .123456789 is 123456789 nanoseconds. Preserving 123456789 passes a lossless contract. Rounding to .123457 produces 123457000 nanoseconds, changing the value by 211 nanoseconds and failing that contract. The expected integer comes from the original fraction, not the glob output. [1, 3]
Compare exact integer fractions
ColumnPairValidator in Truthound Core Python SDK 3.3.2 can compare prepared non-null integer-nanosecond columns with ==. Extract units and exact values before conversion. A dtype check cannot recover rounded information, and the SDK does not choose DuckDB file schemas. [1, 2, 3]
Account for schema inspection cost
Compare an explicit target schema with union_by_name and account for the cost of examining all file schemas. Do not assume maximum_sample_files is supported by this read_parquet path. A supported option must be verified for the actual interface and version. [1, 3]
Document accepted rounding
Accept a cheaper read path only if it preserves the required precision across file-order changes. If microsecond rounding is acceptable, document it as a changed contract rather than calling the result lossless. [1, 3]
Application conditions
Preserve file order, physical timestamp units, and original fractions.
Alternatives
Verify preservation and cost with an explicit schema and union_by_name.
Limitations
A dtype check cannot recover precision already rounded away.
Evidence
- Event checked
- Subject checked
- Modified
