Truthound · Ingestion and parsing · 3/4
Find losses at CSV, JSON and columnar file boundaries.
77 articles
Check Doris VARIANT field presence when UTC Z parsing depends on aliases
After loading a Zulu timestamp, check presence and non-null state at the typed VARIANT path even if the row succeeded. Record the BE conversion context without treating later host state or a compaction observation as proof of cause.
Require an explicit Doris auto-partition RPC status
Do not fill an absent wire status with default OK in an automatic-partition response. Check for an explicit response and then visible keys in the new partition as separate completion boundaries.
Follow the corrected Unity Catalog URI contract for literal percent paths
Check percent-bearing locations by distinguishing URI text from the storage key it represents. Follow the correction toward UC's input and output boundary instead of repeating the withdrawn Delta fix as a remedy.
Preserve mixed-case Hadoop option keys through Delta DataFrame maps
Link each case-sensitive Hadoop option key to its effective Configuration lookup. A forwarded string alone cannot establish that the I/O setting took effect; keep credential values out of the record.
Keep Delta partition-filter casts consistent with scan values under ANSI
Apply the same parsing policy to malformed partition strings in full-scan and filtered paths. Preserve raw values and ANSI settings so a cast error stays distinct from a valid empty result.
Reject invalid unsigned nested input before ClickHouse defaults hide it
Type compatibility cannot approve a nested unsigned array when negative input was replaced by an empty array. Link raw input to the rejection policy and retain the scope of any lost valid siblings.
Do not hide a DuckDB CSV TRY_CAST failure inside a missing-value ratio
Unparseable values, quarantined records and a failed execution are different outcomes.
Separate successful Doris external-scan completion from data completeness
A successful Spark job can still contain a short read if cancellation or failure is exposed as normal end-of-stream.
Before Doris Arrow Stream Load, align actual array types with the destination
A valid Arrow IPC representation is not necessarily a compatible input for the destination column's serde.
Why ClickHouse NATS checks must separate messages from rows
A successful message receipt does not establish complete landing when one message expands into several rows.
PapaParse workers in Vite production: validating the returned data envelope
Approve a new worker path with a small test of the production bundle's returned envelope, not only its development behavior.
PapaParse unparse: testing optional fields that first appear after the initial row
Design the CSV export test around the explicit field contract, not an assumption that the first object represents all rows.
pandas CSV numeric separators: preserving the meaning of Unicode locale settings
Internationalization settings affect interpretation, not just display. Preserve the raw token and the actual separator used to decode it.
pandas parallel read_csv: checking whether lower missingness is a parsing change
Before comparing missingness trends across a processing-speed change, verify continuity of the missing-value decision for the same raw tokens.
pandas ODS import: distinguishing covered cells with values from empty merged placeholders
Separate visual cell coverage from value presence so that a calculation entered by the document author is not silently reclassified as missing.
pandas object uint64 export: preventing signed JSON identifiers from replacing originals
Identifier preservation requires comparison with the source sign and exact decimal representation, not just a range check.
pandas CSV and Excel MultiIndex headers: preserving the meaning of each level
Even when the value array matches, column-level names explain the axes to readers and require a separate agreement.
pandas Excel with mixed values: preserving attribute codes before choosing a dtype
Before calling a mixed-type column dirty, agree whether it holds quantities or a list of attribute values.
pandas read_json and large integers: choosing exact text without changing the field's purpose
Decide whether a large integer is an identifier, an exact count, or a timestamp before reducing its meaning merely to make parsing succeed.
pandas JSON normalization: preserving missing-record positions during migration
Before deleting an empty list entry to avoid a parser failure, decide whether it represents a missing observation or merely packaging.
