Separate local file durability from PyIceberg commit acknowledgment
Read the durability report within the scope of the deprecated filesystem-table commit path. In an isolated recovery test, reconcile acknowledged references with file contents and distinguish process termination from power loss.
A process kill differs from power loss
A durable catalog pointer and durable referenced files are separate conditions. A process-kill test cannot substitute for a power-loss contract. Truthound Core can compare recovered file properties with a separately preserved pre-acknowledgment record. [1, 2, 3]
Tracker and deprecated-path scope
Iceberg issue #18606 describes local PyIceberg files becoming 0 bytes after power or kernel loss when close precedes a pointer change without fsync. A maintainer follow-up says this belongs in the Python tracker and concerns a deprecated filesystem-table commit mechanism. It must not be generalized to modern catalogs or object storage. [1]
Preserve isolated recovery evidence
In an isolated recovery environment, retain acknowledged snapshot references and pre-failure lengths, hashes, and parse status. Record ordinary process termination and power-loss scenarios separately. Do not induce failures on production equipment for this acceptance check. [1, 3]
Length is only a prerequisite
Define an illustrative metadata.json length of 2048 bytes before acknowledgment, preserving a hash separately. Recovered 2048 bytes passes the length prerequisite and still needs hash and parse checks. A 0-byte file fails even when its path exists, because it cannot be the same valid metadata. [1, 3]
Compare files without claiming fsync
ColumnPairValidator in Truthound Core Python SDK 3.3.2 can compare prepared non-null byte lengths. The operator runs the isolated recovery procedure and captures the files. The SDK neither performs fsync nor cuts power, and size equality alone does not establish content integrity. [1, 2, 3]
Review supported durability boundaries
Review supported FileIO durability boundaries for files and directories, with the deprecated commit mechanism kept explicit. A source report filed in the wrong tracker is not evidence that every current warehouse implementation has the same behavior. [1, 3]
State the recovery stage checked
Approve recovery only at the level actually checked: reference existence, length, content, and parsing are separate stages. Preserve the report’s process-kill versus power-loss distinction when communicating what the test can establish. [1, 3]
Application conditions
Prepare isolated-environment pre/post-ACK file length, hash, and parse records.
Alternatives
Check supported FileIO file/directory durability through isolated recovery tests.
Limitations
Do not extend the deprecated filesystem-commit report to modern catalogs or object storage.
Evidence
- Event checked
- Subject checked
- Modified
