Logstash HTTP pipelining: method locks do not establish record assembly order
Protecting individual methods is different from preserving complete-request order.
Why complete ordering matters
Protecting individual methods is different from preserving complete-request order. Verify that a record split across requests on one connection is reassembled in source order. Truthound needs the request sequence and reconstructed record; a few successfully published events cannot validate the damaged middle record. [1, 2, 3]
Serial keep-alive and pipelining
Logstash #19400 describes multiple workers using one connection’s json_lines codec out of order under HTTP pipelining. Serial keep-alive differs from multiple in-flight requests. Synchronizing individual methods does not guarantee ordering across compound tokenizer operations; the public explanation is not a new execution by this article. [1]
Retain one connection’s source
Split one JSON record across two pipelined requests, then compare reconstructed record identity and original content. Isolate publication from a connection with mismatched assembly. Retain raw chunks and arrival order for that TCP connection; a count drop alone can hide an unintended reconstructed payload. [1, 3]
The boundary that must retain second
The illustrative first request contains the front of the second record and the next request its remainder. Source-order assembly should yield id second. second meets the identity condition; CORRUPTED fails regardless of total event count. Equal ids still require comparison of the remaining fields against source content. [1, 3]
Check reconstructed results
Truthound Core Python SDK 3.3.2 can compare prepared source_record_id and reconstructed_record_id through ColumnPairValidator with relationship="==". The operator extracts ordered chunks and results and retains parse failures separately. The SDK does not change Netty scheduling or tokenizer critical sections. [1, 2, 3]
Receipt and assembly have different success conditions
Test serial requests or complete JSON per request, and verify processing and response-order corrections. A fresh connection or one request at a time awaiting a response has a different concurrency condition. Successful request receipt cannot establish correct record assembly. [1, 3]
Application conditions
Preserve request order and raw chunks on the same TCP connection, distinguishing pipelining from serial keep-alive.
Alternatives
Test serial requests or complete JSON per request and validate processing and response ordering.
Limitations
The SDK does not repair scheduling or tokenizer synchronization; correct counts do not establish record meaning.
Evidence
- Event checked
- Subject checked
- Modified
