Styxis
Company
Knowledge

Airbyte MySQL-to-ClickHouse type errors: start with the corrected CDC selectedFields diagnosis

Before changing a date conversion, check whether CDC system fields match the transmitted schema.

Do not preserve a superseded diagnosis

Before fixing only the column named by an error, inspect alignment between the transmitted schema and the complete selected-field list. The useful Truthound connection is verification of delivered fields against the approved schema, rather than changing the format of one apparently failing column. [1, 2, 3]

The reporter corrected the cause

The public Airbyte issue initially blamed DATETIME and numeric VARCHAR columns, then explicitly corrected that diagnosis. The reporter excluded five ab_cdc* system fields from selectedFields for a CDC source and observed the same connector versions succeed after including them. The explanation of the internal code path is separately described as inference from symptoms. [1]

Catch omissions while creating configuration

Check required CDC fields when generating configuration. Reconcile transmitted fields and destination schema using the corrected diagnosis. Record actual order and types so an equal-length but misaligned array cannot pass. API acceptance of the configuration is not approval of the data contract. [1, 3]

Count and identity both matter

If an illustrative contract requires its specified 5 system fields, a selected count of 5 meets a necessary condition and 0 fails. This does not permit substituting 5 unrelated columns. Obtain the expected name set from the CDC contract and compare actual names as well. [1, 3]

The Truthound check

Truthound Core's ColumnExistsValidator can verify required system columns in the prepared transmission table. Add order and type checks separately. It does not automatically amend Airbyte settings or repair already misaligned Protobuf values. Use the corrected report's successful configuration as a control, without repeating the superseded DATETIME explanation as the solution. [1, 2, 3]

Application conditions

Preserve selectedFields, actual transmitted-field order and destination schema for the same CDC run, comparing them with the required system-field contract.

Alternatives

Reproduce the corrected report’s successful configuration that includes CDC system fields as a control.

Limitations

  • Truthound does not automatically amend Airbyte API settings or recover misaligned Protobuf values. Do not retain the superseded DATETIME diagnosis.

Evidence

Event checked
Subject checked
Modified