Styxis
Company
Knowledge

Doris MySQL CDC snapshots: is the INVISIBLE primary key actually extracted?

A key declared in the source schema may still be absent from the rows delivered by a SELECT.

Declared keys and delivered keys differ

The existence of a primary key and its inclusion in an actual extraction are different conditions. Inspect the snapshot's real column list with Truthound, not only the source catalog. A result missing its required key cannot simply be assumed suitable for linking later change events. [1, 2, 3]

The INVISIBLE column omitted by SELECT *

The public Doris MySQL streaming-snapshot report describes SELECT * returning fewer columns than the schema for a table with an INVISIBLE primary key, causing the job to pause. Creating a destination table does not establish snapshot completion. An extraction path capable of using an explicit column list is needed. [1]

An illustrative key-presence condition

If an approved snapshot requires a particular primary-key column, a true presence state passes and a false state fails. NULL key values and duplicate keys are separate checks. This rule concerns the delivered column itself. Substituting the source catalog definition for the actual extraction inventory would manufacture a pass. [1, 3]

Truthound in CDC registration

Before registering CDC, compare declared keys with actual snapshot columns. Approve only a path that can explicitly include the invisible key. Truthound Core's ColumnExistsValidator can verify required columns in the received table. It neither guarantees binlog-reader compatibility nor changes MySQL visibility, and it does not establish complete CDC operation. Test live changes after the snapshot separately. [1, 2, 3]

Application conditions

Compare the approved primary-key column list with the actual completed snapshot’s columns for the same extraction scope.

Alternatives

Choose a snapshot path using an approved explicit column list instead of SELECT *.

Limitations

  • Column presence does not guarantee binlog-reader compatibility or complete CDC operation. Truthound does not change MySQL column visibility.

Evidence

Event checked
Subject checked
Modified