[source-mysql] MySQL CDC Issue - Debezium error - Can't incremental sync after the first full load

[somativa-mauricio-macri]

Connector Name

source-mysql

Connector Version

3.9.4

What step the error happened?

During the sync

Relevant information

Issue

Hi!

We are trying to use the MySQL Source CDC feature to SYNC some tables using the incremental option. This feature will be needed because some tables have deletion in them and are too large to be fully synced, each time.

The problem that we are having at this moment is that, after the initial full load of the tables (Which is a success), the next sync is giving a source error:

Example:

2025-01-16 11:53:43 source ERROR blc-192.168.51.177:3306 i.d.c.b.BinlogStreamingChangeEventSource(validateChangeEventWithTable):1045 Error processing before of row in clientes because it's different column size with internal schema size 65, but before size 1, restart connector with schema recovery mode.
2025-01-16 11:53:43 source ERROR blc-192.168.51.177:3306 i.d.c.b.BinlogStreamingChangeEventSource(logStreamingSourceState):1145 Error during binlog processing. Last offset stored = {file=sonepar-crm-dr-2-binlog.008564, pos=436795413, gtids=0372b537-adda-11ea-bd1f-005056beb7c7:1-3242904880:3242953547-3549105881:3549105883-10732882737,587a2ab3-66f7-11e9-a979-005056be3110:1->2494160359:2494299644-3027638910,5b008243-704b-11ed-a31b-005056beb7c7:1-56,b3ed6440-3c3c-11e8-8aec-00505682309f:1089461085-1100560699:1130215027-1146474858,da5fbd79-42f5-11ee-a41a-005056beb7c7:1-3750, server_id=176, event=2}, binlog reader near position = sonepar-crm-dr-2-binlog.008564/436795472
238
2025-01-16 11:53:43 source ERROR blc-192.168.51.177:3306 i.d.p.ErrorHandler(setProducerThrowable):52 Producer failure io.debezium.DebeziumException: Error processing binlog event

We encountered this issue in two separated MySQL Databases that we tried so far. The only change in the message is the table and the Schema Size.

We tried cleaning the source, creating the source from scratch, changing which table is going to be synced... And everytime the problem is the same.

I'm attaching the log of a execution with the complete messages.

From what we saw, all the specifications needed for the CDC to work were activated following the Airbyte recommendations and instructions on the following links:
https://docs.airbyte.com/understanding-airbyte/cdc
https://docs.airbyte.com/integrations/sources/mysql#quick-start
https://docs.airbyte.com/integrations/sources/mysql/mysql-troubleshooting

We thought the MySQL version could be the problem, but in the troubleshooting link, it is said that the version used (5.7) is compatible / can be used.

Any help on undestanding this and solving the issue would be appreciated!

Thanks!

Airbyte Version

1.3.1 - Latest install using ABCTL

Abctl Version

v.0.24.0

MySQL Source Version

3.9.4

MySQL Database Version

5.7

Relevant log output

Contribute

• Yes, I want to contribute

[somativa-mauricio-macri]

Updating here because of my other issue (#51014)

Even after upgrading from 3.9.4 MySQL Source to 3.11.0, this error in question still persists.

When doing the full sync, it works, but in the incremental sync, there is a error:

"
Failure in source: MySQL Schema Change Error: Your connection could not be completed because changes were detected on an unknown table (see detailed error for the table name), please refresh your schema or reset the connection. -https://kafka.apache.org/11/javadoc/org/apache/kafka/connect/errors/ConnectException.html
"

And when checking is the same error listed above

We are still trying to figure it out if it's something with the SOURCE or the MySQL... But as listed before, from what I know, all the configurations were made done following the recommended airbyte instructions.

Will be posting here if anything changes...