[destination-gcs] version betwen 0.4.4 and 0.4.8 introduces JsonMappingException

[kev-datams]

Connector Name

destination-gcs

Connector Version

v0.4.8

What step the error happened?

During the sync

Relevant information

Hello,

We have Airbyte v1.3.0 deployed on GKE with Helm chart v1.3.1.

We recently upgraded our GCS destination connector from v0.4.4 to v0.4.8.
Our sources use Cloud SQL connector in v3.6.28.

Majority of sync work well, but for one of them we observed below blocking error:

2025-01-15 14:15:56 destination WARN i.a.c.j.Jsons(handleDeserThrowable):424 Failed to deserialize json due to class com.fasterxml.jackson.databind.JsonMappingException
	at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:402)
	at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:361)
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.wrapAndThrow(BeanDeserializerBase.java:1937)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:312)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:177)
	at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:129)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:310)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:177)
	at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:342)
	at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4899)
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3846)
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3814)
	at io.airbyte.commons.json.Jsons.tryDeserialize(Jsons.java:122)
	at io.airbyte.cdk.integrations.base.Destination$ShimToSerializedAirbyteMessageConsumer.consumeMessage(Destination.java:113)
	at io.airbyte.cdk.integrations.base.Destination$ShimToSerializedAirbyteMessageConsumer.accept(Destination.java:91)
	at io.airbyte.cdk.integrations.base.IntegrationRunner.consumeWriteStream(IntegrationRunner.java:321)
	at io.airbyte.cdk.integrations.base.IntegrationRunner.consumeWriteStream(IntegrationRunner.java:301)
	at io.airbyte.cdk.integrations.base.IntegrationRunner.runInternal(IntegrationRunner.java:190)
	at io.airbyte.cdk.integrations.base.IntegrationRunner.run(IntegrationRunner.java:125)
	at io.airbyte.integrations.destination.gcs.GcsDestination.main(GcsDestination.java:16)
Caused by class com.fasterxml.jackson.core.exc.StreamConstraintsException
	at com.fasterxml.jackson.core.StreamReadConstraints._constructException(StreamReadConstraints.java:549)
	at com.fasterxml.jackson.core.StreamReadConstraints.validateStringLength(StreamReadConstraints.java:484)
	at com.fasterxml.jackson.core.util.ReadConstrainedTextBuffer.validateStringLength(ReadConstrainedTextBuffer.java:27)
	at com.fasterxml.jackson.core.util.TextBuffer.finishCurrentSegment(TextBuffer.java:939)
	at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._finishString2(ReaderBasedJsonParser.java:2241)
	at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._finishString(ReaderBasedJsonParser.java:2207)
	at com.fasterxml.jackson.core.json.ReaderBasedJsonParser.getText(ReaderBasedJsonParser.java:327)
	at com.fasterxml.jackson.databind.deser.std.BaseNodeDeserializer._deserializeContainerNoRecursion(JsonNodeDeserializer.java:572)
	at com.fasterxml.jackson.databind.deser.std.JsonNodeDeserializer.deserialize(JsonNodeDeserializer.java:100)
	at com.fasterxml.jackson.databind.deser.std.JsonNodeDeserializer.deserialize(JsonNodeDeserializer.java:25)
	at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:129)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:310)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:177)
	at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:129)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:310)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:177)
	at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:342)
	at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4899)
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3846)
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3814)
	at io.airbyte.commons.json.Jsons.tryDeserialize(Jsons.java:122)
	at io.airbyte.cdk.integrations.base.Destination$ShimToSerializedAirbyteMessageConsumer.consumeMessage(Destination.java:113)
	at io.airbyte.cdk.integrations.base.Destination$ShimToSerializedAirbyteMessageConsumer.accept(Destination.java:91)
	at io.airbyte.cdk.integrations.base.IntegrationRunner.consumeWriteStream(IntegrationRunner.java:321)
	at io.airbyte.cdk.integrations.base.IntegrationRunner.consumeWriteStream(IntegrationRunner.java:301)
	at io.airbyte.cdk.integrations.base.IntegrationRunner.runInternal(IntegrationRunner.java:190)
	at io.airbyte.cdk.integrations.base.IntegrationRunner.run(IntegrationRunner.java:125)
	at io.airbyte.integrations.destination.gcs.GcsDestination.main(GcsDestination.java:16)
2025-01-15 14:15:56 destination ERROR i.a.c.i.b.Destination$ShimToSerializedAirbyteMessageConsumer(consumeMessage):120 Received invalid message:**********

If we rollback to GCS destination connector v0.4.4, the issue disappears.

Please could you investigate and fix the issue ? 🙏

Complementary question: when do you plan to release an official first version of GCS destination connector like Amazon S3 ?

Thanks !