Trying to sink __consumer_offsets topic with Lenses S3 connector

Apache Kafka Connectors
1

We’re attempting to use the S3 Connector to back up our Kafka Cluster for Disaster Recovery purposes.

However, we’re encountering an issue when trying to sink the __consumer_offsets topic into the S3 bucket.

org.apache.kafka.connect.errors.ConnectException: Exiting WorkSinkTask due to unrecoverable exception

We’ve tried different configurations for the S3 Connector for this topic, but it continues to fail due to the binary format of the data.

We’re unsure which value converter to use for this topic.

image (21)
image (21)1916×1021 218 KB

image (20)
image (20)1905×800 82.2 KB

We’ve included the connector configuration and a trace from the SinkTask for reference.

Can you please provide some advice on how to resolve this issue?

2

We are working on a method of simplifying sinking offsets to S3 for backup/restore purposes. Stay tuned soon or email info@lenses.io for latest information.