S3 Source connector: should topic name and bucket path match?

Apache Kafka Connectors
1

Hello!

I’m wondering if the S3 Source connector supports the following situation:

  1. Using Lenses’ S3 Sink, topic my-topic is backed up in my-bucket:topics-backup. This leads for the concrete backup to be at s3://my-bucket:topics-backup/my-topic.
  2. Using Lenses’ S3 Source, s3://my-bucket:topics-backup/my-topic gets restored to a topic named ´another-topic-name´.
    Reading the documentation I do not see explicit support for this, but wanted to check with the experts just in case.

Thanks!

2

I’d like to clarify my understanding of your inquiry:

Are you asking whether data backed up from my-topic should automatically be restored to a topic with the same name?

If so, we intentionally don’t enforce this one-to-one mapping during restoration. Here’s why:

  1. Design Philosophy
  • The source connector is designed to be flexible about target topics
  • This allows for more versatile data restoration scenarios
  1. Key Considerations
  • Many valid use cases require restoring data to different target topics
  • Enforcing same-name restoration would:
    • Add unnecessary complexity to the connector
    • Limit functionality for scenarios like disaster recovery, data migration, or testing
    • Prevent common use cases such as restoring to staging/dev environments

Would you mind sharing your specific use case? This would help me provide more targeted guidance for your situation.