# Lenses s3 connector sink missing metadata information

**URL:** <https://ask.lenses.io/t/lenses-s3-connector-sink-missing-metadata-information/190>\
**Category:** Apache Kafka Connectors\
**Created:** [February 20, 2024, 3:28am UTC](https://ask.lenses.io/t/lenses-s3-connector-sink-missing-metadata-information/190 "2024-02-20T03:28:25Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Rudy\_Moniaga](https://dub1.discourse-cdn.com/flex017/user_avatar/ask.lenses.io/rudy_moniaga/32/105_2.png) [@Rudy\_Moniaga](https://ask.lenses.io/u/Rudy_Moniaga)\
**Post date:** [February 20, 2024, 3:28am UTC](https://ask.lenses.io/t/lenses-s3-connector-sink-missing-metadata-information/190/1 "2024-02-20T03:28:25Z")

</div>

Hi,

I am testing to use lense s3 sink connector in for backing up AWS MSK using AWS MSK Connect. I am able to backup the kafka topics to s3, however they are missing the metadata information in the s3 file. Here are my configurations below.

```auto
connector.class=io.lenses.streamreactor.connect.aws.s3.sink.S3SinkConnector
tasks.max=2
topics=sampletopic
connect.s3.aws.auth.mode=Default
schema.enable=false
security.protocol=SASL_SSL
connect.s3.kcql=INSERT INTO `test-msk-backup:backup` SELECT * FROM `sampletopic` STOREAS `JSON` WITH_FLUSH_COUNT = 1 WITH_FLUSH_SIZE = 10000 WITH_FLUSH_INTERVAL = 60 PROPERTIES('store.envelope'=true)
sasl.mechanism=AWS_MSK_IAM
sasl.jaas.config=software.amazon.msk.auth.iam.IAMLoginModule required;
value.converter.schemas.enable=false
sasl.client.callback.handler.class=software.amazon.msk.auth.iam.IAMClientCallbackHandler
connect.s3.aws.region=ap-southeast-1
value.converter=org.apache.kafka.connect.json.JsonConverter
errors.log.enable=true
key.converter=org.apache.kafka.connect.storage.StringConverter

```

I am using Stream Reactor 6.0.1. Please help to advise what is wrong with my configuration

---

<div class="post-metadata">

**Author:** ![Rudy\_Moniaga](https://dub1.discourse-cdn.com/flex017/user_avatar/ask.lenses.io/rudy_moniaga/32/105_2.png) [@Rudy\_Moniaga](https://ask.lenses.io/u/Rudy_Moniaga)\
**Post date:** [February 21, 2024, 9:35am UTC](https://ask.lenses.io/t/lenses-s3-connector-sink-missing-metadata-information/190/2 "2024-02-21T09:35:18Z")

</div>

I have found out that mentioning the specific topic as below will output the metadata information

```auto
connect.s3.kcql=INSERT INTO `test-msk-backup:backup` SELECT * FROM `sampletopic` STOREAS `JSON` WITH_FLUSH_INTERVAL = 60 PROPERTIES('store.envelope'=true)

```

however, if I am using FROM `*`, it won’t include the metadata. Is this a bug?

```auto
connect.s3.kcql=INSERT INTO `test-msk-backup:backup` SELECT * FROM `*` STOREAS `JSON` WITH_FLUSH_INTERVAL = 60 PROPERTIES('store.envelope'=true)

```

---

<div class="post-metadata">

**Author:** ![stheppi](https://dub1.discourse-cdn.com/flex017/user_avatar/ask.lenses.io/stheppi/32/30_2.png) [@stheppi](https://ask.lenses.io/u/stheppi)\
**Post date:** [February 23, 2024, 3:50pm UTC](https://ask.lenses.io/t/lenses-s3-connector-sink-missing-metadata-information/190/3 "2024-02-23T15:50:20Z")

</div>

Hi Rudy,

Indeed, as per the docs the ‘store.envelope’ backs up the entire Kafka message.

When it comes to storing with envelope mode, at the moment `FROM *`

```auto
connect.s3.kcql=INSERT INTO `test-msk-backup:backup` SELECT * FROM `*` STOREAS `JSON` WITH_FLUSH_INTERVAL = 60 PROPERTIES('store.envelope'=true)

```

will default to non-envelope. It’s a feature we haven’t yet added.

Stefan

---

<div class="post-metadata">

**Author:** ![Priya](https://avatars.discourse-cdn.com/v4/letter/p/eada6e/32.png) [@Priya](https://ask.lenses.io/u/Priya)\
**Post date:** [June 13, 2024, 2:56pm UTC](https://ask.lenses.io/t/lenses-s3-connector-sink-missing-metadata-information/190/4 "2024-06-13T14:56:11Z")

</div>

Hi,

Is there any update on this feature?

Regards,  
Priya

---

<div class="post-metadata">

**Author:** ![guillaume.ayme](https://dub1.discourse-cdn.com/flex017/user_avatar/ask.lenses.io/guillaume.ayme/32/64_2.png) [@guillaume.ayme](https://ask.lenses.io/u/guillaume.ayme)\
**Post date:** [June 14, 2024, 11:48am UTC](https://ask.lenses.io/t/lenses-s3-connector-sink-missing-metadata-information/190/5 "2024-06-14T11:48:24Z")

</div>

Can you email [info@lenses.io](mailto:info@lenses.io) for our Product team to better understand the usecase behind this need.