Using the Lenses S3 Connector kafka-connect-aws-s3-6.0.0, I have the following kcql statement but in the JSON output file in S3, i’m not seeing the metadata of the event in the envelope information:
connect.s3.kcql: "insert into aws.demo.lenses.dev select * from topic1 STOREAS
JSON PARTITIONBY abc, def, ghi WITH_FLUSH_COUNT=100000 WITH_FLUSH_SIZE=10000 WITH_FLUSH_INTERVAL=600 PROPERTIES('store.envelope'=true);"
I was expecting to see:
"metadata": {"offset": xxxx, "partition": 3, "timestamp": 134324234324, "topic": "topic1"}
as described in this blog:
if i’m missing something, please let me know.
Thanks