I am using MQTT Source Connector to pull MQTT messages from an external broker to my AWS MSK Kafka. My Kafka topic has 6 partitions, but all the messages are going to one single partition. On checking the messages received, I realized that the MQTT connector is by default setting the KEY value and this is a constant which makes kafka to push all messages to just 1 partition. Is there a way to edit the KEY value pre-set by the MQTT Connector to a random value or at the least to make it NULL.
@noeltoms You can use an SMT to update the key value. Connect will pass it through them before its written to Kafka. Or you can use MQTT | Kafka Connectors | Lenses Docs
email us at info@lenses.io and we can discuss.