Hi James,
Thank you for trying K2K and going through the trouble of filling this report.
My name is Carlos and I’m working directly on the K2K development.
So, in a nutshell, it seems like when using MSK, some topic properties are added that are non standard (partition.size.max.bytes).
Additionally it also seems like MSK does not allow updating some properties like file.delete.delay.ms .
The current way K2K works is that we use the Apache KafkaAdminClient and when we attempt to auto-create a topic, we simply read the topic’s properties from your source cluster and we attempt to create one with the same properties in the target.
It seems however that the KafkaAdminClient will happily read the non-standard properties but fails when attempting to apply them.
There are two possible workarounds at the moment:
- manually create the target cluster topic ahead of time
- repeat the process done before (setting
replication.common.config.”prop”to null) with all properties that are causing an issue . This should make these not be sent to the target cluster
In the meantime we’re actively working and iterating on K2K so we should have a way to better address this issue soon.