autoCreateTopics problems in k2k kafka replication configuration for MSK Serverless

K2K
1

Seems like when I try to set autoCreateTopics on k2k config, such as:

name: "test-k2k"
features:
  autoCreateControlTopics: enabled
  autoCreateTopics: enabled
coordination:
  kafka:

…I’m getting the errors:

00.0, 5000.0, 7500.0, 10000.0], getCounts=[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2], getExemplars=[]}, ImmutableHistogramPointData{getStartEpochNanos=1753355849817497766, getEpochNanos=1753355851885144572, getAttributes={arena="Main", thread.name="kafka-admin-client-thread | adminclient-1"}, getSum=581064.0, getCount=17, hasMin=true, getMin=7184.0, hasMax=true, getMax=262160.0, getBoundaries=[0.0, 5.0, 10.0, 25.0, 50.0, 75.0, 100.0, 250.0, 500.0, 750.0, 1000.0, 2500.0, 5000.0, 7500.0, 10000.0], getCounts=[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 14], getExemplars=[]}, ImmutableHistogramPointData{getStartEpochNanos=1753355849817497766, getEpochNanos=1753355851885144572, getAttributes={arena="TLAB", thread.name="io-compute-1"}, getSum=372240.0, getCount=1, hasMin=true, getMin=372240.0, hasMax=true, getMax=372240.0, getBoundaries=[0.0, 5.0, 10.0, 25.0, 50.0, 75.0, 100.0, 250.0, 500.0, 750.0, 1000.0, 2500.0, 5000.0, 7500.0, 10000.0], getCounts=[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1], getExemplars=[]}]}}
k2k_1  | 11:17:31.992 [io-compute-0] ERROR io.lenses.k2k.main.Main - K2K application stopped with an unexpected exception.
k2k_1  | org.apache.kafka.common.errors.InvalidPartitionsException: Number of partitions is below 1.


k2k_1  | 11:22:09.527 [io-compute-1] ERROR io.lenses.k2k.main.Main - K2K application stopped with an unexpected exception.
k2k_1  | org.apache.kafka.common.errors.InvalidConfigurationException: Amazon MSK Serverless can't alter topic configuration file.delete.delay.ms. Only the following topic configurations are alterable: [compression.type, max.message.bytes, message.timestamp.difference.max.ms, message.timestamp.type, retention.bytes, retention.ms, cleanup.policy]. For all topic configurations, MSK Serverless uses defaults that follow best practices.
2

Setting this should work:

topicCreation:
  replication:
    common:
      config:
        "file.delete.delay.ms": null
3

…full example:

name: "my-first-replication"
features:
  autoCreateControlTopics: enabled
  autoCreateTopics: enabled
source:
  kafka:
    connection:
      servers: "kafka-source:9092"
    consumer:
      "group.id": "k2k.my-first-k2k"
target:
  kafka:
    connection:
      servers: "kafka-target:9092"
coordination:
  kafka:
    commit:
      group: "k2k.my-first-k2k"
replication:
  - source:
      topic: ".*"
  - sink:
      topic: source
      partition: source
topicCreation:
  replication:
    common:
      config:
        "file.delete.delay.ms": null
4

Now also getting the error:

k2k_1 | 20:08:32.997 [io-compute-blocker-0] ERROR io.lenses.k2k.main.Main - K2K application stopped with an unexpected exception. k2k_1 | org.apache.kafka.common.errors.InvalidConfigurationException: Unknown topic config name: partition.size.max.bytes

5

Hi James,

Thank you for trying K2K and going through the trouble of filling this report.
My name is Carlos and I’m working directly on the K2K development.

So, in a nutshell, it seems like when using MSK, some topic properties are added that are non standard (partition.size.max.bytes).

Additionally it also seems like MSK does not allow updating some properties like file.delete.delay.ms .

The current way K2K works is that we use the Apache KafkaAdminClient and when we attempt to auto-create a topic, we simply read the topic’s properties from your source cluster and we attempt to create one with the same properties in the target.

It seems however that the KafkaAdminClient will happily read the non-standard properties but fails when attempting to apply them.

There are two possible workarounds at the moment:

  • manually create the target cluster topic ahead of time
  • repeat the process done before (setting replication.common.config.”prop” to null) with all properties that are causing an issue . This should make these not be sent to the target cluster

In the meantime we’re actively working and iterating on K2K so we should have a way to better address this issue soon.