Adding connectors to lenses-box

Apache Kafka Connectors
1

Hi! I have been experimenting with lenses box and was trying to add an Iceberg Sink Kafka connector

Im currently mapping this into the container as per below:
- ./tabular-iceberg-kafka-connect-0.6.18:/opt/lensesio/connectors/third-party/tabular-iceberg-kafka-connect-0.6.18

and also providing metadata for the connector via environment variable :

 LENSES_CONNECTORS_INFO= |
        [
            {
            class.name = "io.tabular.iceberg.connect.IcebergSinkConnector"
            name = "Iceberg Sink Connector"
            instance = "file"
            sink = true
            extractor.class = "io.lenses.config.kafka.connect.SimpleTopicsExtractor"
            icon = "iceberg.png"
            description = "Used to sink data to Iceberg tables"
            author = "Tabular"
        }
        ]

Unfortunately I cant see any of the content of the connector being loaded as part of
http://127.0.0.1:3030/fdd/logs/connect-distributed.log or http://127.0.0.1:3030/fdd/logs/lenses.log

And it is not available as a sink in the connector page

Is there anywhere else I need to define the connector in order for it to be picked up by the classloader?

Any pointers would be gratefully received! Thanks

2

Hello Chris! Try to add the connector under /connectors:

- ./tabular-iceberg-kafka-connect-0.6.18:/connectors/tabular-iceberg-kafka-connect-0.6.18

The path /opt/lensesio/connectors is a little special. In Box we allow you to explicitly enable or disable the default connectors, so we process each directory in there separately and we expect a naming convention of kafka-connect-[connectorname].

So if you mounted your connector as /opt/lensesio/connectors/third-party/kafka-connect-tabular-iceberg it would work. But still the recommended place for bring-your-own connectors is under /connectors. :slight_smile:

3

Awesome that worked - thanks for the clear explanation!