I’m using the Lenses Kafka Connect Http Sink Connector and need to publish operational metrics (like 2xx/4xx/5xx error counts, API response times, etc.) to Prometheus. These metrics will help in setting up dashboards and alerts for operational excellence.
The connector version is 8.1.16, and I am using Kafka version 3.4. The configuration includes properties like retry settings, batching, request content, and HTTP endpoints. Here’s a sample of the configuration:
connector.class: io.lenses.streamreactor.connect.http.sink.HttpSinkConnector
errors.tolerance: all
connect.http.retries.max.retries: 3
connect.http.retries.on.status.codes: 408,429,500,502,503,504
connect.http.method: POST
connect.http.endpoint: https://example.com/api
connect.http.request.headers: Content-Type: application/json
Is there a way to directly expose these metrics to Prometheus from the connector? If not, what would be the best practice for integrating these metrics into a Prometheus monitoring setup?