Hello. It appears that Lenses is missing information for some of my partitions. This has the following effects:
- I cannot see data in the explore page for the affected topics, or I cannot perform queries for these topics in SQL Studio
- Consumer groups that are consuming from these topics miss the consumer lag, start and latest offsets information for the affected partitions
- Lenses reports these partitions as having zero messages
This may be an indication of a performance problem. Usually, it will not show until you have a few thousands of partitions and hundreds of consumers.
To work around this issue, you can try to increase the intervals that Lenses is querying the Kafka cluster for information and the allowed memory usage.
You can start by doubling these intervals in
lenses.conf (default values are
To learn more about these↑ settings, check here.
Then to increase allowed memory usage, use this environment variable:
If you are using the Helm Charts, you can try these values (tested with Lenses 5.4):
heapOpts: -Xmx10g -Xms5g