Hello
I have been working with Kafka topics that contain deeply nested JSON structures, and while Lenses SQL has been great for querying; I am running into challenges when trying to extract and flatten specific fields for analysis. ![]()
In particular, when arrays and nested objects are mixed, the queries start to get complicated, and performance sometimes drops noticeably.![]()
I am wondering what the best practices are for handling these scenarios. Should I pre-process the JSON before ingestion, or are there advanced SQL functions within Lenses that can efficiently flatten and query this type of data?
Checked JSON | Lenses Docs guide related to this and found it quite informative. When I reviewed what is javascript, I noticed its simple JSON handling, and it made me think about whether Lenses SQL supports comparable transformations.![]()
Has anyone here designed an efficient strategy for working with deeply nested JSON in Lenses SQL, particularly when balancing readability, maintainability, and query performance? ![]()
Any tips, examples, or real-world use cases would be really helpful for anyone facing similar challenges.
Thank you !!![]()