How to Handle Complex JSON Structures in Lenses SQL Queries

Hello

I have been working with Kafka topics that contain deeply nested JSON structures, and while Lenses SQL has been great for querying; I am running into challenges when trying to extract and flatten specific fields for analysis. :innocent:

In particular, when arrays and nested objects are mixed, the queries start to get complicated, and performance sometimes drops noticeably.:thinking:

I am wondering what the best practices are for handling these scenarios. Should I pre-process the JSON before ingestion, or are there advanced SQL functions within Lenses that can efficiently flatten and query this type of data?:thinking: Checked JSON | Lenses Docs guide related to this and found it quite informative. When I reviewed what is javascript, I noticed its simple JSON handling, and it made me think about whether Lenses SQL supports comparable transformations.:slightly_smiling_face:

Has anyone here designed an efficient strategy for working with deeply nested JSON in Lenses SQL, particularly when balancing readability, maintainability, and query performance? :thinking:

Any tips, examples, or real-world use cases would be really helpful for anyone facing similar challenges.

Thank you !!:slightly_smiling_face:

Can you show an example of a JSON event with the nesting?