I would like to see if and how it would be possible to a search on a combination of elastic data and topic data.
For instance , I find in my elastic index all contract id’s sent & processed by SAP
I would like to select the contracts of this week in elastic and based upon that list search per contract id in for any topics that contain contract number in the message body and display this body.
Thus in general can we combine two or more datasources in one sql query ?
Is there some video on this ?
If Elastic data is in one topic of Kafka and the data from another data source is also in a topic, you can perform JOINs on them using Lenses. The documentation provides more details os how to do this, take a look if that’s what you need: Stream to Stream Join | Lenses.io Documentation
Currently it’s not possible to query an Elastic Search index and a Kafka Topic in the same query in SQL Studio. For you to meet your usecase, you could consider using the Lenses Source Elasticsearch connector to bring the data in first and then doing a join.
There will be some tricks with SQL Processors you could adopt to make this join efficient. We recommend you reach out to the team for more info.