SQL Processor - "SQL Invalid" error when using FORMAT_DATE(_meta.timestamp)

Lenses SQL
1

Getting errors while attempting to create an SQL Processor in Lenses:

SQL Processor statement:

INSERT INTO my_topic
   STORE VALUE AS AVRO
   SELECT STREAM
   field_a as _key,
   field_b,
  FORMAT_DATE(_meta.timestamp, 'yyyyMMddhhmmss') as event_time
FROM xyz_topic;

Error message:

SQL Invalid: Identifier _meta in FORMAT_DATE(_meta.timestamp, yyyyMMddhhmmss') can not be resolved. Hint: verify that there are no typos in _meta.

Same select in SQL Studio work fine.

Currently using Lenses Version 5.5.7

2

Hi wkolbenschlag

At the moment the SQL processors do not support accessing the Kafka message metadata. It is something we are looking to add in the future.

3

Understood. However, the Lenses 6.0 documentation indicates that particular metadata functionality is available for both SQL Studio and Processors and should be updated accordingly.

4

Has the ability to accessing the Kafka message metadata within SQL Processors been dded to Lenses? Is there an estimated availability? T

5

Has this feature been incorporated into the newest versions of Lenses?

6

Hi @wkolbenschlag - The product team are looking into this. Can you email info@lenses.io and they can schedule a call with you.

7

Hi @wkolbenschlag ,
We plan to include this feature in an upcoming release. If it is not ready for the next two-week cycle, it will be included in the following one at the latest.