"Topic cannot be found" error when creating SQL Processor

Lenses
1

Trying to create a SQL Processor

SET defaults.topic.autocreate=true;
– When automatically creating topic(s) set non-default values


INSERT INTO orders_failed
SELECT STREAM * FROM airline-payments
WHERE paymentStatus = "Failed";

INSERT INTO orders_ok
SELECT STREAM * FROM airline-payments
WHERE paymentStatus != "Failed"

Getting the error that "SQL is invalid" and the topic cannot be found:

The topic airline-payments can't be found in Kafka.
Please make sure that:
  - airline-payments is a valid topic
  - airline-payments has a registered Lenses Schema or Schema Registry schema (if AVRO)
  - the current user has permissions to access this topic.

image
image1748×976 80.9 KB

Yet this topic does exist, it has JSON serialization and I can browse the events in the Explore page.

What’s the issue?

2

Hello there!

One scenario where this happens is when your records do not have a key but you have set a key type for the topic.

As an example, the topic in my screenshot does not have records with keys, there are only values.

Screenshot_20240917_150532
Screenshot_20240917_1505321485×878 120 KB

If the key type for this topic was set to anything but BYTES, then I would get the same error as you when trying to use it in SQL Processors.

So the solution is to go to the topic page, hit Actions →Change Types and set the key type to BYTES.

Best,
Marios

1 Like