Hello Zeeks
Has anyone succeeded to enable Kafka plugin with Zeek 3.1.2 ? I am trying to modernize the metron-kafka plugin and have partial success. My problem seems to be with script-land referencing.
The logger node is loading the plugin OK and connects to the Kafka broker. The broker IP is redef information found from site/local.zeek.
$ bin/zeekctl diag logger-1
[logger-1]
No core file found.
Zeek 3.1.2-debug
Zeek plugins:
Apache::Kafka - Writes logs to Kafka (dynamic, version 0.3.0)
==== No reporter.log
==== stderr.log
%7|1587948661.341|RECV|rdkafka#producer-3| […kafka messages…] …
But the worker node has a problem referencing existing variable declaration. The logs-to-kafka.bro script expects it. There is also suspicion with the Zeek plugins info that is different from the logger node and maybe the problem.
$ bin/zeekctl diag worker-1-1
[worker-1-1]
No core file found.
Zeek 3.1.2-debug
Zeek plugins: (none found) <<< ??? Normal for worker node ???
==== No reporter.log
==== stderr.log
error in /opt/zeek/spool/installed-scripts-do-not-touch/site/custom_plugins/APACHE_KAFKA/scripts/Apache/Kafka/./logs-to-kafka.bro, line 24: unknown identifier logs_to_send, at or near “logs_to_send”
The configuration is not default and explained below:
The Kafka logger was installed to site/custom_plugins/APACHE_KAFKA
share/zeek/site/local.zeek uses:
@load custom_plugins/APACHE_KAFKA/scripts/Apache/Kafka
lib/zeek/plugins/custom_plugins is a symlink to share/zeek/site/custom_plugins
Using the lib symlink seems to be the only way to load the plugin, then the @load statement brings redef customizations and scripts. This works ok for the logger node but not the worker who cannot interface with the plugin ?
Another idea is have non-logger nodes bypass loading logs-to-kafka.bro but this isn’t fully understood.
TIA
/hovsep