Hi,
I’m testing Zeek 7.2.1 in Docker with the new ZeroMQ cluster backend enabled via:
@load frameworks/cluster/backend/zeromq/connect
Issue: command “zeekctl netstats” hangs indefinitely without returning any output or errors. Other zeekctl commands work fine, and the cluster runs normally.
Has anyone experienced this issue? Any workarounds or suggestions?
@thiennt - awsome! Thanks for testing and reporting here.
Yeah, that’s unfortunately expected right now because zeekctl is using Broker to interact with the cluster. Should also happen for print id and peerstatus.
I had started a conversation about this on Slack proposing switching zeekctl to Zeek’s WebSocket API instead:
Hey - zeekctl currently uses Broker’s Python bindings to communicate with individual nodes. I’ve been procrastinating on thinking about this for ZeroMQ or other cluster backends in the hope that zeekctl will magically vanish, but I don’t think it will.
So, I’d like to propose the Zeek manager in a zeekctl setup listen on a WebSocket port by default (localhost, probably port 27759) and change the zeekctl commands that interact with individual nodes to use the WebSocket API, rather than establishing connections to individual nodes and interacting with them. The manager would broker between zeekctl and individual nodes.
This would mean that when the manager (or whichever nodes hosts the WebSocket API) isn’t functional, zeekctl will not be able to connect to other nodes in the cluster and commands non-functional, but honestly that seems acceptable.
I’m looking for reactions if that appears reasonable to people? (edited)
Has anyone experienced this issue? Any workarounds or suggestions?
Hmm, one ugly workaround might be to run Broker::listen() using the ports from the cluster-layout in the interim, but that’d be pretty sad. So, probably changing the implementation to WebSocket, but that’d probably come after the 8.0 release…
While that should work for worker, proxy and manager, it wouldn’t work for loggers - they reuse the port numbers to open ZeroMQ PULL sockets.
Thank you so much for your quick response!
I’m really keen on using ZeroMQ for the Zeek cluster backend. Since zeekctl isn’t working with ZeroMQ, I’m wondering if Zeek 7.2.1 offers any API endpoints or alternative approaches to programmatically retrieve packet drop statistics in a more proactive and accessible way?
I’m wondering if Zeek 7.2.1 offers any API endpoints or alternative approaches to programmatically retrieve packet drop statistics in a more proactive and accessible way?
Well, we’re heavily pushing Prometheus metrics and dropped, received and link packets metrics are available as metrics through the misc/stats scripts.
The docs are here - we call it the “Telemetry framework”.
I thought zeekctl today enables them by default? Otherwise check for MetricsPort config and how to use it.
Does using Prometheus work for you? I’d suggest that much over running zeekctl netstats and doing manual scraping 
Really appreciate all your help!
I’m just trying to upgrade to Zeek 7 while maintaining integration with my legacy system. Looks like I should take a deeper dive into the Telemetry framework and explore how to better utilize it for my needs.
Thanks again for all your assistance!
Hey @thiennt- how’s your migration going?
Just wanted to let you know that with Zeek 8.0 (rc2 has been released by now), you’ll be able to use ZeroMQ and Zeekctl together by placing (or uncommenting) the following lines in your zeekctl.cfg:
# The backend to use for cluster communication. Broker is the default
# in 8.0 with ZeroMQ planned to become the default in Zeek 8.1 and later.
ClusterBackend = ZeroMQ
UseWebSocket = 1
If you have further feedback, would be glad to hear