Single Zeek multi core?

New to the platform, but impressed already.

I get that you can make a cluster of several zeek instances for LB. But when running in standalone does it not support multiple cores? In HTOP I see several threads, when running as standalone.

How can I determine if I am hitting a bottleneck?

Doing 10gbe input and appears to catch all. Had to adjust my listener nic (disable offload etc.) and capture loss is now not noticable anymore.

Am I all good? Thanks

I get that you can make a cluster of several zeek instances for LB. But when running in standalone does it not support multiple cores? In HTOP I see several threads, when running as standalone.

The part of Zeek that performs packet analysis is single-threaded and on the main thread. There is a number of threads fulfilling other functions (like logging), but the parsing is limited by a single CPU.

How can I determine if I am hitting a bottleneck?

I think traditional ways are to look at the output of policy/misc/capture-loss.zeek, and to look at zeekctl netstats and to check if the Zeek workers hit 100% CPU.

That being said, I am no expert at this - and it can be surprisingly difficult to figure out if your capture pipeline losses packets somewhere.

It is essentially impossible that you will be able to have 10gbe input and not loose nearly all packets with a single Zeek instance. The only case where this could happen is if the link is barely used.

My numbers are probably a bif off nowadays - in the past, a rule of thumb was that a single process cannot sustain more than 500mbit/s. That is probably quite a bit higher nowadays, but will not be 20x it :slight_smile:

1 Like

If I want to try out multi workers listening on the same interface, and logging to the same dir, what should I do?

Currently just using the standard docker image.

I am guessing I just need to modify the node.cfg, but can you help me do it right?

Below is an example clustered configuration. If you use this,

remove the [zeek] node above.

#[logger-1]
#type=logger
#host=localhost

#[manager]
#type=manager
#host=localhost

#[proxy-1]
#type=proxy
#host=localhost

#[worker-1]
#type=worker
#host=localhost
#interface=eth0

#[worker-2]
#type=worker
#host=localhost
#interface=eth0