Large Difference Between Zeekctl and Zeek in terms of processed pps

Zeek
1

Hello all,

Recently I’ve been spending more time working on using zeek for larger volumes of traffic. And I noticed that running the zeek binary standalone, while still loading the same local.bro information, provides significantly higher processing rates than using a localhost cluster with Zeekctl. I was hoping someone might know why this is the case.

I’ve tested this with the custom J-Gras/zeek-af_packet-plugin and the standard pf_ring load balancing method. Comparatively speaking with 1Gbps of traffic continuously I could get about 68kpps from running the Zeek binary manually, but with Zeekctl this number stays in the 20-30kpps range consistently on both load balancing methods.
Additionally I saw a marked decrease in performance from Zeekctl when I changed it from localhost communications, to communicating over an actual nic on my server.

Is this slow down on speed because of connection limitations between the logger, manager, and worker?

Thanks in advance for any help you can give me!

2

Hello all,

Recently I've been spending more time working on using zeek for larger volumes of traffic. And I noticed that running the zeek binary standalone, while still loading the same local.bro information, provides significantly higher processing rates than using a localhost cluster with Zeekctl. I was hoping someone might know why this is the case.

I've tested this with the custom J-Gras/zeek-af_packet-plugin and the standard pf_ring load balancing method. Comparatively speaking with 1Gbps of traffic continuously I could get about 68kpps from running the Zeek binary manually, but with Zeekctl this number stays in the 20-30kpps range consistently on both load balancing methods.

How were you running the zeek binary?

What does your node.cfg look like? How many (real!) cpus do you have
and what model are they?

Additionally I saw a marked decrease in performance from Zeekctl when I changed it from localhost communications, to communicating over an actual nic on my server.

Definitely not.. On a single machine you were never actually using the
nic, regardless of what IP address you were listening on.

Is this slow down on speed because of connection limitations between the logger, manager, and worker?

Not likely.. but not enough information to say what is happening.