First I would like to thank @Vern and @Aashish_Sharma1 again for your kind replies, we did some initial tests about the performance based on ZAM, and we’d like to share some results.
In summary, we actually focus the cps (connections per second) more than network traffic, and found that under the same test setting, without/with the ZAM was enabled, we noticed roughly 20% performance increasement in general of cps according to our tests.
Some test details are as below:
Physical server of zeek
- cpu: 40 core xeon @2.1ghz
- mem: 256GB
- network adapter: x520 intel 10 Gigabit network adapter
Traffic generator generates a hybrid network traffic includes
- (99%) http (v4/v6), 4KB payload
- (1%) dns(v4/v6)
with ~20K - 24K cps, and each tcp connection maintains for ~30 sec, total open tcp connections are ~800K, and total network traffic is ~1gbps
We port mirror the generated traffic over switch to the 10gb network adapter of the server and then processed by zeek.
To capture the network traffic we use pfring zc and did load-balance by zbalance_ipc to create a total of 31 rss queues:
zbalance_ipc -i zc:ens192 -n 31 -m 1 -c 10 -g 1
And for zeek configurations:
- version 5.1.0
- 31 workers (1 core per worker, and each worker processed traffic from one particular pfring rss queue)
- 1 logger, manager and proxy process
- disable/enable ZAM by the zeekargs mentioned above
Under this setting, without installing any other third party plugins and scripts, and the traffic generator keep generating traffic for about 1hr, the results showed that
- without ZAM, zeek can process ~20K cps without reporting dropped packets (by the capture_loss and the stat log)
- with ZAM is on, zeek can process ~24K cps without(or little) reporting dropped packets
and conn, dns, files and http logs had been functioning normally during the test and had correct analyzed entries; avg cpu loads of workers are ~95%, and memory occupied ~90GB.
We haven’t done a thorough tests but based on the initial results, ZAM does have a significant performance enhancement.
I notice that other than
-O ZAM, there are some additional options of ZAM according to ZAM/README. So I wonder if there is any options can print some debug information, such that which can be utilized to increase the performance further.
Also we appreciate that if there is any comments, ideas and suggestions about our settings/configs in favor of performance enhancement, or if you would like to know more details of our tests.
Again thank for having this great feature of zeek in new releases!