I have two workers that are constantly pegged at dropping 50% of the packets I am processing. It is always the same two workers. This is on bro 2.4.1, so I don’t have misc-stats (yet). Is there a way I can troubleshoot why I have problems with these two workers?
On second thought, I am getting in excess of 1.1 Mpps. According to Robin’s paper here, https://www.sans.org/reading-room/whitepapers/intrusion/open-source-ids-high-performance-shootout-35772, I should be able to process about 880 kpps with 24 workers.
However, I have 20 workers and 400 gigs of ram. When I move the workers up to 24, my box gets crushed with a load of 20, up from a load of 13-15, and I drop even more packets on the floor. Is the only way out of this to stand up another box and try to use broctrl to load balance between those systems?
How much traffic you can handle depends a lot on the kinds of packets that
your traffic consists of. So - for some traffic, 880k kpps might be ok,
for other kinds of traffic, you might not even be able to handle half of
that, even with the same hardware. So - you always have to take numbers
like these with a grain of salt; you will never get exactly the same
performance.
That being said - if there are two specific workers that always drop
packets, that might point to streams with high data rates that are handled
by these two processes.
2.4.1 actually does have misc/stats.bro, so you can try loading that to
see what is going on. It does not give as much information, but it might
still be helpul.
If you have too much traffic for your current hardware to handle, yes,
youd only choice might be to either disable scripts or add more hadrware.
I hope this helps,
Johanna