Hi.
How's the BRO real world performance? You know, 10Gbit links and up. How many workers do I need for every 1Gbit of traffic (sure, it depends on the rules heavily)?
Or just how much traffic can I expect a single worker to handle? How about the memory?
That's what I have here:
Intel(R) Xeon(R) CPU E5-2620 @ 2.00GHz x 2 so it gives 24 threads with HT enabled.
Also, I have 64GB of RAM in each NSM sensor. Expected traffc? A few Gbit/sec, depending on a sensor location.
Do you have some real world examples, such as "we have server with <CPU> and <mem> and it handles Gbit/sec of traffic on average/peak"
I know that's a lot of questions, but trying to establish a baseline and do some capacity planning here 
And there's nothing in google, apart from some (i guess old) statement, that a single bro process can handle up to 80Mbit/sec.
How's the BRO real world performance? You know, 10Gbit links and up. How
many workers do I need for every 1Gbit of traffic (sure, it depends on
the rules heavily)?
[…]
Do you have some real world examples, such as "we have server with <CPU>
and <mem> and it handles Gbit/sec of traffic on average/peak"
There was a thread about exactly this on here just a few weeks ago - to cite a
bit from it:
[…]
I keep meaning to write this up, but on *my* configuration:
* 16 cores of model name : Intel(R) Xeon(R) CPU X5677 @ 3.47GHz
* 72GB of RAM
* Endace DAG (9.2)
* some config magic by Seth, which I'd be happy to share.
6 workers keep up with ~2.5-3Gbps peaks, no problem.
[…]
It doesn't actually consume all of the above resources - I'm running other things on the box too - but bro itself consumes ~4.5GB resident per worker, and can be counted on to pin most of its allocated cores at peak loads.
Just to throw another data point out there:
* 16 physical cores of model name : Intel(R) Xeon(R) CPU E5-2680 @ 2.70 GHz
* 96GB of RAM
* Myricom NIC
28 workers (I have Hyperthreading turned on) keep up with a 6-7 Gbps average, and I've seen them do fine with short peaks of 9 Gbps or so. The Myricom cards definitely won't break the bank: card + SR optics + perpetual license is $895.
[…]
Full thread at:
http://mailman.icsi.berkeley.edu/pipermail/bro/2013-March/006242.html
I hope that helps,
Johanna
How's the BRO real world performance? You know, 10Gbit links and up. How
many workers do I need for every 1Gbit of traffic (sure, it depends on
the rules heavily)?
(...)
Full thread at: http://mailman.icsi.berkeley.edu/pipermail/bro/2013-March/006242.html I hope that helps,
Johanna
Thanks, now I'm like stupid because I should have checked 
Now that's what I call the real world numbers, awesome!
Yeah, I begrudgingly wrote that because the question came up so frequently. It was based on old estimates and doesn't seem to be as relevant anymore. I know of sites doing everything from 100Mbps/core to >500Mbps/core, it depends heavily on the clock rate of the CPU and how you are capturing packets.
In the case of the site with >500Mbps/core, they are using an Endace DAG card and skipping the OS nearly completely to acquire packets and their per-core clock rate is 3.7Ghz I believe.
With 2GHz cores, you likely won't hit that speed, but it will almost certainly be faster than that horribly documented 80Mbps.
.Seth
Also try running just bro command line instead of using broctl in your tests.
James
I just got my filthy paws on another host similar to the one I specced earlier. It will be getting similar-but-different loads to that one, on a slower CPU and with an Intel NIC instead of the DAG. Once I've got some performance numbers, I'll post those too.
I don't mind if people want to contact me, either on or off-list, to see how things are running and what I'm doing.
Seth, have you considered collecting these so they're not stashed in the mailing list archives? A "here's some performance numbers from real installations" kind of page. Maybe link it off http://bro.org/community/index.html ?
Mike
Keep bugging us about it. That's a good idea.
Oh yeah, for the benefit of the list, most sites run pretty stock Bro so performance differences are mediated by that.
.Seth
What sort of packet rate can you handle per worker?
I've maxed out gig links with only minimal impact. I use bro command line though and not broctl.
James