Dropped packets in PF_RING install

Hi Bro,

We have an install of bro running on a single machine with PF_RING load balancing.

Previously we were seeing a huge amount of dropped traffic — in the realm of ~90% average packet loss per hour. The history column in our conn.log was trash as expected, with only one or two letters per connection.

After some tweaking (adding memory & upping # of bro processes & changing PF_RING buffer size), the logs look much better and the packet loss is drastically reduced, to about 0.5%-1% loss per hour. However, both broctl netstats and cat /proc/net/pf_ring/*eth0* report some packet loss still.

Is the sub-1% packet loss we’re seeing expected/optimal or are there additional tweaks that we could add to push this down to 0%?

some notes

both tcpdump -nn -s0 -vv -i eth0 -w /dev/null and the pfcount.c utility from pf_ring report 0% packet loss. It’s not until we start using bro that we start seeing dropped packets.

we’re currently using 16 bro processes pinned to 16 of 32 total processors

PF_RING buffer size is currently 65536

packet loss does seem to go down during low-traffic hours but during the day when traffic is 2.5-3 gbps is when the dropped packet count peaks (while still being a small percentage of the overall traffic)

Let me know if you guys have any thoughts on this, thanks!


Nicholas Siow
Washington University in St. Louis :: Information Security

Can you paste your node.cfg here? I’m having similar problems, but my packet loss is much, much higher.

Cheers,

JB

Are the pins to actual CPUs or hyper threads? How much throughput are you dealing with?

Sure. It’s pretty standard and is more or less copied from the bro page on Load Balancing —


[manager]
type=manager
host=localhost

Actual CPUs, based on /proc/cpuinfo.

During the day I’m usually seeing traffic in the realm of 2.75-3.25 gbps.