It appears PF_RING is not properly load balancing between Bro instances. For example, I have a single Bro node with 5 bro procs. Every entry in http.log is duplicated 5 times (exact timestamp and all fields are identical except the UID). My conclusion is pf_ring is not splitting the traffic and that all procs are seeing all the traffic.
my node.cfg:
[bro-worders]
type=worker
host=localhost
interface=eth5
lb_method=pf_ring
lb_procs=5
pf_ring was loaded with:
enable_tx_capture=0 min_num_slots=32768
Bro is correctly linked to libpcap libraries:
ldd /usr/local/bro/bin/bro | grep pcap
libpcap.so.1 => /opt/pfring-6.6/lib/libpcap.so.1 (0x00007effe684d000)
pf_ring info:
[root@bro-box]# cat /proc/net/pf_ring/info
PF_RING Version : 6.7.0 (dev:9b0e7c81718edb0ff6d070793bc868e3c3456bd5)
Total rings : 6
Standard (non ZC) Options
Ring slots : 32768
Slot version : 16
Capture TX : No [RX only]
IP Defragment : No
Socket Mode : Standard
Cluster Fragment Queue : 0
Cluster Fragment Discard : 0
I am not sure where to go from here. Does anyone have any suggestions?
Jereme Lamps
The default load balancing for bro pf_ring is to use 4-tuple.
If you have a lot of asymmetric traffic (hot IP/port combo like a syslog or something), you’ll see some “buckets” with more packets.
You may want to try a different load balancing scheme as outlined here:
https://www.bro.org/sphinx/components/broctl/README.html#pfringclustertype
You may be running into an issue that was recently fixed in broctl and will be resolved in the next release. Depending on the order you install things in, pf_ring load balancing can end up disabled.
What does the following output for your host?
[root@bro-dev ~]# broctl config | grep pfring
pfringclusterid = 21
pfringclustertype = 4-tuple
ringfirstappinstance = 0
if you have pfringclusterid set to 0, that's the problem that was just fixed. You can easily workaround that by adding
PFRINGClusterID = 21
to your /usr/local/bro/etc/broctl.cfg
Once that is there, a broctl deploy should get everything working.
Justin,
Your solution seems to have fixed it.
Thanks!
Jereme
>
> It appears PF_RING is not properly load balancing between Bro instances. For example, I have a single Bro node with 5 bro procs. Every entry in http.log is duplicated 5 times (exact timestamp and all fields are identical except the UID). My conclusion is pf_ring is not splitting the traffic and that all procs are seeing all the traffic.
You may be running into an issue that was recently fixed in broctl and will be resolved in the next release. Depending on the order you install things in, pf_ring load balancing can end up disabled.
What does the following output for your host?
[root@bro-dev ~]# broctl config | grep pfring
pfringclusterid = 21
pfringclustertype = 4-tuple
ringfirstappinstance = 0
if you have pfringclusterid set to 0, that's the problem that was just fixed. You can easily workaround that by adding
PFRINGClusterID = 21
to your /usr/local/bro/etc/broctl.cfg
Once that is there, a broctl deploy should get everything working.