bro cluster with pf ring dna+libzero

Zeek
1

we're deploying a new bro cluster and am a huge newbie on all of this; so please excuse my ignorance. i have yet to actually start capturing on the cluster (awaiting delivery of a front-end device)

on each worker i have the dna+libzero ixgbe driver installed and insmodded. so i run:

$ sudo insmod pf_ring.ko enable_tx_capture=0 min_num_slots=32768
$ sudo insmod ixgbe.ko RSS=1,1,1,1 num_rx_slots=32768 mtu=9000

$ sudo /usr/sbin/setcap cap_net_raw,cap_net_admin=eip /usr/bin/pfdnacluster_master
$ /usr/bin/pfdnacluster_master -d -P /var/run/pfdnacluster-dna0.pid -D bromaint -c 0 -i dna0 -n 10

i do the setcap as i am running bro as non-root user. looks good…

$ cat /proc/net/pf_ring/13979-dna0.1
Bound Device(s) :
Active : 1
Breed : DNA
Sampling Rate : 1
Capture Direction : RX+TX
Socket Mode : RX only
Appl. Name : dna-cluster-0-socket-0
IP Defragment : No
BPF Filtering : Disabled
# Sw Filt. Rules : 0
# Hw Filt. Rules : 0
Poll Pkt Watermark : 128
Num Poll Calls : 0
Channel Id : 0
Num RX Slots : 32768
Num TX Slots : 8192
Tot Memory : 672399360 bytes
Cluster: Tot Recvd : 11
Cluster: Tot Sent : 0

then on my manager i have the following nodes.cfg:

[manager]
type=manager
host=sec-broman

[proxy-0]
type=proxy
host=sec-broman

[proxy-1]
type=proxy
host=sec-broman

[sec-bro01-0]
type=worker
host=sec-bro01
interface=dnacluster:0
lb_method=pf_ring
lb_procs=10

using bro 2.3; so i believe the lb_pf_ring.py script understands the dnacluster interface spec.

so i do an 'broctl install' (as user bromaint) from the manager, then log onto my worker and run

$ sudo /usr/sbin/setcap cap_net_raw,cap_net_admin=eip /opt/bro/bin/capstats
$ sudo /usr/sbin/setcap cap_net_raw,cap_net_admin=eip /opt/bro/bin/bro

then a 'broctl start' on the manager. everything looks fine so far… then i run 'broctl capstats' and i get:

Interface kpps mbps (10s average)

2

Hello,

Capstats is a separate application as far as pfdnacluster_master is concerned. You can tell pfdnacluster_master that you want to send the same traffic to another application using the -n flag by using a "," and then specifying how many instances of the second app you intend to run. When you call pfdnacluster_master try "-n 10,1" instead of "-n 10". You actually want to run two applications against the same traffic, but the second app, capstats, will only run one process that needs to consume all of the traffic instead of having slices of traffic load balanced between multiple processes.

Regards,
Gary

3

Hi Gary,

ah, that makes sense! -n 10,1 works great :slight_smile: thanks very much.

is anyone using ZC pf_ring for bro?

cheers,

Yee.

4

2.3 should support using the zc load balancing tool. If you follow the directions here:
  http://bro.org/documentation/load-balancing.html

you should be able use the same config as for pf_ring+DNA but you'll be sniffing an interface named zc:21 (or similar, the "21" is the cluster number that is auto assigned by broctl). One issue with the zbalance_ipc tool that pf_ring ships with though is that you can't load balance to multiple applications like you could with the older pfdnacluster_master tool. I couldn't figure out how to do it at least.

  .Seth