Try the following pcap filtering trick:
‘
[manager]
type=manager
host=localhost
[proxy-1]
type=proxy
host=localhost
[worker-1]
type=worker
host=localhost
interface=em0
aux_scripts=q1of2
[worker-2]
type=worker
host=localhost
aux_scripts=q2of2
Add the following scripts in share/bro/site
q1of2.bro:
redef restrict_filters += { [“q0”] = “(ip[12:4] + ip[16:4]) & 1 == 0” };
q2of2.bro:
redef restrict_filters += { [“q0”] = “(ip[12:4] + ip[16:4]) & 1 == 1” };
Nice one!
In the 2.0-beta you don't even need to define that aux_scripts field either. You should be able to just make files named worker-1.local.bro and worker-2.local.bro in your <prefix>/share/bro/site/ directory and they will automatically get loaded by the correct nodes.
.Seth
Nice!
Something changed in the way packet filters work on 2.0 Bata bro. It does not seem to have the old behavior.
What I wont is the ability to set a filter per work. That way I could write I could spread the tcp load for ip and iptv6 acros n works and run a n+1 work that take care of the no ip traffic.
Bill Jones
Is there a reason you can't do PF_RING? It's a lot more efficient
than BPF load balancing.
Thanks for the tip William,
This trick doesn’t work on Bro2.0
I tried both aux_scripts and creating the worker-1.local.bro file
I installed the new policy stop/start and again the workers are not load-balancing traffic
tcpreplay -i ethX --mbps=1000.0 680000_packets.pcap
[BroControl] > netstats
worker-1: 1320349321.810072 recvd=639496 dropped=30101 link=669597
worker-2: 1320349322.010123 recvd=639549 dropped=30048 link=669597
This is Out of the box Bro2.0 with PF_RING and libpcap compiles with PF_RING.
What's the output of cat /proc/net/pf_ring/* ? There should be a file
per Bro worker in there when Bro is running.
PF_RING Version : 5.0.0 ($Revision: exported$)
Ring slots : 4096
Slot version : 13
Capture TX : Yes [RX+TX]
IP Defragment : No
Socket Mode : Standard
Transparent mode : Yes (mode 0)
Total rings : 0
Total plugins : 0
[BroControl] > status
Name Type Host Status Pid Peers Started
manager manager localhost running 13328 3 03 Nov 19:08:41
proxy-1 proxy localhost running 13359 3 03 Nov 19:08:43
worker-1 worker localhost running 13403 2 03 Nov 19:08:45
worker-2 worker localhost running 13402 2 03 Nov 19:08:45
[BroControl] > netstats
worker-1: 1320362018.486131 recvd=669511 dropped=286 link=669797
worker-2: 1320362018.686087 recvd=669650 dropped=147 link=669797
Ok, that shows that Bro is not in fact using PF_RING, as there are no
entries for the pids.
Is PF_RING part of bro?
By studding the pcap filter code I found a work around. I set the “cmd_linebpdf_filter:” to my packfilter definition.
It would be nice to have a more formal way to do this.
Don’t you need more foo to get PF_RING to load balance it looks like you have to bind a bro instances to a cpu?
Nope, and if you build Bro against the PF_RING libpcap wrapper BroControl automatically takes care of everything to begin load balancing. I'm still waiting to hear back from Tomer with the output from the commands I asked him for earlier to actually figure out what's going wrong for him.
.Seth
Thank!
I will give it a try!
I do bro mentoring with a tap. That means that each bro instances needs to read from two Ethernet interface to see tranmit and receive side the same tcp connection. The pcap filters insure that this happens. What happens when I use the PF_RING pcap interface with bro. Will each bro worker see the same connection pair?
Actually, I recommend setting up a bonded interface, which recent
PF_RING's will happily monitor.
Ah, nice! Thanks for pointing that out.
.Seth
Just install bro with PF_RING without my filter to see what happens with load pf_ring load balancing.
As I though the load balancing does a good jobs of distributing the load across my 8 bro workes. The down side is that bro is not working correctly sense each bro work only see part of the tcp connections for example the wired log:
1320440533.316479 B1zdmt0vxHf 129.114.13.51 54999 8.20.213.28 80 above_hole_data_without_any_acks - F worker-2
1320440533.316479 F1NuRpLxmri 129.114.13.51 54999 8.20.213.28 80 above_hole_data_without_any_acks - F worker-4
1320440533.316479 GBvErIhMFH3 129.114.13.51 54999 8.20.213.28 80 above_hole_data_without_any_acks - F worker-1
1320440533.316479 Jgz4LByaW62 129.114.13.51 54999 8.20.213.28 80 above_hole_data_without_any_acks - F worker-8
1320440533.316479 JgQfacLEqNf 129.114.13.51 54999 8.20.213.28 80 above_hole_data_without_any_acks - F worker-5
1320440533.316479 a5JEFET8tid 129.114.13.51 54999 8.20.213.28 80 above_hole_data_without_any_acks - F worker-6
1320440533.316479 Olp5WQZeFsk 129.114.13.51 54999 8.20.213.28 80 above_hole_data_without_any_acks - F worker-7
There are a lot of other functions that don't seem to work.
I am putting the filter back but I will continue to run pf_ring with load blancing turned off and see what happens.
I backed down from the PF_RING pcap library. I couldn't find a way to run off the load balancing.
Hm, I'm not totally convinced that you have pf_ring fully working yet. My guess is that each of those workers saw the same packet. Connection unique IDs will be generated differently on different hosts so you can't expect those to be the same and everything else, including the timestamp is exactly the same.
Can you send the output of:
broctl config | grep -i pfring
and:
ldd <prefix>/bin/bro
If there is a problem with pf_ring not being enabled correctly on some machines, we'd certainly like to figure it out.
Thanks!
.Seth