Trying to get Bro to share Myricom cards with tcpdump or Snort

Zeek
1

Folks,

Can anyone point to a Bro+Snort HOWTO that would help me get Myricom cards to share?

1. Following the directions at

   https://www.myricom.com/software/sniffer10g/995-how-can-i-direct-sniffer10g-traffic-to-multiple-applications-using-snf-app-id.html

doesn't really help, because my Bro deployment is a cluster, and the environmental variables don't propagate to my worker hosts - in fact,
/proc/{bro_pid}/environ is 0-length on all the processes on the worker hosts.

2. I tried to reverse-engineer how Security Onion does it, but I didn't really glean anything that would help.

Thanks for any info,

2

Just to verify, you’re using the Sniffer10G v3 driver, yes?

Assuming you are, keep in mind that each interface is still limited to 32 ring buffers (this is what got me). So plan on running something like 16 for Bro and 16 for Snort/Suricata.

3

I am using the Sniffer10G driver, and in support of getting proof of concept, dropped the number of workers/host from 12 to 11 (we're actually RAM-limited because of the quantity of data we're trying to process).

Abridged utput of myri_endpoint_info:
The myri_snf driver is configured to support a maximum of:
         160 endpoints per NIC, 32 NICs per host

4

Hello Glenn:

(You'd need myricom sniffer V3 drivers inorder to run multiple applications. V2 only allows one application to listen. )

for Bro workers:

Try setting your worker nodes like the following:

[worker-1]
type=worker
host=bro-worker.site.edu
interface=myri0
lb_method=myricom
lb_procs=10
pin_cpus=3,5,7,9,11,13,15,17,19,21
env_vars="LD_LIBRARY_PATH=/usr/local/opt/snf/lib:$PATH SNF_DATARING_SIZE=0x20000000 SNF_NUM_RINGS=10 SNF_FLAGS=0x1"

- Also, I have put the following in broctl.cfg :

env_vars="LD_LIBRARY_PATH=/usr/local/opt/snf/lib"

- And have LD_LIBRARY_PATH=/usr/local/opt/snf/lib in my .bash_profile of bro user.

(depending on what shell you are using)

This config works quite alright. Others can chimein if they have a more optimal config.

Oh btw, CPU numbering is different on FreeBSD vs linux so depending on your OS, make sure you are running a worker on each core, instead of 2 workers on core+hyperthread leaving buch of other cores free. Above pin_cpu scheme is for FreeBSD. I believe linux is 1,2,3,4,5,6,7...... (not sure).

Hope this helps.

Thanks,
Aashish

5

From my Bro's node.cfg - look at the "env_vars". I don't use the

multi-application mode but that is how you pass the variables to Bro
workers.

[nsm7-eth4]
type=worker
host=a.b.c.d
interface=eth4
lb_method=myricom
lb_procs=12
pin_cpus=1,2,3,4,5,6,7,8,9,10,11,12
env_vars=SNF_DEBUG_MASK=0x3,SNF_DESCRING_SIZE=4294967296,SNF_DATARING_SIZE=17179869184

For other applications you will most likely have the modify the SO
startup/stop scripts, putting variables just before the application is
started.

6

Folks,

What a resource this list is! Thanks to Brandon, Aashish, and Micha³, I have the answer I need - configuring "env_vars=" in Bro's node.cfg file
did the trick.

Many thanks!

   -g