Ennobled bro users and developers,
I'm looking for some clarification on the use of bro and multiple
interfaces.
FreeBSD 6.1 machine with two em* (Intel 1000 fibre) interfaces. Each
interface's RX port is connected to one of the two TX ports on a
regenerative tap.
Bro.cfg was originally configured as:
BRO_CAPTURE_INTERFACE="em0 em1"
Additionally, we tried enabling and disabling:
BRO_BPFBOND_ENABLE="YES"
and
BRO_BPFBOND_FLAGS="em0 em1"
In all cases above, we got indications that this configuration was not
correct, and that bro might not be getting all of the traffic across
both interfaces properly (see example #1 below, with content gaps in the
smtp log).
We then set up a bond interface:
ifconfig bridge0 create
ifconfig bridge0 addm em0 addm em1 up
..and changed our bro.cfg to:
BRO_CAPTURE_INTERFACE="bond0"
BRO_BPFBOND_ENABLE="NO"
This seems to work properly now; at least we no longer get content gaps
logged to the smtp log (see example #2 below).
My questions are: Is this (bridge device method) the "right" way to
handle multiple interfaces for my hardware/software? The documentation
mentions kernel patches to enable bpf bonding on FreeBSD 4.1. Is this
not necessary on later FreeBSD releases?
Thanks,
Matt Cuttler
Matt,
We are running a very similar configuration and are using netgraph to
bond the two interfaces into one virtual interface which we monitor
(again similar to your method) and it has been working fairly well for
us. My understanding is that the kernel patch is no longer necessary
because netgraph is already in the source code, it just needs to be
compiled in by adding "options NETGRAPH" to the kernel config file and
then running a script during startup that creates the virtual
interface. The one problem I have seen with two of our systems is that
the interface periodically goes deaf and doesn't come back unless with
ifconfig down and up all of the interfaces involved (so I wrote a script
that tests the interface every few minutes and restarts it and notifies
me if there is no traffic). This only seems to happen on two or the 5
boxes I use this on (not the bro box), and I suspect it is partially a
function of something else I may be running (or is based upon load).
John
Matt Cuttler wrote:
Matt,
We also use netgraph, but don't have the same problems John has.
Here's the netgraph config:
# load module
kldload ng_ether
# bring up the real interfaces
ifconfig em0 promisc -arp up
ifconfig em1 promisc -arp up
# create ngeth0 and bind em1 and em2 to it
ngctl mkpeer . eiface hook ether
ngctl mkpeer ngeth0: one2many lower one
ngctl connect em0: ngeth0:lower lower many0
ngctl connect em1: ngeth0:lower lower many1
# bring up ngeth0
ifconfig ngeth0 -arp up
Mark
John Ives wrote:
John Ives wrote:
Matt,
> The one problem I have seen with two of our systems is that
the interface periodically goes deaf and doesn't come back unless with
ifconfig down and up all of the interfaces involved (so I wrote a script
that tests the interface every few minutes and restarts it and notifies
me if there is no traffic).
I had this problem under FreeBSD 6.0, but it went away when I upgraded
to 6.1
Brian Tierney wrote:
John Ives wrote:
Matt,
The one problem I have seen with two of our systems is that
the interface periodically goes deaf and doesn't come back unless with
ifconfig down and up all of the interfaces involved (so I wrote a script
that tests the interface every few minutes and restarts it and notifies
me if there is no traffic).
I had this problem under FreeBSD 6.0, but it went away when I upgraded
to 6.1
Thanks for confirming I am not crazy. 
Actually when I upgraded to
6.1 (rebuilt world) it got a lot better, but did not go away completely
(whereas it hasn't been a problem the boxes that were built at 6.1 from
the start). At some point I will probably just need to bite the bullet
and rebuild the machines, but probably not until sometime after 6.2.
John