Question about capture loss script vs. broctl netstats

I apologize if this has been answered already - I was searching through the list archives and did’t seem to find the answer.

I have configured a RHEL 6 server with the latest Bro from the repository and pf_ring 5.2.2.

It seems pf_ring works - I run pfcount on my capture interface and it sees traffic and reports no packet loss.

I have Bro configured per the post at http://ossectools.blogspot.com/2011/09/bro-quickstart-cluster-edition.html and everything starts fine and Bro is up and running.

I run netstats in the Broctl shell and get:

worker-0-1: 1372179895.260001 recvd=64969350 dropped=0 link=64969350
worker-0-2: 1372179895.461289 recvd=66422051 dropped=0 link=66422051
worker-0-3: 1372179895.660990 recvd=64099315 dropped=0 link=64099315
worker-0-4: 1372179895.861853 recvd=61738222 dropped=0 link=61738222

But in the notice.log file I see:
1372179930.880560 - - - - - - CaptureLoss::Too_Much_Loss The capture loss script detected an estimated
loss rate above 38.520% - - - - - worker-0-3 Notice::ACTION_LOG 3600.000000 F - - - –
1372179930.908354 - - - - - - CaptureLoss::Too_Much_Loss The capture loss script detected an estimated
loss rate above 37.415% - - - - - worker-0-4 Notice::ACTION_LOG 3600.000000 F - - - –
1372179930.923939 - - - - - - CaptureLoss::Too_Much_Loss The capture loss script detected an estimated
loss rate above 40.462% - - - - - worker-0-1 Notice::ACTION_LOG 3600.000000 F - - - –
1372179930.923939 - - - - - - CaptureLoss::Too_Much_Loss The capture loss script detected an estimated
loss rate above 42.910% - - - - - worker-0-2 Notice::ACTION_LOG 3600.000000 F - - - –

So my question is, am I dropping packets or am I good to go?

Best Regards,

Derek Banks

As seth mentions, the capture-loss script is quite robust, because it
essentially computes an end-to-end value. I don't know of any situations
where in practice it makes poor estimates. These stats:

worker-0-1: 1372179895.260001 recvd=64969350 dropped=0 link=64969350
worker-0-2: 1372179895.461289 recvd=66422051 dropped=0 link=66422051
worker-0-3: 1372179895.660990 recvd=64099315 dropped=0 link=64099315
worker-0-4: 1372179895.861853 recvd=61738222 dropped=0 link=61738222

on the other hand come from the kernel's statistics. If packets are lost
prior to the kernel even seeing them (such as due to an overwhelmed SPAN
port - quite common), then while it reports no drops, that's not a
useful end-to-end measure. (Also, some kernels have bugs in how these
statistics are captured, for example missing out on packets dropped by
the NIC rather than the kernel.)

    Vern

If you’re seeing nearly 50% of dropped traffic, perhaps the SPAN session is monitoring one direction of traffic flow across a NAT’d interface or proxy server? On the external interface monitoring inbound, and on the internal interface monitoring inbound? Any introduction of non bi-directional traffic would very likely confuse the capture loss script.

Since you're using a DAG card this could depend on what sort of flow splitting configuration you have in place there too.

  .Seth

Thanks for all the responses. I put an Intel Pro 1000 in this morning and still using PF_Ring. I three hours of running Bro, I don’t see any reported packet loss. Looks like the Broadcom Card was most likely the problem.

-Derek

Oh yeah, I would never use Broadcom for sniffing. I've had way too many problems with them.

  .Seth

Thanks for all the responses. I put an Intel Pro 1000 in this morning and still using PF_Ring. I three hours of running Bro, I don't see any reported packet loss. Looks like the Broadcom Card was most likely the problem.

Oh yeah, I would never use Broadcom for sniffing. I've had way too many problems with them.

Use Intel or ... use Intel. Too many stability problems with other vendors, and all of them resolved using X520.

Unless you can afford DAG or similar of course.

myrinet seems to be pretty popular aswell, so i would guess work ok aswell, or?

//Kristoffer

Yep, for people with 10G links they're monitoring I'd almost always go with Myricom with the sniffer drivers. They have commodity pricing on their NICs and they do the hash based load balancing and it's extremely easy to setup (and it works on freebsd and linux).

  .Seth