Big Packet loss and PacketFilter::Dropped_Packets

So I am running Bro 2.5.2 in cluster mode using pf_ring and using it to monitor a SPAN port interface. I am running 8 workers and each of them are pinned to a CPU.

When I am performance testing by sending upto 1 gbps of network traffic having a random mix of HTTP, FTP and SMTP data I find that I am getting massive packet loss notices.

{“ts”:1512212763.169748,“note”:“PacketFilter::Dropped_Packets”,“msg”:“4135277 packets dropped after filtering, 4371549 received, 236272 on link”,“peer_descr”:“worker-1-5”,“actions”:[“Notice::ACTION_LOG”],“suppress_for”:3600.0,“dropped”:false}

{“ts”:1512212771.177625,“note”:“PacketFilter::Dropped_Packets”,“msg”:“4827328 packets dropped after filtering, 5073087 received, 245759 on link”,“peer_descr”:“worker-1-7”,“actions”:[“Notice::ACTION_LOG”],“suppress_for”:3600.0,“dropped”:false}

{“ts”:1512212773.214689,“note”:“PacketFilter::Dropped_Packets”,“msg”:“4767851 packets dropped after filtering, 5028737 received, 260886 on link”,“peer_descr”:“worker-1-6”,“actions”:[“Notice::ACTION_LOG”],“suppress_for”:3600.0,“dropped”:false}

{“ts”:1512212783.667576,“note”:“PacketFilter::Dropped_Packets”,“msg”:“5563389 packets dropped after filtering, 5818919 received, 255530 on link”,“peer_descr”:“worker-1-3”,“actions”:[“Notice::ACTION_LOG”],“suppress_for”:3600.0,“dropped”:false}

I am running Bro on a 8 core 8 GB machine with an SSD and not sure why I am getting such high packet loss.

Here is my BroControl netstats and they are also not encouraging.

[BroControl] > netstats

worker-1-1: 1512212665.151426 recvd=297260 dropped=7862632 link=297260

worker-1-2: 1512212659.639980 recvd=251046 dropped=7934351 link=251046

worker-1-3: 1512212652.110004 recvd=261434 dropped=7896026 link=261434

worker-1-4: 1512212662.089539 recvd=291058 dropped=7887963 link=291058

worker-1-5: 1512212666.662180 recvd=246944 dropped=7934732 link=246944

worker-1-6: 1512212661.373981 recvd=254560 dropped=7910802 link=254560

worker-1-7: 1512212657.278461 recvd=255041 dropped=7922435 link=255041

worker-1-8: 1512212671.643251 recvd=214359 dropped=7966526 link=214359

Any help or advise would be greatly appreciated.

Regards,

Vikram Basu

Hello there Vikram!

We are running the same Bro 2.5.2 with pf_ring and we also had the pinned CPUs and had a lot of packet drops.
After a couple tests, we managed to get the packet drops to 0 by unpinning the CPU procs, letting the OS do the dirty job.
We have being running like that for a couple days now, without drops.

Hope you can get it working!

Regards,

Felipe Tavares
OpenCloud Factory