Dropped data

Hello,

Can someone give me some direction on trying to figure out why I have dropped data?

This output is from a machine getting about 3G of traffic a minute or so into starting Bro 2.5.3 with PF_RING 7.0.0.

How much data per worker should I expect to budget for? Ideally I’d like Bro to be able to do 10G of traffic.

Has anyone used PF_RING ZC with success?

worker-0-1: 1525291681.760081 recvd=564836 dropped=0 link=564836
worker-0-2: 1525291681.961074 recvd=723187 dropped=0 link=723187
worker-0-3: 1525291682.162178 recvd=682598 dropped=4619 link=682598
worker-0-4: 1525291682.364202 recvd=1094776 dropped=0 link=1094776
worker-0-5: 1525291682.566055 recvd=6722748 dropped=30902 link=6722748
worker-0-6: 1525291682.768050 recvd=2180528 dropped=0 link=2180528
worker-0-7: 1525291682.969023 recvd=3252824 dropped=0 link=3252824
worker-0-8: 1525291683.179065 recvd=414112 dropped=0 link=414112
worker-0-9: 1525291683.379083 recvd=2228892 dropped=52543 link=2228892
worker-0-10: 1525291683.579973 recvd=1735298 dropped=0 link=1735298
worker-0-11: 1525291683.780260 recvd=2720785 dropped=1437 link=2720785
worker-0-12: 1525291683.981421 recvd=5835651 dropped=7610 link=5835651
worker-0-13: 1525291684.181057 recvd=566766 dropped=0 link=566766
worker-0-14: 1525291684.381979 recvd=335114 dropped=0 link=335114
worker-0-15: 1525291684.582077 recvd=743998 dropped=0 link=743998
worker-0-16: 1525291684.782897 recvd=6124252 dropped=54604 link=6124252
worker-0-17: 1525291684.980916 recvd=3476401 dropped=17138 link=3476401
worker-0-18: 1525291685.184047 recvd=1286574 dropped=0 link=1286574

Hi,

this actually does not look very bad to me - on most interfaces you do not
seem to have any drops. One of them has a bit over 2% which is not that
pretty but also not catastrophic.

I have no experience with ZC, but generally packet loss can be caused by a
number of issues. Single high-speed connections can be problematic
(because they add to the normal load of a single Bro process). Microbursts
also happen and can lead to a bit of packet loss.

If there is a setting to increase the available buffer, that might be
worth playing around with.

Johanna

What kind of cards and distribution do you have? Maybe you could just switch to afpacket to avoid the problem entirely

Michal,

Could you explain what you meant by switching to AF_PACKET and avoiding the problem all together?

Thanks,

Carl

There’s no advantage using crazy solutions that make you jump through multiple hoops when most of the time the default and built in packet capture mechanism works well.

I think he may have been looking for pointers to a next step to take. :slight_smile:

Carl, I think Michal might be telling you to look into the AF_Packet plugin by Jan Grashofer…
https://github.com/J-Gras/bro-af_packet-plugin

That page has full instructions on how to install and use the plugin.

.Seth