missed bytes without gaps

Zeek
1

I am confused on how I am getting “missed_bytes.” When I read the documentation it says that these are due to content gaps, and is caused by packet loss. So I have tried to look for signs of this, but I have yet to find anything that shows content gaps or any significant packet loss. I see no mention of content gaps in my notices.log file, and my packet loss report from broctl shows almost no dropped packets (recvd=2770338 dropped=92 link=2770430)

Perhaps I am looking at this wrong, so if someone could help out here it would be greatly appreciated. I am logging correct streams, vs streams with missed bytes, to see how often these missed bytes show up. To do this I am filtering my logs in the following way

Missed: only looking at bro_conn, orig_bytes AND resp_bytes > 0, only TCP packets, missed_bytes > 0
Non-missed: only looking at bro_conn, orig_bytes AND resp_bytes > 0, only TCP packets, missed_bytes = 0

Using these two definitions, I see almost 40% of my packets fall into the “missed” streams, while around 60% fall into the non-missed. I was doing this to check my setup and see if I had everything working. From everything else (no gaps reported, and no almost no dropped packets) I thought everything was working. Now I question if something else is wrong, and so I am weary about using this to look at other data as it may not be complete.

2

There are a lot of reasons that you could be missing traffic that have nothing to do with the packet drop statistics reported by your NIC. I have a guess about what's happening in your traffic though. Have you disabled the special features on your NIC? Refer to this blog post on how to do it on linux:
  http://blog.securityonion.net/2011/10/when-is-full-packet-capture-not-full.html

If you want a much better mechanism to see if you're receiving all of the traffic you should be I recommend loading the misc/capture-loss script. By default it will write out to capture_loss.log every 15 minutes and due to it taking measurements of TCP streams themselves it can even detect packet loss occurring before the packets arrive at your monitoring interface. A number of people have detected faulty packet distribution boxes and overloaded switch SPAN ports with it.

  .Seth

3

Wow thank you this was exactly my problem! I have turned these settings
off and in the last 5 minute window my missed bytes went from ~40% to 0%.

Perhaps a quick note in the doc's could point others to offloading features
in NIC's as well because this fixed everything for me!