unreported packet loss

Hello everybody,

I am quite new to bro, so I am not sure whether I am missing something.
It seems to me that bro (v2.1 and v2.2) does in some cases not report
packet loss.

I discovered this by comparing resp_bytes, resp_ip_bytes and
missed_bytes from conn.log. I found several TCP streams, for which
resp_ip_bytes < resp_bytes but missed_bytes is 0, that is, there are
more TCP-bytes than IP-bytes but at the same time no packet losses,
which seemed strange.
Analyzing the corresponding TCP sequence numbers more in detail, I found
that this seems to be caused by packet loss that is not reflected in
bro's missing_bytes field. Also capture_loss.log did not show any loss.

You can find two example TCP streams, bro's output and Wireshark screen
shots here:
(Note: This is no real user traffic, these traffic samples have been
automatically generated in a testbed using mechanized Firefox instances.)

Am I missing something or did anybody encounter something similar?

Thank you,

Hello everybody,

sorry for bothering you again. As I posted two weeks ago (see below), it
seems to me that bro sometimes does not report packet loss in TCP
connections (missed_bytes in conn.log is 0 even though there are packets

These are two examples for such connections:
1331764471.664131 CJukZE3ew98dfK4qAd 60648 80 tcp - 10.048360 538 2902 SF - 0 ShADafF
5 750 4 172 (empty)
1331765540.407398 CpZaKz4sihR23hO2a6 64860 80 tcp - 6.424619 1270 10052 SF - 0
ShAaDdfF 8 1602 9 6198 (empty)

You can find the corresponding pcap files and additional information here:

Am I missing something or did anybody encounter something similar?

Thank you,

You did seem to find situations where Bro doesn’t accurately report missed content. Generally, I think it’s situations where packets are missed towards the end of connections, but specifically your example pcaps show slightly different special cases in the TCP reassembler…

example1.pcap: a gap is seen at the same time as a FIN from an endpoint and no content has been reassembled yet. Code in TCP_Reassembler.cc skips reporting that on the chance the pcap trace if pre-filtered. Associated comments:

     // We could be running on a SYN/FIN/RST-filtered trace - don't
    // complain about data missing at the end of the connection.
    // ### However, note that the preceding test is not a precise
    // one for filtered traces, and may fail, for example, when
    // the SYN packet carries data.
    // Skip the undelivered part without reporting to the endpoint.

example2.pcap: a gap is seen, but the connection state is no longer established at both ends (a FIN has been seen from one). Associated comments:

       // Only report on content gaps for connections that
      // are in a cleanly established state. In other
      // states, these can arise falsely due to things
      // like sequence number mismatches in RSTs, or
      // unseen previous packets in partial connections.
      // The one opportunity we lose here is on clean FIN
      // handshakes, but Oh Well.

The second case has a workaround by redefining the “report_gaps_for_partial” [1] option to true at risk of unreliable gap reporting for partial connections. There’s no workaround for the first case, but I think adding a similar redef-able option for explicitly indicating whether a filtered trace is used would be better than Bro just making a guess.

Would you create a bug report for this at bro-tracker.atlassian.net ?

- Jon

[1] http://www.bro.org/sphinx/scripts/base/init-bare.html#id-report_gaps_for_partial