Connection summary email inaccuracies

Zeek
1

Hello,

I've used Bro on and off for a couple years and love its unix-ness and
application-layer smarts. I use it to augment my NetFlow and SNMP data,
and it gives me just enough information to complement those logs. I
haven't dug into the scripting and IDS aspects yet, but I hope to soon.

I have an issue with the connection summary email. Aside from the fact
that I could do without it altogether, because it doesn't really tell me
anything that NetFlow can't, I'm confused by how inaccurate the
information in the email seems to be.

To take the example that always jumps out at me, here are the incoming
port statistics from this morning's email.

== Incoming === 2013-08-25-23-50-18 - 2013-08-26-23-19-23

   - Connections 306.0 - Payload 137.0m -
     Ports |
     9997 78.1% |
     3 9.2% |
     514 5.2% |
     50664 3.6% |
     22 1.3% |
     52145 0.7% |
     51222 0.7% |
     52140 0.3% |
     51735 0.3% |
     51724 0.3% |

The reason I know something is strange about this is that I get NetFlow
data around the clock from three different sites on ports 9997, 9998,
and 9999. How could it be that one site accounts for 78.1% of all of my
incoming traffic and the other two are nowhere to be seen? Also, the
number of connections and payload information is way off. Here is the
same information queried from NetFlow:

Port Flows(%)
0 4831(13.0)
9999 1985( 5.3)
9997 1797( 4.8)
9998 1510( 4.1)
22 559( 1.5)
123 398( 1.1)
64115 349( 0.9)
65138 162( 0.4)
40767 135( 0.4)
13496 120( 0.3)

Summary: total flows: 37254, total bytes: 2.1 G, total packets: 1.7 M,
avg bps: 194612, avg pps: 19, avg bpp: 1237
Time window: 2013-08-25 23:49:42 - 2013-08-26 23:24:49
Total flows processed: 102134, Blocks skipped: 0, Bytes read: 5318892

Netstat doesn't indicate any dropped packets, and conn.log doesn't
indicate any missed_bytes. Can anyone shed some light on why bro could
be so wrong about these statistics? Would it matter that I am using a
single instance of bro to monitor two interfaces (bro -i em0 -i em1)?

Thanks for any help you can provide...

Best,
Chris

2

Netstat doesn't indicate any dropped packets, and conn.log doesn't
indicate any missed_bytes. Can anyone shed some light on why bro could
be so wrong about these statistics? Would it matter that I am using a
single instance of bro to monitor two interfaces (bro -i em0 -i em1)?

The interface thing shouldn't matter. What version of Bro are you using? I think there was some race in how log rotation postprocessing occurred that was fixed in git [1] that may be a cause for what you're seeing. So you might try testing from git sources as a first step to see if suddenly the summary starts looking correct.

Else the approach to finding where it's going wrong would be: Does conn.log look correct? If no, then it's a Bro problem. If yes, then it's a problem with how conn.log is parsed by $prefix/bin/trace-summary or how that python script is invoked by BroControl. You should be able to run that trace-summary script manually on one of your conn.log's to see if it actually gives sane output. That looks like:

    PYTHONPATH=/usr/local/bro/lib/broctl/ /usr/local/bro/bin/trace-summary -c -r -l /usr/local/bro/etc/networks.cfg conn.log

Doing a quick test myself I don't think I see anything overtly wrong, though there's some warnings from it that make me think payload may be under-reported. Another weird thing is that if a connection is between two "local" hosts specified in networks.cfg, that's categorized as "outgoing", not "incoming".

- Jon

[1] [BIT-970] - Bro Tracker

3

More specifically, could you paste a few lines from your conn.log? (feel free to obfuscate ip addresses).

  .Seth

4

Sure... any particular details you'd like to see?

5

Just a few full connection records for tcp traffic. There is typically a lot of detail you can tease out if you know what you're looking for. :slight_smile:

  .Seth

6

I’m curious about this statement. Can you share some examples?