# Connection summary email inaccuracies

**URL:** <https://community.zeek.org/t/connection-summary-email-inaccuracies/2793>\
**Category:** Zeek\
**Created:** [August 27, 2013, 3:00pm UTC](https://community.zeek.org/t/connection-summary-email-inaccuracies/2793 "2013-08-27T15:00:26Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Chris\_Roose](https://avatars.discourse-cdn.com/v4/letter/c/2bfe46/32.png) [@Chris\_Roose](https://community.zeek.org/u/Chris_Roose)\
**Post date:** [August 27, 2013, 3:00pm UTC](https://community.zeek.org/t/connection-summary-email-inaccuracies/2793/1 "2013-08-27T15:00:26Z")

</div>

Hello,

I've used Bro on and off for a couple years and love its unix-ness and  
application-layer smarts. I use it to augment my NetFlow and SNMP data,  
and it gives me just enough information to complement those logs. I  
haven't dug into the scripting and IDS aspects yet, but I hope to soon.

I have an issue with the connection summary email. Aside from the fact  
that I could do without it altogether, because it doesn't really tell me  
anything that NetFlow can't, I'm confused by how inaccurate the  
information in the email seems to be.

To take the example that always jumps out at me, here are the incoming  
port statistics from this morning's email.

> == Incoming === 2013-08-25-23-50-18 - 2013-08-26-23-19-23

&nbsp;&nbsp;&nbsp;- Connections 306.0 - Payload 137.0m -  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Ports |  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;9997 78.1% |  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;3 9.2% |  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;514 5.2% |  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;50664 3.6% |  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;22 1.3% |  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;52145 0.7% |  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;51222 0.7% |  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;52140 0.3% |  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;51735 0.3% |  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;51724 0.3% |

The reason I know something is strange about this is that I get NetFlow  
data around the clock from three different sites on ports 9997, 9998,  
and 9999. How could it be that one site accounts for 78.1% of all of my  
incoming traffic and the other two are nowhere to be seen? Also, the  
number of connections and payload information is way off. Here is the  
same information queried from NetFlow:

Port Flows(%)  
0 4831(13.0)  
9999 1985( 5.3)  
9997 1797( 4.8)  
9998 1510( 4.1)  
22 559( 1.5)  
123 398( 1.1)  
64115 349( 0.9)  
65138 162( 0.4)  
40767 135( 0.4)  
13496 120( 0.3)

Summary: total flows: 37254, total bytes: 2.1 G, total packets: 1.7 M,  
avg bps: 194612, avg pps: 19, avg bpp: 1237  
Time window: 2013-08-25 23:49:42 - 2013-08-26 23:24:49  
Total flows processed: 102134, Blocks skipped: 0, Bytes read: 5318892

Netstat doesn't indicate any dropped packets, and conn.log doesn't  
indicate any missed\_bytes. Can anyone shed some light on why bro could  
be so wrong about these statistics? Would it matter that I am using a  
single instance of bro to monitor two interfaces (bro -i em0 -i em1)?

Thanks for any help you can provide...

Best,  
Chris

---

<div class="post-metadata">

**Author:** ![Siwek\_Jon](https://avatars.discourse-cdn.com/v4/letter/s/90db22/32.png) [@Siwek\_Jon](https://community.zeek.org/u/Siwek_Jon)\
**Post date:** [August 28, 2013, 5:30pm UTC](https://community.zeek.org/t/connection-summary-email-inaccuracies/2793/2 "2013-08-28T17:30:48Z")

</div>

> Netstat doesn't indicate any dropped packets, and conn.log doesn't  
> indicate any missed\_bytes. Can anyone shed some light on why bro could  
> be so wrong about these statistics? Would it matter that I am using a  
> single instance of bro to monitor two interfaces (bro -i em0 -i em1)?

The interface thing shouldn't matter. What version of Bro are you using? I think there was some race in how log rotation postprocessing occurred that was fixed in git [1] that may be a cause for what you're seeing. So you might try testing from git sources as a first step to see if suddenly the summary starts looking correct.

Else the approach to finding where it's going wrong would be: Does conn.log look correct? If no, then it's a Bro problem. If yes, then it's a problem with how conn.log is parsed by $prefix/bin/trace-summary or how that python script is invoked by BroControl. You should be able to run that trace-summary script manually on one of your conn.log's to see if it actually gives sane output. That looks like:

&nbsp;&nbsp;&nbsp;&nbsp;PYTHONPATH=/usr/local/bro/lib/broctl/ /usr/local/bro/bin/trace-summary -c -r -l /usr/local/bro/etc/networks.cfg conn.log

Doing a quick test myself I don't think I see anything overtly wrong, though there's some warnings from it that make me think payload may be under-reported. Another weird thing is that if a connection is between two "local" hosts specified in networks.cfg, that's categorized as "outgoing", not "incoming".

- Jon

[1] [[BIT-970] - Bro Tracker](https://bro-tracker.atlassian.net/browse/BIT-970)

---

<div class="post-metadata">

**Author:** ![Seth\_Hall3](https://avatars.discourse-cdn.com/v4/letter/s/d6d6ee/32.png) [@Seth\_Hall3](https://community.zeek.org/u/Seth_Hall3)\
**Post date:** [August 28, 2013, 11:23pm UTC](https://community.zeek.org/t/connection-summary-email-inaccuracies/2793/3 "2013-08-28T23:23:10Z")

</div>

More specifically, could you paste a few lines from your conn.log? (feel free to obfuscate ip addresses).

&nbsp;&nbsp;.Seth

---

<div class="post-metadata">

**Author:** ![Chris\_Roose](https://avatars.discourse-cdn.com/v4/letter/c/2bfe46/32.png) [@Chris\_Roose](https://community.zeek.org/u/Chris_Roose)\
**Post date:** [August 29, 2013, 12:04am UTC](https://community.zeek.org/t/connection-summary-email-inaccuracies/2793/4 "2013-08-29T00:04:13Z")

</div>

Sure... any particular details you'd like to see?

---

<div class="post-metadata">

**Author:** ![Seth\_Hall3](https://avatars.discourse-cdn.com/v4/letter/s/d6d6ee/32.png) [@Seth\_Hall3](https://community.zeek.org/u/Seth_Hall3)\
**Post date:** [August 29, 2013, 1:24am UTC](https://community.zeek.org/t/connection-summary-email-inaccuracies/2793/5 "2013-08-29T01:24:50Z")

</div>

Just a few full connection records for tcp traffic. There is typically a lot of detail you can tease out if you know what you're looking for. 🙂

&nbsp;&nbsp;.Seth

---

<div class="post-metadata">

**Author:** ![Mike\_Sconzo](https://avatars.discourse-cdn.com/v4/letter/m/d6d6ee/32.png) [@Mike\_Sconzo](https://community.zeek.org/u/Mike_Sconzo)\
**Post date:** [August 29, 2013, 2:54am UTC](https://community.zeek.org/t/connection-summary-email-inaccuracies/2793/6 "2013-08-29T02:54:03Z")

</div>

I’m curious about this statement. Can you share some examples?

---

<div class="post-metadata">

**Author:** ![system](https://canada1.discourse-cdn.com/flex011/uploads/zeek/original/1X/f09d732bc2cc7c7cc7e35db67cf4e1d5233ce7a7.png) [@system](https://community.zeek.org/u/system)\
**Post date:** [May 6, 2022, 3:41pm UTC](https://community.zeek.org/t/connection-summary-email-inaccuracies/2793/7 "2022-05-06T15:41:11Z")

</div>


