Stats.log Growing Out of Control!!!

Zeek
1

@Dan: Both those files are there.

What my main issue seems to be is that my stats.log file is growing by 20-30MB every 5 minutes when the cron runs. I then get the email below in my original post.

I’m circling back here to hopefully find a resolution. I opened a thread in the Security Onion and tried limiting these events in my broctl.cfg. doesn’t seem to work. I’ve stopped Bro, deleted the stats dir, did brotcl install and then start, no go there either.

Here’s my SO thread for ref: https://groups.google.com/forum/#!topic/security-onion/bdmFGn3oj24

If anyone has any ideas or thoughts, please let me know. Any help is truly appreciated!

Thanks
Damon

2

I'd like to know why the stats-to-csv script is failing.
Could you apply the attached patch, and then send me
the contents of the "stats-to-csv failed" email?

To apply the patch you'll need to change directory to (where <prefix>
is the Bro install prefix directory):
<prefix>/lib/broctl/BroControl
In that directory you should see a file named "cron.py".

broctlcron.patch (388 Bytes)

3

Here’s the output after patching the cron.py file

stats-to-csv failed

[‘manager …’, ‘Traceback (most recent call last):’, ’ File “/opt/bro/share/broctl/scripts/stats-to-csv”, line 134, in ‘, ’ processNode(stats, wwwdir, “manager”, False)’, ’ File “/opt/bro/share/broctl/scripts/stats-to-csv”, line 87, in processNode’, ’ if m[1] != node:’, ‘IndexError: list index out of range’]

4

Your spool/stats.log file became corrupt somehow, and then you started
getting "stats-to-csv failed" emails every time cron ran. This was
preventing broctl from removing this file, which explains why you were
seeing such a fast rate of growth in the size of your
logs/stats/stats.log file (broctl cron always appends spool/stats.log
to logs/stats/stats.log).

To fix this, you could just delete the spool/stats.log file, then
you should no longer see the "stats-to-csv failed" emails.

I will improve broctl in the next release to mitigate this problem.
Thanks for reporting this issue.

5

Thanks Dan! That worked like a charm…no emails and my /nsm/bro/logs/stats/stats.log is no longer growing out of control.

Thanks again and I really appreciate all your help on this!

Damon