log rotation leaving conn.log unrotated

Zeek
1

Hi.

Before I ask yet another question, I wanted to say thanks to all who helped me
during the last weeks. :slight_smile: All tips where very helpful and fast! I hope I can repay
for this by contributing in the future.

I have this simple bro file:

redef Log::default_rotation_postprocessor_cmd = “./postrotate.sh”;
redef Log::default_rotation_interval = 10 sec;

with postrotate.sh just printing the parameters:

#!/bin/sh
echo “-1-”
echo $1
echo $2
echo $3
echo $4
echo $5
echo $6
echo “-2-”

Now when bro is terminated via CTRL-C, the script is called:

1430147916.038582 received termination signal
1430147916.038582 1865 packets received on interface eth0, 45 dropped

-1-
files.2015-04-27-17-18-30.log
files
15-04-27_17.18.30
15-04-27_17.18.36
1
ascii
-2-
-1-
http.2015-04-27-17-18-30.log
http
15-04-27_17.18.30
15-04-27_17.18.36
1
ascii
-2-
-1-
weird.2015-04-27-17-18-30.log
weird
15-04-27_17.18.30
15-04-27_17.18.36
1
ascii
-2-
-1-
conn.2015-04-27-17-18-30.log
conn
15-04-27_17.18.30
15-04-27_17.18.36
1
ascii
-2-
-1-
reporter.2015-04-27-17-18-36.log
reporter
15-04-27_17.18.36
15-04-27_17.18.36
1
ascii
-2-

After that there is still a conn.log around. Why is this file not rotated?

When I restart bro now, the conn.log seems to be overwritten and
entries for example in files.log reference a uid not found in any of
the conn.logs.

I could not figure out why rotation works for most of the logs, but
not for conn.log.

Franky

2

In your example, conn.log was rotated (the new filename was conn.2015-04-27-17-18-30.log). The conn.log file you saw
after terminating Bro was most likely created in the short time span
between rotating conn.log and Bro termination.

3

hi,

In your example, conn.log was rotated (the new filename was conn.2015-04-27-17-18-30.log). The conn.log file you saw after terminating Bro was most likely created in the short time span between rotating conn.log and Bro termination.

I think there is more to this. If repeat the following steps I do loose some entries in conn.log:

  1. start bro
  2. produce some traffic
  3. stop bro via CTRL-C
  4. restart bro
  5. wait for log rotation
  6. stop bro via CTRL-C

grep for conn_uids from files.log. For some entries in files.log there will be no match in any
of the conn.logs. My workaround for now is to append the stale
conn.log to the last rotated log on shutdown:

file_name=$1
base_name=$2
from=$3
to=$4
terminating=$5
writer=$6

echo “[+] LOG: Rotating $file_name (base: $base_name, from: $from, to: $to, terminating: $terminating, writer: $writer)”

if [ $terminating -eq 1 -a -f $base_name.log ]; then
echo "terminating. appending stale $base_name.log "

cut timestamp

head -n-1 $file_name > $file_name.tmp
mv $file_name.tmp $file_name

cut header

tail -n +9 $base_name.log >> $file_name
mv $base_name.log done.log
fi

I don’t have the time right now, but I will look further into this.

Franky