Hey All,
So I run bro instead of broctl. Currently, if I stop a running bro, and start it again, bro overwrites any previous log files...is there a way to change this behavior? Thank you.
James
Any takers on this? Would be really nice not to have to move the current logs every time I want to test something. Thanks all.
James
How would you like it to behave instead?
.Seth
To give me an option to append instead of overwrite. I imagine that since broctl does all the file management that this could be a command line option...
bro -i eth0 -n local.bro
where -n would be a no overwrite option. In a nutshell "if the files don't exist, create them, if they do, just append, without the header, to the current file". It could just be a single check on start.
How's that? Thanks Seth.
James
Seth,
Additionally, it would be wonderful to have bro re-load it's local.bro (or whatever) on SIGHUP. During testing my process is:
killall bro
move log files
make changes to scripts
bro -i eth0 local
Repeat. It's pretty tedious. Would be nice too see:
make changes to scripts
killal -HUP bro
That would reload bro local.bro and not overwrite the current log files.
Just some more thoughts...thanks Seth.
James
The problem with that approach is, that Bro would have to check that the mapping in the files still match. If you change the scripts in-between, the order or even the number of columns in the log-files might be different. Which would mean that the header do not fit the file content anymore.
hat might give you really difficult to parse log-files if you do it by accident.
Johanna
That makes sense, thanks Johanna. I'm guessing that not a lot of folks run bro outside of brocontrol in a production environment, and to be honest, if the cpu usage gets reduced in subsequent versions then I'll hop on the brocontrol boat and enjoy all the benefits. But until then bro commandline is where I sit. I agree that creating broken log files is not gonna work for anyone, which is why maybe having to specify if via command line and not make it a default would be the way to go. But maybe not. Thanks again Johanna and everyone really...bro is a crucial part of my continuous monitoring...I find more uses for it every day. I just which I was smart enough to give back to the community.
James
Ah! You just want to have file management (and perhaps full rotation?) added as a standalone script and not something that is added by broctl?
Johanna is right that with our current logging scheme we can't really append log files for multiple reasons but I could certainly pull together something that would give you decent log rotation without running broctl.
.Seth
Seth, that would absolutely rock!
James
To get basic log rotation working without running broctl, you only
need to add this in one of your Bro scripts:
redef Log::default_rotation_interval = 3600 secs;
However, that does not compress the rotated logs, and it will not
move them to another directory. If you want those features, then
you need to have broctl installed, and you need to add this line also:
redef Log::default_rotation_postprocessor_cmd = "archive-log";
The "archive-log" script will be executed by Bro (so it either needs
to be in Bro's PATH or you need to give the pathname).
In order to get the archive-log script to work, you need to
edit broctl.cfg as needed and run "broctl install". Then start Bro
manually and when Bro runs archive-log it should have all the
info it needs.
Excellent...thanks so much Daniel...I will start testing that out this weekend.
James
Yea so I lied I tested this already 
This works really well. I'm assuming that the number of seconds in "redef 3600 secs" and "LogRotationInterval = 3600" in broctl.conf have to match up. And as I have bro symlinked to /usr/local/bin/bro I also had to symlink:
lrwxrwxrwx 1 root root 47 Aug 21 14:24 /usr/local/bin/archive-log -> /usr/local/bro/share/broctl/scripts/archive-log
lrwxrwxrwx 1 root root 52 Aug 21 14:38 /usr/local/bin/broctl-config.sh -> /usr/local/bro/share/broctl/scripts/broctl-config.sh
But after that it ran like a champ. My last question is if I have these rotate every 24 hours, if I say...start this at 15:00, will it rotate at 15:00? Thank you.
James
To give me an option to append instead of overwrite. I imagine
that
since broctl does all the file management that this could be a
command
line option...Ah! You just want to have file management (and perhaps full
rotation?) added as a standalone script and not something that is
added by broctl?Johanna is right that with our current logging scheme we can't
really append log files for multiple reasons but I could certainly
pull together something that would give you decent log rotation
without running broctl..Seth
To get basic log rotation working without running broctl, you only
need to add this in one of your Bro scripts:redef Log::default_rotation_interval = 3600 secs;
However, that does not compress the rotated logs, and it will not
move them to another directory. If you want those features, then
you need to have broctl installed, and you need to add this line
also:redef Log::default_rotation_postprocessor_cmd = "archive-log";
The "archive-log" script will be executed by Bro (so it either needs
to be in Bro's PATH or you need to give the pathname).In order to get the archive-log script to work, you need to
edit broctl.cfg as needed and run "broctl install". Then start Bro
manually and when Bro runs archive-log it should have all the
info it needs.Yea so I lied I tested this already
This works really well. I'm
assuming that the number of seconds in "redef 3600 secs" and
"LogRotationInterval = 3600" in broctl.conf have to match up. And as I
Those values don't really need to match (but it might be best to
keep them in sync just to avoid confusion). Since you're not
starting Bro with broctl, then the only broctl config options
that will be used are the ones that the archive-log script uses
(you can look in that script to see which variables it uses,
if you're curious).
But after that it ran like a champ. My last question is if I have
these rotate every 24 hours, if I say...start this at 15:00, will it
rotate at 15:00? Thank you.James
In that case I think it will rotate at midnight.
That's beautiful...thanks so much Daniel!
James
FYI…this absolutely rotated at midnight…which is just perfect…thanks again.
James