# Append instead of overwrite **URL:** \ **Category:** Zeek\ **Created:** [July 17, 2014, 3:33pm UTC](https://community.zeek.org/t/append-instead-of-overwrite/3186 "2014-07-17T15:33:21Z")\ **Posts on this page:** 16\ **Page:** 1
**Author:** ![James\_inthe\_box](https://avatars.discourse-cdn.com/v4/letter/j/6f9a4e/32.png) [@James\_inthe\_box](https://community.zeek.org/u/James_inthe_box)\ **Post date:** [July 17, 2014, 3:33pm UTC](https://community.zeek.org/t/append-instead-of-overwrite/3186/1 "2014-07-17T15:33:21Z")
Hey All, So I run bro instead of broctl. Currently, if I stop a running bro, and start it again, bro overwrites any previous log files...is there a way to change this behavior? Thank you. James ---
**Author:** ![James\_inthe\_box](https://avatars.discourse-cdn.com/v4/letter/j/6f9a4e/32.png) [@James\_inthe\_box](https://community.zeek.org/u/James_inthe_box)\ **Post date:** [August 15, 2014, 11:59am UTC](https://community.zeek.org/t/append-instead-of-overwrite/3186/2 "2014-08-15T11:59:46Z")
Any takers on this? Would be really nice not to have to move the current logs every time I want to test something. Thanks all. James ---
**Author:** ![Seth\_Hall3](https://avatars.discourse-cdn.com/v4/letter/s/d6d6ee/32.png) [@Seth\_Hall3](https://community.zeek.org/u/Seth_Hall3)\ **Post date:** [August 15, 2014, 3:46pm UTC](https://community.zeek.org/t/append-instead-of-overwrite/3186/3 "2014-08-15T15:46:54Z")
How would you like it to behave instead?   .Seth ---
**Author:** ![James\_inthe\_box](https://avatars.discourse-cdn.com/v4/letter/j/6f9a4e/32.png) [@James\_inthe\_box](https://community.zeek.org/u/James_inthe_box)\ **Post date:** [August 15, 2014, 3:53pm UTC](https://community.zeek.org/t/append-instead-of-overwrite/3186/4 "2014-08-15T15:53:38Z")
To give me an option to append instead of overwrite. I imagine that since broctl does all the file management that this could be a command line option... bro -i eth0 -n local.bro where -n would be a no overwrite option. In a nutshell "if the files don't exist, create them, if they do, just append, without the header, to the current file". It could just be a single check on start. How's that? Thanks Seth. James ---
**Author:** ![James\_inthe\_box](https://avatars.discourse-cdn.com/v4/letter/j/6f9a4e/32.png) [@James\_inthe\_box](https://community.zeek.org/u/James_inthe_box)\ **Post date:** [August 15, 2014, 4:28pm UTC](https://community.zeek.org/t/append-instead-of-overwrite/3186/5 "2014-08-15T16:28:54Z")
Seth, Additionally, it would be wonderful to have bro re-load it's local.bro (or whatever) on SIGHUP. During testing my process is: killall bro move log files make changes to scripts bro -i eth0 local Repeat. It's pretty tedious. Would be nice too see: make changes to scripts killal -HUP bro That would reload bro local.bro and not overwrite the current log files. Just some more thoughts...thanks Seth. James ---
**Author:** ![johanna](https://avatars.discourse-cdn.com/v4/letter/j/50afbb/32.png) [@johanna](https://community.zeek.org/u/johanna)\ **Post date:** [August 15, 2014, 4:37pm UTC](https://community.zeek.org/t/append-instead-of-overwrite/3186/6 "2014-08-15T16:37:07Z")
The problem with that approach is, that Bro would have to check that the mapping in the files still match. If you change the scripts in-between, the order or even the number of columns in the log-files might be different. Which would mean that the header do not fit the file content anymore. hat might give you really difficult to parse log-files if you do it by accident. Johanna ---
**Author:** ![James\_inthe\_box](https://avatars.discourse-cdn.com/v4/letter/j/6f9a4e/32.png) [@James\_inthe\_box](https://community.zeek.org/u/James_inthe_box)\ **Post date:** [August 15, 2014, 4:54pm UTC](https://community.zeek.org/t/append-instead-of-overwrite/3186/7 "2014-08-15T16:54:28Z")
That makes sense, thanks Johanna. I'm guessing that not a lot of folks run bro outside of brocontrol in a production environment, and to be honest, if the cpu usage gets reduced in subsequent versions then I'll hop on the brocontrol boat and enjoy all the benefits. But until then bro commandline is where I sit. I agree that creating broken log files is not gonna work for anyone, which is why maybe having to specify if via command line and not make it a default would be the way to go. But maybe not. Thanks again Johanna and everyone really...bro is a crucial part of my continuous monitoring...I find more uses for it every day. I just which I was smart enough to give back to the community. James ---
**Author:** ![Seth\_Hall3](https://avatars.discourse-cdn.com/v4/letter/s/d6d6ee/32.png) [@Seth\_Hall3](https://community.zeek.org/u/Seth_Hall3)\ **Post date:** [August 15, 2014, 6:13pm UTC](https://community.zeek.org/t/append-instead-of-overwrite/3186/8 "2014-08-15T18:13:39Z")
Ah! You just want to have file management (and perhaps full rotation?) added as a standalone script and not something that is added by broctl? Johanna is right that with our current logging scheme we can't really append log files for multiple reasons but I could certainly pull together something that would give you decent log rotation without running broctl.   .Seth ---
**Author:** ![James\_inthe\_box](https://avatars.discourse-cdn.com/v4/letter/j/6f9a4e/32.png) [@James\_inthe\_box](https://community.zeek.org/u/James_inthe_box)\ **Post date:** [August 15, 2014, 6:24pm UTC](https://community.zeek.org/t/append-instead-of-overwrite/3186/9 "2014-08-15T18:24:18Z")
Seth, that would absolutely rock! James ---
**Author:** ![Daniel\_Thayer](https://avatars.discourse-cdn.com/v4/letter/d/8dc957/32.png) [@Daniel\_Thayer](https://community.zeek.org/u/Daniel_Thayer)\ **Post date:** [August 21, 2014, 7:36pm UTC](https://community.zeek.org/t/append-instead-of-overwrite/3186/10 "2014-08-21T19:36:02Z")
To get basic log rotation working without running broctl, you only need to add this in one of your Bro scripts: redef Log::default\_rotation\_interval = 3600 secs; However, that does not compress the rotated logs, and it will not move them to another directory. If you want those features, then you need to have broctl installed, and you need to add this line also: redef Log::default\_rotation\_postprocessor\_cmd = "archive-log"; The "archive-log" script will be executed by Bro (so it either needs to be in Bro's PATH or you need to give the pathname). In order to get the archive-log script to work, you need to edit broctl.cfg as needed and run "broctl install". Then start Bro manually and when Bro runs archive-log it should have all the info it needs. ---
**Author:** ![James\_inthe\_box](https://avatars.discourse-cdn.com/v4/letter/j/6f9a4e/32.png) [@James\_inthe\_box](https://community.zeek.org/u/James_inthe_box)\ **Post date:** [August 21, 2014, 7:57pm UTC](https://community.zeek.org/t/append-instead-of-overwrite/3186/11 "2014-08-21T19:57:16Z")
Excellent...thanks so much Daniel...I will start testing that out this weekend. James ---
**Author:** ![James\_inthe\_box](https://avatars.discourse-cdn.com/v4/letter/j/6f9a4e/32.png) [@James\_inthe\_box](https://community.zeek.org/u/James_inthe_box)\ **Post date:** [August 21, 2014, 9:13pm UTC](https://community.zeek.org/t/append-instead-of-overwrite/3186/12 "2014-08-21T21:13:25Z")
Yea so I lied I tested this already 😃 This works really well. I'm assuming that the number of seconds in "redef 3600 secs" and "LogRotationInterval = 3600" in broctl.conf have to match up. And as I have bro symlinked to /usr/local/bin/bro I also had to symlink: lrwxrwxrwx 1 root root 47 Aug 21 14:24 /usr/local/bin/archive-log -\> /usr/local/bro/share/broctl/scripts/archive-log lrwxrwxrwx 1 root root 52 Aug 21 14:38 /usr/local/bin/broctl-config.sh -\> /usr/local/bro/share/broctl/scripts/broctl-config.sh But after that it ran like a champ. My last question is if I have these rotate every 24 hours, if I say...start this at 15:00, will it rotate at 15:00? Thank you. James ---
**Author:** ![Daniel\_Thayer](https://avatars.discourse-cdn.com/v4/letter/d/8dc957/32.png) [@Daniel\_Thayer](https://community.zeek.org/u/Daniel_Thayer)\ **Post date:** [August 21, 2014, 10:07pm UTC](https://community.zeek.org/t/append-instead-of-overwrite/3186/13 "2014-08-21T22:07:00Z")
> > > > To give me an option to append instead of overwrite. I imagine > > > > that > > > > since broctl does all the file management that this could be a > > > > command > > > > line option... > > > > > > Ah! You just want to have file management (and perhaps full > > > rotation?) added as a standalone script and not something that is > > > added by broctl? > > > > > > Johanna is right that with our current logging scheme we can't > > > really append log files for multiple reasons but I could certainly > > > pull together something that would give you decent log rotation > > > without running broctl. > > > > > >     .Seth > > > > To get basic log rotation working without running broctl, you only > > need to add this in one of your Bro scripts: > > > > redef Log::default\_rotation\_interval = 3600 secs; > > > > However, that does not compress the rotated logs, and it will not > > move them to another directory. If you want those features, then > > you need to have broctl installed, and you need to add this line > > also: > > > > redef Log::default\_rotation\_postprocessor\_cmd = "archive-log"; > > > > The "archive-log" script will be executed by Bro (so it either needs > > to be in Bro's PATH or you need to give the pathname). > > > > In order to get the archive-log script to work, you need to > > edit broctl.cfg as needed and run "broctl install". Then start Bro > > manually and when Bro runs archive-log it should have all the > > info it needs. > > Yea so I lied I tested this already 😃 This works really well. I'm > assuming that the number of seconds in "redef 3600 secs" and > "LogRotationInterval = 3600" in broctl.conf have to match up. And as I Those values don't really need to match (but it might be best to keep them in sync just to avoid confusion). Since you're not starting Bro with broctl, then the only broctl config options that will be used are the ones that the archive-log script uses (you can look in that script to see which variables it uses, if you're curious). > But after that it ran like a champ. My last question is if I have > these rotate every 24 hours, if I say...start this at 15:00, will it > rotate at 15:00? Thank you. > > James In that case I think it will rotate at midnight. ---
**Author:** ![James\_inthe\_box](https://avatars.discourse-cdn.com/v4/letter/j/6f9a4e/32.png) [@James\_inthe\_box](https://community.zeek.org/u/James_inthe_box)\ **Post date:** [August 21, 2014, 10:29pm UTC](https://community.zeek.org/t/append-instead-of-overwrite/3186/14 "2014-08-21T22:29:21Z")
That's beautiful...thanks so much Daniel! James ---
**Author:** ![James\_inthe\_box](https://avatars.discourse-cdn.com/v4/letter/j/6f9a4e/32.png) [@James\_inthe\_box](https://community.zeek.org/u/James_inthe_box)\ **Post date:** [August 23, 2014, 11:43am UTC](https://community.zeek.org/t/append-instead-of-overwrite/3186/15 "2014-08-23T11:43:54Z")
FYI…this absolutely rotated at midnight…which is just perfect…thanks again. James ---
**Author:** ![system](https://canada1.discourse-cdn.com/flex011/uploads/zeek/original/1X/f09d732bc2cc7c7cc7e35db67cf4e1d5233ce7a7.png) [@system](https://community.zeek.org/u/system)\ **Post date:** [May 6, 2022, 3:41pm UTC](https://community.zeek.org/t/append-instead-of-overwrite/3186/16 "2022-05-06T15:41:54Z")