# Beta schedule **URL:** \ **Category:** Development\ **Tags:** development\ **Created:** [October 12, 2011, 4:15pm UTC](https://community.zeek.org/t/beta-schedule/2060 "2011-10-12T16:15:20Z")\ **Posts on this page:** 15\ **Page:** 1
**Author:** ![robin](https://yyz1.discourse-cdn.com/flex011/user_avatar/community.zeek.org/robin/32/599_2.png) [@robin](https://community.zeek.org/u/robin)\ **Post date:** [October 12, 2011, 4:15pm UTC](https://community.zeek.org/t/beta-schedule/2060/1 "2011-10-12T16:15:20Z")
Sounds like everybody feels we're getting ready to do our beta release. Let's aim for early next week. Here's what I think is left to do:     - #511: Cleanup distribution a bit more (Robin)     - #601: Write docs for notice framework (Seth)     - #601: Cleanup and organize the documentation we have (Robin[1])     - #544: Get scan.bro into shape (Seth)     - #622: Some of these seem to addressed, others not. (John, can             you take the lead closing this ticket? Some might just need a             decision/reminder if what's we're currenty doing is the right             thing.) I'm fine pushing the rest to the final release. Let me know if anybody see any other showstopper. Robin [1] I'll make sure things get online and make a pass over the content if time permits. Anybody else, please also look through and edit what you find. ---
**Author:** ![Seth\_Hall3](https://avatars.discourse-cdn.com/v4/letter/s/d6d6ee/32.png) [@Seth\_Hall3](https://community.zeek.org/u/Seth_Hall3)\ **Post date:** [October 12, 2011, 4:22pm UTC](https://community.zeek.org/t/beta-schedule/2060/2 "2011-10-12T16:22:31Z")
Sounds great to me.   .Seth ---
**Author:** ![Gregor\_Maier](https://avatars.discourse-cdn.com/v4/letter/g/94ad74/32.png) [@Gregor\_Maier](https://community.zeek.org/u/Gregor_Maier)\ **Post date:** [October 12, 2011, 10:12pm UTC](https://community.zeek.org/t/beta-schedule/2060/3 "2011-10-12T22:12:38Z")
Maybe I missed them but what's with (all from #601):   \* Upgrade guide   \* How-to for new logging framework (how to filter, log file format,     maybe how to add columns to the log file (i.e., how to use     c$PROTOCOL))   \* Complete script reference (though that can wait to final release) cu gregor ---
**Author:** ![Jonathan\_Siwek](https://avatars.discourse-cdn.com/v4/letter/j/e68b1a/32.png) [@Jonathan\_Siwek](https://community.zeek.org/u/Jonathan_Siwek)\ **Post date:** [October 13, 2011, 2:50am UTC](https://community.zeek.org/t/beta-schedule/2060/4 "2011-10-13T02:50:10Z")
> Maybe I missed them but what's with (all from #601): > > \* Upgrade guide tracked in more detail here: [http://tracker.bro-ids.org/bro/ticket/510](http://tracker.bro-ids.org/bro/ticket/510) and put up on the site here: [http://www.bro-ids.org/documentation/upgrade.html](http://www.bro-ids.org/documentation/upgrade.html) > \* How-to for new logging framework (how to filter, log file format, >    maybe how to add columns to the log file (i.e., how to use >    c$PROTOCOL)) Think that's mostly done: [http://www.bro-ids.org/development/logging.html](http://www.bro-ids.org/development/logging.html) > \* Complete script reference (though that can wait to final release) Does that mean the generated bro script docs (think I'm going to start calling them "Broxygen docs" for short) ? If so, I didn't look at how far Robin got on the www new-git branch, but part of that will help incorporate those docs into the website. Another general task might be to cleanup the scattered TODO/XXX's. - Jon ---
**Author:** ![robin](https://yyz1.discourse-cdn.com/flex011/user_avatar/community.zeek.org/robin/32/599_2.png) [@robin](https://community.zeek.org/u/robin)\ **Post date:** [October 18, 2011, 4:27am UTC](https://community.zeek.org/t/beta-schedule/2060/5 "2011-10-18T04:27:46Z")
Updates: - I have spent some time redoing how git files are pulled in by   [www.bro-ids.org](http://www.bro-ids.org). That now allows us to have separate sets of docs   for release, beta, and master; and generally makes usage easier. - I have also cleaned up the submodules to unify README structure,   licenses, etc. - I'm making releases (with tarballs) for all the submodules, except   broctl. There shouldn't be changing much for them during the Bro   beta period, and even if, we can always release new versions. Jus   trying to get them out of the way. I'm planing to push all this out tomorrow. Robin ---
**Author:** ![robin](https://yyz1.discourse-cdn.com/flex011/user_avatar/community.zeek.org/robin/32/599_2.png) [@robin](https://community.zeek.org/u/robin)\ **Post date:** [October 18, 2011, 7:36pm UTC](https://community.zeek.org/t/beta-schedule/2060/6 "2011-10-18T19:36:34Z")
> - I'm making releases (with tarballs) for all the submodules. I'm trying to build tar-files for all our submodules but I'm running into a number of things. First, afaics, not all modules yet provide the capability to create the tarballs (or at least I can't directly figure out how). Specifically:     - capstats, binpac, and bro-aux: they have "dist" Makefile       targets, but running thise gives me:         make[1]: \*\*\* No rule to make target `package\_source'. Stop.     - broccoli-ruby: No "dist" Makefile target and I have no idea how     to do that for Ruby packages (I've just added an empty dummy one.) Then, the top-level Bro "make-src-packages" also builts the Broccoli and BroControl tgz. I think it would be better to leave that to the submodules themselves, otherwise things get inconsistent (because we have all these further submodules as well, which do it themselves). So I've removed those lines from make-src-packages. I also noticed that Bro's make-src-packages essentially just tars everything that's in the source directory (except for some tmp files etc.). There's a problem with that: I'm sure that sometimes I will have additional files lying around there which will then end up in the distribution. Is there a way to make sure that only stuff under version control gets actually into the tgz? (And please remind me: what was the reason that we can't just let CMake do the tarballs?) Here's how I'd like to do the tarballs: I've written a script that recursively goes through all submodules and runs "make dist" there. It's in aux/devel-tools now. The script then collects all the resulting tarballs at the top-level with the right naming/path scheme for copying them over to the web server. For that to work, we need to ensure that (1) each Makefile has indeed a working "dist" target, and (2) that that targets outputs the path to the generated tarball so that the top-level script can pick it up. Most CMake-builds do (2) already and for the Python modules I've added output in the form of "Package: relative/path/to/tgz" (which is then grepped for).[1] Robin [1] It works to write out multiple lines like that if more than one tgz is built (like bro vs bro-all). Btw, I'm wondering if we should name them the opposite way: bro-2.0.tgz would include all submodules, and bro-2.0-minimal.tgz would not. Same scheme for BroControl and Broccoli (where we don't do two tarballs yet). ---
**Author:** ![Jonathan\_Siwek](https://avatars.discourse-cdn.com/v4/letter/j/e68b1a/32.png) [@Jonathan\_Siwek](https://community.zeek.org/u/Jonathan_Siwek)\ **Post date:** [October 18, 2011, 8:41pm UTC](https://community.zeek.org/t/beta-schedule/2060/7 "2011-10-18T20:41:14Z")
> > - I'm making releases (with tarballs) for all the submodules. > > I'm trying to build tar-files for all our submodules but I'm running > into a number of things. > > First, afaics, not all modules yet provide the capability to create > the tarballs (or at least I can't directly figure out how). > Specifically: > >    - capstats, binpac, and bro-aux: they have "dist" Makefile >      targets, but running thise gives me: > >        make[1]: \*\*\* No rule to make target `package\_source'. Stop. > >    - broccoli-ruby: No "dist" Makefile target and I have no idea how >    to do that for Ruby packages (I've just added an empty dummy one.) The package\_source target is something provided by CMake, and adding it would require changing top-level CMakeLists.txt to include the cmake/ConfigurePackaging.cmake script, but I don't think that's ultimately the way to go... > Then, the top-level Bro "make-src-packages" also builts the Broccoli > and BroControl tgz. I think it would be better to leave that to the > submodules themselves, otherwise things get inconsistent (because we > have all these further submodules as well, which do it themselves). > So I've removed those lines from make-src-packages. Yeah, sounds good -- repos for which we we want to create recursive packages should handle that themselves... > I also noticed that Bro's make-src-packages essentially just tars > everything that's in the source directory (except for some tmp files > etc.). There's a problem with that: I'm sure that sometimes I will > have additional files lying around there which will then end up in the > distribution. Is there a way to make sure that only stuff under > version control gets actually into the tgz? I think we need to use a combination of `git ls-files` to generate a package manifest as input to `tar -T`. The trick is that all non-recursive source packages need to include the contents of their cmake/ submodule if it exists. For the recursive package bundles (Bro, BroControl, Broccoli), the output of a recursive `git ls-files` needs to be altered to prepend the right relative path of submodules. > (And please remind me: > what was the reason that we can't just let CMake do the tar balls? So that the packager doesn't need to have CMake as a dependency. And actually it's worse than that: the packaging configuration provided by CMake is also currently tied to the full suite of ./configure dependency checks as if one were going to actually build the project. (This was originally pointed out as a pretty bad inconvenience by Craig Leres) > Here's how I'd like to do the tarballs: I've written a script that > recursively goes through all submodules and runs "make dist" there. > It's in aux/devel-tools now. The script then collects all the > resulting tarballs at the top-level with the right naming/path scheme > for copying them over to the web server. For that to work, we need to > ensure that (1) each Makefile has indeed a working "dist" target, and > (2) that that targets outputs the path to the generated tarball so > that the top-level script can pick it up. Most CMake-builds do (2) > already and for the Python modules I've added output in the form of > "Package: relative/path/to/tgz" (which is then grepped for).[1] > > Robin > > [1] It works to write out multiple lines like that if more than one > tgz is built (like bro vs bro-all). Btw, I'm wondering if we should > name them the opposite way: bro-2.0.tgz would include all submodules, > and bro-2.0-minimal.tgz would not. Same scheme for BroControl and > Broccoli (where we don't do two tarballs yet). Sounds good, I'll work on cleaning things up. - Jon ---
**Author:** ![robin](https://yyz1.discourse-cdn.com/flex011/user_avatar/community.zeek.org/robin/32/599_2.png) [@robin](https://community.zeek.org/u/robin)\ **Post date:** [October 18, 2011, 9:13pm UTC](https://community.zeek.org/t/beta-schedule/2060/8 "2011-10-18T21:13:14Z")
> Yeah, sounds good -- repos for which we we want to create recursive > packages should handle that themselves... Right. > I think we need to use a combination of `git ls-files` to generate a package manifest as input to `tar -T`. I just remembered "git clean". That could delete everything we don't want? > So that the packager doesn't need to have CMake as a dependency. And > actually it's worse than that: the packaging configuration provided by > CMake is also currently tied to the full suite of ./configure > dependency checks as if one were going to actually build the project. > (This was originally pointed out as a pretty bad inconvenience by > Craig Leres) Hmm, ok. Not sure I'd see that as a showstopper, but I don't recall the full discussion right now. > Sounds good, I'll work on cleaning things up. Ok, thanks! Robin ---
**Author:** ![robin](https://yyz1.discourse-cdn.com/flex011/user_avatar/community.zeek.org/robin/32/599_2.png) [@robin](https://community.zeek.org/u/robin)\ **Post date:** [October 19, 2011, 1:24am UTC](https://community.zeek.org/t/beta-schedule/2060/9 "2011-10-19T01:24:05Z")
We are getting really close! >     - #511: Cleanup distribution a bit more (Robin) >     - #601: Cleanup and organize the documentation we have (Robin[1]) Done with these, except that I'm still planing to make a pass over the upgrade docs. >     - #622: Some of these seem to addressed, others not. (John, can >             you take the lead closing this ticket? Some might just need a >             decision/reminder if what's we're currenty doing is the right >             thing.) Closed. >     - #601: Write docs for notice framework (Seth) >     - #544: Get scan.bro into shape (Seth) I think Seth is working on these. Seth, one more: Almost all tests pass for me now (just updated a few more baselines). The one exception is bro-testing:tests.m57-long, which still gives me a long list of diffs. That's waiting for you I'm afraid ... Once these are done, I'll be happy to push out the beta. ---
**Author:** ![Jonathan\_Siwek](https://avatars.discourse-cdn.com/v4/letter/j/e68b1a/32.png) [@Jonathan\_Siwek](https://community.zeek.org/u/Jonathan_Siwek)\ **Post date:** [October 19, 2011, 4:13am UTC](https://community.zeek.org/t/beta-schedule/2060/10 "2011-10-19T04:13:17Z")
> I just remembered "git clean". That could delete everything we don't > want? Added that part to the `make-release` script. And all the `make dist` targets should all work better I think; try it out? - Jon ---
**Author:** ![robin](https://yyz1.discourse-cdn.com/flex011/user_avatar/community.zeek.org/robin/32/599_2.png) [@robin](https://community.zeek.org/u/robin)\ **Post date:** [October 19, 2011, 7:40pm UTC](https://community.zeek.org/t/beta-schedule/2060/11 "2011-10-19T19:40:07Z")
Thanks, looks good, but one more request: > Added that part to the `make-release` script. The disadadvantage of this is that is erases all local changes right inside the repository I'm working in. Could "make dist" instead run the "git clean" inside the temporary copy that it's creating? (Which would probably mean that it needs to copy .git over first, then run git clean, and then rm .git). That way nothing I have lying around locally would be touched. I'm sure otherwise I will at some point accidentally delete something. Also, the "git submodule foreach ..." descends into all subdirectories but doesn't execute inside the top-levle parent module. I think we need another "clean ..." at the top-level as well, right? Robin ---
**Author:** ![Jonathan\_Siwek](https://avatars.discourse-cdn.com/v4/letter/j/e68b1a/32.png) [@Jonathan\_Siwek](https://community.zeek.org/u/Jonathan_Siwek)\ **Post date:** [October 19, 2011, 8:04pm UTC](https://community.zeek.org/t/beta-schedule/2060/12 "2011-10-19T20:04:24Z")
> The disadadvantage of this is that is erases all local changes right > inside the repository I'm working in. Could "make dist" instead run > the "git clean" inside the temporary copy that it's creating? (Which > would probably mean that it needs to copy .git over first, then run > git clean, and then rm .git). That way nothing I have lying around > locally would be touched. I'm sure otherwise I will at some point > accidentally delete something. Ok, I'll change that. > Also, the "git submodule foreach ..." descends into all subdirectories > but doesn't execute inside the top-levle parent module. I think we > need another "clean ..." at the top-level as well, right? yeah, forgot about that, but I'll remove that entirely since it will be done in the `make dists` - Jon ---
**Author:** ![robin](https://yyz1.discourse-cdn.com/flex011/user_avatar/community.zeek.org/robin/32/599_2.png) [@robin](https://community.zeek.org/u/robin)\ **Post date:** [October 20, 2011, 12:50am UTC](https://community.zeek.org/t/beta-schedule/2060/13 "2011-10-20T00:50:41Z")
It just happened ... Typing "make distclean" is deleting the repos in testing/external/\*. So let's please switch that back to the previous "rm -rf build". Robin ---
**Author:** ![Jonathan\_Siwek](https://avatars.discourse-cdn.com/v4/letter/j/e68b1a/32.png) [@Jonathan\_Siwek](https://community.zeek.org/u/Jonathan_Siwek)\ **Post date:** [October 21, 2011, 4:28pm UTC](https://community.zeek.org/t/beta-schedule/2060/14 "2011-10-21T16:28:38Z")
> It just happened ... Typing "make distclean" is deleting the repos in > testing/external/\*. So let's please switch that back to the previous > "rm -rf build". Sorry. I will fix it. - Jon ---
**Author:** ![system](https://canada1.discourse-cdn.com/flex011/uploads/zeek/original/1X/f09d732bc2cc7c7cc7e35db67cf4e1d5233ce7a7.png) [@system](https://community.zeek.org/u/system)\ **Post date:** [May 6, 2022, 3:39pm UTC](https://community.zeek.org/t/beta-schedule/2060/15 "2022-05-06T15:39:53Z")