[Announcement] Major NSF funding for Bro development

Some of you folks may remember that back in January we did a survey
soliciting input on operational Bro deployments in the hope of
attracting funding for Bro. Today, the Bro team is jazzed to
announce that the National Science Foundation has awarded a grant of
almost $3M to the International Computer Science Institute (ICSI)
and the National Center for Supercomputing Applications (NCSA) for
extensive Bro development.

The funded project aims specifically at addressing much of the
feedback that we have received from Bro users over the years. It
will enable us to refine many of the rough edges that the system has
accumulated over time[*], improve Bro's performance significantly,
and also make it much easier for the community to contribute to the
project.

For further information, see the joint ICSI/NCSA press release at:

    http://www.ncsa.illinois.edu/News/10/0824NSFawards.html

While we are still in process of planning our next steps, we'd
already like to encourage you folks to take an active role in
shaping the course of Bro's future development. In response to our
earlier survey, many of you have already sent in ideas on what kind
of improvements and new functionality you would like to see. If you
have further thoughts, feel free to send them either to the list or
to Robin personally. Now is also the time to file your favorite Bro
quirk with our tracker at http://tracker.icir.org/bro ...

Thanks to everybody who helped make this happen!

The Bro Team

[*] Yes, that includes documentation!

The funded project aims specifically at addressing much of the
feedback that we have received from Bro users over the years.

Following up on my earlier mail, I'm also happy to announce that
Seth Hall has joined us over here at ICSI to work with us on moving
Bro forward. Many of you folks already know Seth very well, as he
has been contributing to Bro and the Bro community for a long time.
It's great to have him on the core team now. Welcome, Seth!

Robin

I wanted to point out too, that I'll try and frequently be in the #bro-ids channel on the Freenode IRC network in case anyone wants to talk or has questions.

Thanks! I'm looking forward to making Bro much more prevalent in the network monitoring world. :slight_smile:

  Seth

hi Robin

my intern has finished his project, and has returned to his university. We
are still wrapping up the documentation. I'll see him next week at OSDI,
so he and I can discuss this in person. If you, Vern or any of the
developers happen to be at OSDI, we could have an informal discussion
then. We can also plan for a more formal presentation a bit later.

thanks
Martin

Sounds good, thanks, Martin. I don't think anybody from us will be
at OSDI unfortunately but we're looking forward to hear more
(doesn't need to be formal in any way, just something like an email
summary will already be helpful).

Robin

hi Robin

the technical report is now available at:
http://www.hpl.hp.com/techreports/2010/HPL-2010-164.html

The work has more focus on Apache than Bro, primarily because I couldn't get Sergey access to Bro on a production network. However, he did integrate DataSeries with Bro and ran some tests. I think his work does show that DataSeries has clear benefits for log collection and analysis with these types of applications.

There is at least one thing we would do differently if we started over again, and that is to use an in-memory buffer for log entries before writing an extent to disk. Sergey used a temporary file because he was concerned about messing up Apache's memory management, and then followed the same approach when he added DataSeries logging to Bro. Obviously for those familiar with the Bro source, this shouldn't be an issue.

if you or anyone else has questions about this, please let me know.

thanks
Martin

Hi Martin! I'm really excited about the prospects of integrating DataSeries into Bro. Are the changes to Bro available anywhere?

For the benefit of anyone that hasn't found it yet, here's the link to where you can download the DataSeries source code:
   http://tesla.hpl.hp.com/opensource/

.Seth

Thanks, Martin!

Robin

hi Seth

the modified source should be available at

http://www.sfu.ca/~sba70/files/dataseries/

if there are any issues, please let me know.

thanks
Martin

Seth Hall wrote: