Bro Web Frontend

In order to really drill down on some SSL stuff, I decided it was time
to finally put a lightweight web framework on top of my Bro logs. So,
unless the server shoots it down, attached is a tarball that contains
an alpha-quality framework for reading Bro logs and writing them to
syslog or a database or both. If written to a database, they can be
accessed through a very light web frontend included which can be run
right from the Bro server, or from a traditional web server, if you
prefer. The web frontend solely consists of a query bar for searching
on terms contained in the tab separated Bro logs. So you can do
things like this:

This would look for any connections to Gmail with an invalid certificate.

Or you could do

id.resp_h:74.125.*

Which would find any traffic destined for Google's 74.125/16, or just
74.125.*
which would do the same thing, but for either originator or responder.

At this point, there's not much else you can do, as I mainly wanted to
get a quick web frontend up so I could more easily explore the data
that Bro outputs. The key part of the framework is that it doesn't
care what the fields are. It will take whatever is at the top of the
log files and use that as the field names. If it can't find the field
list, then it won't use the file. The database doesn't have a
standard table structure, rather it uses a key-value store, so the
schema is very flexible. It's been tested on MySQL, but it should
work on almost any database. It would be very easy to create entirely
ad-hoc reporting using this structure.

Aside from cosmetics, the big to-do is database table rollover, but if
you just want syslog output from Bro, this will get you going very
quickly. See the INSTALL doc for a quick how-to, or email if you have
any questions; feedback is appreciated! If there is any interest, I
will create a project page for the files for future downloading.

Thanks,

Martin

broweb.tar.gz (5.69 KB)

Hi Martin:

This is cool!

A few notes:

*) There's a ticket open (http://tracker.bro-ids.org/bro/ticket/558) which is looking at revising the ASCII log header (topic/gilbert/ascii-header is a branch containing what's described in that ticket) to offer more information about the log fields themselves, and to support better classification of log files after they've been rotated / compressed. The idea there is to use that type information to automate the table creation process and / or do some simple type conversion.

*) topic/gilbert/log-util is (what sounds like, at least) a similar log wrapper I'm building in Python. The library's in a holding pattern while the new log header format gets pushed out, but should find itself in reasonable shape after that.

If you're interested, I'd love to migrate over to bro-dev and chat about stuff that's useful to have in this kind of a library :slight_smile:

--Gilbert Clark