Python script/module to parse new log files


has anybody already written a python script or module to easily parse the new Bro ASCII log files?


I've got some code for this, but it's a work in progress; you're welcome to take a look at what's there, though, and do whatever you'd like with it.

*** Note that this code demands a slightly modified ASCII header (which includes the bro path name and the separator char).

Anyway, branch is:


and check: aux/log-util

I think the usage is pretty straightforward:

manager = BroLogManager()
#If this is a directory, *all files from all subdirectories* of this path will be loaded.
# Otherwise, only the referenced files will be loaded.
# .log, .log.gz, and .log.bz2 are treated as new-style bro logs

print manager['conn'].get_stats('orig_bytes')
# This prints some interesting statistics about the orig_bytes column (min / max / mean / std_dev).
# Note that the above will actually load *all* relevant log files and perform the calculation, then cache
# the results; the first get_stats() will be very slow, but the rest should work pretty quickly.

for e in manager['conn'].entries():
     print e['ts'] # Note that 'e.ts' will also work, but that this doesn't work for field names that don't map nicely to Python.

See 'bro-logtool' for a toy script I've been using to play with the library.