Hi all:
I'm in the process of processing our syslogs for apache logs (which
will allow us visibility into ssl sessions into our webservers), and
am at the point where I am able to import the data into bro using the
input framework. There's enough data to fill in most of a connection
record, and to fake other stuff. What would be really cool would be
to create a connection record, and have it go thru the normal
processing, feed the http data in for processing via the standard http
processes, and close down the connection. This would allow for
standard logging, and standard IDS processes to act upon this info.
Does anyone have suggestions on how to proceed with this?
Thanks in advance,
Jim Mellander
NERSC Cybersecurity
510-486-7204
It wouldn't work very well. 
Nearly all of the detections rely on the various http_ events. I would go down a slightly different route with logs than I would with raw traffic. This is something that I've been talking about for quite a while and I suspect something related to happen in the next year.
I think it's really cool that you're importing logs into Bro!
.Seth
Well, its unfortunate that we can't feed in data from other sources
and subject it to the same policies that network traffic is subject
to.
In the meantime, I may just write some code that fakes the data into
pcap files that can be read by bro directly.
Reviving my old thread - this project was on hiatus for a while, but
is now generating useful data, although not yet in production.
What I ended up doing was transport the syslogs of interest (apache
logs) via stunnel to my bro box, which then runs a custom python
script:
1. Parses and extracts the logs into python variables
2. Construct plausible http session conversation.
3. Packetize the session into TCP packets
4. Push tcp packets onto virtual interface
5. bro listening to virtual interface performs normal IDS monitoring of traffic.
Here's an example:
Syslog entry:
Jul 3 09:34:54 128.55.22.194 httpd[4148]: www.nersc.gov 66.249.73.109
- - [03/Jul/2013:09:34:54 -0700] "GET /robots.txt HTTP/1.1" 200 82 "-"
"Mozilla/5.0 (compatible; Googlebot/2.1;
+http://www.google.com/bot.html)"
Bro http logs:
Jul 3 09:34:58 u3qPWFy8m9 66.249.73.109 64555 128.55.22.194 80 GET www.nersc.gov /robots.txt Mozilla/5.0
(compatible; Googlebot/2.1;
+http://www.google.com/bot.html) 0 0 200 <empty>(empty) - - - - - -
A few points:
1.Tested creating packets to make the conn logs show the correct
amount of data returned by the server, as reported in the log, but
eventually chose to not do that, as that is of limited value.
2. Support both ipv4 & ipv6 - if a host is dual homed, typically the
syslog entry will be from the ipv4 address, but the requesting ip may
be ipv6, in which case we convert both address to ipv6 using several
strategies.
The value of this, from our perspective, is that we can now perform
the usual http IDS functions on https connections to our syslogging
webservers, without having to store the certs in our bro system for
decryption. We also have visibility on intrasite traffic to those
same webservers.
Alpha quality code available for the asking.
Jim