Still a bro newbie, so I’m looking for some guidance.
I’d like to add a packet post-processor to bro. It’ll be written in C++. Essentially I’d like to see every packet that goes through bro, along with its Connection record (or the equivalent) if there is one. Ideally it would be structured it as a plugin.
It looks like I could do it by triggering off of events like new_packet (although currently that only triggers for IP packets). However, IIUC, that still has bro packaging all the info up into a RecordVal, then I have to decode it in my plugin event handler. That seems like quite a bit of additional overhead on each packet.
It looks like I might also be able to do it as a PktDumper but then I just get the raw packet data and I’d have to go re-parse headers and re-lookup connection info - redoing work that’s already been done.
What I’d really like is to simply get a call at the end of analysis for every packet, where I’d get passed a pointer to the packet data along with a pointer to the existing Connection record (if any). Maybe there’s some obvious way to do this that I’ve overlooked?
Anybody have advice for the best way to go? I’m willing to do work to make this happen, but also would prefer not to fork bro so looking for “right ways”.
TIA,
Jeff
Why do you want to do this ? what is that your packet port-processor going to do that you think bro isn't doing for you?
Maybe there's some obvious way to do this that I've overlooked?
Yes, use scripting layer for your analysis. If you think a particular protocol parsing is deficient, instead of writing a packet post-processor, might as well write your new protocol parser.
Aashish
Why do you want to do this ? what is that your packet port-processor going to do that you think bro isn't doing for you?
First, I need to see every packet (including non-IP packets),
preferably *after* bro has had a chance to analyze them. Second, I'm
intending to interface to another existing system that (among other
things) collects network inventory; it's largely orthogonal to
protocol parsing (although informed by it which is why I would like to
see the Connection data).
Maybe there's some obvious way to do this that I've overlooked?
Yes, use scripting layer for your analysis. If you think a particular protocol parsing is deficient, instead of writing a packet post-processor, might as well write your new protocol parser.
I'm not adding a protocol parser. If/when I add one of those, I will
definitely do it the usual bro way.
I think the performance penalty of trying to do what I need to do in
bro script would be too high (if it were even possible). The bro
documentation warns about the overhead of new_packet, packet_contents
events, which would be the closest analog to what I need. It seems
clear to me that the need to marshal the various data into the form
needed by bro script is the primary source of the overhead and I'd
like to avoid that.
Hi Jeff:
Not claiming to be an expert, so Right Way (tm) is relative. This is probably a terrible idea, but until there's a better one ... 
I believe an analyzer plugin can get you access to the raw packets / connection object on a per-packet basis. There's a video that explains how to write analyzer plugins from a previous bro gathering ... can dig up a link if google isn't helpful.
Offhand, I think my first attempt at doing this would go something like:
* Write the new analyzer to do what I wanted with the packets / connection object
* Possibly build a little plugin that would allow me to register new analyzers through script-land (sounds like something that should already exist, but can't remember offhand ...)
* Write a connection hook that would use the register new analyzers function to register my new analyzer to handle this connection
] Since we want this analyzer to run last, register my analyzer as a child of all other analyzers currently assigned to this connection ...
The challenge, I think, would be interfacing with the external network device from an analyzer. Doing that directly would be an issue ... so, maybe create some kind of ring buffer to interface between:
* the main bro thread running the analyzer, and ...
* ... some kind of external thread that polled the ring buffer populated by the analyzer
The external thread could then shoot gathered information from the local bro instance to the interested network device.
Hope something in there is interesting / useful.
Cheers,
Gilbert Clark
Thanks, Gilbert. That's helpful.
I have been sort of fumbling around that same basic notion. Not sure
though. I'd still need to figure out how to handle the non-IP case;
the IP_Hdr is embedded in the analyzer interface. And analyzers
(except for the special case of ARP) only get called for IP packets,
and they get called via the Connection class (which is only created
for IP).
What does handle mean in this context?
Cheers,
Gilbert
What does handle mean in this context?
A primary goal is just to identify the endpoints represented by the
various layers in the packet: mac addresses, vlan tag, layer3 proto,
IP addresses, IP proto, TCP/UDP ports, etc.
Currently, Bro decodes most of that but then basically discards
anything that isn't IP. (Not intending to criticize: it's perfectly
sensible considering bro's design goals. Anything that isn't IP is
non-routable and hence doesn't typically represent a threat that an
IDS system cares about. But for my purposes I still want to see those
packets.)
Ohhhh, now this whole thread makes sense. There has been some discussion internally and on the bro-dev list lately about how to expose that information to scripts in a way that doesn’t overload Bro. Unfortunately there isn’t a timeline yet on actually implementing what has been discussed.
.Seth
It does sound like an analyzer might not completely work after all...
One option here might be to duplicate / filter the stuff an analyzer-based solution won't support (e.g. via either a custom packet source or something like [1]) and forward it to a one-off application written to parse / gather information from that specific type of traffic. The advantage I see there is that it might let bro do most of the heavy lifting for the complex stuff, and might also yield faster per-packet performance numbers for the other stuff (since the one-off doubles as a fast-path processor for those specific types of traffic). Once bro adds support for the stuff that's interesting, then look at taking the one-off code and making it part of the bro plugin proper.
Just a thought,
Gilbert
[1] https://github.com/bro/packet-bricks