A lower level interface

Hi All,

I’ve seen many discussions referring to the Bro as an alternative of libnids. I wonder that can we use the similar lower-level interface similar to libnids in Bro (e.g., for the tcp assembly)? We would like to explore the string features of packets, while keeping to leverage Bro’s high-level events.

Regards,
Shuai

Hi,

I've seen many discussions referring to the Bro as an alternative of
libnids.

Out of curiosity - where did you see discussions like that?

I do not really know much about libnids, but from the readme it seems that
libnids is a library that mostly implements TCP reassembly. While this is
a part of what Bro does, it only is a small part of it; obviously the
main focus of Bro is on a different layer.

I wonder that can we use the similar lower-level interface similar
to libnids in Bro (e.g., for the tcp assembly)?

I am not quite sure what you are looking for here, could you perhaps
expand a bit on that?

The lowest level access that you can get is probably by writing a custom
(c++) analyzer that gets passed either the reassembled TCP payloads. (Or
just the raw packets in case of UDP).

We would like to explore the string features of packets, while keeping
to leverage Bro's high-level events.

You can use something like the new_packet events in Bro to get access to
individual packet information. However, there is a performance penalty
associated with this (script-level events are fairly expensive and usually
there are a few per connection, not a few per packet).

For anything carrying a significant amount of traffic that approach
probably is not viable.

It depends a bit on what you want to do - preprocessing in C++ and then
bubbling up more high-level events should be a more realisitic choice.

I hope this helps,
Johanna

Thanks for your reply, Johanna.

Basically I am looking for an interface by which we can examine and extract the features of byte stream (or strings) from the traffic (TCP payload), and then we will feed the stream to our analyzer (e.g., via BinPac). Currently I am looking at the tcp_contents; I think it might be sufficient so I don’t have to use tcp_packet or new_packet.

Although there is also a performance note for tcp_contents in the manual, I assume that it would be better than using tcp_packet or new_packet (is this true? I do care about the performance).

Thank you very much!

Cheers,

Shuai

Basically I am looking for an interface by which we can examine and extract
the features of byte stream (or strings) from the traffic (TCP payload),
and then we will feed the stream to our analyzer (e.g., via BinPac).
Currently I am looking at the tcp_contents; I think it might be sufficient
so I don't have to use tcp_packet or new_packet.

I still don't quite get what you are planning to do here. Do you plan to
do some kind of signature to figure out that something is a specific
protocol (so match certain byte sequences)? Or do you really want to do
something more complex that needs scripting?

tcp_contents is probably less expensive than the other named choices, but
it probably still is pretty heavyweight.

Johanna

The thing we are planing to do is similar to the [protocol reverse engineering + traffic pattern recognition]; so we consider that we may need the lower level interface to inspect the byte stream since the patterns that we want to identify (e.g., a serious of connection activities) would involve various protocols.

We do have the signature part to accomplish the payload matching. But it may be not sufficient when we consider the traffic recognition (e.g., generating a signature that involve various protocols and network components).

Thanks for your relies. It do helps much.

Hi again,

The thing we are planing to do is similar to the [protocol reverse
engineering + traffic pattern recognition]; so we consider that we may need
the lower level interface to inspect the byte stream since the patterns
that we want to identify (e.g., a serious of connection activities) would
involve various protocols.

It sadly is really a bit hard to tell what exactly the best starting point
is without knowing the exact problem. You mention connection activities -
does that mean activities inside the same connection or activities within
different connetions?

If it is the latter - you could potentially use signatures to identify
"interesting" connections and use Bro script level events to tie
cross-connection information together.

If signatures for some reason are not enough, it depends a bit on your
traffic. If you only want this to run in rather low-traffic environments,
it might be ok to use the low-level events like tcp_payload. If not - your
only choice quickly becomes to write a C++ analyzer, which then once again
can raise events. You even can write several analyzers, one which only
tries to deduce if a connection is interesting, which then in turn can
forward data to more specific analyzers if interesting data is found.

I hope that helps; I am sorry that I am not more specific, but given that
I still don't 100% understand what you are trying to do this is the best I
can do.

Johanna

It sadly is really a bit hard to tell what exactly the best starting point
is without knowing the exact problem. You mention connection activities -
does that mean activities inside the same connection or activities within
different connetions?

If it is the latter - you could potentially use signatures to identify
"interesting" connections and use Bro script level events to tie
cross-connection information together.

Johanna, thanks for your patient and detailed replies.

I think it is the latter. For example, some external connections (let's say
a HTTP Get/Post to server A, with a binary string with signature S1) will
raise some activities of local network components B and C, where the
traffic is associated with a signature S2. Then we would like to *learn*
such a pattern (HTTP_Get/Post_A, S1, B, C, S2) as a pattern signature. So
we consider "tokenize" the byte stream to extract and cluster the strings
from raw payload.