Traffic characteristics extraction with Bro

Zeek
1

Hi,
I am trying to extract some flow characteristics from static data with Bro. I've checked the analyzer Conn.bro, but didn't find any suitable information.
At present, the characteristics I need are: mean packet size and mean packet inter-arrival time, all per flow. Future work may require packet related information, also per flow.

Does anybody know how to do this ? Bro's manual doesn't provide much information about static traffic analysis.
Where should I start and what should I do now?

I am also quite new with Bro, so any detail is appreciated.
Thanks in advance,
Duc.

2

I am trying to extract some flow characteristics from static data with
Bro. I've checked the analyzer Conn.bro, but didn't find any suitable
information.

Check out analy.bro, which does this sort of analysis on whatever connections
Bro is processing (so you need to load additional scripts to capture the
packets of interest). You may need to extend it by editing
TCPStats_Endpoint::DataSent in TCP.cc.

At present, the characteristics I need are: mean packet size and mean
packet inter-arrival time, all per flow.

Note, if you just want means, then you can track this quite cheaply.
(And mean inter-arrival time is just duration divided by number of packets.)

    Vern

3

Thank you all for the tips.

For those who are interested, here're some observations:

1> I used "tcp_packet" event to capture packets. To test this event, I summed up all packet sizes from the input argument "len" and "is_orig" for each flow. Finally, I compared the sum values (for both directions) with the values returned by "conn_size", as in the analy.bro
The result is interesting: for ALL connections, the total size from the ORIGINATOR side is an exact match, but for many connections, my values for the RESPONDER side are higher and the discrepancies depend on the total size.
Is there a bug, Vern ?

2> Applying similar method for event "new_packet" (using the field "dl" in tcp_hdr) gived me the discrepancies of ten folds, for each direction. So I guess tcp_packet is more suitable :), although I don't know why.

3> I found that handling packet level events (such as tcp_packet) made Bro run out of memory when analyzing a CRedII trace with lots of scans - even if the handler does nothing. Bro works fine, though, if I don't capture these events.

4> It would be nice if there's an overview explanation about the Bro CC code, for someone who needs to extend or modify the code. Doesn't have to be long, one or 2 pages are fine. Also it would be great if we have a page for people to share useful policy/scripts files. I'd be happy to contribute.

5> I really like the Bro language and is learning a lot from Bro. Thanks for creating such a wonderful tool.

Duc Ha

Vern Paxson wrote:

4

The result is interesting: for ALL connections, the total size from the
ORIGINATOR side is an exact match, but for many connections, my values
for the RESPONDER side are higher and the discrepancies depend on the
total size.
Is there a bug, Vern ?

It's likely that the responder side is more likely to retransmit packets.

The usual way to diagnose something like this is pick the largest "offender",
extract the corresponding packet trace, and inspect it by hand.

3> I found that handling packet level events (such as tcp_packet) made
Bro run out of memory when analyzing a CRedII trace with lots of scans -
even if the handler does nothing. Bro works fine, though, if I don't
capture these events.

That certainly suggests a leak. As we don't use tcp_packet in production
use, we haven't tripped across this before.

4> It would be nice if there's an overview explanation about the Bro CC
code, for someone who needs to extend or modify the code. Doesn't have
to be long, one or 2 pages are fine.

Well, I don't know about "doesn't have to be long" - it is, after all,
more than 100,000 lines of code. In any case, we're aware that this would
be nice to have, but it's in fact a great deal of work, beyond our present
resources (since we're funded primarily for research rather than development).

Also it would be great if we have
a page for people to share useful policy/scripts files.

We have the beginnings of this in

  http://www.bro-ids.org/wiki/index.php/Category:User_Contributions

though at present we do not provide public edit-access to the Wiki, but
rather populate this based on contributions folks send us. Alternatively,
one can use the mailing list for this, until a contribution is developed
and stable enough to merit widespread use.

5> I really like the Bro language and is learning a lot from Bro.
Thanks for creating such a wonderful tool.

Very pleased to hear it!

    Vern