Tuning Bro

Hi,

I am a grad student trying to get acquainted with Bro. I have tried using it on a few datasets available (including the old DARPA ones). I am able to get logs and notices and weirds, but I have doubts about Bro configuration:

  1. How do the IP ranges specified in Site::local_nets and networks.cfg affect Bro’s monitoring? Do they have different use cases, or can they be used interchangeably? Or do they have nothing to do with each other? (As of now, my Site::local_nets and networks.cfg are identical.)

  2. From my layman’s understanding, given a PCAP, the larger the window of time and the number of packets that a system looks at the more accurate its detection could be, the tradeoff being that of memory/performance. (I guess there won’t be any packet dropping when reading PCAPs.) Is that true?

  3. For someone who doesn’t know much about the Bro language, are there any generic configuration settings or tunables that might improve detection rates? Like the maximum size up to which a packet is read, or the number of packets that Bro simultaneously analyzes. (Snort has some parameters along these lines.)

  4. How does Bro handle packet defragmentation and stream reassembly? Is there documentation for the internals, about the various components and analyzers and how they analyze traffic? I am looking for a basic understanding.

Thank you.

Regards,
Sravan

1. How do the IP ranges specified in Site::local_nets and networks.cfg affect Bro's monitoring? Do they have different use cases, or can they be used interchangeably? Or do they have nothing to do with each other? (As of now, my Site::local_nets and networks.cfg are identical.)

Those are the same thing. If you run with BroControl, it uses networks.cfg as a convenience method for setting Site::local_nets.

2. From my layman's understanding, given a PCAP, the larger the window of time and the number of packets that a system looks at the more accurate its detection could be, the tradeoff being that of memory/performance. (I guess there won't be any packet dropping when reading PCAPs.) Is that true?

There definitely won't be any packet loss when reading from a tracefile. I don't understand the rest of what you are saying, it mostly comes down to a question of if your host's CPU is overwhelmed or not. If it's overwhelmed then you will drop packets on your NIC. If you run out of memory then Bro will crash because it can't allocate any more memory.

3. For someone who doesn't know much about the Bro language, are there any generic configuration settings or tunables that might improve detection rates?

There are a number of settings which could affect Bro's analysis rate, but I'm not intimately familiar with too many. The normal deployment is that people deploy enough hardware to handle their traffic.

Like the maximum size up to which a packet is read

This is not something that is done in Bro because Bro's primary thrust is in protocol analysis. If you analyze partial packets you can't correctly analyze any protocols.

, or the number of packets that Bro simultaneously analyzes. (Snort has some parameters along these lines.)

I don't understand this comment.

4. How does Bro handle packet defragmentation and stream reassembly? Is there documentation for the internals, about the various components and analyzers and how they analyze traffic? I am looking for a basic understanding.

Most of it is documented with various settings here:
  http://bro-ids.org/documentation/scripts/base/init-bare.html

There are a lot of settings in there, but look for things like "tcp" and "defrag".

  .Seth

  1. From my layman’s understanding, given a PCAP, the larger the window of time and the number of packets that a system looks at the more accurate its detection could be, the tradeoff being that of memory/performance. (I guess there won’t be any packet dropping when reading PCAPs.) Is that true?

There definitely won’t be any packet loss when reading from a tracefile. I don’t understand the rest of what you are saying, it mostly comes down to a question of if your host’s CPU is overwhelmed or not. If it’s overwhelmed then you will drop packets on your NIC. If you run out of memory then Bro will crash because it can’t allocate any more memory.

What I mean is the following. Suppose an attack involves sending n packets. Suppose the alarm related to that attack is set to trigger when the IDS sees m packets within a time interval t. (I guess alarms for portscans are defined in such a way.) Then the attacker could send the n packets at a slower rate to avoid detection through the alarm. If t is set to a larger value, then the slower attacks could also be detected. Something along the lines of: greater the history maintained, greater the context and greater the possibility of detecting an attack. Does this make sense?

, or the number of packets that Bro simultaneously analyzes. (Snort has some parameters along these lines.)

I don’t understand this comment.

  1. How does Bro handle packet defragmentation and stream reassembly? Is there documentation for the internals, about the various components and analyzers and how they analyze traffic? I am looking for a basic understanding.

Most of it is documented with various settings here:
http://bro-ids.org/documentation/scripts/base/init-bare.html

There are a lot of settings in there, but look for things like “tcp” and “defrag”.

Thank you very much. This looks like a good start.

Thinking in terms of packets is usually the wrong approach with Bro, but I will take "packets" just to mean any arbitrary event that was seen. Right now, Bro actually doesn't have any form of scan detection we ship with. We have a script for detecting scanning in our contributed scripts repository which had very minimal porting from the 1.5 release of Bro but doesn't have a modern feel to it. Scan detection was removed because it became really difficult when we moved to a clustered architecture because scan detection involves a global state but in a cluster you have lots of processes with partial state.

We are actively working on adding probabilistic data structures to Bro now so that ultimately we will be able to keep longer periods of state without using too much memory. (that's the hope at least, no promises!)

  .Seth

What I mean is the following. Suppose an attack involves sending n packets. Suppose the alarm related to that attack is set to trigger when the IDS sees m packets within a time interval t. (I guess alarms for portscans are defined in such a way.)

Thinking in terms of packets is usually the wrong approach with Bro, but I will take “packets” just to mean any arbitrary event that was seen.

What would be a right approach with Bro? Would it be thinking along the lines of events? I can see that Bro core generates many different events, like those defined in event.bif. Are Bro users expected to write scripts where some actions are taken based on the events seen (say their combinations and counts)?

Right now, Bro actually doesn’t have any form of scan detection we ship with. We have a script for detecting scanning in our contributed scripts repository which had very minimal porting from the 1.5 release of Bro but doesn’t have a modern feel to it. Scan detection was removed because it became really difficult when we moved to a clustered architecture because scan detection involves a global state but in a cluster you have lots of processes with partial state.

I am also using the scan.bro script right now.

As a supplement, the materials from the most recent bro workshop might be a good way to get an overview of how bro works and what the scripting language is like.

Take a look at: http://bro-ids.org/bro-workshop-2011/index.html

Good luck,
Gilbert

Btw, you may have already found these, but for some context on how Bro
does things internally, I suggest reading the original Bro paper [1]
and also a later one on resource management [2]. Both are a bit dated
in their specifics by now, but conceptually they still apply.

Robin

[1] http://www.icir.org/vern/papers/bro-CN99.pdf
[2] http://www.icir.org/robin/papers/ccs04.ps

Thank you, Gilbert. I found some of these materials very useful.

Changing constants defined in init-bare.bro didn’t effect the detection rate on my pcaps much. Because my problem was not with the detection rate itself. It turned out to be that some of the notices (like those generated by scan.bro, detect-bruteforcing.bro) are in a format different from the default – they don’t contain the 4-tuple of source and destination IP addresses and port numbers. So they were being missed by my analysis scripts that compare notices with labels. Modifying those scripts for more verbose notice generation is showing more positive results.

Regards,
Sravan

Thank you, Robin. I had seen the 1st paper before. I will read both of them.