Signature Matching Performance

Zeek
1

Hi,
currently I'm dealing with quite a lot (enough to significantly impact the runtime performance) of signatures in my Bro setup. I understand that signature matching isn't part of Bro's main focus; after reading a response of Robin to the mailing list from 2010 (http://mailman.icsi.berkeley.edu/pipermail/bro/2010-October/004621.html) made me abandon this illusion :wink:
However, I wonder if there is a way to speed up things a little? Some points that come to my mind are:
- obviously reduce the number of signatures
- properly anchor the signatures rather than prefixing them with ".*" This seems to be the critical point in my situation. So if you have ideas how to resolve this without giving up matching at arbritrary positions.... :wink:
- clusters of Bro instances
- ...

Thanks, Chris.

2

- properly anchor the signatures rather than prefixing them with ".*" This seems to be the critical point in my situation. So if you have ideas how to resolve this without giving up matching at arbritrary positions.... :wink:

Could you give us some example signatures? If they have private data in them, you could defang them a little bit, I'm only asking so that we can see more about how you are using signatures. In general though, lots of signatures with .* at the beginning are going to be really, really bad.

- clusters of Bro instances

That's always an option, but it may be more worthwhile to find out if you are using signatures for an appropriate task first. :slight_smile:

  .Seth

3

Hi,
pretty much any signature is prefixed with .* followed by a potentially short body of actual signature data. The reason for this is that those signatures are automatically generated and do not have much information about their location within the payload. Hence, it might happen that e.g. a signature for a http request type might end up as /.*GET\ / although that obviously isn't what one would usually go for.
Seems like I'm misusing the concept of the signature engine a bit hehe

Chris.

4

Are you trying to auto convert snort signatures?

  .Seth

5

No, I'm just trying to generate some signatures on my own. With a lot of leading .* hehe
Well I guess I have to rethink the idea a bit :wink:

Thanks, Chris.

6

>
>> Hence, it might happen that e.g. a signature for a http request type might end up as /.*GET\ / although that obviously isn't what one would usually go for.
>
> Are you trying to auto convert snort signatures?

No, I'm just trying to generate some signatures on my own. With a lot of leading .* hehe
Well I guess I have to rethink the idea a bit :wink:

If you have a large list of urls what you want to do is generate a set
of those urls ...

    redef bad_urls += {
        "http://bad.example.com",
        "http://evil.example.com",
        ...
    }

then in a policy somewhere you can simply do

    if(url in bad_urls)
        ...

If you still need a regular expression then you can build up a single pattern
like this:

    redef bad_urls =
          /bad\.example\.com\/some_regex_here/
        > /evil\.example\.com/;

and then use it like this

    if(bad_urls in url)
        ...

both methods will be a huge improvement over building multiple
signatures.