Hi Seth,
I don’t have quick internet access, only a *dsl access.
This is why I need feedback please.
Anyone tested please?
What’s performance impact? (only 33sigs)
Regards
Rmkml
Seth Hall
There are a number of potential and definite problems.
- For each http_request event, you are doing a lot of if & if else statements which *could* impact performance.
- For each http header you are similarly doing a lot of if statements which will almost certainly cause a performance impact. Also, you are accessing collected state in the c$http record when you should probably be using the name and value variables directly. If you want to look through data before things are logged, your best bet is to use the HTTP::log_http logging framework event.
- Again, lots of if statements for every dns request is probably going to have a severe performance impact.
- For every single chunk of http entity data, you are running lots of if statements with pattern conditions again.
- Handling the packet_contents event at all is generally really bad. The auto-generated documentation even comments on the fact that using that event is not really feasible for any traffic volume:
http://www.bro-ids.org/documentation/scripts/base/event.bif.html?highlight=packet_contents#id-packet_contents
This is one of the interesting things about Bro. Due to it primarily being a programming language, you can absolutely do things that will negatively impact performance and break other analysis. So like any other language you have to constantly be aware of what you are doing and the potential impacts. We are actively working now to make it possible for you and others to do these detections more easily and with less potential performance impact. Unfortunately we're still at the very beginning of a newly-found operational security engineering focus so this stuff is taking a bit longer than most people would like (me included!).
.Seth
Oh! I forgot to include an alternate approach I thought of. If you are still interested in going down this route, could you start by pulling out malicious software user-agents from the ET signatures? That's something that would fit well and easily into Bro right now and into the intelligence framework in the future.
What do you think about that? We can certainly start small with very well defined goals and move from there.
.Seth
Hi:
You might try obtaining a few rather large traces and running bro against those traces with '-r'. Record how long it takes to process these traces both without the changes you've made and with the changes you've made. The difference in these two times might give you a rough idea of how your modifications impact bro's performance (given observed traffic similar to that of the analyzed trace).
--Gilbert
Thx you for reply Seth,
ok I have started very small bench on my local network: (wget, one cnx)
-without et_bro2_11aug, download http at --limit-rate=85m, bro cpu around (top) 40%-45%
-all sigs et_bro2_11aug, download http at --limit-rate=85m, bro cpu around (top) 75%-90%
-disabled only "packet_contents" on et_bro2_11aug, download http at --limit-rate=85m, bro cpu around (top) 75%-90%
-disabled only "entity_data" on et_bro2_11aug, download http at --limit-rate=85m, bro cpu around (top) 75%-90%
-disabled only "dns_request" on et_bro2_11aug, download http at --limit-rate=85m, bro cpu around (top) 75%-90%
-disabled only "http_header" on et_bro2_11aug, download http at --limit-rate=85m, bro cpu around (top) 75%-90%
-disabled only "http_request" on et_bro2_11aug, download http at --limit-rate=85m, bro cpu around (top) 75%-90%
well, no special sig penalty.
I have discovered one pb on my case: in ids mode, bro not fire immediatly, after 5mn not fire, fire only when I kill bro, it's possible to fire immediatly on my rule set please?
Best Regards
Rmkml
You need to do this with a decent sized tracefile (>1GB) of mixed traffic and run Bro with the "time" command to see how long it takes for it to analyze the full file. I suspect the performance degradation will become much more obvious there.
.Seth
ok Im look on user-agent ET sigs.
Regards
Rmkml
starting hard works...
question please: it's possible to detect POST and uri (/abc) and argument (arg=test) ?
example:
POST /abc HTTP/1.0
...
\r\n
arg=test
not work but like:
("POST"==c$http$method)&&(/\/abc/ in c$http$uri)&&(/arg\=test/ in c$http$body???)
Regards
Rmkml
Hi,
ok I have advance my performance penalty, simply disable "packet_contents" and "entity_data", results my performance go to 40%-45%...
(download one file size 1.9Go with wget multiple times)
good news.
Anyone tested partial ET open-gpl on live trafic please ?
Regards
Rmkml