all,
I will apologize up front for my lack of knowledge in this subject but after 3 weekends of 8 to 12 hours searching I have officially hit the end of the road so I am reaching out to the community hoping you all might have some answers. What I’m trying to do is simple in context I just don’t know the language good enough to do it here is the logic.
if (http connection established and method is post)
check to see have we visited this site before (compare against master list (or table))
if visited this site before
------ignore connection
if site is newly visited
------add site to list or table, and alert
really simple in logic but for the life of me I cannot figure out how to add to a list or table after comparing to that table. Hopefully I explained this well enough, but if I didn’t please let me know and I will try my best to explain it better.
thanks,
Brian,
Brian,
Taking a guess, your difficulty might be stemming from trying to use a table type when instead you may want to use a set type. In the Bro language, a table always has an index and yield, but a set is just a list … if you only want a list of sites users are visiting (no indexes or yields, only a list), then the set type is what you should use. Here’s a simple example (hopefully the script formatting comes through correctly):
global websites: set[string];
event http_header(c: connection, is_orig: bool, name: string, value: string)
{
if ( name == “HOST” )
if ( value !in websites )
add websites[value];
}
event bro_done()
{
print websites;
}
Keep in mind that in production, the websites set is likely to grow enormously and could cause memory issues. For local testing with pcap, the above script will print the seen host values to stdout when Bro finishes. Alternatively, if you want a table of the originating hosts by the websites they connected to …
global websites: table[string] of set[addr];
event http_header(c: connection, is_orig: bool, name: string, value: string)
{
if ( name == “HOST” )
{
if ( value !in websites )
websites[value] = set();
if ( value in websites )
{
add websites[value][c$id$orig_h];
print "found a new website! "+ value;
^^^ could raise notice here instead
}
}
}
event bro_done()
{
print websites;
}
Tables also have nice key/value expire functionality that can help curb their size.
-AK
Yep, there are quite a few attributes that can make them manageable … lots of good information on this page: http://www.bro.org/sphinx/scripts/builtins.html
Looks like I made a mistake in the script I shared … if you want to generate a notice when a new website is found, then the notice should go under the if statement where the value is added to the set/table websites. If you have a master list (set) of websites you’d like to manage separately from a dynamic set/table, then you can add an if statement to check if the value is or isn’t in that list as well …
const master_list: set[string] = { “google.com”, “evil.com” };
{ … }
if ( name == “HOST” )
if ( value in master_list )
#do something
All,
Thanks for the insight, what I am looking at doing is setting a notice log every time we Post to a website when there is no referrer. I believe trimming down to just see this type of info would be less of a load then if we looked at every post to a site we had never visited before. Would there be a large performance impact if I decided to use the input framework with this idea? It seems like that is the logical choice for this project, just not sure on how to edit lists within the input framework on the fly.
Yep, there are quite a few attributes that can make them manageable … lots of good information on this page: http://www.bro.org/sphinx/scripts/builtins.html
Looks like I made a mistake in the script I shared … if you want to generate a notice when a new website is found, then the notice should go under the if statement where the value is added to the set/table websites. If you have a master list (set) of websites you’d like to manage separately from a dynamic set/table, then you can add an if statement to check if the value is or isn’t in that list as well …
const master_list: set[string] = { “google.com”, “evil.com” };
{ … }
if ( name == “HOST” )
if ( value in master_list )
#do something