Seth, what do you think of this approach? If loaded (which I'd like to
do by default), it will replace the standard raw notice alarms with
pretty printed versions.
Note, it's not finished yet; in particular, the actual pretty printing
is missing. But is this a good way to hook into the processing?
Robin
Yep. Handling the Notice::notice event is definitely the right way to hook in.
As I'm reading through it I'm having a few thoughts. Maybe we should just make an extensions/ directory in the notice framework instead of the current extend-email/ and actions/ directories? They're all basically just extensions they're just using different extension mechanisms. I wouldn't say this exactly fits into actions and I didn't abstract ACTION_ALARM into alarms (which possibly should have) and then you could have just directly implemented this there.
And I'm going to back up on what I said before. I can see this script taking a shell script or something to process the log file through as an alternate way of converting the log into the email body. That might even make more sense as the primary approach. I guess it would be like a prerotation filter or something? We may be able to use something like that in more ways than we are even thinking of right now. That way someone could provide a script that can turn Bro logs into webpages (as a sort of obvious example) if people want to receive their alarm emails in a "pretty" format. Or, it could even attach it to the email as an attachment if people prefer that way.
I'm pretty sure you were right on that one, I just hadn't fully thought through it yet. 
.Seth
As I'm reading through it I'm having a few thoughts. Maybe we should
just make an extensions/ directory in the notice framework instead of
the current extend-email/ and actions/ directories?
Yeah, sounds good, I was wonderng where to put this.
And I'm going to back up on what I said before. I can see this script
taking a shell script or something to process the log file through as
an alternate way of converting the log into the email body.
That's how I wanted to start but then one needs to parse the log in
shell. Thinking about it more, I agreed with what you said earlier and
did it in pure Bro, which is easier (or will be once I actually add
the code for the formatting).
That might even make more sense as the primary approach. I guess it
would be like a prerotation filter or something?
Maybe, but not sure. That may be better to postpone until we have an
better idea how it might look like.
Ok, I'll finish this up then. Would like to get that into the beta as
it would be step backwards from 1.5 otherwise. Are you ok turning it
on by default?
One more thing that bothers me a bit: by default, there are no
alarms... Everything just goes into notice.log. To get any mails, one
needs to configure things appropiately, which is hard at the beginning
because one won't have a good idea what can be generated.
I'm sure you have thought about this: what's the reason for not using
ACTION_ALARM as the default action, now that there aren't actually
that many notices generated anymore?
Robin
Ok, I'll finish this up then. Would like to get that into the beta as
it would be step backwards from 1.5 otherwise. Are you ok turning it
on by default?
Sure, it seems reasonable to me.
One more thing that bothers me a bit: by default, there are no
alarms... Everything just goes into notice.log. To get any mails, one
needs to configure things appropiately, which is hard at the beginning
because one won't have a good idea what can be generated.
Eventually (maybe before the final?) I'd like to have a page in the auto-generated docs with all of the Notice::Type values and the associated documentation so users could browse that single location and see what notices they have available for emailing, alarming, etc along with the script that needs to be loaded in order for that notice to be raised.
I'm sure you have thought about this: what's the reason for not using
ACTION_ALARM as the default action, now that there aren't actually
that many notices generated anymore?
In my mind that violates the "Bro is policy neutral" rule (or whatever I've seen in slides). Bro generates the notices and it's a site local decision if those notices are applicable to their environment. Making ACTION_ALARM the default would essentially be saying that all notices are applicable to their environment. It always sort of bothered me that anything was turned into an alarm out of the box after hearing about that. That said, I think that shipping with some notices added to the Notice::emailed_types and Notice::alarmed_type variables would in local.bro would be cool since I see local.bro as our chance to give a suggested configuration.
.Seth
Eventually (maybe before the final?) I'd like to have a page in the
auto-generated docs with all of the Notice::Type values and the
That would be nice. Can Broxygen already extract that information?
> I'm sure you have thought about this: what's the reason for not using
> ACTION_ALARM as the default action, now that there aren't actually
> that many notices generated anymore?
In my mind that violates the "Bro is policy neutral" rule (or whatever
I've seen in slides).
That argument usually applies to the event engine. For scripts,
considering policy is hard to avoid (and as you say, by default, Bro
1.x did turn everything into an alarm).
That said, I think that shipping with some notices added to the
Notice::emailed_types and Notice::alarmed_type variables would in
local.bro would be cool since I see local.bro as our chance to give a
suggested configuration.
Or we could provide an option "default_alarm" or so that enable
ACTION_ALARM as the default, and then put that option into local.bro
for everybody to tune. The question would then be what to default to.
I'm leaning towards alarming by default, in the spirit of showing off
what Bro can do (and showing that *is* doing something).
Robin
Eventually (maybe before the final?) I'd like to have a page in the
auto-generated docs with all of the Notice::Type values and theThat would be nice. Can Broxygen already extract that information?
Yeah, it already has somewhat, but finding it in the sphinx-generated docs might not be so obvious right now.
All that info can be found by looking in the index for everything under "Notice::Type (enum values)",
and then the link leads to the documentation for it and the bro script in which it's declared.
- Jon
That argument usually applies to the event engine. For scripts,
considering policy is hard to avoid (and as you say, by default, Bro
1.x did turn everything into an alarm).
I don't see any reason to not apply it all the way up to the scripts. 
I think it just comes down to the correct mix of documentation and shortcuts.
Or we could provide an option "default_alarm" or so that enable
ACTION_ALARM as the default, and then put that option into local.bro
for everybody to tune. The question would then be what to default to.
I'm leaning towards alarming by default, in the spirit of showing off
what Bro can do (and showing that *is* doing something).
Yuck, I *really* don't want to have that option. We could actually just implement that as an item in Notice::policy anyway. It's a big mental change to adjust initially to multiple actions being applied to a notice. You think that people would want to start receiving all of their notices in email prior to getting a chance to look through the notices to see what they want?
Personally, I can see adding the following to local.bro to cover the example you are trying to accomplish...
redef Notice::alarmed_types += {
HTTP::SQL_Injection_Attack_Against,
HTTP::SQL_Injection_Attacker,
HTTP::Malware_Hash_Registry_Match,
HTTP::Incorrect_File_Type,
FTP::Site_Exec_Success,
SSH::Interesting_Hostname_Login,
SSH::Password_Guessing,
SSH::Watched_Country_Login,
Software::Vulnerable_Version,
};
Well crap, I guess that actually includes many of the notices that we're shipping right now (there are some higher volume ones filtered out still). Maybe this?
# Uncomment the following line to begin receiving hourly emails containing all of your notices.
#redef Notice::policy += { [$action = Notice::ACTION_ALARM, $priority = 0] };
.Seth
Way too much work. 
Would it be a lot of effort to pull all of that into a single page? (i know i already asked you this, but i don't remember exactly what you said)
.Seth
Way too much work.
Would it be a lot of effort to pull all of that into a single page? (i know i already asked you this, but i don't remember exactly what you said)
Probably not difficult, the effort is just in looking into the best way to do that (which I'm not sure of right now, i.e. dunno if I want to do it inside Sphinx or inside Bro).
- Jon
You think that people would want to start receiving all of their
notices in email prior to getting a chance to look through the notices
to see what they want?
Yes, I do actually. It's the push vs. pull model; seeing what I want
and adjusting the config takes effort. Also, at least for smaller
networks, it's actually not a problem; and I'm hoping we'll be getting
plenty of those in the future as well.
# Uncomment the following line to begin receiving hourly emails containing all of your notices.
#redef Notice::policy += { [$action = Notice::ACTION_ALARM, $priority = 0] };
I like that! It's short and even better than the option I suggested
because one can start tweaking right there. And I'm fine leaving it
commented out. I'll add that.
Robin