Having some unidentifiable issues working with the intelligence and Sumstats frameworks that I hope the development team or community can comment on. I’ve written multiple Sumstats scripts that return results, but I have an example where it doesn’t when running on a cluster. Locally, the following snip of script runs and returns the expected results:
@load base/frameworks/sumstats
@load base/frameworks/notice
@load base/frameworks/intel
module Intel;
export {
redef enum Notice::Type += {
Test_Indicators
};
}
event log_intel(rec: Info)
{
SumStats::observe(“intel.indicators”, [$str=cat(rec$uid,"",rec$id$orig_h,"",rec$id$resp_h)], [$str=rec$seen$indicator]);
}
event bro_init()
{
local r1: SumStats::Reducer = [$stream=“intel.indicators”, $apply=set(SumStats::UNIQUE)];
SumStats::create([$name=“test-intel”,
$epoch=2mins,
$reducers=set(r1),
$epoch_result(ts: time, key: SumStats::Key, result: SumStats::Result) =
{
NOTICE([$note=Test_Indicators,
#$src=to_addr(parts[3]),
#$dst=to_addr(parts[5]),
$msg=“passed sumstats”,
#$sub=sub_msg,
$identifier=key$str]);
}]);
}
Each time a log is sent to intel.log, this observes the indicators seen in unique connections. For testing purposes, I have it writing a notice whenever the Sumstats event finishes. While this works as expected locally, when run in production (on a cluster), no notices are written-- that suggests to me that the data is not being sent to Sumstats. (And I am generating lines in intel.log in prod, so it cannot be the lack of intel.log activity.)
The only identifiable difference between my local version of Bro and the one running in production is clustering. Is this expected behavior? I have used log_ events successfully with Sumstats in the past, so I can’t think of what is preventing this notice from firing. I also verified the syntax of the above script by changing the event from log_intel to something more common (http_reply) and it worked both locally and in prod; changing log_intel to Intel::match (mimicking intel/do_notice.bro) worked locally but not in prod.
Interested in reading thoughts on this…
Thanks for reading,
Josh Liburdi