I wonder that does anyone have experience to tackle the “unknown protocol” when DPD cannot recognize the protocol and/or all existing analyzers fail.
At this time we assume that all protocols are tcp-based. According to one of previous discussions, http://mailman.icsi.berkeley.edu/pipermail/bro/2014-July/007222.html
we first attempt to create a signature which matches everything. Such signature will eventually capture ALL connections even when there is available analyzer can process the stream (e.g., HTTP). However, we want the analyzer for unknown protocol only be triggered when no existing analyzer can be used.
(1) One possible way we are considering is that if there is a mechanism can control the process of analyzers. For example, when one of analyzers is successful, it sends a signal to the Unknown-Protocol-Analyzer to terminate it.
(2) Another way is that we set a global variable which captures and indicates the failed/successful analyzers after the DPD; then if all analyzers fail, the Unknown-Protocol-Analyzer is triggered.
In addition, according to this message http://mailman.icsi.berkeley.edu/pipermail/bro/2007-December/002593.html,
Robin mentioned the method DPM::BuildInitialAnalyzerTree() in DPM.{h, cc} (Manage::BuildInitialAnalyzerTree() in current distribution). With the source code, https://github.com/bro/bro/blob/master/src/analyzer/Manager.cc
it seems that we can initiate an analyzer here when seeing a connection which is non-TCP, non-UDP, and non-ICMP. However, if we assume all TCP-based protocols, where we should look at if we have to touch the source code?
We haven’t investigated the PIA implementation; is this part related and worth to explore?
As such, anyone have ideas how to deal with such a case? Thanks for any comments~
The first thing that comes to my mind would still be trying to
unconditionally add your analyzer in
analyzer::Manager::BuildInitialAnalyzerTree. If it's just TCP that
you need, that's fine, see the other tcp->AddChildAnalyzer() calls
there for ideas.
Then, the other part of your problem would be disabling that analyzer
when any other protocol analyzer is confirmed. An idea would be to
periodically (e.g. every DeliverPacket) walk the analyzer tree (e.g.
analyzer::Parent() and analyzer::GetChildren()) and check whether
analyzer::ProtocolConfirmed() is true for anything. You could also
maybe just handle this in scripts via Analyzer::protocol_confirmation
event and Analyzer::disable_analyzer() functions.
The PIA implementation is related to protocol matching, but not sure
whether you'd need to modify anything there to get what you want.
Thanks, Jan and Jon! I noticed that the “Analyzers of Last Resort” is very useful for our case and PacketSled would like to share this part with community. I cannot find the email addresses of speakers from PacketSled, anyone can help?
Also, following Jon’s suggestion with the solution of script level, we write a sample code as follows for testing. We here assume that anytime we capture a protocol_confirmation from any analyzer, there is an available analyzer responsible for the stream so we disable the Unknown_Protocol analyzer which matches anything.
event Unknown_event(c: connection) {
// unknown protocol process
}
If we test with uncommon protocol trace and there is no corresponding protocol analyzer, the Unknown_Protocol Analyzer successfully captures the stream. However, this Analyzer::disable_analyzer() doesn’t work here. With normal protocol traces, we still see the analyzer is processing the stream and produce the logs.
Any ideas how this Analyzer::disable_analyzer() should be used in such scenario?
In addition, the Log::disable_stream() works here if we only terminate the log stream for the Unknown Protocol analyzer. However, we essentially would like to disable the process of analysis instead of only closing the log stream.
event Unknown_event(c: connection) {
// unknown protocol process
}
If we test with uncommon protocol trace and there is no corresponding
protocol analyzer, the Unknown_Protocol Analyzer successfully captures the
stream. However, this Analyzer::disable_analyzer() doesn't work here. With
normal protocol traces, we still see the analyzer is processing the stream
and produce the logs.
Yeah, Analyzer::disable_analyzer() doesn't seem like what you want (it
should disable the analyzer, but the not ones associated w/ current
connections, just future ones).
Any ideas how this Analyzer::disable_analyzer() should be used in such
scenario?
You might actually need to write your own function (unless I
misremember, it looks like Bro doesn't actually have a generic way to
do that from scripts), though it shouldn't be hard. Basically I would
try turning your disable_unknown() function into a BIF (there's a
bunch of *.bif files you can use as reference). E.g. an idea on how
the BIF code would look:
function disable_unknown%(c: connection%) : bool
%{
auto ua = c->GetRootAnalyzer()->FindChild(ANALYZER_UNKNOWN);
if ( ! ua )
return new Val(false, TYPE_BOOL);
ua->Parent()->RemoveChildAnalyzer(ua);
return new Val(true, TYPE_BOOL);
%}
The RemoveChildAnalyzer() call then marks the analyzer as "removed"
and shouldn't receive any further stream data.