Hi everybody,
I have noticed that in many app level analyzers there is a sort of “safety check” in the method DeliverStream which essentially returns when the tcp session is partial, i.e.
if ( TCP() && TCP()->IsPartial() )
return;
This is true for example for the HTTP, SSH, SSL analyzers and more. My understanding is that this is to prevent app layer analyzers or scripts relying on them from breaking down or missing some information when processing packets with possible missing bytes.
Am I right? How much reliable is this check TCP()->IsPartial() for partial tcp sessions in the tcp analyzer?
Thanks,
Mauro
if ( TCP() && TCP()->IsPartial() )
return;
This is true for example for the HTTP, SSH, SSL analyzers and more. My understanding is that this is to prevent app layer analyzers or scripts relying on them from breaking down or missing some information when processing packets with possible missing bytes.
Mre related to the "breaking down" part: current protocol parsers
don't have any type of "re-synchronization" mechanism so particularly
if we miss the TCP handshake and assume we may be starting in the
middle of the app-layer protocol stream (or else have a content gap),
the parser won't know what to do with the incoming data and so the
IsPartial() checks just exit early, before attempting to parse
further.
How much reliable is this check TCP()->IsPartial() for partial tcp sessions in the tcp analyzer?
Should be reliable in detecting the problematic scenario AFAIK, but in
the case where just the TCP handshake packets are missing and not any
segment data, analyzers that exit early like that are skipping streams
they actually should be able to parse.
- Jon
Hi Jon,
Thanks for your reply. As a follow up to my previous question here, I am analyzing zeek processing network traffic which is quite messy, lots of retransmissions, duplicate acks, etc. and a weird log reporting a lot of potential issues. We are looking into fixing these issues. In this context, tcp conns are detected as partials quite often by Zeek and several analyzers do not process correctly the related traffic.
As I would like to get a better understanding of what is going on, I examined a single tcp+http connection in more details. Zeek starts processing it correctly at first, there is the tcp handshake and the first GET/POST are logged correctly. Then Zeek stops processing the http traffic. By debugging, I noticed that in the middle of the conn, zeek calls again the ctor of the tcp analyzer and and as a consequence resets "is_partial" and other variables. Again from debugging it seems this happens because of the timer TCPConnectionExpireTimer is dispatched at a certain point.
Is this what you would expect in the middle of a conn? It doesn't seem to me that there is a long inactivity on this conn.
Best wishes,
Mauro
-----Messaggio originale-----
Hard to say without looking directly at a pcap which reproduces the
behavior, but yes, there do exist various inactivity timers you might
expect/suspect to interfere or cause things like that to happen. You
might look more closely at TCP_Analyzer::ExpireTimer() to find which
condition is being met and see if it makes sense for the particular
connection(s). You'll also find out from that whether there's
particular timeout interval options to try tuning for your use-case.
E.g. the relevant ones look like they're 5-6 seconds by default:
"tcp_SYN_timeout", "tcp_session_timer", and "tcp_connection_linger".
- Jon
Actually the pcap file I was looking into had only half the traffic in it (only one direction). That's why I was seeing a quite odd behavior. After fixing this, zeek disables the http analyzer when it detects some gaps in the TCP flux, i.e. some packets are missing. This makes much more sense.
Thanks again,
Mauro
-----Messaggio originale-----