How to activate an application-layer analyzer when signature dpd_reassemble_first_packets is off?

Zeek
1

Hello everyone,

I have a question about the dpd_reassemble_first_packets option of the signature framework.

According to the doc signatures options, as I quote:

  • dpd_reassemble_first_packets
    If true, Zeek reassembles the beginning of every TCP connection (of up to dpd_buffer_size bytes, see below also), to facilitate reliable matching across packet boundaries. If false, only connections are reassembled for which an application-layer analyzer gets activated (e.g., by Zeek’s dynamic protocol detection).

I wonder how to activate an application-layer analyzer as for the second case? I did some tests of the options with a simple http pcap (it does nothing but a simple http curl request and response on port 80):

  • with the option is on
    the http analyzer works as expected, both conn.log and http.log were generated, and the log shows that http protocol were correctly recognized and analyzed:
{
  "ts": 1658803008.807233,
  "uid": "CtIc132gciBxhf92s9",
  "id.orig_h": "192.168.41.66",
  "id.orig_p": 47706,
  "id.resp_h": "14.215.177.38",
  "id.resp_p": 80,
  "proto": "tcp",
  "service": "http",
  "duration": 1.0121288299560547,
  "orig_bytes": 77,
  "resp_bytes": 2781,
  "conn_state": "SF",
  "missed_bytes": 0,
  "history": "ShADadFf",
  "orig_pkts": 7,
  "orig_ip_bytes": 449,
  "resp_pkts": 6,
  "resp_ip_bytes": 3101
}
  • however with the option is off, only conn.log were generated (no http.log)
{
  "ts": 1658803008.807233,
  "uid": "CtwPDZ18cbYiD8XNh2",
  "id.orig_h": "192.168.41.66",
  "id.orig_p": 47706,
  "id.resp_h": "14.215.177.38",
  "id.resp_p": 80,
  "proto": "tcp",
  "duration": 1.0121288299560547,
  "orig_bytes": 77,
  "resp_bytes": 2781,
  "conn_state": "SF",
  "missed_bytes": 0,
  "history": "ShADadFf",
  "orig_pkts": 7,
  "orig_ip_bytes": 449,
  "resp_pkts": 6,
  "resp_ip_bytes": 3101
}

At first I thought the http analyzer should work even dpd is turn off for standard 80 ports, as which are defined in the http analyzer script base/protocols/http/main.zeek:

...
const ports = {
	80/tcp, 81/tcp, 631/tcp, 1080/tcp, 3128/tcp,
	8000/tcp, 8080/tcp, 8888/tcp,
};
redef likely_server_ports += { ports };
...
# Initialize the HTTP logging stream and ports.
event zeek_init() &priority=5
	{
	Log::create_stream(HTTP::LOG, [$columns=Info, $ev=log_http, $path="http", $policy=log_policy]);
	Analyzer::register_for_ports(Analyzer::ANALYZER_HTTP, ports);
	}

But the test result show that it is not the case, and it seems that zeek didn’t activate the http analyzer at all. So how to enable the http analyzer in this cases?

I tested based on zeek lts 4.0.x version, thanks for help!

2

Hey @mchen - interesting!

But the test result show that it is not the case, and it seems that zeek didn’t activate the http analyzer at all. So how to enable the http analyzer in this cases?

I wonder if you found a very very old bug in Zeek/Bro. I tried your test on try.zeek.org and no Zeek or Bro versions appears to do HTTP analysis if dpd_reassemble_first_packets is set to F.

Running with -B dpd,packet_analysis does indicate the HTTP analyzer is attached to the root analyzer based on the known port 80:

1362692526.869344/1666704174.963249 [packet_analysis] Analysis in IP succeeded, next layer identifier is 0x6.                                                                 
1362692526.869344/1666704174.963308 [dpd] HTTP[3] added originator support CONTENTLINE[4]                                                                                     
1362692526.869344/1666704174.963327 [dpd] HTTP[3] added responder support CONTENTLINE[5]                                                                                      
1362692526.869344/1666704174.963343 [dpd] TCP[1] added child HTTP[3]                                                                                                          
1362692526.869344/1666704174.963357 [dpd] 141.142.228.5:59856 > 192.150.187.43:80 activated HTTP analyzer due to port 80                                                      
...

However, TCP reassembly isn’t activated for the connection. My understanding is that this is due to the following lines in the TCPSessionAdapter and GetChildren() not taking into account Analyzer.new_children (which is where the HTTP analyzer is added first), As TCP reassembly is not enabled, DeliverStream() is never called on the HTTP analyzer. Ouch.

	bool reass = (! GetChildren().empty()) || zeek::detail::dpd_reassemble_first_packets ||
	             zeek::detail::tcp_content_deliver_all_orig ||
	             zeek::detail::tcp_content_deliver_all_resp;

The following seems to fix it:

--- a/src/packet_analysis/protocol/ip/IPBasedAnalyzer.cc
+++ b/src/packet_analysis/protocol/ip/IPBasedAnalyzer.cc
@@ -216,6 +216,8 @@ void IPBasedAnalyzer::BuildSessionAnalyzerTree(Connection* conn)
                        }
                }
 
+       root->AppendNewChildren();
+
        root->AddExtraAnalyzers(conn);
 
        if ( pia )

Let me open this as a PR and see what @robin and others think. Presumably no one sets dpd_reassemble_first_packets to F…

Thanks!

3

PR 2502 opened here.

4

Thank you for your kind reply!
I did the tests because in our environment most traffic are with standard ports, so I assume that if disable the dpd might improve the performance.
Thanks again!