Hello,
I am attempting to get Bro working sitting behind a reverse proxy (nginx), which is receiving connections, terminating TLS, and forwarding cleartext HTTP to a local app server (Tomcat). I have a really simple test case that demonstrates the problem I’m running into, which is that Bro HTTP events are only detected when requests are sent plaintext (without TLS). Here is the test case I’m using:
[guest-vm-bro]$ cat ~/bro-custom/detect-http-request.bro
##! Basic rule to detect HTTP request
@load base/protocols/http
module HTTP;
event http_request(c: connection, method: string, original_URI: string,
unescaped_URI: string, version: string) &priority=3
{
print "HTTP request. URI: ", original_URI;
}
event connection_established(c: connection)
{
print "Connection: ", c;
}
[guest-vm-bro]$ sudo /usr/local/bro/bin/bro -C -i eth1 ~/bro-custom/detect-http-request.bro
listening on eth1
Everything works as expected when requests are sent as plaintext to nginx port 80:
[host-os]$ curl [http://uis.local/sample/](http://uis.local/sample/)
<html>
<head>
<title>Sample "Hello, World" Application</title>
</head>
snip...
[guest-vm-bro]$ sudo /usr/local/bro/bin/bro -C -i eth1 ~/bro-custom/detect-http-request.bro
listening on eth1
Connection: , [id=[orig_h=192.168.43.1, orig_p=53030/tcp, resp_h=192.168.43.10, resp_p=80/tcp], orig=[size=0, state=4, num_pkts=1, num_bytes_ip=64, flow_label=0, l2_addr=0a:00:27:00:00:03], resp=[size=0, state=4, num_pkts=0, num_bytes_ip=0, flow_label=0, l2_addr=08:00:27:18:49:26], start_time=1522451133.083394, duration=0.000023, service={
}, history=Sh, uid=CSecNs4PkfexRaPAp4, tunnel=<uninitialized>, vlan=<uninitialized>, inner_vlan=<uninitialized>, dpd=<uninitialized>, conn=<uninitialized>, extract_orig=F, extract_resp=F, thresholds=<uninitialized>, dce_rpc=<uninitialized>, dce_rpc_state=<uninitialized>, dce_rpc_backing=<uninitialized>, dhcp=<uninitialized>, dnp3=<uninitialized>, dns=<uninitialized>, dns_state=<uninitialized>, ftp=<uninitialized>, ftp_data_reuse=F, ssl=<uninitialized>, http=<uninitialized>, http_state=<uninitialized>, irc=<uninitialized>, krb=<uninitialized>, modbus=<uninitialized>, mysql=<uninitialized>, ntlm=<uninitialized>, radius=<uninitialized>, rdp=<uninitialized>, rfb=<uninitialized>, sip=<uninitialized>, sip_state=<uninitialized>, snmp=<uninitialized>, smtp=<uninitialized>, smtp_state=<uninitialized>, socks=<uninitialized>, ssh=<uninitialized>, syslog=<uninitialized>]
HTTP request. URI: , /sample/
Note the last line, where the http_request event triggered correctly. When I send the request to the TLS encrypted nginx port, however, I only see the connection_established event fire:
[host-os]$ curl -k [https://uis.local/sample/](https://uis.local/sample/)
<html>
<head>
<title>Sample "Hello, World" Application</title>
<head>
snip...
Connection: , [id=[orig_h=192.168.43.1, orig_p=53078/tcp, resp_h=192.168.43.10, resp_p=443/tcp], orig=[size=0, state=4, num_pkts=1, num_bytes_ip=64, flow_label=0, l2_addr=0a:00:27:00:00:03], resp=[size=0, state=4, num_pkts=0, num_bytes_ip=0, flow_label=0, l2_addr=08:00:27:18:49:26], start_time=1522451394.887197, duration=0.000024, service={
}, history=Sh, uid=Ce2kbh4wXUDiXtRC1i, tunnel=<uninitialized>, vlan=<uninitialized>, inner_vlan=<uninitialized>, dpd=<uninitialized>, conn=<uninitialized>, extract_orig=F, extract_resp=F, thresholds=<uninitialized>, dce_rpc=<uninitialized>, dce_rpc_state=<uninitialized>, dce_rpc_backing=<uninitialized>, dhcp=<uninitialized>, dnp3=<uninitialized>, dns=<uninitialized>, dns_state=<uninitialized>, ftp=<uninitialized>, ftp_data_reuse=F, ssl=<uninitialized>, http=<uninitialized>, http_state=<uninitialized>, irc=<uninitialized>, krb=<uninitialized>, modbus=<uninitialized>, mysql=<uninitialized>, ntlm=<uninitialized>, radius=<uninitialized>, rdp=<uninitialized>, rfb=<uninitialized>, sip=<uninitialized>, sip_state=<uninitialized>, snmp=<uninitialized>, smtp=<uninitialized>, smtp_state=<uninitialized>, socks=<uninitialized>, ssh=<uninitialized>, syslog=<uninitialized>]
A similar event pattern happens if I change the Bro agent to listen on the loopback interface:
[host-os]$ curl [http://uis.local/sample/](http://uis.local/sample/)
<html>
<head>
<title>Sample "Hello, World" Application</title>
</head>
snip...
[host-os]$ curl -k [https://uis.local/sample/](https://uis.local/sample/)
<html>
<head>
<title>Sample "Hello, World" Application</title>
</head>
snip...
[guest-vm-bro]$ sudo /usr/local/bro/bin/bro -C -i lo ~/bro-custom/detect-http-request.bro
listening on lo
Connection: , [id=[orig_h=127.0.0.1, orig_p=58658/tcp, resp_h=127.0.0.1, resp_p=8080/tcp], orig=[size=0, state=4, num_pkts=1, num_bytes_ip=60, flow_label=0, l2_addr=<uninitialized>], resp=[size=0, state=4, num_pkts=0, num_bytes_ip=0, flow_label=0, l2_addr=<uninitialized>], start_time=1522451684.380294, duration=-1513581075.701362, service={
}, history=Sh, uid=C3NJSO38EkH9aVSRe8, tunnel=<uninitialized>, vlan=<uninitialized>, inner_vlan=<uninitialized>, dpd=<uninitialized>, conn=<uninitialized>, extract_orig=F, extract_resp=F, thresholds=<uninitialized>, dce_rpc=<uninitialized>, dce_rpc_state=<uninitialized>, dce_rpc_backing=<uninitialized>, dhcp=<uninitialized>, dnp3=<uninitialized>, dns=<uninitialized>, dns_state=<uninitialized>, ftp=<uninitialized>, ftp_data_reuse=F, ssl=<uninitialized>, http=<uninitialized>, http_state=<uninitialized>, irc=<uninitialized>, krb=<uninitialized>, modbus=<uninitialized>, mysql=<uninitialized>, ntlm=<uninitialized>, radius=<uninitialized>, rdp=<uninitialized>, rfb=<uninitialized>, sip=<uninitialized>, sip_state=<uninitialized>, snmp=<uninitialized>, smtp=<uninitialized>, smtp_state=<uninitialized>, socks=<uninitialized>, ssh=<uninitialized>, syslog=<uninitialized>]
Connection: , [id=[orig_h=127.0.0.1, orig_p=58660/tcp, resp_h=127.0.0.1, resp_p=8080/tcp], orig=[size=0, state=4, num_pkts=1, num_bytes_ip=60, flow_label=0, l2_addr=<uninitialized>], resp=[size=0, state=4, num_pkts=0, num_bytes_ip=0, flow_label=0, l2_addr=<uninitialized>], start_time=1522451702.019632, duration=-1513505330.101316, service={
}, history=Sh, uid=CqvuUv2nDCAjXdUcAg, tunnel=<uninitialized>, vlan=<uninitialized>, inner_vlan=<uninitialized>, dpd=<uninitialized>, conn=<uninitialized>, extract_orig=F, extract_resp=F, thresholds=<uninitialized>, dce_rpc=<uninitialized>, dce_rpc_state=<uninitialized>, dce_rpc_backing=<uninitialized>, dhcp=<uninitialized>, dnp3=<uninitialized>, dns=<uninitialized>, dns_state=<uninitialized>, ftp=<uninitialized>, ftp_data_reuse=F, ssl=<uninitialized>, http=<uninitialized>, http_state=<uninitialized>, irc=<uninitialized>, krb=<uninitialized>, modbus=<uninitialized>, mysql=<uninitialized>, ntlm=<uninitialized>, radius=<uninitialized>, rdp=<uninitialized>, rfb=<uninitialized>, sip=<uninitialized>, sip_state=<uninitialized>, snmp=<uninitialized>, smtp=<uninitialized>, smtp_state=<uninitialized>, socks=<uninitialized>, ssh=<uninitialized>, syslog=<uninitialized>]
In this case, I don’t receive an http_request event for either the plaintext or encrypted request sent to nginx. The last case I tested was sending a request directly to the app server from the guest VM running Bro. In this case I only saw the connection_established event again, not the http_request event:
[guest-vm-bro]$ curl localhost:8080/sample/
<html>
<head>
<title>Sample "Hello, World" Application</title>
</head>
snip...
Connection: , [id=[orig_h=127.0.0.1, orig_p=58664/tcp, resp_h=127.0.0.1, resp_p=8080/tcp], orig=[size=0, state=4, num_pkts=1, num_bytes_ip=60, flow_label=0, l2_addr=<uninitialized>], resp=[size=0, state=4, num_pkts=0, num_bytes_ip=0, flow_label=0, l2_addr=<uninitialized>], start_time=1522451893.753416, duration=-1512682034.693552, service={
}, history=Sh, uid=CdewMq37K1zzhpTl91, tunnel=<uninitialized>, vlan=<uninitialized>, inner_vlan=<uninitialized>, dpd=<uninitialized>, conn=<uninitialized>, extract_orig=F, extract_resp=F, thresholds=<uninitialized>, dce_rpc=<uninitialized>, dce_rpc_state=<uninitialized>, dce_rpc_backing=<uninitialized>, dhcp=<uninitialized>, dnp3=<uninitialized>, dns=<uninitialized>, dns_state=<uninitialized>, ftp=<uninitialized>, ftp_data_reuse=F, ssl=<uninitialized>, http=<uninitialized>, http_state=<uninitialized>, irc=<uninitialized>, krb=<uninitialized>, modbus=<uninitialized>, mysql=<uninitialized>, ntlm=<uninitialized>, radius=<uninitialized>, rdp=<uninitialized>, rfb=<uninitialized>, sip=<uninitialized>, sip_state=<uninitialized>, snmp=<uninitialized>, smtp=<uninitialized>, smtp_state=<uninitialized>, socks=<uninitialized>, ssh=<uninitialized>, syslog=<uninitialized>]
This was with Bro still listening on lo interface. This case surprises me. It’s a simple GET request to a local HTTP server on 8080, so I’m curious why this doesn’t trigger an http_request event.
Just to be complete, I also re-ran this last test with Bro listening on eth1, and neither the connection_established or the http_request event fired.
I would love some help if any of you are willing to provide it. My two questions are essentially:
- What needs to be observed by the agent for a http_request event to fire?
- Does Bro treat traffic traversing the localhost only differently than traffic it receives over the network?
Thanks very much, in advance, for reading through this and for helping me to understand what is happening in this case.
Best,
Brandon Sterne