Sometimes HTTP.log fails to generate from PCAP file

Zeek
1

Hi Everyone,

This is my first time using this mailing list so I apologise in advance if I’ve followed the wrong format/protocol etc.

I am doing some malware research and making use of the HTTP.log generated by Bro. I’ve noticed some PCAPs fail to generate a HTTP log. I’ve looked at a couple of examples and thought maybe it is because there is no SYN-ACK before the HTTP connection in the PCAP (the researcher who generated the PCAP may have cut this out or not captured it).

Can anybody confirm why the HTTP.log fails to generate (is it the missing SYN-ACK at the start?) and advise if there is some way I can still extract the HTTP traffic from the PCAP using Bro (since it’s clearly all visible in Wireshark).

Note: I’m unable to attach screenshots of any of the problematic PCAPs due to email size…

Thanks in advance,

Jonah (@_CryptoCat)

2

Can anybody confirm why the HTTP.log fails to generate (is it the missing SYN-ACK at the start?)

Yes, that's likely the reason -- the HTTP parser (or any parser
really) don't make an attempt to (re)synchronize with the HTTP
protocol in the case we may be starting somewhere in the middle of the
TCP stream.

and advise if there is some way I can still extract the HTTP traffic from the PCAP using Bro (since it’s clearly all visible in Wireshark).

Not out of the box, but attached is a patch that removes the checks
which currently cause the HTTP analysis to be skipped for such
connections. If all that's missing is the TCP handshake, then the
http.log produced with the patched version I think will be the same or
similar enough to if the handshake were there. If there's more
packets missing than that, it will likely still fail to parse out much
HTTP data.

- Jon

http-partial.patch (718 Bytes)

3

Try this on your pcap file to see all analyzed events:

Bro Dump Events policy script:

bro -C -r file.pcap policy/misc/dump-events.bro

I would also expect if your trace is a partial capture that Bro will create a weird log entry with a description of what’s going on.

Charles “Chuck” A. Fair
chuck.fair@perched.io

Chuck