Logging TCP server banners

Development
1

Hi everyone,

[This mail has been sent to bro@ first, but I think I might have more
luck (and answers) here. Sorry for the inconvenience to those who have
already read it.]

For a network recon framework I am working on (https://ivre.rocks/ --
for those interested), I would like to log each "TCP server banner"
Bro sees.

I call "TCP server banner" the first chunk of data a server sends,
before the client has sent data (if the client sends data before the
server, I don't want to log anything).

Here is what I have done so far (`PassiveRecon` is my module's name):

export {
        redef tcp_content_deliver_all_resp = T;
        
        [...]
}

[...]

event tcp_contents(c: connection, is_orig: bool, seq: count, contents: string)
        {
        if (! is_orig && seq == 1 && c$orig$num_pkts == 2)
                {
                Log::write(PassiveRecon::LOG, [$ts=c$start_time,
                                               $host=c$id$resp_h,
                                               $srvport=c$id$resp_p,
                                               $recon_type=TCP_SERVER_BANNER,
                                               $value=contents]);
                }
        }

Basically, I consider we have a "TCP server banner" when `is_orig` is
false, when `seq` equals 1 and when we have seen exactly two packets
from the client (which should be a SYN and the first ACK).

This solution generally works **but** I sometimes log a data chunk
when I should not, particularly if I have missed part of the
traffic.

As an example, the following Scapy script creates a PCAP file that
would trick my script into logging a "TCP server banner" while the
client has actually sent some data (and we have missed an ACK packet,
left as a comment in the script):

wrpcap("test.cap", [
    Ether() / IP(dst="1.2.3.4", src="5.6.7.8") /
    TCP(dport=80, sport=5678, flags="S", ack=0, seq=555678),
    Ether() / IP(src="1.2.3.4", dst="5.6.7.8") /
    TCP(sport=80, dport=5678, flags="SA", seq=111234, ack=555679),
    # Ether() / IP(dst="1.2.3.4", src="5.6.7.8") /
    # TCP(dport=80, sport=5678, flags="A", ack=111235, seq=555679),
    Ether() / IP(dst="1.2.3.4", src="5.6.7.8") /
    TCP(dport=80, sport=5678, flags="PA", ack=111235, seq=555679) / "DATA",
    Ether() / IP(src="1.2.3.4", dst="5.6.7.8") /
    TCP(sport=80, dport=5678, flags="PA", seq=111235, ack=555683) / "DATA"
])

Is there a way to know that I have not missed any packet from the
client and/or a way to know that the client has not sent any data on
the connection (like an equivalent of the `seq` parameter, but for the
`ack`)?

Also, when `seq` equals 1, am I certain that I have not missed any
packet from the server?

One more question: is there a better, cleaner, etc. way to do what I'm
trying to do?

Thanks a lot,

Pierre

2

This fits with a feature that I've been talking to several people about for quite a while which would make a bit of the beginning of each direction of a connection available in script-land. That would help with your problem a bit, but it sounds like since there is a particular packet that you want, you may want to write your own analyzer that gets the exact data that you are looking for because you should be able to do packet level stuff easily there.

   .Seth

3

I call "TCP server banner" the first chunk of data a server sends,
before the client has sent data (if the client sends data before the
server, I don't want to log anything).

A solution could be to blacklist such connections, i-e if there is data

        if (! is_orig && seq == 1 && c$orig$num_pkts == 2 && c$orig$size == 0)

Another thing that comes to me is what if you miss the SYN or the
SYN-ACK segment sent by your client? You will not log the banner so I am
not sure about the second condition : c$orig$num_pkts == 2. I would
remove it.

With the pcap generated with the scapy script you gave, I do not log
anymore, however if I change it to this:

wrpcap("test.cap", [
    Ether() / IP(dst="1.2.3.4", src="5.6.7.8") /
    TCP(dport=80, sport=5678, flags="S", ack=0, seq=555678),
    Ether() / IP(src="1.2.3.4", dst="5.6.7.8") /
    TCP(sport=80, dport=5678, flags="SA", seq=111234, ack=555679),
    Ether() / IP(dst="1.2.3.4", src="5.6.7.8") /
    TCP(dport=80, sport=5678, flags="A", ack=111235, seq=555679),
    # Ether() / IP(dst="1.2.3.4", src="5.6.7.8") / no more data sent by the client
    # TCP(dport=80, sport=5678, flags="PA", ack=111235, seq=555679) / "DATA",
    Ether() / IP(src="1.2.3.4", dst="5.6.7.8") /
    TCP(sport=80, dport=5678, flags="PA", seq=111235, ack=555679) / "DATA"
])

I do have an entry in the log.

Also, when `seq` equals 1, am I certain that I have not missed any
packet from the server?

No idea about that, I think the answer is in Bro's TCP implementation in
src/analyzer/protocol/tcp somewhere.

Regards,

4

Another thing that comes to me is what if you miss the SYN or the
SYN-ACK segment sent by your client?

I meant SYN or ACK (third one in the handshake) segment sent by the
client. Sorry.

Regards,

5

Hi bro team,

Do you have plans to integrate oss-fuzz ? like
https://github.com/google/oss-fuzz/issues/624

All the best,
Philippe

6

Hi,

This fits with a feature that I've been talking to several people
about for quite a while which would make a bit of the beginning of
each direction of a connection available in script-land.

I think that would be great!

That would help with your problem a bit, but it sounds like since
there is a particular packet that you want, you may want to write
your own analyzer that gets the exact data that you are looking for
because you should be able to do packet level stuff easily there.

I wanted to avoid that, but actually I think you're right.

Thanks for your answer,

Pierre

7

Hi,

A solution could be to blacklist such connections, i-e if there is data
sent by the client, then do not log:
> if (! is_orig && seq == 1 && c$orig$num_pkts == 2 && c$orig$size == 0)

Another thing that comes to me is what if you miss the SYN or the
SYN-ACK segment sent by your client? You will not log the banner so I am
not sure about the second condition : c$orig$num_pkts == 2. I would
remove it.

Thanks! Indeed, changing `c$orig$num_pkts == 2` to `c$orig$size == 0`
is a good move, I wish I had this idea!

With the pcap generated with the scapy script you gave, I do not log
anymore, however if I change it to this:

wrpcap("test.cap", [
    Ether() / IP(dst="1.2.3.4", src="5.6.7.8") /
    TCP(dport=80, sport=5678, flags="S", ack=0, seq=555678),
    Ether() / IP(src="1.2.3.4", dst="5.6.7.8") /
    TCP(sport=80, dport=5678, flags="SA", seq=111234, ack=555679),
    Ether() / IP(dst="1.2.3.4", src="5.6.7.8") /
    TCP(dport=80, sport=5678, flags="A", ack=111235, seq=555679),
    # Ether() / IP(dst="1.2.3.4", src="5.6.7.8") / no more data sent by the client
    # TCP(dport=80, sport=5678, flags="PA", ack=111235, seq=555679) / "DATA",
    Ether() / IP(src="1.2.3.4", dst="5.6.7.8") /
    TCP(sport=80, dport=5678, flags="PA", seq=111235, ack=555679) / "DATA"
])

I do have an entry in the log.

> Also, when `seq` equals 1, am I certain that I have not missed any
> packet from the server?

No idea about that, I think the answer is in Bro's TCP implementation in
src/analyzer/protocol/tcp somewhere.

I think, as suggested by Seth Hall, that I would have to write my own
analyzer for that.

Thanks a lot,

Pierre