Several protosig questions

Zeek
1

Wow…so here’s my sig:

signature protosig_ntp_apple {
dst-ip == 17.0.0.0/8
ip-proto == udp
dst-port == 123
payload /.*\x00/
payload-size == 48
eval ProtoSig::match
}

First, IP is 192.168.1.95 → 17.253.4.253 udp port 123.

Issue #1: CIDR doesn’t appear to work…with the above dst-ip entry this fails to identify ntp_apple, commenting out the dst-ip the line matches.
Issue #2: Payload-size; of interest, if you don’t set a payload entry, then setting payload-size with “>” and “==” won’t match, but ANY number with “<” fired off. Ironically I could set payload-size < 1 and this would fire.

this is using latest beta. Thank you.

James

2

Also if interest, header ip[16:4] == 17.0.0.0/8 DOES in fact work, so I believe there’s an issue with the dst-ip item.

James

3

Do you have a trace that you can send demonstrating the two issues?

Robin

4

Do you have a trace that you can send demonstrating the two issues?

Robin

Included! Sigs below (in 2.4.1 order mattered..I think last matched gets the protosig tag, but I've swapped these around with the same results)..in either case only ntp matches, not ntp_apple. Maybe it's a beta thing?

signature protosig_ntp {
   ip-proto == udp
   dst-port == 123
   payload /.*\x00/
   payload-size == 48
   eval ProtoSig::match
}

signature protosig_ntp_apple {
   #header ip[16:4] == 17.0.0.0/8
   dst-ip = = 17.0.0.0/8
   ip-proto == udp
   dst-port == 123
   payload /.*\x00/
   payload-size == 48
   eval ProtoSig::match
}

Thank you.

James

ntp-1.pcap (1.27 KB)

5

Do you have a trace that you can send demonstrating the two issues?

Robin

Included! Sigs below (in 2.4.1 order mattered..I think last matched
gets the protosig tag, but I've swapped these around with the same
results)..in either case only ntp matches, not ntp_apple. Maybe it's
a beta thing?

signature protosig_ntp {
  ip-proto == udp
  dst-port == 123
  payload /.*\x00/
  payload-size == 48
  eval ProtoSig::match
}

signature protosig_ntp_apple {
  #header ip[16:4] == 17.0.0.0/8

-> dst-ip == 17.0.0.0/8

6

Thanks, I'll take a look, give me a little bit.

Robin

7

Thanks Robin...I err on the side of "maybe I hosed something up" in general :slight_smile:

James

8

So both the problems turn out to be bugs: dst-ip is indeed not working
with IPv4 CIDR ranges, and payload-size is behaving oddly with (I
believe only) UDP. I have patches for both in git branch
topic/robin/sig-fixes, could you give that a try?

Robin

9

Can do…those patches in the bro git repository or somewhere else? I wasn’t able to find them on github. Thanks for the work Robin!

James

10

Yeah: Fix a couple of problems with signature matching. · bro/bro@5cf2320 · GitHub

Robin

11

So ok here’s where I’m at. With only the ntp_apple rule this works. However if I have a generic ntp rule, either before or after the ntp_apple, I only get the ntp match:

signature protosig_ntp {
ip-proto == udp
dst-port == 123
payload /.*\x00/
payload-size == 48
eval ProtoSig::match
}

signature protosig_ntp_apple {
#header ip[16:4] == 17.0.0.0/8
dst-ip == 17.0.0.0/8
ip-proto == udp
dst-port == 123
payload /.*\x00/
payload-size == 48
eval ProtoSig::match
}

#signature protosig_ntp {

ip-proto == udp

dst-port == 123

payload /.*\x00/

payload-size == 48

eval ProtoSig::match

#}

Currently with 2.4.1 protosig you put the generic one first, and the specific ones after. This appears to have changed? Anyway at least it does now match, so that’s a plus.

12

However if I have a generic ntp rule, either before or after the
ntp_apple, I only get the ntp match:

Let me clarify one thing:

eval ProtoSig::match

"eval" is not for flagging a match. It's a condition by itself that
influences the matching of the signature. To learn about a match use
"event" instead and then hook into the "signature_event" event. If I
do that, things seem to work for me correctly with the sig-fixes
branch:

    # cat test.sig
    signature protosig_ntp {
      ip-proto == udp
      dst-port == 123
      payload /.*\x00/
      payload-size == 48
      event "match"
    
    }
    signature protosig_ntp_apple {
      dst-ip == 17.0.0.0/8
      ip-proto == udp
      dst-port == 123
      payload /.*\x00/
      payload-size == 48
      event "match"
    }

    # cat test.bro
    event signature_match(state: signature_state, msg: string, data: string)
            {
            print "signature match", state$sig_id;
            }

    # bro -s ./test.sig -r ntp-1.pcap ./test.bro
    signature match, protosig_ntp
    signature match, protosig_ntp_apple
    signature match, protosig_ntp
    signature match, protosig_ntp_apple
    signature match, protosig_ntp
    signature match, protosig_ntp_apple
    signature match, protosig_ntp
    signature match, protosig_ntp_apple
    signature match, protosig_ntp
    signature match, protosig_ntp_apple
    signature match, protosig_ntp
    signature match, protosig_ntp_apple
    
If I add "eval", I do see it execute for both signatures, though more
often for the generic one. That's probably an artefact of the order in
which conditions are run internally; having the dst-ip in there may
change that.

Btw, the order of matches is undefined, and might have well changed
since 2.4.

Robin

13

However if I have a generic ntp rule, either before or after the
ntp_apple, I only get the ntp match:

Let me clarify one thing:

eval ProtoSig::match

"eval" is not for flagging a match. It's a condition by itself that
influences the matching of the signature. To learn about a match use
"event" instead and then hook into the "signature_event" event. If I
do that, things seem to work for me correctly with the sig-fixes
branch:

Ok cool...I've just been going per the docs here:

https://github.com/broala/bro-protosigs

"You must add the eval ProtoSig::match condition into your signature that does the final match. That call is what ties the signature match into the protosigs Bro scripts."

I'll give event a go and report my findings. Thanks so much Robin!

James

14

However if I have a generic ntp rule, either before or after the
ntp_apple, I only get the ntp match:

Let me clarify one thing:

eval ProtoSig::match

"eval" is not for flagging a match. It's a condition by itself that
influences the matching of the signature. To learn about a match use
"event" instead and then hook into the "signature_event" event. If I
do that, things seem to work for me correctly with the sig-fixes
branch:

Ok first off thanks for that test setup...now I can just test a sig vs. a pcap, so that's tight. My results:

[13:36:27 @tester:~/dev/bro$] bro -s ./test.sig -r pcaps/ntp-1.pcap ./test.bro
signature match, protosig_ntp
signature match, protosig_ntp_apple
signature match, protosig_ntp
signature match, protosig_ntp_apple
signature match, protosig_ntp
signature match, protosig_ntp_apple
signature match, protosig_ntp
signature match, protosig_ntp_apple
signature match, protosig_ntp
signature match, protosig_ntp_apple
signature match, protosig_ntp
signature match, protosig_ntp_apple

So it does indeed match...however in the official conn.log, this is what I get:

[13:36:32 @tester:~/dev/bro$] ./testhome pcaps/ntp-1.pcap
[13:36:37 @tester:~/dev/bro$] cat conn.log
#separator \x09
#set_separator ,
#empty_field (empty)
#unset_field -
#path conn
#open 2016-10-24-13-36-37
#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p proto service duration orig_bytes resp_bytes conn_state local_orig local_resp missed_bytes history orig_pkts orig_ip_bytes resp_pkts resp_ip_bytes tunnel_parents protosig
#types time string addr port addr port enum string interval count count string bool bool count string count count count count set[string] string
1476535656.489094 ClixtHWwLmpBYkZRh 192.168.1.95 123 17.253.4.253 123 udp - 0.040715 48 48 SF T F 0 Dd 1 76 1 76 (empty) ntp
1476535656.533910 CJFnxQiLgFYwVcEFi 192.168.1.95 123 17.253.4.125 123 udp - 0.040804 48 48 SF T F 0 Dd 1 76 1 76 (empty) ntp
1476535657.111868 Cds9uP3GtXbb1jHh3d 192.168.1.95 123 17.253.26.253 123 udp - 0.037826 48 48 SF T F 0 Dd 1 76 1 76 (empty) ntp
1476535738.400766 CTkIhX1qHjadF6iple 192.168.1.100 123 17.253.4.253 123 udp - 0.040577 48 48 SF T F 0 Dd 1 76 1 76 (empty) ntp
1476535738.360132 Chm9Q6WalLZnpFx4g 192.168.1.100 123 17.253.26.253 123 udp - 0.037825 48 48 SF T F 0 Dd 1 76 1 76 (empty) ntp
1476535739.752622 CRWW8j41rCTK6gYZSk 192.168.1.100 123 17.253.4.125 123 udp - 0.040857 48 48 SF T F 0 Dd 1 76 1 76 (empty) ntp
#close 2016-10-24-13-36-37

Swapping which sig is first gets me this:

[13:46:24 @tester:~/dev/bro$] bro -s ./test.sig -r pcaps/ntp-1.pcap ./test.bro
signature match, protosig_ntp_apple
signature match, protosig_ntp
signature match, protosig_ntp_apple
signature match, protosig_ntp
signature match, protosig_ntp_apple
signature match, protosig_ntp
signature match, protosig_ntp_apple
signature match, protosig_ntp
signature match, protosig_ntp_apple
signature match, protosig_ntp
signature match, protosig_ntp_apple
signature match, protosig_ntp

But the same results as above in conn.log. So I guess that's a feature request? To hard define either a first rule that matches gets logged, or the last rule that matches gets logged. This will allow granular flow identification..which, to be honest, is the whole reason I'm doing this in the first place :slight_smile: Thanks again Robin.

15

It's a feature, not a bug. :slight_smile: The signature engine always reports all
matches, actually with the intention to *not* make order matter. What
you could do is add logic in scriptland that selects which match to
continue working with, based on some scheme you come up with (like
having a table of signature names map to priorities).

Robin

16

Thanks Robin…that helps. Truth be told I wouldn’t have a clue on where to start have a table of sigs to map priorities, so I guess I’ll suck it up and just make specific sigs and leave out the generics. I’ll keep testing and report anything else…thanks again for the work on this.

James