Question on file hashes and cyrmu db

Zeek
1

Hello,

I’m helping to customize an existing deployment of Bro and while I think I’m collecting all the file info correctly, I’m not hitting any matches when I run the hashes against cymru’s database. I was wondering if someone could confirm that none of these hashes match either. I’ve run them against the DNS,Whois and web queries and had no luck. I work at a very open place and I find it almost impossible that not one of the 1.7M hashes match. In the event there are no matches, could someone point me to some sample pcap files so I can test my scripts?

If someone wanted to help cross correlate my findings, I could send offline a .gz of 1.7M hashes from a few hours of collection.

Thanks again for any help or assistance

2

Hi Dave,

The following pcap generates a TeamCymruMalwareHashRegistry::Match for me:
https://github.com/markofu/workshop/blob/master/samples/pcaps/netforensics_evidence05.pcap

3

Most of those hashes are likely just web pages your user's are visiting so it think it's very possible that none of them would match.

I see that the pcap file Doug pointed you to isn't working for you either. It's very possible that you're using a DNS server that isn't very fast and Bro is finishing reading the tracefile before you get a DNS response which will cause you to not have a match. Try this...

bro -r netforensics_evidence05.pcap frameworks/files/detect-MHR exit_only_after_terminate=T

Wait for a few seconds and then hit ctrl-c and see if you get a notice. That "exit_only_after_terminate" bit I added at the end will ensure that Bro doesn't terminate as soon as it reaches the end of the tracefile, giving your DNS server a bit of time to respond.

  .Seth

4

Thanks to a few of you for helping me offline for this, I was sidetracked on other projects. I’m noticing some inconsistencies with Bro and the Cymru Hash Servce on my Bro box (2.3)

  1. When i download some files I expect to match through the service, it fails but it matches virustotal when I enter the MD5/SHA1 hash on their site.
  2. When I do get some matches from Cymru, I don’t get the entry in notice.log via the detect bro script.

I did change the detect-MHR.bro and made the following changes: changed the percent down to 1 (just to test) and added the .zip mime extension

I am running with checksums disabled and I’ve experienced this on a few bro boxes including a virtual I have loaded. For others who are doing dynamic analysis of files for malware/viruses, is this the best approach? Is there anything else I could try before I dig deeper into the code? I’ve verified it’s nothing stupid like DNS queries failing, what I haven’t done is started to dump the SHA256 to see if I have better luck with this hash value.

Also, the script seems to work with pcap files that people have provided so the network could be the issue but I don’t see any signs of packet loss, frame errors or other data.

Thanks again
Dave

5

Hi Dave,

Is it possible that NIC offloading functions are a factor?

http://blog.securityonion.net/2011/10/when-is-full-packet-capture-not-full.html

6

2. When I do get some matches from Cymru, I don't get the entry in notice.log via the detect bro script.

How do you know you get a match from Team Cymru if it doesn't show up in your notice.log?

I did change the detect-MHR.bro and made the following changes: changed the percent down to 1 (just to test) and added the .zip mime extension

You should really avoid making changes to that file. Instead you should have done this in local.bro (or elsewhere, just a script you control):

redef TeamCymruMalwareHashRegistry::match_file_types += /application\/zip/;
redef TeamCymruMalwareHashRegistry::notice_threshold = 1;

  .Seth

7
  1. When I do get some matches from Cymru, I don’t get the entry in notice.log via the detect bro script.

How do you know you get a match from Team Cymru if it doesn’t show up in your notice.log?

I manually dumped the MD5/SHA1 hashes from files.log and imported it into their web portal. For the ones that matches, I confirmed that the DNS query returned a match also.

I did change the detect-MHR.bro and made the following changes: changed the percent down to 1 (just to test) and added the .zip mime extension

You should really avoid making changes to that file. Instead you should have done this in local.bro (or elsewhere, just a script you control):

redef TeamCymruMalwareHashRegistry::match_file_types += /application/zip/;
redef TeamCymruMalwareHashRegistry::notice_threshold = 1;

Thanks again
Dave