I have a few questions on behavioral issues with the intel framework and SMB / SMB file logging:
I’m not sure if this is expected behavior or not, but it doesn’t look like filenames parsed in smb_files.log are properly being logged in files.log. We had a red team exercise recently where our red team was able to successfully retrieve the ntds.dit file off of one of our domain controllers. This transfer occurred over SMB, so I figured we could add ntds.dit to the Intel framework so that next time we don’t have to dig in logs to find out that our domain is owned – we’ll have a handy alert to tell us I did some testing with this though, and while I see ‘ntds.dit’ logged clearly in the name field in smb_files.log, I don’t have a corresponding entry in files.log for this file transfer, and therefore no Intel match. What makes this weirder is I have other irrelevant files from this connection logged in files.log, that I didn’t actually touch or move during this connection:
bro@SObro:/nsm/bro/logs/current$ cat /opt/bro/share/bro/intel/intel.dat | grep ntds.dit
ntds.dit Intel::FILE_NAME domain ownage - update your resume! F
If I search for “ntds.dit” in files log, I get nothing. If I search for the connection UID in files.log, there are some files logged – but not the only file I actually transferred over this connection!
bro@ SObro:/nsm/bro/logs/2018-03-31$ zcat files.16:00:00-17:00:00.log.gz | bro-cut conn_uids tx_hosts rx_hosts source filename | grep C35jBF1HlcrVNLiXW2
C35jBF1HlcrVNLiXW2 1.1.1.1 2.2.2.2 SMB desktop.ini
C35jBF1HlcrVNLiXW2 1.1.1.1 2.2.2.2 SMB share path\and more\more\not my testing directory!? \desktop.ini
C35jBF1HlcrVNLiXW2 1.1.1.1 2.2.2.2 SMB share path\and more\more\my testing directory \random file that lives at this path.exe
C35jBF1HlcrVNLiXW2 1.1.1.1 2.2.2.2 SMB desktop.ini
C35jBF1HlcrVNLiXW2 1.1.1.1 2.2.2.2 SMB favorites\desktop.ini
C35jBF1HlcrVNLiXW2 1.1.1.1 2.2.2.2 SMB Random excel file that lives in my testing directory.xls
C35jBF1HlcrVNLiXW2 1.1.1.1 2.2.2.2 SMB random executable that lives in my testing directory.exe
Is there something wrong with my Bro instance? I feel like filenames from smb_files ‘name’ field should all be fed into files.log. I tested this with two different share paths and similar results – everything gets logged as I would expect in smb_files.log but this filename never shows up in files.log. How can I reliably alert on file names transferred over SMB?
As part of the above red team exercise, I found (what I suspect) are some instances of Meterpeter being transferred from popped hosts back to the adversary system over SMB. These were logged in “smb_files.log” with names like: “Temp\PBetVKZU.tmp” and “Temp\FapcPatS.tmp”. I don’t think the Intel framework supports wildcards – is there a way to alert on files transferred that match a regex such as “Temp\[a-zA-Z]{8}.tmp”, or even: “Temp\*.tmp”?
Unrelated to the Intel framework - it seems like smb_files.log is super noisy. If I browse to a share drive, a massive amount of the contents of the share are enumerated in the smb_files log without taking any action (with the ‘action’ field indicating SMB::FILE_OPEN). This feels like expected behavior in SMB. Is there any way to ‘filter’ the log to only log files that are actually opened, written to, moved, deleted, or had any real operation occur against them?
We’re running Bro 2.5.3 in Security Onion (Ubuntu 14.04). The intel framework is loaded and successfully fires on other indicators we have running.
I don’t think the Intel framework supports wildcards – is there a way to alert
on files transferred that match a regex such as “Temp\[a-zA-Z]{8}.tmp”,
or even: “Temp\*.tmp”?
Use the ‘file_over_new_connection’ event. You could get the file name from the SMB info in the ‘c : connection’ input argument to the event handler. Here is some sample code:
I checked in our autogenerated documentation and I'm now noticing a glaring lack of documentation about *what* that log represents.
That log is meant to represent that an action against a file was seen. Considering that since SMB is a file system protocol you will see a lot of actions that aren't necessarily file transfers. Most of what I've typically seen in smb_files.log is files being opened. This seems be a very common action in SMB, I don't know all of the reasons that an application might open a file but not read nor write to it but it's definitely seen a lot. I think that it could be a way for the client to get a timestamp for a remote file or other metadata. Other actions that fits into that log are file renames and deletes. File renames are represented through the "name" and "prev_name" fields. Deletes just give the name of the file before it was deleted.
The important field to look at is the "action" field since that's where the log tells you what the client was actually doing. Here are all of the potential actions that you will typically see there...
FILE_OPEN,
FILE_RENAME,
FILE_DELETE,
If you wanted, you could add FILE_CLOSE as action that you want logged if you add this to your local.bro (I chose to leave this out by default since it didn't seem to provide much extra context, but had a great increase in log size)...
You could also have it include FILE_WRITE and FILE_READ, but those are really not great to include and lead to a massive increase in log size but I felt like transfers were better represented through the files.log anyway. Generally if data is seen as being transferred that will be represented in files.log since that's the aggregation point for all "files" seen by Bro. smb_files.log can be really helpful if you need to know when someone is poking around at files even if they aren't transferring them and there is a lot of information there that can be useful forensically (MAC times being the big one in my mind).
If you have thoughts on other SMB logs that you'd like to see or if you have thoughts on extensions to the smb_files.log I'd love to hear them! There is definitely no right or wrong answer to what should be in those logs or even what logs should exist.
First off, thanks for the info! Mark, I’ll work on flushing out that code - looks like exactly what I need to alert on those types of filenames (if there turns out to be value there )
Seth - in my testing on this, I was transferring a text document titled ‘ntds.dit’ over an SMB connection that Bro was monitoring. I think my confusion on this is over what the SMB analyzer chooses to send into files.log. I would expect to see the filename of a file that was written over that connection, where in this case I had files that were opened (as you mentioned, probably just by the client caching timestamps), but not the file that was actually sent over that connection.
In my original email, I’d mentioned a red team ‘adversary’ pulled the ntds.dit file off a domain controller and sent it to his system over an SMB connection. For this connection, I have an smb_files log entry AND a files.log entry that log a filename of: temp//ntds.dit being transferred.
When I created a test text document named ‘ntds.dit’ and sent it to a remote fileshare, bro recorded the correct name in smb_files.log, but nothing about that file in files.log!
I’d like to rephrase my questions a little:
1.When does a filename seen over SMB appear in files.log?
As far as flushing out some of the noise in smb_files.log - would it be possible to only log FILE_OPEN’s where the file has a ‘time accessed’ timestamp that falls in the duration of the connection? This might not be foolproof, but could cut down on file_open noise substantially. Just brainstorming on this - but it seems like that would eliminate a lot of the noise where the smb client “opens” a file but really just grabs that file’s timestamps or other metadata.
Now that starts to sound more like a bug or packet loss.
Any details about the connection where you'd expect to see the file or other logs that showed up that were related would be appreciated. In an ideal world I'd also ask for a PCAP, but that sounds unlikely... maybe? Basically I'd like to hear any more of the specifics about what Bro *did* see so that we can possibly figure out what did actually happen.
I did some testing with this though, and while I see ‘ntds.dit’ logged clearly in the name field in smb_files.log, I don’t have a corresponding entry in files.log for this file transfer, and therefore no Intel match. What makes this weirder is I have other irrelevant files from this connection logged in files.log, that I didn’t actually touch or move during this connection:
