File Extraction: doc/xls=ok, docx/xlsx=ko

Zeek
1

Hello,
regarding filename I’m trying something like this:


      local fname = fmt("/bro/extracted/%s.%s", f$info$filename, ext);
           Files::add_analyzer(f, Files::ANALYZER_EXTRACT, [$extract_filename=fname]);
           break;

No errors but files are not being saved. :slight_smile:
Can you give me some help?
Thanks

2

Add this outside of any event handler:

redef FilesExtract::prefix = "/bro/extracted/";

Then change the code you gave to:

  local fname = fmt("%s.%s", f$info$filename, ext);
  Files::add_analyzer(f, Files::ANALYZER_EXTRACT, [$extract_filename=fname]);
  break;

.Seth

3

Hello,
I put

redef FilesExtract::prefix = "/bro/extracted/";

at the beginning of the script but when run gave me an error:


 error in ./extract.bro, line 1: "redef" used but not previously defined (FilesExtract::prefix)
internal warning in ./extract.bro, line 1: Can't document redef of FilesExtract::prefix, identifier lookup failed
4

Sorry, typing mistake...

redef FileExtract::prefix = "/bro/extracted/";

  .Seth

5

Hello,
now no errors but behavior is like after first suggestion of Peter: files not created.
Any idea?

6

Does the user you are running Bro as have permission to write to that directory? Does the directory exist?

  .Seth

7

Of course the user has rights to write in that folder and the folder exist, in fact with previous conf everything is ok, apart the name of the files…

8

Just an idea

What would the output be if you try to print the contents to stdout? Nice way of learing what the variable actually contains…

Put this somewhere just before the call to write the file. Then you know if you have a valid filename or not. Good practise is to use print when unsure about the contents of a variable, it will quickly reveal stuff instead of fumbling in total darkness.

print f$info$filename

As said earlier, I’ve never tried using that variable but it should be there and hold the right value at least according to the documentation.

/Peter

2016-02-19 09:05 skrev puntogtg@tiscali.it:

9

Ugh, I just realized the problem...

if ( f?$mime_type )
        ext = ext_map[f$mime_type];

That code can't work in the file_new event. In Bro 2.4, there is a new event named file_sniff. It's at the point where some content from the file has been seen and Bro has had a chance to look at it and take a guess about the file type. You aren't seeing any file extraction because you have a return statement that's returning if there is no known file extension (which there isn't at that point!).

event file_sniff(f: fa_file, meta: fa_metadata)
  {
  if ( meta?$mime_type )
    {
    # put your code here...
    }
  }

  .Seth

10

Hi,
I added but tells me

identifier not defined: fa_metadata
11

Apologies if I missed it, but which version of Bro are you running?

Josh

12

Josh,
I have to say apologies…
Was a good idea to check the version: I was running 2.3!
Now compiled the new one: 2.4.1 :slight_smile:
Into extracted.bro put again:

event file_sniff(f: fa_file, meta: fa_metadata)

 {

     if ( meta?$mime_type )

            local fname = fmt("/bro/extracted/%s.%s", f$info$filename, ext);
            Files::add_analyzer(f, Files::ANALYZER_EXTRACT, [$extract_filename=fname]);
        return;
          break;

it is working and files are coming with name!

A question now: previously I was using the map

global ext_map: table[string] of string = {
    ["application/x-dosexec"] = "exe",

    ["application/vnd.ms-excel"] = "xls",
}  &default ="";

to select what file type to save, now it seems all extensions are saved…
How can select what to keep?

Thanks

13

Your if statement isn't checking for a particular mime type before adding the extraction analyzer.

  .Seth