# Extracted files don't rotate

**URL:** <https://community.zeek.org/t/extracted-files-dont-rotate/4644>\
**Category:** Zeek\
**Created:** [February 6, 2017, 5:32pm UTC](https://community.zeek.org/t/extracted-files-dont-rotate/4644 "2017-02-06T17:32:55Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![James\_inthe\_box](https://avatars.discourse-cdn.com/v4/letter/j/6f9a4e/32.png) [@James\_inthe\_box](https://community.zeek.org/u/James_inthe_box)\
**Post date:** [February 6, 2017, 5:32pm UTC](https://community.zeek.org/t/extracted-files-dont-rotate/4644/1 "2017-02-06T17:32:55Z")

</div>

Hey all,

So I recently changed the way I run bro at a site. Originally this was run via command line, now I have the below:

[logger]  
type=logger  
host=localhost

[manager]  
type=manager  
host=localhost

[proxy-1]  
type=proxy  
host=localhost

[worker-1]  
type=worker  
host=localhost  
interface=ethx

[worker-2]  
type=worker  
host=localhost  
interface=ethx

extract\_files shows up in worker-2. Here's the extract-files script:

global ext\_map: table[string] of string = {  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;["application/x-dosexec"] = "exe",  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;["application/zip"] = "zip",  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;["application/msword"] = "xls",  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;["application/vnd.openxmlformats-officedocument.wordprocessingml.document"] = "docx",  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;["application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"] = "xlsx",  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;["application/vnd.openxmlformats-officedocument.presentationml.presentation"] = "pptx"

};

event file\_sniff(f: fa\_file, meta: fa\_metadata)  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;if ( f$source != "SMTP" )  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;return;

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;if ( ! meta?$mime\_type || meta$mime\_type !in ext\_map )  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;return;

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;local ext = "";

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;if ( meta?$mime\_type )  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;ext = ext\_map[meta$mime\_type];

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;local fname = fmt("%s-%s.%s", f$source, f$id, ext);  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Files::add\_analyzer(f, Files::ANALYZER\_EXTRACT, [$extract\_filename=fname]);  
}

In looking, I see that the files are accumulating and not rotating out. Anything I can do to troubleshoot this? Thank you.

James

---

<div class="post-metadata">

**Author:** ![James\_inthe\_box](https://avatars.discourse-cdn.com/v4/letter/j/6f9a4e/32.png) [@James\_inthe\_box](https://community.zeek.org/u/James_inthe_box)\
**Post date:** [February 8, 2017, 10:01pm UTC](https://community.zeek.org/t/extracted-files-dont-rotate/4644/2 "2017-02-08T22:01:43Z")

</div>

Any takers on the below? Thank you.

James

---

<div class="post-metadata">

**Author:** ![Azoff\_Justin\_S](https://avatars.discourse-cdn.com/v4/letter/a/dec6dc/32.png) [@Azoff\_Justin\_S](https://community.zeek.org/u/Azoff_Justin_S)\
**Post date:** [February 8, 2017, 10:10pm UTC](https://community.zeek.org/t/extracted-files-dont-rotate/4644/3 "2017-02-08T22:10:54Z")

</div>

Ah yes.. Extracted files aren't managed by anything. If you want files archived in a specific way you need to extract them to the full path that you want or setup a cron job to move+compress them periodically.

If you had something specific in mind I could probably whip up an example script for you.

---

<div class="post-metadata">

**Author:** ![James\_inthe\_box](https://avatars.discourse-cdn.com/v4/letter/j/6f9a4e/32.png) [@James\_inthe\_box](https://community.zeek.org/u/James_inthe_box)\
**Post date:** [February 8, 2017, 10:17pm UTC](https://community.zeek.org/t/extracted-files-dont-rotate/4644/4 "2017-02-08T22:17:49Z")

</div>

Thanks Justin that's helpful. So as I look at my old setup I see that I indeed had move and compress manually on a cron job, so I'll just do that for the extract\_files dir. Maybe a feature request down the road would be (maybe in broctl.conf) to be able to add "pre" rotate and "post" rotate scripts. Just a thought.

James

---

<div class="post-metadata">

**Author:** ![Seth\_Hall3](https://avatars.discourse-cdn.com/v4/letter/s/d6d6ee/32.png) [@Seth\_Hall3](https://community.zeek.org/u/Seth_Hall3)\
**Post date:** [February 9, 2017, 3:15am UTC](https://community.zeek.org/t/extracted-files-dont-rotate/4644/5 "2017-02-09T03:15:54Z")

</div>

This has been a bit of a sticking point for quite a while. Part of the issue is the diversity in how clusters are run and managed. It's hard to create one solution which works for everyone's deployment.

I've been hoping to spend some time rejiggering how file extraction happens a little bit this year but I'd be glad to see anyone else beat me to it. It's a deceptively sneaky issue.

.Seth

---

<div class="post-metadata">

**Author:** ![James\_inthe\_box](https://avatars.discourse-cdn.com/v4/letter/j/6f9a4e/32.png) [@James\_inthe\_box](https://community.zeek.org/u/James_inthe_box)\
**Post date:** [February 9, 2017, 1:09pm UTC](https://community.zeek.org/t/extracted-files-dont-rotate/4644/6 "2017-02-09T13:09:48Z")

</div>

Appreciate the feedback all.

James

---

<div class="post-metadata">

**Author:** ![system](https://canada1.discourse-cdn.com/flex011/uploads/zeek/original/1X/f09d732bc2cc7c7cc7e35db67cf4e1d5233ce7a7.png) [@system](https://community.zeek.org/u/system)\
**Post date:** [May 6, 2022, 3:44pm UTC](https://community.zeek.org/t/extracted-files-dont-rotate/4644/7 "2022-05-06T15:44:35Z")

</div>


