While working through the file-extraction demo posted by @hectaman (http://www.youtube.com/watch?v=-7p3yLHxug4), I noticed my http-item_* files would go missing whenever I stopped the bro processes. It looks like files are properly written to bro/spool/bro, but not rotated or archived.
I’m digging through the scripts in bro/share/broctl/scripts and trying to identify the process for log rotation, but figured someone may have already solved this if they’ve seen similar behavior.
Thanks,
Carl
This e-mail and any attachments are confidential, may contain legal,
professional or other privileged information, and are intended solely for the
addressee. If you are not the intended recipient, do not use the information
in this e-mail in any way, delete this e-mail and notify the sender. -EXCIP
Ah! Now your twitter posts make sense. Unfortunately we don't support file extraction very well when run with BroControl. *Technically* we should be writing them out to some directory other than the spool directory, but honestly I'm not ever sure how this might interact with log rotation (although log rotation for non-logging framework files should be disabled anyway).
This is an area that you might have a bit of a hard time getting anyone to focus on right now because we're going to be ripping out most of the code that Liam pointed out in his video in the coming weeks and replacing the functionality with the in-development file analysis framework.
Probably not a very satisfying answer for you right now, but it is what it is. 
If you tell us more about what you're trying to accomplish we may be able to figure out some easy way for you to get it working though.
.Seth
For the time being, I extract the files into a different directory - extracted files will be written to these folders and would persist restarts.
in your local.bro:
redef SMTP::extraction_prefix = "/data/bro/smtp-extract/smtp-entity" &redef;
redef HTTP::extraction_prefix = "/data/bro/http-extract/http_item" &redef;
Needless to mention, you need to create the two directories smtp-extract and http-extract.
You should be grep on the extracted filename in the rotated log files to get more detailed information about the connection etc.
Hope this helps,
Aashish