http.log reorder and skip fields, how?

Hello All. It appears that the data in http.log is a listing of the Info fields which have the &log attribute. I can see how to add fields by redefining record Info using the += syntax. However, I want to customize the output by removing some fields and reordering others. What is the proper way to do this? Can the field separator be adjusted? I don’t want to actually “remove” fields, I just don’t want some of them displayed. I also didn’t want to parse the bro output with a shell script to reformat it, I would like to have bro write the data out the way that I need it.

Thanks,
Dalton

Read this blog post:
  http://blog.bro-ids.org/2012/02/filtering-logs-with-bro.html

And this documentation:
  http://www.bro-ids.org/documentation/logging.html#filtering

If you still have questions I'd be glad to answer. I do agree that we are missing the ability to modify flags on record fields though. I haven't been able to think of a good syntax for that though.

  .Seth

Seth, thanks for the info. I tried this:
event bro_init() &priority=5
{
Log::create_stream(HTTP::LOG, [$columns=Info, $ev=log_http]);
local filter: Log::Filter = [$name=“myfilt”, $path=“myfilt”, $include=set(“id.orig_h”,“id.resp_h”,“ts”)];
Log::add_filter(HTTP::LOG,filter);
}

But in the output file, the fields are ordered ts,orig,resp. Can I control the ordering?
Is there an easy way to change field separator?
Thank you.

You can't control ordering (sets aren't ordered either). If you need to change the order, you could do that by processing the logs through bro-cut like this:

  cat myfilt.log | bro-cut -f id.orig_h, id.resp_h, ts

Also, your code above should look like this…

event bro_init()
  {
  Log::add_filter(HTTP::LOG,[$name="myfilt", $path="myfilt", $include=set("id.orig_h","id.resp_h","ts")]);
  }

You shouldn't be redefining the stream. Keep in mind that this will still create the full http log since you aren't removing the default filter.

  .Seth