host field

Zeek
1

Is there a non-invasive way to rename the host field in bro log output?

2

In 2.5....

redef Log::default_field_name_map = {
  ["host"] = "something_else",
};

You can do this per-filter too, but this setting is a global default for all writers and filters.

.Seth

3

Ah shoot, but not in 2.4. Ok, thanks!

4

Seth, in 2.5 is this the way to make elastic happy, so you can rename ‘id.orig_h’ natively to whatever you want in Bro (minus the dots)?

5

Dot's were fixed in 2.4.0:

https://www.elastic.co/blog/elasticsearch-2-4-0-released

"You can disable the check which prohibits dots in field names by starting Elasticsearch as follows:

export ES_JAVA_OPTS="-Dmapper.allow_dots_in_name=true"
./bin/elasticsearch"

James

6

The way to make elasticsearch happy is probably this...
  redef Log::default_scope_sep = "_";

It changes all of the periods in field names to anything you want (underscore in this case).

  .Seth

7

Haha, I replied to the other email before reading the whole thread. I recommend this method instead!

  .Seth

8

As far as I know as I understand it, going this route in 2.4 and then later upgrading to 5.x may create conflict and cause unforeseen issues, as suggested here:

https://www.elastic.co/guide/en/elasticsearch/reference/current/dots-in-names.html

9

Thanks for the link. If I understood that right, the only scenario where that's a problem would be if we had a field named "id" and another field named "id.orig_h". Due to the way the logging framework functions, Bro will never write out logs that would have that issue so I think what this link describes isn't an issue for us.

  .Seth

10

OK, so. I absolutely must rename these fields, and I can not wait to deploy 2.5, and can not deploy beta. Does anyone know all the analyzers that have host explicitly defined in them so I can hack this manually?

There is absolutely no way that Splunk can keep up with json format, because it has to run a regex against every event processed to rename the host field. (The value host in splunk is reserved…). We can not do this with the bro json app, because that just puts us right back at square one with tsidx file size issues.

11

Hi Seth,

It works perfect ! I have the git version running with
elastic 2.4 (2.5 gave some trouble again) without
my nasty JSON.cc patch.

Regards,

Daniel