Elastic Search

Seth,

I read on the security-onion list that Bro's Elastic Search writer doesn't write the correct metadata indexes for use with Kibana. Can you elaborate?

https://groups.google.com/forum/#!topic/security-onion/mmxZfVfO0D0

Previous versions of Kibana were designed to only work with Logstash data, which introduces certain assumptions about the way the data is stored. At one point, I had a fork of Kibana that worked with Bro, because it allowed extra configuration. I haven't updated it with the release of Kibana3, and don't intend to. Kibana3 might be flexible enough to support Bro's data, but I haven't checked yet.

If you are interested in using Kibana, I would advise you to be *very* careful about the risks introduced by allowing a user's browser to directly query ElasticSearch. ElasticSearch has no security protections, and all it takes is a curl -XDELETE es.local:9200 to delete all your log data. Kibana does come with a sample nginx config, so I would definitely look at that at least.

Also, I read in the comments of "elasticsearch.bro" that there is some sort of memory leak either in the elastic search writer or elastic search itself? Can you also elaborate on this?

http://www.bro-project.org/sphinx-git/_downloads/elasticsearch.bro

Bro queues undelivered log messages in memory. If Bro is logging faster than ElasticSearch can accept logs, more and more memory is used. There's no memory leak - if your ElasticSearch server can't keep up, the logs simply have nowhere to go, and instead of being dropped, they just queue up on the manager.

  --Vlad

Vlad,

Thank you for the quick response!

What is on the roadmap for Brownian?