Hi,
I’ve been working a while on the elasticsearch integration with bro.
There have been some issues like timestamp, the elstic 2.0 no dot
and the name/type changes in the logging (version …). See my changes
in https://github.com/danielguerra69/bro-debian-elasticsearch/blob/master/Dockerfile
It was made pragmatic, some changes where just a quick hack.
The latest release is stable.
https://hub.docker.com/r/danielguerra/bro-debian-elasticsearch/
Regards,
Daniel
Thanks for this Daniel…I’ve been looking at the new ES as well…seems like a large pain now…this will help me out.
James
Hi,
Diving deeper in the problem, beside the dot & timestamp, it can be solved with bro config andelastic mapping. I didn’t find the exact place where the dot is placed in the fieldnames, but I found
the point it was writing the json and changes JSON.cc (ugly but pragmatic). About the bro script
script structure there is a need for naming conventions and type. Like the version field which changes
type all the time (ssl ssh socks etc.). Check /scripts/bro-map.sh for geo_point and not analyzed fields (when
you let elastic index the data it cuts the results into words). In this script also the shards and copies are set.
Mapping needs to be done before writing.
There may be more updates later, but I just pushed a fix for the fields in the SIP log that were accidentally strings instead of counts (along with some other cleanup).
.Seth