Real-time reporting from multiple sensors to multiple analysis points

Zeek
1

Dear bro mailing list,

I have a question regarding the configuration of bro and its real-time reporting features.

Right now, I have two sensors (s1, s2), each running one bro node with log files rotating every hour. After the rotation, I send the files from each sensor to an analysis point (a1) via scp and perform my analysis steps.

My requirements changed now: I want to know what happens on the sensors in almost real-time. How do I send reports from (s1,s2) with a max. delay of 10 seconds to another analysis point (a2)? The reports can still reach (a1) every hour to keep the load low. My intuition tells me, that a very low rotation interval and scp are not the best practice here.

Regards,

Marcin Nawrocki

2

Based on your requirements you probably want to use something like the bro kafka log writer, or a process running on each system like logstash that can forward logs.

3

You can also have a look at lsyncd (https://github.com/axkibe/lsyncd) which is a synching daemon that uses rsync in the background. By default, lsyncd triggers copying when the file is closed but you can change this behavior by modifying the inotifyMode option - https://axkibe.github.io/lsyncd/manual/config/file/