Hello all,
I know that I can update bro parameters using the INPUT framework (reading input files and updating a table for instance).
The thing is that the INPUT framework (STREAM) and generally reading from files is relatively slow.
Can I add elements to a table inside bro from lets say a syslog message or any other faster method ?
thanks
B
I am also now reading about PYBROKER.
Can anyone provide me with an example of how can I use PYBROKER and a python script to update a table inside a running bro cluster (bro workers to be exact) ?
Thanks
B
Can I add elements to a table inside bro from lets say a syslog message or
any other faster method ?
There is a syslog analyzer you could theoretically use (
https://www.bro.org/sphinx/script-reference/proto-analyzers.html#bro-syslog)
but I would strongly discourage mixing monitored traffic and control
traffic. If you want to interact with Bro, broker might be of interest
for you
(https://www.bro.org/sphinx/components/broker/broker-manual.html). For
example, I have used broker to write a python script that allows to
delete intel items.
Jan
There is still ongoing work on the upcoming config framework which will enable dynamic reconfiguration of lots of parts of Bro. We built it within Corelight and an initial implementation has already been pushed into a branch in the Bro repository but it's going to be improved before being merged into master.
I would still recommend playing with pybroker if you're interested in it though. If you can solve a point solution more easily and quickly for yourself you shouldn't let upcoming yet imcomplete work stop you! 
.Seth
What exactly do you mean by relatively slow? How large are these tables that you are reading?
Indeed, I was also going to ask that. We did some performance measurements when we first wrote it - and it actually is quite fast. There only is a relatively low amount of components between the input reader and it storing things in a table; I cannot be 100% sure, but I doubt that other ingestion methods can be much faster. (I actually doubt that they will be faster at all).
Johanna
Hi and thank you for your answers !
By slow I mean that writing to a file on a remote machine will have network and IO (read and write) strains.
I suppose having something like ZeroMQ or some syslog messaging framework will be more efficient.
On my case, I have a file that is being updated with 3+ lines per sec (each line has 3 fields). This file is being mapped to a table (&create_expire=10min).
Upon a new connection I check if orig_h is in this table and assign a field accordingly.
I see that many orig_h’s are not recognized even though they exist in the file.
Seth, can you please address me to a branch that includes this reconfigurable bro framework ?
thanks again
B
What is the time difference between when the file is updated and the table is checked?
Well, its hard to provide you with this information
As a process, writing to a remote file and reading from that remote file into a bro table, it is not the most efficient way to perform such a task.
I do see events that have recognized their orig_h as part of the updated table, but they are very infrequent.
Thanks
Well, its hard to provide you with this information
Right but you are saying things are not efficient and slow, so I would expect that you have numbers to back up these statements.
How fast are you expecting this table to update? 1ms? 1s? 30s? 'slow' is relative, and you haven't said what your expectations are, just that things aren't working.
As a process, writing to a remote file and reading from that remote file into a bro table, it is not the most efficient way to perform such a task.
What remote file? Are you using NFS or something? Again you are saying things are not efficient, but you have not provided any information that proves this.
You said you are writing 3 lines per second and that each line has 3 fields. This means that you are writing maybe 300 bytes/second to a file. This is not an extreme amount of data and it's not really possible to be inefficient with 3 lines per second.