using broccoli to send events to bro

Zeek
1

I've got a .bro file that looks like:

module A;
global f: file = open("wtf.txt");
redef Communication::nodes += {
        ["test"] = [$host = 127.0.0.1, $events = /test1/],
};
event test1(a: string)
    {
    print f, "got here";
    }

and a python script that looks like:
#! /usr/bin/python

from broccoli import *

bc = Connection("127.0.0.1:47761")
bc.send("test1", "aaaaaaaaaaaaaaaa")
bc.processInput();
print "done"

I get the following log messages:
1379985413.067179 manager child - - - info
[#10005/127.0.0.1:34609] accepted clear connection
1379985413.068412 manager parent - - - info
[#10005/127.0.0.1:34609] added peer
1379985413.068412 manager parent - - - info
[#10005/127.0.0.1:34609] peer connected
1379985413.068412 manager parent - - - info
[#10005/127.0.0.1:34609] phase: version
1379985413.068412 manager script - - - info
connection established
1379985413.068412 manager script - - - info
requesting events matching /^?(test1)$?/
1379985413.068412 manager script - - - info
accepting state
1379985413.069943 manager parent - - - info
[#10005/127.0.0.1:34609] phase: handshake
1379985413.270825 manager parent - - - info
[#10005/127.0.0.1:34609] peer does not support 64bit PIDs; using
compatibility mode
1379985413.270825 manager parent - - - info
[#10005/127.0.0.1:34609] peer is a Broccoli
1379985413.270825 manager parent - - - info
[#10005/127.0.0.1:34609] phase: running
1379985413.270825 manager script - - - info
connection closed
1379985413.272093 manager parent - - - info
[#10005/127.0.0.1:34609] peer disconnected
1379985413.273243 manager child - - - info
[#10005/127.0.0.1:34609] connection closed
1379985413.851595 worker-1 child - - -
info selects=3100000 canwrites=0 timeouts=3098508
1379985416.921436 manager child - - - info
selects=3200000 canwrites=0 timeouts=3198460
1379985411.837037 proxy-1 child - - - info
selects=3100000 canwrites=0 timeouts=30999

But nothing gets persisted to the "wtf.txt" file. I'm sure (I hope)
I'm missing something super easy. Any ideas?

Thanks!

2

processInput() doesn't guarantee that any events are actually processed. It returns True if the send-queue is non-empty so calling it in a loop to make sure the event is sent is probably what you want:

    while bc.processInput():
        pass # or sleep or do other stuff

- Jon

3

Thanks that's good to know. I changed it to

while bc.processInput():
    sleep(2)

and it still seems to exit immediately.

Any additional thoughts? I'm pretty lost on this one. I'm using 2.1
with the broccoli.py included with it.

Thanks again.

4

Any additional thoughts? I'm pretty lost on this one. I'm using 2.1
with the broccoli.py included with it.

If output to the "wtf.txt" file is buffered, you probably aren't going to see anything in there right away. Maybe not even until you terminate the bro process since there's so little data. You can put a regular print statement to stdout in the event handler in your bro script to verify you actually get events, but nothing has yet been written to disk. You could also have your python script send a whole bunch of events and hope you actually cause output to be flushed.

- Jon

5

-1 for me. It was a buffering problem (sending 1000 events worked quite well).

Thanks for the assistance, I feel "special".

-=Mike

6

Actually I'm not sure if processInput() returns true if there's
something in *send* queue (as opposed to the receive queue). There
might still be a race condition there. Does it work with an infite
loop:

    while True:
        bc.processInput():
        sleep(1)

Often the most reliable way to get such stuff working is sending an
acknowledge event back and only terminating once that has been
received.

Robin

7

Actually I'm not sure if processInput() returns true if there's
something in *send* queue (as opposed to the receive queue).

If it doesn't, there's a couple code comments in broccoli/bindings that need fixing :slight_smile:

- Jon

8

Actually I'm not sure if processInput() returns true if there's
something in *send* queue (as opposed to the receive queue).

If it doesn't, there's a couple code comments in broccoli/bindings that need fixing :slight_smile:

That was my confusion exactly.

Regardless it seems to be working, I'm less concerned with termination
of the python script and I'm entirely ok with waiting a while. I just
wasn't getting any output and was confused.

Thanks again.