Module prefix in sending and receiving Broker events

When I receive events from Bro via Broker, they have the prefix of the enclosing module:

    module Foo;

    event foo() { ... }
    event bar() { ... }

When I publish "foo" via Broker, it arrives as "Foo::foo". However, when I publish an event "Foo::bar" from Broker, Bro doesn't recognize it. I
must published it as "bar" - without the module prefix. Is this intentional?

    Matthias

Maybe not so much intentional, but expected at this point. Does the
suggestion [1] to always explicitly scope events by their
namespace/module address your problem? There's some longstanding
oddities [2] with the way events interact with module namespacing.

- Jon

[1] https://www.bro.org/sphinx-git/frameworks/broker.html#a-reminder-about-events-and-module-namespaces
[2] https://bro-tracker.atlassian.net/browse/BIT-71

Does the
suggestion [1] to always explicitly scope events by their
namespace/module address your problem?

That's what I thought would work, but it's the opposite: when I add
the module name as prefix, Bro silently ignores the event. I found a
surprising solution though: if I declare the event first, then it
works. Here's an example:

    module Foo;
    global foo: event();
    event foo() { ... }

This works as expected: when I create an event with name Foo::foo, the
handler gets dispatched. Without the declaration, it doesn't work.

As mentioned in your reference [1], explicit module qualification works as well:

    event Foo::foo() { ... }

I'm essentially running into the inverse of BIT-71, the ticket you referenced.

    Matthias

> Does the
> suggestion [1] to always explicitly scope events by their
> namespace/module address your problem?

That's what I thought would work, but it's the opposite: when I add
the module name as prefix, Bro silently ignores the event.

So [1] didn't work...

As mentioned in your reference [1], explicit module qualification works as well:

    event Foo::foo() { ... }

But [1] also worked? :slight_smile:

I'm essentially running into the inverse of BIT-71, the ticket you referenced.

Yeah, I think I see how it's the reverse of the original example, but
it's likely the same underlying module/namespacing ambiguities with
events, so I'd still suggest explicitly scoping events always and
everywhere (which was the [1] suggestion).

It might also help if you send actual examples that can be run if that
still doesn't work because it's hard to interpret what you mean by
"publish via Broker". That could mean Bro's Broker::publish() API or
the standalone Broker API itself and they are potentially different.

I found a
surprising solution though: if I declare the event first, then it
works. Here's an example:

    module Foo;
    global foo: event();
    event foo() { ... }

This works as expected: when I create an event with name Foo::foo, the
handler gets dispatched. Without the declaration, it doesn't work.

Sure, but that's also not the [1] suggestion either. Adding the
namespace scoping always and everywhere means to the event
declaration, the handlers, event/schedule dispatching, any strings
that contain the event name, etc.

- Jon

It might also help if you send actual examples that can be run if that
still doesn't work because it's hard to interpret what you mean by
"publish via Broker".

Okay, so first a pure Bro example. This one works:

    module Foo;

    global foo: event(); // declaration

    event foo()
      {
      print "foo";
      }

    event bro_init()
      {
      event Foo::foo();
      }

If I remove the event declaration, I get:

    error in ./foo.bro, line 10: not an event (Foo::foo())

If I remove the Foo:: qualification in bro_init *and* the declaration, I get the expected output again. If I don't remove the declaration, nothing gets printed.

Now with Broker, a working example:

    redef exit_only_after_terminate = T;

    module Foo;

    global foo: event(); // declaration

    event foo()
      {
      print "foo";
      }

    event Broker::peer_added(endpoint: Broker::EndpointInfo, msg: string)
      {
      Broker::publish("foo", Broker::make_event(foo));
      }

    event Broker::peer_lost(endpoint: Broker::EndpointInfo, msg: string)
      {
      terminate();
      }

    event bro_init()
      {
      Broker::subscribe("foo");
      Broker::peer("localhost", 55555/tcp);
      }

The Broker Python script:

    #!/usr/bin/env python

    import broker
    import broker.bro

    # Setup endpoint and subscribers.
    endpoint = broker.Endpoint()
    endpoint.listen("localhost", 55555)
    subscriber = endpoint.make_subscriber("foo")
    while True:
        topic, data = subscriber.get()
        print(data)
        endpoint.publish("foo", broker.bro.Event("Foo::foo"))

The Python script prints

    [<broker._broker.Count object at 0x10ac9f5e0>, <broker._broker.Count object at 0x10ace2880>, ['Foo::foo', []]]

and the Bro script "foo". So far so good. If I remove the event declaration, the Python script prints

    [<broker._broker.Count object at 0x10ac7c810>, <broker._broker.Count object at 0x10a04e8b8>, ['foo', []]]

and the Bro script nothing.

I hope this illustrates the issue a bit better.

    Matthias

If I remove the event declaration, I get:

     error in ./foo.bro, line 10: not an event (Foo::foo())

The following works for me:

     module Foo;

     event Foo::foo()
       {
       print "foo";
       }

     event bro_init()
       {
       event Foo::foo();
       }

Just as Jon suggested:

Adding the
namespace scoping always and everywhere means to the event
declaration, the handlers, event/schedule dispatching, any strings
that contain the event name, etc.

Event handlers just don't seem to inherit the surrounding module namespace, which kind of makes sense if you want to handle events generated in the global namespace.

Jan

Thanks, that helps. Underlying issue/pitfall still seems related to
module interactions with event handler definition/declaration, so
added a link back from [#163]. Namely, seems you can try to reason
about events + modules by trying to remember that event declarations
will implicitly get prefixed by module names while event handlers
don't (i.e. a handler for something that hasn't been declared yet
isn't considered an implicit declaration... or at least not the same
kind of declaration). It's not in the scope of Bro 2.6 to try
improving anything here, so I still suggest the simplest rule/approach
to remember is to always/everywhere prefix event names with an
intended module name.

- Jon

[#163] https://github.com/bro/bro/issues/163

Jan:

Event handlers just don't seem to inherit the surrounding module
namespace, which kind of makes sense if you want to handle events
generated in the global namespace.

I agree that it does make sense in that regard. It's certainly prudent
- as mentioned - to always qualify your events. The problematic part
that I see is that absence/presence of a forward declaration
interferes with module scoping. This subtlety took me a while to
figure out.

Jon:

[..] so I still suggest the simplest rule/approach
to remember is to always/everywhere prefix event names with an
intended module name.

Yeah, learned that the hard way. :wink:

    Matthias