Broker cluster discovery and load balancing

I've been working on an application that will fire a large volume of events into bro through broker. Basically, I want to fire events into Broker and load balance them across a pool of workers without the need for more than one worker to handle the same event.

I'm having difficulty determining the most right way to accomplish a couple tasks.

I have looked at the implementations of publish_hrw and publish_rr in bro. I could easily implement those features in my application if that is the recommended way to handle this issue.

Unfortunately, that leaves me with another unfortunate problem. I have been unsuccessful in determining how to 'discover' members of a Bro cluster via Broker. Is there a way to do discovery, or do I need to know who the cluster members are and what port they are listening on via a broctl configuration equivalent?

I have looked at the implementations of publish_hrw and publish_rr in bro. I could easily implement those features in my application if that is the recommended way to handle this issue.

There's been some ideas on pushing loading balancing mechanisms into
broker itself, but I doubt that's something that will get done in the
near term. Explicitly implementing HRW or Round-Robin load balancing
in your application is definitely one way to handle things for now.

Unfortunately, that leaves me with another unfortunate problem. I have been unsuccessful in determining how to 'discover' members of a Bro cluster via Broker. Is there a way to do discovery, or do I need to know who the cluster members are and what port they are listening on via a broctl configuration equivalent?

At the moment, the cluster configuration is defined by the
Cluster::nodes table that gets auto-generated by broctl into
cluster-layout.bro. Does that table have all the info you need?

You still might need a well-known node to connect your broker
application (e.g. manager) to, but then you should be able to send
back the information you need from that table (e.g. here's the list of
worker ip/port pairs).

- Jon