1) I see that stores-listener.bro has clone created into it and store-connector.bro has master in it.
Does that mean the idea is to have workers run listener and manager run connector ? Which fundamentally means manager connects to the workers ?
Or this is open to 'case-by-case' basis ?
You’re free to chose which endpoints do listen and which do connect and there’s no restrictions related to messaging patterns or data stores related to which one is chosen for a given endpoint.
2) What exactly does "bro/event/ready" mean ? Is idea here to compartmentalize various events for various policies ?
something like bro/event/tor-ban/balh ?
It’s an arbitrary choice of a topic string that other endpoints may chose to subscribe to. It doesn’t have to be in a hierarchical/directory-like format, but that model probably works nice for things you’d want to do. For example you could have an endpoint subscribe to the prefix “bro/event” and it will receive all events that peers publish that use that prefix (and are willing to send to others). Or it could use “bro/event/http” to get all http related events, or it can match a topic string exactly if it only cares about just that one event, etc. The degree of specificity is left up to the subscriber.
2b) Is it right to understand that with auto_event the event will be automatically called on workers if called on manager ?
For the most part, yes. There’s other conditions that matter like if the manager allowed the event to be published (the default is “yes”) and if the worker subscribed to the event (they need to explicitly set up the subscriptions they are interested in).
2c) How do I trigger a clone to update the master (how often or can I trigger updates on certain conditions ? )
A clone attempts to be as close a copy of the master data store as it can. Any modifications done to the master or via the clone will automatically trigger updates to be propagated to all clones and you don’t have control over that.
The primary goal of the clone is to keep a local copy of the master data store so that queries can use that local cache and maybe eliminate some latency. If that’s not a concern for your use-case, you could just create a plain “frontend” to the master data store instead of a clone. With a plain frontend, queries always are made against the master store, there’s no local cache.
3) Since all the action happens in "event BrokerComm::outgoing_connection_established" I don't see way to pass data to it.
Do I need to create global variables and then use them in this event ? I mean whats a good way to "pass"/use data to this event ?
You might keep the store handle as a global variable and initialize it in bro_init(), but typically I don’t think you’d actually want to do any data store operations in that event. You can do the operations inside any event depending on what you want the data store to actually do, e.g. handle http_request instead and do store operations there.
3b) How is BrokerComm::outgoing_connection_established event triggered ?
It’s raised automatically when a connection to a peer has completed. If you start trying to send messages before you’ve seen the connection has been established, there’s no guarantee the other side will actually see them.
Does using BrokerStore::insert in some other event also trigger the updates to master from the clone ?
Yes, you can perform data store operations inside any event and modifications to a master store are automatically propagated to all clones of it.
4) Somewhat whimsical issue: Why is peer_address of string type when we have peer_port as port data type. Shouldn't peer_address be address data type ? I was hoping may be one can use dns-names thats why but I cannot seem to get that working ?
Yeah, the idea was that either names or IPs should work and a string is a good way to represent either. Can you give more detail on what’s not working?
4b) Shouldn't this event be better off as : event BrokerComm::outgoing_connection_established(p: peer)
Possibly, but the available information for incoming/outgoing and established/broken is different enough that providing a single “peer” record would make it hard to tell what each one actually can provide.
Oh also, I see that it supports sets but seems like doesn't support tables ?
Tables are a supported data type, but broker data stores don’t support any in-place modification operations on them. Maybe if you are wanting to do a lot of modifications (that aren’t simply whole value replacements) to a table stored inside a broker data store, it’s worth considering mapping the keys in the table to keys in the data store instead.
- Jon