# Broker::publish API **URL:** \ **Category:** Development\ **Tags:** development\ **Created:** [July 27, 2018, 5:39pm UTC](https://community.zeek.org/t/broker-publish-api/5394 "2018-07-27T17:39:29Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![robin](https://yyz1.discourse-cdn.com/flex011/user_avatar/community.zeek.org/robin/32/599_2.png) [@robin](https://community.zeek.org/u/robin)\ **Post date:** [July 27, 2018, 5:39pm UTC](https://community.zeek.org/t/broker-publish-api/5394/1 "2018-07-27T17:39:29Z")
The other day when merging Johanna's code to clusterize the configuration framework, I noticed this code in there:      # [Send id=val to everyone else]      Broker::publish(change\_topic, Config::cluster\_set\_option, ID, val, location);      if ( Cluster::local\_node\_type() != Cluster::MANAGER )          Broker::relay(change\_topic, change\_topic, Config::cluster\_set\_option, ID, val, location); It took me a bit to understand that ... The goal here is that a change in a configuration value gets propagated out to all nodes in the cluster. The Broker::publish() sends it to a node's immediate neighbors, but not further. That means that for workers it goes (only) to their manager; for the manager it means, it goes to all workers. If we're not a manager, we then separately (through Broker::relay()) ask our neighbors (that's the manager) to forward the change to \*their\* neighbors (that's the other workers), without reraising it locally. I remember we have discussed this API before, but I wanted to bring it up again as I keep finding it confusing. I believe the code above could be simplified by using the newer Broker::publish\_and\_relay(), which was added to combine the two operations. Still, I'm realizing now that I don't like thinking about this in terms of separate publishing and relaying operations. It all won't become easier once we add multi-hop routing to the mix (which is in the works). And on top of all that, we also have Cluster::publish\_rr, Cluster::publish\_hew, Cluster::relay\_rr, and Cluster::relay\_hew -- another set of separate publishing & relay options. I'm wondering if we should give it another try to simply this API while we still can (i.e., before 2.6 goes out). To me, the most intuitive publish operation is "send to topic T and propagate to everybody subscribed to that topic". I'd structure the API around that, making that the main publish function for that simply:     Broker::publish(topic, args); That would send to all neighbors, which then process locally and relay to their neighbors. Right now, that would propagate just across one hop but once we have multihop that'd start being broadcasted out broadly. To support the other use cases, we can then add modifiers & functions to tweak this default, e.g.:     - Give publish() another argument "relay: bool &default=T" to prevent       it from going beyond the immediate receiver. Or maybe instead:       "relay\_hops: int &default=-1" to specify the max number of hops       to relay across, with -1 meaning no limit. (I recall concerns       about loops being too easy to create; we could set the default       here to F/0 to default to no forwarding, although conceptually I       don't really like that 🙂     - Give publish() another argument "relay\_topic: string &default=""       to change the topic when relaying on the 1st hop.     - Give publish() another argument "process\_on\_relays: bool &default=T"       to change whether a relaying hop also sees the event locally.     - Add a second function publish\_pool() that has all the same       options, but receives a pool type instead of a topic (just an       enum: RR, HRW). What I'm not quite sure about is if some of these modifiers are better to leave for the receiver to specify (e.g., whether to raise events received on a given topic locally, or just forward). I think I can see that either way. Robin ---
**Author:** ![Jon\_Siwek](https://avatars.discourse-cdn.com/v4/letter/j/71c47a/32.png) [@Jon\_Siwek](https://community.zeek.org/u/Jon_Siwek)\ **Post date:** [July 27, 2018, 7:47pm UTC](https://community.zeek.org/t/broker-publish-api/5394/2 "2018-07-27T19:47:46Z")
> I'm wondering if we should give it another try to simply this API > while we still can (i.e., before 2.6 goes out). To me, the most > intuitive publish operation is "send to topic T and propagate to > everybody subscribed to that topic". I'd structure the API around > that, making that the main publish function for that simply: > >     Broker::publish(topic, args); > > That would send to all neighbors, which then process locally and relay > to their neighbors. Right now, that would propagate just across one > hop but once we have multihop that'd start being broadcasted out > broadly. Can you remind/clarify what's meant by "multihop" ? I thought: Broker already has manual multihop if you set up subscriptions on all relevant nodes on the path yourself. Bro doesn't use it right now. Broker does not yet have automatic multihop where subscriptions are globally flooded automatically. A difference between "manual multihop" and "automatic multihop" would be that in the later, some relaying nodes may not actually hold a subscription to the message they are relaying and so, in the case of Bro events, I think they would not process them locally. >     - Give publish() another argument "relay: bool &default=T" to prevent >       it from going beyond the immediate receiver. Or maybe instead: >       "relay\_hops: int &default=-1" to specify the max number of hops >       to relay across, with -1 meaning no limit. Going with the generalized approach of configurable number of hops per message via "relay\_hops" from the start would be better than finding out we need it later. Possibly a downside is now you need to store original hop limit in addition to current TTL in each message if you want to detect the "is 1st hop" condition for the "relay\_topic" option below. > . (I recall concerns >       about loops being too easy to create; we could set the default >       here to F/0 to default to no forwarding, although conceptually I >       don't really like that 🙂 It's maybe both a concern and a reality -- Bro clusters currently contain cycles (e.g. worker -\> manager -\> proxy -\> worker) >     - Give publish() another argument "relay\_topic: string &default="" >       to change the topic when relaying on the 1st hop. > >     - Give publish() another argument "process\_on\_relays: bool &default=T" >       to change whether a relaying hop also sees the event locally. Those seem fine to me. >     - Add a second function publish\_pool() that has all the same >       options, but receives a pool type instead of a topic (just an >       enum: RR, HRW). What's the goal of the enums instead of just publish\_hrw() and publish\_rr() ? Instead of the API being 2 functions, it then seems like 2 enums that are never used elsewhere + 1 function that now always branches internally. - Jon ---
**Author:** ![Azoff\_Justin\_S](https://avatars.discourse-cdn.com/v4/letter/a/dec6dc/32.png) [@Azoff\_Justin\_S](https://community.zeek.org/u/Azoff_Justin_S)\ **Post date:** [July 27, 2018, 8:48pm UTC](https://community.zeek.org/t/broker-publish-api/5394/3 "2018-07-27T20:48:21Z")
This would do weird things on workers, since they connect to both the manager and proxies. Worker 1 would send to it's neighbors [manager, proxy1, proxy2] but then those 3 nodes would relay to all of the other workers. The TTL would stop the propagation, but you'd still end up sending 3 copies of the same message to each worker. I do agree that there's room for a lot of simplification, for example a worker broadcasting a message efficiently to all other workers needs to do something like this from the docs:         Cluster::relay\_rr(Cluster::proxy\_pool, "example\_key",                           Cluster::worker\_topic, worker\_to\_workers,                           Cluster::node + " (via a proxy)"); But a lot of that could have defaults: Most use cases would want to relay through the default proxy pool Since round robin is in use, they key shouldn't matter. The round robin part itself is really an implementation detail for proxy load balancing and maybe not something that should be exposed in the API. Now that I think of it I'm not sure why one would ever use relay\_hrw over relay\_rr. Removing a lot of that gets close to what you are suggesting:     Cluster::relay(Cluster::worker\_topic, worker\_to\_workers, Cluster::node + " (via a proxy)"); which is I guess just     Cluster::relay(topic, args) like you said. ---
**Author:** ![Jon\_Siwek](https://avatars.discourse-cdn.com/v4/letter/j/71c47a/32.png) [@Jon\_Siwek](https://community.zeek.org/u/Jon_Siwek)\ **Post date:** [July 27, 2018, 10:10pm UTC](https://community.zeek.org/t/broker-publish-api/5394/4 "2018-07-27T22:10:40Z")
> I do agree that there's room for a lot of simplification, for example a worker broadcasting a message efficiently to all > other workers needs to do something like this from the docs: > >         Cluster::relay\_rr(Cluster::proxy\_pool, "example\_key", >                           Cluster::worker\_topic, worker\_to\_workers, >                           Cluster::node + " (via a proxy)"); > > But a lot of that could have defaults: > > Most use cases would want to relay through the default proxy pool > Since round robin is in use, they key shouldn't matter. At the moment, one could write their own wrapper function around that if they find it too verbose and always want to use certain defaults? > The round robin part itself is really an implementation detail for proxy load balancing and maybe not something that > should be exposed in the API. Now that I think of it I'm not sure why one would ever use relay\_hrw over relay\_rr. Theoretically, a more favorable load distribution that's consistent over time? e.g. if you do RR of the same messaging pattern from multiple nodes, you could have waves of "randomly" overlapping loads on the relayer-node since everyone is cycling through all the proxies at their own rate when choosing the relayer. With HRW, you'd stick with the same relayer over time and only change on outages, but everyone should have chosen their relayer in a uniformly distributed fashion. - Jon ---
**Author:** ![Azoff\_Justin\_S](https://avatars.discourse-cdn.com/v4/letter/a/dec6dc/32.png) [@Azoff\_Justin\_S](https://community.zeek.org/u/Azoff_Justin_S)\ **Post date:** [July 28, 2018, 12:30am UTC](https://community.zeek.org/t/broker-publish-api/5394/5 "2018-07-28T00:30:13Z")
> > I do agree that there's room for a lot of simplification, for example a worker broadcasting a message efficiently to all > > other workers needs to do something like this from the docs: > > > >        Cluster::relay\_rr(Cluster::proxy\_pool, "example\_key", > >                          Cluster::worker\_topic, worker\_to\_workers, > >                          Cluster::node + " (via a proxy)"); > > > > But a lot of that could have defaults: > > > > Most use cases would want to relay through the default proxy pool > > Since round robin is in use, they key shouldn't matter. > > At the moment, one could write their own wrapper function around that > if they find it too verbose and always want to use certain defaults? Yeah.. The wrapper would be trivial.. Should bro include it so that the API scripts use is simpler? > > The round robin part itself is really an implementation detail for proxy load balancing and maybe not something that > > should be exposed in the API. Now that I think of it I'm not sure why one would ever use relay\_hrw over relay\_rr. > > Theoretically, a more favorable load distribution that's consistent > over time? e.g. if you do RR of the same messaging pattern from > multiple nodes, you could have waves of "randomly" overlapping loads > on the relayer-node since everyone is cycling through all the proxies > at their own rate when choosing the relayer. With HRW, you'd stick > with the same relayer over time and only change on outages, but > everyone should have chosen their relayer in a uniformly distributed > fashion. > > - Jon I'd expect that round robin would give the most uniform load distribution, for N proxies each proxy would see 1/N relay messages, but I guess in general round robin isn't the best load balance mechanism since it doesn't take into account the responsiveness of each proxy. With some of the information CAF provides it may be possible to also support weighted round robin. That way if a proxy node doesn't die outright but starts having issues for one reason or another, relay\_rr could avoid sending it messages. ---
**Author:** ![robin](https://yyz1.discourse-cdn.com/flex011/user_avatar/community.zeek.org/robin/32/599_2.png) [@robin](https://community.zeek.org/u/robin)\ **Post date:** [July 30, 2018, 4:01pm UTC](https://community.zeek.org/t/broker-publish-api/5394/6 "2018-07-30T16:01:01Z")
> Broker does not yet have automatic multihop where subscriptions are > globally flooded automatically. Yep, that's what I meant: dynamic multihop where each node tracks what its peers are subscribing to, and forwards messages independent of its own subscriptions. > Possibly a downside is now you need to store original hop limit in > addition to current TTL in each message if you want to detect the "is > 1st hop" condition for the "relay\_topic" option below. Yeah, that's right. Actually I think ideally the 1st hop wouldn't have any special role anyways if we didn't need that "relay\_topic". > It's maybe both a concern and a reality -- Bro clusters currently > contain cycles (e.g. worker -\> manager -\> proxy -\> worker) True, although it's not cycles in the connection topology that matter, it's cycles in topic subscriptions. I need to think about this a bit more (and I need to remind myself how our topics currently look like) but could we set up topics so that even in a cluster, messages don't go into a cycle? Is there a summary somewhere of what events & topics the cluster nodes are currently exchanging? > \> - Add a second function publish\_pool() that has all the same > \> options, but receives a pool type instead of a topic (just an > \> enum: RR, HRW). > > What's the goal of the enums instead of just publish\_hrw() and publish\_rr() ? Similar to what Justin wrote, it would more directly express the intent, with less emphasis on the mechanism; we could set a default to whatever we recommend people normally use; and it'd be more extensible. > At the moment, one could write their own wrapper function around that > if they find it too verbose and always want to use certain defaults? They could, but my general point is that it'd be nice to have a simple API that covers the most common uses cases directly and intuitively. Then let people change defaults if they have to and know what they are doing. Robin ---
**Author:** ![Jon\_Siwek](https://avatars.discourse-cdn.com/v4/letter/j/71c47a/32.png) [@Jon\_Siwek](https://community.zeek.org/u/Jon_Siwek)\ **Post date:** [July 30, 2018, 4:01pm UTC](https://community.zeek.org/t/broker-publish-api/5394/7 "2018-07-30T16:01:51Z")
Maybe. We can see how it fits in the mix of what Robin suggested:   # Supports variadic args in place of Broker::Event.   Broker::publish(topic: string, args: Broker::Event, relay\_topic: string &default="", process\_on\_relayer: bool &default=F)   # Supports variadic args in place of Broker::Event.   Cluster::publish(pool: Cluster::pool, key: any, strategy: enum, args: Broker::Event, relay\_topic: string &default="", process\_on\_relayer: bool &default=F)   # Supports variadic args in place of Broker::Event. Use proxy pool and RR method w/ arbitrary, internal key by default.   Cluster::publish\_via\_proxy(relay\_topic: string, args: Broker::Event) That last one being the wrapper you're asking for. Also, I compressed the ideas of having a separate "relay: bool" / "relay\_hops: int" and "relay\_topic: string" args -- a non-empty relay topic implicitly means you want to enable relaying on the receiving node. Thinking more about original idea of giving the number of relay hops: it may be better to leave that until Broker multihop is more robust and allow it's automatic forwarding mechanisms to take care of those scenarios whereas a "relay" is a simple mechanism at the Bro application level (has it's own unique message format) that serves to aid load-balancing use-cases (rather than routing use-cases). - Jon ---
**Author:** ![Jon\_Siwek](https://avatars.discourse-cdn.com/v4/letter/j/71c47a/32.png) [@Jon\_Siwek](https://community.zeek.org/u/Jon_Siwek)\ **Post date:** [July 30, 2018, 4:15pm UTC](https://community.zeek.org/t/broker-publish-api/5394/8 "2018-07-30T16:15:13Z")
> True, although it's not cycles in the connection topology that matter, > it's cycles in topic subscriptions. Right, good point. > I need to think about this a bit > more (and I need to remind myself how our topics currently look like) I think we just have the "broadcast\_topic" to which all nodes subscribe, but not sure if there's more. > but could we set up topics so that even in a cluster, messages don't > go into a cycle? I don't see why not, but it takes planning and prudence on everyone's part (including users) to not break that rule. I'd be more comfortable if one could automate answering the question: "if I add a subscription to a given node in the network, will I create a cycle?". - Jon ---
**Author:** ![robin](https://yyz1.discourse-cdn.com/flex011/user_avatar/community.zeek.org/robin/32/599_2.png) [@robin](https://community.zeek.org/u/robin)\ **Post date:** [July 30, 2018, 5:16pm UTC](https://community.zeek.org/t/broker-publish-api/5394/9 "2018-07-30T17:16:21Z")
> I don't see why not, but it takes planning and prudence on everyone's > part (including users) to not break that rule. Yeah, question is we can pre-configure the cluster so that user's don't need to worry about it most of the time. > I'd be more comfortable if one could automate answering the question: > "if I add a subscription to a given node in the network, will I create > a cycle?". Hmm ... What about a test mode where we'd spin up a dummy cluster (similar to what the bests do), have each node send a message to all subscribed topics, and watch for TTL violations? Robin ---
**Author:** ![Jon\_Siwek](https://avatars.discourse-cdn.com/v4/letter/j/71c47a/32.png) [@Jon\_Siwek](https://community.zeek.org/u/Jon_Siwek)\ **Post date:** [July 30, 2018, 6:30pm UTC](https://community.zeek.org/t/broker-publish-api/5394/10 "2018-07-30T18:30:30Z")
Seems clunky and could get dicey -- subscriptions that transient/dynamic may not be well-tested and you'd probably want to guarantee that sending such a dummy message actually does not result in any side-effects at the Bro-layer. If nodes start raising random events at unusual/unintended times I start to doubt the stability of things. - Jon ---
**Author:** ![robin](https://yyz1.discourse-cdn.com/flex011/user_avatar/community.zeek.org/robin/32/599_2.png) [@robin](https://community.zeek.org/u/robin)\ **Post date:** [July 30, 2018, 6:35pm UTC](https://community.zeek.org/t/broker-publish-api/5394/11 "2018-07-30T18:35:26Z")
Agreed. 🙂 It'd just be a heuristic to catch some obvious errors. I don't think there's more we can do, we can't really catch loops statically by looking at the code. Robin ---
**Author:** ![robin](https://yyz1.discourse-cdn.com/flex011/user_avatar/community.zeek.org/robin/32/599_2.png) [@robin](https://community.zeek.org/u/robin)\ **Post date:** [August 3, 2018, 5:21pm UTC](https://community.zeek.org/t/broker-publish-api/5394/12 "2018-08-03T17:21:34Z")
Can somebody remind me what the use-case is for changing the topic on relay? Grepping over our standard scripts, I see only one use of relay(), and that's the one above. Robin ---
**Author:** ![Jon\_Siwek](https://avatars.discourse-cdn.com/v4/letter/j/71c47a/32.png) [@Jon\_Siwek](https://community.zeek.org/u/Jon_Siwek)\ **Post date:** [August 3, 2018, 8:57pm UTC](https://community.zeek.org/t/broker-publish-api/5394/13 "2018-08-03T20:57:07Z")
Another use is hidden within Cluster::relay\_rr(): event Intel::new\_item(item: Item) &priority=5     {     if ( Cluster::proxy\_pool$alive\_count == 0 )         Broker::publish(indicator\_topic, Intel::insert\_indicator, item);     else         Cluster::relay\_rr(Cluster::proxy\_pool, "Intel::new\_item\_relay\_rr",                           indicator\_topic, Intel::insert\_indicator, item);     } That is, if the manager is currently connected to some proxy, it picks one to do the work of distributing the event to workers. Manager sends 1 message instead of N. I don't know if there's currently other use-cases for Broker::relay specifically, but Cluster::relay\_rr/Cluster::relay\_hrw is essentially an extension of that which just also does the work of choosing the initial topic based upon a given pool and partition strategy. Might have been Justin who originally pointed out potential for avoiding manager overload in this way. - Jon ---
**Author:** ![robin](https://yyz1.discourse-cdn.com/flex011/user_avatar/community.zeek.org/robin/32/599_2.png) [@robin](https://community.zeek.org/u/robin)\ **Post date:** [August 6, 2018, 6:57pm UTC](https://community.zeek.org/t/broker-publish-api/5394/14 "2018-08-06T18:57:32Z")
Yeah, though at least from an API perspective this is different: The caller gives relay\_rr() only one topic to send to (indicator\_topic). It's then using a different topic internally to get it over to the proxy first, but that feels more like an implementation detail. So in that sense I would argue that this is not a use-case for the Broker API letting users change the topic on relay. (I'm not saying that that capability can't be useful, I'm just still looking for actual use cases.) I have another question about this specific case: we use relay\_rr() only for sending Intel::insert\_indicator. Intel::remove\_indicator gets published normally through auto\_publish(). Why the difference? Robin ---
**Author:** ![robin](https://yyz1.discourse-cdn.com/flex011/user_avatar/community.zeek.org/robin/32/599_2.png) [@robin](https://community.zeek.org/u/robin)\ **Post date:** [August 6, 2018, 7:50pm UTC](https://community.zeek.org/t/broker-publish-api/5394/15 "2018-08-06T19:50:11Z")
So I went through the exercise of collecting this information: what connections do we have between nodes, who's subscribing to what, and who's publishing what; see the attached PDF. This is based on all the standard scripts, with some special cases ignored (like the control framework). I'm not fully sure yet what to conclude from this, but a few quick observations:     - The main topics are bro/cluster/\ and       bro/cluster/node/\. For these we wouldn't have a problem       with loops if we enabled automatic, topic-driven forwading as       far as I can see.     - bro/cluster/broadcast seems to be the main case with a looping       problem, because everybody subscribes to it. It's hardly used       though. (bro/config/change is used similarly though).     - Relaying is hardly used.     - There are a couple of script-specific topics where I'm wondering       if these could switch to using bro/cluster/\ instead       (bro/intel/\*, bro/irc/dcc\_transfer\_update). In other words: when       clusterizing scripts, prefer not to introduce new topics.     - There's a lot of checks in publishing code of the type "if I am       (not) of node type X".     - Pools are used for two different things: 1. the known-\* scripts       pick a proxy to process and log the information; whereas 2. the       Intel scripts pick a proxy just as a relay to broadcast stuff       out, reducing load. That 1st application is a good, but the 2nd       feels like should be handled differently. Need to mull over this more, thoughts welcome. Overall I have to say I found it pretty hard to follow this all because we don't have much consistency right now in how scripts structure their communication. That's not surprising, given that we're just starting to use all this, but it suggests that we have room for improvement in our abstractions. 🙂 Robin [Broker Communication.pdf](https://community.zeek.org/uploads/short-url/jaHZe2yOcBqnCLLfSgmXAHeuNSh.pdf) (31.9 KB) ---
**Author:** ![Jan](https://avatars.discourse-cdn.com/v4/letter/j/ce7236/32.png) [@Jan](https://community.zeek.org/u/Jan)\ **Post date:** [August 7, 2018, 10:05am UTC](https://community.zeek.org/t/broker-publish-api/5394/16 "2018-08-07T10:05:53Z")
To be honest, I have somehow lost track of the discussion. What I can recall, it's about simplifying the API in the light of multi-hop routing, which is not fully functional yet. Regarding multi-hop routing I am even not sure what the actual goal is that we are currently aiming at. However, from a conceptual perspective I think "routing" either needs routing algorithms or strict conventions of how the network, to route messages through, is structured. So, what would a "deep cluster" look like and what kind of message flows do we expect in there? Some comments on the observations: >      - The main topics are bro/cluster/\ and >        bro/cluster/node/\. For these we wouldn't have a problem >        with loops if we enabled automatic, topic-driven forwading as >        far as I can see. How does forwarding work if I add another node type? Do we assume a certain cluster structure here? If yes: Is that a valid assumption? >      - bro/cluster/broadcast seems to be the main case with a looping >        problem, because everybody subscribes to it. It's hardly used >        though. (bro/config/change is used similarly though). The topic-concept is a multicast scheme, isn't it? Having a broadcast functionality on top of that feels odd. However, it's limited to the cluster topic. This leads me to the question which domains do we operate on? If I think of messages, I start to think about a cluster but that might be only one domain of application. I think it would be good to define layers of abstraction more precise here. >      - There are a couple of script-specific topics where I'm wondering >        if these could switch to using bro/cluster/\ instead >        (bro/intel/\*, bro/irc/dcc\_transfer\_update). In other words: when >        clusterizing scripts, prefer not to introduce new topics. communication patterns. What's the point of having topics if we don't use them? >      - There's a lot of checks in publishing code of the type "if I am >        (not) of node type X". That's something I would have expected. I don't think this is necessarily an indicator of bad design. Having these kind of checks means that roles are somehow fixed and responsibilities are explicitly codified. >      - Pools are used for two different things: 1. the known-\* scripts >        pick a proxy to process and log the information; whereas 2. the >        Intel scripts pick a proxy just as a relay to broadcast stuff >        out, reducing load. That 1st application is a good, but the 2nd >        feels like should be handled differently. I think we should be careful about introducing too much abstractions. Communication patterns tend to be complex and the more of the complexity is hidden, the easier it will be to generate misunderstandings. For example, in case of the intel framework, proxy nodes might be able to implement some more logic than just relaying at some point. Having the relay abstraction would mean to deal with two different levels of abstractions regarding intel on proxy nodes in this case. > Overall I have to say I found it pretty hard to follow this all > because we don't have much consistency right now in how scripts > structure their communication. That's not surprising, given that we're > just starting to use all this, but it suggests that we have room for > improvement in our abstractions. 🙂 I totally agree here! I think it could help to come up with some more use cases to identify the best abstractions. Jan ---
**Author:** ![Jon\_Siwek](https://avatars.discourse-cdn.com/v4/letter/j/71c47a/32.png) [@Jon\_Siwek](https://community.zeek.org/u/Jon_Siwek)\ **Post date:** [August 7, 2018, 6:32pm UTC](https://community.zeek.org/t/broker-publish-api/5394/17 "2018-08-07T18:32:25Z")
Potentially no reason other than no one reviewed whether it had potential to be optimized in a similar way. e.g. I first ported scripts in a direct fashion without trying to change too much structurally about comm. patterns or doing any optimization except in cases where a change was specifically talked about. I only recall Justin had called out Intel::insert\_indicator, so it got changed. - Jon ---
**Author:** ![Jon\_Siwek](https://avatars.discourse-cdn.com/v4/letter/j/71c47a/32.png) [@Jon\_Siwek](https://community.zeek.org/u/Jon_Siwek)\ **Post date:** [August 7, 2018, 6:39pm UTC](https://community.zeek.org/t/broker-publish-api/5394/18 "2018-08-07T18:39:03Z")
How much is due to new API usage and how much is due to things mainly being a direct port of old communication patterns (which I guess are written by various people over extended lengths of time and so there's inconsistencies to be expected) ? Or due to being a mishmash of both old and new? - Jon ---
**Author:** ![Azoff\_Justin\_S](https://avatars.discourse-cdn.com/v4/letter/a/dec6dc/32.png) [@Azoff\_Justin\_S](https://community.zeek.org/u/Azoff_Justin_S)\ **Post date:** [August 8, 2018, 2:20pm UTC](https://community.zeek.org/t/broker-publish-api/5394/19 "2018-08-08T14:20:31Z")
I think these 2 are somewhat related. Since there weren't higher level things like relaying, in order to relay a message from one worker to all other workers you had to jump through hoops with worker2manger and manager2worker events and often lots of @if stuff. There's also a bunch of places that I think were written standalone first and then updated to work on a cluster in place resulting in some awkwardness.. like notice/main.bro: function NOTICE(n: Notice::Info)     {     if ( Notice::is\_being\_suppressed(n) )         return;     @if ( Cluster::is\_enabled() )         if ( Cluster::local\_node\_type() == Cluster::MANAGER )             Notice::internal\_NOTICE(n);         else             {             n$peer\_name = n$peer\_descr = Cluster::node;             Broker::publish(Cluster::manager\_topic, Notice::cluster\_notice, n);             }     @else         Notice::internal\_NOTICE(n);     @endif     } event Notice::cluster\_notice(n: Notice::Info)     {     NOTICE(n);     } So on a worker, calling NOTICE publishes a cluster\_notice event that then re-calls NOTICE on the manager, which then does the right thing. You end up with a single small function with nested @if/if that works 3 different ways. But if this was written in a more 'cluster by default' way, it would just look like: function NOTICE(n: Notice::Info)     {     if ( Notice::is\_being\_suppressed(n) )         return;     n$peer\_name = n$peer\_descr = Cluster::node;     Broker::publish(Cluster::manager\_topic, Notice::cluster\_notice, n);     } event Notice::cluster\_notice(n: Notice::Info)     {     if ( Notice::is\_being\_suppressed(n) )         return;     Notice::internal\_NOTICE(n);     } Which other than the suppression check, has no branching at all. Broker::publish could possibly be optimized for standalone to raise the event directly if not being ran in a cluster. The only small downside is on a standalone you'd call is\_being\_suppressed twice, could always add a @if if you really wanted, but is\_being\_suppressed is just a set lookup. Then this stuff would be a good use for efficient relaying/broadcasting instead of making the manager do all the work:     Broker::auto\_publish(Cluster::worker\_topic, Notice::begin\_suppression);     Broker::auto\_publish(Cluster::proxy\_topic, Notice::begin\_suppression); ---
**Author:** ![robin](https://yyz1.discourse-cdn.com/flex011/user_avatar/community.zeek.org/robin/32/599_2.png) [@robin](https://community.zeek.org/u/robin)\ **Post date:** [August 8, 2018, 3:48pm UTC](https://community.zeek.org/t/broker-publish-api/5394/20 "2018-08-08T15:48:15Z")
> What I can recall, it's about simplifying the API in the light of > multi-hop routing, which is not fully functional yet. To level up a bit, what I'm hoping for is that we can find some easy ways to simplify the API a bit more now, with an eye towards dynamic multi-hop coming later. I don't know if it'll work out before 2.6 still, but changing the API later is more painful. We don't need to (or even can) solve multi-hop topologies right now, I think nobody really has the use cases clear in their heads yet. But if we could simplify the API a bit more for our current use cases in a way that may extend to multihop naturally later, that would probably save us some headaches at that point. > How does forwarding work if I add another node type? That's actually something I realized yesterday: we don't have direct worker-to-worker communication right now, correct? A worker cannot just publish to "bro/cluster/workers". > Do we assume a certain cluster structure here? If yes: Is that a valid > assumption? I think it's safe to assume we have the cluster structure under our own control; it's whatever we configure it to be. That's something that's easier to change later than the API itself. Said differently: we can always adjust the connections and topics that we set up by default; it's much harder to change how the publish() function works. > From my understanding this would mean going back to the old > communication patterns. What's the point of having topics if we don't > use them? Let me try to phrase it differently: If there's already a topic for a use case, it's better to use it. That's easier and less error-prone. So if, e.g., I want to send my script's data to all workers, publishing to bro/cluster/worker will do the job. And that will even automatically adapt if things get more complex later. For example, I can see having multiple otherwise independent cluster sharing a communication channel. In that case, we could internally change the topic to "bro/cluster/\/workers", and everybody using the predefined worker topic would still reach "their" workers without any further changes. > That's something I would have expected. I don't think this is > necessarily an indicator of bad design. Maybe it's a \*necessary\* design, but that doesn't make it nice. 😉 It makes it very hard to follow the logic; when reading through the scripts I got lost multiple times because some "@if I-am-a-manager" was somewhere half a page earlier, disabling the code I was currently looking at for most nodes. We probably can't totally avoid that, but the less the better. Robin [Next page](https://community.zeek.org/t/broker-publish-api/5394.md?page=2)