# Broker status update **URL:** \ **Category:** Development\ **Tags:** development\ **Created:** [May 21, 2016, 4:01pm UTC](https://community.zeek.org/t/broker-status-update/4166 "2016-05-21T16:01:32Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![Matthias\_Vallentin1](https://yyz1.discourse-cdn.com/flex011/user_avatar/community.zeek.org/matthias_vallentin1/32/596_2.png) [@Matthias\_Vallentin1](https://community.zeek.org/u/Matthias_Vallentin1)\ **Post date:** [May 21, 2016, 4:01pm UTC](https://community.zeek.org/t/broker-status-update/4166/1 "2016-05-21T16:01:32Z")
This is just a brief summary of where the Broker overhaul is currently at. The new Broker API will come two types of endpoints: blocking and nonblocking. The former provides a synchronous API to receive messages, whereas the latter operates fully asynchronous. Performance-wise, the asynchronous version has a major advantage because it does not introduce buffering. The callback provided upon endpoint construction will be immediately executed by the Broker runtime, for each message as it arrives. For a blocking endpoint, it's the users responsibility to extract messages, otherwise they queue up. Endpoints can only be constructed through a broker::context object:     // Encapsulates global states, such as announced types, thread pool,     // CAF scheduler, etc.     context ctx;     // Creates a synchronous endpoint.     auto be = ctx.spawn\();     auto msg = be.receive(); // Block and wait for next message.     be.receive(       [&](const topic&, const message&) {         // As above, but use a lambda to handle the topic-message pair.       }     ); // Block and wait for next message.     // Creates an asynchronous endpoint.     auto ne = ctx.spawn\(       [=](const topic&, const message&) {         // invoked for every subscribed message       }     ); Other than that, both brokers have a publish/subscribe interface, e.g.:     be.subscribe("foo");     be.subscribe("bar");     auto t = topic{"bar"};     auto msg = make\_message(42, "string");     be.publish(t, msg); I'll post more updates as the development progresses, but all of the above examples are now working. The TODO list currently includes:     - Peerings: managing subscriptions across multiple endpoints. Here's       an example:                // Make two endpoints.         auto x = ctx.spawn\<...\>(...);         auto y = ctx.spawn\<...\>(...);       I'm torn between two interfaces. This is the first:         peer(x, y); // Create a peering between the two endpoints.         peer(y, x); // Idempotent. Peerings are symmetric.         // Peer with a remote endpoint. Requires that the remote side         // called e.listen(addr, port) beforehand.         auto r = ctx.spawn\("1.2.3.4", 42000);         peer(y, r);         // Undo a peering.         unpeer(x, y);         unpeer(r, y);       The second is:         x.peer(y); // Create a peering between the two endpoints.         y.peer(x); // Idempotent. Peerings are symmetric.         x.peer("1.2.3.4", 42000); // Peer with a remote endpoint.         x.unpeer(y);         x.unpeer("1.2.3.4", 42000); // Unpeer from a remote endpoint,                                        must match previous peering data.       Personally, I'm favoring the first version, as the functional API       makes the symmetry of the peering relationship clearer.     - Data store: integrate the new "main API" with the existing data       stores. My plan is to use as much as possible from the existing       data store API.     - Bindings: For Python, I'm considering switching to pybind11 [1],       which provides a much more convenient API than SWIG and supports       modern C++11. Please chime in if you have questions/comment or see opportunities for improvement.     Matthias [1] [http://pybind11.readthedocs.io/en/latest/classes.html](http://pybind11.readthedocs.io/en/latest/classes.html) ---
**Author:** ![robin](https://yyz1.discourse-cdn.com/flex011/user_avatar/community.zeek.org/robin/32/599_2.png) [@robin](https://community.zeek.org/u/robin)\ **Post date:** [May 23, 2016, 3:23pm UTC](https://community.zeek.org/t/broker-status-update/4166/2 "2016-05-23T15:23:26Z")
>         peer(x, y); // Create a peering between the two endpoints. >         peer(y, x); // Idempotent. Peerings are symmetric. >         x.peer(y); // Create a peering between the two endpoints. >         y.peer(x); // Idempotent. Peerings are symmetric. I would prefer the 2nd way for consistenct, as all the other operations use the method-based scheme. The idempotency seems secondary to that I would say. Related question: what exactly are the semantics if only one side of the peering is set up? >     - Bindings: For Python, I'm considering switching to pybind11 [1], >       which provides a much more convenient API than SWIG and supports >       modern C++11. Hmm ... I see the appeal but it would introduce a new dependency and its Python-specific (I assume), whereas with SWIG it's easier to add more languages later. Is that worth the benefit of switching? Robin ---
**Author:** ![Matthias\_Vallentin1](https://yyz1.discourse-cdn.com/flex011/user_avatar/community.zeek.org/matthias_vallentin1/32/596_2.png) [@Matthias\_Vallentin1](https://community.zeek.org/u/Matthias_Vallentin1)\ **Post date:** [May 27, 2016, 5:35pm UTC](https://community.zeek.org/t/broker-status-update/4166/3 "2016-05-27T17:35:23Z")
> \> peer(x, y); // Create a peering between the two endpoints. > \> peer(y, x); // Idempotent. Peerings are symmetric. > > \> x.peer(y); // Create a peering between the two endpoints. > \> y.peer(x); // Idempotent. Peerings are symmetric. > > I would prefer the 2nd way for consistenct, as all the other > operations use the method-based scheme. The idempotency seems > secondary to that I would say. Okay, after mulling over this a bit more, I think the second version makes sense also for another reason: overloads. Both of these statements should be valid:     x.peer("127.0.0.1", 42000);     x.peer(y); With a free function, this becomes not as clear anymore. > Related question: what exactly are the semantics if only one side of > the peering is set up? The peering occurs asynchronously. The function endpoint::peer returns void, and when the peering result (failure/success) becomes available, a dedicated status message will be enqueued in the endpoint's mailbox. This message has to be extracted via endpoint::receive, or in the asynchronous case, there needs to be handler for the status present (otherwise the message gets dropped). > Hmm ... I see the appeal but it would introduce a new dependency and > its Python-specific (I assume), whereas with SWIG it's easier to add > more languages later. Is that worth the benefit of switching? We already need SWIG and its Python-specific bindings, which are two separate packages. Since every language has its idiosyncrasies, it's quite unwieldy to have one framework to rule them all. At least SWIG feels not like a productive toolkit for modern C++. Also, since pybind11 is a small header-only package, we can easily ship it directly with Broker, which would render the existing package dependencies obsolete. That is, we would not introducing a new dependency, but remove two instead ;-). That said, this is my impression after just briefly looking at the bindings space. I'll perform a more thorough evaluation once I'll actually start delving into writing the bindings.     Matthias ---
**Author:** ![system](https://canada1.discourse-cdn.com/flex011/uploads/zeek/original/1X/f09d732bc2cc7c7cc7e35db67cf4e1d5233ce7a7.png) [@system](https://community.zeek.org/u/system)\ **Post date:** [May 6, 2022, 3:43pm UTC](https://community.zeek.org/t/broker-status-update/4166/4 "2022-05-06T15:43:42Z")