'async' keyword

At BroCon a few folks asked me about the proposed "async" keyword for
Bro's scripting language. "async" is coroutine-style language
construct that puts blocking operations on hold until they conclude,
working on other stuff first. It could replace most uses of "when" and
is arguably much nicer to use.

"async" is implemented as a proof-of-concept in the topic/robin/async
branch. Note that that Bro branch is not fully functional at the
moment, nor are performance implications clear. However, before going
any further with it I'd like to reach consensus if the current
implementation is acceptable for the Bro code base, as it's very
low-level and not easy to follow. I'd be interested in opinions here.

The commit to look at is:

The exercise here is: Can you understand how "async" works? (If you
can honestly answer "yes" in under 15 minutes, I buy you a beer. :wink:

Robin

PS: See the TODOs in that commit for the current state of the code.)

A feat like that deserves a larger reward.

Understanding the new code also requires understanding the context in which it is implemented and I wonder if the later is more of a hurdle here. Maybe it’s understandable for the exercise to take on the order of days/weeks for a person that’s not done much hacking/exploring of the bro internals? Just trying to give a sane reference point so anyone taking the challenge isn't discouraged after only 15 mins :slight_smile:

- Jon

Hey, this is bro-dev, are you saying not everybody here is intimately
familiar with the Bro source code? :wink:

So yes, I'll allow for more time to understand the context. :slight_smile: The
point though is that conceptually it's both simple and difficult at
the same time. Even without all the context, one might get the basic
idea pretty quickly if looking at the right parts---or maybe one
doesn't, I don't know, that's part of why I keep looking for feedback.

Robin

What are your thoughts on error handling?

Exec::run() returns an Exec::Result, which is nice in that we can recover if something goes wrong. I would think one would want most calls of Exec::run() in an async context, but we lose the return value.

…alan

It looks like this wouldn’t be lost at all, you would just go from

    when (local res = Exec::run([$cmd="…"])){
        do_something_with(res);
    }

to

    local res = async Exec::run([$cmd="…"]);
    do_something_with(res);

Good question, two parts to that.

First, along the lines of Justin's response, the function is free to
return whatever it considers the appropiate result for an error case.
The only constraint is that it *has* to return something because
there's no way to get back without passing on a result.

Second, one thing that "async" doesn't have right now is an equivalent
of "when"'s "timeout", i.e., alternative code to execute if the
asynchronous code takes too long. I'm not sure if we could support
that with "async" because we'd need an alternative way to leave the
calling function and could run into an issue with memory leaks (not
completely sure here, but it's smells similar to the problem we have
with propagating interpreter exceptions up the stack). That said, I'm
also not sure how a timeout would even fit in syntactically, so maybe
the nest answer here is just sticking to "when" if one needs a
timeout.

Robin