Draft API for new communication library

I have been mulling over how an API for a new communication library
could look like. In short, the idea is to (1) overhaul Bro's current
communication model to make it more flexible and easier to control;
and (2) provide the new functionality in the form of a C library that
replaces Broccoli yet will also be used by Bro itself (i.e., we;ll no
longer have two independent implementations of the same protocol to
maintain).

Draft is here:

    http://www.bro.org/development/projects/comm-ng-v2.html

Feedback welcome.

Robin

   http://www.bro.org/development/projects/comm-ng-v2.html

Feedback welcome.

Would something like Cap'n Proto or Protocol Buffers help in defining/maintaining a serialization format? Never used either, just had a good impression from skimming their docs.

- Jon

http://kentonv.github.io/capnproto/

https://code.google.com/p/protobuf-c/

I didn't know Cap’n Proto so far but I have been wondering about using
Protocol Buffers already as well. We'd have to add another dependency
but it would make this stuff quite a bit less cumbersome. Do you know
if their C version is well maintained? It looks rather old compared to
the standard protobuf distribution. Does Cap'n Proto have a C API?

Robin

FWIW, nanopb is a lightweight (read: targeted at embedded devices) C library that is compatible with protobuf.

http://koti.kapsi.fi/~jpa/nanopb/

--Gilbert

Nanopb - protocol buffers with small code size

Nice, that looks like an interesting option.

Btw, another advantage of using something like PB is that it would
make the serialization more portable. I believe this will be one part
where we'll have to keep separate implementastions for Bro and
external clients (because in Bro everythingn is already stored in Val,
and if we'd use an additional C interface we'd have to convert
everything over first; might just as well serialize it directly, but
then having a "middle layer" like PB would certainly be helpful).

Another alternative would be reusing the code that logging and input
framework currently deploy for reading/writing Vals. That was my
initial thought, but it limits us to types they support (essentially
things which can be easily represented in ASCII); probably not ideal.

On the other hand, maybe we could switch logging/input over to using
the new implementation we come up with here instead. It would
certainly be nice to unify the whole serialization code in one place.

Robin