I was emailing with Justin this morning and he reminded me that we still have the issue with people that install Bro from binary packages and their inability to install packages with binary plugins. I don't know if I'll be able to kick start a brief discussion on this right now or not, but I'll throw a couple of ideas out there to see if any stick.
- Include header files with the binary package installs.
PROS
- Solves the problem. Package users could install plugins.
CONS
- "Feels" nasty since we don't have a fully formalized API and we'd just have to include all Bro header files.
- Requires users to have a compiler.
- Create mechanism to release binary plugins in the bro package manager.
PROS
- Super easy installation for users.
CONS
- Here be (library linking) dragons. I don't know how many we could avoid with static linking.
- We would probably have to create more project infrastructure to build packages.
Other thoughts?
.Seth
I was emailing with Justin this morning and he reminded me that we still have the issue with people that install Bro from binary packages and their inability to install packages with binary plugins.
Is there something that makes this currently impossible or just that the user would additionally need to download Bro source code and tell bro-pkg where to find it before installing a package where the build_command will build a plugin?
- Create mechanism to release binary plugins in the bro package manager.
I wonder if pre-built plugins can’t already be distributed in packages? E.g. put the pre-built plugin in the package, specify its location via plugin_dir, and leave the build_command blank. Then after the package is installed, I think Bro should automatically try to load the plugin, provided the architecture matches.
In other words, I think bro-pkg is already set up to handle distributing pre-built binary plugins, but the more difficult problems to address will relate to how to actually produce plugins that will be binary-compatible w/ a user’s Bro installation. E.g. lack of stable/standard ABI in Bro and C++ itself means there’s myriad ways to produce unportable binaries.
The only other idea I have, is if bro-pkg is not able to locate Bro source code (i.e. the user installed Bro from a binary package), then it could automatically download the appropriate version (e.g. clone Bro’s git repo and checkout a specific version tag).
- Jon
In other words, I think bro-pkg is already set up to handle distributing pre-built binary plugins, but the more difficult problems to address will relate to how to actually produce plugins that will be binary-compatible w/ a user’s Bro installation. E.g. lack of stable/standard ABI in Bro and C++ itself means there’s myriad ways to produce unportable binaries.
This, I think, is the big gotcha. Different linux distros, different compilers, different libraries. It even goes into which version of boost is used. There’s also matching to the version of Bro whether that’s master or release N.
I’m trying to come up with parallels with the Linux kernel. Loadable modules have vermagic and there’s DKMS. Neither fully covers the breadth of issues that can occur.
The only other idea I have, is if bro-pkg is not able to locate Bro source code (i.e. the user installed Bro from a binary package), then it could automatically download the appropriate version (e.g. clone Bro’s git repo and checkout a specific version tag).
From a deployment perspective, there really shouldn’t be a compiler on the system running a deployed Bro. Even further, the system might be limited on where on the internet it can reach. At least with many of our customers, that’s the case.
…alan
I imagine bro-pkg could still use its bundling system for when the Bro nodes have limited network/compiler functionality. E.g package management and building occurs on a dedicated host that does have network/compiler access and then a bundle file can be copied over to the Bro nodes to reproduce the desired package ecosystem.
- Jon