We've been talking a bit about our Bro community script archive BPAN [1]
quite a bit lately. Below, I summarize how I envision this project to
manifest. Feedback would be much appreciated.
Matthias
Objectives
- Facilitate sharing Bro scripts in the community
- Facilitate the integration of user-provided scripts
- Specify a set of repositories (or paths inside) to load Bro scripts
Requirements:
- Sharing scripts should be simple: one should be able to point
someone to a remote repository and the integration happens
automatically.
- Organization by topic tags: similar to package managers, we need a
way to categorize scripts and bundle them. For example
- Separation between officially supported and user/experimental
scripts. For the official ones, we offer support and make sure
they'll always work with the latest/current version of Bro. We do
not make such a guarantee for external scripts.
- Integration with the Input/Output framework. That is, users should
be able to provide custom data sources and sinks.
Implementation details:
- github's popular fork-pull-merge model is an excellent fit for
sharing community repositories. We may want to use
https://github.com/broids as base for our community scripts. Users
that want to extend the base simply perform the changes, open a
pull request, and we can review it. The nice thing is that people
can comment on it and follow-up changes are nicely reflected.
Moreover, github automatically lists contributors to a certain
file, so this encourages coders to share, participate, and get
"reputation"---akin to stackoverflow.
- Unlike homebrew, I believe it does not make sense for Bro to have
single huge repository that contains all scripts and that users
continuously branch from. Bro users probably want to organize
their scripts into different repositories and combine multiple
repositories ad libitum, because some repositories may contain
sensitive elements and should not be public. Yet, private
repositories can still be easily integrated with this new
framework.
Having separate repositories also keeps the repository size under
control. Although the input framework will be the main mechanism
to source external data, I could imagine some folks would like to
stuff some file-based intelligence in their repositories.
However, separate repositories require integration efforts, but
this is manageable in my eyes and outlined below.
- Directory structure: the directory structure provides implicit
naming and some meta data. If the directly structure is violated,
Bro will not be able to automatically load scripts. These are the
rules:
* Sub-directories in the repository represent "addressable
units," e.g., names that users can refer to. I call these from
now on a "rebro". Each rebro must have at least one file
called '__load__.bro'. This is the entry point for Bro.
Example:
- repo := https://www.github.com/joe-user/bromising/
- repo/
repo/a/
repo/a/__load__.bro
repo/a/aa.bro
repo/a/ab.bro
repo/b/__load__.bro/
repo/b/x/__load__.bro
repo/b/x/xx1.bro
repo/b/x/xx2.bro
repo/b/y/yy1.bro
In this example, we have the following rebros:
repo/a
repo/b
repo/b/x
By placing __load__.bro files judiciously, users can
explicitly decide what parts of the repo may be interpreted as
a rebro. We may also allow putting a __load__.bro file in the
top-level directory.
* Each __load__.bro file contains additional meta data. Here is
an example
## @version: 0.42
## @author: Joe Bro <bro@bro-ids.org>