I'm just wondering whether we should split out the Broccoli Python
bindings into their own package as well? Probably makes sense, and
makes them easier to install.
Robin
How does it make them easier to install? I would guess that if the build system for Broccoli automatically detected the python header and library (or ruby for that matter) and installed the binding along with the library that would be easier.
I don't know how the packager maintainers for various OSs feel about it though.
.Seth
How does it make them easier to install?
Thas was ambigious, sorry: I meant easier for creating the build
system. As it is, it needs to do a mix of the CMake installation for
libbroccoli, and the Python setup.py for the bindings. The old
autotools setup avoids that by just not installing the bindings by
default, but that's not great, in particuar because broctl depends
on them.
I don't know how the packager maintainers for various OSs feel about it though.
That's another point, right.
Robin
Thas was ambigious, sorry: I meant easier for creating the build
system.
Ah, ok. That makes sense I think. 
I don't know how the packager maintainers for various OSs feel about it though.
That's another point, right.
I was planning on merging the ruby bindings into broccoli at some point, but now it sounds like I shouldn't?
.Seth
Not sure yet, waiting for more opinions ...
Robin
Just got back into town and will talk with jon about this when he gets back Monday and give you our opinion from a packaging perspective.
I was planning on merging the ruby bindings into broccoli at some
point, but now it sounds like I shouldn't?
It seems to me the language should ship together with Broccoli. When
adding more new languages in the future, some "configure" script could
help in selecting only a subset. But conceptionally, the bindings *are*
Broccoli, yet merely offer a different interface. That said, my vote
would be to ship them together.
Matthias
I see three major areas to think about (some thoughts may be applicable to more than just Broccoli bindings). Here's my decomposition:
1) The build system.
I don't see an issue right now with integrating the Python setup.py into CMake -- it should basically involve adding a custom target that delegates the build to setup.py at "make" time and then also delegating to setup.py at "make install" time.
So if we kept broccoli + bindings together, it seems like it's not hard for us to make it so that the bindings are built/installed by default (or selected) which differs from current autotools setup.
Separating bindings would offer a little simplification to its build system -- it can assume that broccoli is already built/installed.
2) The standard package management system per OS.
Seems like it's pretty standard for Debian packages to separate the bindings and library packages.
I don't think it's incredibly harder to generate separate packages even if you're working from a single source tree, but it does seem that if they were separate to begin with, then it simplifies the packaging process a little and makes it easier to add optional bindings for other languages.
3) The git repo organization.
If bindings are kept in their own repos, then don't we gain some flexibility on assigning maintainers that might differ from the broccoli maintainer? How useful may that be?
The broccoli repo could also reference binding repos via git submodules so it's not like they're totally disconnected. I think we should plan on using submodules more generally to keep coherence between Bro and subcomponents.
One question here for clarification: above you write that CMake
delegates to setup.py; however your Wiki page mentions in the
pysubnettree section:
* Will probably have a thin CMake script to build extension and install module+extension
* Keeping the setup.py around may be useful for packagers... not sure
That sounds like replacing setup.py with CMake (rather than
combining the two). So, I was wondering which of the two options
you're planing to follow?
I'd prefer to keep using the setup.py in both cases, as that's the
standard way to build Python extensions and thus probably more
robust.
Robin
Thanks for the summary. One additional advantage of having it
separate is that installing Broccoli does not automatically pull in
the bindings, which for various reasons may not be desired (for
example, there's the problem that Python development headers are
needed for compiling them, which is something we are currently
running into with the broctl installation; their absense could
autodetected, but that then introduces new complications when other
packages assume the bindings have been installed).
So, I think I'm following your leaning, though I'm still not really
feeling strong either way.
Robin
One question here for clarification: above you write that CMake
delegates to setup.py; however your Wiki page mentions in the
pysubnettree section:* Will probably have a thin CMake script to build extension and
install module+extension
* Keeping the setup.py around may be useful for packagers... not sureThat sounds like replacing setup.py with CMake (rather than
combining the two). So, I was wondering which of the two options
you're planing to follow?
Yeah, I vacillated over the weekend and found that it's pretty easy to have CMake completely handle the build/install of Python extensions, but...
I'd prefer to keep using the setup.py in both cases, as that's the
standard way to build Python extensions and thus probably more
robust.
Yes, distutils would probably be the more standard/robust way if it were a completely independent component. But if we want to provide a "unification interface" to have it built as part of a larger Bro source distribution via CMake, then it would be even less standard to have CMake delegate to distutils rather than do it itself. It definitely "feels" like I would be abusing CMake by mixing it with a completely different build system.
Maybe one way to go is to provide both a CMake build system and a distutils one (separated), so that people that need the extra robustness (e.g. developers/packagers) can still use the setup.py while people that just want to build Bro from source can follow our established CMake method. Not quite sure what the best option is for that use-case yet.
- Jon
Maybe one way to go is to provide both a CMake build system and a
distutils one (separated),
That sounds good.
just want to build Bro from source can follow our established CMake
method. Not quite sure what the best option is for that use-case
yet.
Not sure what it is you're not sure about?
Robin
> just want to build Bro from source can follow our established CMake
> method. Not quite sure what the best option is for that use-case
> yet.Not sure what it is you're not sure about?
Putting Humpty Dumpty back together again.
For those that want an easy way to build/install all Bro components from source. I'm not sure if mixing in another build system w/ CMake will work well.
- Jon
So, I think I'm following your leaning, though I'm still not really
feeling strong either way.
Doesn't seem like anyone is strongly opposed, so I think we can commit to the decision to put bindings in their own repos, right?
I can go ahead and try to follow our internal doc for creating a new repo, or let me know if you wanted to do it, Robin.
- Jon
Yes, please go ahead; good test whether the doc has everything it
needs ... 
Robin