[I had posted this on Slack in #development originally, copying here
for visibility]
We now have the nice list of platforms that Zeek currently supports
at: https://github.com/zeek/zeek/wiki/Zeek-Operating-System-Support-Matrix.
We however don’t have a policy yet what platform we generally aim to
support going forward. Dominik and I discussed this further the other
day and came up with a proposal:
1. For each distribution, we decide on a dedicated policy which
versions we want to support at any point. A general rule of thumb
would be “the two most recent stable releases still within their
support window”. But we’d actually look at each distribution and
see what policy makes sense, including any additional dependencies
we will rely on (like devtools on CentOS). And we do
mini-request-for-comments to get feedback from users if we’re
making a reasonable choice.
2. We update CI to test those versions, and *only* those versions.
3. We then rely on CI to decide if changes are ok: If, e.g.,
somebody wants to use a new C++ feature, that’s ok as long as CI
passes. If it breaks CI, it won’t go in.
The advantage of this is that we won’t rely on having to specify
specific compiler versions, which isn’t well-defined (because things
also depend on libstdc++ version, system libs, etc.), but instead on
whether people can actually compile Zeek on the platforms they are
using.
A disadvantage is that we’ll need to spend the time to ensure our CI
setup keeps matching the policies, and gets updated as distribution
updates come out.
I like this a lot! At the end of the day, what matters for our builds is what the distros ship, and this covers that. Plus, it's not that hard to update the CI images to keep them current.
We could put together a release calendar to remind us when it's time to update CI. I took a quick look and couldn't immediately find one. I'd be happy to put one together.
Yes, that would be great. The other thing we'll need to do is define
the per distribution policies. You could add a 1st stab at that to the
calendar as well, or we can do it afterwards.
This includes CentOS, Debian, Fedora, FreeBSD, and Ubuntu.
I tried a few things and landed on one entry per release that starts on release date and ends on (LTS, when available) EOL. The benefit is that on any given day you see directly which releases are currently supported, and a click on any of the entries shows you start & end. The downside is that every day looks crowded, and it's a bit harder to see the starts/ends (week view brings it out okay though for me).
The other thing we'll need to do is define
the per distribution policies. You could add a 1st stab at that to the
calendar as well, or we can do it afterwards.
I think we've got this, too: when a new release starts, it should go into CI, and when one drops off LTS EOL, it can go. What do you think?
Want to start a Wiki page where we collect the pieces for our new
policy? Can link to those calendars from there.
I think we've got this, too: when a new release starts, it should go into
CI, and when one drops off LTS EOL, it can go. What do you think?
Yeah, though I think we'll need to discuss on per distribution basis
what exactly the versions are that we want to support at any point of
time (and hence have in CI), plus any additional assumptions we're
making (e.g., requiring devtools-X on CentOS).
That might be as easy as supporting anything currently in either
release or LTS, but spelling it out in the terms of the corresponding
distributions would be helpful to make sure everybody's on the same
page (e.g., some phrasing of "on Ubuntu, we support the current
release as well as any LTS release not EOL yet; and we assume just
standard packages"). Could you give that a try on that Wiki page as
well for those distros, and then we can send it around to the users'
list to see if people can get behind it?
One more OS to add is macOS. While nobody knows the deadlines there,
we should record how far back we go in supporting previous versions,
and what package management we're assuming for dependencies (Homebrew
I suppose).
I've also added an entry to the calendar for macOS (Catalina, using the availability of images in Cirrus CI as a driver), and I'm putting together a PR (still a draft) for bringing our CI in line with what's in that page:
Which of these come with additional requirements beyond just OS base
packages? We should note those. What I can think of:
CentOS: We require devtoolset (which version?) and probably
EPEL? I don’t recall how people get some of the dependencies in
place (bison, flex, cmake).
macOS: Homebrew I suppose.
On CentOS, if we limit support to “full updates”, CentOS 7 will drop
off soon. I’m wondering if we should use versions with maintenance
updates instead? (I’m glad CentOS 6 just went out of life support
Does anybody know if Cirrus keeps previous macOS images available?
If so, I’d suggest we keep the two most recent macOS versions.
- Which of these come with additional requirements beyond just OS base
packages? We should note those. What I can think of:
- CentOS: We require devtoolset (which version?) and probably
EPEL? I don't recall how people get some of the dependencies in
place (bison, flex, cmake).
Mhmm ... this seems a bit tricky. At least in our CI setups each platform requires some stuff to be installed to allow a build, so I'm wondering whether instead of spelling them out in the wiki page we could just point at our CI? That'd be Dockerfiles for anything Linux and prepare.sh for FreeBSD and macOS. The benefit would be that we'd maintain this in one place only. We could invest a bit of time in documenting the Dockerfiles/prepare.sh scripts so they explain these?
- macOS: Homebrew I suppose.
Yeah, worth calling out because Cirrus includes it in their image, so you won't notice it when looking purely at only our CI setups.
- On CentOS, if we limit support to "full updates", CentOS 7 will drop
off soon. I'm wondering if we should use versions with maintenance
updates instead? (I'm glad CentOS 6 just went out of life support
-- yes, I mislabeled that one. I actually meant to say "Maintenance Updates". Fixed in the wiki page.
- Does anybody know if Cirrus keeps previous macOS images available?
If so, I'd suggest we keep the two most recent macOS versions.
I'm definitely not the expert here but it all looks like Catalina with varying additions:
Btw I didn't include anything about architectures ... for Debian 9 we currently have a 32-bit container, for some other platforms those are still available too. Do we still care about 32-bit?
we could just point at our CI? That’d be Dockerfiles for anything
Linux and prepare.sh for FreeBSD and macOS. The benefit would be
that we’d maintain this in one place only. We could invest a bit of
time in documenting the Dockerfiles/prepare.sh scripts so they
explain these?
Yeah, agree, that sounds better than maintaining the information separately.
– yes, I mislabeled that one. I actually meant to say
“Maintenance Updates”. Fixed in the wiki page.
Perfect. :).
I’m definitely not the expert here but it all looks like Catalina
with varying additions:
Yeah, I saw that, but not sure if that means they are actively removing older images. I’ll see if I can find out.
Btw I didn’t include anything about architectures … for Debian 9
we currently have a 32-bit container, for some other platforms those
are still available too. Do we still care about 32-bit?
Limiting to 64-bit seems fine to me for our current CI platforms. I’m wondering about supporting ARM (32- & 64-bit for Linux, 64bit for future macOS), but looks like CI doesn’t support that yet either way: https://github.com/cirruslabs/cirrus-ci-docs/issues/218
Hah, I asked them too and they actually mentioned their email reply to you!
Yeah, and in a further followup email they went a bit further even and committed to generally keeping the previous base image around in the future when new macOS release come out (but not the Xcode image).
I’m really glad to see the new policy taking shape.
One more thing besides available C++ features worth tracking is CMake availability. In the CAF CI (Jenkins), I’ve made all the various compiler and CMake versions used throughout a build available as artifact. Still work in progress, but this is the current list for the Docker images (the Linux versions currently listed in the Zeek Wiki minus Fedora 33):
Compiler:
Clang 7.0.1
Clang 8.0.0
GNU 10.2.1
GNU 7.3.1
GNU 8.3.0
GNU 8.3.1
GNU 8.4.0
GNU 9.3.0
GNU 9.3.1
I’m a bit sad to see CMake 3.5 still around (released in 2016). Anyway, "it is what it is”. Versions to watch out for in particular IMHO are 3.12 (allows target_link_libraries for object libraries) and 3.15 (introduced FetchContent).
In any case, might be worth tracking the current lower bound for CMake and library dependencies (like OpenSSL) to know when it’s safe to drop existing workarounds or start using new features. Not sure if Cirrus CI can collect data from all builds for providing a single artifact, but otherwise one could provide a script to combine the individual artifacts locally via curl or some other means.
At least CMake 3.5 seems safe as minimum requirement in Zeek/Broker for now. Already a step up from the current 3.0 dependency.
