We’re proud to announce the release of Zeek 8.0! This release introduces a range of technical innovations and culminates architectural improvements we’ve been working on since the release of Zeek 7 a year ago.
Customizable Flow Tuples
For the first time ever, it’s now possible to customize Zeek’s notion of flow tuples via plugins. This enables users to disambiguate colliding flows in complex networks via additional context, such as VLAN tags or the network identifiers provided by encapsulations such as VXLAN and Geneve. Zeek 8 ships with optional support for including VLAN tags in the tuple, and we detail how to build custom context in our documentation. This was one of the most long-standing user requests in our issue tracker, and we’re thrilled to address it in this release.
Framework Enhancements
Much has happened in our frameworks as well. Most importantly, the pluggable cluster backends we introduced in 7.1 have matured to a point where we now encourage users to try our future default backend, powered by ZeroMQ, in production. While Broker remains the default for Zeek 8.0, switching to ZeroMQ is a one-liner in scripting, and just as easy to do via zeekctl. Cluster backends drive all of Zeek’s intra-cluster message communication, and the switch to ZeroMQ is far from a technical detail: common communications patterns, such as worker-wide broadcasts, are now natural, without the need for users to explicitly route events via proxies or the manager. That’s because in contrast to Broker, ZeroMQ’s message pub/sub now operates decoupled from node-to-node connectivity and purely via cluster-wide topics.
We’ve also streamlined telemetry for cluster operation, making it easy to trace message volumes across nodes regardless of the active backend. The storage framework, first introduced in 7.2 as a replacement for Broker’s data stores, has matured further. We’ve added telemetry, improved handling of expirations from stores, and made store I/O automatically synchronous when processing pcaps.
Improved Protocol Analysis
On the logs & protocols front, Zeek 8 offers several enhancements as well. We’ve included a Redis parser and corresponding redis.log. The SMTP analyzer can now hand off top-level mail messages to the file analysis framework, simplifying full RFC 822 message extraction. FTP has gained AUTH TLS support, DNS now understands NAPTR resource records, and PPPoE now reports session IDs.
Zeek 8 also streamlines analyzer.log and dpd.log, replacing them with a single new analyzer.log that reports analyzer violations of confirmed protocols. The service field in conn.log now reports protocols in predictable order as Zeek confirms them.
Users migrating from Zeek 7.0 will enjoy additional protocol analysis enhancements from the 7.1 and 7.2 releases, including a new ip_proto field in conn.log to label non-TCP/UDP/ICMP flows, OUI awareness in SNAP decapsulation, TKEY resource records in DNS, StartTLS and GSS-API WRAP tokens for LDAP, a new event for mid-session user changes in MySQL, support for cookie-less RDP connections, more robust SSH header parsing, and a new analyzer for Postgres, with a corresponding postgres.log.
More Spicy
Zeek 8 ships with Spicy 1.14, which sets the groundwork for upcoming performance improvements in Spicy parsers. We added a framework allowing optimization passes to take control flow into account which will enable more impactful optimizations. Spicy also now removes unused function parameters. Most code will not see huge speedups with Zeek 8.0, but expect upcoming releases to fully utilize the new capabilities. We also made a number of usability improvements — for example, if you’ve ever seen a C++ compiler error when developing a Spicy parser, you now will get a link directly to Spicy’s issue tracker to submit a bug.
That’s Not All
Zeek now offers log schema support via the logschema package. Supporting a wide range of Zeek versions, the package renders log schemas in popular flavors (including JSON Schema, CSV, and via Zeek’s own logging) that explain each Zeek log’s fields in detail.
As always, please consult Zeek’s and Spicy’s release notes for the full list of changes, and consult our documentation. As the first release in the new cycle, Zeek 8.0 builds on the developments of 7.1 and 7.2, and becomes our latest long-term support release, meaning it will see security fixes and relevant backports for a bit over a year. Our past LTS release, Zeek 7.0, will stop seeing updates when we release 8.1 in about four months.
We encourage all users to upgrade to Zeek 8 at this time. Feedback and questions are always welcome, so please feel free to get in touch via our community channels.
Thanks to our contributors!
Work on Zeek 8 began at the end of April and includes some 1,300 commits in 350 merged pull requests. As always we’re particularly grateful to our community members who contributed to this release: @aidans111, Anthony Verez (@netantho), Baa (@Baa14453), Bhaskar Bhar (@bhaskarbhar), @dwhitemv25, EdKo (@ephikos), @edoardomich, Fupeng Zhao (@AmazingPP), Hendrik Schwartke (@hendrikschwartke), @i2z1, Jan Grashöfer (@J-Gras), Jean-Samuel Marier, Justin Azoff (@JustinAzoff), Mario D (@mari0d), Markus Elfring (@elfring), Peter Cullen (@pbcullen), Sean Donaghy, Simeon Miteff (@simeonmiteff), Steve Smoot (@stevesmoot), @timo-mue, @wojciech-graj, and Xiaochuan Ye (@XueSongTap) — thank you!