"zq" command-line processor for Zeek logs

Zeek community,

We’re writing to let you know about zq, an open source command-line processor for structured logs, built for Zeek. (In fact, we’ve been told zq is “like zeek-cut on steroids”.)

Those of you who were on the “Ask the Zeeksperts” call on January 16th saw Seth Hall and Justin Azoff give an early peek of zq (thanks guys!), so this is just an “official” announcement. Come one, come all!

You can get involved by:

• Checking out the zq GitHub repo for install info, code, and docs

• Joining our public Slack workspace for announcements, Q&A, and to trade query ideas

• Contacting us directly via email to schedule a Zoom videoconference

All you need is some Zeek logs (and there’s sample logs to help you get started). Here’s just a taste of what’s possible:

  • A table of top hosts in a subnet that are experiencing the most SYNs-without-ACK:
    zq -f table “ conn_state=S0 | count() by id.orig_h | sort -r” *

  • A regex search for certain HTTP methods, with full events output as NDJSON:
    zq -f ndjson “method=/^(PUT|PATCH|UPDATE)$/” *

  • Connections open a long time with low traffic, printed as a Zeek TSV log:
    zq -f zeek “duration>1000 orig_bytes<10 resp_bytes<10” *

Of course, that’s just scratching the surface. Please try it out and let us know what you think on GitHub or Slack.

Happy hunting, Zeeking, & zq’ing!

Zeek community,

I’m reaching out to announce another open source project… specifically the Brim desktop application.

In its first version, the Brim workflow is tuned for starting from a packet capture (even a big one), which the app turns into Zeek logs for you. Then you’ve got an intuitive UI experience for querying those Zeek logs using the same ZQL language you may already know from zq (see prior announcement below). And should your Zeek explorations lead you to a flow for which you want to see the packets, a single click in the app quickly extracts the flow from the big pcap and opens it immediately in Wireshark.

For more details, here’s some links for Brim:

There’s more coming soon, so keep your eye on the repo for updates.

Happy hunting!

Zeek community,

It’s been a while since we first notified folks here when we initially released our Brim desktop app. Here’s an update that we expect will interest the Zeek community.

Whereas the first version of Brim was all about starting from packet captures and turning those into Zeek logs, the new version v0.9.1 we just released introduces direct import of Zeek logs (default TSV format or JSON). So you can now have all the querying, workflows, etc. for working with your Zeek data in Brim even if you don’t have pcaps.

For more details, here’s some links for Brim:

On a separate-but-related topic, while it’s not visible in the Brim app yet, our related project zq includes an experimental prototype for working with archived Zeek logs called “zar” that’s referenced in a new README. If this topic interests you, check it out and come talk to us on our Slack in the #zar channel.

Happy hunting!