Best way to modify bundled scripts

Zeek
1

Hello,

We have deployed Zeek in production environment where most of user HTTP traffic is behind proxies. The proxies use a mix of Digest (mostly) and Basic Auth (single proxy for special use case).

Our use case is to extract username for all users trying to browse using proxies. I have written a custom script that defines a new handler for event http_header. It is heavily influenced by the bundled scripts/base/protocols/http/main.zeek and seems to do the job. However, if a request has both Authorization and Proxy-Authorixation headers, it can lead to a race between two event handlers (both have same priority).

Ideal solution would be to extract all proxy usernames to a new Log attribute (maybe proxy_username) irrespective of whether they are from basic or digest auth. This would mean that my custom script would handle Proxy-Authorization header for both basic and digest case.

To do so, I need to edit the bundled script so that it only looks for Authorization header here.

What is the “best” way to achieve this so that this is repeatable across builds and sensors? I build zeek myself and deploy it to multiple sensors using Salt. I can think of

  • Creating a patch with necessary changes and deploying it before building the RPM. Then keeping the patch up to date for future releases.
  • Use salt’s file editing capabilities to modify the file inplace after the install
  • Something else?

Please help.

2

Hello redbaron :wave:

The Zeek-Way to go about this is to create a separate Zeek script which extends the HTTP::Info record (what ends-up in the http.log) with the proxy_username field you proposed and have a new http_header event handler that populates just that field. Then load this script in addition.

$ cat http-proxy-username.zeek 
@load base/protocols/http


# Extend the http.log with a new proxy_username field
redef record HTTP::Info += {
    proxy_username: string &log &optional;
};


# http_header handler that populates the proxy_username field
event http_header(c: connection, is_orig: bool, original_name: string, name: string, value: string)
        {
        if ( ! c?$http )
                return;

        if ( name != "PROXY-AUTHORIZATION" )
                return;

        # TODO: extract from value and split on ':' like
        # in http/main.zeek
        local userpass = "...";
        local up = vector("user", "pass");
        if ( |up| >= 1 )
                c$http$proxy_username = up[0];
        }

You can test the script with a pcap of your traffic as follows:

zeek -r path-to-pcap-with-http-traffic.pcap ./http-proxy-username.zeek

For deployment, you’ll need to ensure that Zeek loads http-proxy-username.zeek. This can be done by putting it on Zeek’s command-line, or add an @load http-proxy-username.zeek into local.zeek (assuming local is on the command-line).

Modifying or patching the base scripts should not be needed, unless you’re doing something very exotic. The scripting language provides primitives to extend and customize the logs as shown above. The one file in Zeek’s base distribution that’s meant to be modified is local.zeek. You could use salt to append the @load /path/to/http-proxy-username.zeek to local.zeek, or roll-out your version of local.zeek to the systems within the RPM. If you’re not using local.zeek today, placing http-proxy-username.zeek Zeek’s command-line will work, too.

As of how you get http-proxy-username.zeek deployed: It seems you have enough infrastructure in place to either deploy it via salt as a plain file, or package it as an additional script into your own RPM builds.

There’s, however, also the Zeek package manager / zkg you can leverage. E.g. you could create a Zeek package from the http-proxy-usernames.zeek script. If you’re building your own RPMs today, though, that’s probably extra complexity and I’d suggest to stick with just a single package manager (minimally on the production systems). You can still use zkg during the RPM build process to fetch/build additional community packages if that’s of value to you.

Hope this helps. Sorry for being verbose. Please let us know if that works for you.

2 Likes
3

Thanks for the detailed explanation @awelzel !! Yes I was thinking along the same lines about writing a custom script. My query was about what to do about the bundled script that also looks at both Authorization and Proxy-Authorization headers. Sorry if the original post was not clear.

Since my custom script will be looking at the Proxy-Authorization header for both Basic and Digest cases, I would like the bundled script to skip the Proxy-Authorization check it performs (instead only checking the Authorization header). So I wanted to know if there is a “Zeeky” way of modifying the bundled scripts for such use-cases?

4

I’d still recommend leaving the bundled script as it is and introduce new fields for the data you’re specifically interested, even if it seems duplicated. Re-using and changing well-known fields from the existing http.log can be a source of confusion down the road. The previous snippet should provide enough pointers how to go about this.

If you do want to re-use the fields, and alternative is to write a custom event http_header() with a lower priority (&priority=-5 or so) that will be executed after the handler in the bundled script. Within that handler you can undo what http/main.zeek did by clearing and/or overwriting the c$http$username and c$http$password again based on the Basic/Digest values you extract. Essentially, instead of skipping logic in the bundled script, you can undo the effects in your own script. Again, I’d recommend to use new fields in the log for this purpose, however.

Hope this helps.

1 Like