ZeekCTL log to different directories

Zeek
1

Hello, just wanted to know if it is possible to log the logs of different worker nodes to different directories. This is my node.cfg, pretty straightforward:

[manager]
type=manager
host=localhost

[proxy-1]
type=proxy
host=localhost

[worker-1]
type=worker
host=localhost
interface=enp2s0
env_vars=LogDir=/data/zeek/storage01

[worker-2]
type=worker
host=localhost
interface=enx0050b68cac0d
env_vars=LogDir=/data/zeek/storage02

I understand that this wouldn’t work (sort of), because the manager node just places all the logs together. It would just find it strange if ZeekCTL isn’t able to separate logs from different worker nodes/interfaces.

2

Hey,

It would just find it strange if ZeekCTL isn’t able to separate logs from different worker nodes/interfaces.

Interesting question. I’m not sure anyone is running or has asked for such a configuration.

Seeing different approaches:

  • Use the add_node_name or add-interfaces package and post process the resulting logs into separate directory. Would that be sufficient?

  • Enable Log::enable_local_logging and disable Log::enable_remote_logging on the worker nodes. Setting up log rotation might be a bit tricky in this scenario, the upside is that logging won’t go through the manager and should reduce some extra CPU usage that way.

  • Setting $path of all filters to prefix with peer_description. I suspect this would also require changes in the log rotation, but haven’t tested. Something like below could be a starting point

    event zeek_init()
    {
    # Only run this on the manager?
    mkdir(peer_description);

    for ( stream in Log::active_streams )
            {
            local f = Log::get_filter(stream, "default");
            # Is this pushing the API too much?
            f$path = peer_description + "/" + f$path;
            Log::add_filter(stream, f);
            }
    }

Except for the first one, I suspect there might be some issues with log rotation and zeekctl that would require straightening out. Maybe you can explore these a bit?

Thanks,
Arne

3

Thank you so much! The first approach is perfect in my case—I just need to be able to separate the logs based on interface in ElasticSearch. That being said, everything still is logged in the same directory, but that doesn’t seem like a problem (in my case at least).