Errors in workers using Spicy 1.2.0

Zeek
1

Hi all,

After some problems to install spicy 1.1.0 in my Zeek cluster under FreeBSD 13-p3, this morning I have tried to install spicy 1.2.0 an all works as expected. Good work!! …

But after installing spicy-plugin and spicy-analysers and executing “zeekctl deploy” (only in manager side) the following error appears in all workers:

root@stirling:/nsm/zeek/spool/idps-prod # more stderr.log
error in /opt/zeek/share/zeek/base/init-bare.zeek, line 1: cannot load plugin library /opt/zeek/lib/zeek/plugins/packages/spicy-plugin//lib/_Zeek-Spicy.freebsd-amd64.so: Shared object "libhilti.so" not found, required by "_Zeek-Spicy.freebsd-amd64.so"
fatal error in /opt/zeek/share/zeek/base/init-bare.zeek, line 1: aborting after plugin errors

Do I need to install spicy in all Zeek cluster components: workers, managers, loggers, etc?

Best regards,
C. L. Martinez

2

Hi Carlos,

Moving this to the Spicy mailing list.

After some problems to install spicy 1.1.0 in my Zeek cluster under FreeBSD 13-p3, this morning I have tried to install spicy 1.2.0 an all works as expected. Good work!! …

That’s great to hear.

But after installing spicy-plugin and spicy-analysers and executing “zeekctl deploy” (only in manager side) the following error appears in all workers:

root@stirling:/nsm/zeek/spool/idps-prod # more stderr.log
error in /opt/zeek/share/zeek/base/init-bare.zeek, line 1: cannot load plugin library /opt/zeek/lib/zeek/plugins/packages/spicy-plugin//lib/_Zeek-Spicy.freebsd-amd64.so: Shared object "libhilti.so" not found, required by "_Zeek-Spicy.freebsd-amd64.so"
fatal error in /opt/zeek/share/zeek/base/init-bare.zeek, line 1: aborting after plugin errors

Do I need to install spicy in all Zeek cluster components: workers, managers, loggers, etc?

Yes, you need to install the same Spicy on all nodes where you expect to deploy spicy-plugin. When you install spicy-plugin it will detect Spicy and its configuration during its build phase; the results of these checks get baked into the plugin, so the setup on the node you build it on (manager) should mirror the target nodes.

Cheers,

Benjamin

3

Ok, installed on all Zeek components: workers and manager …. But now Zeek processes don’t start:

Stderr is:

/opt/zeek/share/zeekctl/scripts/run-zeek: line 110: 1330 Abort trap nohup "$myzeek" "$@“

Best regards,
C. L. Martinez

4

Subcribed to spicy mailing list ..

Best regards,
C. L. Martinez

5

Hi Carlos,

Ok, installed on all Zeek components: workers and manager …. But now Zeek processes don’t start:

Stderr is:

/opt/zeek/share/zeekctl/scripts/run-zeek: line 110: 1330 Abort trap nohup "myzeek" "@“

Could you provide any logs, e.g., on the affected nodes (which ones)? The stderr you shared doesn’t really help understand what is going on here.

Cheers,

Benjamin

6

Good morning Benjamin,

That is the only log that appears in the stderr.log of the workers ... And it happens in all workers ....

Maybe I need to enable a flag to return a more accurate log?

7

Hi Carlos,

That is the only log that appears in the stderr.log of the workers ... And it happens in all workers ....

Maybe I need to enable a flag to return a more accurate log?

I am not familiar with zeekctl.

Could you maybe as a first step check whether you can start Zeek on the workers outside of zeekctl after installing the plugins (i.e., log into a worker node, run Zeek)?

Cheers,

Benjamin

8

Hi Benjamin,

Sorry for this later response. If I try to run manually Zeek in a worker, outside of zeekctl/cluster, always returns the same message: “Abort”. For example, trying to display all plugins installed:

root@stirling:~ # zeek -NN
Abort
root@stirling:~ #

Best regards,
C. L. Martinez