Errors in workers using Spicy 1.2.0

Hi all,

After some problems to install spicy 1.1.0 in my Zeek cluster under FreeBSD 13-p3, this morning I have tried to install spicy 1.2.0 an all works as expected. Good work!! …

But after installing spicy-plugin and spicy-analysers and executing “zeekctl deploy” (only in manager side) the following error appears in all workers:

root@stirling:/nsm/zeek/spool/idps-prod # more stderr.log
error in /opt/zeek/share/zeek/base/init-bare.zeek, line 1: cannot load plugin library /opt/zeek/lib/zeek/plugins/packages/spicy-plugin//lib/_Zeek-Spicy.freebsd-amd64.so: Shared object "libhilti.so" not found, required by "_Zeek-Spicy.freebsd-amd64.so"
fatal error in /opt/zeek/share/zeek/base/init-bare.zeek, line 1: aborting after plugin errors

Do I need to install spicy in all Zeek cluster components: workers, managers, loggers, etc?

Best regards,
C. L. Martinez

Hi Carlos,

Moving this to the Spicy mailing list.

After some problems to install spicy 1.1.0 in my Zeek cluster under FreeBSD 13-p3, this morning I have tried to install spicy 1.2.0 an all works as expected. Good work!! …

That’s great to hear.

But after installing spicy-plugin and spicy-analysers and executing “zeekctl deploy” (only in manager side) the following error appears in all workers:

root@stirling:/nsm/zeek/spool/idps-prod # more stderr.log
error in /opt/zeek/share/zeek/base/init-bare.zeek, line 1: cannot load plugin library /opt/zeek/lib/zeek/plugins/packages/spicy-plugin//lib/_Zeek-Spicy.freebsd-amd64.so: Shared object "libhilti.so" not found, required by "_Zeek-Spicy.freebsd-amd64.so"
fatal error in /opt/zeek/share/zeek/base/init-bare.zeek, line 1: aborting after plugin errors

Do I need to install spicy in all Zeek cluster components: workers, managers, loggers, etc?

Yes, you need to install the same Spicy on all nodes where you expect to deploy spicy-plugin. When you install spicy-plugin it will detect Spicy and its configuration during its build phase; the results of these checks get baked into the plugin, so the setup on the node you build it on (manager) should mirror the target nodes.

Cheers,

Benjamin

Ok, installed on all Zeek components: workers and manager …. But now Zeek processes don’t start:

Stderr is:

/opt/zeek/share/zeekctl/scripts/run-zeek: line 110: 1330 Abort trap nohup "$myzeek" "$@“

Best regards,
C. L. Martinez

Subcribed to spicy mailing list ..

Best regards,
C. L. Martinez

Hi Carlos,

Ok, installed on all Zeek components: workers and manager …. But now Zeek processes don’t start:

Stderr is:

/opt/zeek/share/zeekctl/scripts/run-zeek: line 110: 1330 Abort trap nohup "$myzeek" "$@“

Could you provide any logs, e.g., on the affected nodes (which ones)? The stderr you shared doesn’t really help understand what is going on here.

Cheers,

Benjamin

Good morning Benjamin,

That is the only log that appears in the stderr.log of the workers ... And it happens in all workers ....

Maybe I need to enable a flag to return a more accurate log?

Hi Carlos,

That is the only log that appears in the stderr.log of the workers ... And it happens in all workers ....

Maybe I need to enable a flag to return a more accurate log?

I am not familiar with zeekctl.

Could you maybe as a first step check whether you can start Zeek on the workers outside of zeekctl after installing the plugins (i.e., log into a worker node, run Zeek)?

Cheers,

Benjamin

Hi Benjamin,

Sorry for this later response. If I try to run manually Zeek in a worker, outside of zeekctl/cluster, always returns the same message: “Abort”. For example, trying to display all plugins installed:

root@stirling:~ # zeek -NN
Abort
root@stirling:~ #

Best regards,
C. L. Martinez