Hi, I am working on a custom analyzer, and I am able to use it in offline mode, however, I would like to know how to configure Zeek to use it for live traffic analysis, i.e., how to make Zeek recognize the new analyzer.
I am using Zeek 6.0.3, and created the analyzer using the following command:
Also, I’ve noticed an error in the automatically created zeek_myprotocol.spicy file:
on MyProtocol::SUITABLE_UNIT::%done {
zeek::confirm_protocol();
}
on MyProtocol::MyProtocolUnit::%error {
zeek::reject_protocol("error while parsing MyProtocol");
}
SUITABLE_UNIT was not substituted by the unit I chose, causing an error on the creation of a binary file related to the protocol. I simply wrote the chosen unit, just reporting this to let you know, in case this bug was not solved in newer versions of Zeek.
I am guessing by “offline analysis” you mean the analyzer activates if you explicitly load it in your zeek invocation?
The suggested approach to install Spicy analyzers is with Zeek’s package manager zkg, e.g.,
$ zkg install <path/to/your/analyzer/repo>
To enable zkg-managed packages you need to modify your local.zeek file. Near then end of it you find a line
# Uncomment this to source zkg's package state
# @load packages
Uncomment this @load statement like the comment says. To use your local settings you need to load local.zeek, by passing it to your zeek invocation (zeek ... local means "load a script local.zeek); the docs explain this
AFAICT we do not generate code like that. Instead we generate a commented out block with instructions to pick a value for SUITABLE_UNIT which makes sense for your parser,
# TODO: Protocol analyzers should confirm once they are reasonably sure that
# they are indeed parsing the right protocol. Pick a unit that's a little bit
# into the parsing process here.
#
# on MyProtocol::SUITABLE_UNIT::%done {
# zeek::confirm_protocol();
# }
Hi, I put my analyzer in a git repository, then I cloned it and executed sudo zkg install . in the repository’s directory, however, I got this error message:
error: failed to run tests for /home/mayara/dnstunnelling: no "zeek-config" or "bro-config" found in PATH
I configured Zeek’s path after installing it, and when I execute which zeek-config I get the correct path.
When I executed the test command cd testing && sudo btest -c btest.cfg, I got the same zeek-config not found error, and I would like to know what caused this error and how to solve it.
# TODO: Protocol analyzers should confirm once they are reasonably sure that
# they are indeed parsing the right protocol. Pick a unit that's a little bit
# into the parsing process here.
#
# on MyProtocol::SUITABLE_UNIT::%done {
# zeek::confirm_protocol();
# }
Sorry for not paying close attention to these comments, I suposed that SUITABLE_UNIT was going to be automatically replaced by the chosen unit.
This error message comes out of a zkg before zkg-3.0.0 (you can query the version with zkg --version). Something might be wrong with the zkg you are using. The simplest fix would be to use the zkg which comes with the installation of Zeek you are trying to run.
This should not be the same error, but instead a different one (e.g., zeek-config not found) coming not out of zkg, but instead the scripts generated for the plugin. This indicates that zeek-config was not found which could be for a number of reasons, the simplest being that you did not export your modifications to PATH; this might also cause above error from zkg. Try
Hello, I’ve installed Zeek from source and, indeed, zkg version was outdated. The installed version was 2.12.0, and I installed the newest version from github using the command pip3 install git+https://github.com/zeek/package-manager@master. This solved the zeek-config not found error. However, the unit tests failed and I am not able to install the custom analyzer. I got the following error message:
Running unit tests for "/home/mayara/dnstunnelling"
error: failed to run tests for /home/mayara/dnstunnelling: test_command failed with exit code 1
In relation to the tests executed with sudo btest -c btest.cfg, I made some changes on the get-zeek-env file, due to the fact that there were some commands trying to access a file structure that was different from what was really installed in my computer. zeek-path-dev file was located in Downloads/zeek-6.0.3 and zeek-config path is /usr/local/zeek/bin. These changes solved the zeek-config not found error.
On the other hand, while making tests, I just uncommented the @TEST-EXEC lines on test files, to execute the default tests, and still I could not get successful results on the tests.
I could send the link to the repository I am using so you could see better the changes made.
Probably it would be a good idea to make sure that your package tests and installs successfully in a clean environment. I would suggest you test your package inside our Docker images.
I’ve managed to use zeek on a docker container, and executed btest -c btest.cfg and also tried to install the analyzer using zkg, however, I still got error messages.
When trying to install the analyzer, I received this error message, different from the previous one, with information stored in dnstunnelling-build.log:
error: failed to run tests for /dnstunnelling: package build_command failed, see log in /usr/local/zeek/var/lib/zkg/logs/dnstunnelling-build.log
These are the changes I made in get-zeek-env file in order to try to solve the zeek-config not found error while running tests:
base="/zeek-6.0.3"
#base="$(dirname "$0")"
zeek_dist="/usr/local/zeek/bin"
#zeek_dist=$(cat "${base}/../../build/CMakeCache.txt" 2>/dev/null | grep ZEEK_DIST | cut -d = -f 2)
if [ -n "${zeek_dist}" ]; then
if [ "$1" = "zeekpath" ]; then
#"${zeek_dist}/build/zeek-path-dev"
"${base}/build/zeek-path-dev"
base="/zeek-6.0.3" is a directory imported from the local files to the docker container.
/usr/local/zeek/include/zeek/iosource/Packet.h:18:10: fatal error: pcap.h: No such file or directory
18 | #include <pcap.h> // For DLT_ constants
This seems to indicate that you did not install Spicy development dependencies. This is documented in the Docker setup section I linked to,
Similarly, you’ll need g++ for Spicy’s JIT compilation, as well as cmake and make to build Spicy analyzer packages.
apt-get update
apt-get install -y --no-install-recommends g++ cmake make libpcap-dev
With that an analyzer generated from the default template should build, test, and install.
You should not need to touch this generated code at all. What you changed here is a code path which is taken for packages which want access to non-public APIs in Zeek and build against a Zeek source tree. AFAIK this is mostly to support legacy plugins and certainly not needed for Spicy analyzers.
The code path your analyzer should take is on the else branch which follows,
else
# Use Zeek installation for testing. In this case zeek-config must be in PATH.
if ! which zeek-config >/dev/null 2>&1; then
echo "zeek-config not found" >&2
exit 1
fi
I suspect you ran into issues there since your environment was not properly set up.
I would suggest you revert all the build system changes (they could make your package not installable on other systems). To get you back on track, you should not need to modify anything but Spicy grammars (*.spicy files in analyzer/), Zeek glue code (*.evt files in analyzer/), Zeek scripts (files in scripts/), and possibly tests in testing/tests/ and their traces in testing/Traces/); the generated zkg.meta should work and you only might need to adjust the description field and possibly add dependencies. It might be easiest for you to regenerate a clean setup and copy your above changes over; validate that this works in a properly set up Docker container.
Hi, I’ve managed to install the custom protocol on the docker container ( ), then I copied the folder to my local files and created a github repository containing the folder, however I did not succeed in installing the analyzer in my computer.
I did not change anything on the test files, however, my package fails on the trace test, I don’t know if this have any relation with the fail in installing in my computer.
So, to solve the problem should I make some changes in trace.zeek or try to reinstall Zeek on my computer? What type of changes should I make on trace.zeek, if necessary?
Also, when I create another container, importing the folder with the analyser, I cannot install the analyzer as well.
error: failed to run tests for /home/mayara/DNStunnelling: test_command failed with exit code 1
Hello, in the last few days I made some changes in the custom analyzer, specifically in the .spicy file, adding 2 units, in order to promote further analysis over the packets and now, when I install the analyzer in a Docker container skipping the tests, I get errors related to tests.standalone and tests.trace errors.
I have doubts about how should I change these test files according to the features of my analyzer, and I would really appreciate some orientation in this sense, or, if possible, to see some examples that could help me understand about the changes I should make.
), then I copied the folder to my local files and created a github repository containing the folder, however I did not succeed in installing the analyzer in my computer.