Unit tests in plugins?

Has anyone put any thought to how we could create and run unit tests
in Zeek plugins? I don't think any work has been done on that yet,
but I'd love to be able to create unit tests in plugins.

thanks!
  .Seth

Has anyone put any thought to how we could create and run unit tests
in Zeek plugins? I don't think any work has been done on that yet,
but I'd love to be able to create unit tests in plugins.

Yeah I've tinkered a bit with this. I was able to get a unit test to run out of an plugin in a loaded Zeek package (for current master) by doing two things:

- Include doctest.h in the plugin source tree.

- Move the doctest execution in src/zeek-setup.cc down until after the plugin registration.

With this, zeek --test -ltc shows the test case, and it also runs:

==============================================================================
/home/christian/.zkg/clones/package/utest/src/Plugin.cc:21:
TEST CASE: plugin doctest

/home/christian/.zkg/clones/package/utest/src/Plugin.cc:23: SUCCESS: CHECK( true ) is correct!
  values: CHECK( true )

But that's obviously a hacky setup. To do this properly, we should add doctest.h to the set of things that Zeek installs (so plugins don't have to ship their own), and we'll have to rework a bit of the setup logic. And, I don't know how/whether this will work with other compilers/platforms; this was just my local system.

Are you up for creating an issue for this one, Seth?

Best,
Christian

So, I agree that doctest.h is a great way to do this, but I think you could take a two step approach here as well:

1. Separate all BIF logic into its own C++ files/classes. If at all possible, have that logic NOT use Zeek types in any way (BIF should convert/manage that). If this is possible (and still performant), you can unit test that code completely without the Zeek source/libraries.
2. If you have done that, the BIF code could theoretically be unit tested from Zeek script itself. Perhaps with ZTest (shameless plug): https://github.com/corelight/ztest :slight_smile:

I think the issue is when the conversion of Zeek types to C++ types would not be performant enough to make this a viable approach. Far too often I'm seeing situations where writing testable code seems to potentially have an impact on performance, but that might be a cost of doing business in Zeek where every nanosecond counts.

-Ryan Victory

Yeah! I'll create a ticket today. Thanks for your thoughts on how to pull it off currently! It saves me from having to play around with it too.

   .Seth