Trouble building Bro 2.2 on ARMv6 (Raspberry Pi / Wheezy)

Hi, I have been using Bro 2.1 on my Raspberry Pi device. It worked mostly out of the box. (There was a small configuration change due to RPI’s missing realtime clock but otherwise, worked fine.) However, Bro 2.2 does not build on the same device. It builds fine on the same configuration of Linux (Debian Wheezy) under Hyper-V on x64 so I suspect the ARM aspect of the RPI. I haven’t used cmake before so really I’m not even sure how to debug the build issue but here’s stdout:

[ 8%] [BIFCL] Processing top-k.bif

usage: bifcl [-p] *.bif
make[3]: *** [scripts/base/bif/top-k.bif.bro] Error 1
make[3]: Leaving directory /home/pi/bro/build' make[2]: *** [src/probabilistic/CMakeFiles/bif-alt-probabilistic-top-k.bif.dir/all] Error 2 make[2]: Leaving directory /home/pi/bro/build’
make[1]: *** [all] Error 2
make[1]: Leaving directory `/home/pi/bro/build’
make: *** [all] Error 2

I assume it’s a problem with bifcl so I tried replacing the Bro 2.2 build tree bifcl with the one from the Bro 2.1 build tree but that didn’t work either. (different error when doing that: “Error: cannot open file: -s”) Maybe it’s an option mismatch…?

Has anyone gotten Bro 2.2 to build on ARMv6? Any tricks / tweaks needed? Thanks!

Jonathan

P.S. The new work on file carving is fantastic! Enables all kinds of interesting new scenarios!

It’s not just you. I tried compiling on the RPI about a week ago and encountered the exact same issue. I tried cross compiling for ARM from my laptop but didn’t get very far.

-AK

How does that part look like with "make VERBOSE=1"?

Robin

Sorry for the bad formatting

Linking CXX executable bifcl
cd /opt/bro_src/build/src && /usr/bin/cmake -E cmake_link_script CMakeFiles/bifcl.dir/link.txt --verbose=1
/usr/bin/c++ -Wall -Wno-unused -O2 -g CMakeFiles/bifcl.dir/bif_parse.cc.o CMakeFiles/bifcl.dir/bif_lex.cc.o CMakeFiles/bifcl.dir/bif_arg.cc.o CMakeFiles/bifcl.dir/module_util.cc.o -o bifcl -rdynamic
make[3]: Leaving directory /opt/bro_src/build' /usr/bin/cmake -E cmake_progress_report /opt/bro_src/build/CMakeFiles 8 [ 8%] Built target bifcl make -f src/probabilistic/CMakeFiles/bif-alt-probabilistic-top-k.bif.dir/build.make src/probabilistic/CMakeFiles/bif-alt-probabilistic-top-k.bif.dir/depend make[3]: Entering directory /opt/bro_src/build’
cd /opt/bro_src/build && /usr/bin/cmake -E cmake_depends “Unix Makefiles” /opt/bro_src /opt/bro_src/src/probabilistic /opt/bro_src/build /opt/bro_src/build/src/probabilistic /opt/bro_src/build/src/probabilistic/CMakeFiles/bif-alt-probabilistic-top-k.bif.dir/DependInfo.cmake --color=
Dependee “/opt/bro_src/build/src/probabilistic/CMakeFiles/bif-alt-probabilistic-top-k.bif.dir/DependInfo.cmake” is newer than depender “/opt/bro_src/build/src/probabilistic/CMakeFiles/bif-alt-probabilistic-top-k.bif.dir/depend.internal”.
Dependee “/opt/bro_src/build/src/probabilistic/CMakeFiles/CMakeDirectoryInformation.cmake” is newer than depender “/opt/bro_src/build/src/probabilistic/CMakeFiles/bif-alt-probabilistic-top-k.bif.dir/depend.internal”.
Scanning dependencies of target bif-alt-probabilistic-top-k.bif
make[3]: Leaving directory /opt/bro_src/build' make -f src/probabilistic/CMakeFiles/bif-alt-probabilistic-top-k.bif.dir/build.make src/probabilistic/CMakeFiles/bif-alt-probabilistic-top-k.bif.dir/build make[3]: Entering directory /opt/bro_src/build’
/usr/bin/cmake -E cmake_progress_report /opt/bro_src/build/CMakeFiles
[ 8%] [BIFCL] Processing top-k.bif
cd /opt/bro_src/build/src/probabilistic && …/bifcl -s /opt/bro_src/src/probabilistic/top-k.bif || ( rm -f /opt/bro_src/build/scripts/base/bif/top-k.bif.bro top-k.bif.h top-k.bif.cc top-k.bif.init.cc && exit 1 )
usage: bifcl [-p] *.bif
make[3]: *** [scripts/base/bif/top-k.bif.bro] Error 1
make[3]: Leaving directory /opt/bro_src/build' make[2]: *** [src/probabilistic/CMakeFiles/bif-alt-probabilistic-top-k.bif.dir/all] Error 2 make[2]: Leaving directory /opt/bro_src/build’
make[1]: *** [all] Error 2
make[1]: Leaving directory `/opt/bro_src/build’
make: *** [all] Error 2

Hmm ... No immediate idea. Does this work from the top-level
directory?

# ./build/src/bifcl -s src/probabilistic/top-k.bif
# ls top-k*
top-k.bif.bro top-k.bif.cc top-k.bif.h top-k.bif.init.cc

(Note that bifcl's usage message isn't up to date, it does now in fact
support an -s option.)

Is this a fresh checkout/build, or on top of an older build/install?

Robin

For me, it was a fresh checkout/build. I will try that exact command when I get home tonight. I remember trying a couple different combinations from the build/src directory and not being successful. Not sure I tried exactly that one from exactly there though.

Fresh install here, too. Command output below.

root@raspberrypi:/opt/bro_src# ./build/src/bifcl -s src/probabilistic/top-k.bif
usage: bifcl [-p] .bif
root@raspberrypi:/opt/bro_src# find ./ -iname top-k

./src/probabilistic/top-k.bif

I checked the dependency versions from the RPi’s repos and they are equal to or newer than a vanilla Debian 6 system (which Bro builds fine on).

-AK

Can you try the patch below?

- Jon

diff --git a/src/builtin-func.l b/src/builtin-func.l
index c70b99b..b032fd1 100644
--- a/src/builtin-func.l
+++ b/src/builtin-func.l
@@ -275,7 +275,7 @@ void finish_alternative_mode()

int main(int argc, char* argv[])
        {
- char opt;
+ int opt;

        while ( (opt = getopt(argc, argv, "p:s")) != -1 )
                {

That got past the problem for me. If it ever finished building I'll run the test suite.

.Seth

Thanks Jon, the patch worked for me too. If anyone is interested, running make test on the RPi results in the following failures.

[#1] bifs.bloomfilter … failed
[#1] doc.sphinx.using_bro … failed
[#1] istate.broccoli … failed
[#1] istate.broccoli-ipv6 … failed
[#1] istate.broccoli-ssl … failed
[#1] istate.pybroccoli … failed
[#1] scripts.base.frameworks.cluster.start-it-up … failed
[#1] scripts.base.frameworks.control.configuration_update … failed
[#1] scripts.base.frameworks.control.id_value … failed
[#1] scripts.base.frameworks.control.shutdown … failed
[#1] scripts.base.frameworks.file-analysis.input.basic … failed
[#1] scripts.base.frameworks.input.event … failed
[#1] scripts.base.frameworks.input.optional … failed
[#1] scripts.base.frameworks.input.predicate … failed
[#1] scripts.base.frameworks.input.raw.basic … failed
[#1] scripts.base.frameworks.input.reread … failed
[#1] scripts.base.frameworks.input.sqlite.basic … failed
[#1] scripts.base.frameworks.input.sqlite.error … failed
[#1] scripts.base.frameworks.input.sqlite.port … failed
[#1] scripts.base.frameworks.input.sqlite.types … failed
[#1] scripts.base.frameworks.input.tableevent … failed
[#1] scripts.base.frameworks.intel.cluster-transparency … failed
[#1] scripts.base.frameworks.intel.input-and-match … failed
[#1] scripts.base.frameworks.intel.read-file-dist-cluster … failed
[#1] scripts.base.frameworks.logging.remote-types … failed
[#1] scripts.base.frameworks.logging.sqlite.error … failed
[#1] scripts.base.frameworks.logging.sqlite.types … failed
[#1] scripts.base.frameworks.logging.sqlite.wikipedia … failed
[#1] scripts.base.frameworks.notice.suppression-cluster … failed
[#1] scripts.base.frameworks.sumstats.basic-cluster … failed
[#1] scripts.base.frameworks.sumstats.cluster-intermediate-update … failed
[#1] scripts.base.frameworks.sumstats.on-demand-cluster … failed
[#1] scripts.base.frameworks.sumstats.sample-cluster … failed
[#1] scripts.base.frameworks.sumstats.topk-cluster … failed
[#1] scripts.base.protocols.http.http-methods … failed

Patch worked for me as well. Thanks.

I believe all of these test failures are from the Pi running out of memory. :slight_smile:

  .Seth

What happens if you try to re-run only the failed tests? (using
the "-r" option to btest)

   cd bro/testing/btest
   ../../aux/btest/btest -r

Dan,

The same failures occur.
root@raspberrypi:/opt/bro_src/testing/btest# …/…/aux/btest/btest -r
[ 0%] bifs.bloomfilter … failed
[ 2%] doc.sphinx.using_bro … failed
[ 5%] istate.broccoli … failed
[ 8%] istate.broccoli-ipv6 … failed
[ 11%] istate.broccoli-ssl … failed
[ 14%] istate.pybroccoli … failed
[ 17%] scripts.base.frameworks.cluster.start-it-up … failed
[ 20%] scripts.base.frameworks.control.configuration_update … failed
[ 22%] scripts.base.frameworks.control.id_value … failed
[ 25%] scripts.base.frameworks.control.shutdown … failed
[ 28%] scripts.base.frameworks.file-analysis.input.basic … failed
[ 31%] scripts.base.frameworks.input.event … failed
[ 34%] scripts.base.frameworks.input.optional … failed
[ 37%] scripts.base.frameworks.input.predicate … failed
[ 40%] scripts.base.frameworks.input.raw.basic … failed
[ 45%] scripts.base.frameworks.input.sqlite.basic … failed
[ 48%] scripts.base.frameworks.input.sqlite.error … failed
[ 51%] scripts.base.frameworks.input.sqlite.port … failed
[ 54%] scripts.base.frameworks.input.sqlite.types … failed
[ 57%] scripts.base.frameworks.input.tableevent … failed
[ 60%] scripts.base.frameworks.intel.cluster-transparency … failed
[ 62%] scripts.base.frameworks.intel.input-and-match … failed
[ 65%] scripts.base.frameworks.intel.read-file-dist-cluster … failed
[ 68%] scripts.base.frameworks.logging.remote-types … failed
[ 71%] scripts.base.frameworks.logging.sqlite.error … failed
[ 74%] scripts.base.frameworks.logging.sqlite.types … failed
[ 77%] scripts.base.frameworks.logging.sqlite.wikipedia … failed
[ 80%] scripts.base.frameworks.notice.suppression-cluster … failed
[ 82%] scripts.base.frameworks.sumstats.basic-cluster … failed
[ 85%] scripts.base.frameworks.sumstats.cluster-intermediate-update … failed
[ 88%] scripts.base.frameworks.sumstats.on-demand-cluster … failed
[ 91%] scripts.base.frameworks.sumstats.sample-cluster … failed
[ 94%] scripts.base.frameworks.sumstats.topk-cluster … failed
[ 97%] scripts.base.protocols.http.http-methods … failed
34 of 35 tests failed

I'm almost certain that these are just memory exhaustion. (I think you meant to run that with -d too instead of -r).

  .Seth

Have you tried looking in the diag.log file?
That file contains diagnostic output for all failed tests,
and is automatically created when using "make test" (if
running btest manually, then you need to use the "-f"
option to create that file).