I've managed to build Bro on CentOS 5 successfully after a bit of work to get needed dependencies in place, but I can't seem to get around an issue where Bro complains about not being able to find any valid magic files when it starts. I get this error:
internal error: can't load magic file <default>: could not find any valid magic files!
I have my Bro executable installed to /opt/dcod/bin with all of the magic files from Bro in /opt/dcod/share/bro/magic (e.g., /opt/dcod/share/bro/magic/animation, /opt/dcod/share/bro/magic/archive, etc.). Permissions are fine. The "libmagic mime magic database search path" is set to /opt/dcod/share/bro/magic (Bro has actually been built with that path in it, but I've also set the BROMAGIC environment variable to the same for the hell of it to no avail).
Any ideas? Is the search path for libmagic set correctly? I've been fighting with this problem for quite a while now.
I've had the same problem when upgrading to libmagic 5.18 (file-5.18), using Bro 2.2.
It worked fine with file-5.16.
Are there any quick fixes out there without having to downgrade?
I did discover a solution based on the following bug report, though I'm still fighting through some other issues.
https://bro-tracker.atlassian.net/browse/BIT-1111
Basically, you have to also (or rather?) set the MAGIC environment variable, not just the BROMAGIC environment variable.
My conundrum seems to be happening because I need to be able to run on CentOS 5 (or RHEL 5), but that distro is stuck with libmagic version 4, and I can't figure out how to upgrade the system to version 5 without ruining a bunch of things.
I did discover a solution based on the following bug report, though I'm still fighting through some other issues.
[BIT-1111] - Bro Tracker
Basically, you have to also (or rather?) set the MAGIC environment variable, not just the BROMAGIC environment variable.
I’d suggest just setting both to point at the same libmagic installation (e.g. the one Bro has been configured to use via `./configure —with-libmagic=<path>`).
My conundrum seems to be happening because I need to be able to run on CentOS 5 (or RHEL 5), but that distro is stuck with libmagic version 4, and I can't figure out how to upgrade the system to version 5 without ruining a bunch of things.
If you build/install the latest libmagic from source doing something like `./configure —prefix=/opt && make && make install` (you can choose whatever prefix you want), does that work for you as far as isolating that version from everything?
- Jon
Thanks for the thoughts. This is kind of what I'm working towards, though my complication is that I'm building an RPM that incorporates libmagic, which is built and temporarily installed into the RPM build root. If I want to have libmagic installed to somewhere in /opt, my RPM build is not very clean anymore. Does anyone know if there's a way to easily statically link Bro against libmagic? That would take some of this headache away.
Another thing I'm baffled about is that when I build libmagic 5, the resulting shared library and symbolic links have the same filenames as the libmagic 4 that's already installed. How can this be?
[dcod@localhost lib]$ ls -al /usr/lib64 | grep libmagic
-rw-r--r-- 1 root root 105348 Jun 22 2012 libmagic.a
lrwxrwxrwx 1 root root 17 Apr 1 12:22 libmagic.so -> libmagic.so.1.0.0
lrwxrwxrwx 1 root root 17 Apr 1 12:22 libmagic.so.1 -> libmagic.so.1.0.0
-rwxr-xr-x 1 root root 65608 Jun 22 2012 libmagic.so.1.0.0
[dcod@localhost lib]$ ls -al
total 340
drwxr-xr-x 2 dcod dcod 4096 May 7 16:37 .
drwxr-xr-x 6 dcod dcod 4096 May 7 16:37 ..
-rwxr-xr-x 1 dcod dcod 921 May 7 16:37 libmagic.la
lrwxrwxrwx 1 dcod dcod 17 May 7 16:37 libmagic.so -> libmagic.so.1.0.0
lrwxrwxrwx 1 dcod dcod 17 May 7 16:37 libmagic.so.1 -> libmagic.so.1.0.0
-rwxr-xr-x 1 dcod dcod 331757 May 7 16:37 libmagic.so.1.0.0
I think I've figured out a solution in case anyone is interested, and it's basically what Jon recommended--install libmagic 5 from source to somewhere so that it doesn't overwrite the existing libmagic 4 packaged installation already present (I installed to /usr/local, which works fine).
As for my question about the shared library version of libmagic not changing between version 4 and 5 of the project, I guess that's because the interface did not change at all (which seems unlikely, but it's possible).
If it were possible to arrange things so that only libmagic.a is in the paths being searched, I think Bro would just go with that. Or a quick/dirty way would be to edit Bro’s cmake/FindLibMagic.cmake to unconditionally do [1] instead of just when on Darwin.
Also just FYI, Bro 2.3 won’t depend on libmagic.
- Jon
[1] https://github.com/bro/cmake/blob/master/FindLibMagic.cmake#L29
Good to hear! Libmagic in general seems a little hokey given how it must resolve the magic file. Thanks.