Zeek Broker Python Module Fails | Python3.11

Zeek
1

Hi All,

I’m trying to install zeek broker python module using python3.11 on a red hat system. I’m following the official zeek documentation here 6. Python Bindings — Broker User Manual

which includes the steps:

virtualenv -p python3 /Users/user/sandbox/broker/venv . /Users/user/sandbox/broker/venv/bin/activate
./configure --prefix=/Users/user/sandbox/broker --python-prefix=(python3 -c ‘import sys; print(sys.exec_prefix)’)
make install python3 -c ‘import broker; print(broker.file)’
/Users/user/sandbox/broker/venv/lib/python3.7/site-packages/broker/init.py

It works perfectly of python3.6 and I was successfully able to use broker module in python but as soon as I switched to python3.11 the compilation failed at this step:

[ 21%] Building CXX object auxil/broker/bindings/python/CMakeFiles/_broker.dir/_broker.cpp.o
[ 21%] Building CXX object auxil/broker/bindings/python/CMakeFiles/_broker.dir/data.cpp.o
[ 21%] Building CXX object auxil/broker/bindings/python/CMakeFiles/_broker.dir/enums.cpp.o
[ 21%] Building CXX object auxil/broker/bindings/python/CMakeFiles/_broker.dir/store.cpp.o
[ 21%] Building CXX object auxil/broker/bindings/python/CMakeFiles/_broker.dir/zeek.cpp.o
[ 21%] Linking CXX shared module …/…/python/broker/_broker.so
/usr/bin/ld: /usr/local/lib/python3.11/config-3.11-x86_64-linux-gnu/libpython3.11.a(abstract.o): relocation R_X86_64_32 against .rodata.str1.8' can not be used when making a shared object; recompile with -fPIC /usr/bin/ld: /usr/local/lib/python3.11/config-3.11-x86_64-linux-gnu/libpython3.11.a(boolobject.o): relocation R_X86_64_32S against symbol _Py_TrueStruct’ can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /usr/local/lib/python3.11/config-3.11-x86_64-linux-gnu/libpython3.11.a(bytearrayobject.o): relocation R_X86_64_32S against symbol `PyByteArray_Type’ can not be used when making a shared object; recompile with -fPIC

I tried by putting CXXFLAGS=‘-fPIC’ while running the configure command but that also gave the same error. What can be the reason for that?

Thanks,

2

Hello @ashish21 - what stands out here a bit is that the /usr/bin/ld line shows a static libpython library: libpython3.11.a. Is that a custom Python 3.11 build? Is there a .so file available, too?

The following is compiling latest broker within Ubuntu 22.10 using python3.11 (without an venv) and that seems to work out fine. Would it be possible for you to create a reproducer with Docker or otherwise that makes it easier to reproduce the issue?

FROM ubuntu:22.10

RUN apt-get update && apt-get install -y --no-install-recommends \
        build-essential \
        ca-certificates \
        cmake \
        git \
        libssl-dev \
        python3.11 \
        python3.11-dev \
        python3.11-distutils


RUN update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.11 2
RUN python3 --version

WORKDIR /src
RUN git clone --recurse-submodules https://github.com/zeek/broker.git
WORKDIR /src/broker
RUN ./configure
RUN make -j8
RUN make install
RUN python3 -c 'import broker, sys; print(sys.version); print(broker)'

Output of the last line:

#14 0.237 3.11.0rc2 (main, Sep 12 2022, 16:20:24) [GCC 12.2.0]
#14 0.237 <module 'broker' from '/usr/lib/python3/dist-packages/broker/__init__.py'>
3

Hi @awelzel,

Thank you for your response. Actually it was to do with the Python3.11.1 installation. I manually installed the python and did not provide --enabled-shared option while running ./configure.

After compiling with that option the problem was solved!!

Thanks,
Ashish

1 Like