zkg unbundle fails

Zeek
1

Hello,

I have a IDS cluster with Zeek and I try to add some packages using the Zeek Package Manager.
The thing is that this cluster has not internet access, so I try to use the bundle/unbundle method.
I have a different system with zeek installed where I have done the package installation, and then zkg bundle.
When I do the unbundle on my Zeek Manager node I get the following error:

[zeek@idslog01 ~]$ zkg -vvv unbundle zeek-packages.bundle
2020-10-19 18:00:38 DEBUG init Manager version 2.3.1
2020-10-19 18:00:38 DEBUG creating source clone of “zeek” at /opt/zeek/.zkg/clones/source/zeek
2020-10-19 18:00:38 WARNING failed to clone git repo: ‘git clone --recursive --depth=1 https://github.com/zeek/packages /opt/zeek/.zkg/clones/source/zeek --no-single-branch’ returned with exit code 128
stderr: ‘fatal: unable to access ‘https://github.com/zeek/packages/’: Failed to connect to 140.82.118.4: Network is unreachable’
warning: skipped using package source named “zeek”: failed to clone git repo
2020-10-19 18:00:38 DEBUG getting bundle info for file “zeek-packages.bundle”
2020-10-19 18:00:38 DEBUG getting info for bundled package “hassh”
2020-10-19 18:00:38 DEBUG getting info on “/opt/zeek/.zkg/scratch/bundle/hassh”
2020-10-19 18:00:38 INFO getting info on “/opt/zeek/.zkg/scratch/bundle/hassh”: invalid git repo path: ‘git fetch --tags --no-recurse-submodules’ returned with exit code 128
stderr: 'fatal: attempt to fetch/clone from a shallow repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.’
2020-10-19 18:00:38 INFO getting info on “/opt/zeek/.zkg/scratch/bundle/hassh”: matched no source package
2020-10-19 18:00:38 DEBUG getting info for bundled package “ja3”
2020-10-19 18:00:38 DEBUG getting info on “/opt/zeek/.zkg/scratch/bundle/ja3”
2020-10-19 18:00:38 INFO getting info on “/opt/zeek/.zkg/scratch/bundle/ja3”: invalid git repo path: ‘git fetch --tags --no-recurse-submodules’ returned with exit code 128
stderr: 'fatal: attempt to fetch/clone from a shallow repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.’
2020-10-19 18:00:38 INFO getting info on “/opt/zeek/.zkg/scratch/bundle/ja3”: matched no source package
error: bundle zeek-packages.bundle contains invalid package /opt/zeek/.zkg/scratch/bundle/hassh: package name not found in sources and also not a usable git URL (invalid or inaccessible, use -vvv for details)

Zeek itself and the package manager are installed using RPM.
Zeek version 3.0.11
zkg version 2.3.1

Could someone help me getting this fixed, or help me understand what I’m doing wrong?

jebfpocigebbmeoo.png
jebfpocigebbmeoo.png752×66 13.7 KB

2

Hi Jan,

There are two things going on here. The network connectivity errors are due to zkg's startup behavior: upon startup it checks whether it has an internal copy of the configured package repositories ("package sources"). In your case zkg cannot find its internal copy of the standard package source, so proceeds to clone it, which fails. The error has nothing to do with the unbundling itself, and isn't fatal.

The second problem is more subtle. If I'm reading things correctly, your git version is dated and zkg hits problems in its handling of shallow repos (which are in play here, and for which zkg has some precautions ... but that doesn't seem to catch here).

What does git --version say on your system?

Thanks,
Christian

3

Hi Christian,

[jhp@idslog01 ~]$ git --version
git version 1.8.3.1

It’s a Centos7 server.

Jan Hugo Prins

jebfpocigebbmeoo.png
jebfpocigebbmeoo.png752×66 13.7 KB

4

Thanks Jan. Yeah that's a wee bit dusty (released in 2013) ... I'll try to reproduce with this and follow up.

Best,
Christian

5

Confirmed Jan, I get the same error on Centos 7. I've created a PR with a potential fix, so you can follow along there:

https://github.com/zeek/package-manager/pull/77

Best,
Christian

6

zkg v2.4.1 is available now and should fix the issue / behave better
with older versions of Git.

- Jon