I’ve successfully run smaller Bro clusters, but now that I’m scaling out I’m seeing the manager and logger threads crash immediately when I deploy the configuration. What I’m trying to run: 1 manager, 1 logger on 1 host 8 proxies and 32 workers on 8 hosts I’m using Bro 2.5.1. Each worker h…

8 total proxies, 32 total workers. (1 proxy node + 4 worker nodes) * 8 hosts

Okay, I think I’m following now, but I want to restate it so that other’s with more large cluster experience can chime in. 1 physical host = manager and logger 8 x physical hosts = proxy + 4*(worker w/ 9lb_procs) I’m not sure if having multiple worker nodes per physical host is all that common. I…

Yes.. this is a problem. Bro currently uses select() internally for the IO loop and select can't handle more than 1024 file descriptors. Around 170 worker processes is where the manager will accumulate more than 1024 fds. There are a few options here: * Run less lb_procs per port to stay under t…

Manager and logger threads crash immediately on deploy

Zeek

dopheide July 12, 2017, 6:56pm 2

Is that (8 proxies and 32 workers) on ALL 8 hosts? For 64 total proxies?

That seems like a lot to me.

Topic		Replies	Views
Bro manager dies in a large cluster Zeek	1	131	May 6, 2022
Manager swapping.. Zeek	11	160	May 6, 2022
cluster manager crash Zeek	7	167	May 6, 2022
Bro's limitations with high worker count and memory exhaustion Zeek	18	161	May 6, 2022
Bro cluster requirements and manager logging backlog bug Zeek	34	215	May 6, 2022

Manager and logger threads crash immediately on deploy

Related topics