I’ve noticed on our bro cluster that the proxy keeps crashing and restarting. The cluster seems to be working, logs are being written, etc. Has anyone seen this behavior? What am I missing here?
From the crash report:
internal error: unknown msg type 115 in Poll()
/opt/bro/share/broctl/scripts/run-bro: line 85: 18479 Aborted (core dumped) nohup $mybro “$@”
It attempts to restart it, but the status always shows with ??? in the Peers column:
proxy-1 proxy 1.2.3.4 running 12386 ??? 04 Jun 07:30:05
Thanks,
matt
This cluster has two hosts, one proxy, 12 workers, and sees 900+ mbps of traffic.
Would it make more sense to have a proxy on each host?
This is communication system overload. One of the many reasons we've been planning to replace the communication infrastructure in Bro.
Try adding another proxy to your cluster.
.Seth
Updated the cluster to run a proxy on each host now, and so far so good, no crashes.
Thanks,
-matt
Cool, the general rule of thumb I've been giving people lately is somewhere around 14-15 workers per proxy.
.Seth
Is that with a dedicated proxy? I now have manager, proxy, and six workers on one host; proxy and six workers on the other.
Ah, it could be a system resource issue causing trouble for you too.
.Seth