I’ve noticed on our bro cluster that the proxy keeps crashing and restarting. The cluster seems to be working, logs are being written, etc. Has anyone seen this behavior? What am I missing here?
From the crash report:
internal error: unknown msg type 115 in Poll()
/opt/bro/share/broctl/scripts/run-bro: line 85: 18479 Aborted (core dumped) nohup $mybro “$@”
It attempts to restart it, but the status always shows with ??? in the Peers column:
proxy-1 proxy 22.214.171.124 running 12386 ??? 04 Jun 07:30:05
This cluster has two hosts, one proxy, 12 workers, and sees 900+ mbps of traffic.
Would it make more sense to have a proxy on each host?
This is communication system overload. One of the many reasons we've been planning to replace the communication infrastructure in Bro.
Try adding another proxy to your cluster.
Updated the cluster to run a proxy on each host now, and so far so good, no crashes.
Cool, the general rule of thumb I've been giving people lately is somewhere around 14-15 workers per proxy.
Is that with a dedicated proxy? I now have manager, proxy, and six workers on one host; proxy and six workers on the other.
Ah, it could be a system resource issue causing trouble for you too.