[localhost]$ sudo broctl netstats
worker-1: <error: cannot connect to 192.168.0.5:47763>
Everything seems to be running like it should though: (Except the ???'s)
All of the output indicates that there is either a problem with your broccoli-python bindings, a firewall issue (not likely in your case since they all seem to be running on a single host), or there could be other Bro processes that have accidentally been forgotten about. To help debug this, could you send me…
- The content of node.cfg
- The output from the "ps.bro" command in broctl
- A snippet from your manager's communication.log when you try to run "netstats".
You might also want to try removing the old installation and reinstalling (save your site/ directory!). I'm starting to suspect that something may have happened recently that is causing this to be a problem with the broccoli-python bindings if you reinstall in place.
Also, I am wondering what kind of issues I might run into managing
several geographically disparate clusters from a single manager.
Currently, I have each setup as a separate bro cluster. I am most
concerned about the amount of traffic and possible congestion this
might cause.
This is a very similar deployment model to the deep cluster we've been talking about for a little while but this is more of a shallow cluster model. I don't have any experience yet with people using remote managers, I suppose a lot of potential performance problems could come from the workers -> manager connection not being fast enough. I'd be glad to work on it directly with you, it would be great to finally get some relevant experience with that deployment model.
Is there a way to measure the amount of traffic between
the workers and manager if all are on the same server?
You can always run tcpdump on your loopback interface. Capstats should even work on the loopback interface. Unfortunately, you'd only be able to filter down easily to traffic that is being sent to your manager. Traffic sourced from your manager process would be a bit harder, but there isn't much of that fortunately.
[localhost]$ sudo broctl netstats
worker-1: <error: cannot connect to 192.168.0.5:47763>
Everything seems to be running like it should though: (Except the ???'s)
All of the output indicates that there is either a problem with your broccoli-python bindings, a firewall issue (not likely in your case since they all seem to be running on a single host), or there could be other Bro processes that have accidentally been forgotten about.
Just FYI, Seth nailed it. I had re-installed new bro over old bro
before stopping the running processes. I had about 16 processes still
running from back on Christmas day that were keeping the new processes
from binding to the port. That's what I get for trying to work at
family get togethers...sheesh.
Good to hear that it's working for you now. This is actually a problem we're planning on addressing too. The related ticket is here: http://tracker.bro-ids.org/bro/ticket/253
I don't have any experience yet with people using remote managers, I suppose a lot of potential performance problems could come from the workers -> manager connection not being fast enough. I'd be glad to work on it directly with you, it would be great to finally get some relevant experience with that deployment model.