Sending logs to remote public cloud entity

Zeek
1

I am looking into setting up a Bro manager in AWS cloud, which will receive logs from
multiple Bro workers on premise. I then plan to take the logs received on the manager and
load them into my database on AWS. The Bro manager itself is merely to receive the logs
and does not generate logs of its own.

I read some documentation about Broccoli whereby I can configure an SSL tunnel by furnishing the manager
with the a public cert, key and CA. I have not found much documentation nor
discussion on this kind of a setup usage.

I am wondering:

  • is this the recommended approach to send logs to a remote public cloud entity
  • the alternative is to send syslog’s but then I would need to set up stunnel or some other
    encrypted tunneling, which instead I am hoping to leverage the Broccoli SSL tunneling functionality
  • will the Bro manager scale to receive logs from multiple workers (like 10)
  • I can work around this by running multiple Bro managers listening on different ports
  • will the logs be written into the normal place on disk with the default writer

Thanks in advance for your input.

Shankar

2

I read some documentation about Broccoli whereby I can configure an SSL tunnel by furnishing the manager
with the a public cert, key and CA. I have not found much documentation nor
discussion on this kind of a setup usage.

Broccoli is going to be marked as deprecated beginning with the next release so it’s on it’s last legs at the moment. There also isn’t a way with Broccoli to hook into the remote logging. Only Bro can send or receive logs remotely. Our replacement mechanism for Broccoli however will actually be able to send and receive logs remotely in non-Bro processes.

- is this the recommended approach to send logs to a remote public cloud entity

There isn’t a recommended approach to this at the moment. I know of some companies using Bro and forwarding logs off to public cloud servers but they tend to compress and shuttle logs in bulk over other mechanisms (scp for example). I don’t know of anyone streaming logs off to cloud servers.

  - the alternative is to send syslog’s but then I would need to set up stunnel or some other

Yeah, that doesn’t sound fun. We also don’t support writing logs directly to syslog so you’d still end up running something else to pick the logs up off the disk and forward them off to the remote server.

- will the Bro manager scale to receive logs from multiple workers (like 10)
  - I can work around this by running multiple Bro managers listening on different ports


Yes, this is fine. There are quite a few people around that have dozens of Bro processes sending logs to the manager and it takes them just fine.

- will the logs be written into the normal place on disk with the default writer

Yep.

.Seth