known_*

Zeek
1

Hi everybody,

there are a number of scripts (known_services, known_hosts, known_certs) which are implemented both using a broker store and sending broker events. It is possible to switch from one mode to the other using the option use_service_store.

Is there any particular reason for this? Is one option more efficient than the other?

Thanks,

Mauro

2

I think it can get kind of confusing with the naming of different functions.

For instance, in bro_init() you’ll see Cluster::create_store(), but later you’ll see Broker::put_unique(). Those are both required bits for the data store to work and the Cluster* functions use Broker underneath.

The variables like use_service_store toggle whether or not to use the data store method for persistent data or if you’d rather just use Cluster::publish_hrw. I imagine that’s faster because it’s doing less, then then you obviously lose the persistence.

-Dop

3

   there are a number of scripts (known_services, known_hosts, known_certs) which are implemented both using a broker store and sending broker events. It is possible to switch from one mode to the other using the option use_service_store.

Is there any particular reason for this?

Mostly due to not knowing which version better fits the common use-case.

Is one option more efficient than the other?

No, there's two choices because there's a trade-off: using the Broker
store version is theoretically less scalable because it relies on a
single, centralized node, but the version using events partitions the
data across many nodes in a consistent way via HRW. With the Broker
store version you have the option of turning on persistence (e.g.
saving data across restarts can prevent duplicate logs, etc.). With
the version using events there's no option for persistence, although
there's nothing preventing one from combining the two approaches
except complexity of implementing it right: partitioned events +
storing data inside a Broker store.

- Jon