can I send an opaque of bloomfilter over Cluster::manager2worker_event ?

Development
1

I tried doing that and then merging with an existing (initialized) bloomfilter on worker.

I see this error:

1493427133.170419 Reporter::INFO calling inside the m_w_add_bloom worker-1 -
1493427133.170419 Reporter::ERROR incompatible hashers in BasicBloomFilter merge (empty) -
1493427133.170419 Reporter::ERROR failed to merge Bloom filter (empty) -
1493427115.582247 Reporter::INFO calling inside the m_w_add_bloom worker-6 -
1493427115.582247 Reporter::ERROR incompatible hashers in BasicBloomFilter merge (empty) -
1493427115.582247 Reporter::ERROR failed to merge Bloom filter (empty) -
1493427116.358858 Reporter::INFO calling inside the m_w_add_bloom worker-20 -
1493427116.358858 Reporter::ERROR incompatible hashers in BasicBloomFilter merge (empty) -
1493427116.358858 Reporter::ERROR failed to merge Bloom filter (empty) -
1493427115.935649 Reporter::INFO calling inside the m_w_add_bloom worker-7 -
1493427115.935649 Reporter::ERROR incompatible hashers in BasicBloomFilter merge (empty) -
1493427115.935649 Reporter::ERROR failed to merge Bloom filter (empty) -
1493427115.686241 Reporter::INFO calling inside the m_w_add_bloom worker-16 -
1493427115.686241 Reporter::ERROR incompatible hashers in BasicBloomFilter merge (empty) -
1493427115.686241 Reporter::ERROR failed to merge Bloom filter (empty) -
14934271

Not sure if the error is because an opaque of bloomfilter cannot be sent over worker2manager_events and manager2worker_events or if I am doing something not quite right.

Aashish

2

Bloom filters should work over communication. What's the code for the
two sides? The error messages sound like these are filters of
different types.

Robin

3

1493427133.170419 Reporter::ERROR incompatible hashers in BasicBloomFilter merge (empty) -

Not sure if the error is because an opaque of bloomfilter cannot be
sent over worker2manager_events and manager2worker_events or if I am
doing something not quite right.

Bloom filters should work over communication. What's the code for the
two sides? The error messages sound like these are filters of
different types.

Actually - I am not sure if we ever implemented consistent hashing over the cluster; if I remember it correctly, the bloom filters need to use the same seed for the hash-functions in all cluster nodes to be mergeable.

That might be the reason for this to fail (and the error message also seems to indicate this).

Johanna

4

Ah, good point, we did implement that, but it needs to be configured:

    ## Seed for hashes computed internally for probabilistic data structures. Using
    ## the same value here will make the hashes compatible between independent Bro
    q## instances. If left unset, Bro will use a temporary local seed.
    const global_hash_seed: string = "" &redef;

Robin

5

    const global_hash_seed: string = "" &redef;

Yes, with setting of global_hash_seed, bloomfilter movement across workers is working fine and as expected, I see from initial tests.

While we are on this thread, is the following good or there is a better way to copy/merge bloomfilter once its sent to workers:

@if (( Cluster::is_enabled() && Cluster::local_node_type() != Cluster::MANAGER ) || !Cluster::is_enabled())

event Blacklist::m_w_add_bloom(val: opaque of bloomfilter)
{
        blacklistbloom=bloomfilter_merge(val, val);

}
@endif

I figured merging same bloom to itself won't make a difference, I primarily want to copy it to blacklistbloom.

Aashish

6

Speaking of this, I think we still need to have broctl autogenerate a file with this configured to a random value when it starts up (if that file doesn't already exist). That way everyones cluster will end up with a random value that stays consistent across restarts.

  .Seth

7

I figured merging same bloom to itself won't make a difference, I
primarily want to copy it to blacklistbloom.

That's correct behavioar. A bloomfilter merge is idempotent when
executed in reflexive fashion (because it boils down to a bitwise OR of
a bitvector with itself).

    Matthias