non-deterministic notice_policy order

Development
1

I saw that the suppression-disable.bro test periodically failed because the order of elements in Notice::ordered_policy varies between runs (run `bro -e ''` multiple times, looking at notice_policy.log each time for differences). And I think the reason for that variance is because the table/set hashing is partly computed based on the memory address of PolicyItem$pred functions.

If that's right, is there an idea of how to make hash computations for functions deterministic?

- Jon

2

How about printing out the function's code and hashing that when we're
in "deterministic hashing mode" (i.e., have_random_seed() return
true)?

Robin

3

How about printing out the function's code and hashing that when we're
in "deterministic hashing mode" (i.e., have_random_seed() return
true)?

Yeah, I think that would work as far as making the test work consistently, but I think we want a fix that will always be in effect so that the ordering of Notice::policy's PolicyItem's with equal priority are applied in a consistent order during actual operation?

- Jon

4

Makes sense. How about (always) hashing function name and location
information?

Robin

5

I would have expected that to be true already due to the seeding of the random number generator. Here's the code that writes the notice policy to the logging stream...

  local tmp: table[count] of set[PolicyItem] = table();
  for ( pi in policy )
    {
    if ( pi$priority < 0 || pi$priority > 10 )
      Reporter::fatal("All Notice::PolicyItem priorities must be within 0 and 10");
      
    if ( pi$priority !in tmp )
      tmp[pi$priority] = set();
    add tmp[pi$priority][pi];
    }
  
  local rev_count = vector(10,9,8,7,6,5,4,3,2,1,0);
  for ( i in rev_count )
    {
    local j = rev_count[i];
    if ( j in tmp )
      {
      for ( pi in tmp[j] )
        {
        pi$position = |ordered_policy|;
        ordered_policy[|ordered_policy|] = pi;
        Log::write(Notice::POLICY_LOG, pi);
        }
      }
    }

All it's doing is taking the unordered set of PolicyItems and turning it into a vector ordered by the $priority field. It's surprising to me that due to the seeding we're seeing the order of the log change. It's the index value of the vector changing that's causing the difference in notice.log on the line just before the Log::write call.

  .Seth