&expire_func/&create_expire question

Zeek
1

Hiya,

Below is a script that I would think should cause the &expire_func to execute, but doesn't. (I would expect the expire function to execute assuming you run the script on a trace that has packets with arrival times separated by more than EXPIRE time, which is set to 1 second below).

2

Hey Mike,

mhmmm sorry but this seems to work here. For comparison, does this more
basic one give you output?

3

We have some progress on the non-triggering of the expiration callback.
It is triggered, but a *long* time after the &create_expire interval.
Below are the timings using a 20 minute trace for this code:

function expire(t: table[count] of count, idx:count): interval
{
        print fmt("%s %s (expire)", current_time(), network_time());
        return 0 sec;
}

global state: table[count] of count &create_expire=1sec &expire_func=expire;
global idx: count = 0;

event new_packet(c: connection, p: pkt_hdr)
{
        ++idx;
        state[idx] = idx;
        print fmt("%s %s", current_time(), network_time());
}

4

Real time is out, since 1s hasn't yet passed. But network time has
advanced 91s before I see the callback!?

Timer expiration is driven off of packet arrivals. Is there a lengthy
lull in arriving packets that causes the 91 second delay?

    Vern

5

Uh-oh. I've discovered that packets in that trace were not in
chronological order, sorry. So I switched to one that covers ~3 minutes,
definitely sorted and without substantial gaps, and now the delay is at
60s, when a burst of expirations is triggered.

I've uploaded the stdout output I get with that trace and the below code
at http://www.cl.cam.ac.uk/~cpk25/bro/expiration-log.txt.gz .

6

Another factoid: for *any* &create_expire delay between 1s and 60s
(inclusive) the first expiration is triggered at exactly the same time,
1039100508.06149. Once it's at 61 seconds, the first expiration is at
1039100568.33063 -- pushed back by another minute.

Cheers,
Christian.

7

To clarify how table expiration works: we do not install an
individual timer for every table entry; that would way too many.
Instead, every table gets *one* timer which periodically triggers
the expiration of all outdated entries. By default, this is done
every 10s (table_expire_interval). Furthermore, when entries are
expired, only 5000 (table_incremental_step) are expired in a row,
then a delay of 0.01 (table_expire_delay) is inserted to avoid
dropping packets.

Robin