However, the numbers I get soon become negative resp. I get a runtime
error - counter negative. A quick check showed me that
connection_state_removed gets thrown up to four times per connection
in only the first few minutes of my trace.
I then tried to replace connection_state_removed() with
connection_reset() and connection_finished(). However I am not
convinced this is enough because even after more then 90 minutes trace
time concurrent_conn_count is still increasing significantly (~1300
per minute on a 1 Gig uplink).
So my question now is: which events are thrown when exactly? Do I have
to track the established connections in the scripting layer? Is there
a way to just query for the size of the bro-internal connection
tracker?
BTW: I am using a header trace. In my opinion this shouldn't make a
difference, but maybe ...
So my question now is: which events are thrown when exactly?
The event you're looking for is new_connection(). That one is raised
for all connections for which Bro instantiates internal state, i.e.,
it's the counterpart of connection_state_remove().
The other connection events are only raised for a subset of all
connections. connection_established() for those with a full 3w
handshake, connection_finished() for regular tear-downs,
connection_reset() for connection aborted with a reset, etc.
Is there
a way to just query for the size of the bro-internal connection
tracker?
Actually there is: the built-in resource_usage() returns a record
which, among other stuff, contains the numbers of TCP, UDP, ICMP
connections in memory. Caveat: I'm just realizing that this
reporting doesn't take the connection-compressor into account, which
means that by default the values will be too small for TCP
connections. Turning off the compressor with
use_connection_compressor=F will fix that for the cost of some
performance decrease (both cpu and memory).
BTW: I am using a header trace. In my opinion this shouldn't make a
difference, but maybe ...
The event you're looking for is new_connection(). That one is raised
for all connections for which Bro instantiates internal state, i.e.,
it's the counterpart of connection_state_remove().
No, it is not I only want fully established tcp connections. I
tried out new_connection() however, and it gives me about 8 times more
connections than there are fully-established tcp-connections (450k vs.
60k). By the way, I got my numbers now by using
connection_established() to detect new connections,
connection_state_remove() for decreasing the counter and a set of
conn_id to ensure that a connection is removed only once. The price -
of course - is the memory consumption of the extra table.
Actually there is: the built-in resource_usage() returns a record
which, among other stuff, contains the numbers of TCP, UDP, ICMP
connections in memory.
I tried out the built-in resource_usage() as well, it gives pretty
much the same results as the new_connection() approach:
However, sometimes, odd things happen. Like here, where
resource_usage()$max_TCP_conns almost doubles for a short period of
time (this is still in the startup phase):
No, it is not I only want fully established tcp connections. I
(Ok, sorry, I misunderstood you and thought that you do want to
count all connections.)
connection_established() to detect new connections,
connection_state_remove() for decreasing the counter and a set of
conn_id to ensure that a connection is removed only once.
Yeah, that's actually the best approach then.
I tried out the built-in resource_usage() as well, it gives pretty
much the same results as the new_connection() approach:
Right, that makes sense because it also counts all currently active
connection independent of their state.
After looking into the code this seems to happen exactly when the
underlying PDict object does a table resize.
Yepp, that looks indeed like a bug in the accounting code. While the
resize is on its way, there are actually two tables kept internally
and it seems the code calculates the max size wrong during this
time.
I only want fully established tcp connections. I