I've got a question on tm's RAM usage, and I was hoping someone could
point me in the right direction: I'm trying to get as much duration
as possible out of tm so that I can go back many hours or even days
for packets. I have a lot of disk to throw at it, and a fair amount
of RAM. The problem I'm running into is that when I move the
conn_timeout up to 86400 but keep the mem settings low, tm still
consumes a massive amount of RAM. I am concluding that the RAM usage
must be the connection tables, and not the mem setting for the traffic
class. Is there a way to allow tm to maximize for longevity? My
understanding is that if I move the conn_timeout down, those packets
will not be available for query.
That's what I originally thought. What was throwing me was when I
would try to find packets any older than the cutoff, the queries would
come up empty, the log showing something like "query not found in
connection table." So I ran "show conn sample" to see the connections
table, and the oldest connections were always at the cutoff. When I
looked through the source code, it appeared that connections older
than the cutoff were evicted from the connections table, but the query
depended on the connections table to find the packets on disk/ram.
That sound's weird. I'm going to look into that.
Which kind of query did you use?
Can you maybe copy-paste a sample query plus the error message into an
e-mail?
looking at the source code, it seems that the message 'not found in
connection table' is related to subscriptions (i.e., the query request
that all future packets for this connections should be included in the
query results. And subscriptions only work for connections that are
currently active). So this message is ok. (Actually it is commented out
in the current svn-snapshot. Did you uncomment it or was it in your
version of the TM source code).
As others already pointed out the conn_timeout is indeed the idle time
until a connection is expired from the connection table (we only use the
timeout to expire connections). Setting this to high value is
counter-productive: the memory consumption increases significantly.
Furthermore, long timeouts will reduce visibility. No new packets will
be recorded for connections (actually 5-tuples) that aren't expired but
have exceeded the cutoff. So long timeouts can be problematic in the
case of 5-tuple reuse.
To check your current retention times, you can check the classes.tm.log
file. mem_dt and disk_dt will tell you how many seconds of packet data
are currently retained in memory and on disk. Can you check whether the
packets you want to retrieve fall into this time-frame?
Thanks for looking into it. I blew away all of my buffers and indexes
and restarted with more sane settings and queries seem to be behaving.
I'm certainly not ruling out user error. In any case, thanks for
your help, it's very appreciated.