Intermittent bro crashes

Good morning!

We have been seeing occasional core dumps from bro, currently running on 2.5-870. We’ve tried a few things to reproduce it on-demand but haven’t been successful. We were wondering if you might have some insight into the crash. This is the backtrace we get:

#0 TableEntryVal::ExpireAccessTime (this=0x9b349bd32ebb5614) at /hostname/bro-devel-src/src/Val.h:741

#1 TableVal::DoExpire (this=0x3955a40, t=1539447574.1403711) at /hostname/bro-devel-src/src/Val.cc:2353

#2 0x00000000006a34cb in PQ_TimerMgr::DoAdvance (this=0x2e520c0, new_t=1539447574.1403711, max_expire=300) at /hostname/bro-devel-src/src/Timer.cc:166

#3 0x0000000000664d38 in expire_timers (src_ps=src_ps@entry=0x2eab8c0) at /hostname/bro-devel-src/src/Net.cc:232

#4 0x0000000000664de1 in net_packet_dispatch (t=1539447574.1403711, pkt=pkt@entry=0x2eab8f0, src_ps=src_ps@entry=0x2eab8c0) at /hostname/bro-devel-src/src/Net.cc:249

#5 0x0000000000925a89 in iosource::PktSrc::Process (this=0x2eab8c0) at /hostname/bro-devel-src/src/iosource/PktSrc.cc:263

#6 0x00000000006652c5 in net_run () at /hostname/bro-devel-src/src/Net.cc:315

#7 0x00000000005acab1 in main (argc=, argv=) at /hostname/bro-devel-src/src/main.cc:1227

Thank you for your help!

–Tim

We have been seeing occasional core dumps from bro, currently running on 2.5-870.

May be nice to try most recent master version to see if it still pops up.

We’ve tried a few things to reproduce it on-demand but haven’t been successful. We were wondering if you might have some insight into the crash. This is the backtrace we get:

#0 TableEntryVal::ExpireAccessTime (this=0x9b349bd32ebb5614) at /hostname/bro-devel-src/src/Val.h:741

#1 TableVal::DoExpire (this=0x3955a40, t=1539447574.1403711) at /hostname/bro-devel-src/src/Val.cc:2353

I don't see an immediate culprit there -- it's the table entry
expiration algorithm that I guess is well-traveled by most people
running Bro, so maybe need to work on narrowing down a way to
reproduce or what specific script is triggering it. Do you have any
custom scripts that make use of &expire_func or &{read, write,
create}_expire table attributes? That could be a first place to
inspect. I also fixed a bug [1] in related code just last week, but
I'd expect that to give a different stack trace if it was the same
problem here (still doesn't hurt to try to rule that out as a
contributing factor by testing w/ latest git/master).

- Jon

[1] https://github.com/bro/bro/commit/8792f5545cd5b7de433d0eee510fde94371fdee3

Hi Jon,

Our scripts do make extensive use of &read_expire and &create_expire, but we don't have expire_func anywhere.

I'll check against the latest master and see how it goes!

Thanks!
--Tim