# Memory leaks on git eds2245 **URL:** \ **Category:** Development\ **Tags:** development\ **Created:** [September 7, 2011, 5:03pm UTC](https://community.zeek.org/t/memory-leaks-on-git-eds2245/2012 "2011-09-07T17:03:07Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Martin\_Holste](https://avatars.discourse-cdn.com/v4/letter/m/dc4da7/32.png) [@Martin\_Holste](https://community.zeek.org/u/Martin_Holste)\ **Post date:** [September 7, 2011, 5:03pm UTC](https://community.zeek.org/t/memory-leaks-on-git-eds2245/2012/1 "2011-09-07T17:03:07Z")
I'm finding that Bro leaks memory heavily whenever it drops packets, such that if overwhelmed, it will consume all memory on a beefy box in minutes. Even under a slight load, there appears to be memory leakage. Here's the tail end of a valgrind: ==4477== 479,944 (128 direct, 479,816 indirect) bytes in 1 blocks are definitely lost in loss record 5,304 of 5,306 ==4477== at 0x4C274A8: malloc (vg\_replace\_malloc.c:236) ==4477== by 0x554FF21: CRYPTO\_malloc (in /lib/libcrypto.so.0.9.8) ==4477== by 0x55E3D96: X509\_STORE\_new (in /lib/libcrypto.so.0.9.8) ==4477== by 0x5AAED4: BifFunc::bro\_x509\_verify(Frame\*, ValPList\*) (bro.bif:3449) ==4477== by 0x59B88E: BuiltinFunc::Call(ValPList\*, Frame\*) const (Func.cc:463) ==4477== by 0x5888BD: CallExpr::Eval(Frame\*) const (Expr.cc:4649) ==4477== by 0x578169: AssignExpr::Eval(Frame\*) const (Expr.cc:2598) ==4477== by 0x63AFAF: ExprStmt::Exec(Frame\*, stmt\_flow\_type&) const (Stmt.cc:369) ==4477== by 0x633E00: StmtList::Exec(Frame\*, stmt\_flow\_type&) const (Stmt.cc:1404) ==4477== by 0x59C9A0: BroFunc::Call(ValPList\*, Frame\*) const (Func.cc:320) ==4477== by 0x553A45: EventHandler::Call(ValPList\*, bool) (EventHandler.cc:73) ==4477== by 0x5531B4: EventMgr::Dispatch() (Event.h:46) ==4477== ==4477== 974,513 (5,760 direct, 968,753 indirect) bytes in 72 blocks are definitely lost in loss record 5,306 of 5,306 ==4477== at 0x4C27CC1: operator new(unsigned long) (vg\_replace\_malloc.c:261) ==4477== by 0x4FD9AD: yyparse() (parse.y:610) ==4477== by 0x50D437: main (main.cc:745) ==4477== ==4477== LEAK SUMMARY: ==4477== definitely lost: 285,262 bytes in 14,318 blocks ==4477== indirectly lost: 2,282,276 bytes in 39,468 blocks ==4477== possibly lost: 139,163 bytes in 2,955 blocks ==4477== still reachable: 9,741,975 bytes in 133,386 blocks ==4477== suppressed: 0 bytes in 0 blocks ==4477== Reachable blocks (those to which a pointer was found) are not shown. ==4477== To see them, rerun with: --leak-check=full --show-reachable=yes ==4477== ==4477== For counts of detected and suppressed errors, rerun with: -v ==4477== Use --track-origins=yes to see where uninitialised values come from ==4477== ERROR SUMMARY: 222225 errors from 373 contexts (suppressed: 4 from 4) ---
**Author:** ![robin](https://yyz1.discourse-cdn.com/flex011/user_avatar/community.zeek.org/robin/32/599_2.png) [@robin](https://community.zeek.org/u/robin)\ **Post date:** [September 8, 2011, 12:10am UTC](https://community.zeek.org/t/memory-leaks-on-git-eds2245/2012/2 "2011-09-08T00:10:04Z")
Seth/Gregor, could this still be SSL-related? Martin, any chance you could try without SSL to see if it shows the same behaviour? Robin ---
**Author:** ![Gregor\_Maier](https://avatars.discourse-cdn.com/v4/letter/g/94ad74/32.png) [@Gregor\_Maier](https://community.zeek.org/u/Gregor_Maier)\ **Post date:** [September 8, 2011, 5:25am UTC](https://community.zeek.org/t/memory-leaks-on-git-eds2245/2012/3 "2011-09-08T05:25:37Z")
I would guess so. The valgrind output definitely hints at that. I just don't know SSL or its analyzer at all, so I'm afraid I might not be much help ☹ cu Gregor (BTW, the memory problems I have/had weren't "real" leaks. One a SSL connection was done Bro would free the memory for it again. The problem is that many SSL connections can live for days and thus they ultimately consume memory like a "real" leak would). ---
**Author:** ![Seth\_Hall3](https://avatars.discourse-cdn.com/v4/letter/s/d6d6ee/32.png) [@Seth\_Hall3](https://community.zeek.org/u/Seth_Hall3)\ **Post date:** [September 8, 2011, 5:37am UTC](https://community.zeek.org/t/memory-leaks-on-git-eds2245/2012/4 "2011-09-08T05:37:56Z")
I implemented the code yesterday to stop analyzing connections with the skip\_further\_processing bif and it caused Bro to peak using more memory on the tracefile I was using it with than not stopping analysis of connections. One thing the SSL scripts are currently doing that I probably need to change is after logging the SSL log, I should probably do "delete c$ssl". The certificate and certificate chain are stored in there. Actually, as I think about it more that's probably most of the problem. We may want to look into the real traffic implications of calling the skip\_further\_processing bif eventually too though. I was pretty disheartened to see more memory used from calling that than not calling it. Perhaps it results in more memory use to remember which connections to ignore? I suppose I wasn't checking completion time which is probably where the savings should mostly come from.   .Seth ---
**Author:** ![Seth\_Hall3](https://avatars.discourse-cdn.com/v4/letter/s/d6d6ee/32.png) [@Seth\_Hall3](https://community.zeek.org/u/Seth_Hall3)\ **Post date:** [September 8, 2011, 6:56am UTC](https://community.zeek.org/t/memory-leaks-on-git-eds2245/2012/5 "2011-09-08T06:56:59Z")
I implemented this and started running it on live traffic on a cluster and so far it seems to be holding up much better than it was previously. I'll have a better feel about it tomorrow but the initial indications seem to be that this is fixing most of the problem.   .Seth ---
**Author:** ![Gregor\_Maier](https://avatars.discourse-cdn.com/v4/letter/g/94ad74/32.png) [@Gregor\_Maier](https://community.zeek.org/u/Gregor_Maier)\ **Post date:** [September 8, 2011, 3:41pm UTC](https://community.zeek.org/t/memory-leaks-on-git-eds2245/2012/6 "2011-09-08T15:41:21Z")
> > (BTW, the memory problems I have/had weren't "real" leaks. One a SSL > > connection was done Bro would free the memory for it again. The problem > > is that many SSL connections can live for days and thus they ultimately > > consume memory like a "real" leak would). > > I implemented the code yesterday to stop analyzing connections with the skip\_further\_processing bif and it caused Bro to peak using more memory on the tracefile I was using it with than not stopping analysis of connections. One thing the SSL scripts are currently doing that I probably need to change is after logging the SSL log, I should probably do "delete c$ssl". The certificate and certificate chain are stored in there. Actually, as I think about it more that's probably most of the problem. skip\_further\_processing() actually just sets the skip flag which means that no further data is delivered to the analyzers, but the analyzers aren't removed. The disable\_analyzer() bif will actually remove the analyzer, however it needs an analyzer\_id, so the SSL analyzer would somehow need to add it's analyzer\_id to one of it's events. > We may want to look into the real traffic implications of calling the skip\_further\_processing bif eventually too though. I was pretty disheartened to see more memory used from calling that than not calling it. Perhaps it results in more memory use to remember which connections to ignore? I suppose I wasn't checking completion time which is probably where the savings should mostly come from. How much difference in memory usage did you see? And how much memory usage do you see in general? My Bros usually need about 100-300MB and with SSL I will eventually get to 1GB or more (not on all nodes and it will often take hours until it starts get there). (Disabling SSL altogether will reduce the "baseline" usage by about 25%--50%) However, it still puzzles me that skip\_further\_processing didn't help. In my case memory consumption was ramping up over several minutes so either data is still delivered to the analyzer during this ramp-up or there are additional reasons for the memory consumption? cu Gregor ---
**Author:** ![system](https://canada1.discourse-cdn.com/flex011/uploads/zeek/original/1X/f09d732bc2cc7c7cc7e35db67cf4e1d5233ce7a7.png) [@system](https://community.zeek.org/u/system)\ **Post date:** [May 6, 2022, 3:39pm UTC](https://community.zeek.org/t/memory-leaks-on-git-eds2245/2012/7 "2022-05-06T15:39:49Z")