bro's dlmalloc

So I was testing the threaded code on FreeBSD, and discovered I'd forgotten to check to see if the malloc.c that shipped with bro was thread-safe. Oops.

When I went to browse malloc.c's source, I found out that it was dlmalloc. Further, it appears that dlmalloc is not really meant to support threads [1]. With that in mind, I'm wondering if malloc.c is really necessary; most systems these days have a pretty good default allocator ([2], [3]). From what I've read, jemalloc in particular (default allocator in FreeBSD 7+) is competitive with the Linux default allocator (ptmalloc2 derivative) [4], and is used in one context or another by Firefox and Facebook to help with memory fragmentation issues [5].

For folks worried about performance, there's always tcmalloc [6]; this comes with google perftools, but would it be a good idea to just embed this in the project?

With the above in mind, what would y'all think about either:

*) Removing malloc.c entirely
*) Replacing dlmalloc with e.g. ptmalloc3 (which I believe to be licensed under something BSD-like, but need to check on that) or jemalloc (BSD-ish license [7])

--Gilbert

[1] From dlmalloc's source: "... When USE_LOCKS is defined, each public call to malloc, free, etc is surrounded with either a pthread mutex or a win32 spinlock (depending on WIN32). This is not especially fast, and can be a major bottleneck..."
[2] http://people.freebsd.org/~jasone/jemalloc/bsdcan2006/BSDcan2006_slides.pdf
[3] http://people.freebsd.org/~kris/scaling/ebizzy.html
[4] http://www.sintel.org/development/memory-jemalloc/
[5] http://www.canonware.com/jemalloc/
[6] http://googlecode.blogspot.com/2006/12/tcmalloc-success.html
[7] http://www.canonware.com/jemalloc/license.html

support threads [1]. With that in mind, I'm wondering if malloc.c is
really necessary; most systems these days have a pretty good default
allocator ([2], [3]).

That's an excellent point. We added that many years ago when FreeBSD's
malloc performed pretty badly for Bro, but I have no idea if that
still applies. As you say, it seems reasonable to assume that things
have changed.

*) Removing malloc.c entirely

That sounds like the right thing to do, at least as long as we don't
have any data to back up any of the other options. I'll file a ticket.

Robin