Bro 1.5.2 release now available

Bro release 1.5.2 is now available from:

  ftp://bro-ids.org/bro-1.5.2.tar.gz

This version is a maintenance release coming with quiet a number of
refinements and fixes, per the appended changelog entries.

Robin

--------- cut -------------------------------------------------------

1.5.2 Wed Jan 12 17:34:55 PST 2011

- Portability fixes for --enable-int64 (Vern Paxson).

- Bug fix for Active Mapping support (Kevin Lo).

- Broccoli compiler warning fixes (Kevin Lo).

- Bug fixes for --enable-int64 and for avoiding bogus statistics /
  bad memory references when generating profiling information upon
  exit (Vern Paxson).

- Bug fixes for terminating connections (Tyler Schoenke and Vern Paxson).

- Removed now-quite-stale SSHv1 overflow detection, as it's more prone
  to false positives than useful detection (Vern Paxson).

- The SWIG file now explicitly lists those pieces from broccoli.h which it
  wants to wrap, rather than just including all of broccoli.h (Robin Sommer).

- http-header.bro now includes a global "include_header: set[string]" If it
  contains any strings, then only those headers will be processed. If left
  empty, then you continue to get the current behavior of processing all
  headers. (Robin Sommer).

- Several changes to drop.bro (Robin Sommer):

  * If True, the new flag Drop::dont_drop_locals indicates that
    local hosts should never be dropped. On by default.

  * If True, the new flag Drop::debugging activates extensive debugging
    output for the catch-and-release logic. Off by default.

  * The timeout for tracking dropping information is now 1 day
    rather than 7 days, to better echo the one-restart-a-day semantics
    used in the past.

  * Bug fix for hosts once dropped by later cleared; some state
    for them persisted.

- Portability fix for Broccoli Python bindings on 64-bit platforms (Robin
  Sommer).

- The HTTP analyzer no longer attempts to track Server/User-Agent
  versions, as these are hugely voluminous (Seth Hall).

- HTTP and SMTP no longer have extra-short inactivity timeouts, as
  these were too often leading to premature expiration of a connection
  (Robin Sommer).

- Tracking of HTTP refer[r]er's by setting log_referrer. (Vern Paxson).

- The "rst" tool (aux/rst/) now takes an optional "-I <text>" argument
  that instructs it to inject <text> as payload rather than sending a RST
  packet (Vern Paxson). <text> must be NUL-terminated, and the NUL is not
  included.

- Bug fix for crashes in the DNS analyzer when processing replies for
  which no request was seen (Robin Sommer).

- Addressed a number of lint nits (Vern Paxson).

- Rotation post-processors are now passed an additional argument
  indicating whether Bro is terminating (Robin Sommer).

- Bro now consistently generates a file_opened event for all fopen() calls.
  (Robin Sommer).

- The "cf" utility now ignores a leading "t=" prefix, for compatibility
  with Bro's "tagged" logging format (Robin Sommer).

- You can now redefine the email_notice_to function (Robin Sommer).

- Fix for packet processing resumption when a remote Bro dies during
  state synchronization (Robin Sommer).

- OpenSSL/X509 portability fix, at long last (Gregor Maier & Christian
  Kreibich).

- Fix for compatibility with newer versions of autoconf (Gregor Maier).

- A larger BroControl update (Robin Sommer, if not marked otherwise):

  o Increasing default timeouts for scan detector significantly.

  o Increasing the manager's max_remote_events_processed to
    something large, as it would slow down the process too much
    otherwise and there's no other work to be interleaved with it
    anyway.

  o Adding debug output to cluster's part of catch-and-release
    (extends the debugging already present in policy/debug.bro)

  o Fixing typo in util.py. Closes #223.

  o Added note to README pointing to HTML version.

  o Disabling print_hook for proxies' remote.log.

  o broctl's capstats now reports a total as well, and stats.log
    tracks these totals. Closes #160.

  o Avoiding spurious "waiting for lock" messages in cron mode.
    Closes #206.

  o Bug fixes for installation on NFS.

  o Bug fix for top command on FreeBSD 8.

  o crash-diag now checks whether gdb is available.

  o trace-summary reports the sample factor in use in its output,
    and now also applies it to the top-local-networks output (not
    doing the latter was a bug).

  o Removed the default twice-a-day rotation for conn.log. The
    default rotation for conn.log now is now once every 24h, just
    like for all other logs with the exception of mail.log (which is
    still rotated twice a day, and thus the alarms are still mailed
    out twice a day).

  o Fixed the problem of logs sometimes being filed into the wrong
    directory (see the (now gone) FAQ entry in the README).

  o One can now customize the archive naming scheme. See the
    corresponding FAQ entry in the README.

  o Cleaned up, and extended, collection of cluster statistics.

    ${logdir}/stats now looks like this:

      drwxr-xr-x 4 bro wheel 59392 Apr 5 17:55 .
      drwxr-xr-x 96 bro wheel 2560 Apr 6 12:00 ..
      -rw-r--r-- 1 bro wheel 576 Apr 6 16:40 meta.dat
      drwxr-xr-x 2 bro wheel 2048 Apr 6 16:40 profiling
      -rw-r--r-- 1 bro wheel 771834825 Apr 6 16:40 stats.log
      drwxr-xr-x 2 bro wheel 2048 Apr 6 16:25 www

    stats.log accumulates cluster statistics collected every time
    "cron" is called.

    - profiling/ keeps the nodes' prof.logs.

    - www/ keeps a subset of stats.log in CSV format for easy plotting.

    - meta.dat contains meta information about the current cluster
    state (in particular which nodes we have, and when the last
    stats update was done).

    Note that there is no Web setup yet to actually visualize the data in
  www/.

  o BroControl now automatically maintains links inside today's log
    archive directory pointing to the current live version of the
    corresponding log file (if Bro is running). For example:

        smtp.log.11:52:18-current -> /usr/local/cluster/spool/manager/smtp.log

  o Alarms mailed out by BroControl now (1) have the notice msg in the
    subject; and (2) come with the full mail.log entry in the body.

  o Fixing broctl's top output. (Seth Hall).

  o Fixing broctl's df output in certain situations.

  o BroControl fix for dealing with large vsize values reported by
    "top" (Craig Leres).

First of all thanks for a new release of bro. It is a great tool which i use regularly.

Could i request you make the releases available via HTTP too, if possible. This would help out people behind firewalls which only open a handful of protocols?

There was a patch which Vern had forwarded to me which i was hoping made it into the release. Not a big deal since it is just a 2 line patch, but wanted to see if it could be rolled into the mainline.

The patch had to do with enabling the HTTP analyser to kick in even for connections where we miss the tcp handshake. There were a single line in ContentLine.cc and HTTP.cc which had to be changed for this to happen. Any reason we couldn’t make that on by default or enable that via a configure option. If you need me to help out wrt this, i am willing to do so.

thanks,
Sridhar

First of all thanks for a new release of bro.

You're welcome.

Could i request you make the releases available via HTTP too, if possible.

It's now available via HTTP as well:

    http://www.bro-ids.org/download/bro-1.5.2.tar.gz

The patch had to do with enabling the HTTP analyser to kick in even for
connections where we miss the tcp handshake.

Hmm ... I don't remember that patch right now. Could you see whether
is already recorded with the tracker at http://tracker.icir.org, and
if not, open a new ticket. Thanks!

However, in any case, it sounds like this patch is changing Bro's
semantics in terms of how/when connections are analyzed, which
generally is something we prefer to avoid doing for maintainace
releases like this one.

Robin

Question from a very new Bro user. (Seth helped us do the original set up earlier this week -- Thanks!!)

If I do a standard 'make install' will that clobber anything vital in an existing config?

Russell

First of all thanks for a new release of bro.

You’re welcome.

Could i request you make the releases available via HTTP too, if possible.

It’s now available via HTTP as well:

http://www.bro-ids.org/download/bro-1.5.2.tar.gz

Thanks.

The patch had to do with enabling the HTTP analyser to kick in even for
connections where we miss the tcp handshake.

Hmm … I don’t remember that patch right now. Could you see whether
is already recorded with the tracker at http://tracker.icir.org, and
if not, open a new ticket. Thanks!

I did not see anything related to partial connections in the tracker. I will add an entry for it.

thanks,
Sridhar

No, it shouldn't. Note that if you're using broctl, you need a "make
install-broctl" to install everything, but that should likewise not
clobber anything.

(Of course, it never hurts to make a backup of anything you couldn't
restore otherwise ...)

Robin

I got this error when installing a bro-close with 1.5.2

[worker-4]

==== stderr.log

/usr/local/bro/share/bro/broctl/cluster-addrs.hot.bro, line 3 (Hot::ssh_notice_hosts): error, “redef” used but not previously defined

I commented the redef line out of /usr/local/bro/share/bro/broctl/cluster-addrs.hot.bro and all the work nodes came up.

Bill Jones

/usr/local/bro/share/bro/broctl/cluster-addrs.hot.bro, line 3 (Hot::ssh_notice_hosts): error, "redef" used but not previously defined

Doh! That fix didn't get backported from the current development
version.

I commented the redef line out of
/usr/local/bro/share/bro/broctl/cluster-addrs.hot.bro and all the
work nodes came up.

That's indeed the right thing to do. In fact you could also just
delete the whole file as there's nothing else in there.

Robin