A new Bro pre-release is now available from:
ftp://ftp.ee.lbl.gov/.vp-bro-pub-0.7a90.tar.gz
The main theme of this release is better state management, especially for
use in high-volume sites. There are now a number of ways to track the
resources consumed by Bro while it runs, and to reduce the amount of memory
Bro consumes. Adding
@load print-resources
will log a summary of the resources consumed by Bro when it exits, and
@load reduce-memory
will change a number of default values in order to significantly diminish
the amount of memory Bro requires.
Other ways to lower the required memory:
- The global "maintain_http_sessions" now controls whether
http.bro tracks multiple HTTP connections between the same
hosts as a single HTTP session. Doing so costs a considerable
amount of state (so set to F for reducing the memory impact
of HTTP analysis).
- The global "suppress_scan_checks", if true, turns off address
and port scan detection. (You can't achieve this by simply
not @load'ing scan.bro, because it's loaded by some of the
default policy scripts.) Turning it off can save a lot of memory.
- Note, the ident.bro is also expensive in terms of state-holding.
A summary of other changes with respect to the last snapshot:
- New library functions:
- resource_usage() returns a record detailing real time,
CPU time, memory, other getrusage info, and the current
and maximum number of TCP/UDP/ICMP connections, and
timers and fragments.
- val_size() returns the size in bytes needed to represent
a given value (which can be a record, a table, or a
simple constant, for example). It's not fully accurate
but is in the right ballpark.
- global_sizes() return a table mapping every global
variable to its size (per val_size()). Useful for
tracking which ones are growing large over time.
- You can now control a number of timers related to Bro's connection
management. Setting them to lower values generally decreases the
amount of state Bro has to keep (see reduce-memory.bro), though
this can also make it easier for an attacker to evade detection:
tcp_SYN_timeout: interval
Controls how long Bro waits after seeing the
beginning of a connection (whether due to a SYN
or not; the timer is misnamed) before checking
whether it elicited any reply.
tcp_session_timer: interval
After a connection has closed, wait this long for
further activity before checking whether to time
out its state.
tcp_connection_linger: interval
When checking a closed connection for further
activity, Bro should consider it inactive if there
hasn't been any for this long. It also complains
if the connection is reused before this much time
has elapsed.
tcp_attempt_delay: interval
Bro waits this long upon seeing an initial SYN
before timing out the connection attempt.
tcp_close_delay: interval
Upon seeing a normal connection close, Bro flushes
state after this much time.
tcp_reset_delay: interval
Upon seeing a RST, Bro flushes state after this
much time.
tcp_partial_close_delay: interval
Bro generates a connection_partial_close event this
much time after one half of a partial connection
closes, assuming there has been no subsequent
activity.
non_analyzed_lifetime: interval
If a connection belongs to an application that you
aren't analyzing, Bro times it out after this
interval. If 0 secs (default), then it doesn't
time it out.
dns_session_timeout: interval
ntp_session_timeout: interval
rpc_timeout: interval
Bro waits this long before timing out a DNS/NTP/RPC
request.
max_timer_expires: count
The maximum number of timers to expire after
processing each new packet. The value trades off
spreading out the timer expiration load with
possibly having to hold state longer. A value of 0
means "process all expired timers with each new
packet".
- Two new flags control other facets of Bro's connection management,
and thus state-holding:
partial_connection_ok: bool
Whether Bro should analyze connections for which
it doesn't see the beginning, only the middle.
This can be very expensive to do in the face of
stealth-scanning, which looks like a bunch of
partial connections.
Note, the HTTP analyzer has been modified to
now always skip partial connections. This should
instead be user controllable.
tcp_SYN_ack_ok: bool
If true, Bro instantiates connection state when
it sees a SYN ack but not the initial SYN (even
if partial_connection_ok is false). The intent
behind this knob (which is not well tested) is
to allow you to filter out initial SYNs and only
react to SYN acks. This keeps Bro from holding
state during SYN scans and SYN flooding, except
for when the destination responds.
- Some other miscellaneous thresholds that you can now modify
from your policy script:
heartbeat_interval: count
How often to generate net_stats_update() events.
This timer really isn't needed any more, since
you can use "schedule" to achieve the same effect.
tcp_storm_thresh: count
If Bro sees this many FINs/RSTs in a row, it
flags them as a "storm".
tcp_storm_interarrival_thresh: interval
The FINs/RSTs must come with this much time or less
between them.
- The state management for line-oriented applications like HTTP
requests has been improved.
- The HTTP analyzer now expands %hex sequences. If anyone has
a Unicode expander to contribute, that'd be terrific.
- The Code Red detection is more robust (fewer false positives).
- A new redefinable variable, skip_services, lists applications
that should not be analyzed for purposes of detecting scans.
(Default: ident) The point of having this (poorly named) hook
is so that code-red.bro can add "http" to it, to avoid reporting
the zillions of Code Red scans that a site can see.
- Bro now matches regular expressions using lazy DFA evaluation.
The upshot of this is (1) it no longer maintains .bro-RE-cache.v1,
(2) it starts up fast regardless of whether you've added new
regular expressions, (3) you can afford to add lots of regular
expressions, and (4) it's actually a bit faster.
- The list of "hot_ids" has been expanded with a number of other
common root-privileged accounts.
- Vern