# Surprising behavior when reading packets from file vs interface.

**URL:** <https://community.zeek.org/t/surprising-behavior-when-reading-packets-from-file-vs-interface/1665>\
**Category:** Zeek\
**Created:** [September 23, 2010, 10:28pm UTC](https://community.zeek.org/t/surprising-behavior-when-reading-packets-from-file-vs-interface/1665 "2010-09-23T22:28:47Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jim\_Mellander](https://avatars.discourse-cdn.com/v4/letter/j/e47c2d/32.png) [@Jim\_Mellander](https://community.zeek.org/u/Jim_Mellander)\
**Post date:** [September 23, 2010, 10:28pm UTC](https://community.zeek.org/t/surprising-behavior-when-reading-packets-from-file-vs-interface/1665/1 "2010-09-23T22:28:47Z")

</div>

In the course of testing a broccoli-enabled bro policy, I discovered a  
surprising behavior, and a workaround.

My broccoli app runs in an event loop, waiting for bro events, and  
sending back events of its own to bro.

When running bro listening to an interface, it binds to the broccoli  
port, and the broccoli app communicates with bro as expected.  
Running 'bro -r' with a tracefile, however, causes bro to not bind to  
the broccoli port, thus not communicating with the app - verified  
w/netstat.

This snippet from RemoteSerialier.cc appears to be the issue:  
if ( reading\_traces && ! pseudo\_realtime )  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;using\_communication = 0;  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;return;  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}

Accordingly, I used the --pseudo-realtime flag to bro, which caused it  
to bind to the port as expected, when reading tracefiles.

However, I'm not sure this behavior is optimal - there are a number of  
bro applications now that may not need to listen to an interface (or  
to a file, for that matter), but are strictly broccoli event driven,  
for instance those which are strictly driven by syslog events.  
Perhaps there could be a flag to force bro to bind to the broccoli  
ports. Are there other options?

I would appreciate any opinions/comments.

---

<div class="post-metadata">

**Author:** ![robin](https://yyz1.discourse-cdn.com/flex011/user_avatar/community.zeek.org/robin/32/599_2.png) [@robin](https://community.zeek.org/u/robin)\
**Post date:** [September 24, 2010, 6:09pm UTC](https://community.zeek.org/t/surprising-behavior-when-reading-packets-from-file-vs-interface/1665/2 "2010-09-24T18:09:20Z")

</div>

> Running 'bro -r' with a tracefile, however, causes bro to not bind to  
> the broccoli port, thus not communicating with the app - verified  
> w/netstat.

This is actually intentional (because the timing can't be kept in  
sync between the trace and the (real-time) communication), and  
pseudo-realtime is indeed the recommended solution.

> However, I'm not sure this behavior is optimal - there are a number of  
> bro applications now that may not need to listen to an interface (or  
> to a file, for that matter), but are strictly broccoli event driven,

That's right, but when Bro is started without -r, it should actually  
work as you'd expect. Communication is only disabled when you give  
Bro a trace to read from . Are you seeing something different?

Robin

---

<div class="post-metadata">

**Author:** ![Jim\_Mellander](https://avatars.discourse-cdn.com/v4/letter/j/e47c2d/32.png) [@Jim\_Mellander](https://community.zeek.org/u/Jim_Mellander)\
**Post date:** [September 24, 2010, 6:41pm UTC](https://community.zeek.org/t/surprising-behavior-when-reading-packets-from-file-vs-interface/1665/3 "2010-09-24T18:41:46Z")

</div>

> > Running 'bro -r' with a tracefile, however, causes bro to not bind to  
> > the broccoli port, thus not communicating with the app - verified  
> > w/netstat.
> 
> This is actually intentional (because the timing can't be kept in  
> sync between the trace and the (real-time) communication), and  
> pseudo-realtime is indeed the recommended solution.

I gathered that from the code, however pseudo-realtime has its own  
problems with timing. I, for instance, want to measure the real  
propagation delay and end-to-end processing time between bro and the  
broccoli client in a controlled environment by passing a tracefile  
thru bro to be processed in part via the broccoli client.

So bro sends current\_time() to the broccoli client, which then takes a  
timestamp, and can then determine the propagation delay - but  
current\_time() is altered by the pseudo-realtime value.. If you want  
to go thru the tracefile as rapidly as possible, you will set  
pseudo-realtime to \>1, which messes up current\_time(), and thus the  
measurements. Thus, I have a need to:

1. run thru a tracefile as rapidly as possible for testing,  
2. keep the notion of current\_time() accurate to wall time, and  
3. run broccoli

Seems to me a few options can fulfill this:

1. Remove the test I mentioned  
2. Have an explicit remote communications flag - probably having  
remote communications be the default, and allow disabling via flag if  
desired.

Anything I'm missing?

> > However, I'm not sure this behavior is optimal - there are a number of  
> > bro applications now that may not need to listen to an interface (or  
> > to a file, for that matter), but are strictly broccoli event driven,
> 
> That's right, but when Bro is started without -r, it should actually  
> work as you'd expect. Communication is only disabled when you give  
> Bro a trace to read from . Are you seeing something different?

I've heard apocryphal stories of needing to have an interface open,  
but that may be due to to old event-loop requiring packets. Will ask  
around and gather more data.

---

<div class="post-metadata">

**Author:** ![Jim\_Mellander](https://avatars.discourse-cdn.com/v4/letter/j/e47c2d/32.png) [@Jim\_Mellander](https://community.zeek.org/u/Jim_Mellander)\
**Post date:** [September 27, 2010, 2:24am UTC](https://community.zeek.org/t/surprising-behavior-when-reading-packets-from-file-vs-interface/1665/4 "2010-09-27T02:24:53Z")

</div>

BTW: I commented out the test I mentioned, and Bro performs as  
desired. Thanks for your assistance.

---

<div class="post-metadata">

**Author:** ![robin](https://yyz1.discourse-cdn.com/flex011/user_avatar/community.zeek.org/robin/32/599_2.png) [@robin](https://community.zeek.org/u/robin)\
**Post date:** [September 27, 2010, 4:23pm UTC](https://community.zeek.org/t/surprising-behavior-when-reading-packets-from-file-vs-interface/1665/5 "2010-09-27T16:23:55Z")

</div>

Even without the problematic test, I'm not sure this will give you  
realistic results. If Bro reads the trace as quickly as possible,  
that can delay processing remote events more than in a live setting  
where it will likely have more slack.

Here's another idea: it seems the only problem with pseudo-realtime  
is that current\_time() doesn't return wall time but the adjusted  
time, right? Internally, current\_time() actually takes a second  
parameter indicating which of the two the caller wants. That option  
is not exposed to the script layer, but you could add a new function  
wall\_time() or so that does that (see src/bro.bif, current\_time()).

One more thought: Bro already comes with a script remote-ping.bro  
that does periodic pings between two Bro's and measures the delay  
for those. Depending on what exactly you want to measure, that might  
already be sufficient?

Robin

---

<div class="post-metadata">

**Author:** ![system](https://canada1.discourse-cdn.com/flex011/uploads/zeek/original/1X/f09d732bc2cc7c7cc7e35db67cf4e1d5233ce7a7.png) [@system](https://community.zeek.org/u/system)\
**Post date:** [May 6, 2022, 3:39pm UTC](https://community.zeek.org/t/surprising-behavior-when-reading-packets-from-file-vs-interface/1665/6 "2022-05-06T15:39:10Z")

</div>


