# connection $history - 'g' for gap

**URL:** <https://community.zeek.org/t/connection-history-g-for-gap/5656>\
**Category:** Development\
**Tags:** development\
**Created:** [April 8, 2019, 8:02pm UTC](https://community.zeek.org/t/connection-history-g-for-gap/5656 "2019-04-08T20:02:17Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![Vern](https://yyz1.discourse-cdn.com/flex011/user_avatar/community.zeek.org/vern/32/630_2.png) [@Vern](https://community.zeek.org/u/Vern)\
**Post date:** [April 8, 2019, 8:02pm UTC](https://community.zeek.org/t/connection-history-g-for-gap/5656/1 "2019-04-08T20:02:17Z")

</div>

I'm finding it would be handy to be able to glance at a connection log line  
and know that the analysis for the connection experienced a content gap.  
For example, this can immediately explain why DPD failed to identify a  
known server.

Proposal: add 'g'/'G' connection history values, scaled in the same  
exponential way as for 'c', 't' and 'w'.

Any thoughts/objections before I go ahead and implement this?

&nbsp;&nbsp;&nbsp;&nbsp;Vern

---

<div class="post-metadata">

**Author:** ![Jim\_Mellander](https://avatars.discourse-cdn.com/v4/letter/j/e47c2d/32.png) [@Jim\_Mellander](https://community.zeek.org/u/Jim_Mellander)\
**Post date:** [April 9, 2019, 12:05am UTC](https://community.zeek.org/t/connection-history-g-for-gap/5656/2 "2019-04-09T00:05:50Z")

</div>

It might be valuable to have some (optional) way of accessing the byte counts consisting the content gap(s). If the content gap is somewhere in a long tail, but DPD still fails, then the explanation could be something other than a content gap.

On the other hand, maybe you’re just thinking about content gaps at the head of a connection before it has been fully analyzed.

---

<div class="post-metadata">

**Author:** ![seth](https://yyz1.discourse-cdn.com/flex011/user_avatar/community.zeek.org/seth/32/642_2.png) [@seth](https://community.zeek.org/u/seth)\
**Post date:** [April 9, 2019, 11:36am UTC](https://community.zeek.org/t/connection-history-g-for-gap/5656/3 "2019-04-09T11:36:21Z")

</div>

I think it would be useful. I can't believe we hadn't already thought of adding that!

&nbsp;&nbsp;.Seth

---

<div class="post-metadata">

**Author:** ![Michal\_Purzynski1](https://avatars.discourse-cdn.com/v4/letter/m/a88e57/32.png) [@Michal\_Purzynski1](https://community.zeek.org/u/Michal_Purzynski1)\
**Post date:** [April 9, 2019, 12:57pm UTC](https://community.zeek.org/t/connection-history-g-for-gap/5656/4 "2019-04-09T12:57:58Z")

</div>

It’s one of these brilliant ideas that I cannot believe we haven’t got yet 😉

Let’s implement it.

---

<div class="post-metadata">

**Author:** ![JustinAzoff](https://avatars.discourse-cdn.com/v4/letter/j/13edae/32.png) [@JustinAzoff](https://community.zeek.org/u/JustinAzoff)\
**Post date:** [April 9, 2019, 4:35pm UTC](https://community.zeek.org/t/connection-history-g-for-gap/5656/5 "2019-04-09T16:35:42Z")

</div>

This is the missed\_bytes field:

missed\_bytes: count &log &default = 0 &optional  
Indicates the number of bytes missed in content gaps, which is representative of packet loss. A value other than zero will normally cause protocol analysis to fail but some analysis may have been completed prior to the packet loss.

---

<div class="post-metadata">

**Author:** ![Jim\_Mellander](https://avatars.discourse-cdn.com/v4/letter/j/e47c2d/32.png) [@Jim\_Mellander](https://community.zeek.org/u/Jim_Mellander)\
**Post date:** [April 9, 2019, 4:53pm UTC](https://community.zeek.org/t/connection-history-g-for-gap/5656/6 "2019-04-09T16:53:57Z")

</div>

Thanks. I was thinking of something a bit different - the total amount of the content gap is useful, but in some cases it might be useful to know where the content gaps occurred, whether in the head of the connection, which likely is impactful for protocol analysis, or in a long tail, where it probably doesn’t affect analysis.

Perhaps some tunable setting indicating that “I only care about content gaps in the first 10K (or whatever) of the connection” could address that…

---

<div class="post-metadata">

**Author:** ![anthony\_kasza1](https://avatars.discourse-cdn.com/v4/letter/a/dfb087/32.png) [@anthony\_kasza1](https://community.zeek.org/u/anthony_kasza1)\
**Post date:** [April 10, 2019, 4:11am UTC](https://community.zeek.org/t/connection-history-g-for-gap/5656/7 "2019-04-10T04:11:00Z")

</div>

I like the idea of logging gap ranges for a connection. Could a vector be used to store gap start and gap stop offsets?

-AK

---

<div class="post-metadata">

**Author:** ![seth](https://yyz1.discourse-cdn.com/flex011/user_avatar/community.zeek.org/seth/32/642_2.png) [@seth](https://community.zeek.org/u/seth)\
**Post date:** [April 10, 2019, 4:55am UTC](https://community.zeek.org/t/connection-history-g-for-gap/5656/8 "2019-04-10T04:55:58Z")

</div>

I don’t think we’d add that to the default log, but you should be able to write a script that does something like that.

.Seth

---

<div class="post-metadata">

**Author:** ![McMahon\_Kevin\_J](https://avatars.discourse-cdn.com/v4/letter/m/ecc23a/32.png) [@McMahon\_Kevin\_J](https://community.zeek.org/u/McMahon_Kevin_J)\
**Post date:** [April 10, 2019, 12:59pm UTC](https://community.zeek.org/t/connection-history-g-for-gap/5656/9 "2019-04-10T12:59:05Z")

</div>

That could get very messy in the real world. How about start of first gap, length of first gap, total number of gaps?

([www.blackberry.com](http://www.blackberry.com))

---

<div class="post-metadata">

**Author:** ![Vern](https://yyz1.discourse-cdn.com/flex011/user_avatar/community.zeek.org/vern/32/630_2.png) [@Vern](https://community.zeek.org/u/Vern)\
**Post date:** [April 10, 2019, 1:29pm UTC](https://community.zeek.org/t/connection-history-g-for-gap/5656/10 "2019-04-10T13:29:30Z")

</div>

> That could get very messy in the real world. How about start of first gap,=  
> length of first gap, total number of gaps?

I think if the goal is to know whether DPD failed due to content gaps,  
much better than trying to infer that from a set of gap information would  
be for dpd.log to include "no DPD decision because ran into a content gap"  
or such.

&nbsp;&nbsp;&nbsp;&nbsp;Vern

---

<div class="post-metadata">

**Author:** ![McMahon\_Kevin\_J](https://avatars.discourse-cdn.com/v4/letter/m/ecc23a/32.png) [@McMahon\_Kevin\_J](https://community.zeek.org/u/McMahon_Kevin_J)\
**Post date:** [April 10, 2019, 7:36pm UTC](https://community.zeek.org/t/connection-history-g-for-gap/5656/11 "2019-04-10T19:36:21Z")

</div>

Agreed, but I think there is broader use for knowing, from the conn.log, that there were gaps. I think the 'g' character would address that.

I've always loved the history field and have found numerous uses for it over the years.

Kevin

---

<div class="post-metadata">

**Author:** ![system](https://canada1.discourse-cdn.com/flex011/uploads/zeek/original/1X/f09d732bc2cc7c7cc7e35db67cf4e1d5233ce7a7.png) [@system](https://community.zeek.org/u/system)\
**Post date:** [May 6, 2022, 3:46pm UTC](https://community.zeek.org/t/connection-history-g-for-gap/5656/12 "2022-05-06T15:46:25Z")

</div>


