# printing stream columns

**URL:** <https://community.zeek.org/t/printing-stream-columns/5867>\
**Category:** Zeek\
**Created:** [October 16, 2019, 7:45pm UTC](https://community.zeek.org/t/printing-stream-columns/5867 "2019-10-16T19:45:46Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![Henri\_Dubois-Ferrier](https://avatars.discourse-cdn.com/v4/letter/h/3ec8ea/32.png) [@Henri\_Dubois-Ferrier](https://community.zeek.org/u/Henri_Dubois-Ferrier)\
**Post date:** [October 16, 2019, 7:45pm UTC](https://community.zeek.org/t/printing-stream-columns/5867/1 "2019-10-16T19:45:46Z")

</div>

I’m trying to print the record type for each log stream at startup. Something like:

for ( id in Log::active\_streams ) {  
local stream = Log::active\_streams[id];  
print stream$path, stream$columns;

}

doesn’t work because $columns is a record type, and gets stringified “”.

Is there a way to do this in zeek script?

Thanks,  
Henri

---

<div class="post-metadata">

**Author:** ![anthony\_kasza1](https://avatars.discourse-cdn.com/v4/letter/a/dfb087/32.png) [@anthony\_kasza1](https://community.zeek.org/u/anthony_kasza1)\
**Post date:** [October 16, 2019, 8:27pm UTC](https://community.zeek.org/t/printing-stream-columns/5867/2 "2019-10-16T20:27:16Z")

</div>

Hi Henri,

Great question.  
The logging framework is extremely flexible and allows for log stream columns to dynamically change during run time. This means at startup, the bro\_init() event, Zeek may not know all the columns of all the logs. Here’s a script I wrote for you which sort of answers your question. If you have more questions about it, just reach back out to the list.

-AK

function pfunk(rec: any): bool {  
print type\_name(rec);  
return T;  
}

event bro\_init() {  
for (id in Log::active\_streams) {  
for (fname in Log::get\_filter\_names(id)) {  
local filter: Log::Filter;  
filter = Log::get\_filter(id, fname);  
filter$pred = pfunk;  
Log::add\_filter(id, filter);  
}  
}  
}

---

<div class="post-metadata">

**Author:** ![Jon\_Siwek](https://avatars.discourse-cdn.com/v4/letter/j/71c47a/32.png) [@Jon\_Siwek](https://community.zeek.org/u/Jon_Siwek)\
**Post date:** [October 16, 2019, 8:37pm UTC](https://community.zeek.org/t/printing-stream-columns/5867/3 "2019-10-16T20:37:34Z")

</div>

Zeek 3.0 should give better descriptions for types. This was the  
relevant patch which is not in any 2.6.x version:

&nbsp;&nbsp;[https://github.com/bro/bro/commit/1f450c05102be6dd7ebcc2c5901d5a3a231cd675](https://github.com/bro/bro/commit/1f450c05102be6dd7ebcc2c5901d5a3a231cd675)

This script may also help demonstrate things related to what you're  
trying to do:

&nbsp;&nbsp;[https://gist.github.com/jsiwek/f843b3321f4227b6ec32d110424ebf70](https://gist.github.com/jsiwek/f843b3321f4227b6ec32d110424ebf70)

It prints field descriptions of all logs either to stdout or a CSV  
file. Example command:

&nbsp;&nbsp;ZEEK\_ALLOW\_INIT\_ERRORS=1 zeek print-log-info.bro PrintLogs::csv=F

Sample of output:

known\_hosts.log | Hosts with complete TCP handshakes  
&nbsp;&nbsp;ts: time - The timestamp at which the host was detected.  
&nbsp;&nbsp;host: addr - The address that was detected originating or responding  
to a TCP connection.

- Jon

---

<div class="post-metadata">

**Author:** ![Henri\_Dubois-Ferrier](https://avatars.discourse-cdn.com/v4/letter/h/3ec8ea/32.png) [@Henri\_Dubois-Ferrier](https://community.zeek.org/u/Henri_Dubois-Ferrier)\
**Post date:** [October 16, 2019, 8:47pm UTC](https://community.zeek.org/t/printing-stream-columns/5867/4 "2019-10-16T20:47:05Z")

</div>

Thanks Jon and Anthony for the quick responses! print-log-info.bro looks promising for what I’m trying to do.

---

<div class="post-metadata">

**Author:** ![Henri\_Dubois-Ferrier](https://avatars.discourse-cdn.com/v4/letter/h/3ec8ea/32.png) [@Henri\_Dubois-Ferrier](https://community.zeek.org/u/Henri_Dubois-Ferrier)\
**Post date:** [November 1, 2019, 11:11am UTC](https://community.zeek.org/t/printing-stream-columns/5867/5 "2019-11-01T11:11:28Z")

</div>

I’ve been playing around with Jon’s script and am getting close to what I want, but still have one outstanding issue related to nested records. Currently they show up as a single feed with a type “record foo” (such as “record conn\_id” or “record FTP::ExpectedDataChannel”).

I’d like to be able to peek into nested records to get the inner fields that will show up in the logs. It doesn’t seem like there’s a way to do record introspection given a string representation of the record type name, but if I’d be delighted to be told I’m missing something.

Thanks for any pointers!

---

<div class="post-metadata">

**Author:** ![Jon\_Siwek](https://avatars.discourse-cdn.com/v4/letter/j/71c47a/32.png) [@Jon\_Siwek](https://community.zeek.org/u/Jon_Siwek)\
**Post date:** [November 1, 2019, 7:53pm UTC](https://community.zeek.org/t/printing-stream-columns/5867/6 "2019-11-01T19:53:13Z")

</div>

No, didn't look like there was a way to do that, but I've made a  
PR/patch that should make recursive introspection possible via  
something like `record_fields("conn_id")` for any arbitrary record  
type name:

[https://github.com/zeek/zeek/pull/675](https://github.com/zeek/zeek/pull/675)

- Jon

---

<div class="post-metadata">

**Author:** ![Henri\_Dubois-Ferrier](https://avatars.discourse-cdn.com/v4/letter/h/3ec8ea/32.png) [@Henri\_Dubois-Ferrier](https://community.zeek.org/u/Henri_Dubois-Ferrier)\
**Post date:** [November 1, 2019, 7:55pm UTC](https://community.zeek.org/t/printing-stream-columns/5867/7 "2019-11-01T19:55:07Z")

</div>

Ooh that looks great. Thanks!

---

<div class="post-metadata">

**Author:** ![Henri\_Dubois-Ferrier](https://avatars.discourse-cdn.com/v4/letter/h/3ec8ea/32.png) [@Henri\_Dubois-Ferrier](https://community.zeek.org/u/Henri_Dubois-Ferrier)\
**Post date:** [November 11, 2019, 7:37pm UTC](https://community.zeek.org/t/printing-stream-columns/5867/8 "2019-11-11T19:37:30Z")

</div>

Now that this patch is merged (thanks again) I’ve upgraded my Zeek script and the record\_fields changes work great.

I still have one outstanding issue which is that for a container type, record\_field$type\_name is just the container name (such as “vector” or “set”). I don’t see a way to get the type of the container elements from zeek script, but once again would be delighted to be corrected.

And if there’s currently no way, I’m happy to put up a PR, but I could use some guidance on how to expose this in Zeek (e.g. a new field on record\_field?).

Thanks,  
Henri

---

<div class="post-metadata">

**Author:** ![Jon\_Siwek](https://avatars.discourse-cdn.com/v4/letter/j/71c47a/32.png) [@Jon\_Siwek](https://community.zeek.org/u/Jon_Siwek)\
**Post date:** [November 11, 2019, 9:17pm UTC](https://community.zeek.org/t/printing-stream-columns/5867/9 "2019-11-11T21:17:05Z")

</div>

Would be great if you want to try making a PR. The first way to do it  
that comes to mind is just alter that "record\_field$type\_name" to  
better describe containers in a format like "vector of XXX",  
"set[XXX]" or "table[XXX] of YYY". This should be the relevant code  
to modify:

[https://github.com/zeek/zeek/blob/b86a8acc2b84089efbbe51216a4f4d1d57a4f430/src/Type.cc#L845-L850](https://github.com/zeek/zeek/blob/b86a8acc2b84089efbbe51216a4f4d1d57a4f430/src/Type.cc#L845-L850)

- Jon

---

<div class="post-metadata">

**Author:** ![Henri\_Dubois-Ferrier](https://avatars.discourse-cdn.com/v4/letter/h/3ec8ea/32.png) [@Henri\_Dubois-Ferrier](https://community.zeek.org/u/Henri_Dubois-Ferrier)\
**Post date:** [November 11, 2019, 9:24pm UTC](https://community.zeek.org/t/printing-stream-columns/5867/10 "2019-11-11T21:24:09Z")

</div>

Cool, that’s exactly the place i was looking (wasn’t sure if changing this might break existing scripts… but since this is all quite new, probably best to make the change soon). I’ll get the PR up soon.

---

<div class="post-metadata">

**Author:** ![system](https://canada1.discourse-cdn.com/flex011/uploads/zeek/original/1X/f09d732bc2cc7c7cc7e35db67cf4e1d5233ce7a7.png) [@system](https://community.zeek.org/u/system)\
**Post date:** [May 6, 2022, 3:46pm UTC](https://community.zeek.org/t/printing-stream-columns/5867/11 "2022-05-06T15:46:48Z")

</div>


