# A lower level interface **URL:** \ **Category:** Zeek\ **Created:** [October 9, 2017, 1:33pm UTC](https://community.zeek.org/t/a-lower-level-interface/5052 "2017-10-09T13:33:42Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Shuai\_Hao](https://avatars.discourse-cdn.com/v4/letter/s/e9bcb4/32.png) [@Shuai\_Hao](https://community.zeek.org/u/Shuai_Hao)\ **Post date:** [October 9, 2017, 1:33pm UTC](https://community.zeek.org/t/a-lower-level-interface/5052/1 "2017-10-09T13:33:42Z")
Hi All, I’ve seen many discussions referring to the Bro as an alternative of libnids. I wonder that can we use the similar lower-level interface similar to libnids in Bro (e.g., for the tcp assembly)? We would like to explore the string features of packets, while keeping to leverage Bro’s high-level events. Regards, Shuai ---
**Author:** ![johanna](https://avatars.discourse-cdn.com/v4/letter/j/50afbb/32.png) [@johanna](https://community.zeek.org/u/johanna)\ **Post date:** [October 10, 2017, 5:01pm UTC](https://community.zeek.org/t/a-lower-level-interface/5052/2 "2017-10-10T17:01:44Z")
Hi, > I've seen many discussions referring to the Bro as an alternative of > libnids. Out of curiosity - where did you see discussions like that? I do not really know much about libnids, but from the readme it seems that libnids is a library that mostly implements TCP reassembly. While this is a part of what Bro does, it only is a small part of it; obviously the main focus of Bro is on a different layer. > I wonder that can we use the similar lower-level interface similar > to libnids in Bro (e.g., for the tcp assembly)? I am not quite sure what you are looking for here, could you perhaps expand a bit on that? The lowest level access that you can get is probably by writing a custom (c++) analyzer that gets passed either the reassembled TCP payloads. (Or just the raw packets in case of UDP). > We would like to explore the string features of packets, while keeping > to leverage Bro's high-level events. You can use something like the new\_packet events in Bro to get access to individual packet information. However, there is a performance penalty associated with this (script-level events are fairly expensive and usually there are a few per connection, not a few per packet). For anything carrying a significant amount of traffic that approach probably is not viable. It depends a bit on what you want to do - preprocessing in C++ and then bubbling up more high-level events should be a more realisitic choice. I hope this helps, Johanna ---
**Author:** ![Shuai\_Hao](https://avatars.discourse-cdn.com/v4/letter/s/e9bcb4/32.png) [@Shuai\_Hao](https://community.zeek.org/u/Shuai_Hao)\ **Post date:** [October 10, 2017, 6:06pm UTC](https://community.zeek.org/t/a-lower-level-interface/5052/3 "2017-10-10T18:06:51Z")
Thanks for your reply, Johanna. Basically I am looking for an interface by which we can examine and extract the features of byte stream (or strings) from the traffic (TCP payload), and then we will feed the stream to our analyzer (e.g., via BinPac). Currently I am looking at the tcp\_contents; I think it might be sufficient so I don’t have to use tcp\_packet or new\_packet. Although there is also a performance note for tcp\_contents in the manual, I assume that it would be better than using tcp\_packet or new\_packet (is this true? I do care about the performance). Thank you very much! Cheers, Shuai ---
**Author:** ![johanna](https://avatars.discourse-cdn.com/v4/letter/j/50afbb/32.png) [@johanna](https://community.zeek.org/u/johanna)\ **Post date:** [October 10, 2017, 6:22pm UTC](https://community.zeek.org/t/a-lower-level-interface/5052/4 "2017-10-10T18:22:22Z")
> Basically I am looking for an interface by which we can examine and extract > the features of byte stream (or strings) from the traffic (TCP payload), > and then we will feed the stream to our analyzer (e.g., via BinPac). > Currently I am looking at the tcp\_contents; I think it might be sufficient > so I don't have to use tcp\_packet or new\_packet. I still don't quite get what you are planning to do here. Do you plan to do some kind of signature to figure out that something is a specific protocol (so match certain byte sequences)? Or do you really want to do something more complex that needs scripting? tcp\_contents is probably less expensive than the other named choices, but it probably still is pretty heavyweight. Johanna ---
**Author:** ![Shuai\_Hao](https://avatars.discourse-cdn.com/v4/letter/s/e9bcb4/32.png) [@Shuai\_Hao](https://community.zeek.org/u/Shuai_Hao)\ **Post date:** [October 10, 2017, 6:41pm UTC](https://community.zeek.org/t/a-lower-level-interface/5052/5 "2017-10-10T18:41:46Z")
The thing we are planing to do is similar to the [protocol reverse engineering + traffic pattern recognition]; so we consider that we may need the lower level interface to inspect the byte stream since the patterns that we want to identify (e.g., a serious of connection activities) would involve various protocols. We do have the signature part to accomplish the payload matching. But it may be not sufficient when we consider the traffic recognition (e.g., generating a signature that involve various protocols and network components). Thanks for your relies. It do helps much. ---
**Author:** ![johanna](https://avatars.discourse-cdn.com/v4/letter/j/50afbb/32.png) [@johanna](https://community.zeek.org/u/johanna)\ **Post date:** [October 10, 2017, 6:50pm UTC](https://community.zeek.org/t/a-lower-level-interface/5052/6 "2017-10-10T18:50:59Z")
Hi again, > The thing we are planing to do is similar to the [protocol reverse > engineering + traffic pattern recognition]; so we consider that we may need > the lower level interface to inspect the byte stream since the patterns > that we want to identify (e.g., a serious of connection activities) would > involve various protocols. It sadly is really a bit hard to tell what exactly the best starting point is without knowing the exact problem. You mention connection activities - does that mean activities inside the same connection or activities within different connetions? If it is the latter - you could potentially use signatures to identify "interesting" connections and use Bro script level events to tie cross-connection information together. If signatures for some reason are not enough, it depends a bit on your traffic. If you only want this to run in rather low-traffic environments, it might be ok to use the low-level events like tcp\_payload. If not - your only choice quickly becomes to write a C++ analyzer, which then once again can raise events. You even can write several analyzers, one which only tries to deduce if a connection is interesting, which then in turn can forward data to more specific analyzers if interesting data is found. I hope that helps; I am sorry that I am not more specific, but given that I still don't 100% understand what you are trying to do this is the best I can do. Johanna ---
**Author:** ![Shuai\_Hao](https://avatars.discourse-cdn.com/v4/letter/s/e9bcb4/32.png) [@Shuai\_Hao](https://community.zeek.org/u/Shuai_Hao)\ **Post date:** [October 10, 2017, 7:28pm UTC](https://community.zeek.org/t/a-lower-level-interface/5052/7 "2017-10-10T19:28:53Z")
> It sadly is really a bit hard to tell what exactly the best starting point > is without knowing the exact problem. You mention connection activities - > does that mean activities inside the same connection or activities within > different connetions? > > If it is the latter - you could potentially use signatures to identify > "interesting" connections and use Bro script level events to tie > cross-connection information together. Johanna, thanks for your patient and detailed replies. I think it is the latter. For example, some external connections (let's say a HTTP Get/Post to server A, with a binary string with signature S1) will raise some activities of local network components B and C, where the traffic is associated with a signature S2. Then we would like to \*learn\* such a pattern (HTTP\_Get/Post\_A, S1, B, C, S2) as a pattern signature. So we consider "tokenize" the byte stream to extract and cluster the strings from raw payload. ---
**Author:** ![system](https://canada1.discourse-cdn.com/flex011/uploads/zeek/original/1X/f09d732bc2cc7c7cc7e35db67cf4e1d5233ce7a7.png) [@system](https://community.zeek.org/u/system)\ **Post date:** [May 6, 2022, 3:45pm UTC](https://community.zeek.org/t/a-lower-level-interface/5052/8 "2022-05-06T15:45:20Z")
Closed on May 6, 2022, 3:45pm UTC