# Custom log file **URL:** \ **Category:** Zeek\ **Created:** [January 2, 2017, 7:58pm UTC](https://community.zeek.org/t/custom-log-file/4571 "2017-01-02T19:58:30Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Beyaz\_Sapka](https://avatars.discourse-cdn.com/v4/letter/b/ac8455/32.png) [@Beyaz\_Sapka](https://community.zeek.org/u/Beyaz_Sapka)\ **Post date:** [January 2, 2017, 7:58pm UTC](https://community.zeek.org/t/custom-log-file/4571/1 "2017-01-02T19:58:30Z")
Hi all, I want to generate custom log files. For example, the columns of one log file should be like the following. ts,uid,fuid,geo\_location,idresp\_h,idresp\_p,method,status\_code,trans\_depth,response\_body\_len,mime\_type,host,uri,referrer,source,filename,md5,sha1,extracted,flash\_version. Those fields are found in files.log and http.log. To get those values I need connection, fa\_file and fa\_metadata records. More precisely, I need connection for resp\_h and resp\_p through conn\_id record (c$id$resp\_h and c$id$resp\_p). I need connection for ts, uid, trans\_depth, method, status\_code, response\_body\_len, host, uri, referrer, \_flash\_version through HTTP::Info record. I need connection for geo location through lookup\_location(resp\_h). I need fa\_metadata for mime\_type, since I extract only particular mime types and also I build the filename that is going to be extracted. I need fa\_file for fuid, source, filename, md5, sha1, and extracted through Files::Info record. connection and fa\_file records are accessible in event file\_over\_new\_connection. However, at this phase md5, sha1, extracted, and response\_body\_len values are not present. For this reason, I used HTTP::log\_http and Files::log\_files events. I can get all values from that events except resp\_h and resp\_p. I use file\_over\_new\_connection for that since I also extract geo location from here. But in this case, due to the nature for network traffic that is not synchronic, resp\_h and geo location values in my custom log file is erroneous. How do you do that? I read related parts of the documentation and source codes of the bro and alse reviewed the Bro archive for last one year. I keep the post short in order not to be wordy for now. I can send my script also. Any help is quite appriciated, Thanks, ---
**Author:** ![Azoff\_Justin\_S](https://avatars.discourse-cdn.com/v4/letter/a/dec6dc/32.png) [@Azoff\_Justin\_S](https://community.zeek.org/u/Azoff_Justin_S)\ **Post date:** [January 3, 2017, 3:26pm UTC](https://community.zeek.org/t/custom-log-file/4571/2 "2017-01-03T15:26:16Z")
Oh? Those two fields are part of the `id` field in the HTTP::Info record. ---
**Author:** ![Beyaz\_Sapka](https://avatars.discourse-cdn.com/v4/letter/b/ac8455/32.png) [@Beyaz\_Sapka](https://community.zeek.org/u/Beyaz_Sapka)\ **Post date:** [January 3, 2017, 4:26pm UTC](https://community.zeek.org/t/custom-log-file/4571/3 "2017-01-03T16:26:17Z")
You are right. They are available also from there. But it is not the solution of the problem. Using both HTTP::log and Files::log\_files makes inconsistency. Because events for all packets occurring concurrently. While I got tcp stream x in HTTP::log\_http, I could get tcp stream 2 in Files::log\_files. So, I left this approach and developed two scripts, but they have problems also, details are below. The data fields are extracted from the event file\_state\_remove(f: fa\_file) in the following script. [https://pastebin.mozilla.org/8958179](https://pastebin.mozilla.org/8958179) There is two problem in here. 1 request is missing in the output of the script, because it returns 302 redirection HTTP status code, I think. Alos, response\_body\_len is not available in here, since it will be available in connection\_state\_remove (c: connection). In the second script, The data fields are extracted from the event connection\_state\_remove(c: connection) in the following script. [https://pastebin.mozilla.org/8958182](https://pastebin.mozilla.org/8958182) There are two problems in here. 5 requests are missing in the output of the script, because in one connection, there is multiple http responses, I think. Secondly, it is not possible to extract 5 fields, because they are only accessible via the f$info record. There is 11 requests in the sample pcap. [http://wikisend.com/download/488246/test.pcap](http://wikisend.com/download/488246/test.pcap) ---
**Author:** ![Beyaz\_Sapka](https://avatars.discourse-cdn.com/v4/letter/b/ac8455/32.png) [@Beyaz\_Sapka](https://community.zeek.org/u/Beyaz_Sapka)\ **Post date:** [January 3, 2017, 6:53pm UTC](https://community.zeek.org/t/custom-log-file/4571/4 "2017-01-03T18:53:24Z")
Hi Justin, HTTP::log\_http and Files::log\_files based approach is working now. [https://pastebin.mozilla.org/8958232](https://pastebin.mozilla.org/8958232) But I came to that point with trial-and-error method. Here is the success story. I should build filename at the event file\_over\_new\_connection . I should update filename with the extension in the file\_sniff and call extract, md5, and sha1 analyzers in here. I do not know why I need to extract filename at the file\_over\_new\_connection method, but not in file\_sniff or something else. This script may work just for that sample, I need some guidance. Thanks, ---
**Author:** ![system](https://canada1.discourse-cdn.com/flex011/uploads/zeek/original/1X/f09d732bc2cc7c7cc7e35db67cf4e1d5233ce7a7.png) [@system](https://community.zeek.org/u/system)\ **Post date:** [May 6, 2022, 3:44pm UTC](https://community.zeek.org/t/custom-log-file/4571/5 "2022-05-06T15:44:26Z")