How to modify http.log

Zeek
1

Hi all,
i'm trying to modify http.log using the script written below

-----script.bro-----
redef record HTTP::Info += {
        host_ip: set[addr] &optional &log;
};

event connection_state_remove(c: connection) &priority=5
{
        local record_flag: bool = F;

        if (/^[hH][tT][tT][pP]:confused: in c$http$uri)
        {

                record_flag = T;

                when (local h = lookup_hostname(c$http$host))
                {
                        record_flag = F;
                        print(h);
                        if (|h|>0 && (0.0.0.0 !in h))
                        {
                                c$http$host_ip = h;
                                Log::write(HTTP::LOG, c$http);
                        }
                return;
                }
        }
        if (record_flag == T)
        {
                return;
        }
}

-----end script.bro----

I've added a new field in http.log (host_ip) in order to see the host
ip using the function lookup_hostname.
The script works well, but the same record is written twice (with and
without the host_ip field).
I've tried to use a state flag (record_flag) to avoid this, but the
result is the same.
How can avoid record duplicantion?
Thanks,
Vito

2

Log::write is already being called in the base scripts. You want to add a field to the record and let the base scripts worry about actually logging it out. See policy/protocols/http/var-extraction-cookies.bro (https://github.com/bro/bro/blob/master/scripts/policy/protocols/http/var-extraction-cookies.bro) as an example.

You might not be able to do what you want, though, because lookup_hostname is an asynchronous function. If it doesn’t return quickly enough, the log will be written without the field filled in. Another thing to keep in mind is that a large number of asynchronous calls can have a significant performance penalty.

c$http, c$http$uri and c$http$host are optional fields[1], so you should check for the presence of those fields with the ?$ operator before accessing them. Finally, the scheme (http://) is not included in the uri field, so I’m not really sure how your if statement is matching. I would replace that if condition with: c?$http && c$http?$host. If the host field is set, you know it’s HTTP and that you saw the request.

–Vlad

[1] - <https://www.bro.org/sphinx-git/scripts/base/protocols/http/main.bro.html#type-HTTP::Info>