I am trying to get the geo scripts in the phirelight git repository to work. The readme says the scripts have to be explicitly enabled.
I have:
@load ./set-country.bro
enabled in my geo/load.bro. However, my conn.log is not showing any country information. There are no errors and bro deploys cleanly. Why exactly is the country information not being inserted into the conn.log? I load geo/conn as well, which includes the 4 add-X-conn.bro scripts. This SHOULD put in the country, but is not doing so. What am I missing. The loadled_scripts.log says the add-X-conn.bro scripts are loaded.
These scripts https://github.com/phirelight/bro-scripts/tree/master/geo ?
"Scripts to set geoip/asn info for conn. Note, these will not be logged as is. There are addional scripts to log the variables for each log type."
You need to load the scripts under conn,dns,files, or ssl if you want the fields to be logged. By default they are just making the fields available.
In your case what you are missing is loading geo/conn/add-country-conn.bro
Sorry, meant to reply to the list…
The loadled_scripts.log says the add-X-conn.bro scripts are loaded.
All the add scripts are already loaded, or loaded_scripts.log from the log directory would not say so?
If the scripts weren’t being loaded, loaded_scripts.log would not indicate they are…
From: philosnef <philosnef@yahoo.com>
Subject: Re: [Bro] problems with geo scripts in phirelight repository
Date: July 27, 2016 at 3:00:31 PM EDT
To: "Azoff, Justin S" <jazoff@illinois.edu>
Reply-To: philosnef <philosnef@yahoo.com>
From my original message:
The loadled_scripts.log says the add-X-conn.bro scripts are loaded.
All the add scripts are already loaded, or loaded_scripts.log from the log directory would not say so?
Oh, derp. I can read.. really.. 
When you say "conn.log is not showing any country information" do you have the new columns?
add-country-conn.bro adds an "orig_country" and "resp_country" column. Do you have those 2 columns but they are blank, or do you not even have those columns?
If you don't have the columns at all that would be very odd. If you have them but they are all blank, that would point to an issue with the geoip bindings or databases.
Yeah, no new columns at all. I am logging in json format, but they should still show up, right?
Ah, that complicates things because optional fields are not logged in json format. unlike the TSV logs it doesn't need a fixed column layout, so fields can only show up when needed..
geoip is probably broken for you in general.
I'd try this experiment. Some errors are ok since it tries some fallbacks, but you should get the result in the end:
$ cat test.bro
print lookup_location(8.8.8.8);
$ bro test.bro
Failed to open GeoIP Cityv6 database: /usr/local/var/GeoIP/GeoIPCityv6.dat
Failed to open GeoIPv6 Country database: /usr/local/var/GeoIP/GeoIPv6.dat
error in ./test.bro, line 1: Can't open GeoIPv6 City/Country database (lookup_location(8.8.8.8))
[country_code=US, region=CA, city=Mountain View, latitude=37.386002, longitude=-122.083801]
Justin,
I get a bit where it says init-bare.bro problem initializing NB-DNS, but other than that it properly reports the exact same information you have there (US, CA, Mountain View).
For some reason, it seems to work fine now. We just pushed a new version of geo from the phirelight repository and that seems to have fixed it.
Yeah, no new columns at all. I am logging in json format, but they should still show up, right?
Ah, that complicates things because optional fields are not logged in json format. unlike the TSV logs it doesn’t need a fixed column layout, so fields can only show up when needed…
geoip is probably broken for you in general.
I’d try this experiment. Some errors are ok since it tries some fallbacks, but you should get the result in the end:
cat test.bro
print lookup_location(8.8.8.8);
bro test.bro
Failed to open GeoIP Cityv6 database: /usr/local/var/GeoIP/GeoIPCityv6.dat
Failed to open GeoIPv6 Country database: /usr/local/var/GeoIP/GeoIPv6.dat
error in ./test.bro, line 1: Can’t open GeoIPv6 City/Country database (lookup_location(8.8.8.8))
[country_code=US, region=CA, city=Mountain View, latitude=37.386002, longitude=-122.083801]