problem with bro json log format

Zeek
1

So, I am not sure whatgs going on, but when I do:

python -m json.tool < $somelog

I get

Extra data: line 2 column 1 - line 3 column 1 (char 507 - 1011)

All I did was turn json format logging on in ascii writer conf. All of my bro logs cant seem to be parsed by json.tool…

2

json.tool tries to read the entire log file as a single json record when it consists of one json record per line.

Use jq instead: https://stedolan.github.io/jq/

3

Yep, I had just gone down that route. :slight_smile: I had mistakenly believed that json.tool did more than one record at once. Thanks for the fast response Justin!

4

Hi,

>
> So, I am not sure whatgs going on, but when I do:
>
> python -m json.tool < $somelog
>
> I get
>
> Extra data: line 2 column 1 - line 3 column 1 (char 507 - 1011)
>
> All I did was turn json format logging on in ascii writer conf. All
> of my bro logs cant seem to be parsed by json.tool....

json.tool tries to read the entire log file as a single json record
when it consists of one json record per line.

Use jq instead: Redirecting to jqlang.github.io

I would propose an alternative sticking to base python:

import json
with open('conn.log') as conn:
  for line in conn:
    print(json.loads(line))

or bash:

for line in $(cat conn.log); do echo $line | python -m json.tool; done

Franky

5

I would propose an alternative sticking to base python:

import json
with open('conn.log') as conn:
for line in conn:
   print(json.loads(line))

This would be closer to what jq does by default:

import json
import pprint
import sys

for line in sys.stdin:
    pprint.pprint(json.loads(line))

or bash:

for line in $(cat conn.log); do echo $line | python -m json.tool; done

$(cat conn.log) will try to expand to the entire contents of the conn log and blow up..

while read line; do echo $line | python -m json.tool;done < conn.log

would work, but since it runs python for each log line it won't be very fast :slight_smile: