So in my internet travels I ran across this:
https://www.rsreese.com/parsing-netflow-using-kibana-via-logstash-to-elasticsearch/
A tad outdated but I thought why not…I have syslogs and Bro’s conn.log going into the ELK stack, so let’s add netflow to the mix. After dinking around with it and getting the data in, I realized that Bro’s conn.log pretty much does everything netflow can…unless I’m missing something? For example, if I want to see what a single IP address is doing I use this as a filter in Kibana:
type:connlog AND conn_state:S* AND src_ip:192.168.1.100 AND proto:TCP
type:connlog AND conn_state:S* AND src_ip:192.168.1.100 AND proto:UDP
What say you all…any reason not to rip out softflowd and just drive on with Bro’s conn.log? Thank you.
James
James,
I think you’re right, but sometimes you can get Netflow from locations where you might not easily be able to put a Bro sensor.
Kind regards,
Andy
Andrew.Ratcliffe@NSWCSystems.co.uk
CISSP, GCIA, GCIH, GPEN, GWAPT, CSTA, CSTP, CWSA, GCFE
Blog.InfoSecMatters.net
Andrew got the exact reason that you’d still collect netflow. The Bro conn log is significantly different than netflow though. It’s bidirectional (IPFIX can be too, but we’ll ignore that for now). The log doesn’t write out until the connection is complete, whereas netflow breaks and writes out frequently which can be great, but can also be super annoying if you’re trying to pay attention to the full life cycle of a connection forensically. There are several extra fields in the Bro logs that netflow doesn’t have too (history and service being two that immediately come to mind).
If you’re generating netflow though, there is almost never any benefit these days unless you have a netflow analysis solution in place that you’d like to feed and you can’t collect from routers anymore, usually because your routers can only do sampled netflow.
.Seth
Thanks Seth,
I took out softflowd and reverted to my previous version of logstash.conf. I have to say, it was pretty cool to have my Kibana graphs up of Bro's conn.log and softflowd side by side to compare....was pretty awesome.
James
Nice, did you notice any major discrepancies?
.Seth
I did not besides the minor timing thing you described. For example there's an Android device that fires off to ssl.analytics.google.com at exact intervals. The netflow graph showed these at pretty close to the same times (squid logs logged the exact time to syslog), whereas Bro had them a little varied, but that was ONLY when you dug in like to a every 5 minute graph. If you zoomed out to say showing the last 12 hours you couldn't tell a difference at all. I didn't notice a difference in the packet count or size either....a good thing 
James