CIDR and limitation on indicators

I have been happily utilizing the criticalstack plugin which allows for integrating a number of feeds as indicators. I am ultimately trying to establish a system of creating, maintaining, and pushing a blacklist of my own, but I have come across an issue/question regarding the limitations of BRO.

I am currently using the intel framework with 5 feeds and 68k+ indicators. Their documentation suggested that it can allow the integration of your own list, and I wanted to try Spamhaus’ list located here: [https://www.spamhaus.org/drop/drop.txt]

I wrote Critical Stack to ask about the integration of CIDR blocks as indicators and I was told:

"The sheer volume of addresses was actually one of the reasons that we did not include lists like Spamhaus. With the current limitations in Bro of only being able to handle 100-200K indicators most of the CIDR lists pushed us way over that limit very quickly.

We did start work on CIDR expansion but it just wasn’t going to fit into the current state of Bro’s Intel Framework."

I am curious because the documentation [https://www.bro.org/sphinx/script-reference/types.html#type-addr] suggests that BRO can read CIDR notation, so is there an upper limitation with respect to BRO’s indicators? If so, what is it? Is it hardware bound, or can it be improved somewhow?

This has come up a few times, there isn't a huge technical reason why the intel framework could not be modified to handle CIDR blocks as well as individual hosts.

The one potential problem is overlap. If one indicator exists for 192.168.1.0/24 and another has 192.168.0.0/16, a match for 192.168.1.1 will only pull up the record for the larger prefix. This can be a problem if you are really need the information present in both entries.

The most recent ticket I can find about this is https://bro-tracker.atlassian.net/browse/BIT-1167

That has a patch, but it needs to be modified as the comments mention.