[JIRA] (BIT-1215) bro-cut should be rewritten in C for speed and to not depend on gawk

I think start with 1M and realloc 2x as needed is the way to go after

Yes. Maybe a bit less than 2x, exponential grows quickly. :slight_smile:

I think the only thing to do would be to add an absolute max line
length of 64M or something to handle the case where someone
accidentally runs bro-cut against a binary file (like a compressed bro
log) that just doesn't contain any newlines.

Would be nicer to recognize that differently, like by not finding a
log header; that way we can give a good error message. If such a check
is in place, I wouldn't actually bother with another double-check on
line length; in the unlikely case that the file has a correct header
but totally broken content, I'm sure there are plenty other cases
where bro-cut would fail, and it seems there's not more here that can
happen in addition than running out of memory (which the OS will