Offline trace: segmentation fault

Hello.

I am trying to analyze the auckland 4 trace files
(http://pma.nlanr.net/Traces/long/auck4.html) in Bro. They are
recorded in DAG format, so first I have to convert them to pcap.

I have been trying to use libtrace which has various utilities for
conversion between various formats. Also, since the auckland4 trace is
split into incoming and outgoing directions (with either -0 or -1 at
the end of the file), they must be merged together to encompass the
complete trace.

Here is what I do:

tracemerge pcapfile:20010301-110023.pcap.gz
legacyatm:20010301-110023-0.gz legacyatm:20010301-110023-1.gz
gunzip 20010301-110023.pcap.gz
/usr/local/bro/bin/bro -r 20010301-110023.pcap conn scan trw worm
analy print-resources

Running Bro produces a segmentation fault. It creates all of the
output files for the various analyzers (e.g. conn.log), but all of the
are 0 bytes.

At first, I thought the issue may be due to the large file size of the
merged trace (4.1G), so I tried it on just one direction as well
(without trying to merge them):

traceconvert legacyatm:20010301-110023-0.gz pcapfile:20010301-110023-0.pcap.gz
gunzip 20010301-110023-0.pcap.gz
/usr/local/bro/bin/bro -r 20010301-110023-0.pcap conn scan trw worm
analy print-resources

Again, this produces a segmentation fault, and the file size is now 2.0G.

I also tried running it (both the merged and single) with only the
connection analyzer, which is really the one I am interested in.
Again, this led to a seg fault.

Some other notes that may be applicable:
-The trace files are stored on an nfs mounted drive
-I am using bro-1.3.2
-The OS is fedora 4 (32bit), and the machine has 2gb of memory
-I can successfully run Bro against the lbnl
(http://www.icir.org/enterprise-tracing/download.html) traces using
the analyzers from above
-If I use the coral reef toolkit, I can print the contents of the
converted trace files just fine, which would indicate they are
converted successfully

Any thoughts?

Thanks.

(1) Can you read the trace successfully using tcpdump?

(2) If so, what's the shortest subset of it that causes Bro to crash?
    You can generate short subsets using tcpdump -c <pkt-cnt> to extract
    just the first <pkt-cnt> packets.

(3) (And if you can't read it with tcpdump, then the problem is elsewhere ...)

    Vern

1) Yes, if I use tcpdump -r on the trace it spits out the packets
fine. One thing I noticed is that many of the packets are truncated
(listed as IP truncated-ip), and the number of bytes missing is not
homogenous between the truncated packets. Could this be the problem?

2) I tried: 10000, 1000, 100, 10, 1 and none worked. I noticed that
the first packet has the "IP truncated-ip" message. Is there a way to
skip some n number of packets?

I have included a 100 packet sample if it helps.

Thanks.

test.pcap (7.45 KB)

1) Yes, if I use tcpdump -r on the trace it spits out the packets
fine. One thing I noticed is that many of the packets are truncated
(listed as IP truncated-ip), and the number of bytes missing is not
homogenous between the truncated packets. Could this be the problem?

Indeed. The message is indicating an inconsistency between the link
layer framing and the IP framing. This doubtless means that the trace
conversion process has failed to construct a correct link-layer header.

When I run Bro on the trace, I get:

  bro: problem with trace file test.pcap - unknown data link type 0xb

and it exits.

    Vern