# how to merge rx and tx from different pcaps / slightly off-topic **URL:** \ **Category:** Zeek\ **Created:** [September 9, 2015, 2:04pm UTC](https://community.zeek.org/t/how-to-merge-rx-and-tx-from-different-pcaps-slightly-off-topic/3818 "2015-09-09T14:04:34Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Franky](https://avatars.discourse-cdn.com/v4/letter/f/958977/32.png) [@Franky](https://community.zeek.org/u/Franky)\ **Post date:** [September 9, 2015, 2:04pm UTC](https://community.zeek.org/t/how-to-merge-rx-and-tx-from-different-pcaps-slightly-off-topic/3818/1 "2015-09-09T14:04:34Z")
Hi! Sorry if this is off-topic, but I hope to find the right audience here. I want to create bro-logs of around 900 Gb of data in 20.000 pcaps. Capturing was done on different interfaces for upstream and downstream (rx/tx). Because of the large number of files I cannot merge them in one step (“to many open files”), so I merged them to one pcap per day with mergecap. After that Bro is called like this: # mergecap -F pcap -w - \*.pcap | bro -r - foo.bro foo.bro reads: redef bits\_per\_uids = 128; redef ignore\_checksums = T; redef Log::default\_rotation\_interval = 1day; No real service logs are written, except for a weird.log full of: connection\_originator\_SYN\_ack data\_after\_reset data\_before\_established inappropriate\_FIN possible\_split\_routing simultaneous\_open SYN\_after\_close SYN\_after\_reset SYN\_inside\_connection SYN\_seq\_jump TCP\_ack\_underflow\_or\_misorder TCP\_seq\_underflow\_or\_misorder window\_recision It looks like Bro not seeing the data in the correct order. But from what I read in mergecap source in merge\_read\_packet() this should work as intended: “Read the next packet, in chronological order, from the set of files to be merged.” I am thankful for any ideas. Bye, Franky ---
**Author:** ![Matthias\_Vallentin1](https://yyz1.discourse-cdn.com/flex011/user_avatar/community.zeek.org/matthias_vallentin1/32/596_2.png) [@Matthias\_Vallentin1](https://community.zeek.org/u/Matthias_Vallentin1)\ **Post date:** [September 9, 2015, 4:16pm UTC](https://community.zeek.org/t/how-to-merge-rx-and-tx-from-different-pcaps-slightly-off-topic/3818/2 "2015-09-09T16:16:28Z")
> It looks like Bro not seeing the data in the correct order. But from what I > read in mergecap source in merge\_read\_packet() this should work as intended: > "Read the next packet, in chronological order, from the set of files to be > merged." You could give this a shot:     ipsumdump --collate -r \*.pcap -w merged.pcap Unlike mergecap, ipsumdump does not assume packets are sorted within the trace.     Matthias ---
**Author:** ![Jeff\_Barber](https://avatars.discourse-cdn.com/v4/letter/j/45deac/32.png) [@Jeff\_Barber](https://community.zeek.org/u/Jeff_Barber)\ **Post date:** [September 9, 2015, 9:09pm UTC](https://community.zeek.org/t/how-to-merge-rx-and-tx-from-different-pcaps-slightly-off-topic/3818/3 "2015-09-09T21:09:16Z")
I ran into some problems trying to process pcaps. One is the checksums issue but I see you’ve already handled that. The other seems like it might possibly be related: If you don’t specify --pseudo-realtime, BRO will apparently run connection timers based on the current wall clock time, comparing the wall clock with the start time recorded in conjunction with the packets in the pcap. This means it may see a connection start, then immediately expire it as having passed the session time limit. [What? That session is six months old!] (This seems fundamentally broken to me, but it’s also quite likely that I didn’t fully understand the code and/or that there’s some good reason for it to work this way; in any case, the --pseudo-realtime switch seems to make it behave more sanely – for this particular case anyway.) Cheers. ---
**Author:** ![Seth\_Hall3](https://avatars.discourse-cdn.com/v4/letter/s/d6d6ee/32.png) [@Seth\_Hall3](https://community.zeek.org/u/Seth_Hall3)\ **Post date:** [September 10, 2015, 1:55am UTC](https://community.zeek.org/t/how-to-merge-rx-and-tx-from-different-pcaps-slightly-off-topic/3818/4 "2015-09-10T01:55:40Z")
That’s actually not how Bro works, it uses the timestamps in the packets to drive it’s packet clock forward. Could you show how you’re running Bro? It sounds to me like you’re replaying traffic to and interface and then sniffing it. .Seth ---
**Author:** ![Franky](https://avatars.discourse-cdn.com/v4/letter/f/958977/32.png) [@Franky](https://community.zeek.org/u/Franky)\ **Post date:** [September 10, 2015, 7:58am UTC](https://community.zeek.org/t/how-to-merge-rx-and-tx-from-different-pcaps-slightly-off-topic/3818/5 "2015-09-10T07:58:45Z")
Hi, ---
**Author:** ![Jeff\_Barber](https://avatars.discourse-cdn.com/v4/letter/j/45deac/32.png) [@Jeff\_Barber](https://community.zeek.org/u/Jeff_Barber)\ **Post date:** [September 10, 2015, 11:34am UTC](https://community.zeek.org/t/how-to-merge-rx-and-tx-from-different-pcaps-slightly-off-topic/3818/6 "2015-09-10T11:34:43Z")
Seth, Thanks for the clarification. Uggh… It appears that shady stuff my plugin is doing is responsible for my problem. I think the problem is that I have opened a live pkt src from within my plugin, but then also trying to read a pcap. Maybe I’ve seeded BRO with a later timestamp than those in the pcap? Having a hard time following the timer logic. Is it possible to instantiate a per-PktSrc timer? Anyway, sorry to be spewing misinformation. ---
**Author:** ![Seth\_Hall3](https://avatars.discourse-cdn.com/v4/letter/s/d6d6ee/32.png) [@Seth\_Hall3](https://community.zeek.org/u/Seth_Hall3)\ **Post date:** [September 10, 2015, 4:37pm UTC](https://community.zeek.org/t/how-to-merge-rx-and-tx-from-different-pcaps-slightly-off-topic/3818/7 "2015-09-10T16:37:55Z")
> Uggh... It appears that shady stuff my plugin is doing is responsible for my problem. Is your plugin posted anywhere? > I think the problem is that I have opened a live pkt src from within my plugin, but then also trying to read a pcap. Maybe I've seeded BRO with a later timestamp than those in the pcap? Having a hard time following the timer logic. You’re doing both in your plugin? That definitely isn’t a supported model. > Is it possible to instantiate a per-PktSrc timer? I assume you mean a per-pktsrc clock? (since timers have a meaning and are something different in Bro). If you meant clock, then no, a Bro process has the notion of a singular clock.   .Seth ---
**Author:** ![system](https://canada1.discourse-cdn.com/flex011/uploads/zeek/original/1X/f09d732bc2cc7c7cc7e35db67cf4e1d5233ce7a7.png) [@system](https://community.zeek.org/u/system)\ **Post date:** [May 6, 2022, 3:43pm UTC](https://community.zeek.org/t/how-to-merge-rx-and-tx-from-different-pcaps-slightly-off-topic/3818/8 "2022-05-06T15:43:03Z")