# Tap configuration

**URL:** <https://community.zeek.org/t/tap-configuration/4596>\
**Category:** Zeek\
**Created:** [January 12, 2017, 4:04pm UTC](https://community.zeek.org/t/tap-configuration/4596 "2017-01-12T16:04:32Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![Daniel\_Manzo](https://avatars.discourse-cdn.com/v4/letter/d/b5a626/32.png) [@Daniel\_Manzo](https://community.zeek.org/u/Daniel_Manzo)\
**Post date:** [January 12, 2017, 4:04pm UTC](https://community.zeek.org/t/tap-configuration/4596/1 "2017-01-12T16:04:32Z")

</div>

Hi all,

I have Bro 2.4 configured on a RHEL 6.8 server and was wondering how to properly configure the network interfaces so that Bro can see as much of the network traffic as possible. My tap is connected in line with the network, and I believe that I was previously seeing the correct traffic, but now Bro has reporting much less information. I want to make sure that I have the interfaces configured correctly before moving on to troubleshooting other areas. Currently, I have two eth interfaces set up in PROMISC mode. Thank you for the help

Best regards,

Dan Manzo

---

<div class="post-metadata">

**Author:** ![Hosom\_Stephen\_M](https://avatars.discourse-cdn.com/v4/letter/h/d26b3c/32.png) [@Hosom\_Stephen\_M](https://community.zeek.org/u/Hosom_Stephen_M)\
**Post date:** [January 12, 2017, 7:21pm UTC](https://community.zeek.org/t/tap-configuration/4596/2 "2017-01-12T19:21:14Z")

</div>

Have you looked into checksum offloading? If enabled, it can result in Bro not producing many of the logs you would expect.

---

<div class="post-metadata">

**Author:** ![Neslog](https://avatars.discourse-cdn.com/v4/letter/n/7cd45c/32.png) [@Neslog](https://community.zeek.org/u/Neslog)\
**Post date:** [January 12, 2017, 9:59pm UTC](https://community.zeek.org/t/tap-configuration/4596/3 "2017-01-12T21:59:20Z")

</div>

I’ve had success disabling checksum.  
ignore\_checksums

---

<div class="post-metadata">

**Author:** ![Daniel\_Manzo](https://avatars.discourse-cdn.com/v4/letter/d/b5a626/32.png) [@Daniel\_Manzo](https://community.zeek.org/u/Daniel_Manzo)\
**Post date:** [January 13, 2017, 1:58pm UTC](https://community.zeek.org/t/tap-configuration/4596/4 "2017-01-13T13:58:25Z")

</div>

I have tried disabling checksum offloading, but still no luck. Here is the ifcfg file for my eth interface:

DEVICE=eth12

ONBOOT=yes

BOOTPROTO=static

PROMISC=yes

USERCTL=no

Freundliche Grüße / Best regards,

Dan Manzo

Asst Analyst I

---

<div class="post-metadata">

**Author:** ![Seth\_Hall3](https://avatars.discourse-cdn.com/v4/letter/s/d6d6ee/32.png) [@Seth\_Hall3](https://community.zeek.org/u/Seth_Hall3)\
**Post date:** [January 13, 2017, 2:28pm UTC](https://community.zeek.org/t/tap-configuration/4596/5 "2017-01-13T14:28:52Z")

</div>

I would recommend leaving checksum validation on in Bro, but disable checksum offloading on the NIC.

I typically point people to this blog post by Doug Burks (of the SecurityOnion project)...  
&nbsp;&nbsp;[http://blog.securityonion.net/2011/10/when-is-full-packet-capture-not-full.html](http://blog.securityonion.net/2011/10/when-is-full-packet-capture-not-full.html)

There is one further thing I would recommend though that we discovered well after this blog post was written. If you are using an Intel NIC with the ixgbe driver, your nic has a feature called "flow director" that you will want to disable because it will negatively impact your analysis by reordering packets. It can be disabled like this on linux:  
&nbsp;&nbsp;ethtool -L eth12 combined 1

This will cause your NIC to have only a single hardware queue which will disable the flow director feature and prevent your NIC from reordering packets. Do that along with the suggestions in the blog post above and things should be better.

&nbsp;&nbsp;.Seth

---

<div class="post-metadata">

**Author:** ![Daniel\_Manzo](https://avatars.discourse-cdn.com/v4/letter/d/b5a626/32.png) [@Daniel\_Manzo](https://community.zeek.org/u/Daniel_Manzo)\
**Post date:** [January 13, 2017, 8:28pm UTC](https://community.zeek.org/t/tap-configuration/4596/6 "2017-01-13T20:28:18Z")

</div>

Thank you for the help. I tried the settings, but I have noticed any difference in packets. The main test that I am doing is that I would open two putty sessions to the server, and have one running capstats on eth12 while my other session was downloading a 1GB file to /dev/null. Last week, I was able to see the packets increase greatly via capstats, but now they stay steady at 7 or 8 packets per second.

Best regards,  
Dan Manzo

---

<div class="post-metadata">

**Author:** ![Dave\_Crawford](https://avatars.discourse-cdn.com/v4/letter/d/919ad9/32.png) [@Dave\_Crawford](https://community.zeek.org/u/Dave_Crawford)\
**Post date:** [January 14, 2017, 7:26pm UTC](https://community.zeek.org/t/tap-configuration/4596/7 "2017-01-14T19:26:30Z")

</div>

This is what I use in my sensor /etc/network/interfaces config along with a custom “post-up” script. I use Debian for my Bro clusters, so your application will differ. I’m also using af\_packet (v4.8.0 kernel) so some of the performance settings may need to be adjusted for your setup. My tuning is aimed at keeping the packets in L3 cache on the CPU vid the NIC hardware, hence the reduced rings.

auto eth6  
iface eth6 inet manual  
up ip link set $IFACE promisc on arp off mtu 1500 up  
down ip link set $IFACE promisc off down  
post-up /opt/tools/post-up\_settings.sh $IFACE

And the /opt/tools/post-up\_settings.sh script:

#!/bin/bash

IFACE=$1

if [[-n “$IFACE”]]; then

# Lower the NIC ring descriptor size

/sbin/ethtool -G $IFACE rx 512

# Disable offloading functions

for i in rx tx sg tso ufo gso gro lro rxhash ntuple txvlan rxvlan; do ethtool -K $IFACE $i off; done

# Enforce a single RX queue

/sbin/ethtool -L $IFACE combined 1

# Disable pause frames

/sbin/ethtool -A $IFACE rx off tx off

# Limit the maximum number of interrupts per second

/sbin/ethtool -C $IFACE adaptive-rx on rx-usecs 100

# Disable IPv6

/bin/echo 1 \> /proc/sys/net/ipv6/conf/$IFACE/disable\_ipv6

# Pin IRQ to local CPU

/opt/tools/set\_irq\_affinity local $IFACE  
fi

-Dave

---

<div class="post-metadata">

**Author:** ![JonZeolla](https://avatars.discourse-cdn.com/v4/letter/j/ccd318/32.png) [@JonZeolla](https://community.zeek.org/u/JonZeolla)\
**Post date:** [January 16, 2017, 4:38am UTC](https://community.zeek.org/t/tap-configuration/4596/8 "2017-01-16T04:38:07Z")

</div>

So I’m not sure I follow exactly why you’d want to specifically emphasize keeping packets in the L3 cache. Is there a specific hardware configuration where this makes more sense?

As of right now, I do pretty much the same thing you posted earlier except I map the # of RX queues to the # of physical CPU cores and maximize the NIC ring descriptor size.

Jon

---

<div class="post-metadata">

**Author:** ![Dave\_Crawford](https://avatars.discourse-cdn.com/v4/letter/d/919ad9/32.png) [@Dave\_Crawford](https://community.zeek.org/u/Dave_Crawford)\
**Post date:** [January 20, 2017, 3:29pm UTC](https://community.zeek.org/t/tap-configuration/4596/9 "2017-01-20T15:29:55Z")

</div>

Sorry for the delayed response Jon,

We’ve been tracking the “Suricata Extreme Performance Tuning” efforts where they’re hitting 20Gbps on a single box. They have a pretty good write-up of their research into the flow of a packet through the hardware/software layers: [https://github.com/pevma/SEPTun](https://github.com/pevma/SEPTun)

The RSS=1 setting is to avoid packet re-ording impacts. Are you using PF\_RING DNA with symmetric RSS?

One new issue we’re working on since switching from pf\_ring to af\_packet is that the CPU that worker-1 is pinned to runs close to 100% while the 9 other workers are all much lower than that. We’re been able to determine that its due to the Intel ‘set\_irq\_affinity’ script and its pinning every IRQ call for the NIC (and single RSS to the same CPU, which is the first core in that NUMA node.

We’re always looking for tuning feedback/best practices so I appreciate your questions.

-Dave

---

<div class="post-metadata">

**Author:** ![system](https://canada1.discourse-cdn.com/flex011/uploads/zeek/original/1X/f09d732bc2cc7c7cc7e35db67cf4e1d5233ce7a7.png) [@system](https://community.zeek.org/u/system)\
**Post date:** [May 6, 2022, 3:44pm UTC](https://community.zeek.org/t/tap-configuration/4596/10 "2022-05-06T15:44:29Z")

</div>


