I tried to follow the guide fairly closely and adapt for Bro with the
exception of BIOS level tuning (which I plan to investigate later).
rxhash is set to off. I was cpu pinning bro before, so I am continuing
to do so. Settings are below as well as a rough script I am tweaking to
load them.
For troubleshooting purposes I decided not to simplify the script with a
loop as I was running into some issues with command order (especially
with set_irq_affinity placement) as well as a couple unsupported options:
Features for eth4:
rx-checksumming: off
tx-checksumming: off
tx-checksum-ipv4: off
tx-checksum-ip-generic: off [fixed]
tx-checksum-ipv6: off
tx-checksum-fcoe-crc: off [fixed]
tx-checksum-sctp: off [fixed]
scatter-gather: off
tx-scatter-gather: off
tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: off
tx-tcp-segmentation: off
tx-tcp-ecn-segmentation: off [fixed]
tx-tcp6-segmentation: off
udp-fragmentation-offload: off [fixed]
generic-segmentation-offload: off
generic-receive-offload: off
large-receive-offload: off
rx-vlan-offload: off
tx-vlan-offload: off
ntuple-filters: off
receive-hashing: off
highdma: on [fixed]
rx-vlan-filter: on
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: off [fixed]
tx-ipip-segmentation: off [fixed]
tx-sit-segmentation: off [fixed]
tx-udp_tnl-segmentation: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off [fixed]
rx-all: off [fixed]
tx-vlan-stag-hw-insert: off [fixed]
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]
l2-fwd-offload: off [fixed]
busy-poll: on [fixed]
hw-tc-offload: off [fixed]
#!/bin/bash
#Unload any existing module and load with new parameters
rmmod ixgbe
modprobe ixgbe MQ=0,0,0,0 RSS=1,1,1,1 VMDQ=0,0,0,0
InterruptThrottleRate=12500,12500,12500,12500 FCoE=0,0,0,0 LRO=0,0,0,0
vxlan_rx=0,0,0,0
sleep 1
#Disable irqbalance to stop bouncing interrupts between cores
killall irqbalance
sleep 1
#Enable interfaces in promisc mode
ip link set eth4 promisc on arp off up
ip link set eth6 promisc on arp off up
sleep 1
#Disable IPv6 on interfaces
echo 1 > /proc/sys/net/ipv6/conf/eth4/disable_ipv6
echo 1 > /proc/sys/net/ipv6/conf/eth6/disable_ipv6
#Enable Jumbo Frames (MTU of 9216 used on routers)
ip link set dev eth4 mtu 9216
ip link set dev eth6 mtu 9216
#Enforce a single RX queue
ethtool -L eth4 combined 1
ethtool -L eth6 combined 1
#Manage interrupts
ethtool -C eth4 adaptive-rx on rx-usecs 100
ethtool -C eth6 adaptive-rx on rx-usecs 100
#Lower the NIC ring descriptor size
ethtool -G eth4 rx 512
ethtool -G eth6 rx 512
#Disable pause frames
#ethtool -A eth4 autoneg off
#ethtool -A eth6 autoneg off
ethtool -A eth4 rx off tx off
ethtool -A eth6 rx off tx off
#Disable offloading features
ethtool -K eth4 rx off
ethtool -K eth4 tx off
ethtool -K eth4 tso off
ethtool -K eth4 ufo off
ethtool -K eth4 gso off
ethtool -K eth4 gro off
ethtool -K eth4 lro off
ethtool -K eth4 tx-nocache-copy off
ethtool -K eth4 rxhash off
ethtool -K eth4 ntuple off
ethtool -K eth4 sg off
ethtool -K eth4 txvlan off
ethtool -K eth4 rxvlan off
ethtool -K eth6 rx off
ethtool -K eth6 tx off
ethtool -K eth6 tso off
ethtool -K eth6 ufo off
ethtool -K eth6 gso off
ethtool -K eth6 gro off
ethtool -K eth6 lro off
ethtool -K eth6 tx-nocache-copy off
ethtool -K eth6 rxhash off
ethtool -K eth6 ntuple off
ethtool -K eth6 sg off
ethtool -K eth6 txvlan off
ethtool -K eth6 rxvlan off
#Set irq affinity
/bin/bash ./set_irq_affinity 2 eth4
/bin/bash ./set_irq_affinity 3 eth6