Questions about NICs, PF_RING, and AF_Packet

Zeek
1

I am new to Zeek and will be setting up a Zeek system which will use a 10G NIC. I am not sure what NIC/driver configuration to use, and have some questions about PF_RING and AF_Packet. At …

https://docs.zeek.org/en/current/cluster-setup.html#using-pf-ring

… it looks like PF_RING, also known as “Vanilla PF_RING” …

https://www.ntop.org/guides/pf_ring/vanilla.html#vanilla-pf-ring

… makes it possible to assign worker processes to CPU cores by using “packet clustering” …

https://www.ntop.org/guides/pf_ring/vanilla.html#packet-clustering

Is this essentially implementing symmetric Receive Side Scaling?

If so, can Vanilla PF_RING take advantage of a NIC which does symmetric hashing in hardware?

The Zeek docs reference PF_RING+DNA …

https://docs.zeek.org/en/current/cluster-setup.html#using-pf-ring

… although from looking at the ntop site, DNA/Libzero was replaced some time ago with PF_RING ZC (Zero Copy) …

https://www.ntop.org/guides/pf_ring/zc.html

Does Zeek support PF_RING ZC?

If so, in Zeek’s node.cfg, how does one know what options to use for …

interface=

lb_method=

It looks like the AF_Packet plugin …

https://github.com/J-Gras/zeek-af_packet-plugin

… does what Vanilla PF_RING does, in that it allows Zeek to have multiple worker processes which use different CPU cores. Can AF_Packet take advantage of a NIC which does symmetric hashing in hardware?

It looks like AF_Packet does not provide a “Zero Copy” type of functionality, found in PF_RING ZC. Is that correct?

I know this is a lot of questions. Any guidance is appreciated.

– Steve.

2

Hey Steve, answers inline.

I am new to Zeek and will be setting up a Zeek system which will use a 10G NIC. I am not sure what NIC/driver configuration to use, and have some questions about PF_RING and AF_Packet. At …

https://docs.zeek.org/en/current/cluster-setup.html#using-pf-ring

… it looks like PF_RING, also known as “Vanilla PF_RING” …

https://www.ntop.org/guides/pf_ring/vanilla.html#vanilla-pf-ring

… makes it possible to assign worker processes to CPU cores by using “packet clustering” …

Indeed it is a best performance practice to assign cores to worker nodes and that’s supported by both AF_Packet and PF_RING

https://www.ntop.org/guides/pf_ring/vanilla.html#packet-clustering

Is this essentially implementing symmetric Receive Side Scaling?

Packet clustering there means they make it possible to distribute flows among many processes to process them, so kind of like symmetric RSS indeed. Supported by both PF_RING and AF_Packet.

If so, can Vanilla PF_RING take advantage of a NIC which does symmetric hashing in hardware?

As far as I know PF_Ring does not use hardware RSS hash (nothing does, AF_Packet doesn’t either). For AF_Packet read on.

The Zeek docs reference PF_RING+DNA …

https://docs.zeek.org/en/current/cluster-setup.html#using-pf-ring

… although from looking at the ntop site, DNA/Libzero was replaced some time ago with PF_RING ZC (Zero Copy) …

https://www.ntop.org/guides/pf_ring/zc.html

Does Zeek support PF_RING ZC?

If so, in Zeek’s node.cfg, how does one know what options to use for …

interface=

lb_method=

It looks like the AF_Packet plugin …

https://github.com/J-Gras/zeek-af_packet-plugin

… does what Vanilla PF_RING does, in that it allows Zeek to have multiple worker processes which use different CPU cores. Can AF_Packet take advantage of a NIC which does symmetric hashing in hardware?

Intel made it so confusing. Let’s disect

  • there is a way to make hashing symmetric by flipping a bit in hardware. No software but some Intel experiments uses that
  • there is a way to use make RSS symmetric with the right hashing key that’s easily set with ethtools. This would be your hardware symmetric RSS hashing.
    You need a couple of those ethtool commands to account for fragmented packets, etc.

I’ll dig them out tomorrow.

It looks like AF_Packet does not provide a “Zero Copy” type of functionality, found in PF_RING ZC. Is that correct?

AF_Packet is as zero-copy as PF_RING is. All of the PF_Ring documentation is outdated by like 10+ years.

I know this is a lot of questions. Any guidance is appreciated.

I’m sure there will be more and please keep sending them meantime read what Peter Manev and the (not-so-humble) myself wrote about Suricata - applies to Zeek as well

https://github.com/pevma/SEPTun
https://github.com/pevma/SEPTun-Mark-II

3

To answer the question about pf_ring-zc, yes Zeek does support it. We actually just fixed a bug in Zeek recently to make pf_ring-zc work correctly with the new IO loop.

Tim

4

Attached is a systemd service file that may help with that hash command (and a bunch of other stuff) if you’re using Intel XL710s. Credit goes to Vlad.

-Dop

tuning_ens3f0.service (10.9 KB)

5

Big thanks for the information. This is very helpful. (The Suricata Extreme Performance Tuning documents are great work!)

I have a question about …