Zeek benchmarks

Development
1

Hi everyone,

I have a Zeek plugin (Zeek-Dpdk) that provides native DPDK support for Zeek as a packet source. I ran some benchmarks and wanted to validate the results with the community.

The benchmarking scenario is as follows: two servers connected back to back via their Mellanox connectX-5 single port 100G NIC cards. One server is sending at line rate (100G) with pktgen-dpdk and the other is running a single zeek instance with my plugin. The enabled logs are the typical stats.log, conn.log, capture_loss.log, reporter.log, packet_filter.log, notice.log. With some dpdk tunning, I was sending synthetic traffic at 100Gbps and observed that zeek is capturing logs at 97Gbps (modified stats.zeek to report the per second load). These results also depend on the packet size generated which is very expected (i.e., large packets → higher the throughput)

  • ~97 Gbps for packets > 1024 Bytes
  • ~62 Gbps for packets between 512 B and 1024 B
  • ~16 Gbps for packets between 128 B and 512 B

I understand that this scenario is not realistic. However, I was wondering if you could provide any feedback on the accuracy of these results as a best case scenario, particularly the 97Gbps for 1024B size packets. Note that, I’m using a single zeek instance with accelerated packet capture via dpdk, and I didn’t modified anything else in zeek.

Many thanks,

Siham

2

Hi everyone,

I have a Zeek plugin (Zeek-Dpdk) that provides native DPDK support for Zeek as a packet source. I ran some benchmarks and wanted to validate the results with the community.

The benchmarking scenario is as follows: two servers connected back to back via their Mellanox connectX-5 single port 100G NIC cards. One server is sending at line rate (100G) with pktgen-dpdk and the other is running a single zeek instance with my plugin. The enabled logs are the typical stats.log, conn.log, capture_loss.log, reporter.log, packet_filter.log, notice.log. With some dpdk tunning, I was sending synthetic traffic at 100Gbps and observed that zeek is capturing logs at 97Gbps (modified stats.zeek to report the per second load). These results also depend on the packet size generated which is very expected (i.e., large packets → higher the throughput)

  • ~97 Gbps for packets > 1024 Bytes
  • ~62 Gbps for packets between 512 B and 1024 B
  • ~16 Gbps for packets between 128 B and 512 B

I understand that this scenario is not realistic. However, I was wondering if you could provide any feedback on the accuracy of these results as a best case scenario, particularly the 97Gbps for 1024B size packets. Note that, I’m using a single zeek instance with accelerated packet capture via dpdk, and I didn’t modified anything else in zeek.

Many thanks,

Siham

3

Thanks Justin for your response.

  • It’s a single connection back to back connected via their 100G NIC cards.

  • The statistics I am looking at are: received packets and bytes received from the NIC.

  • The packet count incrementation occurs whenever I receive a new burst of packets via dpdk (below code) and the drop rate is counted from NIC counters:

bool DpdkSource::ExtractNextPacket(Packet* pkt){

while (true){

for (int j = 0; j < NumQ; j++){

const uint16_t n_pkts = rte_eth_rx_burst(0, j, bufs, BURST_SIZE); // hard coded the port

if (n_pkts == 0){

return false;

}else{

printf(“New burst %d\n”, n_pkts);

stats.received+=n_pkts;

for (int i = 0; i < n_pkts; i++){

pkt_timeval ts = {0, 0};

u_char* data = rte_pktmbuf_mtod(bufs[i], u_char*);

std::string tag = “”;

pkt->Init(props.link_type, &ts, rte_pktmbuf_data_len(bufs[i]), rte_pktmbuf_pkt_len(bufs[i]), data, false, tag);

rte_pktmbuf_free(bufs[i]);

}

return true;

}

}

}

return false;

}

This is a sample of the stats.log file for packet sizes 1024B:

#open 2021-09-09-16-54-48

#fields ts peer mem pkts_proc bytes_recv pkts_dropped pkts_link pkt_lag events_proc events_queued active_tcp_conns active_udp_conns active_icmp_conns tcp_conns udp_conns icmp_conns timers active_timers files active_files dns_requests active_dns_requests reassem_tcp_size reassem_file_size reassem_frag_size reassem_unknown_size

#types time string count count count count count interval count count count count count count count count count count count count count count count count count count

1.000000 zeek 97 8192 8355840 0 0 1631206487.971128 3 5 1 0 0 1 0 0 29 29 0 0 0 0 990 0 0 0

1631206488.992674 zeek 97 253170 258233400 46121 0 0.001103 100 299 1 0 0 98 0 0 320 26 0 0 0 0 98010 0 0 0

1631206489.994346 zeek 98 11684852 11918548080 306953 0 0.001169 14911 14712 1 0 0 4903 0 0 9812 29 0 0 0 0 990 0 0 0

1631206490.994534 zeek 98 11703602 11937674040 269228 0 0.000486 15173 15173 1 0 0 5057 0 0 10117 29 0 0 0 0 990 0 0 0

1631206491.994719 zeek 98 11708472 11942640480 266805 0 0.000465 15177 15177 1 0 0 5058 0 0 10119 29 0 0 0 0 990 0 0 0

1631206492.994852 zeek 98 11753884 11988961680 221098 0 0.000567 13922 13922 1 0 0 4640 0 0 9283 29 0 0 0 0 990 0 0 0

1631206493.995051 zeek 98 11708660 11942832240 267136 0 0.000631 12960 12960 1 0 0 4319 0 0 8641 29 0 0 0 0 990 0 0 0

1631206494.995114 zeek 98 11743798 11978673960 229597 0 0.000481 15152 15152 1 0 0 5050 0 0 10103 29 0 0 0 0 990 0 0 0

1631206495.997833 zeek 98 11688875 11922651540 319413 0 0.001114 15240 15240 1 0 0 5079 0 0 10161 29 0 0 0 0 990 0 0 0

1631206496.998002 zeek 98 11706266 11940391320 266308 0 0.000488 15290 15290 1 0 0 5096 0 0 10195 29 0 0 0 0 990 0 0 0

1631206497.998161 zeek 98 11705765 11939879340 269705 0 0.000613 15135 15135 1 0 0 5044 0 0 10091 29 0 0 0 0 990 0 0 0

1631206498.998660 zeek 98 11714296 11948581920 267145 0 0.001123 15024 15023 1 0 0 5007 0 0 10041 30 0 0 0 0 990 0 0 0

1631206499.998806 zeek 98 11755586 11990696760 216628 0 0.000532 14721 14721 1 0 0 4906 0 0 9815 30 0 0 0 0 990 0 0 0

1631206500.998834 zeek 98 11710386 11944593720 263120 0 0.000511 15032 15032 1 0 0 5010 0 0 10023 30 0 0 0 0 990 0 0 0

1631206501.998863 zeek 98 11710743 11944956900 262846 0 0.000535 15093 15093 1 0 0 5030 0 0 10063 30 0 0 0 0 990 0 0 0

1631206502.998887 zeek 98 11655274 11888379480 318052 0 0.000503 15203 15203 1 0

I am not logging protocol specific logs but mainly the generic ones (e.g., conn.lo, stats.log, reporter.log, packet_filter.log). I’m assuming that the moment I add more protocols this rate will drop by a lot.

What statistics do you recommend I should capture instead? And what would you suggest in order to improve the packet analysis and storage rates?

Thanks again,

Siham

4

Thanks Justin for your response.

I'm actually sending UDP traffic (connectionless). Why does zeek print "active_tcp_conns" = 1?
What stress test do you recommend to find out the real limitation of the server via this dpdk plugin?

Best,
Siham

5

Are you sure you are sending UDP?

The stats show active_udp_conns 0 and udp_conns 0, as in, zeek has
never received a single udp packet.

You need to send the server traffic that is representative of the
traffic that the server would see in your environment.