From ef0c98e4c115a77b18530bf2c4559f0f991254cd Mon Sep 17 00:00:00 2001 From: Vladimir Dubrovin <3proxy@3proxy.ru> Date: Tue, 13 Oct 2026 13:00:21 +0300 Subject: [PATCH] bench: RESULTS with a table for each machine, HTTPS with streams and envoy Every table is now for one machine, ARM or Intel, with no averages of the two, the copying proxies and those using splice() in one table: bold the best of each group and * the best of the row for the HTTP proxy and the relay, only the best of the row for CONNECT and SOCKS. The tables for 13 October replace those of 11-12 October, whose columns for splice by direction and 16 KB buffers are gone with them. The HTTPS section is measured again with today's devel, with the TLS streams and envoy. report.py takes the TLS streams and 1 MB answers for bulk data. Co-Authored-By: Claude Opus 5.5 --- bench/RESULTS.md | 678 ++++++++++++++++++++++--------------- bench/data/m4-tls.csv | 270 ++++++++++----- bench/data/skylake-tls.csv | 270 ++++++++++----- bench/report.py | 7 +- 4 files changed, 762 insertions(+), 463 deletions(-) diff --git a/bench/RESULTS.md b/bench/RESULTS.md index 8be0f0c..dc04b4e 100644 --- a/bench/RESULTS.md +++ b/bench/RESULTS.md @@ -1,57 +1,45 @@ # Benchmark results -3proxy against nginx, haproxy, squid, tinyproxy and dante on two machines, -as an HTTP proxy, through CONNECT and SOCKSv5 tunnels and as a TCP relay; and -3proxy's own versions against each other: 0.9.4, 1.0.1 and devel, with and -without splice(). The current figures are from 11-12 October 2026, for devel -with splice() by direction, pipes kept per connection and the automatic mode -(`-s`); the sessions before them, for the earlier devel, are at the end. How -the benchmark works is in [README.md](README.md); the measurements are in -[data/](data/), and `bench/report.py` turns them into the tables below. +3proxy against nginx, haproxy, squid, tinyproxy, dante and stunnel on two +machines, as an HTTP proxy, through CONNECT and SOCKSv5 tunnels, as a TCP +relay, ending or beginning TLS, and with 20000 connections open at once; and +devel against 1.0.1. The current figures are from 13 October 2026, for devel +with 16 KB buffers, the automatic splice() mode by default and the thread +pool; the sessions before them are at the end. How the benchmark works is in +[README.md](README.md); the measurements are in [data/](data/), and +`bench/report.py` turns them into the tables below. ## In short -* **HTTPS (13 October):** with TLS ended or begun in the proxy, devel moves - 1 MB answers as fast as haproxy and nginx beginning TLS (5.4 Gbit/s), and - needs a third of their memory; it answers 59% of nginx's 1 KB requests and - does 60-94% of its handshakes. See [HTTPS](#https-tls-in-the-proxy). -* **20000 connections (13 October):** devel keeps up with nginx and haproxy - within 5-25% in throughput and CPU, but needs 65-130 KB per connection - against their 8-30 KB. See [the mix scenarios](#many-connections-the-mix-scenarios). +Ranges are over the two machines, ARM and Intel; the tables give each. -* **Bulk data, copying:** devel moves 19-21 Gbit/s per CPU core through any - of its services (HTTP, CONNECT, SOCKS, `tcppm`), as 1.0.1 does: 1.4 times - squid, 1.3 to 2.2 times dante, 1.7 to 2.3 times nginx and haproxy as TCP - relays, 2.5 to 2.9 times 0.9.4 copying and 17 times tinyproxy. -* **Bulk data, splice():** about 40 Gbit/s on one stream through every - service, level with 0.9.4 and with haproxy splicing every connection; with - 16 streams devel gets 33-35 Gbit/s, ahead of 0.9.4 (25-26) and haproxy - (27). 1 MB HTTP answers go 65% faster than copied (31 against 19 Gbit/s), - 15% faster than haproxy splicing. -* **The automatic mode, `-s`,** came within 5% of the best fixed mode in - every scenario measured: it splices the bulk streams in both directions and - copies small requests. Forcing splice() (`-s3`, and 0.9.4 by default) costs - 1 KB requests through tunnels and relays 12-21%. -* **Small requests:** squid (CONNECT) and dante (SOCKS) answer 1 KB requests - over kept-alive tunnels 8-45% faster than devel; nginx and haproxy answer - 1.9 times more as TCP relays and 2.2 to 2.4 times more as HTTP proxies, and - 1.6 to 3 times more when each request opens a new connection. 3proxy runs - a thread per connection; splice() cannot change that. -* **2000 clients, 50 KB answers:** devel is level with squid and dante in - tunnels (13.1-13.6k requests per second), with nginx and haproxy splicing - as an HTTP proxy (10-11k), ahead of haproxy copying (8.5k), and with nginx as a TCP relay copying (13.5k); haproxy - relaying with splice() is the fastest there, 21.9k against devel's 15.2k. -* **1.0.1** is level with devel in tunnels and relays, but held to 6k - requests per second as an HTTP proxy over kept-alive connections by a 10 ms - sleep removed in devel (see the earlier sessions). -* **16 KB buffers (`-b16384`)** save devel 17-22% of its memory with 2000 - connections and cost 20-40% of the copying throughput; with `-s` they cost - nothing. Since 13 October 16 KB is the default, and so is `-s`. The whole -suite run again with those defaults, against 1.0.1, nginx, haproxy, squid, -tinyproxy and dante, is in [data/](data/) `*-16k-*.csv`. -* **Memory:** with few connections 3proxy needs the least (8-17 MB against - 30-54 MB for nginx and haproxy); with 2000 connections at once, a thread for - each costs it 150-200 MB against nginx's 90-115 MB and haproxy's 160-180 MB. +* **Bulk data:** devel splices 29-42 Gbit/s through every service (HTTP, + CONNECT, SOCKS, `tcppm`), the fastest everywhere: 2-2.9 times squid, 2-4.2 + times dante, and 2.6-5.6 times nginx and haproxy copying 1 MB HTTP answers. + haproxy `splice-auto` is level with it as a relay on Intel and behind on + ARM. Copying with its 16 KB buffers (`-s0`) devel does 9.4-13.4 Gbit/s, + level with nginx and haproxy; the automatic mode, the default, is what + makes 16 KB enough. +* **Small requests:** nginx and haproxy answer 1.5-2.2 times more 1 KB + requests as relays and 1.9-2.8 times as HTTP proxies; squid 1.2-2 times + more through CONNECT; through SOCKS dante leads on ARM, devel on Intel. + 3proxy runs a thread per connection; splice() cannot change that. +* **A new connection per request:** nginx and haproxy do 1.2-1.5 times more + as relays, 2.2-2.9 times as HTTP proxies, where they keep their connections + to the origin and a forward proxy cannot. +* **2000 clients, 50 KB answers:** on Intel devel leads as an HTTP proxy, + through CONNECT and through SOCKS; on ARM nginx, squid and dante are 13-47% + ahead of it. As a relay haproxy `splice-auto` leads on both. +* **HTTPS:** with TLS ended or begun in the proxy devel moves bulk data as + fast as nginx, ahead of haproxy and stunnel, with the least memory; envoy + (BoringSSL) leads, by 30-50% on ARM. devel answers half to two thirds of + nginx's 1 KB requests and does 60-95% of its handshakes. See + [HTTPS](#https-tls-in-the-proxy). +* **20000 connections:** devel keeps up with nginx and haproxy within 5-25% in + throughput and CPU, but needs 65-130 KB per connection against their 8-30 + KB. See [the mix scenarios](#many-connections-the-mix-scenarios). +* **Memory with few connections:** 3proxy needs the least, 5-20 MB against + 30-50 MB for nginx and haproxy. ## Setup @@ -100,28 +88,22 @@ tinyproxy and dante, is in [data/](data/) `*-16k-*.csv`. | column | configuration | |---|---| +| devel | devel as of 13 October: 16 KB buffers, the thread pool, splice() in its automatic mode, the default: each direction switched to splice() once two reads in a row fill 16 KB and back to copying after three small ones; an HTTP body of known length is spliced when 16 KB or more of it remain | | devel -s0 | devel, no splice (TCP_NODELAY is on by default) | -| devel -s1 (download) | splices what the server sends: answers, downloads | -| devel -s3 (both) | splices both directions | -| devel -s (auto) | both directions, each switched to splice() once two reads in a row fill 16 KB and back to copying after three small ones; an HTTP body of known length is spliced when 16 KB or more of it remain | -| devel -b16384 | devel with 16 KB buffers instead of 64 KB | -| 0.9.4 | 0.9.4, `-s0` (no splice) `-ocTCP_NODELAY -osTCP_NODELAY` | -| 0.9.4 splice | 0.9.4, which splices both directions by default, `-ocTCP_NODELAY -osTCP_NODELAY` | -| 1.0.1 | 1.0.1, `-ocTCP_NODELAY -osTCP_NODELAY` (built without splice) | +| 1.0.1 | 1.0.1 as it starts by default: built without splice, 64 KB buffers, Nagle's algorithm on (no TCP_NODELAY, which devel sets by default) | | nginx | `proxy_pass` to an upstream with `keepalive 1024`, `stream` for the TCP relay, `reuseport` | | haproxy | `mode http` with `http-reuse always`, `mode tcp` for the TCP relay | -| haproxy splice-response (download) | `option splice-response` | -| haproxy splice (both) | `option splice-request` and `option splice-response` | | haproxy splice-auto | `option splice-auto`: splices a direction once reads fill the buffer three times in a row | | squid | no cache, no logs, `via off` | | tinyproxy | `DisableViaHeader Yes`, critical messages only | | dante | no authentication, one `sockd` process tree (`-N 1`) | +| stunnel | TLS scenarios only | nginx has no splice() support. Neither nginx nor haproxy has CONNECT or SOCKS, so they are measured as HTTP proxies and TCP relays only; squid and tinyproxy -only through CONNECT, dante only through SOCKS. 3proxy versions are built with -the Makefile they shipped with (`-O3 -flto` for 1.0.1 and devel, `-O2` for -0.9.4, which on gcc 14 also needs `-fpermissive`). Logging is off everywhere. +only as HTTP proxies and through CONNECT, dante only through SOCKS. 3proxy +versions are built with the Makefile they shipped with (`-O3 -flto`). Logging +is off everywhere. ### Scenarios @@ -142,227 +124,384 @@ a TCP relay to the origin (3proxy `tcppm`, nginx `stream`, haproxy ## How to read the tables -* Each cell: the median on the M4 / on the Skylake host / their average. TCP - streams and 1 MB HTTP answers in Gbit/s, the other HTTP scenarios in - thousands of requests per second. -* Each kind of proxying has two tables: the proxies copying through user - space (after "no proxy", the client talking to the origin with only the - kernel forwarding, for reference), then the proxies using splice(). -* **Bold**: the best average of its table in the row, and any other whose runs - cannot be told apart from it: the range of its average (from the average of - each machine's slowest run to that of the fastest) reaches the best one's. -* \*: the best of the row, copying and splice tables together, by the same - rule - whether copying or splice() wins the scenario. +* A table for each machine, ARM (the M4) and Intel (the Skylake host): the + two are different machines in different conditions, and an average of them + would mean nothing. Each cell is the median of three runs; TCP streams and + 1 MB answers in Gbit/s, the other HTTP scenarios in thousands of requests + per second. ‡: most runs were limited by the client or the origin rather + than by the proxy, which can then do at least that much. +* Each table has the proxies copying through user space first, then, after + `‖`, those using splice(): devel, whose automatic mode is its default, and + haproxy with `splice-auto`. +* HTTP proxy and TCP relay: **bold** is the best median of its group + (copying, or splice()) in the row, and any other of the group whose runs + cannot be told apart from it: its fastest run reaches the best one's + slowest. + \* marks the best of the row over both groups, by the same rule. +* CONNECT and SOCKS, where devel is the only proxy using splice(): only the + best of the row is marked, bold and with \*. * CPU time per gigabyte or request and peak memory are in the data. CPU adds little: every proxy ran at its CPU limit, so its CPU per unit of work is the inverse of its throughput. ## HTTP proxy -Measured in one session on each machine ([data/](data/) `*-auto-http.csv`). +Measured on 13 October 2026, as were the CONNECT, SOCKS and relay tables +([data/](data/) `*-16k-*.csv`). -| scenario (M4 / Intel / average) | no proxy | ‖ | devel -s0 | devel -s0 -b16384 | nginx | haproxy | -|---|---:|:-:|---:|---:|---:|---:| -| http-ka-1k, k req/s | 120.8 / 129.3 / 125.0 | ‖ | 16.0 / 18.2 / 17.1 | 16.0 / 18.4 / 17.2 | 37.1 / 36.5 / 36.8 | **44.7 / 36.2 / 40.4** * | -| http-ka-1m, Gbit/s | 71.2 / 65.5 / 68.3 | ‖ | **22.4 / 15.7 / 19.1** | 13.7 / 9.4 / 11.6 | 12.8 / 5.6 / 9.2 | 5.8 / 7.3 / 6.6 | -| http-ka-50k-2000, k req/s | 86.1 / 85.2 / 85.7 | ‖ | **10.5 / 11.2 / 10.9** * | 9.2 / 8.2 / 8.7 | 11.3 / 9.0 / 10.1 | 9.8 / 7.2 / 8.5 | -| http-ka-2k-50k-2000, k req/s | 73.7 / 64.8 / 69.2 | ‖ | **9.6 / 9.6 / 9.6** * | 8.4 / 8.2 / 8.3 | 10.3 / 8.2 / 9.3 | 8.9 / 6.7 / 7.8 | -| http-close-1k, k req/s | 48.5 / 46.8 / 47.6 | ‖ | 5.2 / 4.7 / 4.9 | 5.2 / 4.7 / 5.0 | 13.5 / 13.2 / 13.3 | **16.7 / 14.7 / 15.7** | +**ARM (M4)** -| scenario (M4 / Intel / average) | devel -s1 (download) | devel -s3 (both) | devel -s (auto) | devel -s -b16384 | haproxy splice-response (download) | haproxy splice (both) | -|---|---:|---:|---:|---:|---:|---:| -| http-ka-1k, k req/s | 16.0 / 18.6 / 17.3 | 16.0 / 19.1 / 17.6 | 15.8 / 17.8 / 16.8 | 15.9 / 18.2 / 17.0 | **44.2 / 35.3 / 39.8** * | **44.9 / 34.6 / 39.7** * | -| http-ka-1m, Gbit/s | **36.2 / 27.1 / 31.6** * | **36.2 / 26.7 / 31.5** * | **36.3 / 25.4 / 30.9** * | **35.3 / 26.3 / 30.8** * | 27.9 / 26.6 / 27.3 | 27.3 / 27.3 / 27.3 | -| http-ka-50k-2000, k req/s | **10.4 / 10.2 / 10.3** * | **10.4 / 11.1 / 10.8** * | **10.5 / 10.5 / 10.5** * | **10.2 / 10.1 / 10.2** * | **12.0 / 9.2 / 10.6** * | **12.0 / 9.4 / 10.7** * | -| http-ka-2k-50k-2000, k req/s | **9.6 / 9.6 / 9.6** * | 9.6 / 9.4 / 9.5 | 9.4 / 9.8 / 9.6 | **9.4 / 9.7 / 9.6** * | **11.4 / 8.7 / 10.1** * | **11.3 / 8.4 / 9.9** * | -| http-close-1k, k req/s | 5.2 / 4.8 / 5.0 | 5.2 / 4.4 / 4.8 | 5.2 / 4.8 / 5.0 | 5.3 / 4.6 / 4.9 | **16.8 / 15.3 / 16.1** * | **16.9 / 15.1 / 16.0** * | +| scenario (ARM) | devel -s0 | 1.0.1 | haproxy | nginx | tinyproxy | squid | ‖ | devel | haproxy splice-auto | +|---|---:|---:|---:|---:|---:|---:|:-:|---:|---:| +| ka-1k, k req/s | 14.5 | 1.5 | **40.3** * | 33.1 | 3.9 | 19.4 | ‖ | 14.5 | **40.1** * | +| ka-1m, Gbit/s | **12.4** | 8.1 | 5.4 ‡ | 11.8 | 1.0 | 3.0 ‡ | ‖ | **30.2 ‡** * | 5.4 ‡ | +| ka-50k-2000, k req/s | 8.6 | 8.1 | 8.6 | **10.9** * | 1.6 | 4.1 | ‖ | **9.6** | 8.5 | +| ka-2k-50k-2000, k req/s | 7.8 | 6.7 | 7.9 | **9.6** * | 1.5 | 3.1 | ‖ | **8.7** | 8.1 | +| close-1k, k req/s | 5.4 | 4.1 | **15.6** * | 12.2 | 3.9 | 6.8 | ‖ | 5.4 | **15.7** * | -* splice() pays off for 1 MB answers, in any mode: 31 against 19 Gbit/s - copied, ahead of haproxy splicing (27). With 50 KB answers splicing neither - gains nor loses; with 1 KB answers or a new connection per request the mode - makes no difference. -* Since pipes are made when first needed and kept for the connection, `-s1` - no longer loses with 50 KB answers (9.6k against 8.7k requests per second - in the earlier session, against 9.6k copying). -* nginx and haproxy answer 2.2-2.4 times more 1 KB requests, 2.7-3.2 times - more when every request opens a new connection. +**Intel (Skylake)** + +| scenario (Intel) | devel -s0 | 1.0.1 | haproxy | nginx | tinyproxy | squid | ‖ | devel | haproxy splice-auto | +|---|---:|---:|---:|---:|---:|---:|:-:|---:|---:| +| ka-1k, k req/s | 18.9 | 1.6 | **36.2** * | **35.6** * | 3.9 | 9.3 | ‖ | 18.6 | **35.8** * | +| ka-1m, Gbit/s | **9.4** | 7.8 | 7.9 | 5.9 | 1.1 | 2.2 | ‖ | **26.3 ‡** * | 24.8 ‡ | +| ka-50k-2000, k req/s | 8.6 | **9.6** | 7.6 | 9.1 | 1.6 | 2.1 | ‖ | **10.4** * | 7.4 | +| ka-2k-50k-2000, k req/s | 7.8 | **8.0** | 7.3 | **8.1** | 1.5 | 1.8 | ‖ | **9.5** * | 6.9 | +| close-1k, k req/s | 6.0 | 3.9 | **15.8** * | 13.3 | 3.7 | 4.0 | ‖ | 6.1 | **15.5** * | + +* 1 MB answers: devel splices them at 30 Gbit/s on ARM and 26 on Intel, 2.6 + to 5.6 times nginx and haproxy copying. haproxy `splice-auto` keeps up on + Intel (25) but hardly splices on ARM (5.4). Copying (`-s0`), devel is ahead + of the other copying proxies too (12.4 and 9.4 Gbit/s). +* 2000 clients with 50 KB answers: nginx leads on ARM (10.9k requests a + second against devel's 9.6k), devel on Intel (10.4k against 9.1k); haproxy + is behind both on both. With a 2 KB POST each, the same order. +* nginx and haproxy answer 2.3-2.8 times more 1 KB requests over kept-alive + connections on ARM, 1.9 times on Intel, and 2.2-2.9 times more with a new + connection for each. That last gap is not the thread pool's: nginx + (`keepalive 1024`) and haproxy (`http-reuse always`) keep their connections + to the origin and only accept and close the client's, while a forward proxy + such as 3proxy also opens and closes one to the server for every client + connection. As a TCP relay, where every proxy does, the gap is 1.35-1.5 + times on ARM and 1.2-1.3 on Intel. +* 1.0.1 answers 1.5k 1 KB requests a second over kept-alive connections, held + by a 10 ms sleep removed in devel. ## CONNECT tunnel -Measured in one session on each machine, as were the SOCKS and relay tables -([data/](data/) `*-tunnels.csv`). +**ARM (M4)** -| scenario (M4 / Intel / average) | devel -s0 | 0.9.4 | 1.0.1 | tinyproxy | squid | -|---|---:|---:|---:|---:|---:| -| stream1-down, Gbit/s | **22.1 / 20.2 / 21.1** | 9.1 / 5.8 / 7.4 | **22.8 / 19.8 / 21.3** | 1.2 / 1.2 / 1.2 | 15.1 / 13.0 / 14.0 | -| stream16, Gbit/s | **21.1 / 18.9 / 20.0** | 8.7 / 6.0 / 7.3 | **21.1 / 19.1 / 20.1** | 1.3 / 1.2 / 1.2 | 17.6 / 12.1 / 14.8 | -| ka-1k, k req/s | 19.6 / 30.7 / 25.2 | 19.7 / 29.1 / 24.4 | 19.6 / 30.9 / 25.3 | 17.0 / 24.1 / 20.6 | **36.6 / 36.4 / 36.5** * | -| ka-50k-2000, k req/s | 11.2 / 11.5 / 11.3 | 8.3 / 8.2 / 8.3 | 11.2 / 12.7 / 12.0 | 2.3 / 2.4 / 2.3 | **16.3 / 11.0 / 13.6** * | -| close-1k, k req/s | **4.1 / 4.4 / 4.3** * | **4.2 / 4.6 / 4.4** * | 3.8 / 3.7 / 3.7 | **4.1 / 4.8 / 4.4** * | **5.3 / 3.4 / 4.4** * | +| scenario (ARM) | devel -s0 | 1.0.1 | tinyproxy | squid | ‖ | devel | +|---|---:|---:|---:|---:|:-:|---:| +| stream1-down, Gbit/s | 13.3 | 20.5 | 1.2 | 14.3 | ‖ | **41.7 ‡** * | +| stream16, Gbit/s | 12.6 | 19.0 | 1.2 | 16.9 | ‖ | **33.6 ‡** * | +| ka-1k, k req/s | 17.4 | 17.5 | 15.4 | **35.0 ‡** * | ‖ | 17.5 | +| ka-50k-2000, k req/s | 8.4 | 10.0 | 2.1 | **15.0** * | ‖ | 10.2 | +| ka-2k-50k-2000, k req/s | 7.9 | 9.4 | 2.0 | **14.0** * | ‖ | 9.5 | +| close-1k, k req/s | 4.2 ‡ | 3.5 ‡ | 3.7 ‡ | **4.9 ‡** * | ‖ | 4.2 ‡ | -| scenario (M4 / Intel / average) | devel -s1 (download) | devel -s3 (both) | devel -s (auto) | 0.9.4 splice | -|---|---:|---:|---:|---:| -| stream1-down, Gbit/s | **47.5 / 32.0 / 39.8** * | **47.1 / 32.1 / 39.6** * | **47.5 / 31.6 / 39.5** * | **47.4 / 32.5 / 39.9** * | -| stream16, Gbit/s | 20.9 / 18.7 / 19.8 | 37.6 / 28.7 / 33.1 | **38.3 / 31.2 / 34.7** * | 24.1 / 27.7 / 25.9 | -| ka-1k, k req/s | 17.8 / 28.3 / 23.0 | 16.1 / 23.8 / 20.0 | **19.6 / 28.7 / 24.1** | 16.1 / 26.1 / 21.1 | -| ka-50k-2000, k req/s | **11.5 / 15.4 / 13.4** | 10.6 / 14.5 / 12.5 | **11.4 / 15.4 / 13.4** * | 10.6 / 15.0 / 12.8 | -| close-1k, k req/s | 4.0 / 4.2 / 4.1 | 3.8 / 3.8 / 3.8 | **4.1 / 4.4 / 4.3** * | 3.9 / 4.1 / 4.0 | +**Intel (Skylake)** -* tinyproxy stays at 1.2 Gbit/s whatever the load; squid manages 14-15 Gbit/s, devel 20 copying and 34-40 with - splice(). -* With 16 upload streams `-s1` copies, as it should; `-s3` and `-s` splice - them. 0.9.4 splicing is level with devel on one stream but 25% behind it with - 16. -* squid answers 1 KB requests 45% faster than devel and is level with it with - 2000 clients (ahead on the M4, behind on the Skylake host). -* With a new tunnel per request the client is the limit on the M4 - (proxychains does its work there): every proxy shows 4-5k requests per - second. +| scenario (Intel) | devel -s0 | 1.0.1 | tinyproxy | squid | ‖ | devel | +|---|---:|---:|---:|---:|:-:|---:| +| stream1-down, Gbit/s | 9.6 | 19.1 | 1.2 | 13.4 | ‖ | **31.0** * | +| stream16, Gbit/s | 9.7 | 18.7 | 1.3 | 13.1 | ‖ | **29.5** * | +| ka-1k, k req/s | 30.7 | 30.3 | 29.0 | **36.3** * | ‖ | 30.4 | +| ka-50k-2000, k req/s | 10.0 | 13.1 | 2.5 | 11.3 | ‖ | **15.4** * | +| ka-2k-50k-2000, k req/s | 9.7 | 12.3 | 2.5 | 11.0 | ‖ | **14.1** * | +| close-1k, k req/s | **5.2 ‡** * | 3.7 | 4.9 ‡ | 3.6 | ‖ | **5.2 ‡** * | + +* With splice() devel moves 31-42 Gbit/s through a tunnel, 2-2.9 times squid; + copying with 16 KB buffers (`-s0`) it does 10-13, below 1.0.1's 19-20 with + its 64 KB buffers, which is what the automatic mode is for. +* squid answers 1 KB requests twice as fast as devel on ARM, 19% faster on + Intel; with 2000 clients squid leads on ARM (15.0k against 10.2k), devel on + Intel (15.4k against 11.3k). +* With a new tunnel per request proxychains on the client sets the pace on + ARM; on Intel devel does 5.2k a second, squid 3.6k. ## SOCKSv5 tunnel -| scenario (M4 / Intel / average) | devel -s0 | 0.9.4 | 1.0.1 | dante | -|---|---:|---:|---:|---:| -| stream1-down, Gbit/s | **22.5 / 18.9 / 20.7** | 10.6 / 5.7 / 8.2 | **22.6 / 19.7 / 21.2** | 11.2 / 7.8 / 9.5 | -| stream16, Gbit/s | **21.1 / 18.5 / 19.8** | 8.7 / 5.8 / 7.3 | **21.0 / 19.2 / 20.1** | 14.5 / 15.2 / 14.8 | -| ka-1k, k req/s | 19.7 / 28.5 / 24.1 | 19.6 / 29.5 / 24.6 | 19.6 / 29.9 / 24.7 | **26.8 / 25.3 / 26.0** * | -| ka-50k-2000, k req/s | 10.9 / 13.1 / 12.0 | 8.2 / 7.3 / 7.8 | 11.0 / 12.6 / 11.8 | **14.7 / 11.6 / 13.1** * | -| close-1k, k req/s | 3.9 / 4.3 / 4.1 | **3.9 / 4.4 / 4.2** * | 3.6 / 3.8 / 3.7 | 3.8 / 1.5 / 2.7 | +**ARM (M4)** -| scenario (M4 / Intel / average) | devel -s1 (download) | devel -s3 (both) | devel -s (auto) | 0.9.4 splice | -|---|---:|---:|---:|---:| -| stream1-down, Gbit/s | **47.3 / 32.2 / 39.8** * | **47.7 / 32.7 / 40.2** * | **47.0 / 31.7 / 39.4** * | **47.5 / 31.3 / 39.4** * | -| stream16, Gbit/s | 20.9 / 17.8 / 19.4 | **37.5 / 29.4 / 33.5** * | **38.3 / 29.2 / 33.7** * | 24.2 / 26.8 / 25.5 | -| ka-1k, k req/s | 17.8 / 27.3 / 22.5 | 16.1 / 25.1 / 20.6 | **19.6 / 29.2 / 24.4** | 16.1 / 27.2 / 21.7 | -| ka-50k-2000, k req/s | **11.4 / 14.4 / 12.9** * | 10.5 / 14.4 / 12.5 | **11.4 / 15.1 / 13.2** * | **10.6 / 14.9 / 12.7** * | -| close-1k, k req/s | 3.7 / 3.7 / 3.7 | 3.6 / 3.8 / 3.7 | **3.9 / 4.1 / 4.0** | 3.7 / 4.0 / 3.8 | +| scenario (ARM) | devel -s0 | 1.0.1 | dante | ‖ | devel | +|---|---:|---:|---:|:-:|---:| +| stream1-down, Gbit/s | 12.9 | 19.7 | 10.6 | ‖ | **42.3 ‡** * | +| stream16, Gbit/s | 12.8 | 19.1 | 13.5 | ‖ | **33.9** * | +| ka-1k, k req/s | 17.5 | 17.6 | **25.1** * | ‖ | 17.5 | +| ka-50k-2000, k req/s | 8.5 | 10.1 | **13.3** * | ‖ | 10.2 | +| ka-2k-50k-2000, k req/s | 7.8 | 9.4 | **12.4** * | ‖ | 9.5 | +| close-1k, k req/s | **4.0** * | 3.4 | 3.6 | ‖ | **4.0** * | -* dante is 8% faster than devel with 1 KB requests and level with it with 2000 - clients; devel moves 1.3-2.2 times dante's bulk data copying (20 against - 9.5-15 Gbit/s) and 2.3-4.2 times with splice(). With a new tunnel per - request on the Skylake host dante manages 1.5k requests per second, devel - 4.3k. +**Intel (Skylake)** + +| scenario (Intel) | devel -s0 | 1.0.1 | dante | ‖ | devel | +|---|---:|---:|---:|:-:|---:| +| stream1-down, Gbit/s | 9.7 | 19.4 | 8.0 | ‖ | **33.7** * | +| stream16, Gbit/s | 9.7 | 18.5 | 15.8 | ‖ | **31.2** * | +| ka-1k, k req/s | **29.6** * | **30.5** * | 25.0 | ‖ | **30.9** * | +| ka-50k-2000, k req/s | 10.0 | 12.9 | 11.7 | ‖ | **15.0** * | +| ka-2k-50k-2000, k req/s | 9.2 | 12.3 | 11.1 | ‖ | **14.5** * | +| close-1k, k req/s | **4.7** * | 3.9 | 1.5 | ‖ | **5.1** * | + +* devel moves 31-42 Gbit/s with splice(), 2-4.2 times dante. +* On ARM dante answers 43% more 1 KB requests and 30% more with 2000 clients; + on Intel devel leads both (by 24% and 28%). +* With a new tunnel per request devel manages 4.0-5.1k a second, dante 3.6k + on ARM and 1.5k on Intel. With 2000 clients dante needs 0.8-1.4 GB, devel + 200 MB. ## TCP relay -| scenario (M4 / Intel / average) | devel -s0 | 0.9.4 | 1.0.1 | nginx | haproxy | -|---|---:|---:|---:|---:|---:| -| stream1-down, Gbit/s | **22.4 / 19.5 / 20.9** | 10.2 / 5.7 / 8.0 | **22.0 / 19.3 / 20.6** | 13.5 / 9.0 / 11.3 | 13.2 / 8.7 / 11.0 | -| stream16, Gbit/s | **20.9 / 19.0 / 20.0** | 8.8 / 5.8 / 7.3 | **20.9 / 18.3 / 19.6** | 13.5 / 9.7 / 11.6 | 8.9 / 8.7 / 8.8 | -| ka-1k, k req/s | 20.1 / 30.2 / 25.1 | 20.1 / 31.1 / 25.6 | 20.1 / 30.4 / 25.2 | **44.0 / 49.6 / 46.8** * | **45.6 / 46.8 / 46.2** * | -| ka-50k-2000, k req/s | **12.3 / 14.9 / 13.6** | 9.1 / 8.4 / 8.7 | **12.2 / 14.9 / 13.5** | **14.4 / 12.5 / 13.5** | 11.3 / 9.8 / 10.5 | -| close-1k, k req/s | 5.6 / 5.1 / 5.4 | 5.6 / 5.5 / 5.6 | 5.1 / 4.8 / 5.0 | 8.8 / 8.4 / 8.6 | **9.7 / 8.1 / 8.9** | +**ARM (M4)** -| scenario (M4 / Intel / average) | devel -s1 (download) | devel -s3 (both) | devel -s (auto) | 0.9.4 splice | haproxy splice (both) | haproxy splice-response (download) | haproxy splice-auto | -|---|---:|---:|---:|---:|---:|---:|---:| -| stream1-down, Gbit/s | 46.2 / 27.7 / 36.9 | **48.1 / 31.9 / 40.0** * | **47.8 / 32.1 / 39.9** * | **47.5 / 30.1 / 38.8** * | **46.9 / 31.2 / 39.1** * | **47.1 / 27.4 / 37.3** * | 30.2 / 30.7 / 30.5 | -| stream16, Gbit/s | 20.8 / 19.1 / 20.0 | **38.1 / 32.2 / 35.2** * | **38.3 / 30.5 / 34.4** * | 24.3 / 28.5 / 26.4 | 23.0 / 31.5 / 27.3 | 8.8 / 8.4 / 8.6 | 15.7 / 30.0 / 22.9 | -| ka-1k, k req/s | 18.1 / 28.1 / 23.1 | 16.3 / 27.4 / 21.9 | 20.0 / 29.7 / 24.9 | 16.5 / 26.4 / 21.4 | 32.7 / 37.2 / 35.0 | 39.6 / 41.5 / 40.5 | **46.4 / 46.3 / 46.4** * | -| ka-50k-2000, k req/s | 12.6 / 17.9 / 15.2 | 11.5 / 17.7 / 14.6 | 12.5 / 17.6 / 15.0 | 11.6 / 17.0 / 14.3 | 20.3 / 21.9 / 21.1 | **24.3 / 19.4 / 21.9** * | 17.5 / 19.3 / 18.4 | -| close-1k, k req/s | 5.2 / 4.9 / 5.0 | 5.1 / 4.9 / 5.0 | 5.6 / 5.3 / 5.4 | 5.2 / 4.7 / 5.0 | **10.1 / 8.8 / 9.4** * | **10.1 / 8.8 / 9.5** * | 9.6 / 8.1 / 8.9 | +| scenario (ARM) | devel -s0 | 1.0.1 | haproxy | nginx | ‖ | devel | haproxy splice-auto | +|---|---:|---:|---:|---:|:-:|---:|---:| +| stream1-down, Gbit/s | 13.4 | **20.3** | 13.1 | 12.7 | ‖ | **41.0** * | 26.2 | +| stream16, Gbit/s | 12.8 | **19.2** | 8.1 ‡ | 12.7 | ‖ | **34.1 ‡** * | 14.3 ‡ | +| ka-1k, k req/s | 17.9 | 17.9 | **39.0** * | **37.5** | ‖ | 17.9 | **39.8** * | +| ka-50k-2000, k req/s | 9.4 | 11.1 | 10.3 | **12.8** | ‖ | 11.3 | **15.8** * | +| ka-2k-50k-2000, k req/s | 8.7 | 10.3 | 10.1 | **11.9** | ‖ | 10.3 | **15.5** * | +| close-1k, k req/s | 5.9 | 4.7 | **8.9** * | 8.0 | ‖ | 5.9 | **8.9** * | -* As a relay devel copies 1.7-2.3 times what nginx and haproxy do; with splice() - it is level with haproxy splicing on one stream and 26-29% ahead with 16. - haproxy's `splice-auto` reaches 30 Gbit/s on one stream, the others 37-40. -* nginx and haproxy answer 1.9 times more 1 KB requests, 1.6 times more with - a new connection for each. -* With 2000 clients and 50 KB answers devel and nginx are level copying - (13.5k); splice() takes devel to 15.2k and haproxy from 10.5k to 21.9k. With - 2000 connections devel runs 2000 threads on one CPU, haproxy one event loop. -* Where splice() is forced, 1 KB requests lose 13-24% (devel `-s3`, 0.9.4, - haproxy `splice`); `-s` and `splice-auto` lose nothing. +**Intel (Skylake)** -## 16 KB buffers +| scenario (Intel) | devel -s0 | 1.0.1 | haproxy | nginx | ‖ | devel | haproxy splice-auto | +|---|---:|---:|---:|---:|:-:|---:|---:| +| stream1-down, Gbit/s | 9.6 | **19.0** | 9.4 | 9.6 | ‖ | **32.9** * | 31.5 | +| stream16, Gbit/s | 9.9 | **18.7** | 9.1 | 9.8 | ‖ | **31.5** * | **31.0 ‡** * | +| ka-1k, k req/s | 29.5 | 29.5 | 45.1 | **48.7** * | ‖ | 29.8 | **47.1** * | +| ka-50k-2000, k req/s | 11.0 | **14.4** | 9.8 | 11.9 | ‖ | 17.5 | **18.8** * | +| ka-2k-50k-2000, k req/s | 10.4 | **13.3** | 9.5 | 11.6 | ‖ | 15.3 | **17.9** * | +| close-1k, k req/s | 6.4 | 4.8 | 7.9 | **8.6** * | ‖ | 6.5 | **8.3** | -`-b16384` against the default 64 KB, from the HTTP proxy session: - -| | devel -s0 | devel -s0 -b16384 | devel -s | devel -s -b16384 | -|---|---:|---:|---:|---:| -| http-ka-1m, Gbit/s (average) | 19.1 | 11.6 | 30.9 | 30.8 | -| http-ka-50k-2000, k req/s (average) | 10.9 | 8.7 | 10.5 | 10.2 | -| peak memory with 2000 clients (POST + 50 KB), MB, M4 / Skylake | 199 / 156 | 166 / 128 | 192 / 155 | 150 / 128 | -| peak memory with 16 streams, MB, M4 | 72 | 34 | | | - -Copying with 16 KB buffers takes four times as many system calls for the same -data, which costs 20-40% of the throughput; with splice() the bulk of the data -goes through the pipe and the buffer size hardly matters. The memory saved -with 2000 clients, 17-22%, is far less than the buffers' size would suggest. A smaller buffer also limits what data filters see of a body, and -`tlspr` needs more than 16 KB. +* On streams devel splices 32-41 Gbit/s; haproxy `splice-auto` is level on + Intel with one stream (31.5) and sixteen (31.0), behind on ARM (26 and 14). +* nginx and haproxy answer 2.1-2.2 times more 1 KB requests on ARM, 1.5-1.6 + times on Intel. +* With 2000 clients and 50 KB answers haproxy `splice-auto` leads, 40% ahead + of devel on ARM and 7% on Intel. ## HTTPS: TLS in the proxy -Measured on 13 October 2026 with devel as of then: 16 KB buffers, `-s` by -default, the thread pool ([data/](data/) `*-tls.csv`). Every proxy relays -TCP, ending TLS (`term`: HTTPS from wrk, plain TCP to the origin) or -beginning it (`init`: plain HTTP from wrk, HTTPS to the origin): 3proxy +Measured on 13 October 2026 with devel as of the evening: 16 KB buffers, +splice() in its automatic mode, the thread pool, TLS read a record ahead and +relayed a full record per write ([data/](data/) `*-tls.csv`). Every proxy +relays TCP, ending TLS (`term`: TLS from the client, plain TCP to the origin) +or beginning it (`init`: plain TCP from the client, TLS to the origin): 3proxy `tcppm` under `ssl_serv` / `ssl_cli`, nginx `stream` with `ssl` and -`proxy_ssl`, haproxy `mode tcp` with `ssl crt` and `server ... ssl`, stunnel. -ECDSA P-256 certificate, TLS 1.3, OpenSSL in every proxy; "no proxy" is wrk -speaking HTTPS to the origin itself. Every proxy ran at its CPU limit. +`proxy_ssl`, haproxy `mode tcp` with `ssl crt` and `server ... ssl`, envoy +`tcp_proxy` with TLS transport sockets, stunnel. ECDSA P-256 certificate, +TLS 1.3; envoy uses BoringSSL, the others OpenSSL (3.0.13 on ARM, 3.5.8 on +Intel). "no proxy" is the client speaking TLS to the origin itself. The +`*-ka-*` and `*-close-*` scenarios are wrk over HTTPS, the `*-stream*` ones +iperf3 wrapped in TLS by `bench/mix tls`; on one stream that wrapper, a Go +program on one core, can set the pace (‡). -| scenario (M4 / Intel / average) | no proxy | ‖ | devel | nginx | haproxy | stunnel | -|---|---:|:-:|---:|---:|---:|---:| -| tls-term-ka-1k, k req/s | 140.5 / 107.2 / 123.8 | ‖ | 16.4 / 24.4 / 20.4 | **33.1 / 35.6 / 34.3** | 31.8 / 33.7 / 32.8 | 13.8 / 20.2 / 17.0 | -| tls-term-ka-1m, Gbit/s | 31.53 / 22.68 / 27.10 | ‖ | 5.95 / 4.92 / 5.44 | **6.84 / 6.05 / 6.44** | 5.00 / 5.90 / 5.45 | 4.48 / 5.13 / 4.80 | -| tls-term-close-1k, k req/s | 12.4 / 4.2 / 8.3 | ‖ | 2.7 / 1.2 / 2.0 | **3.3 / 1.4 / 2.4** | 2.9 / 1.4 / 2.1 | 2.0 / 1.1 / 1.5 | -| tls-init-ka-1k, k req/s | 140.4 / 105.9 / 123.1 | ‖ | 16.4 / 24.0 / 20.2 | **34.0 / 36.2 / 35.1** | 32.5 / 33.1 / 32.8 | 14.0 / 20.2 / 17.1 | -| tls-init-ka-1m, Gbit/s | 31.87 / 20.96 / 26.41 | ‖ | **5.60 / 5.16 / 5.38** | **5.80 / 5.43 / 5.61** | 4.58 / 5.12 / 4.85 | 4.15 / 4.56 / 4.35 | -| tls-init-close-1k, k req/s | 12.4 / 3.8 / 8.1 | ‖ | 2.0 / 0.9 / 1.4 | **3.4 / 1.3 / 2.3** | 2.5 / 1.0 / 1.7 | 2.0 / 1.0 / 1.5 | +### ARM (M4) -The proxy's CPU per request (per gigabyte for 1 MB answers), its p99 -latency and its peak memory, M4 / Intel: +**throughput, Gbit/s (thousands of requests per second for HTTP)** -| scenario | devel | nginx | haproxy | stunnel | -|---|---:|---:|---:|---:| -| tls-term-ka-1k, µs | 61 / 41 | **30 / 28** | 31 / 30 | 72 / 50 | -| tls-term-ka-1m, s/GB | 1.35 / 1.63 | **1.17 / 1.33** | 1.62 / 1.37 | 1.79 / 1.57 | -| tls-term-close-1k, µs | 370 / 809 | **302 / 699** | 347 / 721 | 503 / 930 | -| tls-init-ka-1m, s/GB | 1.43 / 1.55 | **1.38 / 1.48** | 1.77 / 1.58 | 1.93 / 1.76 | -| tls-init-close-1k, µs | 501 / 1159 | **293 / 796** | 401 / 1010 | 495 / 1055 | -| tls-term-close-1k, p99 ms | 9 / 44 | **2 / 4** | 16 / 41 | 26 / 57 | -| tls-init-close-1k, p99 ms | 46 / 122 | **20 / 71** | 27 / 89 | 49 / 180 | -| peak memory, MB | **12-18 / 14-77** | 31-44 / 32-48 | 36-48 / 38-53 | 13-21 / 16-34 | +| scenario | no proxy | ‖ | devel | haproxy | nginx | envoy | stunnel | +|---|---:|:-:|---:|---:|---:|---:|---:| +| tls-term-ka-1k | 1.43 (139.2k/s) | ‖ | 0.17 (16.8k/s) | 0.33 (31.9k/s) | **0.34 (33.4k/s)** | 0.32 (30.9k/s) | 0.14 (13.9k/s) | +| tls-term-ka-1m | 29.81 (3.5k/s) | ‖ | 7.20 (0.9k/s) | 4.88 (0.6k/s) ‡ | 7.24 (0.9k/s) | **10.57 (1.3k/s) ‡** | 4.66 (0.6k/s) | +| tls-term-close-1k | 0.13 (12.3k/s) | ‖ | 0.03 (2.7k/s) ‡ | 0.03 (2.8k/s) ‡ | 0.03 (3.3k/s) ‡ | **0.04 (3.9k/s) ‡** | 0.02 (1.9k/s) ‡ | +| tls-term-stream1-down | 9.3 | ‖ | 9.2 ‡ | 8.8 ‡ | 8.6 ‡ | **9.6 ‡** | 4.7 | +| tls-term-stream16 | 19.3 | ‖ | 6.7 | 5.0 | 6.7 | **9.4** | 4.3 | +| tls-init-ka-1k | 1.43 (138.8k/s) | ‖ | 0.18 (17.2k/s) | 0.33 (32.4k/s) | **0.35 (33.7k/s)** | 0.32 (31.3k/s) | 0.15 (14.2k/s) | +| tls-init-ka-1m | 28.69 (3.4k/s) | ‖ | 6.14 (0.7k/s) | 4.63 (0.6k/s) | 5.84 (0.7k/s) | **8.59 (1.0k/s)** | 4.19 (0.5k/s) | +| tls-init-close-1k | 0.13 (12.2k/s) | ‖ | 0.02 (2.0k/s) | 0.03 (2.5k/s) | 0.03 (3.4k/s) | **0.06 (5.7k/s)** | 0.02 (1.9k/s) | +| tls-init-stream1-down | 9.2 | ‖ | 7.2 | 5.8 | 7.2 | **10.9** | 4.4 | +| tls-init-stream16 | 19.6 | ‖ | 8.3 ‡ | 5.9 ‡ | 7.8 ‡ | **11.0 ‡** | 5.2 ‡ | -* **1 MB answers:** devel is level with nginx beginning TLS and with haproxy - ending it, 13-25% ahead of stunnel; nginx is fastest ending it. With TLS the - cost is the cipher, so the threads matter little. -* **1 KB requests over kept-alive connections:** devel answers 59% of what - nginx does, the same share as without TLS (`relay-ka-1k`); the thread per - connection, not TLS, sets it. stunnel, a thread per connection too, is - below devel. -* **A handshake for every request:** ending TLS devel does 82-94% of nginx and - haproxy, 30% more than stunnel; beginning it 60-68% of nginx. nginx - (`proxy_ssl_session_reuse`) and haproxy resume the session to the server, so - most of their handshakes there are short; devel and stunnel make a full one - for every connection. -* Handshakes cost two to three times more on the Skylake host than on the M4, - with or without a proxy. +**proxy CPU time per gigabyte or per request** + +| scenario | no proxy | ‖ | devel | haproxy | nginx | envoy | stunnel | +|---|---:|:-:|---:|---:|---:|---:|---:| +| tls-term-ka-1k | – | ‖ | 60 µs/req | 31 µs/req | **30 µs/req** | 32 µs/req | 72 µs/req | +| tls-term-ka-1m | – | ‖ | 1.11 s/GB | 1.66 s/GB | 1.10 s/GB | **0.76 s/GB** | 1.72 s/GB | +| tls-term-close-1k | – | ‖ | 370 µs/req | 358 µs/req | 307 µs/req | **255 µs/req** | 514 µs/req | +| tls-term-stream1-down | – | ‖ | 0.85 s/GB | 0.87 s/GB | 0.91 s/GB | **0.77 s/GB** | 1.70 s/GB | +| tls-term-stream16 | – | ‖ | 1.20 s/GB | 1.60 s/GB | 1.20 s/GB | **0.85 s/GB** | 1.88 s/GB | +| tls-init-ka-1k | – | ‖ | 58 µs/req | 31 µs/req | **30 µs/req** | 32 µs/req | 71 µs/req | +| tls-init-ka-1m | – | ‖ | 1.31 s/GB | 1.75 s/GB | 1.37 s/GB | **0.93 s/GB** | 1.92 s/GB | +| tls-init-close-1k | – | ‖ | 499 µs/req | 408 µs/req | 297 µs/req | **176 µs/req** | 518 µs/req | +| tls-init-stream1-down | – | ‖ | 1.11 s/GB | 1.37 s/GB | 1.11 s/GB | **0.74 s/GB** | 1.81 s/GB | +| tls-init-stream16 | – | ‖ | 0.96 s/GB | 1.35 s/GB | 1.03 s/GB | **0.73 s/GB** | 1.53 s/GB | + +**proxy peak memory, MB** + +| scenario | no proxy | ‖ | devel | haproxy | nginx | envoy | stunnel | +|---|---:|:-:|---:|---:|---:|---:|---:| +| tls-term-ka-1k | – | ‖ | **12** | 36 | 32 | 18 | 13 | +| tls-term-ka-1m | – | ‖ | **15** | 43 | 44 | 41 | **13** | +| tls-term-close-1k | – | ‖ | **14** | 37 | 32 | 19 | 21 | +| tls-term-stream1-down | – | ‖ | 7 | 40 | 34 | 26 | **5** | +| tls-term-stream16 | – | ‖ | **26** | 55 | 53 | 66 | **21** | +| tls-init-ka-1k | – | ‖ | **13** | 37 | 33 | 17 | 14 | +| tls-init-ka-1m | – | ‖ | **16** | 49 | 44 | 41 | **18** | +| tls-init-close-1k | – | ‖ | **17** | 38 | 33 | **19** | **19** | +| tls-init-stream1-down | – | ‖ | **6** | 40 | 37 | 25 | **6** | +| tls-init-stream16 | – | ‖ | **62** | **59** | 72 | 122 | **56** | + +### Intel (Skylake) + +**throughput, Gbit/s (thousands of requests per second for HTTP)** + +| scenario | no proxy | ‖ | devel | haproxy | nginx | envoy | stunnel | +|---|---:|:-:|---:|---:|---:|---:|---:| +| tls-term-ka-1k | 0.97 (95.0k/s) | ‖ | 0.23 (22.5k/s) | 0.33 (32.5k/s) | **0.34 (33.4k/s)** | 0.28 (27.1k/s) | 0.19 (18.2k/s) | +| tls-term-ka-1m | 19.50 (2.3k/s) | ‖ | **5.90 (0.7k/s)** | **5.48 (0.7k/s)** | **5.82 (0.7k/s)** | **5.80 (0.7k/s)** | 4.51 (0.5k/s) | +| tls-term-close-1k | 0.03 (3.3k/s) | ‖ | 0.01 (1.2k/s) ‡ | 0.01 (1.3k/s) ‡ | 0.01 (1.3k/s) ‡ | **0.02 (2.0k/s) ‡** | 0.01 (1.0k/s) ‡ | +| tls-term-stream1-down | 7.1 | ‖ | **5.6** | 6.1 | 5.9 | **6.9** | 5.1 | +| tls-term-stream16 | 16.2 | ‖ | 5.9 | 5.0 | 5.7 | **6.8** | 4.7 | +| tls-init-ka-1k | 0.97 (95.2k/s) | ‖ | 0.24 (23.1k/s) | **0.32 (31.1k/s)** | **0.31 (30.6k/s)** | **0.30 (29.3k/s)** | 0.19 (18.4k/s) | +| tls-init-ka-1m | 20.70 (2.5k/s) | ‖ | 5.22 (0.6k/s) | 4.85 (0.6k/s) | 5.34 (0.6k/s) | **6.35 (0.8k/s)** | 4.41 (0.5k/s) | +| tls-init-close-1k | 0.03 (3.3k/s) | ‖ | 0.01 (0.8k/s) | 0.01 (1.0k/s) | 0.01 (1.2k/s) | **0.03 (3.0k/s)** | 0.01 (0.9k/s) | +| tls-init-stream1-down | 7.5 | ‖ | 6.1 | 5.2 | 6.0 | **7.8** | 4.7 | +| tls-init-stream16 | 16.7 | ‖ | **7.1** | 5.9 ‡ | 6.8 | **7.3** | 5.6 | + +**proxy CPU time per gigabyte or per request** + +| scenario | no proxy | ‖ | devel | haproxy | nginx | envoy | stunnel | +|---|---:|:-:|---:|---:|---:|---:|---:| +| tls-term-ka-1k | – | ‖ | 44 µs/req | 31 µs/req | **30 µs/req** | 37 µs/req | 55 µs/req | +| tls-term-ka-1m | – | ‖ | **1.36 s/GB** | **1.47 s/GB** | **1.38 s/GB** | **1.38 s/GB** | 1.78 s/GB | +| tls-term-close-1k | – | ‖ | 821 µs/req | 774 µs/req | 781 µs/req | **510 µs/req** | 1,022 µs/req | +| tls-term-stream1-down | – | ‖ | **1.38 s/GB** | 1.28 s/GB | 1.32 s/GB | **1.08 s/GB** | 1.57 s/GB | +| tls-term-stream16 | – | ‖ | 1.35 s/GB | 1.61 s/GB | 1.40 s/GB | **1.18 s/GB** | 1.69 s/GB | +| tls-init-ka-1k | – | ‖ | 43 µs/req | **32 µs/req** | **33 µs/req** | **34 µs/req** | 55 µs/req | +| tls-init-ka-1m | – | ‖ | 1.53 s/GB | 1.67 s/GB | 1.50 s/GB | **1.26 s/GB** | 1.82 s/GB | +| tls-init-close-1k | – | ‖ | 1,187 µs/req | 1,041 µs/req | 828 µs/req | **336 µs/req** | 1,070 µs/req | +| tls-init-stream1-down | – | ‖ | 1.30 s/GB | 1.53 s/GB | 1.34 s/GB | **1.03 s/GB** | 1.71 s/GB | +| tls-init-stream16 | – | ‖ | **1.14 s/GB** | 1.37 s/GB | 1.17 s/GB | **1.11 s/GB** | 1.43 s/GB | + +**proxy peak memory, MB** + +| scenario | no proxy | ‖ | devel | haproxy | nginx | envoy | stunnel | +|---|---:|:-:|---:|---:|---:|---:|---:| +| tls-term-ka-1k | – | ‖ | **14** | 37 | 35 | 24 | 16 | +| tls-term-ka-1m | – | ‖ | **20** | 49 | 48 | 46 | **20** | +| tls-term-close-1k | – | ‖ | **24** | **38** | **34** | **26** | **34** | +| tls-term-stream1-down | – | ‖ | 8 | 39 | 36 | 33 | **8** | +| tls-term-stream16 | – | ‖ | 80 | **44** | 85 | 138 | 89 | +| tls-init-ka-1k | – | ‖ | **16** | 41 | 36 | 24 | 17 | +| tls-init-ka-1m | – | ‖ | **18** | 50 | 46 | 48 | 21 | +| tls-init-close-1k | – | ‖ | 59 | 40 | 35 | **27** | 34 | +| tls-init-stream1-down | – | ‖ | **11** | 39 | 35 | 32 | **10** | +| tls-init-stream16 | – | ‖ | 95 | **55** | 114 | 148 | 97 | + +* **Bulk data:** devel is level with nginx and ahead of stunnel on both + machines, and ahead of haproxy but for one stream ending TLS on Intel; envoy leads on ARM by 30-50% with a third less CPU per + gigabyte, and on Intel by 0-30%. BoringSSL's ciphers on ARM are the likely + difference: on Intel, ending TLS for 1 MB answers, the four are level. +* **1 KB requests over kept-alive connections:** devel answers half of what + nginx, haproxy and envoy do on ARM and two thirds on Intel, the same share + as without TLS: the thread per connection sets it, not TLS. stunnel, a + thread per connection too, is below devel. +* **A handshake for every request:** ending TLS devel does 80-95% of nginx + and haproxy, beginning it 60-80%; nginx and haproxy resume the session to + the server, devel makes a full handshake each time. envoy does 1.2-1.5 + times nginx ending TLS and 1.7-2.5 times beginning it. +* **Memory:** with one connection or a few dozen, devel and stunnel need the + least, mostly 6-24 MB (devel 59 MB beginning a TLS connection per request + on Intel), against 32-50 MB for nginx and haproxy and 17-48 MB for + envoy. With 16 streams on Intel haproxy needs the least (44-55 MB, devel + 80-95), and envoy the most everywhere (66-148 MB). ## Many connections: the mix scenarios Measured on 13 October 2026 with the same devel ([data/](data/) `*-mix.csv`), 20000 connections open through the proxy, `--mix-active` 2000; `-3k` at a -fixed 3000 exchanges a second. dante is left out on the Skylake host: with -20000 connections it needs more memory than the machine has (6.6 GB on the -M4) and the OOM killer ended the origin. ‡: limited by the client. +fixed 3000 exchanges a second. dante is left out on Intel: with 20000 +connections it needs more memory than the machine has (6.6 GB on ARM) and the +OOM killer ended the origin. ‡: limited by the client. -| scenario (M4 / Intel), k exchanges/s | devel | devel -s0 | 1.0.1 | nginx | haproxy | haproxy splice-auto | squid | tinyproxy | dante | -|---|---:|---:|---:|---:|---:|---:|---:|---:|---:| -| mix-http | 5.0 / 7.0 | 4.9 / 6.5 | 3.9 / 5.4 | **7.5‡ / 7.9** | 6.5‡ / 7.1 | 6.5‡ / **8.6** | 2.7‡ / 2.2 | 1.5 / 1.7 | | -| mix-connect | 5.7‡ / **9.1‡** | 5.5 / 8.2 | 5.5‡ / 7.3 | | | | **7.3‡** / 7.2 | 2.0 / 2.8 | | -| mix-socks | **5.7‡ / 9.0** | 5.5 / 8.0 | 5.8‡ / 8.4 | | | | | | 3.5 / - | -| mix-relay | 5.9‡ / 9.2 | 5.7 / 8.1 | 5.8‡ / 8.6 | **7.5‡ / 9.9‡** | 6.8‡ / 7.9‡ | 7.1‡ / 8.5 | | | | +### ARM (M4) -| at 3000 a second (M4 / Intel) | devel | devel -s0 | 1.0.1 | nginx | haproxy | haproxy splice-auto | squid | tinyproxy | dante | -|---|---:|---:|---:|---:|---:|---:|---:|---:|---:| -| CPU per exchange, µs: mix-http-3k, mix-relay-3k | 236 / 201, 209 / 171 | 236 / 217, 215 / 174 | 278 / 238, 212 / 170 | 169 / 187, 178 / 162 | 162 / 168, 179 / 166 | **162 / 160, 178 / 159** | 326 / 489 | 678 / 647 | | -| peak memory, MB: mix-http-3k | 2627 / 2659 | 2614 / 2644 | 2752 / 2820 | **139 / 154** | 150 / 179 | 154 / 180 | 603 / 868 | 803 / 765 | | -| peak memory, MB: mix-connect-3k, mix-socks-3k | 2670 / 2660, 1422 / 1410 | 2653 / 2648, 1404 / 1398 | 2831 / 2834, 1559 / 1574 | | | | 1289 / 1371 | **1025 / 1038** | 6747 / - (socks) | -| peak memory, MB: mix-relay-3k | 1306 / 1376 | 1291 / 1358 | 1456 / 1531 | 574 / 604 | **267 / 314** | **264 / 318** | | | | +**throughput, Gbit/s (thousands of requests per second for HTTP)** -* Copying many small exchanges devel does 75-80% of nginx and haproxy on the - M4 and 85-95% on the Skylake host, where it is first through CONNECT and - SOCKS; at a fixed rate it takes 3-8% more CPU than they do as a relay, 7-25% +| scenario | devel -s0 | 1.0.1 | haproxy | nginx | tinyproxy | squid | dante | ‖ | devel | haproxy splice-auto | +|---|---:|---:|---:|---:|---:|---:|---:|:-:|---:|---:| +| mix-http | 1.80 (4.9k/s) | 1.42 (3.9k/s) | 1.83 (6.5k/s) ‡ | **2.76 (7.5k/s) ‡** | 0.59 (1.5k/s) | 0.92 (2.7k/s) ‡ | – (–k/s) | ‖ | **1.84 (5.0k/s)** | **1.81 (6.5k/s) ‡** | +| mix-connect | 2.08 (5.5k/s) | 2.07 (5.5k/s) ‡ | – (–k/s) | – (–k/s) | 0.74 (2.0k/s) | **2.76 (7.3k/s) ‡** | – (–k/s) | ‖ | **2.16 (5.7k/s) ‡** | – (–k/s) | +| mix-socks | 2.03 (5.5k/s) | **2.17 (5.8k/s) ‡** | – (–k/s) | – (–k/s) | – (–k/s) | – (–k/s) | 1.31 (3.5k/s) | ‖ | **2.13 (5.7k/s) ‡** | – (–k/s) | +| mix-relay | 2.07 (5.7k/s) | 2.12 (5.8k/s) ‡ | 2.31 (6.8k/s) ‡ | **2.72 (7.5k/s) ‡** | – (–k/s) | – (–k/s) | – (–k/s) | ‖ | 2.16 (5.9k/s) ‡ | **2.50 (7.1k/s) ‡** | +| mix-http-3k | 1.09 (3.0k/s) | 1.08 (3.0k/s) | 1.08 (3.0k/s) | **1.09 (3.0k/s)** | 0.58 (1.5k/s) | **1.09 (3.0k/s) ‡** | – (–k/s) | ‖ | 1.08 (3.0k/s) | **1.08 (3.0k/s)** | +| mix-connect-3k | **1.09 (3.0k/s)** | 1.08 (3.0k/s) | – (–k/s) | – (–k/s) | 0.74 (2.0k/s) | 1.08 (3.0k/s) | – (–k/s) | ‖ | **1.08 (3.0k/s)** | – (–k/s) | +| mix-socks-3k | 1.08 (3.0k/s) | 1.08 (3.0k/s) | – (–k/s) | – (–k/s) | – (–k/s) | – (–k/s) | **1.13 (3.1k/s)** | ‖ | **1.08 (3.0k/s)** | – (–k/s) | +| mix-relay-3k | **1.08 (3.0k/s)** | 1.08 (3.0k/s) | **1.09 (3.0k/s)** | 1.08 (3.0k/s) | – (–k/s) | – (–k/s) | – (–k/s) | ‖ | **1.08 (3.0k/s)** | **1.09 (3.0k/s)** | + +**proxy CPU time per gigabyte or per request** + +| scenario | devel -s0 | 1.0.1 | haproxy | nginx | tinyproxy | squid | dante | ‖ | devel | haproxy splice-auto | +|---|---:|---:|---:|---:|---:|---:|---:|:-:|---:|---:| +| mix-http | 205 µs/req | 272 µs/req | 156 µs/req | **134 µs/req** | 675 µs/req | 375 µs/req | – µs/req | ‖ | 211 µs/req | **157 µs/req** | +| mix-connect | 190 µs/req | 187 µs/req | – µs/req | – µs/req | 507 µs/req | **139 µs/req** | – µs/req | ‖ | **182 µs/req** | – µs/req | +| mix-socks | 187 µs/req | **179 µs/req** | – µs/req | – µs/req | – µs/req | – µs/req | 298 µs/req | ‖ | **179 µs/req** | – µs/req | +| mix-relay | 186 µs/req | 182 µs/req | 151 µs/req | **136 µs/req** | – µs/req | – µs/req | – µs/req | ‖ | 177 µs/req | **143 µs/req** | +| mix-http-3k | 236 µs/req | 278 µs/req | **162 µs/req** | 169 µs/req | 678 µs/req | 326 µs/req | – µs/req | ‖ | 236 µs/req | **162 µs/req** | +| mix-connect-3k | 218 µs/req | 216 µs/req | – µs/req | – µs/req | 509 µs/req | **194 µs/req** | – µs/req | ‖ | **212 µs/req** | – µs/req | +| mix-socks-3k | **215 µs/req** | **211 µs/req** | – µs/req | – µs/req | – µs/req | – µs/req | 334 µs/req | ‖ | **210 µs/req** | – µs/req | +| mix-relay-3k | 215 µs/req | 212 µs/req | **179 µs/req** | **178 µs/req** | – µs/req | – µs/req | – µs/req | ‖ | 209 µs/req | **178 µs/req** | + +**proxy peak memory, MB** + +| scenario | devel -s0 | 1.0.1 | haproxy | nginx | tinyproxy | squid | dante | ‖ | devel | haproxy splice-auto | +|---|---:|---:|---:|---:|---:|---:|---:|:-:|---:|---:| +| mix-http | 2,407 | 2,729 | 529 | **153** | 817 | 839 | – | ‖ | 2,438 | **556** | +| mix-connect | 2,725 | 3,220 | – | – | **1,033** | 1,768 | – | ‖ | **2,760** | – | +| mix-socks | **1,482** | 2,007 | – | – | – | – | 6,580 | ‖ | **1,506** | – | +| mix-relay | 1,246 | 1,767 | **293** | 608 | – | – | – | ‖ | 1,296 | **263** | +| mix-http-3k | 2,614 | 2,752 | 150 | **139** | 803 | 603 | – | ‖ | 2,627 | **154** | +| mix-connect-3k | 2,653 | 2,831 | – | – | **1,025** | 1,289 | – | ‖ | **2,670** | – | +| mix-socks-3k | **1,404** | 1,559 | – | – | – | – | 6,747 | ‖ | **1,422** | – | +| mix-relay-3k | 1,291 | 1,456 | **267** | 574 | – | – | – | ‖ | 1,306 | **264** | + +### Intel (Skylake) + +**throughput, Gbit/s (thousands of requests per second for HTTP)** + +| scenario | devel -s0 | 1.0.1 | haproxy | nginx | tinyproxy | squid | ‖ | devel | haproxy splice-auto | +|---|---:|---:|---:|---:|---:|---:|:-:|---:|---:| +| mix-http | 2.39 (6.5k/s) | 1.98 (5.4k/s) | 2.29 (7.1k/s) | **2.87 (7.9k/s)** | 0.65 (1.7k/s) | 0.80 (2.2k/s) | ‖ | 2.54 (7.0k/s) | **3.02 (8.6k/s)** | +| mix-connect | **3.08 (8.2k/s)** | 2.72 (7.3k/s) | – (–k/s) | – (–k/s) | 1.02 (2.8k/s) | 2.72 (7.2k/s) | ‖ | **3.39 (9.1k/s) ‡** | – (–k/s) | +| mix-socks | 2.98 (8.0k/s) | **3.12 (8.4k/s)** | – (–k/s) | – (–k/s) | – (–k/s) | – (–k/s) | ‖ | **3.37 (9.0k/s)** | – (–k/s) | +| mix-relay | 2.97 (8.1k/s) | 3.11 (8.6k/s) | 2.54 (7.9k/s) ‡ | **3.61 (9.9k/s) ‡** | – (–k/s) | – (–k/s) | ‖ | **3.35 (9.2k/s)** | 3.00 (8.5k/s) | +| mix-http-3k | **1.10 (3.0k/s)** | **1.09 (3.0k/s)** | 1.08 (3.0k/s) | 1.09 (3.0k/s) | 0.64 (1.6k/s) | 0.63 (2.1k/s) | ‖ | **1.09 (3.0k/s)** | **1.09 (3.0k/s)** | +| mix-connect-3k | **1.08 (3.0k/s)** | 1.08 (3.0k/s) | – (–k/s) | – (–k/s) | 1.01 (2.8k/s) | 1.08 (3.0k/s) | ‖ | **1.08 (3.0k/s)** | – (–k/s) | +| mix-socks-3k | **1.08 (3.0k/s)** | **1.08 (3.0k/s)** | – (–k/s) | – (–k/s) | – (–k/s) | – (–k/s) | ‖ | **1.08 (3.0k/s)** | – (–k/s) | +| mix-relay-3k | 1.08 (3.0k/s) | **1.08 (3.0k/s)** | **1.08 (3.0k/s)** | 1.08 (3.0k/s) | – (–k/s) | – (–k/s) | ‖ | **1.08 (3.0k/s)** | **1.08 (3.0k/s)** | + +**proxy CPU time per gigabyte or per request** + +| scenario | devel -s0 | 1.0.1 | haproxy | nginx | tinyproxy | squid | ‖ | devel | haproxy splice-auto | +|---|---:|---:|---:|---:|---:|---:|:-:|---:|---:| +| mix-http | 158 µs/req | 192 µs/req | 146 µs/req | **131 µs/req** | 623 µs/req | 462 µs/req | ‖ | 148 µs/req | **121 µs/req** | +| mix-connect | **125 µs/req** | **137 µs/req** | – µs/req | – µs/req | 372 µs/req | 145 µs/req | ‖ | **114 µs/req** | – µs/req | +| mix-socks | 129 µs/req | **123 µs/req** | – µs/req | – µs/req | – µs/req | – µs/req | ‖ | **115 µs/req** | – µs/req | +| mix-relay | 128 µs/req | **122 µs/req** | 131 µs/req | **105 µs/req** | – µs/req | – µs/req | ‖ | **113 µs/req** | 123 µs/req | +| mix-http-3k | 217 µs/req | 238 µs/req | **168 µs/req** | 187 µs/req | 647 µs/req | 489 µs/req | ‖ | 201 µs/req | **160 µs/req** | +| mix-connect-3k | **175 µs/req** | **172 µs/req** | – µs/req | – µs/req | 358 µs/req | 189 µs/req | ‖ | **171 µs/req** | – µs/req | +| mix-socks-3k | **181 µs/req** | **171 µs/req** | – µs/req | – µs/req | – µs/req | – µs/req | ‖ | **176 µs/req** | – µs/req | +| mix-relay-3k | 174 µs/req | 170 µs/req | 166 µs/req | **162 µs/req** | – µs/req | – µs/req | ‖ | 171 µs/req | **159 µs/req** | + +**proxy peak memory, MB** + +| scenario | devel -s0 | 1.0.1 | haproxy | nginx | tinyproxy | squid | ‖ | devel | haproxy splice-auto | +|---|---:|---:|---:|---:|---:|---:|:-:|---:|---:| +| mix-http | 2,432 | 2,828 | 486 | **222** | 786 | 639 | ‖ | 2,440 | **428** | +| mix-connect | 2,657 | 3,293 | – | – | **1,066** | 1,812 | ‖ | **2,692** | – | +| mix-socks | **1,412** | 1,985 | – | – | – | – | ‖ | **1,445** | – | +| mix-relay | 1,298 | 1,794 | **479** | 648 | – | – | ‖ | 1,328 | **334** | +| mix-http-3k | 2,644 | 2,820 | 179 | **154** | 765 | 868 | ‖ | 2,659 | **180** | +| mix-connect-3k | 2,648 | 2,834 | – | – | **1,038** | 1,371 | ‖ | **2,660** | – | +| mix-socks-3k | **1,398** | 1,574 | – | – | – | – | ‖ | **1,410** | – | +| mix-relay-3k | 1,358 | 1,531 | **314** | 604 | – | – | ‖ | 1,376 | **318** | + +* Copying many small exchanges devel does 75-80% of nginx and haproxy on ARM + and 85-95% on Intel, where it is first through CONNECT and SOCKS; at a fixed rate it takes 3-8% more CPU than they do as a relay, 7-25% more as an HTTP proxy, with the same p99. * Memory is where a thread per connection shows: about 65 KB a connection for a relay or a SOCKS tunnel (the thread's stack and two 16 KB buffers), about @@ -371,12 +510,21 @@ M4) and the OOM killer ended the origin. ‡: limited by the client. * nginx relays fail exchanges whose client shuts down its sending side first (`proxy_half_close` is off by default), 350-470 a run; with it on (`nginx::halfclose`, [data/](data/) `*-halfclose.csv`) there are none and - nothing else changes: 7.47k against 7.44k on the M4, 9.83k against 9.71k on - the Skylake host. + nothing else changes: 7.47k against 7.44k on ARM, 9.83k against 9.71k on + Intel. +* devel itself lost 2-11 exchanges a run in the tunnels and relays, every + one half-closed by the client with a large answer, cut short. Once the + server had answered and closed, `sockmap()` took the hang-up poll() reported + on its socket for the end of it while the rest of the answer was still + unread there, if its buffer was full at the time; fixed since. ## Earlier sessions -The tables below are from devel before the automatic mode, whose `-s` then +The session of 11-12 October, devel with 64 KB buffers, splice() by direction +(`-s1`, `-s3`) and the automatic mode as an option, against 0.9.4, 1.0.1 and +the same proxies, is in [data/](data/) (`*-auto-http.csv`, `*-tunnels.csv`). + +The tables below are older still: from devel before the automatic mode, whose `-s` then spliced both directions always, making a pair of pipes for each mapping (`sockmap()` call), and before squid, tinyproxy and dante were added. In the first table "devel" is `-s0` and "devel splice" `-s3` of today. The haproxy @@ -398,47 +546,17 @@ M4 and 7.5% or less on the Skylake host, except for one origin-bound case ### Throughput -| scenario (M4 / Intel / average) | no proxy | ‖ | 0.9.4 | 1.0.1 | devel | nginx | haproxy | ‖ | 0.9.4 splice | devel splice | haproxy splice-auto | haproxy splice | -|---|---:|:-:|---:|---:|---:|---:|---:|:-:|---:|---:|---:|---:| -| stream1, Gbit/s | 62.0 / 44.4 / 53.2 | ‖ | 8.3 / 5.5 / 6.9 | 22.1 / 19.1 / 20.6 | **22.0 / 19.9 / 21.0** | 13.5 / 9.8 / 11.6 | 13.6 / 9.7 / 11.7 | ‖ | 25.1 / 29.2 / 27.1 | **45.7 / 31.4 / 38.6** * | 29.5 / 30.6 / 30.1 | **44.7 / 32.3 / 38.5** * | -| stream16, Gbit/s | 141.2 / 150.5 / 145.9 | ‖ | 8.7 / 5.6 / 7.2 | **20.9 / 18.9 / 19.9** | **20.9 / 18.8 / 19.8** | 13.5 / 9.9 / 11.7 | 9.5 / 9.2 / 9.3 | ‖ | 24.5 / 28.3 / 26.4 | **37.8 / 31.2 / 34.5** * | 15.9 / 30.3 / 23.1 | 22.1 / 31.5 / 26.8 | -| stream1-down, Gbit/s | 61.8 / 43.1 / 52.5 | ‖ | 8.8 / 5.6 / 7.2 | **22.4 / 18.8 / 20.6** | **22.1 / 19.4 / 20.8** | 13.6 / 9.7 / 11.7 | 13.6 / 9.3 / 11.5 | ‖ | **47.9 / 34.4 / 41.1** * | 47.4 / 32.0 / 39.7 | 30.4 / 31.0 / 30.7 | **47.6 / 33.7 / 40.6** * | -| http-ka-1k, k req/s | 120.3 / 127.1 / 123.7 | ‖ | 14.1 / 19.2 / 16.7 | 6.0 / 6.1 / 6.1 | 15.8 / 18.1 / 17.0 | 37.0 / 36.5 / 36.8 | **44.3 / 37.3 / 40.8** * | ‖ | 13.4 / 14.6 / 14.0 | 14.9 / 13.8 / 14.4 | **44.1 / 37.2 / 40.7** * | **44.1 / 38.0 / 41.1** * | -| http-ka-1m, Gbit/s | 71.6 / 67.5 / 69.5 | ‖ | 8.3 / 5.8 / 7.1 | 12.5 / 12.4 / 12.4 | **21.9 / 18.3 / 20.1** | 12.6 / 5.8 / 9.2 | 6.0 / 7.6 / 6.8 | ‖ | **33.0 / 26.6 / 29.8** * | **35.3 / 23.9 / 29.6** * | 5.9 / 24.2 / 15.1 | 28.0 / 26.7 / 27.4 | -| http-ka-50k-2000, k req/s | 84.5 / 85.7 / 85.1 | ‖ | 7.6 / 7.0 / 7.3 | 9.0 / 10.3 / 9.6 | **10.4 / 10.5 / 10.5** * | 11.0 / 9.2 / 10.1 | 9.6 / 7.3 / 8.4 | ‖ | 9.1 / 9.4 / 9.3 | 9.9 / 9.0 / 9.4 | 9.6 / 7.7 / 8.6 | **11.9 / 9.0 / 10.4** * | -| http-close-1k, k req/s | 48.7 / 52.1 / 50.4 | ‖ | 5.0 / 4.8 / 4.9 | 4.4 / 4.0 / 4.2 | 5.2 / 4.8 / 5.0 | 13.3 / 12.9 / 13.1 | **16.6 / 15.1 / 15.9** * | ‖ | 4.8 / 4.4 / 4.6 | 5.0 / 4.4 / 4.7 | **16.7 / 15.1 / 15.9** * | **16.8 / 15.5 / 16.1** * | +| scenario (M4 / Intel / average) | no proxy | ‖ | 0.9.4 | 1.0.1 | devel | nginx | haproxy | ‖ | 0.9.4 splice | devel splice | haproxy splice-auto | +|---|---:|:-:|---:|---:|---:|---:|---:|:-:|---:|---:|---:| +| stream1, Gbit/s | 62.0 / 44.4 / 53.2 | ‖ | 8.3 / 5.5 / 6.9 | 22.1 / 19.1 / 20.6 | **22.0 / 19.9 / 21.0** | 13.5 / 9.8 / 11.6 | 13.6 / 9.7 / 11.7 | ‖ | 25.1 / 29.2 / 27.1 | **45.7 / 31.4 / 38.6** * | 29.5 / 30.6 / 30.1 | +| stream16, Gbit/s | 141.2 / 150.5 / 145.9 | ‖ | 8.7 / 5.6 / 7.2 | **20.9 / 18.9 / 19.9** | **20.9 / 18.8 / 19.8** | 13.5 / 9.9 / 11.7 | 9.5 / 9.2 / 9.3 | ‖ | 24.5 / 28.3 / 26.4 | **37.8 / 31.2 / 34.5** * | 15.9 / 30.3 / 23.1 | +| stream1-down, Gbit/s | 61.8 / 43.1 / 52.5 | ‖ | 8.8 / 5.6 / 7.2 | **22.4 / 18.8 / 20.6** | **22.1 / 19.4 / 20.8** | 13.6 / 9.7 / 11.7 | 13.6 / 9.3 / 11.5 | ‖ | **47.9 / 34.4 / 41.1** * | 47.4 / 32.0 / 39.7 | 30.4 / 31.0 / 30.7 | +| http-ka-1k, k req/s | 120.3 / 127.1 / 123.7 | ‖ | 14.1 / 19.2 / 16.7 | 6.0 / 6.1 / 6.1 | 15.8 / 18.1 / 17.0 | 37.0 / 36.5 / 36.8 | **44.3 / 37.3 / 40.8** * | ‖ | 13.4 / 14.6 / 14.0 | 14.9 / 13.8 / 14.4 | **44.1 / 37.2 / 40.7** * | +| http-ka-1m, Gbit/s | 71.6 / 67.5 / 69.5 | ‖ | 8.3 / 5.8 / 7.1 | 12.5 / 12.4 / 12.4 | **21.9 / 18.3 / 20.1** | 12.6 / 5.8 / 9.2 | 6.0 / 7.6 / 6.8 | ‖ | **33.0 / 26.6 / 29.8** * | **35.3 / 23.9 / 29.6** * | 5.9 / 24.2 / 15.1 | +| http-ka-50k-2000, k req/s | 84.5 / 85.7 / 85.1 | ‖ | 7.6 / 7.0 / 7.3 | 9.0 / 10.3 / 9.6 | **10.4 / 10.5 / 10.5** * | 11.0 / 9.2 / 10.1 | 9.6 / 7.3 / 8.4 | ‖ | 9.1 / 9.4 / 9.3 | 9.9 / 9.0 / 9.4 | 9.6 / 7.7 / 8.6 | +| http-close-1k, k req/s | 48.7 / 52.1 / 50.4 | ‖ | 5.0 / 4.8 / 4.9 | 4.4 / 4.0 / 4.2 | 5.2 / 4.8 / 5.0 | 13.3 / 12.9 / 13.1 | **16.6 / 15.1 / 15.9** * | ‖ | 4.8 / 4.4 / 4.6 | 5.0 / 4.4 / 4.7 | **16.7 / 15.1 / 15.9** * | -### Splice by direction - -devel splices what the server sends with `-s1` (downloads), what the client -sends with `-s2` (uploads), both with `-s3`; haproxy's matching settings are -`option splice-response`, `option splice-request` and both. The 2000 client -scenario here sends a 2 KB POST body with each request and gets 50 KB back -(`http-ka-2k-50k-2000`). Same setup and markings as above; measured in a -session of their own ([data/](data/) `*-splice-direction.csv`). - -| scenario (M4 / Intel / average) | no proxy | ‖ | devel -s0 | haproxy | ‖ | devel -s1 | devel -s3 | haproxy splice-response | haproxy splice | -|---|---:|:-:|---:|---:|:-:|---:|---:|---:|---:| -| stream1, Gbit/s | 61.6 / 45.5 / 53.6 | ‖ | **22.4 / 19.0 / 20.7** | 13.1 / 9.5 / 11.3 | ‖ | 22.4 / 19.5 / 21.0 | **44.1 / 31.9 / 38.0** * | 13.6 / 9.7 / 11.7 | **46.3 / 30.7 / 38.5** * | -| stream16, Gbit/s | 140.6 / 147.1 / 143.8 | ‖ | **21.1 / 19.2 / 20.2** | 9.0 / 8.9 / 8.9 | ‖ | 20.9 / 18.7 / 19.8 | **38.2 / 32.3 / 35.2** * | 9.2 / 9.1 / 9.2 | 23.2 / 33.3 / 28.3 | -| stream1-down, Gbit/s | 63.7 / 47.0 / 55.3 | ‖ | **22.3 / 19.5 / 20.9** | 13.8 / 9.5 / 11.7 | ‖ | **47.9 / 33.0 / 40.5** * | **46.3 / 30.7 / 38.5** * | **46.9 / 32.3 / 39.6** * | **47.4 / 32.6 / 40.0** * | -| http-ka-1k, k req/s | 120.0 / 127.6 / 123.8 | ‖ | 15.9 / 19.5 / 17.7 | **44.2 / 37.1 / 40.7** * | ‖ | 15.3 / 17.0 / 16.1 | 15.0 / 14.9 / 14.9 | **44.0 / 37.0 / 40.5** * | **44.1 / 35.0 / 39.5** * | -| http-ka-1m, Gbit/s | 71.0 / 67.2 / 69.1 | ‖ | **22.3 / 17.5 / 19.9** | 5.8 / 7.7 / 6.8 | ‖ | **35.6 / 24.7 / 30.2** * | **34.6 / 24.5 / 29.5** * | 26.4 / 27.7 / 27.1 | 26.5 / 25.9 / 26.2 | -| http-ka-2k-50k-2000, k req/s | 73.7 / 64.7 / 69.2 | ‖ | **9.5 / 9.7 / 9.6** | 8.8 / 7.2 / 8.0 | ‖ | 9.0 / 8.4 / 8.7 | 8.7 / 7.6 / 8.1 | **11.2 / 8.7 / 10.0** * | **11.2 / 9.0 / 10.1** * | -| http-close-1k, k req/s | 49.0 / 43.4 / 46.2 | ‖ | 5.1 / 4.9 / 5.0 | **16.5 / 15.9 / 16.2** * | ‖ | 5.0 / 4.7 / 4.9 | 5.0 / 4.2 / 4.6 | **16.7 / 15.0 / 15.8** * | **16.8 / 15.5 / 16.2** * | - - -* Each option splices the direction it names: with `-s1` an upload stream - goes at the copying speed and a download stream at the splicing one, as with - haproxy's `splice-response`. -* For bulk downloads `-s1` gets all of splice()'s gain; `-s3` is needed only - for bulk uploads (stream1, stream16). -* Small requests and 50 KB answers gain nothing from splicing in 3proxy, in - either direction: devel `-s0` stays ahead of `-s1` and `-s3`. haproxy gains - 25% with 2000 clients from splicing the answers; there it is ahead of devel by - 4-5%. - ### Grace delay with splice `-s1` without grace delay, with the setting suggested in the documentation diff --git a/bench/data/m4-tls.csv b/bench/data/m4-tls.csv index bfdc6bc..01d9431 100644 --- a/bench/data/m4-tls.csv +++ b/bench/data/m4-tls.csv @@ -1,91 +1,181 @@ proxy,scenario,rep,gbit,rate,p50_ms,p99_ms,errors,timeouts,retrans,prx_cpu_s,prx_task_s,cpu_s_per_gb,cpu_us_per_req,peak_mb,cli_util,prx_util,srv_util,cli_thread,srv_thread,bound -direct,tls-term-ka-1k,0,1.43923347456,139788.38,0.418,0.716,0,0,0,1.56,0.0,0.8671282471258088,1.104978637079683,0.0,0.8773911039410003,0.42048517520215634,0.9502362204724409,3.1036970032231808,0.9449767933049799,origin -3proxy-current,tls-term-ka-1k,0,0.16919822336,16434.21,3.67,7.83,0,0,0,10.11,10.102077999999999,47.80192037118089,60.91168709106026,11.7578125,0.44093526435177105,0.9610266159695817,0.4682822523164647,0.5500387716770072,0.18439661348100986,proxy -haproxy,tls-term-ka-1k,0,0.32690405376,31753.11,1.94,3.08,0,0,0,10.02,10.02144,24.520956249398573,31.49121263168481,36.2578125,0.4317390507604911,0.9524714828897338,0.484903695991671,0.7067207427148267,0.30119480289065775,proxy -nginx,tls-term-ka-1k,0,0.34091302912,33110.79,1.91,2.29,0,0,0,10.01,10.017096,23.489861976443315,30.177325704087355,32.421875,0.4595433789954338,0.9524262607040913,0.4998720900486058,0.7783796718107615,0.3079200937141662,proxy -stunnel,tls-term-ka-1k,0,0.14411628544,13997.51,4.39,8.19,0,0,0,10.11,10.105061999999998,56.12134655918036,71.5124421746573,13.41796875,0.41525275494140285,0.9601139601139601,0.44391971664698937,0.46239304043982005,0.15681054770137193,proxy -direct,tls-term-ka-1m,0,28.518582845439997,3396.15,4.92,7.99,0,0,0,6.39,0.0,0.17925154372870206,186.29194484125827,0.0,0.8392857142857143,0.8047858942065491,0.8910614525139665,3.256266527376346,0.9575980353996528,origin -3proxy-current,tls-term-ka-1m,0,5.95389841408,709.52,22.37,36.51,0,0,0,10.03,10.015631,1.3476884289836968,1410.689170182841,12.3046875,0.32870031699585467,0.9534220532319392,0.07265774378585087,1.0561731198566409,0.024725197211463632,client -haproxy,tls-term-ka-1m,0,4.7462744063999995,565.34,28.1,35.21,0,0,0,10.1,10.098108,1.7023878748149743,1768.8266199649738,42.609375,0.48249027237354086,0.9600760456273765,0.19498847926267282,0.8580728221764721,0.021861640453726128,proxy -nginx,tls-term-ka-1m,0,6.842252001279999,814.97,17.07,34.05,0,0,0,10.02,10.019807,1.1715441054349394,1227.0389419544454,42.1015625,0.3050129442221699,0.9524714828897338,0.11587591240875912,0.7839451741608926,0.03706187762990015,proxy -stunnel,tls-term-ka-1m,0,4.47960055808,533.79,29.8,41.51,0,0,0,10.04,10.025791,1.7930170103029432,1876.6355140186915,13.4453125,0.2036326426195958,0.952561669829222,0.0729903536977492,0.5599828036914579,0.018997196463425892,proxy -direct,tls-term-close-1k,0,0.12700352512000002,12382.0,0.558,1.43,0,0,0,5.89,0.0,37.10133238859188,47.09362756856161,0.0,0.9567078972407231,0.6770114942528735,0.9501734468621886,3.787302085470794,0.9500237647279252,origin -3proxy-current,tls-term-close-1k,0,0.02759852032,2689.77,5.42,9.96,0,0,0,10.1,10.101399,292.7693190183321,371.77457945301285,19.26953125,0.9548693586698337,0.9609895337773549,0.37130801687763715,3.78363078824788,0.04752205083144872,client -haproxy,tls-term-close-1k,0,0.02952790016,2881.39,14.06,16.23,0,0,0,10.1,10.085373,273.6395055597478,347.05518521063846,36.9921875,0.7649444918872759,0.9609895337773549,0.36055245941361763,2.831407964527035,0.056069766301595894,client -nginx,tls-term-close-1k,0,0.0339738624,3309.76,1.2,1.53,0,0,0,10.1,10.093163,237.829891251929,302.12384086150166,30.9453125,0.9542857142857143,0.9609895337773549,0.4076662908680947,3.779649775563822,0.054170596120676894,client -stunnel,tls-term-close-1k,0,0.02046820352,1998.12,17.55,26.16,0,0,0,10.01,9.925464,391.2409798043673,499.8252359314925,19.890625,0.8229471316085489,0.9533333333333334,0.27490247074122237,3.1921854543592736,0.03896556963013126,client -direct,tls-init-ka-1k,0,1.44560881664,140408.63,0.42,0.686,0,0,0,1.97,0.0,1.090198110207342,1.3891229557151832,0.0,0.888347260909935,0.49747474747474746,0.9535398230088495,3.1486030140417856,0.9549567226722561,origin -3proxy-current,tls-init-ka-1k,0,0.16592666624000002,16118.5,3.77,8.46,0,0,0,10.01,10.014918,48.262284667475036,62.037507592002676,12.22265625,0.4267361111111111,0.9524262607040913,0.45854241338112306,0.46053905836101444,0.18819565851956507,proxy -haproxy,tls-init-ka-1k,0,0.33218887680000003,32259.89,1.94,2.87,0,0,0,10.11,10.098718000000002,24.347594290068653,31.02911703593669,36.65234375,0.4495581737849779,0.9601139601139601,0.4657842157842158,0.7123230811607429,0.3001226109723907,proxy -nginx,tls-init-ka-1k,0,0.3485466624,33852.58,1.87,2.3,0,0,0,10.09,10.100526,23.159022509119286,29.51215731237551,33.13671875,0.43630632301131095,0.9600380589914367,0.458747648481591,0.6722958894310512,0.32298702524092604,proxy -stunnel,tls-init-ka-1k,0,0.14403239936,13992.44,4.38,8.53,0,0,0,10.12,10.105426999999999,56.20957531759609,71.60951585740366,14.359375,0.39872521246458925,0.9601518026565465,0.4342925659472422,0.38535774385386845,0.16155219887809816,proxy -direct,tls-init-ka-1m,0,31.86865733632,3802.24,4.21,6.77,0,0,0,7.48,0.0,0.18777069698446844,194.77644975652944,0.0,0.9059233449477352,0.85,0.9536131271694541,3.508252963448141,0.9571120725212797,origin -3proxy-current,tls-init-ka-1m,0,5.599983042560001,667.35,21.46,52.65,0,0,0,10.04,9.997523,1.434290057479927,1500.5230907188761,18.234375,0.17424826522744796,0.9534662867996201,0.16935985083903046,0.43199750231871964,0.17489734078750313,proxy -haproxy,tls-init-ka-1m,0,4.5562724351999995,542.44,29.34,37.3,0,0,0,10.1,10.096542,1.7733794708097441,1843.7385907265425,47.91015625,0.44719237786336913,0.9600760456273765,0.20470798569725865,0.6888261180458144,0.1910789643970402,proxy -nginx,tls-init-ka-1m,0,5.8494602444799995,697.07,22.4,35.93,0,0,0,10.03,10.018901999999999,1.371750497419324,1436.139747995418,46.23828125,0.17391304347826086,0.9525166191832859,0.1887599266951741,0.4518170555555568,0.16343421309998707,proxy -stunnel,tls-init-ka-1m,0,4.09624117248,488.1,29.94,80.2,0,0,0,10.04,10.018036,1.960822046797884,2052.33033524121,14.28125,0.1614650502966211,0.9516587677725118,0.12625,0.39455826463615057,0.17000018816441312,proxy -direct,tls-init-close-1k,0,0.1270874112,12387.5,0.557,1.42,0,0,0,5.89,0.0,37.07684306028259,47.074431949872526,0.0,0.9541676561386844,0.6754587155963303,0.949873896595208,3.789037266677241,0.948525093638368,origin -3proxy-current,tls-init-close-1k,0,0.02046820352,1991.68,31.58,46.36,0,0,0,10.05,10.046098,392.8043803230661,502.6005201040208,16.2265625,0.1962303664921466,0.9553231939163498,0.4414856341976174,0.11700812834357528,0.16819766016891818,proxy -haproxy,tls-init-close-1k,0,0.0255852544,2497.16,25.51,27.45,0,0,0,10.05,10.018104,314.24350425845284,401.37385678341786,37.69140625,0.21621621621621623,0.95260663507109,0.48175519630484986,0.14544643294249562,0.1908854172303607,proxy -nginx,tls-init-close-1k,0,0.03506438144,3415.05,18.64,19.96,0,0,0,10.04,10.017447,229.06435733776914,293.36138382421694,31.3046875,0.3150169363944298,0.952561669829222,0.5448717948717948,0.20914579107716558,0.2763155516595777,proxy -stunnel,tls-init-close-1k,0,0.020719861760000003,2020.17,30.65,50.59,0,0,0,10.04,9.952296,387.6473739562246,494.97140603431274,19.15234375,0.15156832638736564,0.9543726235741445,0.40702781844802344,0.11608036637748038,0.15210949476013474,proxy -3proxy-current,tls-term-ka-1k,1,0.16961765376,16473.58,3.66,7.74,0,0,0,10.1,10.101476,47.63655091841308,60.70477644414259,11.890625,0.44235458028561475,0.9600760456273765,0.46807500593401374,0.5555511284364395,0.18530655416507485,proxy -haproxy,tls-term-ka-1k,1,0.3326083072,32302.05,1.91,3.02,0,0,0,10.02,10.022596,24.100420303633353,30.953440094899168,36.2421875,0.4341311894756075,0.9515669515669516,0.4884744884744885,0.7159547740664182,0.3181358575020215,proxy -nginx,tls-term-ka-1k,1,0.34510733312,33517.26,1.88,2.35,0,0,0,10.03,10.020503,23.250737466102787,29.856166503941132,32.359375,0.43843107674901,0.9534220532319392,0.4723779023218575,0.7783193704900336,0.2880103536267642,proxy -stunnel,tls-term-ka-1k,1,0.14227079168,13818.69,4.46,8.3,0,0,0,10.02,10.021258999999999,56.34325855183151,72.3894291204901,13.26953125,0.4207980652962515,0.9533777354900095,0.4372623574144487,0.46534344959924023,0.188215713535384,proxy -direct,tls-term-ka-1k,1,1.4463637913599998,140480.79,0.42,0.664,0,0,0,1.72,0.0,0.9513512494018966,1.2122109106029693,0.0,0.8825017245343757,0.455026455026455,0.9526963103122044,3.143914124870164,0.9521459060053177,origin -3proxy-current,tls-term-ka-1m,1,6.05397450752,721.51,21.98,35.03,0,0,0,10.02,10.015604,1.3240888262814539,1386.0838290219947,12.7421875,0.3317904993909866,0.9524714828897338,0.07711757269279393,1.0398464274029606,0.025655603147065834,client -haproxy,tls-term-ka-1m,1,4.9979326464,594.95,26.83,29.54,0,0,0,10.1,10.101992000000001,1.6166684450659825,1680.8121151605924,41.5234375,0.4844629822732012,0.9600760456273765,0.17418213969938107,0.9117298840407279,0.02279813353131231,client -nginx,tls-term-ka-1m,1,6.531118530560001,778.22,20.01,33.74,0,0,0,10.02,10.022428000000001,1.2273548493251247,1284.944857655809,41.61328125,0.3185115843669553,0.9524714828897338,0.12767911818738517,0.7439290423295145,0.04085399320300185,proxy -stunnel,tls-term-ka-1m,1,4.425410150399999,527.41,30.23,43.92,0,0,0,10.01,10.021142,1.8095497881199059,1894.7567669884536,11.01171875,0.1969302809573361,0.9524262607040913,0.07452618053324767,0.547764871945811,0.020898337347345086,proxy -direct,tls-term-ka-1m,1,31.52505995264,3754.79,4.3,6.62,0,0,0,7.9,0.0,0.20047543159297765,208.30586684245222,0.0,0.913396795913629,0.8652792990142387,0.9504888047934406,3.516977670473943,0.9578710733957263,origin -3proxy-current,tls-term-close-1k,1,0.027682406399999998,2702.88,5.39,8.85,0,0,0,10.11,10.101994000000001,292.1711314808239,370.31610563715617,9.4375,0.9553231939163498,0.9601139601139601,0.3737727910238429,3.7866005305899466,0.049420228363693086,client -haproxy,tls-term-close-1k,1,0.029360128,2862.86,14.52,15.78,0,0,0,10.11,10.086215000000001,275.47563825334817,349.5729746550949,36.484375,0.795193764884174,0.9601139601139601,0.3691130767391824,2.992484396465593,0.054184394584872404,client -nginx,tls-term-close-1k,1,0.033218887679999996,3235.76,1.21,1.65,0,0,0,10.11,10.092725999999999,243.47594290068656,309.3445933541399,31.73828125,0.9547295687395759,0.9601139601139601,0.4058890147225368,3.781842488734503,0.055100514707128095,client -stunnel,tls-term-close-1k,1,0.020048773120000002,1952.73,17.8,26.94,0,0,0,10.11,10.006965,403.4162066471626,512.5735145001014,21.234375,0.8369369369369369,0.9601139601139601,0.27988565488565487,3.229551113263159,0.03894919669621939,client -direct,tls-term-close-1k,1,0.12675186688,12362.82,0.557,1.47,0,0,0,5.9,0.0,37.23811030308984,47.248382343519765,0.0,0.9557879724269075,0.6750572082379863,0.9468051621026126,3.7892235436396366,0.9464910700414535,origin -3proxy-current,tls-init-ka-1k,1,0.16995319808,16510.15,3.67,8.13,0,0,0,10.1,10.104447,47.54250047237475,60.57007838127965,12.40234375,0.42960787700811887,0.9609895337773549,0.46485207100591713,0.4603220748429241,0.20899354055461175,proxy -haproxy,tls-init-ka-1k,1,0.33436991488,32472.92,1.92,2.91,0,0,0,10.0,10.009786,23.925597501411186,30.76042855429062,36.75,0.45118153507968495,0.9514747859181731,0.4600202429149798,0.7115378225129483,0.2812704754783617,proxy -nginx,tls-init-ka-1k,1,0.35039215616,34030.39,1.86,2.41,0,0,0,10.02,10.009271,22.877224444315594,29.397095496552737,33.1796875,0.4374884579870729,0.9515669515669516,0.4701101206082853,0.6698367151793299,0.32405144493317556,proxy -stunnel,tls-init-ka-1k,1,0.14369685504,13957.13,4.41,8.5,0,0,0,10.11,10.105449,56.285156677566775,71.71942170452449,14.3671875,0.4037722668529514,0.9601139601139601,0.4407616361071932,0.3908161003110235,0.20042123513864932,proxy -direct,tls-init-ka-1k,1,1.4440988672000001,140257.87,0.41400000000000003,0.754,0,0,0,1.52,0.0,0.8420476101873366,1.072973491907521,0.0,0.8771566597653554,0.41192411924119243,0.9477494491658798,3.12287468894866,0.9481793060147322,origin -3proxy-current,tls-init-ka-1m,1,5.70232406016,679.46,21.12,53.48,0,0,0,10.03,9.987998,1.4071455629925838,1471.537558685446,20.1796875,0.19841072758877576,0.9534220532319392,0.1715170278637771,0.4375197590627391,0.17766417038041074,proxy -haproxy,tls-init-ka-1m,1,4.59922210816,547.35,29.03,37.71,0,0,0,10.1,10.098744,1.7568188293547202,1827.0622286541245,48.4453125,0.4712334038868578,0.9600760456273765,0.1909942580840133,0.6783993530642565,0.2138962635360921,proxy -nginx,tls-init-ka-1m,1,5.79761864704,690.77,21.14,42.38,0,0,0,10.03,10.02678,1.3840165227315682,1448.584633160023,43.7734375,0.20105820105820105,0.9525166191832859,0.19914477703115455,0.45648111788411194,0.16529398143366705,proxy -stunnel,tls-init-ka-1m,1,4.14749556736,494.21,30.11,68.85,0,0,0,10.02,10.019917000000001,1.9327326261863647,2023.016353725015,16.39453125,0.16917581015565195,0.9524714828897338,0.1315128688010044,0.4003186821500213,0.11491240453161419,proxy -direct,tls-init-ka-1m,1,32.21225472,3837.04,4.14,9.74,0,0,0,7.21,0.0,0.17906228701273605,186.0357105996491,0.0,0.9049836981835119,0.8403263403263403,0.9338790931989924,3.5067157321961746,0.9569534940338696,origin -3proxy-current,tls-init-close-1k,1,0.02046820352,1997.24,31.37,46.46,0,0,0,10.05,10.042808,392.8043803230661,501.4219428229307,18.21875,0.19344675488342786,0.9553231939163498,0.44232558139534883,0.11608032161165885,0.16726423174861682,proxy -haproxy,tls-init-close-1k,1,0.025417482239999997,2481.7,25.53,29.86,0,0,0,10.05,10.026736000000001,316.31771880801364,403.43623298944243,37.80078125,0.19936974789915968,0.9544159544159544,0.48161849710982657,0.14271647506102042,0.19102332527847646,proxy -nginx,tls-init-close-1k,1,0.03506438144,3414.92,18.63,20.03,0,0,0,10.03,10.018757,228.83620558743272,293.1034482758621,30.4296875,0.31871016122984624,0.9525166191832859,0.5444391634980988,0.20913259461550932,0.27349234155048363,proxy -stunnel,tls-init-close-1k,1,0.020635975679999998,2011.13,30.86,49.35,0,0,0,10.06,9.969092,389.9985212620681,497.9951487550121,21.671875,0.14970323147944603,0.9553656220322887,0.4062423799073397,0.1160795948302106,0.1511111707292933,proxy -direct,tls-init-close-1k,1,0.12473860095999999,12161.15,0.56,2.12,0,0,0,5.88,0.0,37.710860662197376,47.86909268530956,0.0,0.9543509272467903,0.6743119266055045,0.9372844151771715,3.7915930704274428,0.9349731514323012,origin -haproxy,tls-term-ka-1k,2,0.32757514239999996,31813.93,1.94,3.09,0,0,0,10.03,10.02569,24.495143133302655,31.447329642008366,36.0078125,0.4365556162388494,0.9525166191832859,0.48237137633846955,0.7193656085444711,0.2879642673226588,proxy -nginx,tls-term-ka-1k,2,0.33814478848,32841.24,1.93,2.33,0,0,0,10.02,10.020349,23.705821509279644,30.44325007519665,32.38671875,0.4483398256353181,0.9524714828897338,0.49432404540763675,0.766660326000989,0.32590565364239055,proxy -stunnel,tls-term-ka-1k,2,0.14101250048,13694.14,4.5,8.3,0,0,0,10.03,10.02326,56.90275665410284,73.10762054010715,13.234375,0.41935483870967744,0.9525166191832859,0.43829787234042555,0.4670283823043739,0.19859143792501524,proxy -direct,tls-term-ka-1k,2,1.45986945024,141790.49,0.41500000000000004,0.664,0,0,0,1.61,0.0,0.8822706713865786,1.1242844034519022,0.0,0.8869364161849711,0.44352617079889806,0.9548611111111112,3.1439647708458303,0.951380301378939,origin -3proxy-current,tls-term-ka-1k,2,0.16827547648,16344.39,3.71,7.95,0,0,0,10.01,10.019722,47.58863363522713,61.13761154102205,11.70703125,0.4419650579484518,0.9524262607040913,0.46368848542761587,0.53861259710042,0.20248401114127337,proxy -haproxy,tls-term-ka-1m,2,5.19036731392,618.34,25.8,30.58,0,0,0,10.11,10.101577,1.558271218745707,1618.8951160928743,43.00390625,0.4926410466067048,0.9601139601139601,0.18352601156069365,0.919041208804482,0.019958293511711238,client -nginx,tls-term-ka-1m,2,6.90868977664,823.11,20.01,32.92,0,0,0,10.01,10.014739,1.1591199285104739,1213.627546071775,42.90234375,0.31381733021077285,0.9515209125475285,0.11250766400980994,0.7740125280194877,0.03514294477338386,proxy -stunnel,tls-term-ka-1m,2,4.703492505600001,560.55,28.42,41.01,0,0,0,10.02,10.028072,1.7042654985537051,1782.918149466192,16.3125,0.21539243078486156,0.9524714828897338,0.06808781419026408,0.5789818824150479,0.021848661098155486,proxy -direct,tls-term-ka-1m,2,31.52505995264,3761.15,4.19,6.81,0,0,0,7.3,0.0,0.18524944944667554,192.17606486600326,0.0,0.9059266227657573,0.8439306358381503,0.9573324905183312,3.511586154017881,0.9589447840056432,origin -3proxy-current,tls-term-ka-1m,2,5.743931555840001,684.37,23.33,37.04,0,0,0,10.03,10.016382,1.3969525789076989,1462.7388070584805,15.43359375,0.3208477082306555,0.9534220532319392,0.07605905006418485,1.0152254629514315,0.025662381385124753,client -haproxy,tls-term-close-1k,2,0.02961178624,2883.24,13.9,16.13,0,0,0,10.1,10.084714,272.8643228244511,346.8168395027814,37.3125,0.828,0.9600760456273765,0.34004024144869216,3.178466778286004,0.057929972653973774,client -nginx,tls-term-close-1k,2,0.0339738624,3309.07,1.2,1.53,0,0,0,10.11,10.093905,238.0653663917824,302.48631182120096,30.9921875,0.9547619047619048,0.9601139601139601,0.4078590785907859,3.7809801968316314,0.05509837817054535,client -stunnel,tls-term-close-1k,2,0.02038431744,1989.2,17.45,25.85,0,0,0,10.1,10.005118000000001,396.38315208654836,502.71265740879,23.3359375,0.8371460928652321,0.9600760456273765,0.2772869022869023,3.254544592720068,0.038007282343487485,client -direct,tls-term-close-1k,2,0.12767461376,12452.45,0.554,1.39,0,0,0,5.88,0.0,36.84365953001806,46.749405694204825,0.0,0.9560361216730038,0.6758620689655173,0.9499370277078085,3.786575286683947,0.9457882376167059,origin -3proxy-current,tls-term-close-1k,2,0.02793406464,2721.22,5.42,9.08,0,0,0,10.1,10.101989999999999,289.2525704415353,367.4865376218891,14.08984375,0.9557248274220423,0.9609895337773549,0.3702576112412178,3.7892973863571653,0.04753876423631224,client -haproxy,tls-init-ka-1k,2,0.33420214272000004,32462.97,1.91,3.39,0,0,0,10.02,10.022186,23.98548356021743,30.798075888672024,36.73046875,0.4501100513573001,0.9533777354900095,0.46357448953869423,0.709869290340063,0.2813518223778306,proxy -nginx,tls-init-ka-1k,2,0.35072770048,34063.66,1.85,2.28,0,0,0,10.01,10.014167,22.832527881431623,29.338147798613107,33.171875,0.4212902012413015,0.9515209125475285,0.46594222104426186,0.6460985894525442,0.333599490097434,proxy -stunnel,tls-init-ka-1k,2,0.14361296896,13952.27,4.41,8.45,0,0,0,10.1,10.102834000000001,56.26232824592946,71.68051780302761,14.4609375,0.41027859491261465,0.9600760456273765,0.4431924882629108,0.39277794381787956,0.17478874644588616,proxy -direct,tls-init-ka-1k,2,1.44703488,140540.9,0.417,0.6930000000000001,0,0,0,1.6,0.0,0.884567481884058,1.1271479212574462,0.0,0.8832675794847993,0.43360433604336046,0.9536569987389659,3.128451216037277,0.9547420572619894,origin -3proxy-current,tls-init-ka-1k,2,0.16936599552,16447.73,3.68,8.2,0,0,0,10.12,10.104624,47.80180327900569,60.92018372371613,12.2734375,0.4296861453095197,0.9601518026565465,0.46746817538896746,0.4622866558988936,0.21663472696348368,proxy -haproxy,tls-init-ka-1m,2,4.57690841088,544.94,29.26,35.93,0,0,0,10.1,10.103178000000002,1.765383808160247,1835.0290697674418,48.20703125,0.4741463414634146,0.9609895337773549,0.207396449704142,0.6994235656888089,0.14259669133018518,proxy -nginx,tls-init-ka-1m,2,5.71062878208,680.35,19.63,40.59,0,0,0,10.02,10.019193000000001,1.4036983151757776,1470.2861335289801,42.8359375,0.21492760858711932,0.9524714828897338,0.20980511571254568,0.454465383622178,0.2051962373631917,proxy -stunnel,tls-init-ka-1m,2,4.1693898342399995,496.86,29.68,73.53,0,0,0,10.03,10.017989,1.9245022219090775,2014.4607350873669,14.3203125,0.1612071189063709,0.9525166191832859,0.12788521522145976,0.40020245897368467,0.12629547315655576,proxy -direct,tls-init-ka-1m,2,31.3532612608,3740.53,4.17,7.43,0,0,0,7.55,0.0,0.19264343666703732,199.8253182648281,0.0,0.9028264424199953,0.8521444695259593,0.950566750629723,3.4989447089055448,0.9585365268418932,origin -3proxy-current,tls-init-ka-1m,2,5.57389447168,664.14,21.7,53.53,0,0,0,10.04,9.994436,1.4410032412363043,1508.4134615384614,15.921875,0.18035714285714285,0.9534662867996201,0.1760519801980198,0.4338860462906276,0.18060928631957865,proxy -haproxy,tls-init-close-1k,2,0.0255852544,2494.53,25.49,27.15,0,0,0,10.04,10.024525,313.9308241547131,401.0866091403004,37.68359375,0.2095970087245534,0.9534662867996201,0.48092485549132946,0.14549104844711475,0.1947076871070426,proxy -nginx,tls-init-close-1k,2,0.03498049536,3409.86,18.67,19.87,0,0,0,10.02,10.016498,229.1562745897032,293.0595770816882,33.66796875,0.3164628410159925,0.9533777354900095,0.5445473984319316,0.20930352375326866,0.27551384990636907,proxy -stunnel,tls-init-close-1k,2,0.02080374784,2025.37,30.67,48.34,0,0,0,10.04,9.959099,386.08427970640116,493.92433708860136,19.828125,0.14982425307557118,0.9534662867996201,0.4061433447098976,0.11692732046212614,0.15287396711535206,proxy -direct,tls-init-close-1k,2,0.12826181632,12507.42,0.558,1.48,0,0,0,5.89,0.0,36.73735594266065,46.62060013139252,0.0,0.9543726235741445,0.6754587155963303,0.9480968858131488,3.78625065105561,0.9399542676846688,origin -3proxy-current,tls-init-close-1k,2,0.0205520896,2007.95,31.25,46.3,0,0,0,10.02,10.025754000000001,390.03333266900506,497.6903591119058,16.20703125,0.19590215346017145,0.9542857142857143,0.44217371110078957,0.11512995354814924,0.1625629104397623,proxy +direct,tls-term-ka-1k,0,1.3920894976,135203.73,0.433,1.07,0,0,0,1.77,0.0,1.0171759807406222,1.296234183379788,0.0,0.8580835603996367,0.43596059113300495,0.9349517584811703,3.0248309276281193,0.9358171938778904,origin +3proxy-devel,tls-term-ka-1k,0,0.17507024896,17001.38,3.53,7.88,0,0,0,10.09,10.104890000000001,46.10720580996196,58.75947052417641,12.08984375,0.429058908045977,0.960952380952381,0.4549483013293944,0.5766794859348302,0.22720038990652688,proxy +haproxy,tls-term-ka-1k,0,0.32933675007999996,31988.77,1.93,3.04,0,0,0,10.02,10.023767,24.339828452344946,31.246978990928397,36.26171875,0.408785727987723,0.9524714828897338,0.46326140398798143,0.7161813017483875,0.286939172841326,proxy +nginx,tls-term-ka-1k,0,0.34351349760000005,33361.89,1.89,2.33,0,0,0,10.02,10.013692,23.3353275955815,29.973974692632147,32.47265625,0.4407484407484408,0.9515669515669516,0.4747123360984747,0.7827712011784499,0.33442126498782127,proxy +envoy,tls-term-ka-1k,0,0.31801212927999994,30886.15,1.98,3.06,0,0,0,10.09,10.084401,25.382679642677562,32.34295715279403,17.52734375,0.517204682511529,0.9591254752851711,0.5630252100840336,0.9037871159143464,0.3431329766735917,client +stunnel,tls-term-ka-1k,0,0.14503903231999998,14084.55,4.35,8.21,0,0,0,10.11,10.107971000000001,55.76429924156847,71.07255587034004,13.49609375,0.4050044682752458,0.9610266159695817,0.41683217477656404,0.47576815277289547,0.18724993216172428,proxy +direct,tls-term-ka-1m,0,29.377576304639998,3499.4,4.82,7.73,0,0,0,6.43,0.0,0.17509953668940134,181.91591693543825,0.0,0.8574433278803458,0.8027465667915106,0.8941139831828091,3.2694043456181623,0.9565431983124729,origin +3proxy-devel,tls-term-ka-1m,0,7.03368003584,838.24,18.66,31.09,0,0,0,10.02,10.006776,1.1396594612144149,1193.7097927090779,14.80859375,0.26465495918872123,0.9524714828897338,0.07633829585049097,0.7677964352440629,0.029453518646857354,proxy +haproxy,tls-term-ka-1m,0,4.8792338432,581.2,27.54,30.64,0,0,0,10.11,10.104375000000001,1.6576372971490048,1722.3168654173764,41.4765625,0.510533569600315,0.9601139601139601,0.14507299270072993,0.8962550719293447,0.02185952816643344,proxy +nginx,tls-term-ka-1m,0,7.27619469312,866.88,15.74,37.69,0,0,0,10.02,10.017831000000001,1.1016747541925345,1154.3778801843318,43.4296875,0.2948284195263412,0.9524714828897338,0.08377618005626758,0.7801570989336393,0.033263906937601666,proxy +envoy,tls-term-ka-1m,0,10.565619548159999,1261.88,12.68,16.97,0,0,0,10.01,10.00575,0.7579299977155235,792.3064745923698,41.40625,0.3592592592592593,0.9515209125475285,0.11862864077669903,1.0442960505988186,0.05415495596845124,client +stunnel,tls-term-ka-1m,0,4.61230833664,549.65,29.07,41.36,0,0,0,10.02,9.988987,1.737958396302607,1819.8329095532147,12.859375,0.2089171974522293,0.9515669515669516,0.06509829197550758,0.5717750085376905,0.027555417032560613,proxy +direct,tls-term-close-1k,0,0.12499025920000001,12190.24,0.555,1.43,0,0,0,5.65,0.0,36.16281803822358,45.8879521790686,0.0,0.9543400713436385,0.6608187134502924,0.9436220472440945,3.78107836386332,0.9352217747404032,origin +3proxy-devel,tls-term-close-1k,0,0.02776629248,2708.48,5.25,8.62,0,0,0,10.12,10.082897,291.57655836952415,369.9100811462826,12.88671875,0.9533222195760895,0.9601518026565465,0.3292407917815084,3.7798582845450444,0.04846762861987139,client +haproxy,tls-term-close-1k,0,0.028940697600000002,2825.09,14.24,18.98,0,0,0,10.03,9.995154,277.25662010303444,354.2166972736262,37.11328125,0.7877919788453063,0.9525166191832859,0.31592621460119513,2.988125024992129,0.0569891153108287,client +nginx,tls-term-close-1k,0,0.03338665984,3255.28,1.21,1.61,0,0,0,10.09,10.08296,241.77321237535332,306.8828127376137,31.3203125,0.9545021438780371,0.9600380589914367,0.37815326035221325,3.7767875149066468,0.055129953873606335,client +envoy,tls-term-close-1k,0,0.04018143232,3916.26,12.03,14.65,0,0,0,10.09,10.030517000000001,200.8888069423604,255.10075089120926,18.6875,0.904420273084934,0.9582146248812915,0.33026315789473687,3.54912460888683,0.07412408106946165,client +stunnel,tls-term-close-1k,0,0.020048773120000002,1957.62,17.56,26.03,0,0,0,10.11,10.001994,403.4162066471626,511.2257281553398,23.7890625,0.8398446424491661,0.9610266159695817,0.23870791130577607,3.2471492042191374,0.03801439829195942,client +direct,tls-term-stream1-down,0,9.293834923854945,nan,nan,nan,0,0,0,0.09,0.0,0.00774636463727282,nan,0.0,0.474320987654321,0.010101010101010102,0.36843800322061193,1.0746396247321204,0.8176578992239385,client +3proxy-devel,tls-term-stream1-down,0,9.163430408938526,nan,nan,nan,0,0,0,9.76,9.108672,0.8519879017494479,nan,6.35546875,0.5072658772874058,0.9207547169811321,0.04691358024691358,1.0566006975425148,0.09849937659157491,client +haproxy,tls-term-stream1-down,0,8.77996528514481,nan,nan,nan,0,0,0,9.52,8.860667,0.8673907989096684,nan,41.08984375,0.46992970580577975,0.8592057761732852,0.06592,1.0556821915231327,0.0910933624177118,client +nginx,tls-term-stream1-down,0,7.51396851840779,nan,nan,nan,0,0,0,9.96,9.718259,1.0603034804198586,nan,33.1484375,0.4508174009420892,0.9205175600739371,0.05337262606417813,1.0368524567142539,0.08161489142052755,client +envoy,tls-term-stream1-down,0,9.727648676631553,nan,nan,nan,0,0,0,9.31,6.887653,0.7656286105433424,nan,24.40625,0.48361270953214913,0.7550689375506894,0.06654170571696345,1.0675906263818422,0.0995115775407132,client +stunnel,tls-term-stream1-down,0,4.716006741302276,nan,nan,nan,0,0,0,10.03,10.012193,1.7012633870261782,nan,4.4375,0.3495561677379859,0.9400187441424555,0.019162065605716143,0.8897344251098557,0.05254501244348075,proxy +direct,tls-term-stream16,0,18.925880502955465,nan,nan,nan,0,0,705,0.24,0.0,0.0101050467848037,nan,0.0,0.5121503897294819,0.030303030303030304,0.9310453400503779,1.5156926322014015,2.758691947536444,origin +3proxy-devel,tls-term-stream16,0,6.744539040269984,nan,nan,nan,0,0,406,10.08,10.028887,1.1925920272156314,nan,32.1328125,0.2334340850964892,0.9438202247191011,0.20575815738963532,0.6048301032185484,0.4510193750789299,proxy +haproxy,tls-term-stream16,0,5.0508450944870456,nan,nan,nan,0,0,730,10.02,10.022953,1.5859289505450904,nan,57.16796875,0.21203703703703702,0.9408450704225352,0.5180420458111076,0.4983960403619521,0.8873812098975168,proxy +nginx,tls-term-stream16,0,6.6625112543267475,nan,nan,nan,0,0,464,10.07,10.051921,1.2043516887648529,nan,61.03515625,0.208363201911589,0.943767572633552,0.2315270935960591,0.5925187446976511,0.45838061659597235,proxy +envoy,tls-term-stream16,0,9.443206579037609,nan,nan,nan,0,0,907,10.04,10.055330000000001,0.8478981746042726,nan,65.64453125,0.27735849056603773,0.9444967074317968,0.1190176322418136,0.7685936825864172,0.23934200455094362,proxy +stunnel,tls-term-stream16,0,4.246478882892315,nan,nan,nan,0,0,471,10.05,10.016542,1.8893481270251706,nan,17.46875,0.1544256120527307,0.9418931583880038,0.18260217472815898,0.3906179274582188,0.3914421897839879,proxy +direct,tls-init-ka-1k,0,1.42891548672,138786.21,0.422,0.729,0,0,0,1.57,0.0,0.8789883038380959,1.1200517078648318,0.0,0.8772413793103448,0.42091152815013405,0.9508816120906801,3.1069088993629594,0.9498230194007785,origin +3proxy-devel,tls-init-ka-1k,0,0.17557356544,17054.78,3.53,8.48,0,0,0,10.12,10.103021,46.11172518887362,58.75386080211793,12.5625,0.4164285714285714,0.9601518026565465,0.45169261181121817,0.4748139450569951,0.21860738817753814,proxy +haproxy,tls-init-ka-1k,0,0.33529266176,32566.71,1.92,2.86,0,0,0,10.11,10.100397000000001,24.12221000467147,30.73845255758519,36.84765625,0.42901115813774526,0.9601139601139601,0.41986827661909987,0.7136189922365787,0.270835638298258,proxy +nginx,tls-init-ka-1k,0,0.34452013056,33463.74,1.89,2.33,0,0,0,10.03,10.015092000000001,23.290366188348397,29.925916201467356,33.1953125,0.42545523520485584,0.9525166191832859,0.455810147299509,0.6670143148430631,0.29940655350770856,proxy +envoy,tls-init-ka-1k,0,0.32405192704,31476.65,1.95,3.01,0,0,0,10.1,10.086791999999999,24.934275422477672,31.770197667249644,17.42578125,0.5011361650061178,0.9591642924976258,0.5558217537134643,0.7600478003035748,0.2946446738179909,proxy +stunnel,tls-init-ka-1k,0,0.14612955136000003,14195.24,4.34,8.37,0,0,0,10.02,10.013202,54.855434273195314,70.50528789659225,14.453125,0.3845597541131803,0.9524714828897338,0.40935233808452115,0.39571583490682516,0.16057848338674605,proxy +direct,tls-init-ka-1m,0,27.74548873216,3302.69,5.22,7.36,0,0,0,6.48,0.0,0.18684118524793503,194.2737220806476,0.0,0.8270607250057723,0.8039702233250621,0.8671732522796353,3.0923785505514267,0.953336216453813,origin +3proxy-devel,tls-init-ka-1m,0,5.84056832,696.0,20.27,61.66,0,0,0,10.02,9.989832,1.3724691777939857,1438.208698148414,20.171875,0.19264668849156383,0.9524714828897338,0.18023617153511498,0.4434411792478279,0.27271998232683625,proxy +haproxy,tls-init-ka-1m,0,4.6313504768,551.57,28.9,36.95,0,0,0,10.11,10.096093,1.7463588731873187,1814.7549811523963,49.21875,0.4540692989524577,0.9610266159695817,0.17063369397217928,0.7050437766371025,0.1491894355858182,proxy +nginx,tls-init-ka-1m,0,5.8539062067200005,697.54,21.17,43.06,0,0,0,10.02,10.016183,1.3693420627064403,1434.2971657600915,43.91015625,0.20135236664162284,0.9515669515669516,0.18321327904791732,0.4564925550297905,0.1852465234565511,proxy +envoy,tls-init-ka-1m,0,8.41813590016,1002.47,15.76,23.84,0,0,0,10.01,10.0073,0.9512794869286675,996.4164841728051,40.79296875,0.1663172606568833,0.9515209125475285,0.2652689152233364,0.39577915164922567,0.30409122909425207,proxy +stunnel,tls-init-ka-1m,0,4.1721580748800005,497.09,29.48,77.56,0,0,0,10.03,10.015576,1.923225308338967,2013.6518771331057,17.89453125,0.15694372741352608,0.9516129032258065,0.12350346565847511,0.3994475964951075,0.17938605756051504,proxy +direct,tls-init-close-1k,0,0.12532580352,12218.5,0.5630000000000001,1.98,0,0,0,5.99,0.0,38.236339727399184,48.53621579574275,0.0,0.9517126546146527,0.6737907761529809,0.9413241292751804,3.7872712953261907,0.9411646148728162,origin +3proxy-devel,tls-init-close-1k,0,0.020635975679999998,2009.76,31.18,46.26,0,0,0,10.03,10.025792,388.83550380303603,498.1870560770874,15.3203125,0.1654315605928509,0.9534220532319392,0.41494082840236685,0.11895511321300963,0.1682994717210808,proxy +haproxy,tls-init-close-1k,0,0.02499805184,2436.74,26.03,28.43,0,0,0,10.02,10.015318,320.66498826814177,410.18503356803666,37.8203125,0.17112998040496408,0.9524714828897338,0.4559748427672956,0.1417205035275168,0.19192313229100294,proxy +nginx,tls-init-close-1k,0,0.03447717888,3361.32,18.79,23.41,0,0,0,10.01,10.003511000000001,232.2695841174346,297.3326204479297,33.03515625,0.26715092816787733,0.9533333333333334,0.5357056402808037,0.210250244059997,0.27654611918467026,proxy +envoy,tls-init-close-1k,0,0.05804916736,5657.94,10.9,16.77,0,0,0,9.99,9.91711,137.67639336558437,176.1780474040632,19.2109375,0.24180934263369266,0.9496197718631179,0.5218150087260035,0.27486978755602454,0.23085421138047788,proxy +stunnel,tls-init-close-1k,0,0.01920991232,1872.01,33.09,54.5,0,0,0,10.03,9.928395,417.7010215526064,534.7051924512208,19.0,0.134568176746262,0.9534220532319392,0.37035123966942146,0.11227889987212178,0.15206495973371348,proxy +direct,tls-init-stream1-down,0,9.2074792391595,nan,nan,nan,0,0,0,0.37,0.0,0.03214644729780898,nan,0.0,0.4862340216322517,0.04057017543859649,0.3595758354755784,1.078637185575463,0.8151964992838053,client +3proxy-devel,tls-init-stream1-down,0,7.308803248275543,nan,nan,nan,0,0,0,10.0,10.002157,1.0944476447066418,nan,4.70703125,0.1378241262683202,0.9514747859181731,0.33733417561591916,0.47579304380184495,0.6988019660576581,proxy +haproxy,tls-init-stream1-down,0,5.821160810375183,nan,nan,nan,0,0,0,10.0,9.995925999999999,1.374145734271717,nan,40.66796875,0.10997150997150996,0.9505703422053232,0.2651635439822166,0.43946900720116777,0.5454691540590876,proxy +nginx,tls-init-stream1-down,0,7.229744269623417,nan,nan,nan,0,0,0,9.99,10.000118,1.1054046608729153,nan,36.828125,0.12457240592930445,0.9496197718631179,0.2848205465920316,0.46892265628063223,0.6496471681828438,proxy +envoy,tls-init-stream1-down,0,10.853111653192572,nan,nan,nan,0,0,1,9.98,9.976779,0.73560132376774,nan,24.9453125,0.06754082137555666,0.9486692015209125,0.3833116036505867,0.22926516893050844,0.8208024804285621,proxy +stunnel,tls-init-stream1-down,0,4.426390368581383,nan,nan,nan,0,0,1,10.0,10.001557,1.8071425769221658,nan,5.9140625,0.1052335396736072,0.9505703422053232,0.21188055908513342,0.4004110010905507,0.4114237983316316,proxy +direct,tls-init-stream16,0,18.283828112558886,nan,nan,nan,0,0,743,0.39,0.0,0.017015777126283136,nan,0.0,0.508459076360311,0.04797047970479705,0.9268830759533564,1.4860198345701823,1.953350246579136,origin +3proxy-devel,tls-init-stream16,0,7.85990244478347,nan,nan,nan,0,0,1032,10.07,10.039336,1.0220159223349807,nan,44.359375,0.07603578154425612,0.956315289648623,0.6354202698028364,0.09130866194988724,1.0447423890358685,origin +haproxy,tls-init-stream16,0,5.86128438345321,nan,nan,nan,0,0,9,10.03,10.025364999999999,1.368238219647359,nan,65.19921875,0.03711790393013101,0.9516129032258065,0.855525965379494,0.07035385453858682,1.2534061634240694,origin +nginx,tls-init-stream16,0,7.792840054565132,nan,nan,nan,0,0,454,10.07,10.060383999999999,1.0290383462320856,nan,72.234375,0.06873094065212292,0.956315289648623,0.6355074991280084,0.08370469003431631,1.0536350634348823,origin +envoy,tls-init-stream16,0,10.864147201801515,nan,nan,nan,0,0,8190,10.04,10.04179,0.7371821331730954,nan,125.82421875,0.07728337236533958,0.9543726235741445,0.6989898989898989,0.12366100208998754,1.3354274559546866,origin +stunnel,tls-init-stream16,0,5.254997688734296,nan,nan,nan,0,0,906,10.04,10.002109,1.5273697651841438,nan,56.0078125,0.037926068170907344,0.9534662867996201,0.566410537870472,0.05707554613487713,0.9207344244023087,origin +3proxy-devel,tls-term-ka-1k,1,0.17272143872,16779.08,3.59,7.93,0,0,0,10.1,10.10052,46.78052741963635,59.596865558912384,12.01953125,0.4264917325664989,0.9600760456273765,0.4481132075471698,0.563270684334641,0.24708187130673714,proxy +haproxy,tls-term-ka-1k,1,0.32480690176,31546.14,1.96,3.14,0,0,0,10.03,10.017436,24.703908557734213,31.734982835266013,36.23046875,0.39679165056049476,0.9525166191832859,0.45339565576024193,0.6954535418365821,0.29740529997501947,proxy +nginx,tls-term-ka-1k,1,0.33688649728,32718.93,1.94,2.37,0,0,0,10.01,10.008228,23.770617298870924,30.5605930159428,32.4609375,0.4192982456140351,0.9524262607040913,0.4618760285244103,0.7640349644007383,0.32229591570628036,proxy +envoy,tls-term-ka-1k,1,0.31784435712000003,30874.68,1.98,3.01,0,0,0,10.08,10.06195,25.370908179928744,32.3244761703192,17.54296875,0.5158506516379007,0.9572649572649573,0.5584072919165267,0.8959950160342254,0.3648243385606514,proxy +stunnel,tls-term-ka-1k,1,0.14235467776,13824.27,4.42,8.49,0,0,0,10.1,10.14796,56.75963815971199,72.33868830620035,13.578125,0.3925763393673432,0.9609895337773549,0.4120603015075377,0.4653459096080538,0.17875867582282284,proxy +direct,tls-term-ka-1k,1,1.43294201856,139176.18,0.369,0.849,0,0,0,1.64,0.0,0.9155988051201558,1.166678523155723,0.0,0.8827331486611265,0.4444444444444444,0.9510886715052067,3.1294172907604505,0.9562392105780603,origin +3proxy-devel,tls-term-ka-1m,1,7.3014444032,871.73,17.94,30.29,0,0,0,10.02,10.009905,1.09786496442907,1147.3720371006527,18.25,0.2613806903451726,0.9524714828897338,0.0740043900909376,0.7760446635763213,0.04177642800938439,proxy +haproxy,tls-term-ka-1m,1,5.08290924544,605.36,26.23,32.27,0,0,0,10.1,10.101548,1.5896408158868394,1651.9463526333006,43.8203125,0.48774008990600737,0.9591642924976258,0.1387618176273254,0.912195062191311,0.03324629169014147,client +nginx,tls-term-ka-1m,1,6.911458017279999,823.51,16.6,35.58,0,0,0,10.01,10.017724,1.1586556671513346,1214.0691328077623,43.640625,0.30174682938502034,0.9515209125475285,0.09606986899563319,0.7579814584045789,0.0370401569429093,proxy +envoy,tls-term-ka-1m,1,10.39382085632,1242.31,12.78,18.35,0,0,0,10.02,10.016006999999998,0.7712274543509996,805.4014950566675,39.97265625,0.3582776882339397,0.9515669515669516,0.12507534659433395,1.055930244987573,0.053193587238450246,client +stunnel,tls-term-ka-1m,1,4.65542578176,554.79,28.75,41.13,0,0,0,10.02,10.020218999999999,1.7218618394490917,1803.7803780378038,13.1640625,0.2159376571141277,0.9524714828897338,0.07064868336544637,0.5765733690493384,0.0209101453966468,proxy +direct,tls-term-ka-1m,1,29.807073034240002,3549.46,3.91,9.65,0,0,0,6.84,0.0,0.1835805881950972,190.80562374469986,0.0,0.8698197143526106,0.8191616766467066,0.9052369077306733,3.311668926885409,0.9535747825977475,origin +3proxy-devel,tls-term-close-1k,1,0.02743074816,2670.97,5.36,9.12,0,0,0,10.11,10.107229,294.85160057697817,374.74979613018013,14.109375,0.9550107117353012,0.9610266159695817,0.3282308657465496,3.7871930248964962,0.048481171862288494,client +haproxy,tls-term-close-1k,1,0.0285212672,2780.7,14.76,18.6,0,0,0,10.01,10.001138000000001,280.7729384478401,359.2964824120603,37.1015625,0.7736015918638072,0.9515209125475285,0.31197916666666664,2.93198166282422,0.05603384040518167,client +nginx,tls-term-close-1k,1,0.03179282432,3101.91,1.26,3.0,0,0,0,10.08,10.071446,253.6421400890501,321.7466245331801,32.98046875,0.9511089911757692,0.9590865842055185,0.3812380038387716,3.7716297701023414,0.06177184221568405,client +envoy,tls-term-close-1k,1,0.039845888,3888.89,11.21,15.71,0,0,0,10.05,10.016436,201.7774079975329,255.86842507255972,19.078125,0.8643533123028391,0.9571428571428572,0.2913797851831451,3.3446102026163405,0.07413301508017986,client +stunnel,tls-term-close-1k,1,0.019881000960000002,1935.27,17.91,26.69,0,0,0,10.1,9.990365,406.4181685950685,516.7033304343378,20.77734375,0.8298548094373865,0.9600760456273765,0.23940967477452857,3.2103454770571536,0.03895485929898251,client +direct,tls-term-close-1k,1,0.12717129728,12396.83,0.5670000000000001,1.5,0,0,0,5.99,0.0,37.681458807872275,47.8384832245853,0.0,0.9562737642585551,0.6814562002275313,0.9517806492278601,3.7895813705102115,0.9453739873830603,origin +3proxy-devel,tls-term-stream1-down,1,9.269733180405584,nan,nan,nan,0,0,0,9.53,8.612098,0.8223599190491607,nan,10.3671875,0.4755386565272497,0.8433628318584071,0.06378927324658838,1.0546488611938987,0.09086527097403839,client +haproxy,tls-term-stream1-down,1,7.519094674708266,nan,nan,nan,0,0,1,9.88,9.61274,1.0510683525119917,nan,40.09375,0.45240793201133145,0.9114391143911439,0.04571611253196931,1.0427517178910461,0.07037176596326697,client +nginx,tls-term-stream1-down,1,9.104725019391479,nan,nan,nan,0,0,0,9.56,8.81892,0.8399591371899257,nan,39.859375,0.48488664987405544,0.855863921217547,0.13503222026947861,1.0526915771243328,0.08811322576773059,client +envoy,tls-term-stream1-down,1,9.42879598589367,nan,nan,nan,0,0,0,9.41,7.081422,0.7983786410491476,nan,27.40625,0.4868080703569581,0.7732128184059162,0.050588609608654155,1.0863265860845686,0.09468485594659978,client +stunnel,tls-term-stream1-down,1,4.524189666969606,nan,nan,nan,0,0,0,10.02,10.010884,1.7716461924246814,nan,4.6015625,0.3325115562403698,0.9390815370196813,0.016787360105332456,0.8789332503100277,0.04504085122005321,proxy +direct,tls-term-stream1-down,1,9.10875118930647,nan,nan,nan,0,0,2,0.26,0.0,0.02283434722893716,nan,0.0,0.4829965500246427,0.02905027932960894,0.3766111285759195,1.0793943246836857,0.7859091664243235,client +3proxy-devel,tls-term-stream16,1,6.697351563038402,nan,nan,nan,0,0,445,10.06,10.021135,1.1984589641856127,nan,26.359375,0.2403443462075384,0.9428303655107779,0.20544269835185894,0.6180199618605883,0.4509232551261653,proxy +haproxy,tls-term-stream16,1,5.004658686563269,nan,nan,nan,0,0,617,10.02,10.026729,1.6004047397392551,nan,53.2578125,0.21793390339727295,0.9408450704225352,0.5203488372093024,0.4983626788859683,0.8901862338824181,proxy +nginx,tls-term-stream16,1,6.682945302994012,nan,nan,nan,0,0,603,10.05,10.038772000000002,1.2006610456939681,nan,46.66015625,0.22403595930920275,0.9410112359550562,0.2548375319459657,0.60745653551399,0.46860140105462306,proxy +envoy,tls-term-stream16,1,9.480483179061055,nan,nan,nan,0,0,726,10.06,10.056433,0.8463177340622898,nan,76.984375,0.27601597369039227,0.9428303655107779,0.11945928953159383,0.7571363232770778,0.23529033093508328,proxy +stunnel,tls-term-stream16,1,4.269192969026591,nan,nan,nan,0,0,461,10.06,10.027892999999999,1.8789587980898699,nan,21.15625,0.16828478964401294,0.9419475655430711,0.18892630003741115,0.39458049083242813,0.3935539468825128,proxy +direct,tls-term-stream16,1,19.743686764960216,nan,nan,nan,0,0,758,0.55,0.0,0.02222746217714254,nan,0.0,0.528691313222953,0.06798516687268233,0.9303058971933144,1.5665236665604825,16.89637765654046,origin +3proxy-devel,tls-init-ka-1k,1,0.17758683136,17250.12,3.5,8.1,0,0,0,10.12,10.107277,45.58896590472956,58.08814244222756,12.6953125,0.41420959147424513,0.9601518026565465,0.4562424242424242,0.4747393192948892,0.2108943989320829,proxy +haproxy,tls-init-ka-1k,1,0.333447168,32387.18,1.92,3.04,0,0,0,10.02,10.020069,24.039790315448112,30.88979249581508,36.734375,0.4266792809839167,0.9506641366223909,0.429831595829992,0.6989222614929111,0.2706432240165258,proxy +nginx,tls-init-ka-1k,1,0.34745614336,33748.56,1.86,2.59,0,0,0,10.1,10.066616,23.254733451721687,29.62989039874205,33.21484375,0.42716795767995464,0.9582542694497154,0.4670078103959063,0.6706931422430661,0.2956026697567241,proxy +envoy,tls-init-ka-1k,1,0.31927042048000004,31007.9,1.97,3.15,0,0,0,10.08,10.0391,25.257585678862323,32.18699109110068,17.46484375,0.49156455336529925,0.9590865842055185,0.5488519785051295,0.7573484689334815,0.30988612174344554,proxy +stunnel,tls-init-ka-1k,1,0.14436794368,14025.62,4.38,9.04,0,0,0,10.03,10.001055,55.58020565691276,71.44281725454442,14.42578125,0.36625744047619047,0.9525166191832859,0.3876331514679137,0.39864048976106936,0.21190585517352017,proxy +direct,tls-init-ka-1k,1,1.4241339801600001,138315.47,0.41100000000000003,0.842,0,0,0,1.86,0.0,1.0448455136452994,1.331388265745098,0.0,0.8664998862861042,0.4603960396039604,0.945716975211798,3.0800727871449856,0.9478631858416178,origin +3proxy-devel,tls-init-ka-1m,1,6.14868189184,732.73,19.62,47.55,0,0,0,10.04,9.971608,1.3062962341017148,1368.22022349414,15.73046875,0.19650872817955112,0.9534662867996201,0.17011278195488722,0.4538056345323639,0.15672117794804047,proxy +haproxy,tls-init-ka-1m,1,4.69351006208,558.9,28.52,31.7,0,0,0,10.09,10.10157,1.7198216032848497,1787.422497785651,48.83203125,0.46153846153846156,0.9600380589914367,0.17016029593094945,0.7010050630755602,0.1339659252668233,proxy +nginx,tls-init-ka-1m,1,5.8355351552,695.41,21.47,44.05,0,0,0,10.02,10.013459,1.3736529361590777,1439.0348987505386,43.09765625,0.19281376518218624,0.9515669515669516,0.18042526579111945,0.4557887752731992,0.1654046191203913,proxy +envoy,tls-init-ka-1m,1,8.76173328384,1046.7,15.28,20.39,0,0,0,10.0,10.005844999999999,0.9130613476622339,953.6524890329964,42.25,0.17040981708728872,0.9523809523809523,0.26814454904342544,0.40727643382389644,0.31557469261748955,proxy +stunnel,tls-init-ka-1m,1,4.25059155968,506.53,29.42,74.82,0,0,0,10.03,10.009055,1.8877372448845862,1977.5236593059938,16.53125,0.1719969780911609,0.9525166191832859,0.12606905289832118,0.4033243735928517,0.1425615490538313,proxy +direct,tls-init-ka-1m,1,28.69038153728,3414.19,5.04,8.61,0,0,0,6.88,0.0,0.19184129680702072,199.5186033697764,0.0,0.865546218487395,0.8309178743961353,0.9150650960942344,3.277400414027292,0.9556988618574584,origin +3proxy-devel,tls-init-close-1k,1,0.0205520896,2005.61,31.29,46.51,0,0,0,10.04,10.028074,390.81184231505097,499.42794607769986,20.33984375,0.16630481980026052,0.9534662867996201,0.4135635018495684,0.11894297993036855,0.16824515511381852,proxy +haproxy,tls-init-close-1k,1,0.025417482239999997,2480.38,25.64,28.03,0,0,0,10.02,10.014808,315.37348681157175,403.02469632370685,37.859375,0.17990907122753844,0.9533777354900095,0.4598698481561822,0.14461803605679738,0.19294756256333243,proxy +nginx,tls-init-close-1k,1,0.034812723200000006,3396.92,18.72,20.49,0,0,0,10.03,10.009061,230.49044321818522,294.92193242965095,32.4609375,0.2654133818475844,0.9525166191832859,0.5373854317414375,0.20830536739664013,0.27548694290475495,proxy +envoy,tls-init-close-1k,1,0.06048186368,5893.82,10.5,14.56,0,0,0,9.99,9.925229,132.13878531065797,169.195853939435,19.1796875,0.25120570350178234,0.9496197718631179,0.5250436082731124,0.2834379861127693,0.23271254204393688,proxy +stunnel,tls-init-close-1k,1,0.02038431744,1983.51,31.48,51.65,0,0,0,10.03,9.939901,393.63594212159205,504.704875962361,20.08203125,0.1386535889433794,0.9534220532319392,0.3755784061696658,0.11414184021191724,0.15294220722219815,proxy +direct,tls-init-close-1k,1,0.12683575296,12370.37,0.5690000000000001,1.65,0,0,0,5.9,0.0,37.213481923259764,47.220777141942456,0.0,0.9520190023752969,0.6758304696449027,0.9468720528135807,3.787419660653997,0.9393004327061868,origin +3proxy-devel,tls-init-stream1-down,1,6.778821779722053,nan,nan,nan,0,0,0,10.0,10.001936,1.1800161675431136,nan,6.90625,0.12013828867761452,0.9514747859181731,0.2888028396256857,0.46055110506776037,0.6084823476155924,proxy +haproxy,tls-init-stream1-down,1,5.846312769726175,nan,nan,nan,0,0,0,10.01,10.003052,1.369599527587606,nan,35.765625,0.11402508551881414,0.9506172839506173,0.263884430176565,0.4413979082194608,0.5587704439401219,proxy +nginx,tls-init-stream1-down,1,7.6017539162414725,nan,nan,nan,0,0,2,10.0,10.000609,1.0522722237738609,nan,33.7890625,0.12517740562021004,0.9505703422053232,0.28846792954469924,0.47366796962412583,0.6688308040119565,proxy +envoy,tls-init-stream1-down,1,10.955224848848047,nan,nan,nan,0,0,0,10.01,9.998869000000001,0.7309485864206171,nan,24.67578125,0.1174812030075188,0.9515209125475285,0.4,0.23601588200569462,0.8395756468434993,proxy +stunnel,tls-init-stream1-down,1,4.795608625183398,nan,nan,nan,0,0,1,10.01,10.000457,1.6698423364559418,nan,5.7421875,0.11237830319888735,0.9506172839506173,0.17884297520661158,0.4215543965934909,0.42401484662024624,proxy +direct,tls-init-stream1-down,1,8.845369392159736,nan,nan,nan,0,0,0,0.19,0.0,0.01718230262475553,nan,0.0,0.5005027652086476,0.023543990086741014,0.3837906023336487,1.1188734081266047,0.8206808597482219,client +3proxy-devel,tls-init-stream16,1,8.348526041879582,nan,nan,nan,0,0,868,10.06,10.039975,0.961174535194798,nan,65.45703125,0.07860975040821087,0.9553656220322887,0.6574106213120444,0.09511296521841255,1.096991829366749,origin +haproxy,tls-init-stream16,1,5.937518659306832,nan,nan,nan,0,0,189,10.03,10.025227999999998,1.3502995738285455,nan,59.19921875,0.029081133919843596,0.9534220532319392,0.8708708708708709,0.06656302739661958,1.2897569968599922,origin +nginx,tls-init-stream16,1,7.838572817371034,nan,nan,nan,0,0,276,10.1,10.077734,1.024392927143987,nan,91.29296875,0.07186046511627907,0.957345971563981,0.6445066480055983,0.07988570806653485,1.0723465706063984,origin +envoy,tls-init-stream16,1,10.981565296725723,nan,nan,nan,0,0,7529,10.07,10.046444999999999,0.7313052365375371,nan,121.828125,0.08134083450539147,0.9554079696394687,0.6933154182062479,0.12175028292203176,1.3100040220041085,origin +stunnel,tls-init-stream16,1,5.157615200482038,nan,nan,nan,0,0,623,10.06,10.028229,1.5558830120489562,nan,56.29296875,0.027489751627682663,0.9553656220322887,0.5785369476420349,0.05801598681117103,0.941318332710401,origin +direct,tls-init-stream16,1,19.66306247313523,nan,nan,nan,0,0,767,0.51,0.0,0.02070314890198838,nan,0.0,0.5320643553138454,0.06480304955527319,0.9295154185022027,1.5642678563259558,28.347345127194817,origin +haproxy,tls-term-ka-1k,2,0.3288334336,31936.39,1.93,3.1,0,0,0,10.03,10.013764,24.40141171825175,31.351587896974245,36.015625,0.4109277177006261,0.9525166191832859,0.45345345345345345,0.7146669159365351,0.332579233598512,proxy +nginx,tls-term-ka-1k,2,0.34410070016000005,33420.64,1.89,2.35,0,0,0,10.02,10.069127,23.29550621743204,29.92125513991621,32.46875,0.42587601078167114,0.9524714828897338,0.4678774735700732,0.7751503992672535,0.30497461469923043,proxy +envoy,tls-term-ka-1k,2,0.3212836864,31204.57,1.96,2.98,0,0,0,10.09,10.105414,25.124213714201208,32.014569960878134,17.55078125,0.5137630971408276,0.9591254752851711,0.562905941784941,0.8884373344802152,0.3107348750724679,proxy +stunnel,tls-term-ka-1k,2,0.1430257664,13894.73,4.42,8.6,0,0,0,10.02,10.026845000000002,56.04584545683651,72.00965878057895,13.48046875,0.406758669497523,0.9533777354900095,0.41379310344827586,0.45961210744019154,0.17874150323549828,proxy +direct,tls-term-ka-1k,2,1.4377235251199998,139641.48,0.423,0.662,0,0,0,1.59,0.0,0.8847319931652592,1.1274316893393888,0.0,0.8828578056135468,0.43089430894308944,0.9529820132533923,3.1301228964205388,0.9510357842846556,origin +3proxy-devel,tls-term-ka-1k,2,0.17263755264,16769.6,3.6,7.88,0,0,0,10.02,10.022396,46.432539603452994,59.63930718409618,12.06640625,0.4245810055865922,0.9524714828897338,0.4507529005183905,0.5565802004880921,0.20526765339971662,proxy +haproxy,tls-term-ka-1m,2,4.88233762816,581.52,27.41,35.73,0,0,0,10.11,10.103785,1.6565835089631262,1721.4370849650945,42.8984375,0.5081123862287297,0.9601139601139601,0.14822673537435588,0.9134818371799617,0.024710959111089923,client +nginx,tls-term-ka-1m,2,7.24289191936,863.15,16.18,33.62,0,0,0,10.0,10.018656,1.1045311857569318,1157.0056693277797,43.71875,0.29292929292929293,0.9514747859181731,0.08688881968234195,0.8124892632445389,0.03136062583842448,proxy +envoy,tls-term-ka-1m,2,10.65151889408,1264.58,12.62,16.87,0,0,0,10.03,10.019421,0.7533197922091329,792.0713890863144,41.12890625,0.35921658986175115,0.9516129032258065,0.11940749697702539,1.0567530907363607,0.046561906006665855,client +stunnel,tls-term-ka-1m,2,4.85650071552,578.79,27.55,40.0,0,0,0,10.03,10.023925,1.6522184325758602,1730.2052785923754,15.66796875,0.21747916140439505,0.9525166191832859,0.061661341853035144,0.587862486055989,0.024702653428661635,proxy +direct,tls-term-ka-1m,2,31.86865733632,3797.73,4.18,6.6,0,0,0,7.23,0.0,0.18149493839541536,188.5119808098454,0.0,0.906857009127077,0.8387470997679815,0.9492113564668769,3.5375807901289584,0.9502110922461952,origin +3proxy-devel,tls-term-ka-1m,2,7.204220436479999,858.41,18.21,30.58,0,0,0,10.01,10.006914,1.111570650927046,1163.953488372093,15.359375,0.270303628733646,0.9524262607040913,0.07348748812163447,0.7831167601209016,0.032324450163658494,proxy +haproxy,tls-term-close-1k,2,0.02860515328,2791.55,15.0,16.49,0,0,0,10.1,10.081886,282.46658638425583,358.24495442131024,36.52734375,0.8370253164556962,0.9600760456273765,0.3230690274570182,3.2458036071078893,0.05511654276584143,client +nginx,tls-term-close-1k,2,0.033470545920000004,3260.1,1.21,1.62,0,0,0,10.1,10.087979,241.40628059406325,306.7484662576687,31.7890625,0.9519505233111323,0.9591642924976258,0.3795811518324607,3.777672140618581,0.05606441963521189,client +envoy,tls-term-close-1k,2,0.0402653184,3929.47,11.79,15.35,0,0,0,10.06,10.0202,199.8742421468099,253.45796276233906,19.01171875,0.9000230361667818,0.9571836346336822,0.3164929163325314,3.50386950987268,0.0731608214001628,client +stunnel,tls-term-close-1k,2,0.01996488704,1947.87,17.85,26.89,0,0,0,10.11,10.004071,405.1112327255121,513.928426189508,20.63671875,0.825663114939923,0.9601139601139601,0.2408619749045281,3.199547979016617,0.038952692175190734,client +direct,tls-term-close-1k,2,0.1266679808,12348.0,0.561,1.47,0,0,0,5.97,0.0,37.70487197977028,47.8679901858593,0.0,0.9524714828897338,0.6791808873720137,0.9464735516372796,3.792224525260157,0.941483701506363,origin +3proxy-devel,tls-term-close-1k,2,0.027850178559999998,2713.89,5.3,8.93,0,0,0,10.11,10.104875,290.4110644237105,368.83003173908287,14.0625,0.954772673173054,0.9610266159695817,0.32941470735367684,3.786314827126038,0.048477728487604,client +haproxy,tls-term-stream1-down,2,8.968599375780224,nan,nan,nan,0,0,1,9.63,8.850935000000002,0.8589691740820903,nan,38.27734375,0.471606824548001,0.8598214285714286,0.1500146498681512,1.0473339437508125,0.08830739845965337,client +nginx,tls-term-stream1-down,2,8.636059642597564,nan,nan,nan,0,0,0,9.82,9.239968,0.9095734353564449,nan,34.4765625,0.4688576729393056,0.8870822041553749,0.08398621121905359,1.0531026050569805,0.08534777220768175,client +envoy,tls-term-stream1-down,2,9.64194246382679,nan,nan,nan,0,0,1,9.02,6.594320000000001,0.7483133289679001,nan,26.2109375,0.4697935253632424,0.7297734627831716,0.0610062893081761,1.0598768657246265,0.09573663899930611,client +stunnel,tls-term-stream1-down,2,4.727783514453538,nan,nan,nan,0,0,0,10.03,10.004134,1.6970001363501541,nan,4.7578125,0.3436827543805718,0.9391385767790262,0.01818772328678142,0.8941120954913153,0.05346378262222705,proxy +direct,tls-term-stream1-down,2,9.280568456863806,nan,nan,nan,0,0,0,0.23,0.0,0.019825564819653148,nan,0.0,0.48358430017279685,0.025755879059350503,0.39379184418746194,1.0751779725198674,0.8094679506634208,client +3proxy-devel,tls-term-stream1-down,2,7.483711544983906,nan,nan,nan,0,0,0,9.86,9.567825000000001,1.0539077878075176,nan,6.95703125,0.465326917633739,0.9087557603686636,0.0685805422647528,1.0525559116722494,0.07785224580457077,client +haproxy,tls-term-stream16,2,4.825166495188452,nan,nan,nan,0,0,661,10.03,10.020363,1.6614448553657888,nan,55.234375,0.21141016793190706,0.9409005628517824,0.5207373271889401,0.48198378635282196,0.8550742835264348,proxy +nginx,tls-term-stream16,2,6.80332160424798,nan,nan,nan,0,0,520,10.05,10.052991,1.1779545111390417,nan,53.390625,0.21821631878557876,0.9427767354596623,0.2176560121765601,0.5990328463711426,0.46895727829320333,proxy +envoy,tls-term-stream16,2,9.350486773219757,nan,nan,nan,0,0,729,10.07,10.067895,0.8586724814154829,nan,61.42578125,0.2726628895184136,0.9446529080675422,0.11827618748033973,0.7550716753873961,0.23141995069069154,proxy +stunnel,tls-term-stream16,2,4.328418471651339,nan,nan,nan,0,0,479,10.07,10.025249,1.8559281797682747,nan,22.92578125,0.16020551144325082,0.943767572633552,0.18752348741074784,0.3898960057929893,0.3918973778300262,proxy +direct,tls-term-stream16,2,19.267557448239685,nan,nan,nan,0,0,770,0.51,0.0,0.021120634928308554,nan,0.0,0.519927536231884,0.06319702602230483,0.9336895034569453,1.525441135865811,1.9436265325616784,origin +3proxy-devel,tls-term-stream16,2,6.525649907980567,nan,nan,nan,0,0,503,10.05,10.010306,1.2290484249521128,nan,15.41796875,0.24103631737219525,0.9418931583880038,0.20954198473282443,0.5961738622476135,0.4472656045201327,proxy +haproxy,tls-init-ka-1k,2,0.33361494016000004,32400.88,1.92,2.92,0,0,0,10.11,10.099167,24.2435185789972,30.89409528582386,36.7265625,0.42649310872894336,0.9601139601139601,0.4293770139634801,0.7089433284844637,0.27944815575151094,proxy +nginx,tls-init-ka-1k,2,0.34997272576,33988.5,1.86,2.33,0,0,0,10.01,10.01811,22.881783094982172,29.407098832529364,33.20703125,0.4274132730015083,0.9524262607040913,0.4494658997534922,0.6716650770493547,0.2888964333763196,proxy +envoy,tls-init-ka-1k,2,0.32220643328,31297.27,1.96,3.05,0,0,0,10.0,9.989739,24.828802822344446,31.886738305538728,17.46875,0.49851007887817705,0.9505703422053232,0.553567167740396,0.7593842531004811,0.3070797150729684,proxy +stunnel,tls-init-ka-1k,2,0.14604566528000001,14187.81,4.33,8.64,0,0,0,10.03,10.023702,54.94171966430032,70.58806970132026,14.453125,0.3899517253709995,0.9507109004739337,0.41609251968503935,0.39661778215746857,0.16135901979361764,proxy +direct,tls-init-ka-1k,2,1.43260647424,139139.45,0.41500000000000004,0.746,0,0,0,2.33,0.0,1.3011249310379216,1.6580231240855967,0.0,0.8789618741387231,0.5393518518518519,0.951496062992126,3.1239657884918812,0.951668403483684,origin +3proxy-devel,tls-init-ka-1k,2,0.1769996288,17191.73,3.52,8.07,0,0,0,10.02,10.019112,45.288230570571685,58.21452225746854,12.43359375,0.4183401457259641,0.9524714828897338,0.45017182130584193,0.4776990446520995,0.2242895288315792,proxy +haproxy,tls-init-ka-1m,2,4.614153830399999,549.43,29.0,38.12,0,0,0,10.1,10.102464,1.7511336416149668,1820.147774373761,48.37109375,0.4638196915776987,0.9591642924976258,0.1913991520290733,0.7077436581546265,0.12540260862770147,proxy +nginx,tls-init-ka-1m,2,5.8165769011199995,692.99,21.61,44.26,0,0,0,10.02,10.020264000000001,1.3781301504767338,1443.5960236277194,44.0859375,0.1933028919330289,0.9515669515669516,0.18346333853354135,0.45558047273462804,0.17384430798558817,proxy +envoy,tls-init-ka-1m,2,8.589934592,1020.64,15.56,21.68,0,0,0,10.0,10.000572,0.9313225746154785,978.3778495254868,39.65234375,0.17091245947197778,0.9514747859181731,0.2697070371488976,0.39958115838789576,0.26797225398575725,proxy +stunnel,tls-init-ka-1m,2,4.19329736704,499.67,29.15,77.7,0,0,0,10.04,10.01838,1.915437732399526,2005.1927301777512,18.15625,0.15841073271413827,0.952561669829222,0.12373417721518987,0.39957156932750854,0.13873068821903145,proxy +direct,tls-init-ka-1m,2,30.4942678016,3635.82,4.43,7.95,0,0,0,6.94,0.0,0.18206700472764567,188.97723559525107,0.0,0.8992686954470394,0.8232502965599051,0.9318323952470294,3.489327784596196,0.955611981392214,origin +3proxy-devel,tls-init-ka-1m,2,6.143397068800001,732.14,19.61,47.43,0,0,0,10.01,9.984368,1.3035133347752526,1365.0620482749216,15.52734375,0.19990079365079366,0.9515209125475285,0.1783359497645212,0.4556760218601475,0.18814525834964735,proxy +haproxy,tls-init-close-1k,2,0.025165824,2452.58,25.91,28.92,0,0,0,10.03,10.019024,318.84511311848956,407.9889358932639,38.03125,0.1740073768713387,0.9525166191832859,0.45713595749818886,0.14362115394287908,0.19379040243355145,proxy +nginx,tls-init-close-1k,2,0.03456106496,3371.32,18.79,21.65,0,0,0,10.03,10.009682,232.16877168822052,297.0531615578262,33.36328125,0.26589945487583283,0.9525166191832859,0.5369208494208494,0.20739408911666185,0.2736890449391615,proxy +envoy,tls-init-close-1k,2,0.05813305344,5667.53,10.87,16.19,0,0,0,9.98,9.900231,137.34011078981786,175.61147281365476,19.25,0.24025423728813558,0.9477682811016145,0.5211864406779662,0.2786747697771259,0.23460509573367108,proxy +stunnel,tls-init-close-1k,2,0.019797114879999998,1931.59,32.12,52.34,0,0,0,10.04,9.932929,405.7156837592691,518.3808343659645,18.0234375,0.13486915678819056,0.9543726235741445,0.375224877923413,0.11322497308445967,0.1511396879125909,proxy +direct,tls-init-close-1k,2,0.12490637312000001,12179.95,0.58,2.0,0,0,0,6.01,0.0,38.492831709882886,48.84630076642365,0.0,0.9545670789724072,0.685290763968073,0.945528967254408,3.7883910301492008,0.9416777150247672,origin +3proxy-devel,tls-init-close-1k,2,0.0201326592,1959.36,31.94,47.36,0,0,0,10.04,10.031596,398.95375569661456,510.88947689802563,16.5703125,0.16138894955230398,0.9543726235741445,0.41196834817012856,0.11703460326425069,0.16918514107144272,proxy +haproxy,tls-init-stream1-down,2,6.2095089349680475,nan,nan,nan,0,0,2,9.98,9.924361,1.285628660563529,nan,39.59375,0.11718086611944523,0.9477682811016145,0.2666232073011734,0.45369101490501135,0.5595078051772462,proxy +nginx,tls-init-stream1-down,2,6.95793759706339,nan,nan,nan,0,0,0,9.99,10.000771,1.1484788645528825,nan,39.546875,0.12242562929061784,0.950523311132255,0.2855747313578639,0.46794344780213487,0.6289645114261664,proxy +envoy,tls-init-stream1-down,2,10.602435969850474,nan,nan,nan,0,0,5,9.97,9.95305,0.7522182679815519,nan,23.296875,0.10244147998993204,0.9930278884462151,0.39627842866988283,0.224566693722654,0.8365168777145631,proxy +stunnel,tls-init-stream1-down,2,4.296702701740668,nan,nan,nan,0,0,0,9.99,10.004566,1.8597835968763725,nan,3.80859375,0.09842075578116187,0.950523311132255,0.20205589463539994,0.3937244983993934,0.40275029697538167,proxy +direct,tls-init-stream1-down,2,9.246768142707195,nan,nan,nan,0,0,0,0.33,0.0,0.028548914210530792,nan,0.0,0.4900437105390966,0.03650442477876106,0.3407821229050279,1.0804634142923428,0.8393525065936837,client +3proxy-devel,tls-init-stream1-down,2,7.22981240533617,nan,nan,nan,0,0,0,10.01,10.00555,1.1075052461361792,nan,5.58984375,0.12827822120866592,0.9506172839506173,0.33886626996554964,0.479473441600731,0.6650384319639997,proxy +haproxy,tls-init-stream16,2,5.96259483329159,nan,nan,nan,0,0,3,10.02,10.024015,1.3436187641156672,nan,47.4140625,0.041939393939393936,0.9515669515669516,0.8697534976682212,0.07989339792745656,1.2757819328115165,origin +nginx,tls-init-stream16,2,7.686542335357298,nan,nan,nan,0,0,629,10.05,10.054053,1.0418987802894677,nan,70.05078125,0.05486641221374046,0.9553231939163498,0.6394819740987049,0.08276288491161637,1.0491590720549449,origin +envoy,tls-init-stream16,2,11.062095301183447,nan,nan,nan,0,0,8642,10.05,10.037702,0.7250835961215212,nan,117.53125,0.0774812734082397,0.9544159544159544,0.7045148247978437,0.12270659389069657,1.3424051373589132,origin +stunnel,tls-init-stream16,2,5.239605769183789,nan,nan,nan,0,0,631,10.05,10.033299999999999,1.530080919521522,nan,46.265625,0.02703355056722182,0.9544159544159544,0.5678373382624768,0.058032006675349565,0.9224683534584275,origin +direct,tls-init-stream16,2,19.643355063016077,nan,nan,nan,0,0,802,0.52,0.0,0.021148139109517843,nan,0.0,0.5287981859410431,0.06508135168961202,0.9277374566109182,1.5566616672365599,1.938116030685343,origin +3proxy-devel,tls-init-stream16,2,8.42007141579248,nan,nan,nan,0,0,900,10.06,10.041384,0.9526444944502712,nan,61.51953125,0.07548054383497421,0.9562737642585551,0.6500172235618326,0.09035494427132373,1.090465747761052,origin diff --git a/bench/data/skylake-tls.csv b/bench/data/skylake-tls.csv index 48194dc..41fefdf 100644 --- a/bench/data/skylake-tls.csv +++ b/bench/data/skylake-tls.csv @@ -1,91 +1,181 @@ proxy,scenario,rep,gbit,rate,p50_ms,p99_ms,errors,timeouts,retrans,prx_cpu_s,prx_task_s,cpu_s_per_gb,cpu_us_per_req,peak_mb,cli_util,prx_util,srv_util,cli_thread,srv_thread,bound -direct,tls-term-ka-1k,0,1.1194597376,109493.64,0.5640000000000001,0.88,0,0,0,7.37,0.0,5.266826310913498,6.664604889306068,0.0,0.6467110125646711,0.7155339805825243,0.9559848005066498,1.9942851466688527,0.8402770418544304,origin -3proxy-current,tls-term-ka-1k,0,0.25014829056,24470.78,2.49,4.56,0,0,0,10.1,7.472335,32.30084036117748,40.867689842558235,14.01953125,0.237912876974629,0.9600760456273765,0.3274647887323944,0.6650230362408502,0.3032660986708979,proxy -haproxy,tls-term-ka-1k,0,0.34493956096,33737.22,1.75,3.11,0,0,0,10.02,7.024958,23.238853721767086,29.674205652314264,39.1171875,0.27087242026266417,0.9524714828897338,0.3939868462261196,0.7478065772405075,0.32990281828206053,proxy -nginx,tls-term-ka-1k,0,0.36683382783999996,35883.47,1.66,2.63,0,0,0,10.11,6.714439,22.04813020550466,27.89681214766823,34.90625,0.27857988165680475,0.9601139601139601,0.410288582183187,0.7848410024270118,0.34317401568043043,proxy -stunnel,tls-term-ka-1k,0,0.20308819968,19862.73,2.99,6.95,0,0,0,10.04,7.650684,39.54931902816501,50.49742986188651,18.52734375,0.20702844848195076,0.9543726235741445,0.28337612323491657,0.5784034584947134,0.23670157509482284,proxy -direct,tls-term-ka-1m,0,22.67742732288,2705.24,5.81,10.12,0,0,0,8.19,0.0,0.2889216623523018,299.75843642485904,0.0,0.7610872675250357,0.8297872340425532,0.9534957291996203,2.5895535072121367,0.8562784735233212,origin -3proxy-current,tls-term-ka-1m,0,4.92251906048,586.56,26.44,59.84,0,0,0,10.02,7.145911,1.6284345274263603,1706.984667802385,19.08984375,0.2776717557251908,0.9524714828897338,0.05218216318785579,0.8296491258355196,0.042785272849970885,proxy -haproxy,tls-term-ka-1m,0,5.8997080064,702.39,22.17,32.44,0,0,0,10.1,7.309323,1.3695593055172934,1423.7383704539047,46.52734375,0.29105771612583603,0.9600760456273765,0.07271577616187164,0.8048629147918888,0.03422650828012293,proxy -nginx,tls-term-ka-1m,0,6.29657305088,750.4,20.67,31.44,0,0,0,10.03,7.495661,1.2743439860319856,1334.664005322688,47.80859375,0.25861244019138757,0.9525166191832859,0.06675062972292191,0.7496642279925745,0.046582996396474605,proxy -stunnel,tls-term-ka-1m,0,5.1266977792,610.99,25.42,59.16,0,0,0,10.03,7.741818,1.5651400463969054,1639.9607586657946,20.0234375,0.2268440145102781,0.9534220532319392,0.05780164245104232,0.6507519932640717,0.04087871882467707,proxy -direct,tls-term-close-1k,0,0.04286578688,4207.33,1.55,5.34,0,0,0,3.6,0.0,67.18644890533268,84.96577767288176,0.0,0.9538973384030418,0.3474903474903475,0.9068736141906873,3.583540794177188,0.8232646501143567,origin -3proxy-current,tls-term-close-1k,0,0.01098907648,1080.27,22.53,48.39,0,0,0,10.12,8.900212,736.7316093153624,927.5043534048208,42.8046875,0.9548371761350131,0.9619771863117871,0.06321473951715374,3.6999085064935566,0.028521737238229684,client -haproxy,tls-term-close-1k,0,0.01400897536,1372.98,27.29,41.4,0,0,0,10.1,8.908869,576.7730895630614,728.2428437522532,38.1015625,0.7685163134079542,0.9619047619047619,0.07070386810399493,2.932285448471752,0.02947374185889227,client -nginx,tls-term-close-1k,0,0.01400897536,1377.42,3.1,4.85,0,0,0,10.11,8.769033,577.3441520279753,726.710753306498,33.359375,0.9557774607703281,0.9610266159695817,0.08386277001270648,3.705869538524049,0.037064883202621905,client -stunnel,tls-term-close-1k,0,0.01098907648,1076.02,21.55,52.77,0,0,0,10.11,8.96967,736.0036136539836,930.253956569746,34.19921875,0.9465304182509505,0.9628571428571429,0.06482364156339371,3.687378250238888,0.029471449078566612,client -direct,tls-init-ka-1k,0,1.08263374848,105887.95,0.5650000000000001,0.94,0,0,0,7.27,0.0,5.37208451904032,6.797670652389376,0.0,0.6480750246791708,0.7044573643410853,0.9579113924050633,1.9966754355184562,0.8476755626979373,origin -3proxy-current,tls-init-ka-1k,0,0.23815258112,23295.97,2.55,9.15,0,0,0,10.1,7.479255,33.92782879782714,42.9272107513537,15.453125,0.17768199233716475,0.9600760456273765,0.36823453608247425,0.4386048730614794,0.33643266951347556,proxy -haproxy,tls-init-ka-1k,0,0.34301018112,33545.49,1.81,14.6,0,0,0,10.09,7.09971,23.53282918204711,29.781142545121824,42.10546875,0.224934194783441,0.960952380952381,0.43178973717146435,0.5679634627033696,0.3802972087446423,proxy -nginx,tls-init-ka-1k,0,0.34091302912,33343.45,1.78,3.2,0,0,0,10.09,6.803458,23.677593141090213,29.96101208239427,38.4453125,0.22347575612097936,0.9591254752851711,0.45906801007556675,0.559362248529181,0.42001702694852067,proxy -stunnel,tls-init-ka-1k,0,0.20644364288,20193.48,2.92,12.26,0,0,0,10.14,7.714791,39.29401693766508,49.7175806072017,16.58984375,0.16155321188878236,0.9629629629629629,0.3395081443628234,0.39387222180360004,0.27941077827725713,proxy -direct,tls-init-ka-1m,0,20.95944040448,2493.51,6.78,13.24,0,0,0,8.04,0.0,0.30687842212739536,319.23764145324594,0.0,0.7303290414878397,0.8080402010050252,0.9269680682895984,2.466781691288365,0.8196875576746906,origin -3proxy-current,tls-init-ka-1m,0,5.164865945600001,615.45,25.05,57.51,0,0,0,10.03,7.472096,1.5535737199211772,1628.5111219353792,18.375,0.17719341361513885,0.9507109004739337,0.26473369051370943,0.4613840727624356,0.23945357085597052,proxy -haproxy,tls-init-ka-1m,0,5.1099205632,608.02,25.44,37.1,0,0,0,10.1,7.5443940000000005,1.5812378881561397,1644.6832763393584,54.546875,0.19995050730017322,0.9600760456273765,0.2572692793931732,0.5051365812803598,0.2785377466673379,proxy -nginx,tls-init-ka-1m,0,5.44319995904,648.56,24.08,34.66,0,0,0,10.03,7.373113999999999,1.4741328741145798,1545.4545454545455,43.8984375,0.1904527559055118,0.9516129032258065,0.2863422291993721,0.4860833250705032,0.2946700749773043,proxy -stunnel,tls-init-ka-1m,0,4.20957126656,501.63,30.5,71.58,0,0,0,10.03,7.781262,1.9061323569317057,1997.6100378410674,20.43359375,0.1535635562086701,0.9534220532319392,0.2157545080670674,0.3881396735403182,0.22816363189430924,proxy -direct,tls-init-close-1k,0,0.034393292799999996,3378.49,1.52,4.64,0,0,0,2.63,0.0,61.17471834508383,77.50338893145518,0.0,0.9528907922912205,0.25191570881226055,0.8751979727589484,3.599362891586791,0.86689710796206,client -3proxy-current,tls-init-close-1k,0,0.0088080384,864.36,70.96,121.97,0,0,0,10.04,9.072577,911.894298735119,1159.2194896663202,76.5703125,0.03870814533364996,0.9561904761904761,0.2731012658227848,0.0646956757172168,0.23294151831009602,proxy -haproxy,tls-init-close-1k,0,0.00973078528,951.83,62.48,96.56,0,0,0,10.03,8.9932,824.5994304788524,1051.8036912751677,42.62890625,0.04185493460166469,0.9534220532319392,0.28748019017432647,0.068502521929934,0.24624148261424658,proxy -nginx,tls-init-close-1k,0,0.01283457024,1261.4,48.86,71.27,0,0,0,10.05,8.818029000000001,626.4331294041053,795.598480050665,31.62109375,0.05712254088646599,0.9553231939163498,0.35991175543649545,0.09704781724589216,0.30791304706611916,proxy -stunnel,tls-init-close-1k,0,0.00973078528,952.99,60.54,330.32,0,0,0,10.07,8.896381,827.8879626043912,1055.2237241957455,28.5625,0.04579971523493118,0.956315289648623,0.24723713293337543,0.075160811692268,0.20726306873233175,proxy -3proxy-current,tls-term-ka-1k,1,0.24184356863999998,23655.24,2.57,4.9,0,0,0,10.1,7.463227,33.410026346524894,42.27444907184564,14.09765625,0.23694298035457595,0.9600760456273765,0.3238308776425368,0.662156994575174,0.25667947228469246,proxy -haproxy,tls-term-ka-1k,1,0.34921775104,34157.4,1.75,2.93,0,0,0,10.03,7.021541999999999,22.977067964339867,29.305463438293447,38.75,0.2665879574970484,0.9534220532319392,0.39593114241001565,0.753481525112189,0.34700718616226184,proxy -nginx,tls-term-ka-1k,1,0.36096180224,35302.14,1.69,2.78,0,0,0,10.02,6.681599,22.207335929329826,28.329573020746746,35.01171875,0.2880833925610045,0.9524714828897338,0.410288582183187,0.8077387679797692,0.3744450896339174,proxy -stunnel,tls-term-ka-1k,1,0.21692940287999998,21218.28,2.84,6.57,0,0,0,10.12,7.70104,37.320897455650616,47.22506101459217,16.0234375,0.21948889419632195,0.9619771863117871,0.30248883216336947,0.6070026201252147,0.2719186248164488,proxy -direct,tls-term-ka-1k,1,1.0890090905599998,106518.51,0.56,0.95,0,0,0,7.25,0.0,5.325942685214383,6.739208435815779,0.0,0.6322836598065956,0.7018393030009681,0.9572243346007605,1.9457815100926927,0.8492458801910123,origin -3proxy-current,tls-term-ka-1m,1,4.83049603072,575.61,26.94,62.46,0,0,0,10.02,7.147285999999999,1.6594569065001779,1739.281374761326,18.0703125,0.2709477202196228,0.9515669515669516,0.050933248971844355,0.8020386626920842,0.04088170992854504,proxy -haproxy,tls-term-ka-1m,1,5.71834630144,680.79,22.24,32.3,0,0,0,10.09,7.391886,1.4115969153472396,1467.4229203025016,48.6875,0.2883617494440326,0.960952380952381,0.06622516556291391,0.7943766224387107,0.03612636365880328,proxy -nginx,tls-term-ka-1m,1,6.0456697856,720.52,21.42,32.06,0,0,0,10.02,7.475741,1.3259076800875018,1388.9658996395897,48.23046875,0.2518002880460874,0.9524714828897338,0.06236220472440945,0.7268675747928643,0.04373694304635185,proxy -stunnel,tls-term-ka-1m,1,5.47306340352,652.13,24.04,55.44,0,0,0,10.04,7.784278,1.467551060131008,1538.9331698344574,20.05859375,0.2372718539865514,0.9534662867996201,0.062460567823343846,0.6736025246218889,0.04088345494128453,proxy -direct,tls-term-ka-1m,1,22.849226014720003,2725.64,5.89,11.59,0,0,0,8.21,0.0,0.2874495615636496,298.2526246957533,0.0,0.7637963843958135,0.8360488798370672,0.9464851171627613,2.5974012039744174,0.8588979777160687,origin -3proxy-current,tls-term-close-1k,1,0.012582912,1237.99,19.37,44.4,0,0,0,10.12,8.865654999999999,643.4122721354166,809.3410108765195,53.9765625,0.9531621493105088,0.9610636277302944,0.06743002544529263,3.699060414906116,0.030413040186725,client -haproxy,tls-term-close-1k,1,0.01417674752,1393.81,26.76,43.86,0,0,0,10.01,8.84597,564.8686335637019,717.4598623853211,38.14453125,0.7505949547834364,0.9524262607040913,0.06622306717363752,2.87981288526459,0.0304195615733104,client -nginx,tls-term-close-1k,1,0.0150994944,1485.46,3.01,4.23,0,0,0,10.13,8.809959,536.7067125108508,675.1532924553452,34.2578125,0.9536376604850214,0.961100569259962,0.08436409768474469,3.703024618254623,0.037079432561640494,client -stunnel,tls-term-close-1k,1,0.011240734720000001,1105.87,22.33,58.85,0,0,0,10.12,8.990985,720.2376180620334,906.0793267078521,33.6484375,0.9365343475160447,0.9628924833491912,0.06263910969793322,3.640140800258993,0.026619042119849314,client -direct,tls-term-close-1k,1,0.044459622399999996,4368.77,1.37,4.12,0,0,0,3.06,0.0,55.06119638119104,69.8582288884323,0.0,0.9522678698646402,0.29423076923076924,0.9164027865737808,3.5838588862622704,0.8535331102029736,origin -3proxy-current,tls-init-ka-1k,1,0.25467813888,24909.29,2.45,9.6,0,0,0,10.1,7.445309,31.726319485188153,40.14627553859607,15.453125,0.17972681524083392,0.9600760456273765,0.3825588796944621,0.44718341123734623,0.3118851039676552,proxy -haproxy,tls-init-ka-1k,1,0.33160167424000003,32433.37,1.82,3.17,0,0,0,10.01,7.015357,24.14945587459287,30.834724643752658,42.5625,0.22458741927768477,0.9515209125475285,0.43470149253731344,0.5679230139372056,0.3916792412681567,proxy -nginx,tls-init-ka-1k,1,0.37044092928,36231.05,1.68,2.58,0,0,0,10.1,6.750587,21.81184464606686,27.602265025470604,37.9921875,0.23570566948130278,0.9600760456273765,0.48954372623574144,0.6050895101616365,0.42590048431961447,proxy -stunnel,tls-init-ka-1k,1,0.20057161728,19614.83,3.06,7.43,0,0,0,10.14,7.742308,40.44440639213457,51.18446092263272,16.53125,0.15878378378378377,0.9629629629629629,0.34713879228580463,0.3967153990647493,0.29566125559174905,proxy -direct,tls-init-ka-1k,1,1.09689438208,107289.65,0.5740000000000001,0.97,0,0,0,7.52,0.0,5.484575450730345,6.9395227970706355,0.0,0.6395262768319763,0.735812133072407,0.9582410629547612,1.9627920411500144,0.844395054470617,origin -3proxy-current,tls-init-ka-1m,1,5.15547070464,614.33,25.29,56.81,0,0,0,10.0,7.447799,1.551749676862654,1627.0745200130166,19.98828125,0.19309100283578243,0.999000999000999,0.2816015883520847,0.46749050126567226,0.33340386717275095,proxy -haproxy,tls-init-ka-1m,1,5.1187286016,609.75,25.05,39.17,0,0,0,10.11,7.567824,1.580079865432184,1641.7668074050016,52.67578125,0.19812252964426877,0.9601139601139601,0.25962145110410095,0.4947301677400869,0.3536716902270825,proxy -nginx,tls-init-ka-1m,1,5.42801657856,646.82,23.59,36.77,0,0,0,10.01,7.3381739999999995,1.4753086848759116,1546.1847389558234,45.08984375,0.1848387890721142,0.9515209125475285,0.27835375431982406,0.4794874384374446,0.24717662919930292,proxy -stunnel,tls-init-ka-1m,1,4.56038285312,543.48,28.92,61.02,0,0,0,10.02,7.748921999999999,1.7577471581176194,1841.9117647058824,19.328125,0.16544117647058823,0.9533777354900095,0.23479357075323037,0.4214135455095488,0.2452523290157053,proxy -direct,tls-init-ka-1m,1,24.30951489536,2893.37,5.24,10.82,0,0,0,8.25,0.0,0.27149863040910593,282.31187763063343,0.0,0.7631141704248754,0.840122199592668,0.9569347688410387,2.590333892278712,0.8539590235113941,origin -3proxy-current,tls-init-close-1k,1,0.008724152320000001,853.33,72.04,123.14,0,0,0,10.08,9.078254000000001,924.3304912860576,1177.2950245269797,106.28515625,0.03732762719923918,0.9572649572649573,0.2735312697409981,0.0646964799238412,0.2338931695429387,proxy -haproxy,tls-init-close-1k,1,0.010317987839999999,1014.39,61.2,89.13,0,0,0,10.03,9.006401,777.6710076060721,987.2047244094488,40.25390625,0.04085510688836105,0.9552380952380952,0.2985781990521327,0.06945446102926073,0.2519581519683936,proxy -nginx,tls-init-close-1k,1,0.0130023424,1280.33,48.97,65.27,0,0,0,10.07,8.833155999999999,619.5806687878024,785.1239669421487,33.2734375,0.05444602948169282,0.956315289648623,0.3599621092516577,0.096097654203865,0.3118427993095415,proxy -stunnel,tls-init-close-1k,1,0.0100663296,987.32,60.97,179.83,0,0,0,10.07,8.884551,800.2916971842449,1017.7885587224581,26.97265625,0.04238095238095238,0.956315289648623,0.25370779425686335,0.07610681090710829,0.21579858590296877,proxy -direct,tls-init-close-1k,1,0.0390070272,3833.45,1.35,4.49,0,0,0,2.97,0.0,60.912101499495975,77.23312963203745,0.0,0.9522451888809693,0.28557692307692306,0.8707526198793267,3.599204587241182,0.7957784211213861,client -haproxy,tls-term-ka-1k,2,0.3388997632,33146.84,1.8,3.07,0,0,0,10.03,7.025264,23.676617310778926,30.20756785408811,38.6328125,0.2640661938534279,0.9525166191832859,0.3856516290726817,0.7468796501643881,0.362099393159481,proxy -nginx,tls-term-ka-1k,2,0.36373004288,35576.38,1.64,2.72,0,0,0,10.02,6.674969,22.03832253318871,28.118738529406812,35.046875,0.279613025011798,0.9524714828897338,0.4095894703854591,0.7849563178023642,0.3289797478182464,proxy -stunnel,tls-term-ka-1k,2,0.20627587072,20172.31,2.93,6.77,0,0,0,10.03,7.639314000000001,38.89936312954326,49.69011795829597,16.109375,0.20660761311946374,0.9534220532319392,0.2871696911811525,0.5822243005878621,0.23768837393448636,proxy -direct,tls-term-ka-1k,2,1.09613940736,107215.88,0.519,1.18,0,0,0,7.24,0.0,5.28399942663293,6.68562774720201,0.0,0.6309435821630943,0.7029126213592233,0.9585311807534029,1.928793342330788,0.8492639797650146,origin -3proxy-current,tls-term-ka-1k,2,0.24905777152,24356.11,2.49,5.12,0,0,0,10.11,7.467399,32.47439319254694,41.09856337959462,14.01171875,0.23248026787849796,0.9601139601139601,0.3248326426522155,0.648899048000669,0.27860010218781583,proxy -haproxy,tls-term-ka-1m,2,6.06773182464,722.77,21.77,29.9,0,0,0,10.1,7.316416,1.3316343295180795,1383.5616438356165,48.5078125,0.2947290274684484,0.9600760456273765,0.07561436672967864,0.8123886312990025,0.03707507168062021,proxy -nginx,tls-term-ka-1m,2,5.82177783808,693.84,21.17,35.66,0,0,0,10.0,7.48522,1.3741506842930937,1440.2995823131212,47.0625,0.24687199230028875,0.9514747859181731,0.055888853804862644,0.7107182919596713,0.04086582169203762,proxy -stunnel,tls-term-ka-1m,2,4.642339553279999,553.13,27.4,70.17,0,0,0,10.03,7.776821,1.728438841646282,1812.1047877145438,19.6953125,0.20920096852300243,0.9534220532319392,0.04724159797083069,0.5917782925101898,0.03612834129274017,proxy -direct,tls-term-ka-1m,2,22.59152797696,2687.25,4.87,15.31,0,0,0,7.98,0.0,0.2825838078110844,293.9767913059495,0.0,0.7588838540424517,0.806060606060606,0.9553656220322887,2.5740331857966865,0.8729504556873563,origin -3proxy-current,tls-term-ka-1m,2,4.91790532608,585.98,26.45,61.09,0,0,0,10.02,7.152266,1.6299622437810228,1708.4398976982097,18.70703125,0.279446828803052,0.9524714828897338,0.05083675402589201,0.8285317016944691,0.03232033074241261,proxy -haproxy,tls-term-close-1k,2,0.01409286144,1387.75,27.01,39.65,0,0,0,10.11,8.923624,573.9075796944753,721.318493150685,38.125,0.7507149666348903,0.961941008563273,0.06586064269805918,2.86757815205729,0.03042395324596571,client -nginx,tls-term-close-1k,2,0.01459617792,1432.73,3.04,4.44,0,0,0,10.12,8.793505999999999,554.6657518408765,699.3296938704996,32.4140625,0.9582047019710282,0.9619771863117871,0.08290978398983481,3.7058639902721384,0.0370772087119028,client -stunnel,tls-term-close-1k,2,0.01023410176,1006.86,23.44,57.25,0,0,0,10.13,8.976559,791.8623627209273,996.0668633235005,44.88671875,0.9324774132192106,0.9629277566539924,0.056615776081424936,3.6263739757288787,0.02660811878950716,client -direct,tls-term-close-1k,2,0.03539992576,3472.31,1.36,4.65,0,0,0,2.8,0.0,63.2769688610782,80.45052292839904,0.0,0.9515324305060584,0.2697495183044316,0.859493670886076,3.5989094224272002,0.8078214488024834,client -3proxy-current,tls-term-close-1k,2,0.01291845632,1265.46,19.2,43.25,0,0,0,10.13,8.866699,627.3195341035918,792.5827400046944,26.51171875,0.9517585551330798,0.961100569259962,0.06925031766200762,3.6878071840084887,0.03137736114803411,client -haproxy,tls-init-ka-1k,2,0.33839644672,33094.55,1.81,3.17,0,0,0,10.1,7.065385,23.877319275416767,30.216632311452702,37.65234375,0.22140133460438513,0.9600760456273765,0.425512104283054,0.548914173442944,0.4049978454420713,proxy -nginx,tls-init-ka-1k,2,0.375390208,36713.93,1.65,2.71,0,0,0,10.1,6.743352,21.524269487604748,27.237889252360393,36.015625,0.24158653846153846,0.9600760456273765,0.5014344915524387,0.6107614796798443,0.44380242302070183,proxy -stunnel,tls-init-ka-1k,2,0.21877489664,21400.55,2.82,7.15,0,0,0,10.12,7.670164000000001,37.00607393416891,46.821721207186116,16.2265625,0.16017316017316016,0.9619771863117871,0.3560126582278481,0.4034064305763258,0.285231512111838,proxy -direct,tls-init-ka-1k,2,1.0775166976,105386.82,0.597,1.04,0,0,0,7.4,0.0,5.494114395800896,6.952240923332754,0.0,0.629866929521932,0.7212475633528265,0.9585049097244219,1.9318361435249158,0.8442186732191183,origin -3proxy-current,tls-init-ka-1k,2,0.24486346752000002,23952.57,2.52,6.79,0,0,0,10.1,7.466133,32.99798080062735,41.74899347723647,15.53125,0.1762866762866763,0.9600760456273765,0.3820366856419987,0.4433571685833449,0.3318177913299525,proxy -haproxy,tls-init-ka-1m,2,5.15161194496,613.07,25.1,36.43,0,0,0,10.11,7.561844000000001,1.5699940302981807,1632.7519379844962,48.96875,0.20385947550717468,0.9610266159695817,0.25702557625513106,0.509897082368172,0.2785221479454837,proxy -nginx,tls-init-ka-1m,2,5.39672707072,643.07,24.31,33.38,0,0,0,10.02,7.355081,1.4853447089238385,1557.109557109557,45.12109375,0.1896212493851451,0.9524714828897338,0.2780395852968897,0.4823316183459257,0.29186636453340337,proxy -stunnel,tls-init-ka-1m,2,4.606184652800001,548.86,28.55,58.96,0,0,0,10.04,7.74968,1.743742512605855,1828.1136198106337,19.22265625,0.16042385411532775,0.9534662867996201,0.23423707440100883,0.39953144165606513,0.34316182816272983,proxy -direct,tls-init-ka-1m,2,20.95944040448,2500.99,6.86,13.27,0,0,0,7.96,0.0,0.30382490548931185,315.09777531470195,0.0,0.7287043664996421,0.8008048289738431,0.9344729344729344,2.4373570409781307,0.8358760954322354,origin -3proxy-current,tls-init-ka-1m,2,5.04801263616,601.58,25.75,57.86,0,0,0,10.01,7.463358,1.586366868941051,1662.7906976744187,17.8671875,0.1805213969503197,0.9515209125475285,0.26418663303909207,0.46422173811209644,0.3127123492444544,proxy -haproxy,tls-init-close-1k,2,0.01015021568,992.76,62.58,83.25,0,0,0,10.05,9.013471,792.1013950316375,1010.4564649105168,40.19921875,0.04467680608365019,0.9553231939163498,0.31021437578814626,0.07706499279148822,0.2605029856855648,proxy -nginx,tls-init-close-1k,2,0.01291845632,1265.49,49.44,70.91,0,0,0,10.08,8.822935999999999,624.2231889204546,795.5173230210718,31.58203125,0.05611031859248693,0.9581749049429658,0.3640654499685337,0.09894778197969596,0.30901439407837994,proxy -stunnel,tls-init-close-1k,2,0.00939524096,919.55,64.89,159.59,0,0,0,10.07,8.890711000000001,857.4553898402622,1093.1393834129397,35.2890625,0.04349893035417162,0.9572243346007605,0.24677368586717027,0.07515966943447495,0.2014633377112483,proxy -direct,tls-init-close-1k,2,0.039258685439999996,3851.0,1.39,4.23,0,0,0,2.94,0.0,59.91030962039263,76.22109302084414,0.0,0.9519961977186312,0.2829643888354187,0.879277566539924,3.5797071030662435,0.8374755337159332,client -3proxy-current,tls-init-close-1k,2,0.009227468800000001,902.41,67.44,116.26,0,0,0,10.1,9.073845,875.6464177911932,1116.8859891628883,74.41015625,0.03851640513552068,0.957345971563981,0.29058749210360074,0.06752228756023755,0.2508956313242431,proxy +direct,tls-term-ka-1k,0,0.9432989696,92260.06,0.665,1.12,0,0,0,7.33,0.0,6.216480870838429,7.866174163320234,0.0,0.6291913214990138,0.7095837366892546,0.9559708584098828,1.927090841878114,0.8392698396127815,origin +3proxy-devel,tls-term-ka-1k,0,0.20401094656,19954.33,3.07,7.89,0,0,0,10.11,7.432369,39.64493149205258,50.166228353098795,14.7421875,0.25184304399524376,0.9601139601139601,0.3185161496642149,0.6991874125041462,0.2555947816036184,proxy +haproxy,tls-term-ka-1k,0,0.33277607936000003,32550.27,1.8,3.11,0,0,0,10.02,7.036519,24.088269852257685,30.740251014704395,37.1484375,0.27379830348727613,0.9524714828897338,0.40442781415653256,0.775291905522719,0.36011944890466585,proxy +nginx,tls-term-ka-1k,0,0.3414163456,33389.97,1.78,2.74,0,0,0,10.03,6.733183,23.50209679006066,29.992045977836387,34.94921875,0.2888154789995281,0.9525166191832859,0.4136634283923535,0.812413432243309,0.3650203770458889,proxy +envoy,tls-term-ka-1k,0,0.27338473472,26737.41,2.32,4.09,0,0,0,10.1,7.345412,29.555417599580007,37.40006591298745,22.74609375,0.27153736316039984,0.9591642924976258,0.36795440151994935,0.7629207873309807,0.3032087888519814,proxy +stunnel,tls-term-ka-1k,0,0.16642998272,16282.1,3.78,12.43,0,0,0,10.12,7.75016,48.645081058625244,61.53883574846913,16.23046875,0.21561161351737268,0.9619771863117871,0.28066263141127745,0.5812563023193552,0.2443092578206486,proxy +direct,tls-term-ka-1m,0,19.49915152384,2322.98,6.73,11.05,0,0,0,7.97,0.0,0.32698858677028037,339.68375740527637,0.0,0.7356239560963971,0.813265306122449,0.9575546404814698,2.476173005554468,0.8529648878713005,origin +3proxy-devel,tls-term-ka-1m,0,5.42440947712,646.39,23.53,55.27,0,0,0,10.02,7.486255999999999,1.477764544474611,1548.2076637824475,20.20703125,0.25281033245635015,0.9515669515669516,0.06204724409448819,0.7220149760631065,0.06271281714312446,proxy +haproxy,tls-term-ka-1m,0,4.7294133043199995,563.48,28.85,37.06,0,0,0,10.09,7.410545,1.7067656135332416,1772.974872605869,47.94140625,0.26380368098159507,0.9600380589914367,0.0591584941474217,0.7220079347290078,0.033269254791884945,proxy +nginx,tls-term-ka-1m,0,5.81674467328,693.24,22.12,32.09,0,0,0,10.01,7.468177000000001,1.3767150613960808,1442.3631123919308,48.2578125,0.25968436154949787,0.9524262607040913,0.06509433962264151,0.752396395476189,0.04847659691732909,proxy +envoy,tls-term-ka-1m,0,5.8040778752,691.44,22.41,35.79,0,0,0,10.01,7.548457,1.3797195992522853,1446.5317919075144,46.5625,0.25225439012814427,0.9515209125475285,0.06649858178380082,0.7429351786134916,0.046581616452105526,proxy +stunnel,tls-term-ka-1m,0,4.1498443776,494.44,30.92,84.04,0,0,0,10.04,7.829454,1.9354942665693853,2028.6926651848858,19.4921875,0.2041062801932367,0.9534662867996201,0.04946100190234623,0.5774295187816934,0.03517211745717436,proxy +direct,tls-term-close-1k,0,0.03145728,3087.8,1.78,4.9,0,0,0,2.54,0.0,64.59554036458334,82.02279846287983,0.0,0.9527091254752852,0.24423076923076922,0.9006664550936211,3.584391528635113,0.8564716246802538,origin +3proxy-devel,tls-term-close-1k,0,0.01241513984,1219.79,19.92,41.48,0,0,0,10.12,8.815469,652.107032569679,821.4285714285714,23.6484375,0.9548264384213029,0.9610636277302944,0.07516650808753568,3.6910616152080116,0.03232031755726188,client +haproxy,tls-term-close-1k,0,0.01224736768,1199.44,30.57,50.81,0,0,0,10.12,8.913373,661.0400056185787,835.3281056541477,42.41015625,0.7505355867650559,0.9610636277302944,0.06351222610352493,2.871992444901631,0.028517067016256066,client +nginx,tls-term-close-1k,0,0.01040187392,1024.81,4.03,5.29,0,0,0,10.12,8.833620999999999,778.3212969380039,977.6833156216791,34.234375,0.9581450653983353,0.9619771863117871,0.07577679137603044,3.7032775538284626,0.032316796774519076,client +envoy,tls-term-close-1k,0,0.01904214016,1872.39,24.79,37.26,0,0,0,9.99,8.37075,419.7007233875964,532.8284175156008,26.3359375,0.8705155618911855,0.950523311132255,0.09490667510281557,3.330418014767844,0.041824944417735765,client +stunnel,tls-term-close-1k,0,0.009227468800000001,903.02,22.46,51.81,0,0,0,10.13,8.976982,878.2473477450285,1110.6238351057998,34.3359375,0.9512601046124584,0.9629277566539924,0.06778587266392144,3.6925623055568226,0.028513045064273084,client +direct,tls-term-stream1-down,0,6.783526652335611,nan,nan,nan,0,0,0,1.12,0.0,0.13208213860636228,nan,0.0,0.4068665129387651,0.10980392156862745,0.36882003955174686,0.7877991910485717,0.7715536520324874,none +3proxy-devel,tls-term-stream1-down,0,5.262776090987485,nan,nan,nan,0,0,0,9.85,7.536672,1.4971816578238932,nan,8.015625,0.38275265628860883,0.9257518796992481,0.07125,0.7503795100068791,0.10799432329246171,proxy +haproxy,tls-term-stream1-down,0,6.112343512559425,nan,nan,nan,0,0,0,9.79,7.79017,1.2812064300822925,nan,40.96875,0.38974736325729703,0.9183864915572233,0.07653858169322086,0.7750593590707735,0.12019293163689658,proxy +nginx,tls-term-stream1-down,0,5.931179961755962,nan,nan,nan,0,0,0,9.79,7.651082,1.3203424511390753,nan,35.58203125,0.38706516413522785,0.9175257731958762,0.07176433719837041,0.7551686024054629,0.11461406856791828,proxy +envoy,tls-term-stream1-down,0,6.601938240534079,nan,nan,nan,0,0,51,9.35,7.677902,1.132928434323632,nan,33.484375,0.38018545632015616,0.8746492048643593,0.07149547299406805,0.7532599921061515,0.1305880549952942,none +stunnel,tls-term-stream1-down,0,5.112206699803147,nan,nan,nan,0,0,4,10.02,7.87134,1.5678767218328273,nan,7.61328125,0.3762644954354799,0.9417293233082706,0.06193306224585549,0.7323742166486193,0.0939037635440876,proxy +direct,tls-term-stream16,0,16.189406506042417,nan,nan,nan,0,0,449,1.84,0.0,0.09048365037545522,nan,0.0,0.5880281690140845,0.1727699530516432,0.926821608040201,1.6943030654309574,2.495587768870536,origin +3proxy-devel,tls-term-stream16,0,5.498303398582655,nan,nan,nan,0,0,255,10.05,7.440002,1.4572365402638408,nan,79.53515625,0.2265975005894836,0.9427767354596623,0.21279373368146215,0.637124348090757,0.3476513426658661,proxy +haproxy,tls-term-stream16,0,4.78856326565202,nan,nan,nan,0,0,379,10.04,7.588107000000001,1.676239602025297,nan,49.16796875,0.21227863046044865,0.9400749063670412,0.21428571428571427,0.5736924021728917,0.37621692329806006,proxy +nginx,tls-term-stream16,0,5.747591231298047,nan,nan,nan,0,0,168,10.12,7.5756049999999995,1.3951586103657325,nan,93.7265625,0.22637051039697542,0.9484536082474226,0.2144026186579378,0.6340051976388752,0.3512914393348748,proxy +envoy,tls-term-stream16,0,6.665260370341934,nan,nan,nan,0,0,436,10.09,8.901069999999999,1.2044401980835575,nan,132.4140625,0.2591369959915114,0.9465290806754222,0.08705148205928237,0.7180528038523158,0.15005205850380832,proxy +stunnel,tls-term-stream16,0,4.737174937751008,nan,nan,nan,0,0,224,10.06,7.887934,1.6948594233051786,nan,88.6171875,0.18702651515151514,0.9446009389671362,0.17488494411571334,0.5298015200391204,0.29820060230862616,proxy +direct,tls-init-ka-1k,0,0.97324630016,95190.89,0.609,1.12,0,0,0,7.72,0.0,6.345772903513402,8.09917927617257,0.0,0.6185643564356436,0.7509727626459144,0.9490022172949002,1.9210891704573578,0.835270770120478,origin +3proxy-devel,tls-init-ka-1k,0,0.23714594816,23195.71,2.54,11.98,0,0,0,10.1,7.376012,34.07184505024098,43.1108075806727,15.6015625,0.18319327731092436,0.9600760456273765,0.38293778062860806,0.45473484308340345,0.30894826610664666,proxy +haproxy,tls-init-ka-1k,0,0.31776047104000005,31082.96,1.93,3.18,0,0,0,10.03,7.039405,25.25172490378745,32.2291450440058,42.71875,0.2213231794383627,0.9516129032258065,0.4260355029585799,0.5498434737168942,0.3850016249369103,proxy +nginx,tls-init-ka-1k,0,0.31255953408,30570.98,1.96,3.14,0,0,0,10.1,6.808431000000001,25.851075136079242,32.71212680645433,38.265625,0.23767352800382957,0.9600760456273765,0.48128172588832485,0.588864108100041,0.45629824283106335,proxy +envoy,tls-init-ka-1k,0,0.29997662208,29339.74,2.12,3.68,0,0,0,10.02,7.2529129999999995,26.72208235567848,34.11342543731232,24.3046875,0.20966969842029679,0.9497630331753555,0.43020667726550077,0.5327364630345152,0.4097274363879263,proxy +stunnel,tls-init-ka-1k,0,0.18429771775999998,18025.81,3.3,13.14,0,0,0,10.12,7.747776,43.928921629637,55.58454398154505,18.3515625,0.16063294174058979,0.9619771863117871,0.3333333333333333,0.3881199282654151,0.3022825202934192,proxy +direct,tls-init-ka-1m,0,20.358144983040003,2424.8,6.56,10.89,0,0,0,8.09,0.0,0.31790715732654934,330.2849677472034,0.0,0.7418815663801337,0.8138832997987927,0.9597464342313787,2.4950566222836454,0.8561016612763134,origin +3proxy-devel,tls-init-ka-1m,0,5.21763028992,621.79,24.78,55.53,0,0,0,10.01,7.305898,1.534796364447429,1608.5489313835772,18.234375,0.18661108386463954,0.9515209125475285,0.28917558212712396,0.47941424412812983,0.3051091966802041,proxy +haproxy,tls-init-ka-1m,0,4.84559552512,577.33,26.34,37.58,0,0,0,10.12,7.546633,1.6707956654717904,1735.551363402504,50.06640625,0.20716049382716048,0.9601518026565465,0.25679090334807325,0.5108422421578771,0.2726915883044597,proxy +nginx,tls-init-ka-1m,0,4.686631403520001,558.53,26.99,39.46,0,0,0,10.0,7.402639000000001,1.7069829716054519,1789.5490336435219,46.67578125,0.18256833419288293,0.999000999000999,0.27695351137487634,0.4453863161567847,0.32628643046837874,proxy +envoy,tls-init-ka-1m,0,6.215958528,740.81,21.32,32.13,0,0,0,10.01,8.448684,1.2882968835663378,1345.6109692162925,48.98828125,0.12704432329936005,0.9533333333333334,0.31930926216640504,0.3957752734324336,0.3660257255032095,proxy +stunnel,tls-init-ka-1m,0,4.395211161600001,523.72,29.56,65.8,0,0,0,10.04,7.7572719999999995,1.827443484439116,1915.2995040061046,21.109375,0.15467714215565923,0.9543726235741445,0.23510872990860385,0.4033770768913009,0.21294925752198746,proxy +direct,tls-init-close-1k,0,0.0402653184,3951.09,1.44,4.63,0,0,0,2.96,0.0,58.809916178385414,74.6833526769945,0.0,0.9520417853751187,0.28516377649325625,0.888712745719721,3.5843472601022723,0.8161202135512972,client +3proxy-devel,tls-init-close-1k,0,0.00864026624,846.34,72.94,118.91,0,0,0,10.06,9.036869,931.4527789365898,1186.8806040585182,34.76953125,0.040152055119980996,0.9571836346336822,0.2846299810246679,0.06849818529302429,0.24146591432248135,proxy +haproxy,tls-init-close-1k,0,0.009479127039999998,931.86,65.2,94.5,0,0,0,10.05,9.000262000000001,848.179369901134,1076.247590490469,42.4609375,0.04183503684335631,0.956232159847764,0.3029728020240354,0.07230383355337092,0.2576277078522508,proxy +nginx,tls-init-close-1k,0,0.012331253759999999,1213.67,51.87,64.22,0,0,0,10.06,8.803065,652.6505866018283,827.7110416323844,34.97265625,0.0600664767331434,0.9562737642585551,0.3702302112898139,0.10274951297821974,0.3165987754612421,proxy +envoy,tls-init-close-1k,0,0.03036676096,2982.08,20.65,32.39,0,0,0,10.03,7.5244480000000005,264.23628158990846,335.9121202987374,27.05859375,0.09073679222932954,0.9525166191832859,0.32179161372299875,0.17980738156982373,0.23672427519203304,proxy +stunnel,tls-init-close-1k,0,0.009898557439999999,968.76,61.9,266.92,0,0,0,10.07,8.876617,813.8559632382152,1036.9683863659768,31.78125,0.047019710282593205,0.959047619047619,0.2644367308299148,0.07991876231528308,0.21676266771260233,proxy +direct,tls-init-stream1-down,0,7.51032531588814,nan,nan,nan,0,0,0,1.47,0.0,0.15657359394850548,nan,0.0,0.3944349524041982,0.13828786453433678,0.3574575204531152,0.8018173519591436,0.7570818855351054,none +3proxy-devel,tls-init-stream1-down,0,6.091370550384604,nan,nan,nan,0,0,0,9.99,7.412558000000001,1.311880811167123,nan,10.7890625,0.16613418530351437,0.950523311132255,0.29310344827586204,0.3710849233465248,0.6274470011467818,proxy +haproxy,tls-init-stream1-down,0,5.22049901865726,nan,nan,nan,0,0,0,9.98,7.435752,1.5291885729007673,nan,38.7890625,0.13622828784119106,0.9486692015209125,0.2842476740455566,0.31206799999373924,0.5968167494416051,proxy +nginx,tls-init-stream1-down,0,5.819827488827418,nan,nan,nan,0,0,0,10.02,7.378045,1.3772164949759493,nan,36.5703125,0.16601178781925344,0.9524714828897338,0.30380556443875917,0.3653853490708076,0.6493793133839354,proxy +envoy,tls-init-stream1-down,0,7.788301416058057,nan,nan,nan,0,0,0,9.98,8.656368,1.0250172640021944,nan,32.48046875,0.06501182033096926,0.9504761904761905,0.3600638977635783,0.1465382931598391,0.7673225486173552,proxy +stunnel,tls-init-stream1-down,0,4.702044255838254,nan,nan,nan,0,0,8,10.04,7.908345,1.7080480108315121,nan,10.13671875,0.1281165144408788,0.9534662867996201,0.23504956827630316,0.29207382522659653,0.5048360994072054,proxy +direct,tls-init-stream16,0,16.499858853486494,nan,nan,nan,0,0,357,1.8,0.0,0.086889656161025,nan,0.0,0.5853543677890275,0.16901408450704225,0.9235368156073002,1.682704799217022,8.10781805270257,origin +3proxy-devel,tls-init-stream16,0,6.842659828981578,nan,nan,nan,0,0,338,10.07,7.837721,1.1716229439974304,nan,94.38671875,0.05892135899263483,0.9572243346007605,0.5597589554737195,0.13211400166511136,0.8271743928075266,proxy +haproxy,tls-init-stream16,0,5.862325867743212,nan,nan,nan,0,0,10,10.03,7.438014,1.3676024439439092,nan,54.84375,0.06178707224334601,0.9525166191832859,0.7143330023171135,0.11787160954103942,1.0741883155495973,origin +nginx,tls-init-stream16,0,6.8431736274537,nan,nan,nan,0,0,422,10.11,7.917688,1.1710265790814582,nan,114.35546875,0.056769596199524944,0.9601139601139601,0.5375703409467064,0.12737086928086372,0.7633546910784156,proxy +envoy,tls-init-stream16,0,7.334330547432625,nan,nan,nan,0,0,5705,10.07,7.92617,1.0932946682844875,nan,148.0234375,0.06285578747628083,0.956315289648623,0.5339425587467362,0.14637366469341512,0.8619641543333603,proxy +stunnel,tls-init-stream16,0,5.626352686394712,nan,nan,nan,0,0,283,10.08,8.019518,1.427484442866689,nan,102.74609375,0.04898929845422117,0.9572649572649573,0.5078464106844741,0.10932185788223023,0.6904925012520765,proxy +3proxy-devel,tls-term-ka-1k,1,0.24058527744,23533.73,2.51,5.95,0,0,0,10.12,7.394785,33.651269463149404,42.576286018402044,14.36328125,0.2523386903334133,0.9610636277302944,0.33365384615384613,0.7163582660819358,0.2728282563404937,proxy +haproxy,tls-term-ka-1k,1,0.33185333248000004,32458.11,1.84,3.11,0,0,0,10.02,7.041486,24.155249369035957,30.825173276236008,37.1640625,0.2658317580340265,0.9524714828897338,0.3942337825133187,0.7515882214030223,0.3156306999875652,proxy +nginx,tls-term-ka-1k,1,0.3686793216,36056.55,1.65,2.6,0,0,0,10.02,6.6448030000000005,21.742472469603243,27.77238836996591,37.1015625,0.2921908378827439,0.9515669515669516,0.4232469993682881,0.8352639706876831,0.38311186136133746,proxy +envoy,tls-term-ka-1k,1,0.27749515263999996,27141.48,2.31,4.03,0,0,0,10.07,7.345847,29.031137745498604,36.73546814921823,23.97265625,0.25587529976019185,0.9581351094196003,0.35412026726057905,0.7325621669718656,0.3204048794971974,proxy +stunnel,tls-term-ka-1k,1,0.20073938944,19637.33,2.95,12.2,0,0,0,10.12,7.700559,40.330898796620346,51.025295840833344,16.22265625,0.21466346153846153,0.9619771863117871,0.2859897172236504,0.6136469613029831,0.22911792815524312,proxy +direct,tls-term-ka-1k,1,1.0461433036799999,102325.03,0.595,1.05,0,0,0,7.36,0.0,5.628292012468929,7.121721443929022,0.0,0.638909541511772,0.7104247104247104,0.9563567362428842,1.9680999514500503,0.8508707648756154,origin +3proxy-devel,tls-term-ka-1m,1,6.1052289023999995,727.53,21.3,49.18,0,0,0,10.01,7.450235,1.311662531908019,1374.6223564954682,18.7421875,0.2612872238232469,0.9515209125475285,0.06255924170616113,0.7590859466090741,0.051330857901977084,proxy +haproxy,tls-term-ka-1m,1,5.480361492479999,653.02,23.66,34.7,0,0,0,10.1,7.325695,1.474355297745806,1531.4632297194844,48.7890625,0.2999752903385224,0.9600760456273765,0.06409851594568992,0.8247263038649035,0.03707373395550695,proxy +nginx,tls-term-ka-1m,1,5.654676766720001,673.85,22.58,35.56,0,0,0,10.02,7.461974,1.4175876589758971,1485.7651245551601,47.93359375,0.2518625330449411,0.9524714828897338,0.06200818382121498,0.7276811631277783,0.04752599678496451,proxy +envoy,tls-term-ka-1m,1,5.73512351744,683.39,22.52,36.18,0,0,0,10.01,7.545095,1.3963082007995793,1463.8783269961978,46.14453125,0.24554737592020898,0.9524262607040913,0.06186868686868687,0.7249132469004583,0.05418122027883975,proxy +stunnel,tls-term-ka-1m,1,4.50921234432,537.25,27.95,76.75,0,0,0,10.05,7.799951,1.783016497355139,1869.419642857143,20.16796875,0.21642512077294687,0.9535104364326376,0.05439595192915876,0.6163941318855918,0.03230501831404746,proxy +direct,tls-term-ka-1m,1,21.131239096319998,2515.02,6.18,11.69,0,0,0,8.22,0.0,0.3111980310300501,323.5838286816518,0.0,0.7578571428571429,0.8311425682507584,0.9588216661387393,2.5521928184605907,0.8548553913300087,origin +3proxy-devel,tls-term-close-1k,1,0.01241513984,1223.08,19.6,42.72,0,0,0,10.1,8.805891,650.8182834934544,817.3504896010359,20.01171875,0.9517814726840855,0.9609895337773549,0.07419150285351934,3.6931023490886203,0.03230757767299401,client +haproxy,tls-term-close-1k,1,0.01350565888,1326.3,28.11,50.66,0,0,0,10.11,8.914606,598.8600831594525,754.533920441824,38.5,0.7743162901307967,0.9601139601139601,0.06571428571428571,2.9645510598126523,0.030401723835461417,client +nginx,tls-term-close-1k,1,0.013841203199999999,1359.44,3.15,4.74,0,0,0,10.13,8.756054,585.498231830019,737.7467045371786,32.37890625,0.9581749049429658,0.9620132953466287,0.08507936507936507,3.7055384378513323,0.03612375334975298,client +envoy,tls-term-close-1k,1,0.022481469440000002,2208.11,21.02,31.49,0,0,0,10.0,8.389932,355.84862552472015,451.9569736961041,27.546875,0.8848441589340947,0.9505703422053232,0.0922930542340628,3.3868945881353416,0.041831216601638604,client +stunnel,tls-term-close-1k,1,0.01040187392,1023.22,22.04,53.28,0,0,0,10.06,8.91971,773.7067437941029,981.6549570647932,32.15625,0.9448538150701212,0.9562737642585551,0.061010486177311724,3.6922447212409866,0.027569361636001734,client +direct,tls-term-close-1k,1,0.03388997632,3325.48,1.51,4.39,0,0,0,2.66,0.0,62.79142776338181,79.70992778160678,0.0,0.952007602755999,0.2550335570469799,0.8679544015199493,3.6044816494783274,0.84014239731083,client +3proxy-devel,tls-term-stream1-down,1,6.777909340164613,nan,nan,nan,0,0,0,9.55,7.609268,1.1270723288902251,nan,8.5,0.39004658004412845,0.895032802249297,0.07538317172349077,0.764707997581304,0.12218539457260721,none +haproxy,tls-term-stream1-down,1,6.511348322089533,nan,nan,nan,0,0,0,9.79,7.76732,1.2027660621729064,nan,38.328125,0.3940511307767945,0.9175257731958762,0.07142857142857142,0.7846908957395419,0.11176733864119884,proxy +nginx,tls-term-stream1-down,1,6.388193022225373,nan,nan,nan,0,0,0,9.96,7.455176,1.2472715109193422,nan,35.66015625,0.398369967893307,0.9369708372530574,0.07478097622027534,0.7523181694608292,0.11835996641196735,proxy +envoy,tls-term-stream1-down,1,6.878771710384555,nan,nan,nan,0,0,37,9.33,7.673685,1.084962976718731,nan,32.8671875,0.3852178169358786,0.8744142455482662,0.07211988760536997,0.7818843898862599,0.13058873653378722,none +stunnel,tls-term-stream1-down,1,5.477174966278608,nan,nan,nan,0,0,3,9.95,7.847276,1.4531484606802736,nan,7.87890625,0.3928202479338843,0.9822309970384995,0.07056120774532326,0.7170010577723507,0.11234308108424469,proxy +direct,tls-term-stream1-down,1,7.7124235050519285,nan,nan,nan,0,0,0,1.43,0.0,0.14831678217060454,nan,0.0,0.41544401544401544,0.14102564102564102,0.37532981530343007,0.8039674922719308,0.7897114282891569,none +3proxy-devel,tls-term-stream16,1,6.399134395557837,nan,nan,nan,0,0,286,10.06,7.417442,1.2537646239504552,nan,76.8671875,0.23665564478034956,0.9428303655107779,0.2221487603305785,0.66607419779506,0.36788902552055647,proxy +haproxy,tls-term-stream16,1,5.25119474979544,nan,nan,nan,0,0,507,10.04,7.545478,1.528894055882118,nan,43.83203125,0.21996695775312722,0.9409559512652296,0.20805592543275633,0.5918508233665358,0.3714092144411958,proxy +nginx,tls-term-stream16,1,5.685491239876813,nan,nan,nan,0,0,214,10.09,7.545533,1.4079926623315013,nan,85.4921875,0.2238241550460884,0.9483082706766918,0.21896383186705767,0.621425753292232,0.3547280028887528,proxy +envoy,tls-term-stream16,1,6.811969600638648,nan,nan,nan,0,0,432,10.1,8.923905999999999,1.1789248304156084,nan,139.29296875,0.26030624263839813,0.9465791940018744,0.08502657080337606,0.7219528311862721,0.14917633696485025,proxy +stunnel,tls-term-stream16,1,4.482254556558468,nan,nan,nan,0,0,275,10.07,7.845625,1.792366623033023,nan,81.6875,0.18577168518080833,0.9446529080675422,0.17371353654539495,0.5233774799132344,0.29346149200910193,proxy +direct,tls-term-stream16,1,16.015143423796545,nan,nan,nan,0,0,396,1.89,0.0,0.09412058291327519,nan,0.0,0.5851938895417156,0.17746478873239438,0.9249371859296482,1.683811797303392,1.7502437122495316,origin +3proxy-devel,tls-init-ka-1k,1,0.23379050496,22864.77,2.6,14.44,0,0,0,10.1,7.405217,34.56085610227171,43.73525017862169,15.765625,0.185789726356217,0.9600760456273765,0.3856082607292675,0.4623346985270433,0.3127445963278598,proxy +haproxy,tls-init-ka-1k,1,0.30685528064,30009.61,1.98,4.1,0,0,0,10.11,7.101384,26.357701855842503,33.35510818140428,40.796875,0.22169249106078665,0.9601139601139601,0.429821819318537,0.5488927974373623,0.36310268041072635,proxy +nginx,tls-init-ka-1k,1,0.32690405376,31971.09,1.84,3.09,0,0,0,10.11,6.791938,24.74120435942311,31.308936518111803,36.09765625,0.2318562874251497,0.9601139601139601,0.47014218009478675,0.5793548719560367,0.4591288786848607,proxy +envoy,tls-init-ka-1k,1,0.3112173568,30438.3,2.01,3.87,0,0,0,10.08,7.339404999999999,25.911151238207548,32.7896582458834,23.56640625,0.2147297943567671,0.9590865842055185,0.44992050874403816,0.546028495545099,0.42755951301028444,proxy +stunnel,tls-init-ka-1k,1,0.18975031296,18562.92,3.17,12.61,0,0,0,10.12,7.720931,42.666596295452024,53.97477266060428,16.71484375,0.15832335329341318,0.9619771863117871,0.3290570878768441,0.3852942957659553,0.2918545363042167,proxy +direct,tls-init-ka-1k,1,0.98977185792,96806.69,0.582,1.31,0,0,0,7.29,0.0,5.89226694347111,7.455947012660792,0.0,0.632112815437902,0.7063953488372093,0.9578446909667195,1.935960064564296,0.8469917496875562,origin +3proxy-devel,tls-init-ka-1m,1,5.01311602688,597.33,25.49,59.61,0,0,0,10.01,7.292004,1.5974096663754893,1674.4730679156908,19.8046875,0.18762230919765166,0.9515209125475285,0.27851291745431633,0.4755833541381246,0.25281457203352964,proxy +haproxy,tls-init-ka-1m,1,4.639403540479999,552.38,27.07,40.96,0,0,0,10.09,7.514806,1.7398788291576075,1808.5678436995877,50.35546875,0.1948629291183008,0.9600380589914367,0.24456350456980774,0.48605896317095715,0.29179928444947434,proxy +nginx,tls-init-ka-1m,1,5.34404661248,636.87,24.15,37.14,0,0,0,10.02,7.348424,1.499986916521305,1572.0112958895513,44.85546875,0.19219440353460973,0.9515669515669516,0.2951181102362205,0.49089435273934007,0.27568115684768685,proxy +envoy,tls-init-ka-1m,1,6.35252506624,757.1,20.76,32.78,0,0,0,10.03,8.454549,1.2631197699073906,1320.9535098116687,48.328125,0.1295287710158655,0.9525166191832859,0.31798245614035087,0.40147456427613315,0.284247782539958,proxy +stunnel,tls-init-ka-1m,1,4.412072263680001,525.71,29.48,64.35,0,0,0,10.03,7.735322,1.8186465498430844,1906.8441064638782,20.3203125,0.16351583599312547,0.9534220532319392,0.2371491643014822,0.41763576489220217,0.25097826572548027,proxy +direct,tls-init-ka-1m,1,22.3338299392,2660.21,5.84,10.38,0,0,0,8.29,0.0,0.2969486209062429,308.556965794469,0.0,0.80732635585157,0.8382204246713852,0.9585443037974684,2.7559459621246556,0.83378802605234,origin +3proxy-devel,tls-init-close-1k,1,0.00661086208,649.15,92.14,167.06,0,0,0,10.21,9.229690999999999,1235.5423394341938,1570.7692307692307,245.3125,0.03233475986685687,0.9677725118483412,0.23042929292929293,0.05327597843602479,0.19478849580430974,proxy +haproxy,tls-init-close-1k,1,0.009814671359999999,964.15,64.22,89.94,0,0,0,10.05,8.996122999999999,819.1817846053688,1040.588113481052,40.48828125,0.042972459639126305,0.9535104364326376,0.3013003488740882,0.0723029348335669,0.25658274132482833,proxy +nginx,tls-init-close-1k,1,0.01199570944,1179.71,52.05,72.02,0,0,0,10.06,8.821459,670.9065470662151,851.748370163407,31.71484375,0.05767861381438405,0.9562737642585551,0.3602523659305994,0.09799210874810978,0.3051810942577959,proxy +envoy,tls-init-close-1k,1,0.02701131776,2648.78,25.85,33.13,0,0,0,10.0,7.53495,296.1721479522515,376.9601930036188,25.37109375,0.08483412322274882,0.9514747859181731,0.3000952683391553,0.17028274795259593,0.22149084640344327,proxy +stunnel,tls-init-close-1k,1,0.00956301312,940.82,64.09,134.51,0,0,0,10.08,8.880436000000001,843.2488692434213,1069.609507640068,34.0625,0.044016178919819175,0.9581749049429658,0.2582886011998737,0.0780159399501272,0.21478690327275426,proxy +direct,tls-init-close-1k,1,0.0339738624,3336.8,1.48,4.44,0,0,0,2.58,0.0,60.75258608217593,77.12082262210797,0.0,0.9515669515669516,0.2473633748801534,0.8701298701298701,3.596700224334168,0.8504363119407453,client +3proxy-devel,tls-init-stream1-down,1,6.133944966178552,nan,nan,nan,0,0,0,10.0,7.496550999999999,1.3040808374213728,nan,10.62890625,0.16373840897999023,0.9514747859181731,0.29824,0.3749050152067912,0.6428098887296081,proxy +haproxy,tls-init-stream1-down,1,4.9296750490498304,nan,nan,nan,0,0,0,10.02,7.670416,1.6258992982076013,nan,39.8046875,0.14485637122514117,0.9515669515669516,0.25143769968051116,0.31400832008877133,0.5391051676669245,proxy +nginx,tls-init-stream1-down,1,5.960508081584328,nan,nan,nan,0,0,0,10.01,7.2652589999999995,1.3434820873922508,nan,33.0859375,0.1706837186424004,0.9515209125475285,0.3100899742930591,0.3787020849981426,0.6646366554210981,proxy +envoy,tls-init-stream1-down,1,7.391248288658627,nan,nan,nan,0,0,0,9.94,8.38443,1.0757515552744128,nan,30.41796875,0.0689410092395167,0.9466666666666667,0.35768985322271857,0.1560433694609397,0.7397200738212938,proxy +stunnel,tls-init-stream1-down,1,4.647629931634322,nan,nan,nan,0,0,7,10.03,7.7758970000000005,1.7262864814211671,nan,6.1796875,0.13894324853228962,0.9525166191832859,0.2464833759590793,0.3120326945206376,0.5190391453461016,proxy +direct,tls-init-stream1-down,1,7.424191430329656,nan,nan,nan,0,0,2,1.52,0.0,0.16378334422004096,nan,0.0,0.4142270159219312,0.15004935834155972,0.3711169861202908,0.806835386562984,0.7853450250871016,none +3proxy-devel,tls-init-stream16,1,7.0580551345748965,nan,nan,nan,0,0,361,10.06,7.850505999999999,1.1359180230934023,nan,99.46484375,0.05827783063748811,0.9553656220322887,0.5584329349269588,0.1321325179278397,0.8063560646592222,proxy +haproxy,tls-init-stream16,1,4.817428192038526,nan,nan,nan,0,0,249,10.01,7.571764000000001,1.661378334010888,nan,55.43359375,0.05576649264356905,0.9515209125475285,0.606425702811245,0.09979419689304676,0.8640512357873212,proxy +nginx,tls-init-stream16,1,6.734199711767152,nan,nan,nan,0,0,363,10.11,7.896966,1.1888774289205675,nan,114.54296875,0.055687767729652546,0.961941008563273,0.529431216931217,0.12547183040801566,0.7332136585318135,proxy +envoy,tls-init-stream16,1,7.226866948849155,nan,nan,nan,0,0,5520,10.07,7.894514,1.1100708413478906,nan,146.3984375,0.061804404451811505,0.956315289648623,0.516590761223162,0.1397480801155369,0.8206870463097911,proxy +stunnel,tls-init-stream16,1,5.672833873410656,nan,nan,nan,0,0,262,10.09,8.017217,1.41722064178196,nan,96.6328125,0.04966730038022814,0.9573055028462998,0.5086493679308051,0.11122284289756777,0.6924045799893166,proxy +direct,tls-init-stream16,1,16.669870786761685,nan,nan,nan,0,0,351,1.8,0.0,0.08614673932646646,nan,0.0,0.5733051841426225,0.16901408450704225,0.9123857548061771,1.6547693509923322,1.730440452660327,origin +haproxy,tls-term-ka-1k,2,0.329252864,32202.84,1.82,3.19,0,0,0,10.02,7.0114920000000005,24.346029682523884,31.0670701458478,37.1640625,0.2688222799150342,0.9524714828897338,0.39617314930991215,0.7610692311183535,0.30325132758570317,proxy +nginx,tls-term-ka-1k,2,0.3372220416,32984.98,1.8,2.74,0,0,0,10.03,6.7356940000000005,23.79441142675295,30.341439509212847,35.05078125,0.28277330809275913,0.9525166191832859,0.4054820415879017,0.80005281595184,0.3897619751259341,proxy +envoy,tls-term-ka-1k,2,0.3112173568,30436.36,1.95,3.75,0,0,0,10.09,7.336266,25.93685674538831,32.82464353217889,24.3828125,0.2793628150261531,0.9591254752851711,0.3863204559848005,0.7924550687823478,0.3202520254040063,proxy +stunnel,tls-term-ka-1k,2,0.1862270976,18211.81,3.22,11.18,0,0,0,10.13,7.726447,43.51676047385276,55.07559982384507,16.23046875,0.21734985700667303,0.9620132953466287,0.28917969996808174,0.5878981119594319,0.2661752418622385,proxy +direct,tls-term-ka-1k,2,0.97106526208,94977.36,0.622,1.03,0,0,0,7.45,0.0,6.137589544943471,7.766451846956072,0.0,0.6391089108910891,0.7233009708737864,0.9588347055098163,1.9585169070902544,0.8379191384919805,origin +3proxy-devel,tls-term-ka-1k,2,0.22984785919999998,22480.59,2.67,6.03,0,0,0,10.1,7.388892,35.15368830548586,44.4835939220436,14.37109375,0.24719101123595505,0.9600760456273765,0.3267931810871663,0.683053267403589,0.3023032110186354,proxy +haproxy,tls-term-ka-1m,2,5.78419687424,688.76,22.31,35.24,0,0,0,10.11,7.352988,1.3982926542524892,1453.421506612996,49.4921875,0.29149198520345254,0.9601139601139601,0.06719242902208201,0.799161700451357,0.038978444116790405,proxy +nginx,tls-term-ka-1m,2,5.92797761536,706.47,21.82,33.15,0,0,0,10.01,7.509201999999999,1.350882294030674,1415.6413520011313,48.171875,0.2593480345158197,0.9515209125475285,0.06090249289996844,0.7562664402028803,0.043728221841974155,proxy +envoy,tls-term-ka-1m,2,6.31133700096,751.94,21.0,31.48,0,0,0,10.03,7.5293719999999995,1.2713629455659698,1332.890365448505,45.64453125,0.2587594696969697,0.9516129032258065,0.06614173228346457,0.7676486679977432,0.0456090921779366,proxy +stunnel,tls-term-ka-1m,2,4.83452256256,575.93,26.25,69.91,0,0,0,10.02,7.787783,1.658074793585269,1738.3761276891048,20.01171875,0.22013034033309195,0.9533777354900095,0.05203045685279188,0.6307144379513957,0.0351734766358953,proxy +direct,tls-term-ka-1m,2,19.49915152384,2325.01,6.72,13.14,0,0,0,8.12,0.0,0.3331427006994575,345.81150717601463,0.0,0.7318409145034532,0.8193743693239153,0.9537974683544304,2.4733511309377647,0.8453802202803354,origin +3proxy-devel,tls-term-ka-1m,2,5.89668810752,702.67,21.49,52.53,0,0,0,10.02,7.462343,1.359407154293485,1424.7120716621641,21.1953125,0.25538535184298705,0.9515669515669516,0.061198738170347,0.7343611511208081,0.05132466972602156,proxy +haproxy,tls-term-close-1k,2,0.013170114560000001,1290.81,28.57,53.51,0,0,0,10.09,8.915245,612.9027931553542,773.9510623609726,38.18359375,0.7568403521294313,0.960952380952381,0.06630710659898477,2.8871627857227327,0.030418026958914324,client +nginx,tls-term-close-1k,2,0.01308622848,1282.22,3.27,5.44,0,0,0,10.12,8.743829,618.6656462840544,781.4671814671815,34.26171875,0.9538863798431186,0.9610636277302944,0.091426763682379,3.6960002972247388,0.03992728533245066,client +envoy,tls-term-close-1k,2,0.01996488704,1958.38,22.97,33.36,0,0,0,10.0,8.365652,400.7034942883404,509.7879282218597,24.765625,0.9170233000475512,0.9514747859181731,0.09838146620120597,3.5248328330071175,0.04468149465514618,client +stunnel,tls-term-close-1k,2,0.00998244352,981.56,21.22,53.08,0,0,0,10.13,8.965724,811.8252794281776,1021.7873713939883,39.875,0.9524714828897338,0.9629277566539924,0.06414734836456018,3.7055428924816685,0.027568887986306533,client +direct,tls-term-close-1k,2,0.03632267264,3569.56,1.73,5.33,0,0,0,2.73,0.0,60.127734036698904,76.04880494735083,0.0,0.9534331195058209,0.2619961612284069,0.9041574103459219,3.588052577307687,0.8343079303115016,origin +3proxy-devel,tls-term-close-1k,2,0.00931135488,915.19,25.74,51.8,0,0,0,10.09,8.895892,866.8985452737893,1091.28271685053,62.00390625,0.9510107015457788,0.9591254752851711,0.06253968253968253,3.704490114385741,0.026617144126606217,client +haproxy,tls-term-stream1-down,2,5.302554843793358,nan,nan,nan,0,0,0,9.85,7.7463049999999996,1.4860203700305017,nan,38.76171875,0.368986568986569,0.9257518796992481,0.0699344364658133,0.7323634714766126,0.10888657700448262,proxy +nginx,tls-term-stream1-down,2,5.25909138117809,nan,nan,nan,0,0,0,9.84,7.618736999999999,1.4968246872196194,nan,36.03515625,0.38342351205115593,0.924812030075188,0.06887914840325611,0.7303719745619792,0.10706374951805704,proxy +envoy,tls-term-stream1-down,2,6.917778773006712,nan,nan,nan,0,0,51,9.12,7.569824,1.0545631725522884,nan,32.0078125,0.3991801178580579,0.8941176470588236,0.07730101539469375,0.7569942127826226,0.13990350085749714,none +stunnel,tls-term-stream1-down,2,4.724236505582047,nan,nan,nan,0,0,8,9.99,7.856866,1.6902324388971128,nan,6.9921875,0.3659135559921415,0.9389097744360902,0.06924516531503432,0.6971319212187429,0.09958308780794305,proxy +direct,tls-term-stream1-down,2,7.12681696258761,nan,nan,nan,0,0,0,1.68,0.0,0.18857134384516924,nan,0.0,0.4154043645699615,0.16535433070866143,0.38365194462755436,0.798939947914091,0.795692847977105,none +3proxy-devel,tls-term-stream1-down,2,5.589366736662486,nan,nan,nan,0,0,0,9.67,7.574899,1.3840398671266767,nan,8.40625,0.4052590873936582,0.9517716535433071,0.08120497707924033,0.7498378177438761,0.12415641967975921,proxy +haproxy,tls-term-stream16,2,4.96613614028804,nan,nan,nan,0,0,466,10.03,7.534289,1.614712226972157,nan,43.26171875,0.21573724007561437,0.9409005628517824,0.21319120586275817,0.5794262362632789,0.37798477295436833,proxy +nginx,tls-term-stream16,2,6.108448277671506,nan,nan,nan,0,0,221,10.1,7.5052389999999995,1.3145825403146698,nan,80.41015625,0.2256689557186834,0.9465791940018744,0.21903820816864294,0.6280822074124501,0.35563504646872457,proxy +envoy,tls-term-stream16,2,7.257016197788897,nan,nan,nan,0,0,535,10.07,8.925799999999999,1.106027711582965,nan,138.19140625,0.26298394711992445,0.9455399061032864,0.08320300281513919,0.732572825428969,0.15015417969546402,proxy +stunnel,tls-term-stream16,2,4.93465688449111,nan,nan,nan,0,0,250,10.08,7.863481,1.6297426649783842,nan,94.40625,0.1862884160756501,0.9447047797563262,0.18262586377097728,0.5232369679199749,0.3093901908498831,proxy +direct,tls-term-stream16,2,17.61709791981296,nan,nan,nan,0,0,448,1.95,0.0,0.08829426836049009,nan,0.0,0.6038488617695377,0.18327067669172933,0.9182903834066625,1.742694135125302,1.7426340139672905,origin +3proxy-devel,tls-term-stream16,2,5.936625616234798,nan,nan,nan,0,0,291,10.06,7.430306,1.3509057288458155,nan,94.56640625,0.22995031937544358,0.9428303655107779,0.21864740643466843,0.6468909149467797,0.3610874148838743,proxy +haproxy,tls-init-ka-1k,2,0.33151778816000005,32428.72,1.86,6.39,0,0,0,10.11,7.107836000000001,24.3968809181862,30.868249669486843,37.9140625,0.23448275862068965,0.9601139601139601,0.4529008109794136,0.5812056025910589,0.39825676089187356,proxy +nginx,tls-init-ka-1k,2,0.30643585024000003,29971.31,2.08,3.03,0,0,0,10.11,6.847041,26.393778644585783,33.39797101545028,36.0,0.22458233890214796,0.9610266159695817,0.4601381042059008,0.5564684768960839,0.3744680837178592,proxy +envoy,tls-init-ka-1k,2,0.28596764672,27967.51,2.22,4.0,0,0,0,10.06,7.35279,28.143043775438176,35.614905864776645,22.72265625,0.20828374196619853,0.9590085795996187,0.42987997473152245,0.5156140540364486,0.3679052006028204,proxy +stunnel,tls-init-ka-1k,2,0.1879048192,18378.01,3.2,13.32,0,0,0,10.13,7.726971,43.128217969621936,54.57562468348293,16.4296875,0.15950334288443171,0.9620132953466287,0.33110687022900764,0.3786416986892288,0.2869732693516453,proxy +direct,tls-init-ka-1k,2,0.94262788096,92194.83,0.617,1.51,0,0,0,7.1,0.0,6.025707614562938,7.624971674841137,0.0,0.6228698444060262,0.6873184898354308,0.9585180493983534,1.8976348522690194,0.8397667601400569,origin +3proxy-devel,tls-init-ka-1k,2,0.23655874559999998,23138.93,2.57,11.49,0,0,0,10.1,7.3944469999999995,34.15642055213874,43.218170459310734,15.703125,0.18974604695735506,0.9591642924976258,0.39451880178457616,0.46614134967155574,0.36408252675269875,proxy +haproxy,tls-init-ka-1m,2,4.91756978176,585.68,26.58,37.15,0,0,0,10.1,7.541793,1.643088020828891,1707.5232459847844,50.94140625,0.19707848477345877,0.9609895337773549,0.25701671397035636,0.4966247525837884,0.23482604770459523,proxy +nginx,tls-init-ka-1m,2,5.49386715136,654.75,23.46,35.21,0,0,0,10.02,7.363644,1.4590815138323192,1529.070654661987,45.5390625,0.1945057640421879,0.9524714828897338,0.2866541353383459,0.49564436941189216,0.2956667776866136,proxy +envoy,tls-init-ka-1m,2,6.6190311424,788.86,20.2,28.69,0,0,0,10.02,8.465634,1.2110533743603857,1268.6756140795137,40.92578125,0.13119810201660737,0.9515669515669516,0.32610744580584355,0.4157245533238602,0.4239715728250539,proxy +stunnel,tls-init-ka-1m,2,4.427339530239999,527.6,29.33,64.28,0,0,0,10.03,7.752134,1.812375117199342,1899.621212121212,20.8203125,0.1608237313066928,0.9525166191832859,0.23768939393939395,0.41280827049799834,0.28608684313989863,proxy +direct,tls-init-ka-1m,2,20.70174236672,2466.35,6.09,13.26,0,0,0,8.03,0.0,0.31031204457104944,322.3604977920513,0.0,0.7590073968026724,0.8135764944275583,0.9433185560481318,2.58756715545147,0.8647377835930178,origin +3proxy-devel,tls-init-ka-1m,2,5.44118669312,648.4,23.92,53.52,0,0,0,10.0,7.283727,1.4702675080264092,1541.0695022345508,17.33984375,0.18867924528301888,0.9514747859181731,0.2910753705455692,0.4860933020282822,0.39922826177484244,proxy +haproxy,tls-init-close-1k,2,0.009898557439999999,971.68,63.22,87.65,0,0,0,10.06,8.991831,813.0477646649896,1033.066338057096,40.4140625,0.04598246029864897,0.9553656220322887,0.3114909781576448,0.07420891244025199,0.26240220709934997,proxy +nginx,tls-init-close-1k,2,0.01249902592,1226.33,50.61,73.25,0,0,0,10.05,8.809909,643.2501261666317,818.6705767350928,34.82421875,0.06105063890203502,0.956232159847764,0.36561514195583594,0.1008433222599802,0.31656271034531436,proxy +envoy,tls-init-close-1k,2,0.033554432,3298.66,19.22,28.01,0,0,0,10.03,7.52569,239.13383483886716,302.8015940103852,27.10546875,0.09157769869513642,0.9534220532319392,0.34790874524714827,0.19218400391023552,0.2614521204420928,proxy +stunnel,tls-init-close-1k,2,0.00931135488,917.07,64.49,282.03,0,0,0,10.07,8.893132999999999,865.1802131721564,1096.47212543554,42.8046875,0.04510921177587844,0.9572243346007605,0.25410872313527183,0.07611102140043158,0.21010458273517538,proxy +direct,tls-init-close-1k,2,0.032380026879999996,3182.26,1.71,4.86,0,0,0,2.59,0.0,63.99006423554891,81.09716003381658,0.0,0.9524827750059397,0.2483221476510067,0.891566265060241,3.594582602385835,0.868531410399534,client +3proxy-devel,tls-init-close-1k,2,0.008724152320000001,854.51,70.94,130.96,0,0,0,10.03,9.019395999999999,919.7455186110276,1170.7715653087428,59.45703125,0.03983874792506521,0.9525166191832859,0.2871161760050649,0.06659426008973492,0.247076553129968,proxy +haproxy,tls-init-stream1-down,2,5.30983748687016,nan,nan,nan,0,0,0,10.0,7.443982,1.5064755017869838,nan,35.140625,0.15938430983118174,0.9514747859181731,0.28630838131797826,0.3643463829261043,0.6102902686360547,proxy +nginx,tls-init-stream1-down,2,6.122829390865141,nan,nan,nan,0,0,0,10.0,7.421562,1.3064479145261825,nan,34.765625,0.16845180136319376,0.9487666034155597,0.3047923322683706,0.38059470645108456,0.6493674048897675,proxy +envoy,tls-init-stream1-down,2,7.999160223164411,nan,nan,nan,0,0,0,9.97,8.580546,0.9969992853509123,nan,33.28515625,0.06983901515151515,0.9477186311787072,0.36058767167039285,0.15890472448885026,0.7578210005940405,proxy +stunnel,tls-init-stream1-down,2,4.99521013590384,nan,nan,nan,0,0,11,10.02,7.927848999999999,1.6045700985060765,nan,10.63671875,0.15476480623933708,0.9515669515669516,0.2399103139013453,0.34256203830421034,0.520834406924631,proxy +direct,tls-init-stream1-down,2,7.575648412068668,nan,nan,nan,0,0,1,1.33,0.0,0.1404461615601946,nan,0.0,0.4143334189571025,0.1310344827586207,0.3648380700594845,0.8089368670720193,0.770897273935217,none +3proxy-devel,tls-init-stream1-down,2,6.5616980626696195,nan,nan,nan,0,0,0,9.96,7.428655,1.214188443232296,nan,11.21484375,0.1782250242483026,0.9485714285714286,0.304694985627595,0.39666298169177927,0.6558373533562405,proxy +haproxy,tls-init-stream16,2,6.1118228658999,nan,nan,nan,0,0,7,10.04,7.414294,1.313045255905343,nan,55.0859375,0.06579572446555819,0.9516587677725118,0.7479379742659188,0.1273587095064298,1.1262821310115851,origin +nginx,tls-init-stream16,2,6.87073422596113,nan,nan,nan,0,0,487,10.11,7.865257,1.1658229604748571,nan,112.50390625,0.05589914367269267,0.9610266159695817,0.5361405835543767,0.1273765617651751,0.7455949793916397,proxy +envoy,tls-init-stream16,2,7.250423926236643,nan,nan,nan,0,0,4922,10.09,7.947361000000001,1.1084477936372372,nan,153.49609375,0.06331515295233578,0.9563981042654028,0.5267275097783573,0.14637536633339554,0.8405408179005287,proxy +stunnel,tls-init-stream16,2,5.505053569840967,nan,nan,nan,0,0,270,10.1,7.992601,1.4612649529824777,nan,95.6484375,0.049168646080760096,0.957345971563981,0.5093395597064709,0.10836711168852431,0.6895314756957547,proxy +direct,tls-init-stream16,2,17.9537542328526,nan,nan,nan,0,0,424,1.83,0.0,0.0813396718411371,nan,0.0,0.6075533661740559,0.17231638418079095,0.9185790631876768,1.7678203990168668,1.736464051627823,origin +3proxy-devel,tls-init-stream16,2,7.39921253835151,nan,nan,nan,0,0,350,10.06,7.851057,1.0835586377024127,nan,95.4453125,0.06012357414448669,0.9553656220322887,0.5556293590169379,0.13688854209297033,0.8042317032742594,proxy diff --git a/bench/report.py b/bench/report.py index 8cbcc5b..1877d17 100644 --- a/bench/report.py +++ b/bench/report.py @@ -36,7 +36,8 @@ import csv import statistics STREAM = ("stream1", "stream1-down", "stream16", "connect-stream1-down", "connect-stream16", - "socks-stream1-down", "socks-stream16", "relay-stream1-down", "relay-stream16") + "socks-stream1-down", "socks-stream16", "relay-stream1-down", "relay-stream16", + "tls-term-stream1-down", "tls-term-stream16", "tls-init-stream1-down", "tls-init-stream16") # connect-ka-1m etc. are bulk too when present @@ -117,7 +118,7 @@ def load(paths): # In the combined table, the bulk scenarios in Gbit/s, the others in # thousands of requests per second. -BULK = STREAM + ("http-ka-1m",) +BULK = STREAM + ("http-ka-1m", "tls-term-ka-1m", "tls-init-ka-1m") def combined(machines, labels, splice, prefix=None, split=False, overall_only=False): @@ -288,7 +289,7 @@ def main(): sel = by.get((s, p), []) if p == "direct": return "–" - if s in STREAM or s in ("http-ka-1m", "http-ka-50k-2000", "http-ka-2k-50k-2000"): + if s in BULK or s in ("http-ka-50k-2000", "http-ka-2k-50k-2000"): return fmt(median(sel, "cpu_s_per_gb"), 2) + " s/GB" return fmt(median(sel, "cpu_us_per_req"), 0) + " µs/req"