diff --git a/bench/README.md b/bench/README.md index e8eda0a..5f1ac81 100644 --- a/bench/README.md +++ b/bench/README.md @@ -230,6 +230,12 @@ variant; `nginx::halfclose` turns on `proxy_half_close` in its relays, which nginx leaves off: then a client shutting down its sending side still gets the answer, as through 3proxy and haproxy. +`ktls` asks for kernel TLS in the `tls-*` scenarios: among 3proxy's options +(`3proxy:bench/bin/3proxy:ktls`) it adds `ssl_ktls` before the TLS services, +for nginx (`nginx::ktls`, or `nginx::halfclose+ktls`) it adds `Options KTLS` +to both sides' OpenSSL configuration. Either takes effect only where the +proxy's OpenSSL is built with kernel TLS and the tls module is loaded. + `--proxy stunnel` measures stunnel in the `tls-*` scenarios. `--proxy envoy` measures envoy where nginx and haproxy are measured: as an HTTP proxy for the origin (its connections there pooled), as a TCP relay (`tcp_proxy`) and in diff --git a/bench/RESULTS.md b/bench/RESULTS.md index 29465a9..45cf244 100644 --- a/bench/RESULTS.md +++ b/bench/RESULTS.md @@ -529,6 +529,62 @@ program on one core, can set the pace (‡). envoy. With 16 streams on Intel haproxy needs the least (44-55 MB, devel 80-95), and envoy the most everywhere (66-148 MB). +## Kernel TLS + +Measured on 14 October 2026 on Intel only, whose OpenSSL (3.5.8) and nginx +are built with kernel TLS and whose kernel has the tls module; the ARM +machine's OpenSSL (Ubuntu 24.04) is not. devel with `ssl_ktls` (bench option +`3proxy::ktls`) and without, nginx with `ssl_conf_command Options +KTLS` / `proxy_ssl_conf_command Options KTLS` (`nginx::ktls`) and without, in +the TLS scenarios of the HTTPS section; `/proc/net/tls_stat` confirmed the +sessions in the kernel. Each cell is throughput and the CPU busy time per +gigabyte or per request. + +**devel splicing over kernel TLS** ([data/](data/) `skylake-ktls-splice.csv`): +a connection whose TLS sockets the kernel took both ways is spliced as a +plain one is. + +| scenario (Intel) | 3proxy | 3proxy ssl_ktls | nginx | nginx KTLS | +|---|---:|---:|---:|---:| +| tls-term-ka-1k | 23.5k/s, 43 µs | 21.2k/s, 47 µs | 34.5k/s, 29 µs | 32.3k/s, 31 µs | +| tls-term-ka-1m | 5.82 Gbit/s, 1.38 s/GB | 6.80 Gbit/s, 1.18 s/GB | 5.77 Gbit/s, 1.39 s/GB | 5.59 Gbit/s, 1.43 s/GB | +| tls-term-close-1k | 1.3k/s, 782 µs | 1.2k/s, 820 µs | 1.3k/s, 781 µs | 1.3k/s, 783 µs | +| tls-term-stream1-down | 6.56 Gbit/s, 1.20 s/GB | 7.49 Gbit/s, 0.79 s/GB | 6.27 Gbit/s, 1.26 s/GB | 6.01 Gbit/s, 1.30 s/GB | +| tls-term-stream16 | 6.31 Gbit/s, 1.27 s/GB | 6.94 Gbit/s, 1.15 s/GB | 6.20 Gbit/s, 1.29 s/GB | 5.63 Gbit/s, 1.43 s/GB | +| tls-init-ka-1k | 24.6k/s, 41 µs | 22.0k/s, 45 µs | 32.7k/s, 31 µs | 31.5k/s, 32 µs | +| tls-init-ka-1m | 5.59 Gbit/s, 1.43 s/GB | 5.90 Gbit/s, 1.36 s/GB | 5.44 Gbit/s, 1.47 s/GB | 5.03 Gbit/s, 1.59 s/GB | +| tls-init-close-1k | 0.8k/s, 1243 µs | 0.8k/s, 1265 µs | 1.3k/s, 798 µs | 1.2k/s, 861 µs | +| tls-init-stream1-down | 6.09 Gbit/s, 1.31 s/GB | 6.91 Gbit/s, 1.16 s/GB | 6.15 Gbit/s, 1.30 s/GB | 5.59 Gbit/s, 1.43 s/GB | +| tls-init-stream16 | 7.03 Gbit/s, 1.14 s/GB | 7.65 Gbit/s, 1.05 s/GB | 6.24 Gbit/s, 1.29 s/GB | 6.01 Gbit/s, 1.33 s/GB | + +* With splice() kernel TLS pays for bulk data: 3proxy moves 6-17% more at + 5-34% less CPU per gigabyte (ending TLS for one stream it is no longer + held by its CPU: 7.5 Gbit/s at 0.79 s/GB against 6.6 at 1.20), ahead of + nginx with or without it by 8-23%. +* Small requests, which are not spliced, are 0-11% slower with it: the + kernel does the same encryption with a layer more for every record. +* nginx, which relays TLS through user space with or without it, is 0-9% + slower with kernel TLS. + +**Before splice() over kernel TLS** ([data/](data/) `skylake-ktls.csv`), the +same with devel copying TLS connections: + +| scenario (Intel) | 3proxy | 3proxy ssl_ktls | nginx | nginx KTLS | +|---|---:|---:|---:|---:| +| tls-term-ka-1k | 24.1k/s, 42 µs | 22.9k/s, 44 µs | 35.1k/s, 29 µs | 33.5k/s, 30 µs | +| tls-term-ka-1m | 6.20 Gbit/s, 1.29 s/GB | 5.48 Gbit/s, 1.46 s/GB | 5.86 Gbit/s, 1.37 s/GB | 5.75 Gbit/s, 1.39 s/GB | +| tls-term-close-1k | 1.3k/s, 785 µs | 1.2k/s, 814 µs | 1.4k/s, 712 µs | 1.3k/s, 782 µs | +| tls-term-stream1-down | 7.11 Gbit/s, 1.06 s/GB | 6.00 Gbit/s, 1.32 s/GB | 5.98 Gbit/s, 1.32 s/GB | 6.70 Gbit/s, 1.16 s/GB | +| tls-term-stream16 | 6.57 Gbit/s, 1.22 s/GB | 5.00 Gbit/s, 1.61 s/GB | 6.31 Gbit/s, 1.27 s/GB | 5.74 Gbit/s, 1.40 s/GB | +| tls-init-ka-1k | 25.7k/s, 39 µs | 22.6k/s, 44 µs | 35.9k/s, 28 µs | 33.3k/s, 30 µs | +| tls-init-ka-1m | 5.66 Gbit/s, 1.42 s/GB | 4.58 Gbit/s, 1.75 s/GB | 5.62 Gbit/s, 1.43 s/GB | 5.07 Gbit/s, 1.58 s/GB | +| tls-init-close-1k | 0.9k/s, 1124 µs | 0.8k/s, 1197 µs | 1.3k/s, 796 µs | 1.2k/s, 834 µs | +| tls-init-stream1-down | 6.43 Gbit/s, 1.24 s/GB | 5.00 Gbit/s, 1.60 s/GB | 6.17 Gbit/s, 1.30 s/GB | 5.58 Gbit/s, 1.43 s/GB | +| tls-init-stream16 | 7.20 Gbit/s, 1.11 s/GB | 6.37 Gbit/s, 1.26 s/GB | 6.43 Gbit/s, 1.25 s/GB | 6.14 Gbit/s, 1.31 s/GB | + +* Without splice() kernel TLS did not pay: 3proxy did 5-24% less, at 4-32% + more CPU per gigabyte or request. + ## Many connections: the mix scenarios Measured on the evening of 13 October 2026 with the same devel @@ -1114,6 +1170,24 @@ proxy's CPUs, 200% for both full. * ARM's client set the pace of the mix again (1.7-1.8k a second); 1 exchange was lost there, none on Intel. +### CONNECT at 10 Mbit/s + +Intel only: 10 Mbit/s, 10 ms and 0.5% loss each way, two CPUs for the proxy +(`--cpus cli=0-3,prx=4-5,srv=6-7`), `mix-connect-3k` only ([data/](data/) +`skylake-10m-connect.csv`); each cell the three runs. + +| | -s0 | -s | -s0 -g8000,2,5 | -s -g8000,2,5 | +|---|---:|---:|---:|---:| +| exchanges a second | 2.98k | 2.98k | 2.98k | 2.98k | +| CPU busy, % of 200 | 98, 98, 105 | 94, 96, 100 | 96, 96, 101 | 94, 97, 102 | +| process CPU µs per exchange | 104, 108, 114 | 98, 98, 102 | 97, 103, 111 | 97, 100, 105 | +| p50 / p90 / p99 ms | 41 / 123-127 / 508 | 41 / 123-127 / 508 | 41 / 127 / 508 | 41 / 127 / 508 | + +* `-s` takes about 9% off the process CPU per exchange; a wait matched to + the rate (5 ms: 8000 bytes take about 6 ms to come) changes nothing beyond + the spread of the runs, as on the other links, and no setting moves the + latency. + ## Earlier sessions The morning session of 13 October ([data/](data/) `*-16k-*.csv`, `*-mix.csv`) diff --git a/bench/bench.py b/bench/bench.py index 22f12a7..738b612 100644 --- a/bench/bench.py +++ b/bench/bench.py @@ -562,6 +562,11 @@ def proxy_conf(kind, work, ncpu, label, opts="", out=PRX_OUT): both 3proxy services, e.g. -s0 to keep a version from using splice(). out is the address the proxy connects from.""" if kind == "3proxy": + # ktls among the options is not a service option: kernel TLS for the + # TLS services (ssl_ktls) + words = opts.split() + ktls = "ssl_ktls\n" if "ktls" in words else "" + opts = " ".join(w for w in words if w != "ktls") conf = os.path.join(work, f"{label}.cfg") with open(conf, "w") as f: f.write(f"""maxconn 100000 @@ -573,7 +578,7 @@ tcppm {opts} -i{PRX_IN} -e{out} {RELAY_MIX} {SRV_ADDR} {ORIGIN_MIX} socks {opts} -i{PRX_IN} -e{out} -p{SOCKS_PORT} """) if TLS: - f.write(f"""ssl_server_cert {TLS["cert"]} + f.write(f"""{ktls}ssl_server_cert {TLS["cert"]} ssl_server_key {TLS["key"]} ssl_serv tcppm {opts} -i{PRX_IN} -e{out} {TLS_TERM} {SRV_ADDR} {ORIGIN_HTTP} @@ -676,17 +681,22 @@ socks pass {{ from: 0.0.0.0/0 to: 0.0.0.0/0 }} if os.path.exists(stream): load = f"load_module {stream};\n" break - halfclose = " proxy_half_close on;\n" if opts == "halfclose" else "" + nopts = opts.split("+") + halfclose = " proxy_half_close on;\n" if "halfclose" in nopts else "" + # kernel TLS, where nginx's OpenSSL is built with it + ktls = "ktls" in nopts nginx_tls = "" if TLS: nginx_tls = "" for port, dest in ((TLS_TERM, ORIGIN_HTTP), (TLS_TERM_TCP, ORIGIN_TCP)): nginx_tls += (f" server {{\n listen {PRX_IN}:{port} ssl backlog=65535 reuseport;\n" f" ssl_certificate {TLS['cert']};\n ssl_certificate_key {TLS['key']};\n" + + (" ssl_conf_command Options KTLS;\n" if ktls else "") + f" proxy_pass {SRV_ADDR}:{dest};\n proxy_bind {out};\n }}\n") for port, dest in ((TLS_INIT, ORIGIN_TLS), (TLS_INIT_TCP, ORIGIN_TLS_TCP)): nginx_tls += (f" server {{\n listen {PRX_IN}:{port} backlog=65535 reuseport;\n" f" proxy_pass {SRV_ADDR}:{dest};\n proxy_ssl on;\n" + + (" proxy_ssl_conf_command Options KTLS;\n" if ktls else "") + f" proxy_bind {out};\n }}\n") f.write(f"""{load}daemon off; worker_processes {ncpu}; @@ -838,8 +848,8 @@ class Proxy: path, _, self.opts = path.partition(":") # nginx has no splice(): its one option is halfclose, for # proxy_half_close on, which its relays leave off by default - if kind == "nginx" and self.opts not in ("", "halfclose"): - sys.exit("nginx options: halfclose") + if kind == "nginx" and not set(self.opts.split("+")) <= {"", "halfclose", "ktls"}: + sys.exit("nginx options: halfclose, ktls, halfclose+ktls") self.binary = path or tool(kind) self.label = kind + ("-" + self.opts if self.opts else "") elif kind == "envoy": @@ -1204,7 +1214,8 @@ def summary(rows): def main(): ap = argparse.ArgumentParser(description=__doc__.split("\n\n")[0]) ap.add_argument("--proxy", action="append", required=True, - help="3proxy:[:], nginx[:[:halfclose]], " + help="3proxy:[:, ktls for ssl_ktls], " + "nginx[:[:halfclose|ktls|halfclose+ktls]], " "tinyproxy, squid, dante (sockd), " "haproxy[:[:splice-auto|splice-response|splice-request|splice]] " "or none (the kernel routing, an upper bound); repeat") diff --git a/bench/data/skylake-10m-connect.csv b/bench/data/skylake-10m-connect.csv new file mode 100644 index 0000000..2f077f8 --- /dev/null +++ b/bench/data/skylake-10m-connect.csv @@ -0,0 +1,13 @@ +proxy,scenario,rep,gbit,rate,p50_ms,p90_ms,p99_ms,errors,timeouts,retrans,prx_cpu_s,prx_task_s,cpu_s_per_gb,cpu_us_per_req,peak_mb,cli_util,prx_util,srv_util,cli_thread,srv_thread,bound +3proxy-devel-s0,mix-connect-3k,0,1.0833090672,2984.7,40.96,126.976,507.904,0,0,0,10.49,3.4134260000000003,7.746635059273138,351.4591081180688,1356.38671875,0.7092015921329899,0.49481132075471695,0.24598677998111426,1.0228413218482582,0.29418743221806337,client +3proxy-devel-s,mix-connect-3k,0,1.084996412,2984.5,40.96,122.88,507.904,0,0,0,9.64,3.03341,7.107857606445247,323.00217791924945,1361.6484375,0.7052681537731371,0.4573055028462998,0.2541646834840552,1.0324029701744581,0.3039691427621107,client +"3proxy-devel-s0-g8000,2,5",mix-connect-3k,0,1.0845063936,2985.5,40.96,126.976,507.904,0,0,0,10.08,3.3130330000000003,7.435640810960729,337.63188745603753,1358.56640625,0.705270621602458,0.4763705103969754,0.2440305635148042,1.0190327482029051,0.29567378104061465,client +"3proxy-devel-s-g8000,2,5",mix-connect-3k,0,1.0827053152000001,2984.5,40.96,126.976,507.904,0,0,0,9.4,2.9779820000000004,6.945564868323277,314.96062992125985,1369.7578125,0.7084223013048636,0.44655581947743467,0.23424723424723426,1.0309091510574537,0.2898800177422385,client +3proxy-devel-s,mix-connect-3k,1,1.0793660167999999,2983.6,40.96,126.976,507.904,0,0,0,9.97,2.910027,7.389522993920518,334.1600750770881,1364.75,0.7185542168674699,0.48071359691417553,0.2453748782862707,1.0341719610882976,0.3025844264339619,client +"3proxy-devel-s0-g8000,2,5",mix-connect-3k,1,1.08644042,2985.4,40.96,126.976,507.904,0,0,0,9.6,2.9069730000000003,7.068956436653931,321.5649494205132,1358.70703125,0.7192598003408814,0.4680643588493418,0.23092072870507138,1.029042164689488,0.2788424858854484,client +"3proxy-devel-s-g8000,2,5",mix-connect-3k,1,1.079747428,2983.7,40.96,126.976,507.904,0,0,0,10.19,3.119586,7.549913793357978,341.5222710057982,1368.4609375,0.7161716171617162,0.48362600854295207,0.24462494027711418,1.027294864636595,0.30028820246596594,client +3proxy-devel-s0,mix-connect-3k,1,1.0821890424,2984,40.96,122.88,507.904,0,0,0,9.84,3.104513,7.274144989069611,329.7587131367292,1350.8671875,0.722448003825006,0.4733044733044733,0.25,1.0400936512304055,0.30273999837722054,client +"3proxy-devel-s0-g8000,2,5",mix-connect-3k,2,1.0854676855999998,2984.5,40.96,126.976,507.904,0,0,0,9.64,3.0625229999999997,7.104771613479343,323.00217791924945,1355.9609375,0.7131560028756291,0.46058289536550406,0.23949782713664897,0.9793776890400269,0.28664305959265485,client +"3proxy-devel-s-g8000,2,5",mix-connect-3k,2,1.0846176912,2984.3,40.96,126.976,507.904,0,0,0,9.68,2.9011819999999995,7.139842972164863,324.36417250276446,1369.375,0.7232613908872901,0.4656084656084656,0.23959341723136496,1.0444379140190048,0.29042166740903086,client +3proxy-devel-s0,mix-connect-3k,2,1.0819248935999999,2984.4,40.96,122.88,507.904,0,0,0,9.82,3.2234460000000005,7.261132493088244,329.04436402626993,1350.875,0.691814946619217,0.46386395843174305,0.2460582895365504,1.0237687152033708,0.30372538157979995,client +3proxy-devel-s,mix-connect-3k,2,1.0825509968,2984.5,40.96,126.976,507.904,0,0,0,9.45,2.92331,6.983504723885724,316.635952420841,1362.32421875,0.6979881656804734,0.44786729857819907,0.2350406893250359,1.0260921713430133,0.2851011848282969,client diff --git a/bench/data/skylake-ktls-splice.csv b/bench/data/skylake-ktls-splice.csv new file mode 100644 index 0000000..66f3e9a --- /dev/null +++ b/bench/data/skylake-ktls-splice.csv @@ -0,0 +1,121 @@ +proxy,scenario,rep,gbit,rate,p50_ms,p90_ms,p99_ms,errors,timeouts,retrans,prx_cpu_s,prx_task_s,cpu_s_per_gb,cpu_us_per_req,peak_mb,cli_util,prx_util,srv_util,cli_thread,srv_thread,bound +3proxy-ksplice,tls-term-ka-1k,0,0.25232932863999996,24678.11,2.44,3.22,5.1,0,0,0,10.03,7.303116,31.7997120796366,40.576895847239925,14.35546875,0.2528763183125599,0.9525166191832859,0.3345104333868379,0.7096641249072052,0.3440782592428632,proxy +3proxy-ksplicektls,tls-term-ka-1k,0,0.21692940287999998,21215.37,2.81,3.88,5.78,0,0,0,10.01,7.3958710000000005,36.91523552678485,47.11829941066822,14.3515625,0.23150586545367488,0.9524262607040913,0.3120408815075056,0.6421146463583584,0.29847090988892605,proxy +nginx,tls-term-ka-1k,0,0.33353105408,32626.2,1.8,2.48,2.82,0,0,0,10.0,6.7322560000000005,23.985772545428826,30.59863162919354,35.0,0.28243913968329,0.9523809523809523,0.39836375078665825,0.7962975450577282,0.3659934460986228,proxy +nginx-ktls,tls-term-ka-1k,0,0.31096569856,30416.15,1.92,2.68,3.01,0,0,0,10.03,6.901791,25.803489057336623,32.91859621715115,34.9921875,0.26900307838029835,0.9525166191832859,0.38396492326965237,0.759151439940726,0.3137059235879253,proxy +3proxy-ksplice,tls-term-ka-1m,0,5.82781763584,694.42,22.08,34.49,52.07,0,0,0,10.02,7.470508,1.3754720035683825,1441.5192058696591,19.08203125,0.255736137667304,0.9524714828897338,0.06780195521917376,0.7391896475795305,0.04658104211527053,proxy +3proxy-ksplicektls,tls-term-ka-1m,0,6.80408383488,810.77,18.68,30.49,48.08,0,0,0,10.01,6.943242,1.1769402309460568,1233.6701996549175,18.88671875,0.3016587677725118,0.9533333333333334,0.07300188797986155,0.8723846290896963,0.0665480020024152,proxy +nginx,tls-term-ka-1m,0,6.22149500928,741.42,21.15,23.76,30.31,0,0,0,10.04,7.483822,1.29100802749491,1351.8244243974686,48.1171875,0.25898418782942023,0.952561669829222,0.06025236593059937,0.7525467852518639,0.042781432485919056,proxy +nginx-ktls,tls-term-ka-1m,0,5.83008256,694.75,22.38,24.77,33.63,0,0,0,10.02,7.438579000000001,1.3749376475382193,1440.483036227717,47.046875,0.2631704410011919,0.9524714828897338,0.058582677165354334,0.7297055955651024,0.04944094241994385,proxy +3proxy-ksplice,tls-term-close-1k,0,0.013254000640000001,1298.34,19.45,30.41,40.33,0,0,0,10.13,8.840598,611.4380269110958,772.4569162726857,22.34765625,0.9517470881863561,0.961100569259962,0.06829733163913596,3.703592039553187,0.030422982955746414,client +3proxy-ksplicektls,tls-term-close-1k,0,0.01291845632,1271.46,21.27,32.1,44.58,0,0,0,10.1,8.873759999999999,625.4617269937095,786.481856408659,20.93359375,0.9491807171693184,0.9600760456273765,0.0673015873015873,3.6912782861765874,0.0294737933035682,client +nginx,tls-term-close-1k,0,0.013925089279999999,1364.54,3.15,3.94,4.74,0,0,0,10.11,8.796314,580.822128847421,733.5655202437963,34.859375,0.9538315088053307,0.9610266159695817,0.12226103524928549,3.7068905218163133,0.036130894930851844,client +nginx-ktls,tls-term-close-1k,0,0.0130023424,1277.85,3.25,4.02,4.94,0,0,0,10.11,8.783344,622.041763797883,783.2352029748993,34.3671875,0.9534441805225653,0.9610266159695817,0.08563273073263558,3.7020836704407114,0.03516204049126043,client +3proxy-ksplice,tls-init-ka-1k,0,0.25383927808,24827.89,2.42,3.27,5.82,0,0,0,10.12,7.386144,31.894197230770814,40.35988897042402,15.6640625,0.18354582631705557,0.9610636277302944,0.3813640730067243,0.45952003112448453,0.3745791568978095,proxy +3proxy-ksplicektls,tls-init-ka-1k,0,0.22515023872,22023.98,2.68,3.8,13.2,0,0,0,10.1,7.437137,35.887148270130865,45.4059108605544,14.625,0.17577937649880096,0.9600760456273765,0.36252414681262074,0.43188869555876896,0.3108608364197206,proxy +nginx,tls-init-ka-1k,0,0.3338665984,32653.35,1.84,2.43,2.75,0,0,0,10.1,6.787256,24.201282903776697,30.626942612387232,35.953125,0.22892145087677157,0.9609895337773549,0.47484973109775386,0.5764912311579603,0.45154066388608277,proxy +nginx-ktls,tls-init-ka-1k,0,0.32245809152,31542.73,1.9,2.55,3.54,0,0,0,10.13,6.914149,25.131948036408204,31.798847958815312,35.77734375,0.2237410071942446,0.961100569259962,0.45176544766708704,0.5574919240792225,0.4010413066161622,proxy +3proxy-ksplice,tls-init-ka-1m,0,5.767503544319999,687.37,22.92,33.0,48.87,0,0,0,9.99,7.297042,1.3856948571571746,1452.2459659834278,18.28515625,0.200982800982801,0.9514285714285714,0.2918373778758273,0.5051842575118066,0.3460434376094258,proxy +3proxy-ksplicektls,tls-init-ka-1m,0,6.08450904064,725.06,21.51,31.67,44.75,0,0,0,10.02,7.009504000000001,1.3174440117450847,1380.9261300992282,18.21484375,0.21006350757205666,0.9515669515669516,0.30861093651791327,0.5393944943516197,0.3270208930737481,proxy +nginx,tls-init-ka-1m,0,5.596795371520001,666.92,23.39,25.94,32.93,0,0,0,10.03,7.333718999999999,1.4336775721390738,1502.8468684447107,45.0703125,0.19740068661108387,0.9516129032258065,0.29478315524827153,0.4994344190692393,0.34981083791259165,proxy +nginx-ktls,tls-init-ka-1m,0,5.243383316479999,624.3,24.65,28.17,36.41,0,0,0,10.01,7.529783,1.5272581683720865,1602.112676056338,47.42578125,0.18719332679097153,0.9515209125475285,0.27329974811083124,0.471779821380507,0.2528257264736601,proxy +3proxy-ksplice,tls-init-close-1k,0,0.00864026624,851.62,72.12,95.7,134.15,0,0,0,10.02,9.002260999999999,927.749189358313,1174.8153359127682,50.94140625,0.04157757186980281,0.9524714828897338,0.2889661713563073,0.0684953157388227,0.2471755535247446,proxy +3proxy-ksplicektls,tls-init-close-1k,0,0.008304721919999999,817.69,74.14,101.9,132.83,0,0,0,10.03,9.045409999999999,966.1973124802714,1223.618396974503,59.81640625,0.03970518307180219,0.9534220532319392,0.26818469323213157,0.06279572622399907,0.23104143182523948,proxy +nginx,tls-init-close-1k,0,0.01308622848,1283.65,48.44,53.0,71.19,0,0,0,10.05,8.817149,614.3863384540265,781.0678479832129,31.60546875,0.059800664451827246,0.9553231939163498,0.3694829760403531,0.10180226429981691,0.3155515015158498,proxy +nginx-ktls,tls-init-close-1k,0,0.011827937279999999,1164.44,53.17,61.48,76.73,0,0,0,10.05,8.834159,679.7465872257314,861.4039598868603,31.75,0.05812574139976275,0.956232159847764,0.3527926790785737,0.09799476941815402,0.30221408295146895,proxy +3proxy-ksplice,tls-term-stream1-down,0,5.6745174544114905,nan,nan,nan,nan,0,0,0,9.86,7.514831,1.3900591325854168,nan,8.23046875,0.39761843127103286,0.9704724409448819,0.06765188834154351,0.7260985708550692,0.10353699791739669,proxy +3proxy-ksplicektls,tls-term-stream1-down,0,7.491891795340206,nan,nan,nan,nan,0,0,0,8.0,6.720459,0.8541641884516346,nan,8.5859375,0.38845780795344326,0.7455731593662628,0.07926065162907268,0.7998961013233421,0.12592285923174693,none +nginx,tls-term-stream1-down,0,4.90106566266301,nan,nan,nan,nan,0,0,0,10.02,7.514945,1.6353948701064285,nan,33.57421875,0.36145169200588523,0.9408450704225352,0.05701754385964912,0.6800787214155922,0.08265666542266957,proxy +nginx-ktls,tls-term-stream1-down,0,6.01294358280279,nan,nan,nan,nan,0,0,0,9.76,7.493136000000001,1.2983939080209241,nan,35.24609375,0.37548923679060664,0.9155722326454033,0.06727158948685857,0.7048591755451242,0.10429664846292247,proxy +3proxy-ksplice,tls-term-stream16,0,5.606848414999827,nan,nan,nan,nan,0,0,213,10.06,7.424974000000001,1.429561157488033,nan,79.390625,0.21173409037142182,0.9428303655107779,0.20433639947437582,0.5991757591037115,0.3365349965365454,proxy +3proxy-ksplicektls,tls-term-stream16,0,6.944057675552902,nan,nan,nan,nan,0,0,260,10.05,7.116556,1.1532565508387105,nan,55.88671875,0.24982307147912244,0.9436619718309859,0.251069430733794,0.6977187832125356,0.41820578843118583,proxy +nginx,tls-term-stream16,0,5.919047942960162,nan,nan,nan,nan,0,0,226,10.11,7.579682,1.354520655210071,nan,90.4453125,0.22358780430158354,0.948405253283302,0.2155453951665578,0.6213573417630854,0.3481461054601926,proxy +nginx-ktls,tls-term-stream16,0,5.231536811243903,nan,nan,nan,nan,0,0,164,10.16,7.758987,1.533273631441005,nan,114.703125,0.20994082840236686,0.9522024367385192,0.19915254237288135,0.5862201652162891,0.3301272248529309,proxy +3proxy-ksplice,tls-init-stream1-down,0,5.670721386411653,nan,nan,nan,nan,0,0,0,10.01,7.414155,1.4120149254485912,nan,5.78515625,0.12825,0.9506172839506173,0.29490874159462055,0.30353537824709853,0.6246701096400474,proxy +3proxy-ksplicektls,tls-init-stream1-down,0,6.909064760896488,nan,nan,nan,nan,0,0,0,9.98,7.1259760000000005,1.1554601487453906,nan,10.203125,0.18659691081215746,0.9513822688274547,0.3175567636712504,0.42232535469997756,0.6872306168981653,proxy +nginx,tls-init-stream1-down,0,6.216553512940169,nan,nan,nan,nan,0,0,0,10.01,7.520242,1.2880361475040898,nan,37.51953125,0.16833618372831663,0.9515209125475285,0.2944746087511977,0.37395280942811426,0.642814391916999,proxy +nginx-ktls,tls-init-stream1-down,0,5.326351689838529,nan,nan,nan,nan,0,0,0,10.01,7.634256,1.503351166492372,nan,34.9296875,0.1658952496954933,0.9515209125475285,0.28255291853752407,0.3644015343977695,0.6027720186295242,proxy +3proxy-ksplice,tls-init-stream16,0,6.445850527000402,nan,nan,nan,nan,0,0,376,10.06,7.80634,1.243929741566182,nan,92.6015625,0.06516587677725119,0.9562737642585551,0.541320942582144,0.13876375010947697,0.7749139160150545,proxy +3proxy-ksplicektls,tls-init-stream16,0,7.336089233450127,nan,nan,nan,nan,0,0,617,10.04,7.297786,1.0910934005719046,nan,78.125,0.07577551503670377,0.9561904761904761,0.6337033299697276,0.155866629641123,0.9922995359969098,origin +nginx,tls-init-stream16,0,6.24231032997711,nan,nan,nan,nan,0,0,433,10.14,7.863570999999999,1.2853829885168477,nan,115.27734375,0.056585829766999524,0.9620493358633776,0.5226364846870839,0.1254802734325599,0.7190536565366029,proxy +nginx-ktls,tls-init-stream16,0,5.541304803934703,nan,nan,nan,nan,0,0,366,10.13,7.7487200000000005,1.44597217163207,nan,115.78515625,0.05370722433460076,0.9629277566539924,0.519433465085639,0.11594640270515653,0.7083536479405714,proxy +3proxy-ksplicektls,tls-term-ka-1k,1,0.19822280704,19391.91,3.2,4.24,6.26,0,0,0,10.02,7.4183769999999996,40.43934257465351,51.62447061732975,14.34765625,0.23452067893856085,0.9524714828897338,0.30448717948717946,0.6592837431323209,0.2813925644977245,proxy +nginx,tls-term-ka-1k,1,0.35257319424,34487.38,1.72,2.35,2.63,0,0,0,10.01,6.695883,22.713014292711307,28.987857533802273,35.01953125,0.29312796208530806,0.9524262607040913,0.41328297135662573,0.8295392898315199,0.4325160642052838,proxy +nginx-ktls,tls-term-ka-1k,1,0.33051115519999996,32326.83,1.83,2.46,2.85,0,0,0,10.1,6.894671000000001,24.446981207368335,30.934055332142933,34.8125,0.28480264712833847,0.9600760456273765,0.3956043956043956,0.7838691983984613,0.33461976558768125,proxy +3proxy-ksplice,tls-term-ka-1k,1,0.23630708736,23113.42,2.52,3.65,4.99,0,0,0,10.11,7.373978999999999,34.2266501202243,43.31007184074231,14.32421875,0.24579529072561268,0.9610266159695817,0.32346314773093016,0.7001982878550356,0.2719050157195778,proxy +3proxy-ksplicektls,tls-term-ka-1m,1,6.14356484096,732.13,20.21,34.97,55.22,0,0,0,10.02,6.972262,1.3047799132119864,1367.35807860262,21.5390625,0.29938417811463763,0.9515669515669516,0.06687697160883281,0.8618505981158419,0.04752724941759543,proxy +nginx,tls-term-ka-1m,1,5.7676713164799995,687.19,22.0,29.32,35.14,0,0,0,10.01,7.495611,1.3884286327340978,1455.153365314726,46.04296875,0.25836520076481834,0.9524262607040913,0.06393700787401575,0.7420092779592163,0.0494318278888597,proxy +nginx-ktls,tls-term-ka-1m,1,4.980736,593.54,26.57,31.45,36.19,0,0,0,10.03,7.556031,1.611006887335526,1688.8365044620307,46.95703125,0.24339894383101296,0.9516129032258065,0.05854579792256846,0.677349510910715,0.03422159569674635,proxy +3proxy-ksplice,tls-term-ka-1m,1,5.55552342016,661.81,22.83,35.1,53.12,0,0,0,10.02,7.429608,1.4428883462017945,1513.1380247659317,19.89453125,0.24640115163147794,0.9515669515669516,0.06364209199747953,0.7086836754478396,0.043723077784222526,proxy +3proxy-ksplicektls,tls-term-close-1k,1,0.011827937279999999,1165.27,20.56,32.41,44.48,0,0,0,10.12,8.873964999999998,684.4811405695922,859.8861415583312,25.1640625,0.9546210501306723,0.9619771863117871,0.07412100095026924,3.696924480090183,0.031372733335726884,client +nginx,tls-term-close-1k,1,0.01308622848,1281.21,3.23,4.14,4.98,0,0,0,10.11,8.758714,618.0543165940504,781.2379259717178,33.38671875,0.9555924958442176,0.9610266159695817,0.08375634517766498,3.70158221506262,0.03612492499451384,client +nginx-ktls,tls-term-close-1k,1,0.01266679808,1247.89,3.21,3.94,4.85,0,0,0,10.1,8.774252,637.8881189207367,801.3329101872422,33.359375,0.9553231939163498,0.9609895337773549,0.08116677235256817,3.703726758168911,0.034198513116633425,client +3proxy-ksplice,tls-term-close-1k,1,0.011240734720000001,1101.58,20.92,34.21,45.2,0,0,0,10.1,8.822521,718.8142235599346,907.7019861597915,40.12890625,0.9532177630016623,0.9609895337773549,0.08095238095238096,3.689034169748212,0.035168992940219336,client +3proxy-ksplicektls,tls-init-ka-1k,1,0.19763560447999998,19329.45,3.16,4.28,16.72,0,0,0,10.09,7.490909,40.84284317716071,51.68395441157639,14.6015625,0.18314231136580708,0.9591254752851711,0.3800064082024992,0.4347143772508123,0.3183963841598975,proxy +nginx,tls-init-ka-1k,1,0.31113347072,30428.75,1.98,2.62,3.18,0,0,0,10.11,6.835453,25.99527457230301,32.89762688038735,35.98046875,0.2393961179007908,0.9601139601139601,0.47424960505529223,0.5840820039129122,0.4410654692531807,proxy +nginx-ktls,tls-init-ka-1k,1,0.2982150144,29165.12,2.06,2.75,3.26,0,0,0,10.11,6.953333,27.12137085476002,34.32272870785621,35.91015625,0.22508960573476702,0.9610266159695817,0.4513801756587202,0.5517220583040033,0.3819970028108927,proxy +3proxy-ksplice,tls-init-ka-1k,1,0.20065550336000001,19628.83,3.2,4.2,18.4,0,0,0,10.09,7.448548,40.22815155743755,50.89507745232054,15.69140625,0.18000958772770853,0.9600380589914367,0.3764441591784339,0.4376077408251847,0.3184626020379277,proxy +3proxy-ksplicektls,tls-init-ka-1m,1,4.9557379481599995,590.54,25.83,42.1,60.65,0,0,0,10.02,7.018789,1.617518941447708,1694.8579161028417,18.921875,0.19990272373540857,0.9515669515669516,0.2959697732997481,0.5050542088882085,0.3534375080910716,proxy +nginx,tls-init-ka-1m,1,4.6087012352,549.25,28.49,34.24,38.67,0,0,0,10.02,7.374137,1.7393186476866809,1821.4870023632068,45.76953125,0.18322295805739514,0.9524714828897338,0.28085642317380355,0.46130032912907387,0.29746923155148386,proxy +nginx-ktls,tls-init-ka-1m,1,4.59494391808,546.51,27.75,36.04,42.56,0,0,0,10.02,7.4923340000000005,1.7445261885480179,1832.144816236972,45.98828125,0.1828641370869033,0.9515669515669516,0.25126582278481013,0.4480683712592145,0.30892245852500166,proxy +3proxy-ksplice,tls-init-ka-1m,1,5.3255077888,634.59,24.29,36.55,54.95,0,0,0,10.0,7.290274,1.5022041685535952,1574.5551881593449,17.48828125,0.1928921568627451,0.9514747859181731,0.2942659105229994,0.4879526377436015,0.3079326668516742,proxy +3proxy-ksplicektls,tls-init-close-1k,1,0.0075497472,740.46,81.93,118.61,170.17,0,0,0,10.08,9.073598,1068.115234375,1358.1244947453517,153.3125,0.04107312440645774,0.9590865842055185,0.27330173775671407,0.06468940271131363,0.2291092712953228,proxy +nginx,tls-init-close-1k,1,0.01056964608,1039.53,60.42,72.59,76.96,0,0,0,10.07,8.810018999999999,762.1825687468998,967.3390970220942,33.3046875,0.061611374407582936,0.9572243346007605,0.3545224541429475,0.09988801432066717,0.30499715769451535,proxy +nginx-ktls,tls-init-close-1k,1,0.011156848640000001,1092.89,55.84,69.76,78.45,0,0,0,10.08,8.868641,722.7847450657895,920.2958093672967,35.625,0.057299096528768426,0.9581749049429658,0.3448275862068966,0.0979886436476056,0.2975576907141273,proxy +3proxy-ksplice,tls-init-close-1k,1,0.0079691776,785.09,77.75,110.08,160.37,0,0,0,10.06,9.043084,1009.8909076891447,1279.0845518118244,92.890625,0.04277566539923954,0.9562737642585551,0.27679697351828497,0.06754613063125449,0.23386108091453345,proxy +3proxy-ksplicektls,tls-term-stream1-down,1,7.438340662396382,nan,nan,nan,nan,0,0,0,7.38,5.963725999999999,0.7936419993040382,nan,8.3046875,0.40966010733452596,0.7178988326848249,0.08144499178981937,0.8009936742385864,0.1409465833349352,none +nginx,tls-term-stream1-down,1,6.26682843060054,nan,nan,nan,nan,0,0,0,9.84,7.698968,1.2560565207298098,nan,35.8671875,0.39297987236131565,0.923943661971831,0.07903780068728522,0.7741569170420605,0.12116514935499652,proxy +nginx-ktls,tls-term-stream1-down,1,6.170527788643289,nan,nan,nan,nan,0,0,0,9.83,7.575132999999999,1.2743096909620155,nan,36.7109375,0.3817243920412675,0.9230046948356807,0.0721875,0.7314557578905811,0.11181236679887911,proxy +3proxy-ksplice,tls-term-stream1-down,1,6.708720283563356,nan,nan,nan,nan,0,0,0,9.76,7.541123,1.1637736094179794,nan,8.45703125,0.386391549987718,0.9155722326454033,0.07524196066187949,0.7524193289334596,0.11833624638337431,proxy +3proxy-ksplicektls,tls-term-stream16,1,6.99919210100683,nan,nan,nan,nan,0,0,245,10.04,7.139615,1.14494515999148,nan,75.4296875,0.23937677053824363,0.9418386491557224,0.247537754432042,0.6713374393642326,0.4152610107232596,proxy +nginx,tls-term-stream16,1,6.4484778465287045,nan,nan,nan,nan,0,0,251,10.1,7.518965,1.2443180642631646,nan,89.4296875,0.22718676122931442,0.9465791940018744,0.22764493940386504,0.637325141038503,0.3685889887391464,proxy +nginx-ktls,tls-term-stream16,1,5.7279699579323395,nan,nan,nan,nan,0,0,198,10.11,7.676056,1.4002573969490333,nan,93.4296875,0.21362668559261888,0.9475164011246485,0.2051871306631648,0.6041936833636724,0.3403285633135482,proxy +3proxy-ksplice,tls-term-stream16,1,6.5965411436163155,nan,nan,nan,nan,0,0,294,10.05,7.43635,1.2143907100058997,nan,87.2421875,0.23406775645581615,0.9445488721804511,0.2291735265064208,0.6613527521098542,0.37643145194140015,proxy +3proxy-ksplicektls,tls-init-stream1-down,1,6.951844882073525,nan,nan,nan,nan,0,0,0,10.02,6.991088,1.1529527668068018,nan,6.390625,0.18883906327852515,0.9506641366223909,0.3199614890885751,0.42050826161764643,0.6845071399431949,proxy +nginx,tls-init-stream1-down,1,6.149148951589873,nan,nan,nan,nan,0,0,1,9.99,7.501031,1.2995558554653532,nan,36.6796875,0.1704297159504734,0.9514285714285714,0.29244074311338886,0.380610144439883,0.6368002142046838,proxy +nginx-ktls,tls-init-stream1-down,1,5.623120566987276,nan,nan,nan,nan,0,0,0,9.99,7.680849,1.421122676148335,nan,38.09765625,0.16532748965181399,0.950523311132255,0.2597568777991043,0.3510439950028623,0.5578171587964821,proxy +3proxy-ksplice,tls-init-stream1-down,1,6.63091117781515,nan,nan,nan,nan,0,0,0,10.01,7.452977,1.2075491628899577,nan,10.18359375,0.16068333745976726,0.9515209125475285,0.2954908858330668,0.36634978047921374,0.6475527725100126,proxy +3proxy-ksplicektls,tls-init-stream16,1,7.64514605703067,nan,nan,nan,nan,0,0,597,10.06,7.289176,1.0484633214629666,nan,81.3359375,0.06747445949156569,0.9571836346336822,0.6198988195615515,0.15113855071788476,0.981071490856779,origin +nginx,tls-init-stream16,1,6.454994865258617,nan,nan,nan,nan,0,0,406,10.13,7.852273,1.2439163477420692,nan,114.04296875,0.06092935040303461,0.961100569259962,0.5233986060404912,0.13116687915808303,0.7237028475409372,proxy +nginx-ktls,tls-init-stream16,1,6.090022089199298,nan,nan,nan,nan,0,0,540,10.11,7.710031,1.3135759317257747,nan,109.02734375,0.05548968460991226,0.961941008563273,0.5203171456888008,0.11882915405735375,0.7019046038205773,proxy +3proxy-ksplice,tls-init-stream16,1,7.23633372486351,nan,nan,nan,nan,0,0,390,10.07,7.828429,1.109766180786773,nan,85.05859375,0.06414825374198146,0.9554079696394687,0.5434350132625995,0.13973422691292683,0.7750263966918097,proxy +nginx,tls-term-ka-1k,2,0.35618029568,34840.36,1.73,2.22,2.64,0,0,0,10.0,6.68388,22.46053500721267,28.661918342194642,34.95703125,0.29092632077706704,0.9523809523809523,0.40139064475347663,0.8286002366108043,0.38692615059913565,proxy +nginx-ktls,tls-term-ka-1k,2,0.34150023168,33397.87,1.8,2.27,2.92,0,0,0,10.12,6.9138850000000005,23.707158147951972,30.00127475772192,34.93359375,0.2768575485092286,0.9601518026565465,0.39491844416562105,0.7715343180386712,0.35825675687555036,proxy +3proxy-ksplice,tls-term-ka-1k,2,0.24016584704,23490.39,2.54,3.51,5.33,0,0,0,10.01,7.324836,33.3436252435437,42.57635884937496,14.18359375,0.23453237410071942,0.9515209125475285,0.3198458574181118,0.66023453643644,0.26893343481462473,proxy +3proxy-ksplicektls,tls-term-ka-1k,2,0.2290089984,22399.82,2.67,3.62,5.58,0,0,0,10.09,7.429782,35.24752326937385,44.599641965213166,14.35546875,0.23389423076923077,0.9600380589914367,0.3203449377195784,0.659270613956578,0.2785606586821733,proxy +nginx,tls-term-ka-1m,2,5.71801075712,681.42,22.44,28.38,39.21,0,0,0,10.01,7.508766,1.4004870470081807,1467.741935483871,46.94140625,0.2527551509343555,0.9515209125475285,0.06393700787401575,0.7200823506409791,0.03896874043153516,proxy +nginx-ktls,tls-term-ka-1m,2,5.5895811686400005,666.17,23.01,28.02,34.11,0,0,0,10.0,7.453766,1.4312342479045668,1499.47518368571,48.09375,0.25966587112171835,0.9514747859181731,0.056954059156702326,0.7229851535962198,0.04846342295609667,proxy +3proxy-ksplice,tls-term-ka-1m,2,5.82454607872,694.09,21.93,33.9,51.44,0,0,0,10.01,7.457523,1.3748710872521477,1440.7023603914795,19.3828125,0.25269719491728604,0.9515209125475285,0.06368221941992434,0.7287500041678996,0.03993086930388327,proxy +3proxy-ksplicektls,tls-term-ka-1m,2,7.156237598720001,852.88,18.18,28.3,42.03,0,0,0,10.03,6.901819,1.121259584985721,1175.0234301780692,18.06640625,0.3110953394842678,0.9516129032258065,0.07311692404664355,0.9047550112747813,0.050382294944584446,client +nginx,tls-term-close-1k,2,0.01266679808,1240.67,3.31,4.32,4.99,0,0,0,10.1,8.815508,637.8881189207367,806.0011172292714,33.3671875,0.9562217463716393,0.9609895337773549,0.07989854153455929,3.711438185651166,0.03422474802772152,client +nginx-ktls,tls-term-close-1k,2,0.013925089279999999,1370.89,2.97,3.48,4.29,0,0,0,10.13,8.779534,581.9711340479106,731.6192402137801,33.30078125,0.9558194774346793,0.9620132953466287,0.08187876864487464,3.7079660501068967,0.03613314829619952,client +3proxy-ksplice,tls-term-close-1k,2,0.0130023424,1277.78,18.79,30.38,40.02,0,0,0,10.09,8.860475000000001,620.8112162928427,781.8069115140245,41.44140625,0.9498693276312663,0.9600380589914367,0.07606973058637084,3.6885838382155707,0.030423535330863474,client +3proxy-ksplicektls,tls-term-close-1k,2,0.01241513984,1216.91,20.19,31.88,43.94,0,0,0,10.08,8.868224999999999,649.5295344172297,820.1122772760557,23.03515625,0.9508196721311475,0.9590865842055185,0.06986344871387742,3.69703290516213,0.030421169430672298,client +nginx,tls-init-ka-1k,2,0.36121346048,35329.44,1.71,2.2,2.63,0,0,0,10.1,6.756221999999999,22.369044578966847,28.30519162054228,35.9609375,0.23106515989420534,0.9609895337773549,0.4731522425773847,0.5870006336847697,0.3898136782871855,proxy +nginx-ktls,tls-init-ka-1k,2,0.34108080127999996,33360.81,1.8,2.33,2.77,0,0,0,10.11,6.906386,23.712856219545472,30.005787466425275,35.76171875,0.22429906542056074,0.9610266159695817,0.4556324732536186,0.5612983227504297,0.43542970353153915,proxy +3proxy-ksplice,tls-init-ka-1k,2,0.25165824,24613.33,2.42,3.35,8.09,0,0,0,10.1,7.37022,32.10703531901042,40.6293118360024,15.69921875,0.17934520943668752,0.9600760456273765,0.3822963438101347,0.45189886745746055,0.3517519548971979,proxy +3proxy-ksplicektls,tls-init-ka-1k,2,0.23337107456,22823.65,2.6,3.68,6.41,0,0,0,10.11,7.4561340000000005,34.657251397797225,43.85774646665336,14.6015625,0.1791866028708134,0.9610266159695817,0.3716475095785441,0.4309747581610235,0.31944173375809926,proxy +nginx,tls-init-ka-1m,2,5.44487768064,648.82,23.72,28.61,33.83,0,0,0,10.02,7.371252,1.4722093810301695,1542.487684729064,44.203125,0.19306443679291688,0.9524714828897338,0.28136762860727726,0.49275367896111943,0.29372628287176855,proxy +nginx-ktls,tls-init-ka-1m,2,5.03115153408,598.71,25.18,32.06,38.42,0,0,0,10.03,7.543864999999999,1.5948635109968465,1673.8985313751668,46.75,0.18244237371260422,0.9516129032258065,0.2543334383863851,0.4603584855543597,0.26706658483509266,proxy +3proxy-ksplice,tls-init-ka-1m,2,5.5901683712,666.19,23.41,34.54,52.23,0,0,0,10.02,7.318258,1.4339460759889893,1502.4741340530813,18.91796875,0.19338235294117648,0.9524714828897338,0.2906574394463668,0.49086036179759523,0.2613927098260128,proxy +3proxy-ksplicektls,tls-init-ka-1m,2,5.9032312217600005,703.46,21.83,33.08,50.02,0,0,0,10.01,7.0073229999999995,1.356545203664321,1421.875,18.16015625,0.20609013398294762,0.9524262607040913,0.30730478589420657,0.5317877411533619,0.324185440195343,proxy +nginx,tls-init-close-1k,2,0.01283457024,1258.26,49.31,57.21,65.89,0,0,0,10.05,8.831911999999999,626.4331294041053,797.6190476190476,31.80859375,0.05961995249406176,0.9571428571428572,0.3632923368022706,0.09894504152568322,0.31471087107528817,proxy +nginx-ktls,tls-init-close-1k,2,0.01241513984,1216.93,51.39,57.08,65.53,0,0,0,10.05,8.857801,647.5964108028928,824.4462674323215,32.28515625,0.05865590121111375,0.956232159847764,0.3560126582278481,0.09989793623876428,0.3071013358779275,proxy +3proxy-ksplice,tls-init-close-1k,2,0.00822083584,807.52,75.14,104.0,140.93,0,0,0,10.06,9.045176,978.9758799027426,1243.2031636183885,127.3515625,0.04088424055146185,0.9580952380952381,0.28078505856283636,0.06849977009845447,0.24717499328128675,proxy +3proxy-ksplicektls,tls-init-close-1k,2,0.00805306368,792.96,77.52,104.09,133.19,0,0,0,10.05,9.057715,998.3777999877931,1264.9465072372561,89.11328125,0.038058991436726926,0.9544159544159544,0.26795317937361596,0.06564788534408178,0.2262654531168982,proxy +nginx,tls-term-stream1-down,2,6.727902250244174,nan,nan,nan,nan,0,0,0,9.76,7.677534,1.1604184360828438,nan,36.08984375,0.3911547911547912,0.9155722326454033,0.07776389756402248,0.769540147868085,0.12400876434597437,proxy +nginx-ktls,tls-term-stream1-down,2,5.853803281954053,nan,nan,nan,nan,0,0,0,9.9,7.467183,1.3529476034781558,nan,34.58203125,0.380905752753978,0.9295774647887324,0.07299498746867168,0.712321272199629,0.11088446677432914,proxy +3proxy-ksplice,tls-term-stream1-down,2,6.562441172882736,nan,nan,nan,nan,0,0,0,9.83,7.573898000000001,1.198206583090021,nan,8.3671875,0.3956016802569805,0.9238721804511278,0.07145095581322469,0.7799864877273652,0.11183463137674145,proxy +3proxy-ksplicektls,tls-term-stream1-down,2,7.5993364325588475,nan,nan,nan,nan,0,0,0,7.24,5.813371,0.7621289016688052,nan,7.66015625,0.38945827232796487,0.6716141001855288,0.07694713794182045,0.8037769173111151,0.13054455655493002,none +nginx,tls-term-stream16,2,6.19990024821253,nan,nan,nan,nan,0,0,220,10.08,7.5725050000000005,1.2906437229382741,nan,90.28125,0.2246342614440774,0.9473684210526315,0.22229444263893403,0.6261797646333028,0.3565453835381354,proxy +nginx-ktls,tls-term-stream16,2,5.626465891702194,nan,nan,nan,nan,0,0,165,10.15,7.69886,1.4278825529140469,nan,99.80078125,0.2090435606060606,0.950374531835206,0.20058997050147492,0.5891206570429849,0.3329684982084061,proxy +3proxy-ksplice,tls-term-stream16,2,6.314996680105168,nan,nan,nan,nan,0,0,260,10.05,7.445434,1.2693344210684785,nan,84.28515625,0.22913218254906598,0.9436619718309859,0.21610871726881006,0.6504614790549056,0.361136731276418,proxy +3proxy-ksplicektls,tls-term-stream16,2,6.793015509881373,nan,nan,nan,nan,0,0,251,10.09,7.133857,1.1838432445531404,nan,62.6796875,0.24899787785899552,0.9456419868791003,0.24242424242424243,0.6905714560715868,0.4071542035470923,proxy +nginx,tls-init-stream1-down,2,6.042833278505757,nan,nan,nan,nan,0,0,0,9.99,7.398124,1.3224195604613083,nan,34.11328125,0.15336658354114713,0.950523311132255,0.29413659793814434,0.3453888655710673,0.634171671965408,proxy +nginx-ktls,tls-init-stream1-down,2,5.58790224349376,nan,nan,nan,nan,0,0,0,10.01,7.679581000000001,1.4329450476173164,nan,38.66015625,0.13487570760521783,0.9515209125475285,0.2733998070119009,0.33114098230336647,0.5990812766832481,proxy +3proxy-ksplice,tls-init-stream1-down,2,6.08842145595632,nan,nan,nan,nan,0,0,0,9.98,7.402287,1.3111995423088514,nan,6.328125,0.1531311154598826,0.9477682811016145,0.30658105939004815,0.3615660292126046,0.6531725375316942,proxy +3proxy-ksplicektls,tls-init-stream1-down,2,6.41210681924807,nan,nan,nan,nan,0,0,0,10.0,7.019907999999999,1.2475096115326127,nan,8.16015625,0.19195430841817732,0.9514747859181731,0.31971920867900444,0.40626087802334354,0.6816651627181615,proxy +nginx,tls-init-stream16,2,6.212600507468642,nan,nan,nan,nan,0,0,557,10.09,7.858582,1.2883996522169827,nan,107.70703125,0.06137012369172217,0.9600380589914367,0.5225763612217795,0.13022324386838557,0.7312911403250917,proxy +nginx-ktls,tls-init-stream16,2,6.007629852592818,nan,nan,nan,nan,0,0,451,10.12,7.708746,1.3342343377384736,nan,108.75,0.05397051830718022,0.9601518026565465,0.5178453403833444,0.11975996444199664,0.6970848367759259,proxy +3proxy-ksplice,tls-init-stream16,2,7.027110423606573,nan,nan,nan,nan,0,0,327,10.06,7.814476,1.1413572403470096,nan,84.91796875,0.061001661523854736,0.9553656220322887,0.5341511936339522,0.1349908246355018,0.7548276852637233,proxy +3proxy-ksplicektls,tls-init-stream16,2,8.448486750397032,nan,nan,nan,nan,0,0,607,10.06,7.287471,0.9490869057905378,nan,80.07421875,0.07144552575361975,0.9553656220322887,0.6671223513328777,0.16063575408787711,1.073213515746919,origin diff --git a/bench/data/skylake-ktls.csv b/bench/data/skylake-ktls.csv new file mode 100644 index 0000000..386113d --- /dev/null +++ b/bench/data/skylake-ktls.csv @@ -0,0 +1,121 @@ +proxy,scenario,rep,gbit,rate,p50_ms,p90_ms,p99_ms,errors,timeouts,retrans,prx_cpu_s,prx_task_s,cpu_s_per_gb,cpu_us_per_req,peak_mb,cli_util,prx_util,srv_util,cli_thread,srv_thread,bound +3proxy-ktls,tls-term-ka-1k,0,0.2457862144,24036.74,2.47,3.4,5.1,0,0,0,10.11,7.366099999999999,32.90664620773784,41.644183201453224,14.35546875,0.24370352602542575,0.9610266159695817,0.33099041533546325,0.6869054426962027,0.3108643663572742,proxy +3proxy-ktlsktls,tls-term-ka-1k,0,0.2382364672,23298.59,2.58,3.42,5.63,0,0,0,10.11,7.403414000000001,33.94946246079996,42.9647824775083,14.359375,0.23883448770002388,0.9601139601139601,0.3232291001914486,0.6516494036511804,0.2604665278299717,proxy +nginx,tls-term-ka-1k,0,0.35836133376,35053.31,1.66,2.37,2.73,0,0,0,10.12,6.72487,22.59172303846266,28.58434075245735,34.95703125,0.2853415559772296,0.9610636277302944,0.40861906259830133,0.8077320438403539,0.3527463144429704,proxy +nginx-ktls,tls-term-ka-1k,0,0.3615490048,35366.07,1.74,1.94,2.75,0,0,0,10.03,6.76899,22.19339534467445,28.30191341251097,34.671875,0.28222748815165877,0.9534220532319392,0.4021363493559535,0.793467369048492,0.35653754743203786,proxy +3proxy-ktls,tls-term-ka-1m,0,6.2058921984,739.54,21.05,32.18,48.49,0,0,0,10.01,7.450712,1.2903865784301922,1352.5199297392244,18.71875,0.2632208662359416,0.9515209125475285,0.06334699023006618,0.7592076451212405,0.03991725092070315,proxy +3proxy-ktlsktls,tls-term-ka-1m,0,5.245564354560001,625.0,24.32,39.46,58.41,0,0,0,10.01,7.534842,1.52662315410135,1599.8082147994246,19.48046875,0.24778761061946902,0.9515209125475285,0.05832282471626734,0.6983323706163362,0.0465850849452155,proxy +nginx,tls-term-ka-1m,0,6.467616768,770.81,20.43,22.55,29.03,0,0,0,10.03,7.48001,1.2406424634960682,1300.2333419756287,48.09765625,0.27010260081126225,0.9516129032258065,0.06163084702907712,0.7820670880647388,0.04563546333337103,proxy +nginx-ktls,tls-term-ka-1m,0,5.74602870784,684.73,22.72,25.67,34.96,0,0,0,10.0,7.4705900000000005,1.3922659295253146,1459.4279042615294,48.078125,0.272294887039239,0.9505703422053232,0.05776515151515151,0.7544410190996059,0.045605420334354344,proxy +3proxy-ktls,tls-term-close-1k,0,0.013254000640000001,1299.74,19.06,30.4,39.95,0,0,0,10.1,8.837986,609.6272528926029,769.3479585618526,24.4375,0.9508313539192399,0.9600760456273765,0.06834075015893197,3.694488345988044,0.030424887734638806,client +3proxy-ktlsktls,tls-term-close-1k,0,0.01249902592,1226.49,20.37,31.96,42.73,0,0,0,10.09,8.873475,645.8103256737626,814.4979011947046,27.06640625,0.9521542489883361,0.960952380952381,0.07201776649746193,3.7017749049670106,0.031373094578846926,client +nginx,tls-term-close-1k,0,0.014344519679999999,1405.34,3.11,3.62,4.45,0,0,0,10.1,8.736243000000002,563.2813213861476,711.5181401902079,34.59375,0.9536266349583828,0.9609895337773549,0.0853697238971755,3.7027041851169824,0.03707623722711079,client +nginx-ktls,tls-term-close-1k,0,0.013589544960000002,1333.28,3.08,3.71,4.54,0,0,0,10.11,8.794594,595.1634159794559,750.7239919803965,33.5546875,0.9559942911512845,0.9610266159695817,0.08370323398858592,3.700821018144337,0.03707794324198707,client +3proxy-ktls,tls-init-ka-1k,0,0.23160946688,22652.6,2.62,3.68,6.79,0,0,0,10.02,7.319382,34.60998424625362,44.191390176456835,15.625,0.17642822851656265,0.9515669515669516,0.36917677089980855,0.4347769414699658,0.3117179106077899,proxy +3proxy-ktlsktls,tls-init-ka-1k,0,0.23009951743999998,22502.45,2.61,3.78,7.19,0,0,0,10.09,7.437909,35.08047339605928,44.39516537089103,14.609375,0.17453847998081995,0.9600380589914367,0.3650590865538167,0.4309761658589577,0.3203970648366826,proxy +nginx,tls-init-ka-1k,0,0.38201720831999997,37364.33,1.64,1.88,2.55,0,0,0,10.1,6.709347,21.150879656792107,26.76397092531792,36.046875,0.242781520692974,0.9609895337773549,0.5025396825396825,0.6212415428983391,0.4515847557360016,proxy +nginx-ktls,tls-init-ka-1k,0,0.34133245951999996,33389.25,1.82,2.29,2.84,0,0,0,10.04,6.814094,23.531310240154216,30.03703770186744,35.87890625,0.22232870148538572,0.952561669829222,0.44952681388012616,0.5641593687858987,0.3812181740005725,proxy +3proxy-ktls,tls-init-ka-1m,0,5.658703298560001,674.32,23.28,33.69,50.74,0,0,0,10.02,7.2941069999999995,1.4165789540582332,1484.6643947251446,17.6484375,0.1928921568627451,0.9524714828897338,0.2849952815350739,0.49278450313989297,0.37537645662994545,proxy +3proxy-ktlsktls,tls-init-ka-1m,0,4.579592765439999,545.7,28.29,43.13,63.03,0,0,0,10.01,7.390827,1.7486270963725319,1832.6620285609667,20.48046875,0.1942270058708415,0.9515209125475285,0.24212846347607053,0.4747393731367436,0.21288886450756445,proxy +nginx,tls-init-ka-1m,0,5.6525796147200005,673.59,23.24,25.36,29.61,0,0,0,10.02,7.333513,1.4181135952734512,1486.646884272997,44.0859375,0.1987211018199705,0.9515669515669516,0.28970495919648465,0.5070504949764314,0.33652952634941036,proxy +nginx-ktls,tls-init-ka-1m,0,5.28968843264,629.82,24.65,26.88,35.29,0,0,0,10.01,7.526526,1.5138887860741796,1588.3846397968898,43.703125,0.18766887742569394,0.9524262607040913,0.2614173228346457,0.4766363628672231,0.3109040114409556,proxy +3proxy-ktls,tls-init-close-1k,0,0.009143582720000001,895.58,69.03,93.43,115.93,0,0,0,10.08,9.080478,881.9300100344037,1123.9964317573595,125.80078125,0.03732762719923918,0.9572649572649573,0.28056872037914693,0.06374846219722821,0.24150507384680797,proxy +3proxy-ktlsktls,tls-init-close-1k,0,0.00864026624,848.85,72.71,96.0,140.34,0,0,0,10.06,9.085586000000001,931.4527789365898,1182.4165491302304,104.78515625,0.03914590747330961,0.9580952380952381,0.271608832807571,0.06279600114982913,0.231044212904108,proxy +nginx,tls-init-close-1k,0,0.01283457024,1264.01,49.92,54.22,61.2,0,0,0,10.07,8.817834,627.6797624974469,795.7329118925326,31.7890625,0.059591642924976256,0.956315289648623,0.36294656971229844,0.09799429373220138,0.31089201678503625,proxy +nginx-ktls,tls-init-close-1k,0,0.01224736768,1206.15,50.88,60.14,75.62,0,0,0,10.07,8.853347,657.7739976856807,833.7473091571452,31.90234375,0.05529188419553868,0.9572243346007605,0.35003150598613736,0.09228161806410955,0.2966304309808959,proxy +3proxy-ktls,tls-term-stream1-down,0,7.326241354030307,nan,nan,nan,nan,0,0,0,9.54,7.589665,1.041622643370041,nan,8.25390625,0.3940137389597645,0.8940955951265229,0.07757272442915233,0.7667020731517917,0.1278124706894706,none +3proxy-ktlsktls,tls-term-stream1-down,0,5.4159560518101655,nan,nan,nan,nan,0,0,0,10.0,7.413972,1.4769619272204584,nan,7.19921875,0.38768115942028986,0.9852216748768473,0.06241787122207622,0.6790990527441514,0.0985724597839295,proxy +nginx,tls-term-stream1-down,0,5.97721596076857,nan,nan,nan,nan,0,0,0,9.84,7.605242,1.3169820045523122,nan,35.8828125,0.38231662138799705,0.9256820319849483,0.0653737879261808,0.7314692440545073,0.10337560034511603,proxy +nginx-ktls,tls-term-stream1-down,0,6.211716152979651,nan,nan,nan,nan,0,0,0,9.9,7.522898,1.2749570551192566,nan,34.78125,0.3942661112472433,0.9287054409005628,0.1061836352279825,0.7180720627272221,0.11653553164597577,proxy +3proxy-ktls,tls-term-stream16,0,6.568801767314768,nan,nan,nan,nan,0,0,270,10.06,7.4393959999999995,1.2192487527303417,nan,74.92578125,0.2557532621589561,0.9446009389671362,0.27978933508887427,0.6592878510575275,0.3650488234020122,proxy +3proxy-ktlsktls,tls-term-stream16,0,4.996096137054575,nan,nan,nan,nan,0,0,266,10.07,7.52867,1.605768689093688,nan,84.29296875,0.19139886578449905,0.943767572633552,0.22728731942215089,0.5367182522300772,0.3697141417519626,proxy +nginx,tls-term-stream16,0,6.40016860563706,nan,nan,nan,nan,0,0,229,10.11,7.532485,1.256117947935666,nan,81.65625,0.2351549562337355,0.948405253283302,0.2370731707317073,0.63914963562628,0.36026992248844736,proxy +nginx-ktls,tls-term-stream16,0,5.55587829662867,nan,nan,nan,nan,0,0,195,10.12,7.702363,1.4446528703573767,nan,88.6640625,0.20269631031220436,0.9484536082474226,0.1962066710268149,0.563219159062359,0.32740477096695647,proxy +3proxy-ktls,tls-init-stream1-down,0,6.434830876283748,nan,nan,nan,nan,0,0,0,10.0,7.54868,1.243098792852022,nan,10.97265625,0.15151515151515152,0.9514747859181731,0.29423076923076924,0.36539748482107154,0.6397111520845147,proxy +3proxy-ktlsktls,tls-init-stream1-down,0,4.84307179943234,nan,nan,nan,nan,0,0,0,10.0,7.3695189999999995,1.6516701011538795,nan,5.62890625,0.17727051966974258,0.9514747859181731,0.2652341244387428,0.3758380570093278,0.5600580723199883,proxy +nginx,tls-init-stream1-down,0,6.169006351308825,nan,nan,nan,nan,0,0,0,9.99,7.351503,1.2954694796748054,nan,37.5,0.19021872695994102,0.9496197718631179,0.3057692307692308,0.39202434886206305,0.6551788659843247,proxy +nginx-ktls,tls-init-stream1-down,0,5.314667326329944,nan,nan,nan,nan,0,0,0,10.02,7.556637,1.5081186270097653,nan,34.078125,0.16962332928311058,0.9515669515669516,0.2865384615384615,0.36723053041858233,0.6037309468165956,proxy +3proxy-ktls,tls-init-stream16,0,5.96382224105992,nan,nan,nan,nan,0,0,327,10.05,7.788124000000001,1.3444751015090743,nan,87.9375,0.06657156443176415,0.9553231939163498,0.533689126084056,0.12736755852122786,0.7408114591330676,proxy +3proxy-ktlsktls,tls-init-stream16,0,5.426779284900564,nan,nan,nan,nan,0,0,305,10.07,7.763995,1.4778879086214776,nan,85.28515625,0.053944866920152094,0.9572243346007605,0.518444666001994,0.11499779945340158,3.916028401009886,origin +nginx,tls-init-stream16,0,6.4309966621507595,nan,nan,nan,nan,0,0,444,10.12,7.873684,1.2470035638001487,nan,107.83203125,0.061103185924869234,0.9610636277302944,0.5270984798413747,0.1302282138548835,0.7284973297332885,proxy +nginx-ktls,tls-init-stream16,0,6.14237436446586,nan,nan,nan,nan,0,0,444,10.14,7.716718999999999,1.3087369831315978,nan,109.859375,0.05674264007597341,0.9620493358633776,0.5275016567263088,0.11883035256410401,0.7133755355572462,proxy +3proxy-ktlsktls,tls-term-ka-1k,1,0.21961375744,21479.29,2.76,3.83,6.3,0,0,0,10.1,7.475782000000001,36.79186629374761,46.55686621584869,14.35546875,0.2973102785782901,0.9609895337773549,0.3635490070467649,0.6735409684181647,0.28518851539199225,proxy +nginx,tls-term-ka-1k,1,0.3498049536,34217.11,1.71,2.39,2.73,0,0,0,10.03,6.722608,22.938497346653925,29.26023816608613,35.00390625,0.28332153809860816,0.9525166191832859,0.4057608015028178,0.7915593267281232,0.335582954819501,proxy +nginx-ktls,tls-term-ka-1k,1,0.34217132031999997,33468.56,1.77,2.39,3.03,0,0,0,10.03,6.836462,23.450241219795753,29.910120535101896,34.76171875,0.28588346755092375,0.9534220532319392,0.40564263322884014,0.7972588983298724,0.33369248074378616,proxy +3proxy-ktls,tls-term-ka-1k,1,0.24595398656,24053.62,2.47,3.41,5.42,0,0,0,10.1,7.372845,32.85167324591789,41.57524914276776,14.25390625,0.24557627929220469,0.9600760456273765,0.3308776425368354,0.6716783012431786,0.27570641446387933,proxy +3proxy-ktlsktls,tls-term-ka-1m,1,5.47784491008,652.67,23.53,37.28,57.53,0,0,0,10.03,7.464381,1.464809634393759,1535.517452541335,18.9296875,0.25949820788530464,0.9525166191832859,0.0645059786028949,0.7286331239376254,0.045623708957824034,proxy +nginx,tls-term-ka-1m,1,5.863636992,698.76,21.8,28.6,34.75,0,0,0,10.02,7.478353,1.3670696209428648,1432.8614328614328,47.96484375,0.26450226784435427,0.9515669515669516,0.0675888085507702,0.759060330841039,0.050379210917972546,proxy +nginx-ktls,tls-term-ka-1m,1,5.2458160128,625.12,24.42,31.05,36.39,0,0,0,10.02,7.497565,1.5280749420949267,1600.894711615274,46.984375,0.2568149210903874,0.9524714828897338,0.05932470810981382,0.7134721748668741,0.04277808533701262,proxy +3proxy-ktls,tls-term-ka-1m,1,5.914388070399999,704.76,21.64,33.68,52.06,0,0,0,10.03,7.467645,1.3566914961427825,1421.8882903317267,18.51171875,0.26227944682880305,0.9516129032258065,0.0705511811023622,0.7647636490503127,0.05415512157471166,proxy +3proxy-ktlsktls,tls-term-close-1k,1,0.01166016512,1143.41,21.92,34.72,49.5,0,0,0,10.1,8.853855,692.9575968131745,874.5345917395446,21.55859375,0.9517470881863561,0.9609895337773549,0.07389787503964479,3.6958147011123708,0.032319595320439314,client +nginx,tls-term-close-1k,1,0.01417674752,1391.3,3.24,3.83,4.67,0,0,0,10.1,8.728051,569.9473725268122,718.7588955308853,33.33984375,0.9560256714998812,0.9609895337773549,0.09082248332804065,3.6931150815491023,0.037074775053594015,client +nginx-ktls,tls-term-close-1k,1,0.01308622848,1281.5,3.13,4.0,4.89,0,0,0,10.13,8.808627000000001,619.2769759740585,782.6019777503091,32.34375,0.9583927722301474,0.9620132953466287,0.12797713559860274,3.708028163909508,0.035171242217625215,client +3proxy-ktls,tls-term-close-1k,1,0.0130023424,1275.78,17.65,29.38,41.5,0,0,0,10.05,8.816282000000001,618.3501212827622,785.4016880275086,45.3671875,0.948005698005698,0.9544159544159544,0.3360785555907507,3.3905999840908603,0.031351852271316726,client +3proxy-ktlsktls,tls-init-ka-1k,1,0.23060283392,22557.81,2.61,3.78,6.7,0,0,0,10.11,7.424079000000001,35.073289701226585,44.374607717055916,14.6171875,0.17833173537871524,0.9601139601139601,0.3781190019193858,0.435729746430215,0.3089820479266111,proxy +nginx,tls-init-ka-1k,1,0.3640655872,35604.92,1.7,2.15,2.84,0,0,0,10.1,6.7278519999999995,22.193803215905817,28.086606859806118,35.98828125,0.2426134998798943,0.9609895337773549,0.49428208386277,0.6126851892407387,0.4829829072123885,proxy +nginx-ktls,tls-init-ka-1k,1,0.33999028224,33253.3,1.81,2.35,3.05,0,0,0,10.11,6.852244,23.78891522049639,30.102455538583843,35.859375,0.23403744599135862,0.9601139601139601,0.46999368288060644,0.5793820609414128,0.4458623866675235,proxy +3proxy-ktls,tls-init-ka-1k,1,0.26717716480000003,26134.64,2.33,2.93,6.62,0,0,0,10.11,7.349978,30.27204815970859,38.30198329260669,15.8203125,0.18611378977820636,0.9601139601139601,0.3950814436282338,0.47191423723577763,0.37364927458346225,proxy +3proxy-ktlsktls,tls-init-ka-1m,1,4.50308866048,536.35,28.67,42.7,61.61,0,0,0,10.01,7.3675559999999995,1.7783349615742188,1863.7125302550735,18.11328125,0.18737769080234834,0.9524262607040913,0.23508993373303883,0.4642601707361724,0.347003670336928,proxy +nginx,tls-init-ka-1m,1,5.55015471104,661.37,23.1,27.57,35.12,0,0,0,9.99,7.3662730000000005,1.4399598598761296,1509.2914337513218,45.5,0.19415807560137457,0.950523311132255,0.28926660371419577,0.4956169180224298,0.30029415049951824,proxy +nginx-ktls,tls-init-ka-1m,1,5.0720879411199995,604.19,25.13,30.51,40.83,0,0,0,10.0,7.4896199999999995,1.5772597188513,1653.9861065167051,46.95703125,0.19984567901234568,0.998003992015968,0.2750413223140496,0.4713519324483768,0.2543528057458968,proxy +3proxy-ktls,tls-init-ka-1m,1,5.65207629824,673.58,23.33,34.15,52.86,0,0,0,10.01,7.2987910000000005,1.4168244689997571,1485.1632047477744,18.26171875,0.19680981595092026,0.9515209125475285,0.29421019509125235,0.49652863675975717,0.3450518345228662,proxy +3proxy-ktlsktls,tls-init-close-1k,1,0.00813694976,801.72,75.27,105.3,163.29,0,0,0,10.09,9.114457,992.0179229421715,1256.53798256538,168.9609375,0.039220347040646544,0.9591254752851711,0.2719797596457938,0.06469530737377825,0.2338898028187404,proxy +nginx,tls-init-close-1k,1,0.01249902592,1227.05,49.97,59.06,68.87,0,0,0,10.06,8.827727,643.8901760434144,818.5516680227828,31.7734375,0.057915974365060525,0.9562737642585551,0.3634069400630915,0.09894200273943439,0.3108730541498211,proxy +nginx-ktls,tls-init-close-1k,1,0.0121634816,1195.68,51.62,60.84,71.24,0,0,0,10.06,8.820988999999999,661.6526636584051,839.7328881469115,31.84765625,0.05738676784443918,0.9553656220322887,0.35394321766561515,0.09608917499029922,0.3032919945014139,proxy +3proxy-ktls,tls-init-close-1k,1,0.00905969664,888.73,68.42,95.59,153.19,0,0,0,10.06,9.036776,888.3299650969328,1129.9561945411658,52.7109375,0.039047619047619046,0.9571836346336822,0.29094351530451246,0.06850511081118076,0.2472120812758378,proxy +3proxy-ktlsktls,tls-term-stream1-down,1,6.00483423494679,nan,nan,nan,nan,0,0,0,9.89,7.52233,1.3175029581125264,nan,8.1484375,0.40587931923671994,0.9734251968503937,0.0761904761904762,0.7200519659586874,0.11534964064004868,proxy +nginx,tls-term-stream1-down,1,5.728632557380473,nan,nan,nan,nan,0,0,0,9.9,7.579065,1.382516168970677,nan,35.91015625,0.3825136612021858,0.9295774647887324,0.06443540819518298,0.7523866867179478,0.09581102658879728,proxy +nginx-ktls,tls-term-stream1-down,1,6.699772201367511,nan,nan,nan,nan,0,0,0,9.68,7.662923,1.1557517850156496,nan,35.42578125,0.38246131171702286,0.9080675422138836,0.07084893882646692,0.7419127075964321,0.10709276695794784,proxy +3proxy-ktls,tls-term-stream1-down,1,7.10911244752434,nan,nan,nan,nan,0,0,0,9.45,7.604518,1.0633106455322245,nan,8.17578125,0.39267399267399267,0.8889934148635936,0.07675438596491228,0.7704737759364104,0.1212659200888465,none +3proxy-ktlsktls,tls-term-stream16,1,4.75532178262193,nan,nan,nan,nan,0,0,256,10.05,7.516392,1.68565557278041,nan,79.5546875,0.1870708027468624,0.9436619718309859,0.22568218298555376,0.5235015126095496,0.3621734595124635,proxy +nginx,tls-term-stream16,1,6.3077413504402235,nan,nan,nan,nan,0,0,180,10.09,7.512531,1.2706422623186402,nan,90.05078125,0.22453320727960294,0.9456419868791003,0.2261356155365372,0.6293042786467311,0.36930598027103306,proxy +nginx-ktls,tls-term-stream16,1,5.858098665468214,nan,nan,nan,nan,0,0,195,10.13,7.701568,1.3691255221804195,nan,91.69921875,0.21438712730714624,0.950281425891182,0.21447282252783234,0.5881806293198875,0.35200258673775897,proxy +3proxy-ktls,tls-term-stream16,1,6.56675110842816,nan,nan,nan,nan,0,0,286,10.07,7.467701,1.2224609437154916,nan,84.51171875,0.22857142857142856,0.943767572633552,0.2257001647446458,0.6439104670085051,0.3706590701640415,proxy +3proxy-ktlsktls,tls-init-stream1-down,1,4.999928236277779,nan,nan,nan,nan,0,0,0,10.01,7.362375,1.601456115958155,nan,5.00390625,0.1593460224499756,0.9515209125475285,0.2501601537475977,0.3615749124417473,0.530579972459699,proxy +nginx,tls-init-stream1-down,1,6.21236151993428,nan,nan,nan,nan,0,0,0,10.01,7.3283130000000005,1.2889056784211925,nan,33.4375,0.17905733722060252,0.9506172839506173,0.31183830606352264,0.3919184656901152,0.6711732268399877,proxy +nginx-ktls,tls-init-stream1-down,1,5.849495579614582,nan,nan,nan,nan,0,0,0,10.0,7.639269,1.3674955694018749,nan,34.78515625,0.16048780487804878,0.9514747859181731,0.26957637997432604,0.36253944476917865,0.582919688342687,proxy +3proxy-ktls,tls-init-stream1-down,1,6.109701578298123,nan,nan,nan,nan,0,0,0,10.0,7.296709000000001,1.3092503442846601,nan,5.1953125,0.17835209825997952,0.9970089730807578,0.31214261688530104,0.38136174272884515,0.6564313304737873,proxy +3proxy-ktlsktls,tls-init-stream16,1,6.3697061225774565,nan,nan,nan,nan,0,0,337,10.08,7.699675999999999,1.2610569473140496,nan,88.97265625,0.05564803804994055,0.9563567362428842,0.5225314777998674,0.12547520008789753,0.72084848211206,proxy +nginx,tls-init-stream16,1,6.929013873941702,nan,nan,nan,nan,0,0,543,10.1,7.900086999999999,1.15643727230689,nan,108.3125,0.05973346025702047,0.9591642924976258,0.5443414956982131,0.13877893298646177,0.7690248379685674,proxy +nginx-ktls,tls-init-stream16,1,6.209615814621383,nan,nan,nan,nan,0,0,465,10.13,7.711684,1.2917344955217616,nan,110.609375,0.0544589774078478,0.9620132953466287,0.5221414408460013,0.12071221020120756,0.6999448137173695,proxy +3proxy-ktls,tls-init-stream16,1,7.195507217636038,nan,nan,nan,nan,0,0,363,10.06,7.840883,1.1146051261163956,nan,94.54296875,0.05920114122681883,0.9553656220322887,0.5397614314115308,0.1349855929752782,0.7709270326820198,proxy +nginx,tls-term-ka-1k,2,0.38335938560000005,37495.77,1.64,1.87,2.38,0,0,0,10.01,6.648461999999999,20.889015114281317,26.654382395858853,34.83203125,0.2953846153846154,0.9524262607040913,0.4280778894472362,0.8323546782540214,0.3783506287504259,proxy +nginx-ktls,tls-term-ka-1k,2,0.33076281344,32354.82,1.81,2.56,2.8,0,0,0,10.1,6.884835000000001,24.428380917329758,30.90726596161379,35.01171875,0.27138783269961975,0.9609895337773549,0.3845425867507886,0.7629774073679495,0.340368902781251,proxy +3proxy-ktls,tls-term-ka-1k,2,0.2541748224,24862.13,2.44,3.12,4.79,0,0,0,10.02,7.312456,31.537348681157177,40.23627770259689,14.35546875,0.24633853541416567,0.9524714828897338,0.3274364747507237,0.7097291248630052,0.252870205064721,proxy +3proxy-ktlsktls,tls-term-ka-1k,2,0.23462936575999999,22945.33,2.6,3.52,6.17,0,0,0,10.11,7.454905,34.47138841211007,43.623842522674906,14.171875,0.2314148681055156,0.9610266159695817,0.30702038175347784,0.6498218563615207,0.30520987384755,proxy +nginx,tls-term-ka-1m,2,5.73629792256,683.64,22.03,28.44,39.6,0,0,0,10.03,7.4726539999999995,1.398811586902209,1465.9456299327683,47.97265625,0.2509587727708533,0.9516129032258065,0.061533606816030297,0.7239849066368242,0.0389754298570397,proxy +nginx-ktls,tls-term-ka-1m,2,5.928397045760001,706.5,22.28,24.5,29.2,0,0,0,10.01,7.443725,1.3507867199494228,1415.8415841584158,47.78515625,0.2717313646106216,0.9515209125475285,0.058804932026557066,0.7639350926038432,0.04848339372485036,proxy +3proxy-ktls,tls-term-ka-1m,2,6.19616141312,738.38,21.01,31.92,46.19,0,0,0,10.03,7.438596,1.294995314842777,1357.4231966436596,19.66015625,0.2596936333173767,0.9525166191832859,0.06731984829329962,0.7468570159727564,0.04563622986360491,proxy +3proxy-ktlsktls,tls-term-ka-1m,2,5.7550045184,685.66,22.51,34.69,53.63,0,0,0,10.02,7.460329000000001,1.3928746666264302,1460.2156805596037,18.1796875,0.2615421227986673,0.9524714828897338,0.06188822229239027,0.7363883132248247,0.04373131024535983,proxy +nginx,tls-term-close-1k,2,0.01451229184,1422.99,3.07,3.63,4.96,0,0,0,10.1,8.771161999999999,556.7693985955564,702.7553576398552,33.34765625,0.9557984790874525,0.9619047619047619,0.08632180260234847,3.7026834407676232,0.03802740299572837,client +nginx-ktls,tls-term-close-1k,2,0.0130023424,1280.61,3.11,4.01,4.82,0,0,0,10.12,8.784732,622.6570375504032,782.4338951600433,33.2890625,0.9555924958442176,0.9610636277302944,0.07933989209774675,3.711139343405604,0.034223106633319705,client +3proxy-ktls,tls-term-close-1k,2,0.01291845632,1270.86,19.58,31.09,42.93,0,0,0,10.12,8.846919,626.7002650669643,788.4076036148333,34.671875,0.9508079847908745,0.9610636277302944,0.06959008579599618,3.6932432487054543,0.03137231948080242,client +3proxy-ktlsktls,tls-term-close-1k,2,0.01266679808,1244.14,20.16,31.74,40.8,0,0,0,10.11,8.867838,638.5196913157077,804.5519656215184,23.0859375,0.9508313539192399,0.9610266159695817,0.07369758576874205,3.6951548021815617,0.030423396028380782,client +nginx,tls-init-ka-1k,2,0.3665821696,35852.81,1.67,2.22,2.77,0,0,0,10.1,6.762233999999999,22.041443010762297,27.892462131149806,36.0859375,0.23610108303249097,0.9609895337773549,0.4771283354510801,0.6012914718531616,0.4658734370288612,proxy +nginx-ktls,tls-init-ka-1k,2,0.34049359872,33307.17,1.78,2.48,3.03,0,0,0,10.11,6.881964,23.753750526896244,30.054222454755166,35.7734375,0.22077922077922077,0.9601139601139601,0.45764854614412137,0.5584748551100605,0.4554346998720907,proxy +3proxy-ktls,tls-init-ka-1k,2,0.26264731648,25691.65,2.36,3.04,7.3,0,0,0,10.09,7.369860999999999,30.733228529348654,38.88501366178131,15.80078125,0.18736435977815288,0.960952380952381,0.4001925545571245,0.4737448256963625,0.36599912009190644,proxy +3proxy-ktlsktls,tls-init-ka-1k,2,0.24595398656,24056.4,2.53,3.19,8.32,0,0,0,10.1,7.420751,32.85167324591789,41.56874688748863,14.60546875,0.1702487321902922,0.9600760456273765,0.3680488588878174,0.4338577692509546,0.3308732342298923,proxy +nginx,tls-init-ka-1m,2,5.6169280307200005,669.34,22.92,26.78,33.17,0,0,0,10.01,7.368054,1.425690333969528,1494.4759629740222,44.40234375,0.19276218611521417,0.9515209125475285,0.28324426281043696,0.49279170121925175,0.37345223426578195,proxy +nginx-ktls,tls-init-ka-1m,2,5.02712500224,598.43,25.04,35.94,38.7,0,0,0,10.01,7.52804,1.592958201045682,1671.3975621973618,43.8203125,0.1784313725490196,0.9515209125475285,0.24794952681388013,0.4519062374146436,0.2680956460995401,proxy +3proxy-ktls,tls-init-ka-1m,2,5.66080045056,674.57,23.34,33.65,51.51,0,0,0,10.02,7.317829,1.416054155240008,1482.906615361847,20.2734375,0.19027743677878714,0.9533777354900095,0.279508970727101,0.4890233400671784,0.3650858996852706,proxy +3proxy-ktlsktls,tls-init-ka-1m,2,4.730503823359999,563.77,27.76,40.39,57.93,0,0,0,10.02,7.371729,1.6945340917844063,1775.6512493354599,18.31640625,0.18641157714005396,0.9524714828897338,0.24179292929292928,0.46996422622958617,0.2528054578622284,proxy +nginx,tls-init-close-1k,2,0.013170114560000001,1296.0,48.28,53.27,57.85,0,0,0,10.05,8.844591,610.4730496740644,773.9699653446285,31.69140625,0.05608365019011407,0.9571428571428572,0.3623142586152387,0.09799864620856737,0.31185763008399014,proxy +nginx-ktls,tls-init-close-1k,2,0.01266679808,1245.59,49.87,53.74,77.81,0,0,0,10.07,8.868394,635.9934017358237,807.279140612474,32.078125,0.05592574964302713,0.9572243346007605,0.34746835443037977,0.0960971680799794,0.29665710009939406,proxy +3proxy-ktls,tls-init-close-1k,2,0.009143582720000001,894.31,67.88,92.34,117.45,0,0,0,10.06,9.042095,880.180148903383,1120.5168188906216,50.93359375,0.040646541478488236,0.9562737642585551,0.2812796959138423,0.06755046977307559,0.24149531835757837,proxy +3proxy-ktlsktls,tls-init-close-1k,2,0.00855638016,841.38,73.51,99.77,124.73,0,0,0,10.09,9.095251,943.3895933861827,1197.1998101566207,135.15234375,0.037089871611982884,0.9591254752851711,0.2691334598355471,0.06279002027592046,0.2300827892822143,proxy +nginx,tls-term-stream1-down,2,6.400770580410245,nan,nan,nan,nan,0,0,0,9.82,7.57843,1.2272216464950285,nan,35.1640625,0.38095238095238093,0.9220657276995305,0.07362155388471178,0.7333625873295194,0.11461207390467829,proxy +nginx-ktls,tls-term-stream1-down,2,6.756428635593354,nan,nan,nan,nan,0,0,0,9.57,7.510962999999999,1.1330256461579196,nan,35.1484375,0.3834678405478112,0.8994360902255639,0.0728125,0.7209993256000874,0.11650758055535027,none +3proxy-ktls,tls-term-stream1-down,2,6.815663635015595,nan,nan,nan,nan,0,0,0,9.66,7.561722,1.1337407496519554,nan,8.51953125,0.41226708074534163,0.9489194499017681,0.07536041939711664,0.7699341022114989,0.11526923178558898,proxy +3proxy-ktlsktls,tls-term-stream1-down,2,6.663556120601091,nan,nan,nan,nan,0,0,1,9.49,7.486928,1.1392889821019732,nan,8.21484375,0.4055926115956901,0.9340551181102362,0.08144499178981937,0.7410934543166012,0.12323287058562432,proxy +nginx,tls-term-stream16,2,6.172591176525737,nan,nan,nan,nan,0,0,191,10.1,7.520776,1.298653174558868,nan,92.5625,0.2277882797731569,0.9465791940018744,0.2220766459220439,0.6427636686469156,0.3554208493193295,proxy +nginx-ktls,tls-term-stream16,2,5.741620190831843,nan,nan,nan,nan,0,0,195,10.13,7.711811,1.395881420368857,nan,98.77734375,0.2251358374675171,0.9493908153701968,0.2167374959136973,0.6208687502737305,0.3495317445389246,proxy +3proxy-ktls,tls-term-stream16,2,5.993015627067362,nan,nan,nan,nan,0,0,297,10.06,7.384076,1.3379536125577445,nan,79.4453125,0.22734800094629762,0.9419475655430711,0.2235371466140697,0.6362054532193276,0.365434158023906,proxy +3proxy-ktlsktls,tls-term-stream16,2,5.233112783786238,nan,nan,nan,nan,0,0,258,10.06,7.519772,1.5326738619401123,nan,77.71484375,0.19900379506641366,0.9437148217636022,0.23743556701030927,0.5604669236236561,0.3847769384524135,proxy +nginx,tls-init-stream1-down,2,5.965662608428549,nan,nan,nan,nan,0,0,0,10.01,7.420767000000001,1.342207055621584,nan,34.61328125,0.16907620681205587,0.9515209125475285,0.2941552219738103,0.3643184788245149,0.6197736902418614,proxy +nginx-ktls,tls-init-stream1-down,2,5.576892797099186,nan,nan,nan,nan,0,0,0,10.0,7.557824,1.4343393677971839,nan,33.78125,0.1704338789295921,0.998003992015968,0.27682475613858054,0.33449215744528493,0.5924306668835934,proxy +3proxy-ktls,tls-init-stream1-down,2,6.831835674522873,nan,nan,nan,nan,0,0,0,9.98,7.321784,1.1685829215874735,nan,6.44140625,0.16812865497076024,0.9504761904761905,0.30816,0.3929992408167308,0.6637291202283812,proxy +3proxy-ktlsktls,tls-init-stream1-down,2,5.18990574090663,nan,nan,nan,nan,0,0,0,10.01,7.520538,1.5428331163194444,nan,10.35546875,0.18314306533883895,0.9506172839506173,0.25376,0.40340246831146215,0.5533321126146946,proxy +nginx,tls-init-stream16,2,5.962113739474437,nan,nan,nan,nan,0,0,358,10.13,7.839964999999999,1.3452466728440324,nan,114.40234375,0.05110530068932731,0.961100569259962,0.5100334448160535,0.11408064977195043,0.699177044184028,proxy +nginx-ktls,tls-init-stream16,2,5.76603490238921,nan,nan,nan,nan,0,0,369,10.12,7.792174999999999,1.3894342647204374,nan,113.16015625,0.052093244529019984,0.9619771863117871,0.5156921043937892,0.1121588081097605,0.7018845929866626,proxy +3proxy-ktls,tls-init-stream16,2,7.23540163258609,nan,nan,nan,nan,0,0,351,10.07,7.8539520000000005,1.109910472835705,nan,91.09765625,0.06349583828775268,0.9545023696682464,0.5584112149532711,0.13782358536257486,0.8263187067194244,proxy +3proxy-ktlsktls,tls-init-stream16,2,6.50055551400149,nan,nan,nan,nan,0,0,282,10.08,7.7149339999999995,1.235509629287493,nan,102.67578125,0.05679657794676806,0.9572649572649573,0.528377032857617,0.12073701272134864,0.7333551591243483,proxy