diff --git a/bench/README.md b/bench/README.md index 95790ca..ca03c81 100644 --- a/bench/README.md +++ b/bench/README.md @@ -89,6 +89,7 @@ host) falls on all of them alike; the summary gives medians. | `mix-*-3k` | the same, exchanges started at 3000 a second whether the proxy keeps up or not: latencies compared at equal load | | `tls-term-ka-1k`, `tls-term-ka-1m`, `tls-term-close-1k` | TLS ended by the proxy: wrk speaks HTTPS to a relay of the proxy, which passes plain TCP to the origin; kept-alive connections with 1 KB and 1 MB answers, and a new connection, so a full handshake, for every request | | `tls-init-*` | TLS begun by the proxy: wrk speaks plain HTTP to a relay, which connects to the origin's HTTPS listener | +| `tls-term-stream1-down`, `tls-term-stream16`, `tls-init-stream1-down`, `tls-init-stream16` | the same with iperf3 streams, one downloading or sixteen uploading | ### The mix scenarios @@ -146,10 +147,11 @@ answer, and `peak_mb` divided by the pool what each open connection costs the proxy. The binaries are built with `bench/mix/build.sh`, on any system with Go, for both Linux architectures. -nginx is not a forward proxy: it answers absolute URIs by passing them to its -upstream, which is the origin here. Neither nginx nor haproxy has CONNECT or -SOCKS, so they take part in the HTTP and `relay-*` scenarios but not in the -`connect-*` and `socks-*` ones. tinyproxy +nginx and envoy are not forward proxies: they answer absolute URIs by passing +them to their upstream, which is the origin here. nginx and haproxy have +neither CONNECT nor SOCKS, and envoy is measured as they are, so they take +part in the HTTP and `relay-*` scenarios but not in the `connect-*` and +`socks-*` ones. tinyproxy and squid are HTTP proxies, measured in the HTTP and `connect-*` scenarios; dante is a SOCKS server, measured in the `socks-*` ones. @@ -160,10 +162,18 @@ The `tls-*` scenarios compare TLS done in the proxy: 3proxy (`tcppm` under `proxy_ssl on`), haproxy (`mode tcp` with `bind ... ssl crt` and `server ... ssl`) and stunnel, which takes part in these scenarios only. Every one relays TCP: no HTTP is read, so what differs is the TLS and how data is moved around -it. The certificate is made for every run: an ECDSA P-256 key signed by a CA +it. envoy takes part too, with BoringSSL where the others use OpenSSL. The certificate is made for every run: an ECDSA P-256 key signed by a CA of its own, with the proxy's and the origin's addresses in it; nobody verifies it. `none` has wrk speak HTTPS to the origin itself. +iperf3 has no TLS of its own, so for the TLS streams `bench/mix tls` wraps +its connections: on the client it begins TLS towards the proxy ending it (or +towards the origin with no proxy), at the origin it ends the TLS a proxy +begins and passes the stream to iperf3. The wrapper is in Go, a goroutine +for each direction of each connection: on one stream it, rather than the +proxy, can be the limit (about 10 Gbit/s a core), which the run's `bound` +shows. + wrk does not resume TLS sessions, so in `tls-term-close-1k` every connection is a full handshake. Towards the origin a proxy may resume them: nginx (`proxy_ssl_session_reuse`) and haproxy do by default, so `tls-init-close-1k` @@ -194,7 +204,9 @@ sudo bench/bench.py \ `bench/report.py ` turns a run into Markdown tables: throughput, the proxy's CPU per gigabyte or per request, and its peak memory. With `--machine NAME=a.csv,b.csv` once per machine it makes a single throughput -table of all of them, each cell their medians and their average. +table of all of them, each cell their medians and their average; `--overall` +then marks only the best of each row, for tables where one group has a +single proxy. After the binary, a 3proxy entry can carry options for both its services: `-s1`, `-s2` and `-s3` splice downloads, uploads or both (`-s0` keeps 0.9.0 to @@ -212,7 +224,12 @@ variant; `nginx::halfclose` turns on `proxy_half_close` in its relays, which nginx leaves off: then a client shutting down its sending side still gets the answer, as through 3proxy and haproxy. -`--proxy stunnel` measures stunnel in the `tls-*` scenarios. +`--proxy stunnel` measures stunnel in the `tls-*` scenarios. `--proxy envoy` +measures envoy where nginx and haproxy are measured: as an HTTP proxy for the +origin (its connections there pooled), as a TCP relay (`tcp_proxy`) and in +the TLS scenarios; its circuit breakers, which allow 1024 connections to a +cluster by default, are lifted for the scenarios that open more. The +official release binaries run as they are (`envoy` on the PATH). `--scenario` picks scenarios, `--reps` and `--duration` set the repetitions and their length, `--netem "delay 1ms rate 10gbit"` puts a slower, longer link diff --git a/bench/bench.py b/bench/bench.py index 04a7647..e8ca288 100644 --- a/bench/bench.py +++ b/bench/bench.py @@ -51,6 +51,10 @@ MIX_PORTS = 16 # (TLS from the client, plain to ORIGIN_HTTP) and initiate it (plain from the # client, TLS to ORIGIN_TLS). TLS holds the certificate files once made. ORIGIN_TLS, TLS_TERM, TLS_INIT = 8443, 5443, 5444 +# the same for TLS streams: iperf3 has no TLS, so bench/mix wraps its +# connections, on the client (CLI_TLS, for TLS ended by the proxy) and at the +# origin (ORIGIN_TLS_TCP, taking the TLS begun by the proxy) +ORIGIN_TLS_TCP, TLS_TERM_TCP, TLS_INIT_TCP, CLI_TLS = 8444, 5445, 5446, 5202 TLS = None # Each measurement takes the next of SLOTS sets of addresses: the proxy's to # connect from, and the mix client's to connect from. What a run leaves in @@ -107,6 +111,10 @@ SCENARIOS = { "tls-init-ka-1k": ("http", {"file": "1k", "conns": 64, "via": "tls-init"}), "tls-init-ka-1m": ("http", {"file": "1m", "conns": 16, "via": "tls-init"}), "tls-init-close-1k": ("http", {"file": "1k", "conns": 64, "close": True, "via": "tls-init"}), + "tls-term-stream1-down": ("stream", {"streams": 1, "reverse": True, "via": "tls-term"}), + "tls-term-stream16": ("stream", {"streams": 16, "via": "tls-term"}), + "tls-init-stream1-down": ("stream", {"streams": 1, "reverse": True, "via": "tls-init"}), + "tls-init-stream16": ("stream", {"streams": 16, "via": "tls-init"}), # a pool of open connections and exchanges on random ones of them, sizes # and pieces from a web-like profile (bench/mix): -active of them going # at once all the time, or started at a fixed rate whatever the proxy @@ -445,6 +453,11 @@ http {{ procs = [spawn("bsrv", "srv", [tool("nginx"), "-c", conf, "-p", work], log), spawn("bsrv", "srv", [tool("iperf3"), "-s", "-B", SRV_ADDR, "-p", str(ORIGIN_TCP)], log)] ports = [ORIGIN_HTTP, ORIGIN_TCP] + ([ORIGIN_TLS] if TLS else []) + if args.wrap: + procs.append(spawn("bsrv", "srv", [args.wrap, "tls", "-listen", f"{SRV_ADDR}:{ORIGIN_TLS_TCP}", + "-connect", f"{SRV_ADDR}:{ORIGIN_TCP}", + "-cert", TLS["cert"], "-key", TLS["key"]], log)) + ports.append(ORIGIN_TLS_TCP) if args.mix: procs.append(spawn("bsrv", "srv", [args.mix, "origin", "-listen", f"{SRV_ADDR}:{ORIGIN_MIX}", "-ports", str(MIX_PORTS)], log)) @@ -458,6 +471,64 @@ http {{ # --- proxies ----------------------------------------------------------------- +def envoy_conf(work, label, out): + """envoy as nginx and haproxy are measured: an HTTP proxy for the origin + (absolute URIs taken, the origin's connections pooled), TCP relays, and + the TLS ones. Its default circuit breakers allow 1024 connections to a + cluster, far fewer than the scenarios open: they are lifted.""" + def addr(host, port): + return {"socket_address": {"address": host, "port_value": port}} + + def cluster(name, port, tls=False): + c = {"name": name, "type": "STATIC", "connect_timeout": "5s", + "upstream_bind_config": {"source_address": {"address": out, "port_value": 0}}, + "circuit_breakers": {"thresholds": [{"max_connections": 1000000, + "max_pending_requests": 1000000, + "max_requests": 1000000}]}, + "load_assignment": {"cluster_name": name, "endpoints": [{"lb_endpoints": [ + {"endpoint": {"address": addr(SRV_ADDR, port)}}]}]}} + if tls: + c["transport_socket"] = {"name": "envoy.transport_sockets.tls", "typed_config": { + "@type": "type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext"}} + return c + + def http(name, port, dest): + return {"name": name, "address": addr(PRX_IN, port), "filter_chains": [{"filters": [{ + "name": "envoy.filters.network.http_connection_manager", "typed_config": { + "@type": "type.googleapis.com/envoy.extensions.filters.network." + "http_connection_manager.v3.HttpConnectionManager", + "stat_prefix": name, "codec_type": "HTTP1", + "route_config": {"virtual_hosts": [{"name": "all", "domains": ["*"], "routes": [ + {"match": {"prefix": "/"}, "route": {"cluster": dest, "timeout": "0s"}}]}]}, + "http_filters": [{"name": "envoy.filters.http.router", "typed_config": { + "@type": "type.googleapis.com/envoy.extensions.filters.http.router.v3.Router"}}]}}]}]} + + def tcp(name, port, dest, tls=False): + chain = {"filters": [{"name": "envoy.filters.network.tcp_proxy", "typed_config": { + "@type": "type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy", + "stat_prefix": name, "cluster": dest}}]} + if tls: + chain["transport_socket"] = {"name": "envoy.transport_sockets.tls", "typed_config": { + "@type": "type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext", + "common_tls_context": {"tls_certificates": [{ + "certificate_chain": {"filename": TLS["cert"]}, + "private_key": {"filename": TLS["key"]}}]}}} + return {"name": name, "address": addr(PRX_IN, port), "filter_chains": [chain]} + + listeners = [http("http", HTTP_PORT, "origin"), http("httpmix", HTTP_MIX, "mix"), + tcp("tcp", TCP_PORT, "tcp"), tcp("tcphttp", RELAY_HTTP, "origin"), + tcp("tcpmix", RELAY_MIX, "mix")] + clusters = [cluster("origin", ORIGIN_HTTP), cluster("mix", ORIGIN_MIX), cluster("tcp", ORIGIN_TCP)] + if TLS: + listeners += [tcp("tlsterm", TLS_TERM, "origin", True), tcp("tlstermtcp", TLS_TERM_TCP, "tcp", True), + tcp("tlsinit", TLS_INIT, "origintls"), tcp("tlsinittcp", TLS_INIT_TCP, "tcptls")] + clusters += [cluster("origintls", ORIGIN_TLS, True), cluster("tcptls", ORIGIN_TLS_TCP, True)] + conf = os.path.join(work, f"{label}.json") + with open(conf, "w") as f: + json.dump({"static_resources": {"listeners": listeners, "clusters": clusters}}, f, indent=1) + return conf + + def proxy_conf(kind, work, ncpu, label, opts="", out=PRX_OUT): """Configuration for one proxy: an HTTP proxy on HTTP_PORT and a TCP relay on TCP_PORT, both towards the origin. Returns the argv. opts are added to @@ -479,12 +550,17 @@ socks {opts} -i{PRX_IN} -e{out} -p{SOCKS_PORT} ssl_server_key {TLS["key"]} ssl_serv tcppm {opts} -i{PRX_IN} -e{out} {TLS_TERM} {SRV_ADDR} {ORIGIN_HTTP} +tcppm {opts} -i{PRX_IN} -e{out} {TLS_TERM_TCP} {SRV_ADDR} {ORIGIN_TCP} ssl_noserv ssl_cli tcppm {opts} -i{PRX_IN} -e{out} {TLS_INIT} {SRV_ADDR} {ORIGIN_TLS} +tcppm {opts} -i{PRX_IN} -e{out} {TLS_INIT_TCP} {SRV_ADDR} {ORIGIN_TLS_TCP} ssl_nocli """) return [conf] + if kind == "envoy": + return ["-c", envoy_conf(work, label, out), "--concurrency", str(ncpu), + "--log-level", "error", "--disable-hot-restart"] if kind == "stunnel": conf = os.path.join(work, f"{label}.conf") with open(conf, "w") as f: @@ -503,6 +579,17 @@ client = yes accept = {PRX_IN}:{TLS_INIT} connect = {SRV_ADDR}:{ORIGIN_TLS} local = {out} +[termtcp] +accept = {PRX_IN}:{TLS_TERM_TCP} +connect = {SRV_ADDR}:{ORIGIN_TCP} +local = {out} +cert = {TLS["cert"]} +key = {TLS["key"]} +[inittcp] +client = yes +accept = {PRX_IN}:{TLS_INIT_TCP} +connect = {SRV_ADDR}:{ORIGIN_TLS_TCP} +local = {out} """) return [conf] if kind == "tinyproxy": @@ -565,12 +652,15 @@ socks pass {{ from: 0.0.0.0/0 to: 0.0.0.0/0 }} halfclose = " proxy_half_close on;\n" if opts == "halfclose" else "" nginx_tls = "" if TLS: - nginx_tls = (f" server {{\n listen {PRX_IN}:{TLS_TERM} ssl backlog=65535 reuseport;\n" - f" ssl_certificate {TLS['cert']};\n ssl_certificate_key {TLS['key']};\n" - f" proxy_pass {SRV_ADDR}:{ORIGIN_HTTP};\n proxy_bind {out};\n }}\n" - f" server {{\n listen {PRX_IN}:{TLS_INIT} backlog=65535 reuseport;\n" - f" proxy_pass {SRV_ADDR}:{ORIGIN_TLS};\n proxy_ssl on;\n" - f" proxy_bind {out};\n }}\n") + nginx_tls = "" + for port, dest in ((TLS_TERM, ORIGIN_HTTP), (TLS_TERM_TCP, ORIGIN_TCP)): + nginx_tls += (f" server {{\n listen {PRX_IN}:{port} ssl backlog=65535 reuseport;\n" + f" ssl_certificate {TLS['cert']};\n ssl_certificate_key {TLS['key']};\n" + f" proxy_pass {SRV_ADDR}:{dest};\n proxy_bind {out};\n }}\n") + for port, dest in ((TLS_INIT, ORIGIN_TLS), (TLS_INIT_TCP, ORIGIN_TLS_TCP)): + nginx_tls += (f" server {{\n listen {PRX_IN}:{port} backlog=65535 reuseport;\n" + f" proxy_pass {SRV_ADDR}:{dest};\n proxy_ssl on;\n" + f" proxy_bind {out};\n }}\n") f.write(f"""{load}daemon off; worker_processes {ncpu}; pid {work}/{label}.pid; @@ -691,6 +781,16 @@ listen tlsinit bind {PRX_IN}:{TLS_INIT} source {out} server origin {SRV_ADDR}:{ORIGIN_TLS} ssl verify none +listen tlstermtcp + mode tcp + bind {PRX_IN}:{TLS_TERM_TCP} ssl crt {TLS["pem"]} + source {out} + server origin {SRV_ADDR}:{ORIGIN_TCP} +listen tlsinittcp + mode tcp + bind {PRX_IN}:{TLS_INIT_TCP} + source {out} + server origin {SRV_ADDR}:{ORIGIN_TLS_TCP} ssl verify none """) return ["-f", conf] raise ValueError(kind) @@ -715,6 +815,9 @@ class Proxy: sys.exit("nginx options: halfclose") self.binary = path or tool(kind) self.label = kind + ("-" + self.opts if self.opts else "") + elif kind == "envoy": + self.binary = path or tool("envoy") + self.label = kind elif kind in ("tinyproxy", "squid", "dante", "stunnel"): # dante's server is sockd, danted on Debian; stunnel stunnel4 there self.binary = path or (tool("sockd", "danted") if kind == "dante" else @@ -731,12 +834,14 @@ class Proxy: PORTS = {"3proxy": (HTTP_PORT, TCP_PORT, SOCKS_PORT, RELAY_HTTP, RELAY_MIX), "nginx": (HTTP_PORT, TCP_PORT, RELAY_HTTP, HTTP_MIX, RELAY_MIX), "haproxy": (HTTP_PORT, TCP_PORT, RELAY_HTTP, HTTP_MIX, RELAY_MIX), + "envoy": (HTTP_PORT, TCP_PORT, RELAY_HTTP, HTTP_MIX, RELAY_MIX), "tinyproxy": (HTTP_PORT,), "squid": (HTTP_PORT,), "dante": (SOCKS_PORT,), - "stunnel": (TLS_TERM, TLS_INIT)} + "stunnel": (TLS_TERM, TLS_INIT, TLS_TERM_TCP, TLS_INIT_TCP)} def ports(self): """What to wait for: with TLS scenarios the relays for them too.""" - extra = (TLS_TERM, TLS_INIT) if TLS and self.kind in ("3proxy", "nginx", "haproxy") else () + extra = (TLS_TERM, TLS_INIT, TLS_TERM_TCP, TLS_INIT_TCP) \ + if TLS and self.kind in ("3proxy", "nginx", "haproxy", "envoy") else () return self.PORTS[self.kind] + extra def serves(self, scenario): @@ -745,14 +850,14 @@ class Proxy: # TLS ended or begun on a relay: what 3proxy, nginx, haproxy and # stunnel do, and only stunnel if via in ("tls-term", "tls-init"): - return self.kind in ("3proxy", "nginx", "haproxy", "stunnel", "none") + return self.kind in ("3proxy", "nginx", "haproxy", "envoy", "stunnel", "none") if self.kind == "stunnel": return False if self.kind == "dante": return via == "socks" if self.kind in ("tinyproxy", "squid"): return via == "connect" or (kind in ("http", "mix") and not via) - if self.kind in ("nginx", "haproxy"): + if self.kind in ("nginx", "haproxy", "envoy"): # no CONNECT, no SOCKS return via in (None, "relay") return True @@ -876,12 +981,38 @@ def run_http(proxy, params, args, work, mark): def run_stream(proxy, params, args, work, mark): - host, port, prefix = target(proxy, params, work, ORIGIN_TCP, TCP_PORT, TCP_PORT) + via = params.get("via") + wrapper = None + if via in ("tls-term", "tls-init"): + # iperf3 talks to a wrapper on the client; with TLS ended by the + # proxy, or with no proxy, it begins TLS there, and the origin's + # wrapper ends it; with TLS begun by the proxy it passes plain TCP + if proxy.kind == "none": + dest, tls = f"{SRV_ADDR}:{ORIGIN_TLS_TCP}", True + elif via == "tls-term": + dest, tls = f"{PRX_IN}:{TLS_TERM_TCP}", True + else: + dest, tls = f"{PRX_IN}:{TLS_INIT_TCP}", False + if tls: + log = open(os.path.join(work, "wrap.out"), "a") + wrapper = spawn("bcli", "cli", [args.wrap, "tls", "-listen", f"{CLI_ADDR}:{CLI_TLS}", + "-connect", dest], log) + wait_port("bcli", CLI_ADDR, CLI_TLS, 10) + host, port = CLI_ADDR, CLI_TLS + else: + host, port = PRX_IN, TLS_INIT_TCP + prefix = [] + else: + host, port, prefix = target(proxy, params, work, ORIGIN_TCP, TCP_PORT, TCP_PORT) argv = prefix + [tool("iperf3"), "-c", host, "-p", str(port), "-t", str(args.duration), "-P", str(params["streams"]), "-J"] if params.get("reverse"): argv.append("-R") - out = run_in("bcli", "cli", argv, args.duration + 30) + try: + out = run_in("bcli", "cli", argv, args.duration + 30) + finally: + if wrapper: + stop(wrapper) res = {"raw": out, "rate": float("nan"), "p50_ms": float("nan"), "p99_ms": float("nan"), "requests": 0, "errors": 0, "retrans": 0, "timeouts": 0} try: @@ -908,7 +1039,7 @@ def run_mix(proxy, params, args, work, mark): addr = f"{PRX_IN}:{RELAY_MIX}" elif via == "socks": addr = f"{PRX_IN}:{SOCKS_PORT}" - elif via == "http" and proxy.kind in ("nginx", "haproxy"): + elif via == "http" and proxy.kind in ("nginx", "haproxy", "envoy"): addr = f"{PRX_IN}:{HTTP_MIX}" else: addr = f"{PRX_IN}:{HTTP_PORT}" @@ -1088,6 +1219,14 @@ def main(): sys.exit(f"{args.mix} is missing: build it with bench/mix/build.sh") else: args.mix = None + # the TLS streams need its TLS wrapper + args.wrap = None + if any(SCENARIOS[s][0] == "stream" and SCENARIOS[s][1].get("via") in ("tls-term", "tls-init") + for s in scenarios): + arch = {"x86_64": "amd64", "aarch64": "arm64"}.get(os.uname().machine, os.uname().machine) + args.wrap = os.path.abspath(os.path.join(os.path.dirname(__file__), "mix", f"mix-linux-{arch}")) + if not os.access(args.wrap, os.X_OK): + sys.exit(f"{args.wrap} is missing: build it with bench/mix/build.sh") stamp = datetime.datetime.now().strftime("%Y%m%d-%H%M%S") global TLS diff --git a/bench/mix/main.go b/bench/mix/main.go index d69ba4b..dabde36 100644 --- a/bench/mix/main.go +++ b/bench/mix/main.go @@ -26,6 +26,7 @@ package main import ( "bufio" "bytes" + "crypto/tls" "encoding/binary" "encoding/json" "errors" @@ -45,6 +46,62 @@ import ( "time" ) +// --- a TLS wrapper --------------------------------------------------------------- + +// wrap relays connections to listen to target, with TLS on one side: given a +// certificate it takes TLS and passes plain TCP on (the end of a TLS stream +// at the origin), without one it takes plain TCP and begins TLS (for a client +// with no TLS of its own, iperf3). A side shut down for writing is shut down +// on the other side too, so half-closed connections work. +func wrap(listen, target, cert, key string) error { + var server *tls.Config + if cert != "" { + pair, err := tls.LoadX509KeyPair(cert, key) + if err != nil { + return err + } + server = &tls.Config{Certificates: []tls.Certificate{pair}} + } + ln, err := net.Listen("tcp", listen) + if err != nil { + return err + } + for { + c, err := ln.Accept() + if err != nil { + return err + } + go func() { + var in, out net.Conn = c, nil + var err error + if server != nil { + in = tls.Server(c, server) + out, err = net.Dial("tcp", target) + } else { + out, err = tls.Dial("tcp", target, &tls.Config{InsecureSkipVerify: true}) + } + if err != nil { + c.Close() + return + } + done := make(chan struct{}, 2) + pass := func(dst, src net.Conn) { + io.Copy(dst, src) + if cw, ok := dst.(interface{ CloseWrite() error }); ok { + cw.CloseWrite() + } + done <- struct{}{} + } + go pass(out, in) + go pass(in, out) + <-done + <-done + in.Close() + out.Close() + }() + } +} + // --- the profile ------------------------------------------------------------- // lognormal draws from a lognormal distribution by its median and sigma, @@ -332,6 +389,7 @@ type client struct { aborts, resets, halfcloses atomic.Uint64 complete, first histogram dialErr, exchErr atomic.Value // the first of each, as text + errlog bool // every failed exchange to stderr } // shard is a share of the pool with a dispatcher of its own: one dispatcher @@ -470,8 +528,10 @@ func (cl *client) release(c int) { func (cl *client) worker(id int, c net.Conn) { scratch := make([]byte, 64<<10) head := make([]byte, 0, 4096) + nth := 0 // exchanges on this connection so far for j := range cl.jobs[id] { if c == nil { + nth = 0 var err error if c, err = cl.dial(id); err != nil { cl.dialErr.CompareAndSwap(nil, err.Error()) @@ -488,6 +548,7 @@ func (cl *client) worker(id int, c net.Conn) { } } n, err := cl.exchange(c, id, j, head, scratch) + nth++ j.slot <- struct{}{} if err == errGone { // closed as drawn, with a reset or not @@ -509,6 +570,9 @@ func (cl *client) worker(id int, c net.Conn) { if err != nil || j.e.close || halfclosed { c.Close() c = nil + if err != nil && cl.errlog { + fmt.Fprintf(os.Stderr, "error: connection %d, exchange %d on it: %v\n", id, nth, err) + } if err != nil && cl.measuring.Load() { cl.exchErr.CompareAndSwap(nil, err.Error()) var ne net.Error @@ -567,7 +631,7 @@ func (cl *client) exchange(c net.Conn, id int, j job, head, scratch []byte) (int } head = append(head, "\r\n"...) if err := send(c, r, clientWrites, head, e.upload); err != nil { - return 0, err + return 0, fmt.Errorf("sending (end %d, upload %d): %w", e.end, e.upload, err) } if halfclose { if cw, ok := c.(interface{ CloseWrite() error }); ok { @@ -624,7 +688,8 @@ func (cl *client) exchange(c net.Conn, id int, j job, head, scratch []byte) (int if want >= 0 && body >= want { break } - return 0, err + return 0, fmt.Errorf("reading (end %d, close %v, upload %d, answer %d, got %d, body %d of %d): %w", + e.end, e.close, e.upload, e.answer, got, body, want, err) } if d := readPause(r); d > 0 { time.Sleep(d) @@ -794,7 +859,7 @@ func runClient(cl *client, active int, rate float64, warmup, duration time.Durat func main() { if len(os.Args) < 2 { - fmt.Fprintln(os.Stderr, "usage: mix origin|client [flags]") + fmt.Fprintln(os.Stderr, "usage: mix origin|client|tls [flags]") os.Exit(2) } fs := flag.NewFlagSet(os.Args[1], flag.ExitOnError) @@ -807,6 +872,16 @@ func main() { fmt.Fprintln(os.Stderr, err) os.Exit(1) } + case "tls": + listen := fs.String("listen", "127.0.0.1:5443", "address to take connections on") + target := fs.String("connect", "127.0.0.1:5201", "address to pass them to") + cert := fs.String("cert", "", "certificate: take TLS and pass plain TCP on; none: the other way round") + key := fs.String("key", "", "the certificate's key") + fs.Parse(os.Args[2:]) + if err := wrap(*listen, *target, *cert, *key); err != nil { + fmt.Fprintln(os.Stderr, err) + os.Exit(1) + } case "client": cl := &client{} fs.StringVar(&cl.via, "via", "http", "http (an HTTP proxy), connect, socks, relay or direct") @@ -815,6 +890,7 @@ func main() { fs.IntVar(&cl.origins, "origins", 1, "the origin's ports to spread over (origin -ports)") sources := fs.String("sources", "", "client addresses to open connections from, comma separated") fs.IntVar(&cl.pool, "pool", 20000, "connections kept open") + fs.BoolVar(&cl.errlog, "errlog", false, "print every failed exchange to stderr") active := fs.Int("active", 2000, "exchanges in progress at most") rate := fs.Float64("rate", 0, "exchanges started per second, 0 to keep -active going all the time") duration := fs.Duration("duration", 10*time.Second, "how long to measure") @@ -848,7 +924,7 @@ func main() { } runClient(cl, *active, *rate, *warmup, *duration, *dialers, *shards) default: - fmt.Fprintln(os.Stderr, "usage: mix origin|client [flags]") + fmt.Fprintln(os.Stderr, "usage: mix origin|client|tls [flags]") os.Exit(2) } } diff --git a/bench/report.py b/bench/report.py index bd17870..8cbcc5b 100644 --- a/bench/report.py +++ b/bench/report.py @@ -27,7 +27,8 @@ of the group whose range (the average of its slowest runs to that of its fastest) reaches the best one's; * marks the best of the row, both groups together, in the same way. --split puts the splice group in a table of its own (* still compares across both), --prefix connect- takes only the scenarios so -named. +named. --overall marks the best of the row only, in bold and with *: for +tables where one group has a single proxy, its group's best says nothing. """ import argparse @@ -119,7 +120,7 @@ def load(paths): BULK = STREAM + ("http-ka-1m",) -def combined(machines, labels, splice, prefix=None, split=False): +def combined(machines, labels, splice, prefix=None, split=False, overall_only=False): """One throughput table for several machines: each cell their medians and their average. Bold: the best of its group; *: the best of the row. split puts the proxies using splice() in a table of their own, * still @@ -175,7 +176,17 @@ def combined(machines, labels, splice, prefix=None, split=False): plain = [p for p in proxies if p != "direct" and p not in splice] spliced = [p for p in proxies if p in splice] direct = [p for p in proxies if p == "direct"] - names = " / ".join(n for n, _ in machines) + " / average" + # one machine: its medians alone, with ‡ where most runs were limited by + # the client or the origin; several: their medians and their average + single = len(machines) == 1 + names = machines[0][0] if single else " / ".join(n for n, _ in machines) + " / average" + + def limited(s, p): + if not single: + return "" + bounds = [r.get("bound", "") for r in data[machines[0][0]].get((s, p), [])] + other = sum(b in ("client", "origin") for b in bounds) + return " ‡" if bounds and other * 2 > len(bounds) else "" def render(tables): """tables: lists of groups, each table printed on its own; bold for @@ -198,13 +209,15 @@ def combined(machines, labels, splice, prefix=None, split=False): for i, g in enumerate(t): if i: cells.append("‖") - best = leaders(s, g) + best = overall if overall_only else leaders(s, g) for p in g: - vals = [value(n, s, p) for n, _ in machines] + [average(s, p)] + vals = [value(n, s, p) for n, _ in machines] + if not single: + vals.append(average(s, p)) if all(v is None for v in vals): cells.append("–") continue - text = " / ".join(fmt(v, 1) for v in vals) + text = " / ".join(fmt(v, 1) for v in vals) + limited(s, p) if p in best: text = f"**{text}**" if p in overall: @@ -231,6 +244,8 @@ def main(): ap.add_argument("--machine", action="append", default=[], help="NAME=a.csv[,b.csv]: one throughput table for all the " "machines given, instead of the tables for one") + ap.add_argument("--overall", action="store_true", + help="with --machine: only the best of the row, bold and *") ap.add_argument("--label", action="append", default=[], help="old=new") ap.add_argument("--title", default="") ap.add_argument("--splice", action="append", default=[], @@ -240,7 +255,8 @@ def main(): if args.machine: combined([(m.split("=", 1)[0], m.split("=", 1)[1].split(",")) - for m in args.machine], labels, args.splice, args.prefix, args.split) + for m in args.machine], labels, args.splice, args.prefix, args.split, + args.overall) return rows = load(args.csv) proxies, scenarios = [], []