mirror of
https://github.com/3proxy/3proxy.git
synced 2026-10-15 16:15:49 +08:00
bench: TLS streams, envoy, a table for each machine
tls-term-stream* and tls-init-stream* run iperf3 streams through the TLS relays. iperf3 has no TLS of its own, so bench/mix gets a TLS wrapper mode, run on the client to begin TLS towards a proxy ending it and at the origin to end the TLS a proxy begins. --proxy envoy measures envoy where nginx and haproxy are measured: as an HTTP proxy for the origin, as a TCP relay and in the TLS scenarios, its circuit breakers lifted for the scenarios opening more than 1024 connections. report.py --machine with a single machine gives its medians alone, with the client- or origin-limited ones marked; --overall marks only the best of each row. The mix client's -errlog prints every failed exchange. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Vladimir Dubrovin
parent
aec167f8b3
commit
97b8b9d7fb
@ -89,6 +89,7 @@ host) falls on all of them alike; the summary gives medians.
|
||||
| `mix-*-3k` | the same, exchanges started at 3000 a second whether the proxy keeps up or not: latencies compared at equal load |
|
||||
| `tls-term-ka-1k`, `tls-term-ka-1m`, `tls-term-close-1k` | TLS ended by the proxy: wrk speaks HTTPS to a relay of the proxy, which passes plain TCP to the origin; kept-alive connections with 1 KB and 1 MB answers, and a new connection, so a full handshake, for every request |
|
||||
| `tls-init-*` | TLS begun by the proxy: wrk speaks plain HTTP to a relay, which connects to the origin's HTTPS listener |
|
||||
| `tls-term-stream1-down`, `tls-term-stream16`, `tls-init-stream1-down`, `tls-init-stream16` | the same with iperf3 streams, one downloading or sixteen uploading |
|
||||
|
||||
### The mix scenarios
|
||||
|
||||
@ -146,10 +147,11 @@ answer, and `peak_mb` divided by the pool what each open connection costs the
|
||||
proxy. The binaries are built with `bench/mix/build.sh`, on any system with
|
||||
Go, for both Linux architectures.
|
||||
|
||||
nginx is not a forward proxy: it answers absolute URIs by passing them to its
|
||||
upstream, which is the origin here. Neither nginx nor haproxy has CONNECT or
|
||||
SOCKS, so they take part in the HTTP and `relay-*` scenarios but not in the
|
||||
`connect-*` and `socks-*` ones. tinyproxy
|
||||
nginx and envoy are not forward proxies: they answer absolute URIs by passing
|
||||
them to their upstream, which is the origin here. nginx and haproxy have
|
||||
neither CONNECT nor SOCKS, and envoy is measured as they are, so they take
|
||||
part in the HTTP and `relay-*` scenarios but not in the `connect-*` and
|
||||
`socks-*` ones. tinyproxy
|
||||
and squid are HTTP proxies, measured in the HTTP and `connect-*` scenarios;
|
||||
dante is a SOCKS server, measured in the `socks-*` ones.
|
||||
|
||||
@ -160,10 +162,18 @@ The `tls-*` scenarios compare TLS done in the proxy: 3proxy (`tcppm` under
|
||||
`proxy_ssl on`), haproxy (`mode tcp` with `bind ... ssl crt` and `server ...
|
||||
ssl`) and stunnel, which takes part in these scenarios only. Every one relays
|
||||
TCP: no HTTP is read, so what differs is the TLS and how data is moved around
|
||||
it. The certificate is made for every run: an ECDSA P-256 key signed by a CA
|
||||
it. envoy takes part too, with BoringSSL where the others use OpenSSL. The certificate is made for every run: an ECDSA P-256 key signed by a CA
|
||||
of its own, with the proxy's and the origin's addresses in it; nobody
|
||||
verifies it. `none` has wrk speak HTTPS to the origin itself.
|
||||
|
||||
iperf3 has no TLS of its own, so for the TLS streams `bench/mix tls` wraps
|
||||
its connections: on the client it begins TLS towards the proxy ending it (or
|
||||
towards the origin with no proxy), at the origin it ends the TLS a proxy
|
||||
begins and passes the stream to iperf3. The wrapper is in Go, a goroutine
|
||||
for each direction of each connection: on one stream it, rather than the
|
||||
proxy, can be the limit (about 10 Gbit/s a core), which the run's `bound`
|
||||
shows.
|
||||
|
||||
wrk does not resume TLS sessions, so in `tls-term-close-1k` every connection
|
||||
is a full handshake. Towards the origin a proxy may resume them: nginx
|
||||
(`proxy_ssl_session_reuse`) and haproxy do by default, so `tls-init-close-1k`
|
||||
@ -194,7 +204,9 @@ sudo bench/bench.py \
|
||||
`bench/report.py <csv>` turns a run into Markdown tables: throughput, the
|
||||
proxy's CPU per gigabyte or per request, and its peak memory. With
|
||||
`--machine NAME=a.csv,b.csv` once per machine it makes a single throughput
|
||||
table of all of them, each cell their medians and their average.
|
||||
table of all of them, each cell their medians and their average; `--overall`
|
||||
then marks only the best of each row, for tables where one group has a
|
||||
single proxy.
|
||||
|
||||
After the binary, a 3proxy entry can carry options for both its services:
|
||||
`-s1`, `-s2` and `-s3` splice downloads, uploads or both (`-s0` keeps 0.9.0 to
|
||||
@ -212,7 +224,12 @@ variant; `nginx::halfclose` turns on `proxy_half_close` in its relays, which
|
||||
nginx leaves off: then a client shutting down its sending side still gets the
|
||||
answer, as through 3proxy and haproxy.
|
||||
|
||||
`--proxy stunnel` measures stunnel in the `tls-*` scenarios.
|
||||
`--proxy stunnel` measures stunnel in the `tls-*` scenarios. `--proxy envoy`
|
||||
measures envoy where nginx and haproxy are measured: as an HTTP proxy for the
|
||||
origin (its connections there pooled), as a TCP relay (`tcp_proxy`) and in
|
||||
the TLS scenarios; its circuit breakers, which allow 1024 connections to a
|
||||
cluster by default, are lifted for the scenarios that open more. The
|
||||
official release binaries run as they are (`envoy` on the PATH).
|
||||
|
||||
`--scenario` picks scenarios, `--reps` and `--duration` set the repetitions
|
||||
and their length, `--netem "delay 1ms rate 10gbit"` puts a slower, longer link
|
||||
|
||||
165
bench/bench.py
165
bench/bench.py
@ -51,6 +51,10 @@ MIX_PORTS = 16
|
||||
# (TLS from the client, plain to ORIGIN_HTTP) and initiate it (plain from the
|
||||
# client, TLS to ORIGIN_TLS). TLS holds the certificate files once made.
|
||||
ORIGIN_TLS, TLS_TERM, TLS_INIT = 8443, 5443, 5444
|
||||
# the same for TLS streams: iperf3 has no TLS, so bench/mix wraps its
|
||||
# connections, on the client (CLI_TLS, for TLS ended by the proxy) and at the
|
||||
# origin (ORIGIN_TLS_TCP, taking the TLS begun by the proxy)
|
||||
ORIGIN_TLS_TCP, TLS_TERM_TCP, TLS_INIT_TCP, CLI_TLS = 8444, 5445, 5446, 5202
|
||||
TLS = None
|
||||
# Each measurement takes the next of SLOTS sets of addresses: the proxy's to
|
||||
# connect from, and the mix client's to connect from. What a run leaves in
|
||||
@ -107,6 +111,10 @@ SCENARIOS = {
|
||||
"tls-init-ka-1k": ("http", {"file": "1k", "conns": 64, "via": "tls-init"}),
|
||||
"tls-init-ka-1m": ("http", {"file": "1m", "conns": 16, "via": "tls-init"}),
|
||||
"tls-init-close-1k": ("http", {"file": "1k", "conns": 64, "close": True, "via": "tls-init"}),
|
||||
"tls-term-stream1-down": ("stream", {"streams": 1, "reverse": True, "via": "tls-term"}),
|
||||
"tls-term-stream16": ("stream", {"streams": 16, "via": "tls-term"}),
|
||||
"tls-init-stream1-down": ("stream", {"streams": 1, "reverse": True, "via": "tls-init"}),
|
||||
"tls-init-stream16": ("stream", {"streams": 16, "via": "tls-init"}),
|
||||
# a pool of open connections and exchanges on random ones of them, sizes
|
||||
# and pieces from a web-like profile (bench/mix): -active of them going
|
||||
# at once all the time, or started at a fixed rate whatever the proxy
|
||||
@ -445,6 +453,11 @@ http {{
|
||||
procs = [spawn("bsrv", "srv", [tool("nginx"), "-c", conf, "-p", work], log),
|
||||
spawn("bsrv", "srv", [tool("iperf3"), "-s", "-B", SRV_ADDR, "-p", str(ORIGIN_TCP)], log)]
|
||||
ports = [ORIGIN_HTTP, ORIGIN_TCP] + ([ORIGIN_TLS] if TLS else [])
|
||||
if args.wrap:
|
||||
procs.append(spawn("bsrv", "srv", [args.wrap, "tls", "-listen", f"{SRV_ADDR}:{ORIGIN_TLS_TCP}",
|
||||
"-connect", f"{SRV_ADDR}:{ORIGIN_TCP}",
|
||||
"-cert", TLS["cert"], "-key", TLS["key"]], log))
|
||||
ports.append(ORIGIN_TLS_TCP)
|
||||
if args.mix:
|
||||
procs.append(spawn("bsrv", "srv", [args.mix, "origin", "-listen", f"{SRV_ADDR}:{ORIGIN_MIX}",
|
||||
"-ports", str(MIX_PORTS)], log))
|
||||
@ -458,6 +471,64 @@ http {{
|
||||
|
||||
# --- proxies -----------------------------------------------------------------
|
||||
|
||||
def envoy_conf(work, label, out):
|
||||
"""envoy as nginx and haproxy are measured: an HTTP proxy for the origin
|
||||
(absolute URIs taken, the origin's connections pooled), TCP relays, and
|
||||
the TLS ones. Its default circuit breakers allow 1024 connections to a
|
||||
cluster, far fewer than the scenarios open: they are lifted."""
|
||||
def addr(host, port):
|
||||
return {"socket_address": {"address": host, "port_value": port}}
|
||||
|
||||
def cluster(name, port, tls=False):
|
||||
c = {"name": name, "type": "STATIC", "connect_timeout": "5s",
|
||||
"upstream_bind_config": {"source_address": {"address": out, "port_value": 0}},
|
||||
"circuit_breakers": {"thresholds": [{"max_connections": 1000000,
|
||||
"max_pending_requests": 1000000,
|
||||
"max_requests": 1000000}]},
|
||||
"load_assignment": {"cluster_name": name, "endpoints": [{"lb_endpoints": [
|
||||
{"endpoint": {"address": addr(SRV_ADDR, port)}}]}]}}
|
||||
if tls:
|
||||
c["transport_socket"] = {"name": "envoy.transport_sockets.tls", "typed_config": {
|
||||
"@type": "type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext"}}
|
||||
return c
|
||||
|
||||
def http(name, port, dest):
|
||||
return {"name": name, "address": addr(PRX_IN, port), "filter_chains": [{"filters": [{
|
||||
"name": "envoy.filters.network.http_connection_manager", "typed_config": {
|
||||
"@type": "type.googleapis.com/envoy.extensions.filters.network."
|
||||
"http_connection_manager.v3.HttpConnectionManager",
|
||||
"stat_prefix": name, "codec_type": "HTTP1",
|
||||
"route_config": {"virtual_hosts": [{"name": "all", "domains": ["*"], "routes": [
|
||||
{"match": {"prefix": "/"}, "route": {"cluster": dest, "timeout": "0s"}}]}]},
|
||||
"http_filters": [{"name": "envoy.filters.http.router", "typed_config": {
|
||||
"@type": "type.googleapis.com/envoy.extensions.filters.http.router.v3.Router"}}]}}]}]}
|
||||
|
||||
def tcp(name, port, dest, tls=False):
|
||||
chain = {"filters": [{"name": "envoy.filters.network.tcp_proxy", "typed_config": {
|
||||
"@type": "type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy",
|
||||
"stat_prefix": name, "cluster": dest}}]}
|
||||
if tls:
|
||||
chain["transport_socket"] = {"name": "envoy.transport_sockets.tls", "typed_config": {
|
||||
"@type": "type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext",
|
||||
"common_tls_context": {"tls_certificates": [{
|
||||
"certificate_chain": {"filename": TLS["cert"]},
|
||||
"private_key": {"filename": TLS["key"]}}]}}}
|
||||
return {"name": name, "address": addr(PRX_IN, port), "filter_chains": [chain]}
|
||||
|
||||
listeners = [http("http", HTTP_PORT, "origin"), http("httpmix", HTTP_MIX, "mix"),
|
||||
tcp("tcp", TCP_PORT, "tcp"), tcp("tcphttp", RELAY_HTTP, "origin"),
|
||||
tcp("tcpmix", RELAY_MIX, "mix")]
|
||||
clusters = [cluster("origin", ORIGIN_HTTP), cluster("mix", ORIGIN_MIX), cluster("tcp", ORIGIN_TCP)]
|
||||
if TLS:
|
||||
listeners += [tcp("tlsterm", TLS_TERM, "origin", True), tcp("tlstermtcp", TLS_TERM_TCP, "tcp", True),
|
||||
tcp("tlsinit", TLS_INIT, "origintls"), tcp("tlsinittcp", TLS_INIT_TCP, "tcptls")]
|
||||
clusters += [cluster("origintls", ORIGIN_TLS, True), cluster("tcptls", ORIGIN_TLS_TCP, True)]
|
||||
conf = os.path.join(work, f"{label}.json")
|
||||
with open(conf, "w") as f:
|
||||
json.dump({"static_resources": {"listeners": listeners, "clusters": clusters}}, f, indent=1)
|
||||
return conf
|
||||
|
||||
|
||||
def proxy_conf(kind, work, ncpu, label, opts="", out=PRX_OUT):
|
||||
"""Configuration for one proxy: an HTTP proxy on HTTP_PORT and a TCP relay
|
||||
on TCP_PORT, both towards the origin. Returns the argv. opts are added to
|
||||
@ -479,12 +550,17 @@ socks {opts} -i{PRX_IN} -e{out} -p{SOCKS_PORT}
|
||||
ssl_server_key {TLS["key"]}
|
||||
ssl_serv
|
||||
tcppm {opts} -i{PRX_IN} -e{out} {TLS_TERM} {SRV_ADDR} {ORIGIN_HTTP}
|
||||
tcppm {opts} -i{PRX_IN} -e{out} {TLS_TERM_TCP} {SRV_ADDR} {ORIGIN_TCP}
|
||||
ssl_noserv
|
||||
ssl_cli
|
||||
tcppm {opts} -i{PRX_IN} -e{out} {TLS_INIT} {SRV_ADDR} {ORIGIN_TLS}
|
||||
tcppm {opts} -i{PRX_IN} -e{out} {TLS_INIT_TCP} {SRV_ADDR} {ORIGIN_TLS_TCP}
|
||||
ssl_nocli
|
||||
""")
|
||||
return [conf]
|
||||
if kind == "envoy":
|
||||
return ["-c", envoy_conf(work, label, out), "--concurrency", str(ncpu),
|
||||
"--log-level", "error", "--disable-hot-restart"]
|
||||
if kind == "stunnel":
|
||||
conf = os.path.join(work, f"{label}.conf")
|
||||
with open(conf, "w") as f:
|
||||
@ -503,6 +579,17 @@ client = yes
|
||||
accept = {PRX_IN}:{TLS_INIT}
|
||||
connect = {SRV_ADDR}:{ORIGIN_TLS}
|
||||
local = {out}
|
||||
[termtcp]
|
||||
accept = {PRX_IN}:{TLS_TERM_TCP}
|
||||
connect = {SRV_ADDR}:{ORIGIN_TCP}
|
||||
local = {out}
|
||||
cert = {TLS["cert"]}
|
||||
key = {TLS["key"]}
|
||||
[inittcp]
|
||||
client = yes
|
||||
accept = {PRX_IN}:{TLS_INIT_TCP}
|
||||
connect = {SRV_ADDR}:{ORIGIN_TLS_TCP}
|
||||
local = {out}
|
||||
""")
|
||||
return [conf]
|
||||
if kind == "tinyproxy":
|
||||
@ -565,12 +652,15 @@ socks pass {{ from: 0.0.0.0/0 to: 0.0.0.0/0 }}
|
||||
halfclose = " proxy_half_close on;\n" if opts == "halfclose" else ""
|
||||
nginx_tls = ""
|
||||
if TLS:
|
||||
nginx_tls = (f" server {{\n listen {PRX_IN}:{TLS_TERM} ssl backlog=65535 reuseport;\n"
|
||||
f" ssl_certificate {TLS['cert']};\n ssl_certificate_key {TLS['key']};\n"
|
||||
f" proxy_pass {SRV_ADDR}:{ORIGIN_HTTP};\n proxy_bind {out};\n }}\n"
|
||||
f" server {{\n listen {PRX_IN}:{TLS_INIT} backlog=65535 reuseport;\n"
|
||||
f" proxy_pass {SRV_ADDR}:{ORIGIN_TLS};\n proxy_ssl on;\n"
|
||||
f" proxy_bind {out};\n }}\n")
|
||||
nginx_tls = ""
|
||||
for port, dest in ((TLS_TERM, ORIGIN_HTTP), (TLS_TERM_TCP, ORIGIN_TCP)):
|
||||
nginx_tls += (f" server {{\n listen {PRX_IN}:{port} ssl backlog=65535 reuseport;\n"
|
||||
f" ssl_certificate {TLS['cert']};\n ssl_certificate_key {TLS['key']};\n"
|
||||
f" proxy_pass {SRV_ADDR}:{dest};\n proxy_bind {out};\n }}\n")
|
||||
for port, dest in ((TLS_INIT, ORIGIN_TLS), (TLS_INIT_TCP, ORIGIN_TLS_TCP)):
|
||||
nginx_tls += (f" server {{\n listen {PRX_IN}:{port} backlog=65535 reuseport;\n"
|
||||
f" proxy_pass {SRV_ADDR}:{dest};\n proxy_ssl on;\n"
|
||||
f" proxy_bind {out};\n }}\n")
|
||||
f.write(f"""{load}daemon off;
|
||||
worker_processes {ncpu};
|
||||
pid {work}/{label}.pid;
|
||||
@ -691,6 +781,16 @@ listen tlsinit
|
||||
bind {PRX_IN}:{TLS_INIT}
|
||||
source {out}
|
||||
server origin {SRV_ADDR}:{ORIGIN_TLS} ssl verify none
|
||||
listen tlstermtcp
|
||||
mode tcp
|
||||
bind {PRX_IN}:{TLS_TERM_TCP} ssl crt {TLS["pem"]}
|
||||
source {out}
|
||||
server origin {SRV_ADDR}:{ORIGIN_TCP}
|
||||
listen tlsinittcp
|
||||
mode tcp
|
||||
bind {PRX_IN}:{TLS_INIT_TCP}
|
||||
source {out}
|
||||
server origin {SRV_ADDR}:{ORIGIN_TLS_TCP} ssl verify none
|
||||
""")
|
||||
return ["-f", conf]
|
||||
raise ValueError(kind)
|
||||
@ -715,6 +815,9 @@ class Proxy:
|
||||
sys.exit("nginx options: halfclose")
|
||||
self.binary = path or tool(kind)
|
||||
self.label = kind + ("-" + self.opts if self.opts else "")
|
||||
elif kind == "envoy":
|
||||
self.binary = path or tool("envoy")
|
||||
self.label = kind
|
||||
elif kind in ("tinyproxy", "squid", "dante", "stunnel"):
|
||||
# dante's server is sockd, danted on Debian; stunnel stunnel4 there
|
||||
self.binary = path or (tool("sockd", "danted") if kind == "dante" else
|
||||
@ -731,12 +834,14 @@ class Proxy:
|
||||
PORTS = {"3proxy": (HTTP_PORT, TCP_PORT, SOCKS_PORT, RELAY_HTTP, RELAY_MIX),
|
||||
"nginx": (HTTP_PORT, TCP_PORT, RELAY_HTTP, HTTP_MIX, RELAY_MIX),
|
||||
"haproxy": (HTTP_PORT, TCP_PORT, RELAY_HTTP, HTTP_MIX, RELAY_MIX),
|
||||
"envoy": (HTTP_PORT, TCP_PORT, RELAY_HTTP, HTTP_MIX, RELAY_MIX),
|
||||
"tinyproxy": (HTTP_PORT,), "squid": (HTTP_PORT,), "dante": (SOCKS_PORT,),
|
||||
"stunnel": (TLS_TERM, TLS_INIT)}
|
||||
"stunnel": (TLS_TERM, TLS_INIT, TLS_TERM_TCP, TLS_INIT_TCP)}
|
||||
|
||||
def ports(self):
|
||||
"""What to wait for: with TLS scenarios the relays for them too."""
|
||||
extra = (TLS_TERM, TLS_INIT) if TLS and self.kind in ("3proxy", "nginx", "haproxy") else ()
|
||||
extra = (TLS_TERM, TLS_INIT, TLS_TERM_TCP, TLS_INIT_TCP) \
|
||||
if TLS and self.kind in ("3proxy", "nginx", "haproxy", "envoy") else ()
|
||||
return self.PORTS[self.kind] + extra
|
||||
|
||||
def serves(self, scenario):
|
||||
@ -745,14 +850,14 @@ class Proxy:
|
||||
# TLS ended or begun on a relay: what 3proxy, nginx, haproxy and
|
||||
# stunnel do, and only stunnel
|
||||
if via in ("tls-term", "tls-init"):
|
||||
return self.kind in ("3proxy", "nginx", "haproxy", "stunnel", "none")
|
||||
return self.kind in ("3proxy", "nginx", "haproxy", "envoy", "stunnel", "none")
|
||||
if self.kind == "stunnel":
|
||||
return False
|
||||
if self.kind == "dante":
|
||||
return via == "socks"
|
||||
if self.kind in ("tinyproxy", "squid"):
|
||||
return via == "connect" or (kind in ("http", "mix") and not via)
|
||||
if self.kind in ("nginx", "haproxy"):
|
||||
if self.kind in ("nginx", "haproxy", "envoy"):
|
||||
# no CONNECT, no SOCKS
|
||||
return via in (None, "relay")
|
||||
return True
|
||||
@ -876,12 +981,38 @@ def run_http(proxy, params, args, work, mark):
|
||||
|
||||
|
||||
def run_stream(proxy, params, args, work, mark):
|
||||
host, port, prefix = target(proxy, params, work, ORIGIN_TCP, TCP_PORT, TCP_PORT)
|
||||
via = params.get("via")
|
||||
wrapper = None
|
||||
if via in ("tls-term", "tls-init"):
|
||||
# iperf3 talks to a wrapper on the client; with TLS ended by the
|
||||
# proxy, or with no proxy, it begins TLS there, and the origin's
|
||||
# wrapper ends it; with TLS begun by the proxy it passes plain TCP
|
||||
if proxy.kind == "none":
|
||||
dest, tls = f"{SRV_ADDR}:{ORIGIN_TLS_TCP}", True
|
||||
elif via == "tls-term":
|
||||
dest, tls = f"{PRX_IN}:{TLS_TERM_TCP}", True
|
||||
else:
|
||||
dest, tls = f"{PRX_IN}:{TLS_INIT_TCP}", False
|
||||
if tls:
|
||||
log = open(os.path.join(work, "wrap.out"), "a")
|
||||
wrapper = spawn("bcli", "cli", [args.wrap, "tls", "-listen", f"{CLI_ADDR}:{CLI_TLS}",
|
||||
"-connect", dest], log)
|
||||
wait_port("bcli", CLI_ADDR, CLI_TLS, 10)
|
||||
host, port = CLI_ADDR, CLI_TLS
|
||||
else:
|
||||
host, port = PRX_IN, TLS_INIT_TCP
|
||||
prefix = []
|
||||
else:
|
||||
host, port, prefix = target(proxy, params, work, ORIGIN_TCP, TCP_PORT, TCP_PORT)
|
||||
argv = prefix + [tool("iperf3"), "-c", host, "-p", str(port), "-t", str(args.duration),
|
||||
"-P", str(params["streams"]), "-J"]
|
||||
if params.get("reverse"):
|
||||
argv.append("-R")
|
||||
out = run_in("bcli", "cli", argv, args.duration + 30)
|
||||
try:
|
||||
out = run_in("bcli", "cli", argv, args.duration + 30)
|
||||
finally:
|
||||
if wrapper:
|
||||
stop(wrapper)
|
||||
res = {"raw": out, "rate": float("nan"), "p50_ms": float("nan"),
|
||||
"p99_ms": float("nan"), "requests": 0, "errors": 0, "retrans": 0, "timeouts": 0}
|
||||
try:
|
||||
@ -908,7 +1039,7 @@ def run_mix(proxy, params, args, work, mark):
|
||||
addr = f"{PRX_IN}:{RELAY_MIX}"
|
||||
elif via == "socks":
|
||||
addr = f"{PRX_IN}:{SOCKS_PORT}"
|
||||
elif via == "http" and proxy.kind in ("nginx", "haproxy"):
|
||||
elif via == "http" and proxy.kind in ("nginx", "haproxy", "envoy"):
|
||||
addr = f"{PRX_IN}:{HTTP_MIX}"
|
||||
else:
|
||||
addr = f"{PRX_IN}:{HTTP_PORT}"
|
||||
@ -1088,6 +1219,14 @@ def main():
|
||||
sys.exit(f"{args.mix} is missing: build it with bench/mix/build.sh")
|
||||
else:
|
||||
args.mix = None
|
||||
# the TLS streams need its TLS wrapper
|
||||
args.wrap = None
|
||||
if any(SCENARIOS[s][0] == "stream" and SCENARIOS[s][1].get("via") in ("tls-term", "tls-init")
|
||||
for s in scenarios):
|
||||
arch = {"x86_64": "amd64", "aarch64": "arm64"}.get(os.uname().machine, os.uname().machine)
|
||||
args.wrap = os.path.abspath(os.path.join(os.path.dirname(__file__), "mix", f"mix-linux-{arch}"))
|
||||
if not os.access(args.wrap, os.X_OK):
|
||||
sys.exit(f"{args.wrap} is missing: build it with bench/mix/build.sh")
|
||||
|
||||
stamp = datetime.datetime.now().strftime("%Y%m%d-%H%M%S")
|
||||
global TLS
|
||||
|
||||
@ -26,6 +26,7 @@ package main
|
||||
import (
|
||||
"bufio"
|
||||
"bytes"
|
||||
"crypto/tls"
|
||||
"encoding/binary"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
@ -45,6 +46,62 @@ import (
|
||||
"time"
|
||||
)
|
||||
|
||||
// --- a TLS wrapper ---------------------------------------------------------------
|
||||
|
||||
// wrap relays connections to listen to target, with TLS on one side: given a
|
||||
// certificate it takes TLS and passes plain TCP on (the end of a TLS stream
|
||||
// at the origin), without one it takes plain TCP and begins TLS (for a client
|
||||
// with no TLS of its own, iperf3). A side shut down for writing is shut down
|
||||
// on the other side too, so half-closed connections work.
|
||||
func wrap(listen, target, cert, key string) error {
|
||||
var server *tls.Config
|
||||
if cert != "" {
|
||||
pair, err := tls.LoadX509KeyPair(cert, key)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
server = &tls.Config{Certificates: []tls.Certificate{pair}}
|
||||
}
|
||||
ln, err := net.Listen("tcp", listen)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for {
|
||||
c, err := ln.Accept()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
go func() {
|
||||
var in, out net.Conn = c, nil
|
||||
var err error
|
||||
if server != nil {
|
||||
in = tls.Server(c, server)
|
||||
out, err = net.Dial("tcp", target)
|
||||
} else {
|
||||
out, err = tls.Dial("tcp", target, &tls.Config{InsecureSkipVerify: true})
|
||||
}
|
||||
if err != nil {
|
||||
c.Close()
|
||||
return
|
||||
}
|
||||
done := make(chan struct{}, 2)
|
||||
pass := func(dst, src net.Conn) {
|
||||
io.Copy(dst, src)
|
||||
if cw, ok := dst.(interface{ CloseWrite() error }); ok {
|
||||
cw.CloseWrite()
|
||||
}
|
||||
done <- struct{}{}
|
||||
}
|
||||
go pass(out, in)
|
||||
go pass(in, out)
|
||||
<-done
|
||||
<-done
|
||||
in.Close()
|
||||
out.Close()
|
||||
}()
|
||||
}
|
||||
}
|
||||
|
||||
// --- the profile -------------------------------------------------------------
|
||||
|
||||
// lognormal draws from a lognormal distribution by its median and sigma,
|
||||
@ -332,6 +389,7 @@ type client struct {
|
||||
aborts, resets, halfcloses atomic.Uint64
|
||||
complete, first histogram
|
||||
dialErr, exchErr atomic.Value // the first of each, as text
|
||||
errlog bool // every failed exchange to stderr
|
||||
}
|
||||
|
||||
// shard is a share of the pool with a dispatcher of its own: one dispatcher
|
||||
@ -470,8 +528,10 @@ func (cl *client) release(c int) {
|
||||
func (cl *client) worker(id int, c net.Conn) {
|
||||
scratch := make([]byte, 64<<10)
|
||||
head := make([]byte, 0, 4096)
|
||||
nth := 0 // exchanges on this connection so far
|
||||
for j := range cl.jobs[id] {
|
||||
if c == nil {
|
||||
nth = 0
|
||||
var err error
|
||||
if c, err = cl.dial(id); err != nil {
|
||||
cl.dialErr.CompareAndSwap(nil, err.Error())
|
||||
@ -488,6 +548,7 @@ func (cl *client) worker(id int, c net.Conn) {
|
||||
}
|
||||
}
|
||||
n, err := cl.exchange(c, id, j, head, scratch)
|
||||
nth++
|
||||
j.slot <- struct{}{}
|
||||
if err == errGone {
|
||||
// closed as drawn, with a reset or not
|
||||
@ -509,6 +570,9 @@ func (cl *client) worker(id int, c net.Conn) {
|
||||
if err != nil || j.e.close || halfclosed {
|
||||
c.Close()
|
||||
c = nil
|
||||
if err != nil && cl.errlog {
|
||||
fmt.Fprintf(os.Stderr, "error: connection %d, exchange %d on it: %v\n", id, nth, err)
|
||||
}
|
||||
if err != nil && cl.measuring.Load() {
|
||||
cl.exchErr.CompareAndSwap(nil, err.Error())
|
||||
var ne net.Error
|
||||
@ -567,7 +631,7 @@ func (cl *client) exchange(c net.Conn, id int, j job, head, scratch []byte) (int
|
||||
}
|
||||
head = append(head, "\r\n"...)
|
||||
if err := send(c, r, clientWrites, head, e.upload); err != nil {
|
||||
return 0, err
|
||||
return 0, fmt.Errorf("sending (end %d, upload %d): %w", e.end, e.upload, err)
|
||||
}
|
||||
if halfclose {
|
||||
if cw, ok := c.(interface{ CloseWrite() error }); ok {
|
||||
@ -624,7 +688,8 @@ func (cl *client) exchange(c net.Conn, id int, j job, head, scratch []byte) (int
|
||||
if want >= 0 && body >= want {
|
||||
break
|
||||
}
|
||||
return 0, err
|
||||
return 0, fmt.Errorf("reading (end %d, close %v, upload %d, answer %d, got %d, body %d of %d): %w",
|
||||
e.end, e.close, e.upload, e.answer, got, body, want, err)
|
||||
}
|
||||
if d := readPause(r); d > 0 {
|
||||
time.Sleep(d)
|
||||
@ -794,7 +859,7 @@ func runClient(cl *client, active int, rate float64, warmup, duration time.Durat
|
||||
|
||||
func main() {
|
||||
if len(os.Args) < 2 {
|
||||
fmt.Fprintln(os.Stderr, "usage: mix origin|client [flags]")
|
||||
fmt.Fprintln(os.Stderr, "usage: mix origin|client|tls [flags]")
|
||||
os.Exit(2)
|
||||
}
|
||||
fs := flag.NewFlagSet(os.Args[1], flag.ExitOnError)
|
||||
@ -807,6 +872,16 @@ func main() {
|
||||
fmt.Fprintln(os.Stderr, err)
|
||||
os.Exit(1)
|
||||
}
|
||||
case "tls":
|
||||
listen := fs.String("listen", "127.0.0.1:5443", "address to take connections on")
|
||||
target := fs.String("connect", "127.0.0.1:5201", "address to pass them to")
|
||||
cert := fs.String("cert", "", "certificate: take TLS and pass plain TCP on; none: the other way round")
|
||||
key := fs.String("key", "", "the certificate's key")
|
||||
fs.Parse(os.Args[2:])
|
||||
if err := wrap(*listen, *target, *cert, *key); err != nil {
|
||||
fmt.Fprintln(os.Stderr, err)
|
||||
os.Exit(1)
|
||||
}
|
||||
case "client":
|
||||
cl := &client{}
|
||||
fs.StringVar(&cl.via, "via", "http", "http (an HTTP proxy), connect, socks, relay or direct")
|
||||
@ -815,6 +890,7 @@ func main() {
|
||||
fs.IntVar(&cl.origins, "origins", 1, "the origin's ports to spread over (origin -ports)")
|
||||
sources := fs.String("sources", "", "client addresses to open connections from, comma separated")
|
||||
fs.IntVar(&cl.pool, "pool", 20000, "connections kept open")
|
||||
fs.BoolVar(&cl.errlog, "errlog", false, "print every failed exchange to stderr")
|
||||
active := fs.Int("active", 2000, "exchanges in progress at most")
|
||||
rate := fs.Float64("rate", 0, "exchanges started per second, 0 to keep -active going all the time")
|
||||
duration := fs.Duration("duration", 10*time.Second, "how long to measure")
|
||||
@ -848,7 +924,7 @@ func main() {
|
||||
}
|
||||
runClient(cl, *active, *rate, *warmup, *duration, *dialers, *shards)
|
||||
default:
|
||||
fmt.Fprintln(os.Stderr, "usage: mix origin|client [flags]")
|
||||
fmt.Fprintln(os.Stderr, "usage: mix origin|client|tls [flags]")
|
||||
os.Exit(2)
|
||||
}
|
||||
}
|
||||
|
||||
@ -27,7 +27,8 @@ of the group whose range (the average of its slowest runs to that of its
|
||||
fastest) reaches the best one's; * marks the best of the row, both groups
|
||||
together, in the same way. --split puts the splice group in a table of its own
|
||||
(* still compares across both), --prefix connect- takes only the scenarios so
|
||||
named.
|
||||
named. --overall marks the best of the row only, in bold and with *: for
|
||||
tables where one group has a single proxy, its group's best says nothing.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
@ -119,7 +120,7 @@ def load(paths):
|
||||
BULK = STREAM + ("http-ka-1m",)
|
||||
|
||||
|
||||
def combined(machines, labels, splice, prefix=None, split=False):
|
||||
def combined(machines, labels, splice, prefix=None, split=False, overall_only=False):
|
||||
"""One throughput table for several machines: each cell their medians
|
||||
and their average. Bold: the best of its group; *: the best of the row.
|
||||
split puts the proxies using splice() in a table of their own, * still
|
||||
@ -175,7 +176,17 @@ def combined(machines, labels, splice, prefix=None, split=False):
|
||||
plain = [p for p in proxies if p != "direct" and p not in splice]
|
||||
spliced = [p for p in proxies if p in splice]
|
||||
direct = [p for p in proxies if p == "direct"]
|
||||
names = " / ".join(n for n, _ in machines) + " / average"
|
||||
# one machine: its medians alone, with ‡ where most runs were limited by
|
||||
# the client or the origin; several: their medians and their average
|
||||
single = len(machines) == 1
|
||||
names = machines[0][0] if single else " / ".join(n for n, _ in machines) + " / average"
|
||||
|
||||
def limited(s, p):
|
||||
if not single:
|
||||
return ""
|
||||
bounds = [r.get("bound", "") for r in data[machines[0][0]].get((s, p), [])]
|
||||
other = sum(b in ("client", "origin") for b in bounds)
|
||||
return " ‡" if bounds and other * 2 > len(bounds) else ""
|
||||
|
||||
def render(tables):
|
||||
"""tables: lists of groups, each table printed on its own; bold for
|
||||
@ -198,13 +209,15 @@ def combined(machines, labels, splice, prefix=None, split=False):
|
||||
for i, g in enumerate(t):
|
||||
if i:
|
||||
cells.append("‖")
|
||||
best = leaders(s, g)
|
||||
best = overall if overall_only else leaders(s, g)
|
||||
for p in g:
|
||||
vals = [value(n, s, p) for n, _ in machines] + [average(s, p)]
|
||||
vals = [value(n, s, p) for n, _ in machines]
|
||||
if not single:
|
||||
vals.append(average(s, p))
|
||||
if all(v is None for v in vals):
|
||||
cells.append("–")
|
||||
continue
|
||||
text = " / ".join(fmt(v, 1) for v in vals)
|
||||
text = " / ".join(fmt(v, 1) for v in vals) + limited(s, p)
|
||||
if p in best:
|
||||
text = f"**{text}**"
|
||||
if p in overall:
|
||||
@ -231,6 +244,8 @@ def main():
|
||||
ap.add_argument("--machine", action="append", default=[],
|
||||
help="NAME=a.csv[,b.csv]: one throughput table for all the "
|
||||
"machines given, instead of the tables for one")
|
||||
ap.add_argument("--overall", action="store_true",
|
||||
help="with --machine: only the best of the row, bold and *")
|
||||
ap.add_argument("--label", action="append", default=[], help="old=new")
|
||||
ap.add_argument("--title", default="")
|
||||
ap.add_argument("--splice", action="append", default=[],
|
||||
@ -240,7 +255,8 @@ def main():
|
||||
|
||||
if args.machine:
|
||||
combined([(m.split("=", 1)[0], m.split("=", 1)[1].split(","))
|
||||
for m in args.machine], labels, args.splice, args.prefix, args.split)
|
||||
for m in args.machine], labels, args.splice, args.prefix, args.split,
|
||||
args.overall)
|
||||
return
|
||||
rows = load(args.csv)
|
||||
proxies, scenarios = [], []
|
||||
|
||||
Loading…
Reference in New Issue
Block a user