-proxy -s -socks -s +proxy -s0 tcppm -s3 8080 192.168.1.1 80--s alone splices both directions where it pays: an HTTP body of known length if at +-s alone, the default, splices both directions where it pays: an HTTP body of known length if at least 16K of it is left to read, a stream (CONNECT, SOCKS, tcppm) once it carries large reads, going back to copying when they turn small. -s1 always splices what the server sends to the client (downloads), -s2 what the client sends to the server (uploads), -s3 both; -s5, -s6 and -s7 (the same as -s) do it for those directions -only where it pays. -s0, the default, turns it off. In builds without it (CMake option 3PROXY_USE_SPLICE=OFF, +only where it pays. -s0 turns it off. In builds without it (CMake option 3PROXY_USE_SPLICE=OFF, or -DWITHSPLICE removed from the Makefile) -s is accepted and ignored. -
splice() pays off when most of the traffic is large transfers: in the -benchmarks it moved -1.3 to 2.1 times more data than the read/write path at 40-55% less CPU per gigabyte. -It does not help small requests and short connections, which go slower with it, and -every spliced direction needs a pipe (two file descriptors) and more kernel -buffer memory. Data filters are not applied to spliced data. +
splice() pays off when most of the traffic is large transfers. Measured in +the benchmarks +on an 8 CPU KVM virtual machine (Intel Xeon Skylake, 2.0 GHz, Linux 6.12), with +3proxy confined to one CPU and the client and the origin on CPUs and in network +namespaces of their own, linked by veth pairs (MTU 1500), default 16K buffers, +the automatic mode (-s) against copying (-s0): +
| load | -s0 | -s | CPU per gigabyte |
|---|---|---|---|
| TCP streams through tcppm, CONNECT or SOCKS | 9.4-10.2 Gbit/s | 28-34 Gbit/s | 65-70% less |
| HTTP proxy, 1 MB answers on kept-alive connections | 9.4 Gbit/s | 25.3 Gbit/s | 62% less |
| 2000 clients, 50 KB answers: tcppm | 11.0k req/s | 18.1k req/s | 39% less |
| 2000 clients, 50 KB answers: CONNECT, SOCKS | 10.4-10.5k req/s | 15.8k req/s | 33-35% less |
| 2000 clients, 50 KB answers: HTTP proxy | 8.1k req/s | 9.6k req/s | 16% less |
| 2000 clients, 2 KB POST and 50 KB answer: tcppm | 10.5k req/s | 16.4k req/s | 36% less |
| 2000 clients, 2 KB POST and 50 KB answer: CONNECT, SOCKS | 9.8-10.1k req/s | 14.6-14.7k req/s | 31-34% less |
| 2000 clients, 2 KB POST and 50 KB answer: HTTP proxy | 7.5k req/s | 9.4k req/s | 20% less |
| 1K requests, kept alive or a connection each | within 5% either way: the automatic mode does not splice them | ||
The mix closer to a proxy giving clients access to the Internet: 20000 +connections, 3000 exchanges a second, web-like answers (median 12K, up to +8M), as a TCP relay, an HTTP proxy and through CONNECT, on the same machine, +with the clients' links emulated as in the grace delay tests below. 3proxy's +own CPU time per exchange with -s against -s0 (and the CPU's busy time, the +kernel's network work included): +
| client link | tcppm | HTTP proxy | CONNECT |
|---|---|---|---|
| no limit, no delay | 8% less (4%) | 9% less (6%) | 12% less (7%) |
| 100 Mbit/s, 2 ms, 0.1% loss | 7% less (+1%) | 1% less (+3%) | 6% less (+1%) |
| 20 Mbit/s, 5 ms, 0.5% loss | +1% (+3%) | 12% less (5%) | +7% (+8%) |
| 5 Mbit/s, 25 ms, 1% loss | 8% less (0%) | 10% less (9%) | 6% less (2%) |
Conclusion: for web traffic -s, the default, and -s0 show no +significant difference: most connections are slow and their exchanges +short, splice() applies to few of them, and the difference stays within +0-12% of 3proxy's CPU at no cost in latency. -s increases the speed and +decreases CPU usage for high-speed large uploads and downloads, 3 times the +speed at 60-70% less CPU per gigabyte here, especially for traffic inside +high-speed local networks and virtual environments, where nothing but the +CPU limits a connection. +
Every spliced direction needs a pipe (two file descriptors) and more kernel +buffer memory. splice() is not used where 3proxy has to see the data: on a connection while +data filters apply to it (an HTTP body is spliced only once the filters are +done with it: they saw it whole, or it is larger than filtermaxsize, which +they skip), and not at all by a service that ends or begins TLS (ssl_serv, +ssl_cli).
TCP_NODELAY and splice() do not conflict and can be combined on high-speed connections.
proxy -g8000,3,10-The first parameter is the average read size we want to keep, the second parameter is -the minimal number of packets in the same direction to apply the algorithm, -and the last value is the delay added after polling and prior to reading data. -The example above adds a 10-millisecond delay before reading data if the average -polling size is below 8000 bytes and 3 read operations have been made in the same -direction. It's especially useful with splice.
logdump 1 1is useful -to see how grace delays work; choose a delay value to avoid filling the read -buffer (typically 64K) but keep the request sizes close to the chosen average -on large file uploads/downloads. +
proxy -g8000,2,2+-gTRAFF,NUM,DELAY: once NUM reads in a row in one direction, within the same +second, have averaged TRAFF bytes or less, 3proxy waits DELAY milliseconds +before it polls and reads again, so that data gathers in the socket and is read +in fewer, larger pieces. The example waits 2 milliseconds after two reads in a +row averaging 8000 bytes or less. The counts start again every second and +whenever the other direction is read, and the wait stops as soon as reads grow +larger than TRAFF on average. +
Fewer reads mean fewer system calls, wakeups and context switches: it +decreases CPU usage on connections which carry data in small pieces, such as +bulk transfers to and from clients on slow links. The cost is the throughput of +a single connection. While 3proxy waits, it reads at most a buffer's worth +after each wait: 16K copying (with the default buffer size; less with -b), up +to 64K (the pipe) with splice(). This caps a connection's direction at one +buffer per DELAY: +
| DELAY | copying, 16K per read | splice(), 64K per read |
|---|---|---|
| 1 ms | 131 Mbit/s | 524 Mbit/s |
| 2 ms | 65 Mbit/s | 262 Mbit/s |
| 3 ms | 44 Mbit/s | 175 Mbit/s |
| 5 ms | 26 Mbit/s | 105 Mbit/s |
| 10 ms | 13 Mbit/s | 52 Mbit/s |
Measured on the same Intel machine as splice() above, the clients' links +emulated for each connection (rate limit, delay and loss each way, with fq and +netem on the client's side), 16 streams through tcppm and a mix of 20000 +connections at a fixed 3000 exchanges a second (web-like answers, median 12K), +against 3proxy without grace delay: +
| client link | setting | 16 streams: throughput | 16 streams: 3proxy CPU per gigabyte | mix, HTTP proxy: CPU busy |
|---|---|---|---|---|
| 100 Mbit/s, 2 ms, 0.1% loss | -g8000,2,2 | the same (100 Mbit/s each) | the same: reads are large, waits do not apply | the same (73%) |
| -g6000,3,2 | ||||
| -g8000,2,3 | ||||
| 20 Mbit/s, 5 ms, 0.5% loss | -g8000,2,2 | 3% less | 41% less | 88% instead of 97% |
| -g6000,3,2 | 3% less | 35% less | 92% | |
| -g8000,2,3 | 7% less | 46% less | 88% | |
| 5 Mbit/s, 25 ms, 1% loss | -g8000,2,2 | the same (held by the loss) | 21-23% more | CPU-bound with or without |
| -g8000,2,10 | 5-8% less | 21-29% less | ||
| -g8000,2,20 | 12-17% less | 31-32% less |
logdump 1 1is useful to see how grace +delays work.