Commit Graph
32 Commits
Author SHA1 Message Date
Vladimir DubrovinandClaude Opus 5.5 1a193eb82d highload.html: kernel TLS, splice() heuristics, grace delay by traffic
A section on ssl_ktls: what it needs, how 3proxy reports it, and that
it pays with splice() for TLS connections carrying large transfers.
The splice() section gets the :MIN,STREAK,SMALL tuning and says TLS
services splice where the kernel does TLS. Grace delay is recommended
by the clients' speed for steady streams only: for web traffic it
saved nothing on any of the links measured, nor with the second CPU
the HTTP proxy needed on the 5 Mbit/s one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 19:14:46 +03:00
Vladimir DubrovinandClaude Opus 5.5 0917b02314 highload.html: grace delay and splice() with measured figures
Grace delay is described as it works: NUM small reads in a row in one
direction within a second start the wait, larger reads end it. It
saves CPU on connections carrying data in small pieces and caps a
connection at one read per DELAY, 16K copying and 64K (the pipe) with
splice(); a table gives the caps for 1-10 ms, another the measured
gains per client link, and DELAY is to be matched to the clients'
speed. It does not help once the CPU is exhausted. The example becomes
-g8000,2,2: the old -g8000,3,10 capped copying at 13 Mbit/s.

splice() gets measured figures: 3 times the speed at 60-70% less CPU
for large transfers on a fast network, within 0-12% for web traffic
in the mix over emulated links. -s, the automatic mode, is the
default; the paragraph still said -s0 was. splice() is not used where
3proxy has to see the data: while data filters apply, and by services
ending or beginning TLS.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 17:06:47 +03:00
Vladimir DubrovinandClaude Opus 5.5 b9297dc4cf highload.html: benchmark figures, linked on devel
The benchmarks live on the devel branch only, so link to their results
there; master has no bench/.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 13:59:20 +03:00
Vladimir DubrovinandClaude Opus 5.5 626aac16bc splice() per direction, pipes per connection, automatic mode
-s takes the directions to splice: -s1 what the server sends (downloads),
-s2 what the client sends (uploads), -s3 both. -s alone (-s7) splices both
only where it pays, as -s5 and -s6 do for one direction:
- a body of known length is spliced if at least 16K of it is left to read
  once what is already buffered is counted, so an answer read whole with
  its headers is just copied;
- a stream of unknown length starts copied, is spliced after 2 reads in a
  row of at least 16K, and goes back to copying after 3 splices in a row
  of less than 4K, once its pipe is empty.

Pipes are made when a direction first splices and kept with the
connection for its next mappings, so a kept-alive connection no longer
makes and closes two pipes per request; one left holding data by an
error or a timeout is closed. The direction not spliced keeps its buffer.

Before, -s spliced both directions and every mapping made both pipes,
which made small answers slower than without splice: with kept-alive
50K answers 2 pipe2() and 4 close() per request, now none.

Also corrects the default buffer size in the -b help (65536 for TCP).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 00:39:06 +03:00
Vladimir DubrovinandClaude Opus 5.5 c13c048f3f Benchmarks: measure the proxy under stress, add results
bench.py:
- the proxy gets one CPU, the client four, the origin three (8 CPUs), so
  the proxy is what runs out first
- CPU time is the busy time of a role's CPUs from /proc/stat, softirqs
  included: with IRQ time accounting the cgroup's own figure leaves out
  much of the network work
- every run records which side set its pace (proxy, client, origin), by
  the busy share of the role's CPUs and of its busiest thread
- a 50 KB keep-alive scenario with 2000 clients
- haproxy with splice: haproxy::splice-auto and haproxy::splice
- wrk timeouts counted apart from errors; tools found by absolute path;
  leftovers of an interrupted run are cleaned up

report.py turns runs into Markdown tables: throughput, the proxy's CPU
per gigabyte or per request, its peak memory, splice and copying proxies
in groups of their own, the best of each group marked.

RESULTS.md has the comparison of 0.9.4, 1.0.1, devel, nginx and haproxy,
with and without splice, on an Apple M4 and an Intel Skylake virtual
machine, and how it was measured; data/ has the measurements.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 22:04:47 +03:00
Vladimir Dubrovin c827f77137 Support = in socket options, make TCP_NODELAY default 2026-10-11 20:56:49 +03:00
Vladimir Dubrovin 75021a2003 Update highload.html 2026-08-12 14:49:44 +03:00
Vladimir Dubrovin bebc3894e2 Support acls for UDP ASSOC, fix udppm via SOCKSv5
C/C++ CI Linux / ${{ matrix.target }} (ubuntu-24.04-arm) (push) Waiting to run
C/C++ CI Linux / ${{ matrix.target }} (ubuntu-latest) (push) Waiting to run
C/C++ CI MacOS / ${{ matrix.target }} (macos-15) (push) Waiting to run
C/C++ CI Windows / ${{ matrix.target }} (windows-2022) (push) Waiting to run
C/C++ CI cmake / ${{ matrix.target }} (macos-15) (push) Waiting to run
C/C++ CI cmake / ${{ matrix.target }} (ubuntu-24.04-arm) (push) Waiting to run
C/C++ CI cmake / ${{ matrix.target }} (ubuntu-latest) (push) Waiting to run
C/C++ CI cmake / ${{ matrix.target }} (windows-2022) (push) Waiting to run
C/C++ CI cmake / ubuntu-latest (wolfSSL) (push) Waiting to run
2026-08-11 19:02:54 +03:00
Vladimir Dubrovin 0ae2754c1e make html formatting compatible with .md
C/C++ CI Linux / ${{ matrix.target }} (ubuntu-24.04-arm) (push) Has been cancelled
C/C++ CI Linux / ${{ matrix.target }} (ubuntu-latest) (push) Has been cancelled
C/C++ CI MacOS / ${{ matrix.target }} (macos-15) (push) Has been cancelled
C/C++ CI Windows / ${{ matrix.target }} (windows-2022) (push) Has been cancelled
C/C++ CI cmake / ${{ matrix.target }} (macos-15) (push) Has been cancelled
C/C++ CI cmake / ${{ matrix.target }} (ubuntu-24.04-arm) (push) Has been cancelled
C/C++ CI cmake / ${{ matrix.target }} (ubuntu-latest) (push) Has been cancelled
C/C++ CI cmake / ${{ matrix.target }} (windows-2022) (push) Has been cancelled
C/C++ CI cmake / ubuntu-latest (wolfSSL) (push) Has been cancelled
2026-08-01 11:47:50 +03:00
Vladimir Dubrovin 763e21e053 Fixed: maxchild dropped to 100 on config reload if not explicitly set, system-specific tuning hints added to doc 2026-08-01 10:22:03 +03:00
Vladimir Dubrovin 7b85f6a684 Disable splice support
C/C++ CI Linux / ${{ matrix.target }} (ubuntu-24.04-arm) (push) Waiting to run
C/C++ CI Linux / ${{ matrix.target }} (ubuntu-latest) (push) Waiting to run
C/C++ CI MacOS / ${{ matrix.target }} (macos-15) (push) Waiting to run
C/C++ CI Windows / ${{ matrix.target }} (windows-2022) (push) Waiting to run
C/C++ CI cmake / ${{ matrix.target }} (macos-15) (push) Waiting to run
C/C++ CI cmake / ${{ matrix.target }} (ubuntu-24.04-arm) (push) Waiting to run
C/C++ CI cmake / ${{ matrix.target }} (ubuntu-latest) (push) Waiting to run
C/C++ CI cmake / ${{ matrix.target }} (windows-2022) (push) Waiting to run
C/C++ CI cmake / ubuntu-latest (wolfSSL) (push) Waiting to run
2026-08-01 09:38:49 +03:00
Vladimir Dubrovin 3d9d76fcf5 Update documentation 2026-04-08 19:21:30 +03:00
Vladimir Dubrovin 55d1bbe155 Grace delay feature added
`proxy -g8000,3,10`

First parameter is average read size we want to keep, second parameter is
minimal number of packets in the same direction to apply algorythm,
last value is delay added after polling and prior to reading data.
An example above adds 10 millisecond delay before reading data if average
polling size is below 8000 bytes and 3 read operations are made in the same
direction. It's specially usefule with splice. `logdump 1 1` is useful
to see how grace delays work, choose delay value to avoid filling the read
pipe/buffer (typically 64K) but keep the request sizes close to chosen average
on large file upload/download.
2022-05-19 18:51:02 +03:00
z3APA3A 37c380f4f7 Documentation example corrected 2019-12-04 17:56:06 +03:00
z3APA3A a905d00008 documentation update 2019-06-26 16:52:55 +03:00
z3APA3A 43b4c893ad Minor documentation updates 2019-06-20 11:18:13 +03:00
z3APA3A d89134be33 minor doc update 2019-03-05 15:02:04 +03:00
z3APA3A 3a1f8c2328 typo corrected 2019-02-22 17:16:36 +03:00
z3APA3A 158bacc087 Highload documentation update 2019-02-22 17:05:22 +03:00
z3APA3A 536c8327ff Minor man updates 2019-02-14 18:35:12 +03:00
z3APA3A 54e9f9cedd Highload docs correction 2019-01-11 19:11:31 +03:00
z3APA3A b56c294c01 Update highload.html with splice() requirements 2018-04-28 01:56:13 +03:00
z3APA3A a816ed2343 force/noforce description usage 2017-12-19 01:22:25 +03:00
z3APA3A 41b7b03db2 updated to highload doc 2017-10-10 18:21:50 +03:00
z3APA3A b3b76fe682 RADIUS example added 2017-10-08 23:35:39 +03:00
z3APA3A 695bfc89ac High load documentation update 2017-10-08 23:20:52 +03:00
z3APA3A 2fd9f945c8 SO_PORTSCALABILITY documented 2017-10-06 21:50:52 +03:00
z3APA3A 6bbe4181fc update to high load documentation 2017-10-06 21:39:28 +03:00
z3APA3A 29b834cf56 update for highload documentation 2017-10-06 21:17:15 +03:00
z3APA3A e588dce959 update highload documentation 2017-10-06 20:40:07 +03:00
z3APA3A 2f5265bee6 highload documentation updated 2017-10-06 18:38:27 +03:00
z3APA3A 4d3062c62a highload documentation added 2017-10-06 18:13:26 +03:00