An answer's headers were sent by themselves, and what hdrread() had read
of the body after them by sockmap() next: two segments where one does,
for most small answers. The headers, still where hdrread() put them in
the server's buffer, move up against the body, and go with what of it
was read in one send; an answer whose whole body came with its headers
is not mapped at all. Not for chunked answers, answers with no body, or
where a data filter is to see the body; the bytes were counted as they
were read.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A connection kept alive waited for its next request holding both of its
buffers, empty: most of the memory of an HTTP proxy under load. They are
let go before the wait and made again as something comes, but for a
request httpsrv keeps there to hand over.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A client whose fiber gave up its stack while relayed never comes back to
the end of proxychild(), which frees what it allocated: the request line
and the FTP base were lost with each CONNECT, the tail having a copy of
the line. The line is the tail's now, taken back if mapping returns, and
the FTP base is freed before mapping.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
give up their fiber while relayed
The children for which mapping is their last thing set their tail too:
- socks, for CONNECT: its log line; the request's buffer is freed before
mapping, nothing needing it after;
- proxy, for CONNECT: the status page its end sends for the result
where one is, now proxystatus() for both, and the log line of its
request, a copy of which is the tail's. Not in a request httpsrv hands
over, whose end is httpsrv's;
- pop3p, imapp (with a copy of its tag), smtpp (its command freed before
mapping): the log line and the error reply their ends send.
A tail may wait now, sending to the client: once mapping is over the end
of a client whose fiber gave up its stack runs on a fiber taken again
for it, which, after a reload, has sockmap() go on with the mapping
first. Only with no stack to be had is it ended there, logged with no
tail.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A fiber waiting on a socket registered it with its loop and took it
off again on every wait: two epoll_ctl() calls a wait, some 39 for an
exchange of the mix benchmark, about an eighth of the proxy's time. A
socket is now registered once, edge-triggered for reading and writing,
until it is closed, and the loop keeps what it was reported ready for
until an operation finds it not to be: read, write, accept, connect,
splice() from a pipe and TLS tell it so through fiberblockfunc, the
close through fiberclosefunc. A wait for what a socket is known to be
ready for is answered at once, with no system call and no switch; only
with none of them ready does the fiber wait in its loop. Readiness is
trusted once: answered again with nothing reported or found since, what
was there may have been taken without its end being found, and the
system is asked with poll(). epoll and kqueue only; with poll() each
wait still registers its sockets.
What takes readability for an end rather than reading, the control
connection of a UDP association, the client waiting for a SOCKS BIND,
the server between requests of a kept-alive HTTP connection, asks
first with sockreadable(), a peek below any TLS layer.
kqueue: the end of either side, the peer's or our own shutdown, is
readable or writable, not a hang up, which poll() reports only with
both: kept, it made a wait for the answer after our shutdown return
at once, and sockmap give up with 93.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The bandwidth limiter between HTTP requests and httpsrv's delay slept
with usleep(); they are waits of sockpoll() with no fds, to be made
other than by sleeping once a client is served by an event loop.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A layer over the sockets, TLS or a plugin's, no longer waits itself.
Its _prepoll only tells what it knows without asking the system: the
fds it knows to be ready, data TLS has already decrypted, or, returning
0, that the sockets have to be polled, for events it may rewrite: a
TLS read waiting for its socket to be writable. The wait is
sockpoll()'s, which calls _prepoll first and then polls, so that a wait
can be made other than with poll() in one place: a client of an event
loop is to wait in the loop. Whatever a rewritten wait gets is given to
the caller as what it asked for, its chance to try again, and the
events it asked for are given back. With no fds sockpoll() sleeps,
which WSAPoll() would refuse.
TLS remembers what a read or a write it could not do waits for. A read
needing to write, or a write to read, was an error which ended the
connection, as when TLS 1.3 answers a key update while the socket
cannot take more; it now waits for what TLS needs, as one needing what
it was asked for always did.
Incompatible for plugins, which have to be built again with the same
version of 3proxy, the structures they share not kept compatible:
- struct sockfuncs has _prepoll(state, fds, nfds) in place of
_poll(state, fds, nfds, timeout). A plugin replacing _poll replaces
_prepoll: it must not wait, returns the number of fds it set revents
of (events then left as they are), 0 for the fds to be polled (with
events as it leaves them, revents left to the poll) or -1, and calls
the _prepoll it replaced when it has nothing to say itself.
- Waiting for sockets is sockpoll(sf, state, fds, nfds, timeout), not
so._poll() of the socket functions.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
rpcndr.h, which windows.h brings in, defines small as char, so the
buffer the rest of a request is read into, unsigned char small[256],
did not compile with MSVC. Renamed, and the splice heuristic's small
in mainconf.c too.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
REQMAX is 4 KB: a log line holds 4000 characters, so a longer copy of
the request line was cut by the log before "..." could show.
The headers this proxy adds are written straight after the header
block, HDRRESERVE (2 KB, what they come to at most) asked to be free
there first: in the side's buffer that is the room hdrread() left, so
nothing is copied, and the buffer on the stack they were built in is
gone.
The scheme and authority of a request are no longer kept in a buffer
of their own: req, the line in absolute form as logged, has them, and
where the authority begins and ends in it is kept instead, for the
next request on the connection to be compared with and for a parent
proxy to be given. An authority beyond what req keeps of the line is
refused. A line a request filter rewrote is logged only when the
rewrite is applied.
Tests: a 5 KB line through a parent arrives whole in absolute form.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A request to a parent HTTP proxy went as two writes, the line as the
client sent it from req and then the rest of the headers from buf,
held together with TCP_CORK, which does not help over TLS nor with
headers bigger than a segment. The line now goes back to absolute form
where it is: a request's header block is read HDRFRONT (300) bytes
into clibuf, room for http://, a host name of 256 characters and a
port; the method moves down into it and the scheme and authority go
in after it, and line and headers are sent in one piece. Where they
do not fit, or the headers are in a buffer of their own by then, both
are put together in one. TCP_CORK is gone.
The scheme and authority of a request are kept for that (auth), from
its line or, for a request with a path only, its Host header; the last
one is what the next request on the connection is compared with. A
CONNECT line is kept whole in buf. The request filters are given the
line in absolute form in a buffer of their own, made only when there
are some, as req was. req is the copy logged and nothing else: at most
REQMAX (6 KB) of the line, "..." after what is cut, never a reason to
refuse the request.
The headers this proxy adds are built in a buffer on the stack, of
bounded parts, and put after the header block once there is room for
them, in a standalone proxy too, which never asked for it. HDRRESERVE
is 1 KB.
Tests: through a parent a POST arrives with its line in absolute form,
its headers and body whole; user and password in the URL; headers in a
buffer of their own (a data filter) put together with the line; a 7 KB
request line served whole and logged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The HTTP proxy read each request into a buffer of 64 KB of its own,
cleared it with memset, and read the answer's headers into it again;
the header lines were copied there one by one from clibuf or srvbuf.
Now the header block of a request is read whole into clibuf, and that
of an answer into srvbuf, and worked on where it is (hdrread()):
- the block is moved to one byte into the buffer, and its lines, as
they are kept, are moved down to its start, so the line being worked
on, and the 0 written after it, never reach the next one; headers
taken out are left behind;
- HDRRESERVE is left free after it, for the headers this proxy adds,
before whatever came after it, a body or another request, from
where the connection is read on;
- the buffer is grown only for a block bigger than it, up to the
limits there were (512 KB for a request, 64 KB for an answer);
- the headers are copied to a buffer of their own only when they need
more room than that, when a filter hands back one, when a body is to
be read whole into the same buffer for a data filter, and for an ftp
listing, which is written where the headers were.
Whether buf is the headers' own buffer is kept with it (bufown), not
found from where it points: the side's buffer may be a bigger one by
then let go of. filterswap() leaves alone a buffer in either side's
buffer, which may be one grown by malloc, not only their fixed arrays.
What took buf for a buffer of LINESIZE does no longer: the request is
logged from a copy of its line, and the rest of a request refused
before its headers were all read is drained through a small buffer of
its own, keeping track of where its lines begin.
Tests: a POST whose body comes in the same packet as its headers,
which are rewritten, with a header added and one taken out; 45 KB of
request headers; two requests pipelined in one packet; empty lines
before a request; an answer with a 30 KB header; a 600 KB header block
refused, the proxy answering after it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
UDP sessions check for a reload in udpsockmap() as TCP connections do
in sockmap(): unless noforce, a session goes by the newest service on
its listener, re-authenticated as its client, or is dropped.
A request on a connection that outlives a reload, an HTTP one kept
alive, is authorized by clientauth(): it moves the connection to the
newest service on its listener first if a reload came since, then
authorizes as the service whose rules apply (PSRV). The proxy uses it
for every request, so a request of a user the new configuration
removed is answered 407 rather than authorized by the old rules and
cut while relayed. reattach() takes whether to authenticate: sockmap()
and udpsockmap() move and authenticate at once, clientauth() moves and
then authorizes the request.
A test case covers the drops: with one slot free in the new service one
of two downloads moves on and the other is cut (27; a service keeps one
of its maxconn slots for the next connection it accepts), and a proxy
download is cut when a socks service takes its port (26). The reload
case checks the kept-alive request.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A reload reads the file into a new configuration, allocated from the
defaults, and prepares its services without starting them. Only if the
whole file is read without an error does it come into force: the old
configuration's services stop accepting, the new ones start. On an
error everything prepared is dropped and the old configuration, which
never stopped, stays in force.
Each service holds its configuration until it is freed, after its last
client; a configuration no longer in force is released when the last
holder lets go (confhold/confput). freeconf() now only releases what a
configuration holds, touching nothing global.
What runs without a service of its own reads the configuration in
force, runconf, rather than conf, which a reload points at the one
being read: the resolver, which holds it while it waits (a client
resolves with its own service's), the log, the thread pool, the admin
pages, which walk the services under config_mutex, and the signal
handlers.
The users table, havelog and the resolver choice are per configuration
now, the resolver applied when the configuration comes into force, so
a refused file leaves the users as they were. The log file, the
configuration file name, the string table, daemon mode and the filter
list belong to the process and are handed from one configuration to
the next. Settings left out of a new file go back to their defaults,
where before they kept the old values.
The gap between the old and the new listening sockets remains: a
connection queued on an old service's socket when it closes is reset.
Handing the socket over is the next phase.
A test case reloads a broken file (nothing changes, alice still gets
in, bob from the refused file does not) and a good one (bob only, the
service on its new port, the old one closed, a download through the
old service finishing whole). 40 reloads, a third refused, under load
show no memory errors under AddressSanitizer.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
conf is now a pointer to the current configuration rather than the
structure itself: at first confboot, which exists before main() runs, as
plugins and the standalone services use it too; later configurations
will start from confdefaults. Both are made from confinit.h, the one list
of initial values. The timeouts are an array of the configuration
instead of a pointer to a global one. Nothing behaves differently yet.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Like the timeouts, each service now keeps the values given before it
instead of reading the global ones while its connections run. setopts()
takes the service's maxseg. The TLS handshakes wait the service's
CONNECT_TO, kept in its SSL configuration, rather than the global one.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
proxychild() allocated its 64 KB buffer for headers once and kept it
for the life of the connection: for a CONNECT tunnel all the time data
was relayed, and for a kept-alive connection between its requests.
With 2000 tunnels that was 128 MB, with 20000 mostly idle connections
over a gigabyte.
The buffer is now freed before a tunnel starts relaying, and when a
kept-alive connection has waited a second for its next request; it is
allocated again once the next request starts coming. logurl() takes a
copy of its own for a request it logs after its buffer was freed.
2000 tunnels: 339 to 213 MB; 20000 connections, 3000 exchanges a
second: 2.65 to 1.63 GB through the HTTP proxy, 2.66 to 1.42 GB through
CONNECT; busy kept-alive connections within 1.5% of before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A kept-alive client connection reuses the connection to the server while
requests go to the same host. The check compared the start of the request
line up to the end of the host, method included, so a POST after a GET to
the same host, or a GET after a POST, closed the server connection and
opened another one. Only the host and port are compared now.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
pop3p, imapp and smtpp are built with MAILPROXY=true, and ftppr and the
ftp:// scheme of the HTTP proxy with FTP=true; with CMake the switches are
3PROXY_USE_MAILPROXY and 3PROXY_USE_FTP. Neither is in a default build, and
the standalone binaries follow what was built.
A configuration naming one of them is still read either way. The three mail
protocols amount to a STARTTLS negotiation now that mail is carried over
TLS, so without a proxy of their own those names are tlspr speaking the
protocol: the file holds a stand-in which sets the protocol for the
connection and returns tlspr for the caller to run, which serves the
service name and a parent chain alike and leaves conf.c and the redirect
table untouched. FTP has no such fallback, so the service is known, answers
nothing and logs the refusal.
The Linux workflow builds both, so all of it is still compiled and run.
A rewrite only reached the copy of the request kept for logging. On a
direct connection the server is sent the request line held in the header
buffer, which was parsed and reduced to its path before the filters ran, so
the rewrite showed up in the log and nowhere else. Through an HTTP parent
the rewritten copy is what goes out, which is why it appeared to work
there.
Splice the new path back into that buffer. The destination was chosen, and
the access rules applied to it, before the rewrite happened, so a rewrite
that changes the method or the authority is left alone: acting on it would
send the request somewhere the rules never saw. Say so in the manual.
The copy needed to notice a rewrite is only taken when a request filter
exists, so a proxy without one does no extra work.
ss pointed into the buf request buffer and was never updated when buf
was realloc()ed while reading request headers (both in the header loop
and in handlehdrfilterscli()). The ftp:// branch then dereferenced the
stale pointer.
Store ss as an offset into buf and recompute it before use.
Requires authenticated access to proxy or configuration with
authentication disabled.
Reported by Calif.io in collaboration with Anthropic
(ref ANT-2026-FHVKZ7VQ).
- authcache switched to use hashtables, size parameter added
- overflow fixed on hashinit
- hashtable prefers new values on insert if table is full
- hashtable is able to compact/grow
`proxy -g8000,3,10`
First parameter is average read size we want to keep, second parameter is
minimal number of packets in the same direction to apply algorythm,
last value is delay added after polling and prior to reading data.
An example above adds 10 millisecond delay before reading data if average
polling size is below 8000 bytes and 3 read operations are made in the same
direction. It's specially usefule with splice. `logdump 1 1` is useful
to see how grace delays work, choose delay value to avoid filling the read
pipe/buffer (typically 64K) but keep the request sizes close to chosen average
on large file upload/download.
