A listener counts the traffic of its clients as it is logged: bytes from
servers (%I) and from clients (%O). clearstat(), which every log writer
calls once a line is logged, as well as lognone and logradius, adds the
counts to the client's listener before clearing them, through
listenertrafficfunc, set where services have listeners and NULL in the
standalone binaries.
The admin services page shows them as listenertrafin and
listenertrafout. The takeover case checks that they go on through three
reloads.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A listener counts its clients, whichever of its services they came to,
old or new: those connected now, raised when a client is spawned, and
those done with, raised as one leaves its service. As the listener
outlives a reload, so do the counts.
The services page of the admin interface shows them for each service as
listenerclients and listenerprocessed; services sharing a socket show
the same counts. The takeover case checks that the count goes on
through three reloads and that no client is left counted.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
UDP sessions check for a reload in udpsockmap() as TCP connections do
in sockmap(): unless noforce, a session goes by the newest service on
its listener, re-authenticated as its client, or is dropped.
A request on a connection that outlives a reload, an HTTP one kept
alive, is authorized by clientauth(): it moves the connection to the
newest service on its listener first if a reload came since, then
authorizes as the service whose rules apply (PSRV). The proxy uses it
for every request, so a request of a user the new configuration
removed is answered 407 rather than authorized by the old rules and
cut while relayed. reattach() takes whether to authenticate: sockmap()
and udpsockmap() move and authenticate at once, clientauth() moves and
then authorizes the request.
A test case covers the drops: with one slot free in the new service one
of two downloads moves on and the other is cut (27; a service keeps one
of its maxconn slots for the next connection it accepts), and a proxy
download is cut when a socks service takes its port (26). The reload
case checks the kept-alive request.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
After a reload, unless noforce, a client of an old service goes by the
rules of the newest service on its listener, the one accepting there
now. It takes one of that service's maxconn slots, which counts it
there and keeps that service and its configuration alive until the
client is done, and is re-authenticated as its client: access lists,
users, authentication, bandwidth limits and traffic counters come from
the new configuration. A later reload moves it on, giving the slot
back. The client stays its own service's child, which keeps its
filters and TLS state.
It is dropped if its service was removed or is of another type now
(26), or if the newest service has no free slot (27); what the
re-authentication refuses ends it as before. A service without a
listener (UDP, connecting back) keeps its clients as they are.
This restores what force did before the configuration was kept per
service: since then a client compared its version with its own
service's configuration, which never changes, and was never checked
again. Services now take their configuration's version as it is: a
reload numbers the new configuration before reading it, and so does
startup now, where services were numbered one ahead.
A listener now stays with its old service until that service's last
client is gone, those clients looking the newest service up through
it; the socket itself is still closed as soon as nobody accepts on it.
Connection limits are given back in the configuration they were taken
in. PSRV() is the service whose rules apply to a client.
The reload case checks that a download of a user the new configuration
keeps finishes under it, and that one of a user it removes is cut.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A service's TCP listening socket is kept as a listener: when a reload
commits, before the old services are told to stop, each new service
takes over the listener of an older configuration's service on the same
address, port, network namespace and device with the same -ol options.
It skips socket() and bind(), calling only listen() again for its own
backlog, and logs "Accepting connections, taken over". The old service
stops accepting at its next poll and lets go of the socket; whoever
lets go last closes it, unlinking a Unix socket's path.
So a service staying where it was no longer leaves a gap: before, the
old one closed its socket and the new one opened another, and a
connection queued on the old socket was reset. UDP services, whose
clients read from the service's socket, and two services of the same
configuration on one address keep their own sockets as before.
A test case makes requests nonstop through three reloads and wants none
refused or reset; against the build before this, connections were
reset. The reload case no longer waits out the switch. SIGUSR1 in
3proxy(8) describes how a reload behaves now.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A reload reads the file into a new configuration, allocated from the
defaults, and prepares its services without starting them. Only if the
whole file is read without an error does it come into force: the old
configuration's services stop accepting, the new ones start. On an
error everything prepared is dropped and the old configuration, which
never stopped, stays in force.
Each service holds its configuration until it is freed, after its last
client; a configuration no longer in force is released when the last
holder lets go (confhold/confput). freeconf() now only releases what a
configuration holds, touching nothing global.
What runs without a service of its own reads the configuration in
force, runconf, rather than conf, which a reload points at the one
being read: the resolver, which holds it while it waits (a client
resolves with its own service's), the log, the thread pool, the admin
pages, which walk the services under config_mutex, and the signal
handlers.
The users table, havelog and the resolver choice are per configuration
now, the resolver applied when the configuration comes into force, so
a refused file leaves the users as they were. The log file, the
configuration file name, the string table, daemon mode and the filter
list belong to the process and are handed from one configuration to
the next. Settings left out of a new file go back to their defaults,
where before they kept the old values.
The gap between the old and the new listening sockets remains: a
connection queued on an old service's socket when it closes is reset.
Handing the socket over is the next phase.
A test case reloads a broken file (nothing changes, alice still gets
in, bob from the refused file does not) and a good one (bob only, the
service on its new port, the old one closed, a download through the
old service finishing whole). 40 reloads, a third refused, under load
show no memory errors under AddressSanitizer.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
-s[N]:MIN,STREAK,SMALL sets where the automatic mode splices, as -g
does for grace delay: a direction is spliced after STREAK reads in a
row of MIN bytes or more (SMALL from a TLS socket), copied again after
more than STREAK splices in a row of less than SMALL, and a body of
known length is spliced if MIN or more of it is left. The defaults,
16384,2,4096, are what was built in; a value left out or 0 keeps its
default.
The man page describes it, and its HTML copy, which still said splice()
was not built by default, now has the current text. A test case relays
answers of every size, chunked ones and a 3 MB upload through every
mode, two of them with heuristics set, and compares them byte for byte.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A service with ssl_ktls keeps its splice() mode. A connection whose TLS
sockets all got kernel TLS both ways may splice, as a plain one does when
no data filters apply; one where the kernel did not take TLS (nosplice)
copies as before.
TLS may hold data for a socket that a splice() from it would not see:
OpenSSL reads more from a kernel TLS socket than it is asked for. sockmap
asks the TLS code (splicetls) and reads that through TLS into the buffer
first, once the pipe has drained, so the order is kept, then splices on.
A read from a TLS socket counts as large at 4K, TLS handing out a record
at a time, and the short read collecting what TLS left from the last one
does not end a run of large ones. A splice() refused because a record is
not data (an alert, a key update, a ticket) falls back to reading it
through TLS instead of ending the connection.
The kernel TLS test compares the bodies with the origin's.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
conf is now a pointer to the current configuration rather than the
structure itself: at first confboot, which exists before main() runs, as
plugins and the standalone services use it too; later configurations
will start from confdefaults. Both are made from confinit.h, the one list
of initial values. The timeouts are an array of the configuration
instead of a pointer to a global one. Nothing behaves differently yet.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
What a numbered counter counted now belongs to the process: a counter
rule points to the value of its number, created the first time the
number is met (from the counter file, if one is open) and shared by the
rules of that number in every configuration loaded. A reload no longer
resets it when there is no counter file, and once configurations can
outlive one another, clients of the old and the new one count into the
same value. An unnumbered counter, and those LdapPlugin makes, keep a
value of their own as before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
conf.haveerror was only initialised and srvparam.srvfds only declared;
the extern declarations of stdlog and demon named globals that do not
exist (both are fields of conf); a local in make_ace() and one in the
webadmin XML page were never used.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
They are the state of the process, not configuration: conf.time (now
curtime), timetoexit and needreload are globals of their own. Plugins
reach needreload through a pointer added at the end of struct
pluginlink; LdapPlugin uses it. The extern for a paused global, never
defined, goes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nservers, numservers and authnserver (resolve.c, common.c) and
radiuslist, nradservers and radiussecret (authradius.c) were globals of
their own; they are configuration, set by nserver, authnserver and
radius, and now live in struct extparam with the rest of it.
MAXNSERVERS, MAXRADIUS and struct radserver move to structures.h, which
plugins include without proxy.h.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Like the timeouts, each service now keeps the values given before it
instead of reading the global ones while its connections run. setopts()
takes the service's maxseg. The TLS handshakes wait the service's
CONNECT_TO, kept in its SSL configuration, rather than the global one.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Outside Linux nssocket() was a macro, and two of its callers have #ifdef
WITH_UN among its arguments. Preprocessor directives in macro arguments
are undefined behaviour: gcc and clang take them, MSVC does not. It is a
function on every system now.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
-s takes the directions to splice: -s1 what the server sends (downloads),
-s2 what the client sends (uploads), -s3 both. -s alone (-s7) splices both
only where it pays, as -s5 and -s6 do for one direction:
- a body of known length is spliced if at least 16K of it is left to read
once what is already buffered is counted, so an answer read whole with
its headers is just copied;
- a stream of unknown length starts copied, is spliced after 2 reads in a
row of at least 16K, and goes back to copying after 3 splices in a row
of less than 4K, once its pipe is empty.
Pipes are made when a direction first splices and kept with the
connection for its next mappings, so a kept-alive connection no longer
makes and closes two pipes per request; one left holding data by an
error or a timeout is closed. The direction not spliced keeps its buffer.
Before, -s spliced both directions and every mapping made both pipes,
which made small answers slower than without splice: with kept-alive
50K answers 2 pipe2() and 4 close() per request, now none.
Also corrects the default buffer size in the -b help (65536 for TCP).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
extport binds a local port out of its range and connects from it. A port
which is free to bind may still be half of a pair which is closing from an
earlier connection to the same destination, and Windows fails the connect
with WSAEADDRINUSE where other systems fail the bind and fall back. The
request then failed with error 13 for a reason another port would have
avoided, which is what the range is searched for in the first place.
Retry the connect with another port from the range. Only for that error, so
a destination which is simply unreachable still costs one attempt, and out of
the same ten tries the bind already spends, so the attempts parentretries
makes are not multiplied. EADDRINUSE joins the errno values proxy.h maps to
their Winsock numbers, without which the test would never match on Windows.
parent_ports gave its three services one window between them, which is how
the case met the collision on the Windows runner. Each has its own now.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
