The part after the @ of a port in a rule names the service: by its
port, by the address it listens on, or by both, the address in
brackets when IPv6 —
connlim 5 60 * * * @1080
connlim 5 60 * * * @10.0.0.1:1080
deny * * * @[::1]
A slash, as the delimiter of the first take, would have read as a
CIDR habit the day a service is named inside an address list, and
@ cannot appear in a port or an address. Digits alone are a port, a
dashed pair a port range, anything else an address with a port behind
a colon; an @ with nothing after it is refused. The destination port
before the @ is unchanged, and so is every rule without one.
A port in a rule's targetportlist may carry a service port after a
slash, X-Y/S, or, for any destination, */S: the rule matches only
requests that arrived through the service listening on the port S,
the port of its proxy -p or socks -p, whatever their destination. The
limiters are shared by all the services of a configuration, so this is
how a limit is given to the connections of one service alone:
connlim 5 60 * * * */1080
denies the sixth concurrent connection of the service on port 1080.
The destination port and the service port are both ranges or single
ports, a slash with nothing after it is refused, and a rule without a
slash is what it always was.
mark <number> extends the allow or deny rule above it, as nolog and
weight do: a connection the rule lets through carries the mark on its
outgoing sockets, towards the destination and towards the parents, so
policy routing and firewall rules can tell its packets apart (SO_MARK
of Linux, SO_USER_COOKIE of FreeBSD; on a system with no mark the
command sets it to nothing). The UDP sockets of a session towards a
destination are marked the same, the SOCKS UDP ASSOCIATE relays among
them, for the rule that let the session through is looked up for each
destination. A mark with no rule above it, or not a number, is refused.
port tcp|udp [ip:]port [socket_options] opens, binds and listens at
the time the line is read, so a privileged port can be taken before
setuid and chroot drop the rights to open it: a service of the
configuration on the same address, of the same type, takes the socket
over when it starts, instead of binding one of its own, and a port no
service takes stays reserved until the configuration is left. A
reservation of the configuration before holds the port over a reload:
the new one shares its socket, each dropping its hold with its
configuration.
The socket is what the reservation made of it: the options which act
only before bind() cannot follow a service's wishes on a socket already
bound, so the service takes it as it is, its other options set on it.
The reservation is built from the same default address a service is, so
it matches in every byte a listener is compared by.
Datagrams on a TCP connection, each with a two byte length in network
order in front of it, which is how udptunnel (the rfc1036 one) and
ProxyGuard (eduVPN) put them there, the framing alone or after a
handshake: udptunnel's 32 bytes, or ProxyGuard's HTTP Upgrade for
UoTLV/1 answered with 101 Switching Protocols.
tcptoudp takes the connections, a UDP socket of its own to the target
for each, the handshake a client speaks recognised by its first bytes
and answered; a parent the ACL gives is a SOCKS5 UDP relay, as for
udppm. udptotcp takes the datagrams, each client a session of the
service's with a TCP connection of its own, connected as the service's
loop sees it writable (dnspr asks a name server over TCP the same way);
-m sends udptunnel's handshake to the target, -m2 ProxyGuard's, -m0
(the default) none, so both ends of either tool are stood in for.
The service names take two places of the string table, the default
configuration path moving two later (stringtable[27]).
Built by the Makefiles and CMake, with the standalone binaries, and
installed by them; a case of the test suite speaks all three handshakes.
ssize_t does not exist under MSVC, and the socket function table
already spells its send and recv results int there and ssize_t
elsewhere. That type is now named SOCKRET, beside SASIZETYPE, and the
event loop's copies of it use the name, so the MSVC build compiles
again.
A connection's buffers came from malloc(), which keeps what it is given
back: a buffer let go while the connection waits (HTTP keep-alive, an
idle relay) left its memory with the process. They now come from a
pool of memory mapped for them, in chunks aligned to their size, the
first unit of a chunk its header. A unit is a page, or a buffer if pages
are smaller: with pages of 4 or 16 KB a buffer is a unit of its own,
with pages of 64 KB a unit holds four. Free units are kept for the next
buffer, the most recently freed first; more than 64 and an eighth of
those in use, and the least recently freed are given back to the system
(MADV_DONTNEED on Linux, MADV_FREE_REUSABLE on macOS, MADV_FREE
elsewhere, MEM_RESET on Windows), and a chunk given back whole is
unmapped. Built for AddressSanitizer the buffers stay malloc()'s, which
it checks for overruns, unless WITHBUFPOOL is defined; the pool then
poisons the free ones.
Buffers are 16384 bytes, not 16384 + 64: a size of pages. The 64 bytes
were room for a TLS record's header, which tlspr needs to have a record
whole; tlspr now takes a larger buffer when a record does not fit, up to
TLS's largest (5 + 16384 + 2048), given back once it is emptied, and
relays read the whole buffer again, a record at most as before. A record
larger than the buffer set with -b is no longer refused.
HTTP proxy, 2000 idle keep-alive connections, after a request (Linux
container): 39.6 KB a connection before, 19.5 KB now; a thread a
connection, 44.2 KB before, 24.1 KB now; a TCP relay, 13.0 KB before,
11.6 KB now.
Incompatible for plugins: CLIENTBUFSIZE is 16384; clibufdata and
srvbufdata are the pool's, never to be passed to free() or realloc().
bufpoolinit() is called before any service starts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
-b, which made a service read less at once into a connection's buffer,
takes the size of its pipes after a comma: -b,262144, -b8192,32768.
From 4096 to 64M (MAXPIPESIZE); a pipe the system will not make so
large (pipe-max-size for an unprivileged process) is not used, and the
data is copied, as with -DPIPESIZE, which stays the default for a
service given none. sockmap() and the callbacks' relay fill a pipe up
to the service's size (SRVPIPESIZE()), and only a pipe of other than
64K costs an fcntl() as it is made.
The man page tells of -b, which it did not before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Win32 "lite" build with Open Watcom (Makefile.watcom, its release
workflow, the CMake branch for it) was the only one built without IPv6
(NOIPV6), without fibers and event loops, with a select() of its own
for poll() (mypoll()), and with the calling conventions of Watcom's
pragmas in the plugins. They are gone with it: every build has IPv6,
and polls with poll() on Unix and WSAPoll() on Windows, which were
what every other build asked for (WITH_POLL, WITH_WSAPOLL, the
3PROXY_USE_POLL and 3PROXY_USE_WSAPOLL options of CMake, FD_SETSIZE
for select()). getip(), the IPv4-only resolver of NOIPV6, goes too.
Plugin binary compatibility: plugins built with Open Watcom, and their
calling convention, are no longer supported.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
loopmode is cb unless a configuration says otherwise, as -L2 had it
for a service: its clients on event loops, a connection's data relayed
by the loop itself where nothing but the sockets is between its sides.
A service whose clients cannot share a thread (threadperclient: PAM,
LDAP, FilePlugin, names resolved by the system, -ne) or listening for
connections back (-R) still gives each a thread, as do the standalone
services and builds without fibers. loopmode on and -L keep the loops
with the relay on the client's stack; loopmode off and -L0 a thread
for each client.
The man page tells of cb. tests/run.py --loopmode takes the mode
(cb, on, off) to give every configuration, with two loops; the Linux
workflow runs the suite once more with a thread for each client, which
is no longer what a configuration gets by default.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A logger's lines are written by a thread of its own, its clients only
queueing them, so a logger whose writes wait cannot hold a client's
thread or a loop: its blocks flag, giving its services a thread for each
client, is gone. ODBC was the logger setting it; its services are served
on loops now too. Only a logger whose thread could not be started is
written by its clients themselves, as before.
Plugins: struct logtype loses its last member, blocks.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A connection's two data buffers are malloc()'d with it (bufalloc())
instead of being part of struct clientparam, which is CLIENTBUFSIZE*2
smaller. The callbacks' relay of a client whose fiber gave up its stack
frees a side's buffer while it is empty and makes it again to read, and
gives both back before sockmap() or the child's tail goes on: a pool of
idle relayed connections holds no buffers. Fibers and threads keep
theirs as before.
mix scenarios, callbacks (-L -L2), peak memory before/after, throughput
the same:
Intel: relay 972/632 MB, connect 981/581, socks 992/594
ARM: relay 1000/673 MB, connect 1000/615, socks 1020/629
struct clientparam's clibufdata and srvbufdata are pointers now:
plugins built against the old structure are not compatible.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A child for which mapping is the last thing it does (tcppm, tlspr) sets
maptail, what it does once mapping is over as its own end would: its
log line, prepared before for tlspr. Relayed by the loop, the client's
fiber then gives up its stack at once: left in the middle of the
child's code, never to be resumed, it is started over from the start of
its stack for the next client (fiberreset), or freed where it cannot be
(Windows). Once mapping is over the loop ends the client as the child
and its fiber would: the result, the tail, the connection released and
freed. After a reload, which sockmap() is to see to, the client is
given a fiber again for sockmap() to go on with the mapping, then its
end. Not while a reload is pending: sockmap() sees to it on the fiber.
The callbacks' state is no longer on the fiber's stack but allocated.
clientparam has maptail and maptaildata: plugins are to be built again.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With -L, or loopmode on for the services which follow, the clients of a
TCP service are served on event loops: loops threads (by default one
for each CPU) each run a loop, and each client is given in turn to one
of them, which serves it on a fiber with the same code a thread runs.
Its waits in sockpoll() wait in the loop, its sockets and a timer
registered there, and the fiber goes on once one is ready or the time
is out; a sleep is a timer alone. A client costs a stack, as large as
its thread's would be, and no thread. The loops start with the first
client served on one and run as long as the process; each keeps up to
256 idle fibers for its next clients.
srvspawn() posts the client to its loop once it is counted, what is
posted made before so that posting cannot fail. A service whose clients
cannot share a thread (threadperclient), or listening for connections
back (-R), all of its clients waiting for one socket, which a loop
waits for once, keeps a thread for each client, as does every service
of a build or a processor without fibers, and the standalone services.
tests/run.py --loopmode runs every configuration with loopmode on, on
two loops.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
struct auth has blocks, for an authentication waiting other than in
sockpoll(): PamAuth, LdapPlugin and WindowsAuthentication set it, and a
service authenticating with one of them gives each client a thread.
Incompatible for plugins, which have to be built again with the same
version of 3proxy: struct auth has blocks.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A flag of the service for what cannot share a thread between clients,
once clients can be served otherwise: what waits other than in
sockpoll(), or a thread which may be unable to serve another client.
It is set for a service
- with FilePlugin, whose _prepoll writes files and sends them;
- with a -ne network namespace: a thread which failed to come back from
it to the main namespace is not to serve another client;
- writing to an ODBC log, should the log have no thread of its own to
write it: struct logtype has blocks, for a log waiting other than in
sockpoll().
Nothing serves clients otherwise yet: as before, each has a thread.
Incompatible for plugins, which have to be built again with the same
version of 3proxy: struct srvparam has threadperclient, struct logtype
has blocks.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A layer over the sockets, TLS or a plugin's, no longer waits itself.
Its _prepoll only tells what it knows without asking the system: the
fds it knows to be ready, data TLS has already decrypted, or, returning
0, that the sockets have to be polled, for events it may rewrite: a
TLS read waiting for its socket to be writable. The wait is
sockpoll()'s, which calls _prepoll first and then polls, so that a wait
can be made other than with poll() in one place: a client of an event
loop is to wait in the loop. Whatever a rewritten wait gets is given to
the caller as what it asked for, its chance to try again, and the
events it asked for are given back. With no fds sockpoll() sleeps,
which WSAPoll() would refuse.
TLS remembers what a read or a write it could not do waits for. A read
needing to write, or a write to read, was an error which ended the
connection, as when TLS 1.3 answers a key update while the socket
cannot take more; it now waits for what TLS needs, as one needing what
it was asked for always did.
Incompatible for plugins, which have to be built again with the same
version of 3proxy, the structures they share not kept compatible:
- struct sockfuncs has _prepoll(state, fds, nfds) in place of
_poll(state, fds, nfds, timeout). A plugin replacing _poll replaces
_prepoll: it must not wait, returns the number of fds it set revents
of (events then left as they are), 0 for the fds to be polled (with
events as it leaves them, revents left to the poll) or -1, and calls
the _prepoll it replaced when it has nothing to say itself.
- Waiting for sockets is sockpoll(sf, state, fds, nfds, timeout), not
so._poll() of the socket functions.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Once 3proxy runs (after the first configuration is in force, so after a
daemon's fork), each logger has a thread of its own writing from a
queue, and a client never waits for a log: ODBC, RADIUS, a full disk or
a pipe nobody reads hold up only that log's thread.
A queued line is fixed size: the part of struct clientparam logging
reads (everything before the filters: addresses, counters, names and
credentials, which hold no pointer it follows but srv), the time it was
logged, and its text, 256 bytes inline or a copy of its own if longer.
The thread formats it: struct tm is per call now, timechanged() and
dologname() included, and the type's format() takes the time.
minlogqueue (64) slots are allocated as the thread starts and kept,
maxlogqueue (4096) lines are queued at most. A line which finds the
queue full is dropped and counted on the last one queued; once that is
written, "Log X has its queue full, N records dropped" goes to where
errors go. A log which fails is no longer written to the error log
instead: its lines are dropped and counted while it is retried every
minute, and the count is told when it is open again, or when it was
reopened after failed writes.
A service which ended is stopped at once (srvstop(): sockets, listener,
its clients gone), but freed (srvrelease()) through its log's queue,
behind the lines which refer to it and to its configuration: the event
is part of struct srvparam, so a full queue never drops it. logfinish()
writes what is queued, 10 seconds at most, also as 3proxy exits on Unix.
Standalone services write as before, as each line comes.
LOGQUEUEMIN, LOGQUEUEMAX, DEFAULT_MAXCHILD, DEFAULT_POOLMAX,
DEFAULT_POOLTIMEOUT and POOLPERSERVICE (the automatic thread pool cap
for each service, which was 32 in the code) can be set with -D at build
time.
Tests: a log nobody reads (a FIFO) holds no client up, drops past its
queue and says so, and lines queued for a service a reload removed are
written as it formatted them; /dev/full says what it dropped.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On Windows proxy.h makes errno WSAGetLastError(), which cannot be
assigned, so the Windows build failed, and the error of fopen() would
not have been read there either. A type's init returns the errno value
(the C library's, read with _get_errno() on Windows), and logget() gives
it to its caller.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The string up to the first comma names an ODBC log; the user and the
password are kept apart as its parameters, never shown, and do not tell
logs apart. Another log or -l naming the same data source with other
parameters gives them to the logger, which reconnects with them. The
password is whatever follows the comma after the user.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A logger (struct logger) is identified by the string log or -l gives it:
"" stdout, "/dev/null" none, "@IDENT" syslog, "&DSN" ODBC, "radius",
anything else a file. Each type (struct logtype) has init, formatting of
a connection into a buffer, writing, close, and a periodic tick for
rotation and flushing; ODBC and RADIUS register their types from 3proxy,
which alone has them. Services and configurations hold the logger they
use; the last to let go closes it, except the standard ones (none,
stdout, stderr, syslog), which are opened at startup and kept.
-l takes every log type now, and a file named by several services or by
both log and -l is opened once. rotate and archiver are bound to the
logger of the last log command when its configuration comes into force,
and every file logger is rotated by the main loop.
Errors with no connection go to a new stderr logger; after daemon, or as
a Windows service, stdout logging writes nothing and errors go to the
last log configured. A log whose writes keep failing is closed and
reopened; while it cannot be, its lines go to stderr and it is retried
every minute. ODBC's own reconnection throttle gives way to this.
Formatting (logbuf) is separate from counting traffic and clearing the
statistics, done once by logdo(), the services' log function, so a later
step can format or copy a connection's line and write it from a thread
of its own. dologname() and timechanged() move to log.c, which the
standalone services link as well.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each service resolves names the way the last of defaultresolve (the name
servers in force, configured or the system's with the hosts file),
fakeresolve (127.0.0.2) and sysresolve (the C library) read before it
says; the last one in the configuration also applies outside services.
defaultresolve is the new command, and the default.
The name servers and the hosts file no longer depend on the mode: nserver
lines, or the system's, are always in force, so dnspr keeps forwarding;
with fakeresolve it does not, as before. fakeresolver moves to common.c,
which the standalone services link without resolve.c.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Without nserver (or fakeresolve) names were resolved by the C library.
3proxy's own resolver is now the default, with the system's name
servers: up to 3 from /etc/resolv.conf, on Windows from the registry
(set for the system, set for an interface with an address, Tcpip6
interfaces, then those DHCP gave). With none found, the C library as
before. nserver takes up to 3 as well (MAXNSERVERS), a fourth is
warned about.
The name servers in force belong to the process, not a configuration:
nserver lines are kept aside while a configuration is read and replace
them only once it is committed, so a reload resolves with those in
force until then. Resolutions and dnspr read a copy of them.
The system's are read at start, before the configuration and a chroot
in it, and are in force while it is read, so its host names resolve
with them; read again at a commit, and checked every 5 seconds from
the main loop while the default is in force. What cannot be read then
(within a chroot) stays as last read.
The hosts file, read with them, is imported into tables of its own
while the default is in force: its first address of a family for a
name. A name is looked up as one nsrecord made first, then the hosts
file, then the name cache, then name servers. A changed file replaces
what was imported. No name cache is made for it: without nscache
nothing is cached, as before.
The name cache's keys are host names of any case now: DNS names are.
Tests: sysdns, a name of the hosts file answered by dnspr without
nserver, in any case, and not with one.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A UDP service now runs in its own thread alone (udploop()): it reads the
service's socket and the sockets of its sessions towards their
destinations, keeps their deadlines, and answers every client from the
service's socket. A session is a struct clientparam as a client thread's
would be, with a slot of maxchild, among the service's children and
counted on its listener, but without a thread. The services supply
handlers (struct udpservice, childdef.udp): udppm keeps a session per
client address, forwarding its datagrams, behind the headers of a
parent chain, from a socket of its own and answering what comes back,
until STRING_L of silence or, with -s, the first answer; dnspr makes a
session of each query, answered at once from the fake address, the
name cache or the resolver, or sent on to the first name server over
UDP, whose answer is waited for in the loop up to DNS_TO (a name server
over TCP is still waited for in place). Authentication stays in the
loop for now; datagrams over a bandwidth limit, or of a client without
a free slot, are dropped.
On a reload the old service goes on serving the socket until it sees
its configuration replaced, then hands its sessions to the listener;
the newest service of its kind there, which until then does not read
the socket so that no client gets two sessions, adopts them: a slot of
its own, the old one given back, its filters and limits, and
re-authentication. A session is ended with 26 if the newest service is
of another kind, 27 without a slot, or with what the authentication
says; sessions nobody is left to adopt are ended by the last service to
stop on the listener.
On Windows a send on a socket waited while another thread polled it,
which delayed udppm's answers by up to a second, and old UDP services
did not exit after a reload; with one thread per socket neither
happens, and the second socket bound to the service's address
(NO_SHARE_UDP_SOCKET) is gone from the code and the build files.
udpinit, the global udp_table, udpnext(), udppmchild() and
dnsprchild() are gone; udpsockmap() serves SOCKS5 UDP ASSOCIATE only.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When a reload cannot take a listening Unix socket over (a bind-time
-ol option changed), the new service binds the path anew, and the old
listener, closing after it, removed the file, the new socket's by then,
leaving nothing to connect to. The socket's file (device and inode) is
now noted on binding and the path removed on closing only if it is
still that file.
structures.h includes <stddef.h> for offsetof() in SASIZE(), which
needs it wherever the header is used.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On Linux a proxy listening on a Unix socket relayed only the first
bytes of an answer (result 95): sockmap() read with recvfrom() into the
client's address, which for a Unix stream socket Linux fills with an
empty AF_UNIX one, and then sent with sendto() to that address, which a
connected Unix stream socket refuses. A TCP connection has no use for
addresses: sockmap() now sends and receives with NULL ones, and
connect-back sends its command with socksend() and reads the answer
with sockrecvfrom() without an address, which a NULL sin now allows.
An abstract socket name (-iunix:@NAME) was bound with the whole size of
struct sockaddr_un, padded with zeros, so a client naming it by its
bytes, as most do, reached another name. SASIZE() now gives an abstract
address its exact size; getsockname() calls that took SASIZE() of the
old contents as the buffer size take the buffer's size instead.
tcppm refused port 0 for a unix: target, though the manual gives
"tcppm 8080 unix:/var/run/app.sock 0" as the example: the port of a
Unix socket target is ignored, so 0 is allowed.
A test case covers a proxy on a Unix socket and on an abstract one on
Linux, tcppm to a Unix socket, a parent reached through one, and a
listening Unix socket across reloads: taken over each time, its file
kept, and removed once no service listens on it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Services of one configuration on the same address, port, network
namespace and device now share one listener and its socket, all
accepting on it, instead of each binding its own: on Linux the second
bind failed unless SO_REUSEPORT was set. A listener keeps a list of the
services accepting on it; the first to start opens the socket under the
listener's open_mutex, as binding may take long, and the others use it
("Accepting connections, shared"). The socket is closed once the last of
them stops.
Such services must be of one type with the same -ol options: otherwise
the configuration is refused ("a service of another type listens
there", "a service with other socket options listens there").
mainclaim() now runs for every service, at startup too: it joins the
listener of its configuration for the address, takes over one of an
older configuration (one of its type first), or makes a new one, under
one hold of listener_mutex so that a listener found is not freed before
it is joined.
Services connecting back (-r) to one place share a pseudo-listener
without a socket, of any type. After a reload a client moves to one of
the newest services of its type on its listener with a slot free: 26 if
there is none of its type, 27 if none has a slot.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
After a reload, unless noforce, a client of an old service goes by the
rules of the newest service on its listener, the one accepting there
now. It takes one of that service's maxconn slots, which counts it
there and keeps that service and its configuration alive until the
client is done, and is re-authenticated as its client: access lists,
users, authentication, bandwidth limits and traffic counters come from
the new configuration. A later reload moves it on, giving the slot
back. The client stays its own service's child, which keeps its
filters and TLS state.
It is dropped if its service was removed or is of another type now
(26), or if the newest service has no free slot (27); what the
re-authentication refuses ends it as before. A service without a
listener (UDP, connecting back) keeps its clients as they are.
This restores what force did before the configuration was kept per
service: since then a client compared its version with its own
service's configuration, which never changes, and was never checked
again. Services now take their configuration's version as it is: a
reload numbers the new configuration before reading it, and so does
startup now, where services were numbered one ahead.
A listener now stays with its old service until that service's last
client is gone, those clients looking the newest service up through
it; the socket itself is still closed as soon as nobody accepts on it.
Connection limits are given back in the configuration they were taken
in. PSRV() is the service whose rules apply to a client.
The reload case checks that a download of a user the new configuration
keeps finishes under it, and that one of a user it removes is cut.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A service's TCP listening socket is kept as a listener: when a reload
commits, before the old services are told to stop, each new service
takes over the listener of an older configuration's service on the same
address, port, network namespace and device with the same -ol options.
It skips socket() and bind(), calling only listen() again for its own
backlog, and logs "Accepting connections, taken over". The old service
stops accepting at its next poll and lets go of the socket; whoever
lets go last closes it, unlinking a Unix socket's path.
So a service staying where it was no longer leaves a gap: before, the
old one closed its socket and the new one opened another, and a
connection queued on the old socket was reset. UDP services, whose
clients read from the service's socket, and two services of the same
configuration on one address keep their own sockets as before.
A test case makes requests nonstop through three reloads and wants none
refused or reset; against the build before this, connections were
reset. The reload case no longer waits out the switch. SIGUSR1 in
3proxy(8) describes how a reload behaves now.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A reload reads the file into a new configuration, allocated from the
defaults, and prepares its services without starting them. Only if the
whole file is read without an error does it come into force: the old
configuration's services stop accepting, the new ones start. On an
error everything prepared is dropped and the old configuration, which
never stopped, stays in force.
Each service holds its configuration until it is freed, after its last
client; a configuration no longer in force is released when the last
holder lets go (confhold/confput). freeconf() now only releases what a
configuration holds, touching nothing global.
What runs without a service of its own reads the configuration in
force, runconf, rather than conf, which a reload points at the one
being read: the resolver, which holds it while it waits (a client
resolves with its own service's), the log, the thread pool, the admin
pages, which walk the services under config_mutex, and the signal
handlers.
The users table, havelog and the resolver choice are per configuration
now, the resolver applied when the configuration comes into force, so
a refused file leaves the users as they were. The log file, the
configuration file name, the string table, daemon mode and the filter
list belong to the process and are handed from one configuration to
the next. Settings left out of a new file go back to their defaults,
where before they kept the old values.
The gap between the old and the new listening sockets remains: a
connection queued on an old service's socket when it closes is reset.
Handing the socket over is the next phase.
A test case reloads a broken file (nothing changes, alice still gets
in, bob from the refused file does not) and a good one (bob only, the
service on its new port, the old one closed, a download through the
old service finishing whole). 40 reloads, a third refused, under load
show no memory errors under AddressSanitizer.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
-s[N]:MIN,STREAK,SMALL sets where the automatic mode splices, as -g
does for grace delay: a direction is spliced after STREAK reads in a
row of MIN bytes or more (SMALL from a TLS socket), copied again after
more than STREAK splices in a row of less than SMALL, and a body of
known length is spliced if MIN or more of it is left. The defaults,
16384,2,4096, are what was built in; a value left out or 0 keeps its
default.
The man page describes it, and its HTML copy, which still said splice()
was not built by default, now has the current text. A test case relays
answers of every size, chunked ones and a 3 MB upload through every
mode, two of them with heuristics set, and compares them byte for byte.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A service with ssl_ktls keeps its splice() mode. A connection whose TLS
sockets all got kernel TLS both ways may splice, as a plain one does when
no data filters apply; one where the kernel did not take TLS (nosplice)
copies as before.
TLS may hold data for a socket that a splice() from it would not see:
OpenSSL reads more from a kernel TLS socket than it is asked for. sockmap
asks the TLS code (splicetls) and reads that through TLS into the buffer
first, once the pipe has drained, so the order is kept, then splices on.
A read from a TLS socket counts as large at 4K, TLS handing out a record
at a time, and the short read collecting what TLS left from the last one
does not end a run of large ones. A splice() refused because a record is
not data (an alert, a key update, a ticket) falls back to reading it
through TLS instead of ending the connection.
The kernel TLS test compares the bodies with the origin's.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A service keeps the configuration it was started with (srvparam.config)
and its threads and its clients' read it there: limits and counters
(alwaysauth, the limiter), RADIUS, dnspr's name servers, and the checks
for a reload in the accept loop and in sockmap(). The configuration is
still reloaded in place, so it is the current one for now; this is what
lets a reload build a new one while the old one still serves.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
conf is now a pointer to the current configuration rather than the
structure itself: at first confboot, which exists before main() runs, as
plugins and the standalone services use it too; later configurations
will start from confdefaults. Both are made from confinit.h, the one list
of initial values. The timeouts are an array of the configuration
instead of a pointer to a global one. Nothing behaves differently yet.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
What a numbered counter counted now belongs to the process: a counter
rule points to the value of its number, created the first time the
number is met (from the counter file, if one is open) and shared by the
rules of that number in every configuration loaded. A reload no longer
resets it when there is no counter file, and once configurations can
outlive one another, clients of the old and the new one count into the
same value. An unnumbered counter, and those LdapPlugin makes, keep a
value of their own as before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
conf.haveerror was only initialised and srvparam.srvfds only declared;
the extern declarations of stdlog and demon named globals that do not
exist (both are fields of conf); a local in make_ace() and one in the
webadmin XML page were never used.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
They are the state of the process, not configuration: conf.time (now
curtime), timetoexit and needreload are globals of their own. Plugins
reach needreload through a pointer added at the end of struct
pluginlink; LdapPlugin uses it. The extern for a paused global, never
defined, goes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nservers, numservers and authnserver (resolve.c, common.c) and
radiuslist, nradservers and radiussecret (authradius.c) were globals of
their own; they are configuration, set by nserver, authnserver and
radius, and now live in struct extparam with the rest of it.
MAXNSERVERS, MAXRADIUS and struct radserver move to structures.h, which
plugins include without proxy.h.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
dialer ran a program when a name could not be resolved, for dial-up
links; with nscache it did not even run. The command is still accepted
and ignored, so configurations using it keep loading.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Like the timeouts, each service now keeps the values given before it
instead of reading the global ones while its connections run. setopts()
takes the service's maxseg. The TLS handshakes wait the service's
CONNECT_TO, kept in its SSL configuration, rather than the global one.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
-s takes the directions to splice: -s1 what the server sends (downloads),
-s2 what the client sends (uploads), -s3 both. -s alone (-s7) splices both
only where it pays, as -s5 and -s6 do for one direction:
- a body of known length is spliced if at least 16K of it is left to read
once what is already buffered is counted, so an answer read whole with
its headers is just copied;
- a stream of unknown length starts copied, is spliced after 2 reads in a
row of at least 16K, and goes back to copying after 3 splices in a row
of less than 4K, once its pipe is empty.
Pipes are made when a direction first splices and kept with the
connection for its next mappings, so a kept-alive connection no longer
makes and closes two pipes per request; one left holding data by an
error or a timeout is closed. The direction not spliced keeps its buffer.
Before, -s spliced both directions and every mapping made both pipes,
which made small answers slower than without splice: with kept-alive
50K answers 2 pipe2() and 4 close() per request, now none.
Also corrects the default buffer size in the -b help (65536 for TCP).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>