struct auth has blocks, for an authentication waiting other than in
sockpoll(): PamAuth, LdapPlugin and WindowsAuthentication set it, and a
service authenticating with one of them gives each client a thread.
Incompatible for plugins, which have to be built again with the same
version of 3proxy: struct auth has blocks.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A flag of the service for what cannot share a thread between clients,
once clients can be served otherwise: what waits other than in
sockpoll(), or a thread which may be unable to serve another client.
It is set for a service
- with FilePlugin, whose _prepoll writes files and sends them;
- with a -ne network namespace: a thread which failed to come back from
it to the main namespace is not to serve another client;
- writing to an ODBC log, should the log have no thread of its own to
write it: struct logtype has blocks, for a log waiting other than in
sockpoll().
Nothing serves clients otherwise yet: as before, each has a thread.
Incompatible for plugins, which have to be built again with the same
version of 3proxy: struct srvparam has threadperclient, struct logtype
has blocks.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A layer over the sockets, TLS or a plugin's, no longer waits itself.
Its _prepoll only tells what it knows without asking the system: the
fds it knows to be ready, data TLS has already decrypted, or, returning
0, that the sockets have to be polled, for events it may rewrite: a
TLS read waiting for its socket to be writable. The wait is
sockpoll()'s, which calls _prepoll first and then polls, so that a wait
can be made other than with poll() in one place: a client of an event
loop is to wait in the loop. Whatever a rewritten wait gets is given to
the caller as what it asked for, its chance to try again, and the
events it asked for are given back. With no fds sockpoll() sleeps,
which WSAPoll() would refuse.
TLS remembers what a read or a write it could not do waits for. A read
needing to write, or a write to read, was an error which ended the
connection, as when TLS 1.3 answers a key update while the socket
cannot take more; it now waits for what TLS needs, as one needing what
it was asked for always did.
Incompatible for plugins, which have to be built again with the same
version of 3proxy, the structures they share not kept compatible:
- struct sockfuncs has _prepoll(state, fds, nfds) in place of
_poll(state, fds, nfds, timeout). A plugin replacing _poll replaces
_prepoll: it must not wait, returns the number of fds it set revents
of (events then left as they are), 0 for the fds to be polled (with
events as it leaves them, revents left to the poll) or -1, and calls
the _prepoll it replaced when it has nothing to say itself.
- Waiting for sockets is sockpoll(sf, state, fds, nfds, timeout), not
so._poll() of the socket functions.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Once 3proxy runs (after the first configuration is in force, so after a
daemon's fork), each logger has a thread of its own writing from a
queue, and a client never waits for a log: ODBC, RADIUS, a full disk or
a pipe nobody reads hold up only that log's thread.
A queued line is fixed size: the part of struct clientparam logging
reads (everything before the filters: addresses, counters, names and
credentials, which hold no pointer it follows but srv), the time it was
logged, and its text, 256 bytes inline or a copy of its own if longer.
The thread formats it: struct tm is per call now, timechanged() and
dologname() included, and the type's format() takes the time.
minlogqueue (64) slots are allocated as the thread starts and kept,
maxlogqueue (4096) lines are queued at most. A line which finds the
queue full is dropped and counted on the last one queued; once that is
written, "Log X has its queue full, N records dropped" goes to where
errors go. A log which fails is no longer written to the error log
instead: its lines are dropped and counted while it is retried every
minute, and the count is told when it is open again, or when it was
reopened after failed writes.
A service which ended is stopped at once (srvstop(): sockets, listener,
its clients gone), but freed (srvrelease()) through its log's queue,
behind the lines which refer to it and to its configuration: the event
is part of struct srvparam, so a full queue never drops it. logfinish()
writes what is queued, 10 seconds at most, also as 3proxy exits on Unix.
Standalone services write as before, as each line comes.
LOGQUEUEMIN, LOGQUEUEMAX, DEFAULT_MAXCHILD, DEFAULT_POOLMAX,
DEFAULT_POOLTIMEOUT and POOLPERSERVICE (the automatic thread pool cap
for each service, which was 32 in the code) can be set with -D at build
time.
Tests: a log nobody reads (a FIFO) holds no client up, drops past its
queue and says so, and lines queued for a service a reload removed are
written as it formatted them; /dev/full says what it dropped.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On Windows proxy.h makes errno WSAGetLastError(), which cannot be
assigned, so the Windows build failed, and the error of fopen() would
not have been read there either. A type's init returns the errno value
(the C library's, read with _get_errno() on Windows), and logget() gives
it to its caller.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The string up to the first comma names an ODBC log; the user and the
password are kept apart as its parameters, never shown, and do not tell
logs apart. Another log or -l naming the same data source with other
parameters gives them to the logger, which reconnects with them. The
password is whatever follows the comma after the user.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A logger (struct logger) is identified by the string log or -l gives it:
"" stdout, "/dev/null" none, "@IDENT" syslog, "&DSN" ODBC, "radius",
anything else a file. Each type (struct logtype) has init, formatting of
a connection into a buffer, writing, close, and a periodic tick for
rotation and flushing; ODBC and RADIUS register their types from 3proxy,
which alone has them. Services and configurations hold the logger they
use; the last to let go closes it, except the standard ones (none,
stdout, stderr, syslog), which are opened at startup and kept.
-l takes every log type now, and a file named by several services or by
both log and -l is opened once. rotate and archiver are bound to the
logger of the last log command when its configuration comes into force,
and every file logger is rotated by the main loop.
Errors with no connection go to a new stderr logger; after daemon, or as
a Windows service, stdout logging writes nothing and errors go to the
last log configured. A log whose writes keep failing is closed and
reopened; while it cannot be, its lines go to stderr and it is retried
every minute. ODBC's own reconnection throttle gives way to this.
Formatting (logbuf) is separate from counting traffic and clearing the
statistics, done once by logdo(), the services' log function, so a later
step can format or copy a connection's line and write it from a thread
of its own. dologname() and timechanged() move to log.c, which the
standalone services link as well.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each service resolves names the way the last of defaultresolve (the name
servers in force, configured or the system's with the hosts file),
fakeresolve (127.0.0.2) and sysresolve (the C library) read before it
says; the last one in the configuration also applies outside services.
defaultresolve is the new command, and the default.
The name servers and the hosts file no longer depend on the mode: nserver
lines, or the system's, are always in force, so dnspr keeps forwarding;
with fakeresolve it does not, as before. fakeresolver moves to common.c,
which the standalone services link without resolve.c.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Without nserver (or fakeresolve) names were resolved by the C library.
3proxy's own resolver is now the default, with the system's name
servers: up to 3 from /etc/resolv.conf, on Windows from the registry
(set for the system, set for an interface with an address, Tcpip6
interfaces, then those DHCP gave). With none found, the C library as
before. nserver takes up to 3 as well (MAXNSERVERS), a fourth is
warned about.
The name servers in force belong to the process, not a configuration:
nserver lines are kept aside while a configuration is read and replace
them only once it is committed, so a reload resolves with those in
force until then. Resolutions and dnspr read a copy of them.
The system's are read at start, before the configuration and a chroot
in it, and are in force while it is read, so its host names resolve
with them; read again at a commit, and checked every 5 seconds from
the main loop while the default is in force. What cannot be read then
(within a chroot) stays as last read.
The hosts file, read with them, is imported into tables of its own
while the default is in force: its first address of a family for a
name. A name is looked up as one nsrecord made first, then the hosts
file, then the name cache, then name servers. A changed file replaces
what was imported. No name cache is made for it: without nscache
nothing is cached, as before.
The name cache's keys are host names of any case now: DNS names are.
Tests: sysdns, a name of the hosts file answered by dnspr without
nserver, in any case, and not with one.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A UDP service now runs in its own thread alone (udploop()): it reads the
service's socket and the sockets of its sessions towards their
destinations, keeps their deadlines, and answers every client from the
service's socket. A session is a struct clientparam as a client thread's
would be, with a slot of maxchild, among the service's children and
counted on its listener, but without a thread. The services supply
handlers (struct udpservice, childdef.udp): udppm keeps a session per
client address, forwarding its datagrams, behind the headers of a
parent chain, from a socket of its own and answering what comes back,
until STRING_L of silence or, with -s, the first answer; dnspr makes a
session of each query, answered at once from the fake address, the
name cache or the resolver, or sent on to the first name server over
UDP, whose answer is waited for in the loop up to DNS_TO (a name server
over TCP is still waited for in place). Authentication stays in the
loop for now; datagrams over a bandwidth limit, or of a client without
a free slot, are dropped.
On a reload the old service goes on serving the socket until it sees
its configuration replaced, then hands its sessions to the listener;
the newest service of its kind there, which until then does not read
the socket so that no client gets two sessions, adopts them: a slot of
its own, the old one given back, its filters and limits, and
re-authentication. A session is ended with 26 if the newest service is
of another kind, 27 without a slot, or with what the authentication
says; sessions nobody is left to adopt are ended by the last service to
stop on the listener.
On Windows a send on a socket waited while another thread polled it,
which delayed udppm's answers by up to a second, and old UDP services
did not exit after a reload; with one thread per socket neither
happens, and the second socket bound to the service's address
(NO_SHARE_UDP_SOCKET) is gone from the code and the build files.
udpinit, the global udp_table, udpnext(), udppmchild() and
dnsprchild() are gone; udpsockmap() serves SOCKS5 UDP ASSOCIATE only.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When a reload cannot take a listening Unix socket over (a bind-time
-ol option changed), the new service binds the path anew, and the old
listener, closing after it, removed the file, the new socket's by then,
leaving nothing to connect to. The socket's file (device and inode) is
now noted on binding and the path removed on closing only if it is
still that file.
structures.h includes <stddef.h> for offsetof() in SASIZE(), which
needs it wherever the header is used.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On Linux a proxy listening on a Unix socket relayed only the first
bytes of an answer (result 95): sockmap() read with recvfrom() into the
client's address, which for a Unix stream socket Linux fills with an
empty AF_UNIX one, and then sent with sendto() to that address, which a
connected Unix stream socket refuses. A TCP connection has no use for
addresses: sockmap() now sends and receives with NULL ones, and
connect-back sends its command with socksend() and reads the answer
with sockrecvfrom() without an address, which a NULL sin now allows.
An abstract socket name (-iunix:@NAME) was bound with the whole size of
struct sockaddr_un, padded with zeros, so a client naming it by its
bytes, as most do, reached another name. SASIZE() now gives an abstract
address its exact size; getsockname() calls that took SASIZE() of the
old contents as the buffer size take the buffer's size instead.
tcppm refused port 0 for a unix: target, though the manual gives
"tcppm 8080 unix:/var/run/app.sock 0" as the example: the port of a
Unix socket target is ignored, so 0 is allowed.
A test case covers a proxy on a Unix socket and on an abstract one on
Linux, tcppm to a Unix socket, a parent reached through one, and a
listening Unix socket across reloads: taken over each time, its file
kept, and removed once no service listens on it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Services of one configuration on the same address, port, network
namespace and device now share one listener and its socket, all
accepting on it, instead of each binding its own: on Linux the second
bind failed unless SO_REUSEPORT was set. A listener keeps a list of the
services accepting on it; the first to start opens the socket under the
listener's open_mutex, as binding may take long, and the others use it
("Accepting connections, shared"). The socket is closed once the last of
them stops.
Such services must be of one type with the same -ol options: otherwise
the configuration is refused ("a service of another type listens
there", "a service with other socket options listens there").
mainclaim() now runs for every service, at startup too: it joins the
listener of its configuration for the address, takes over one of an
older configuration (one of its type first), or makes a new one, under
one hold of listener_mutex so that a listener found is not freed before
it is joined.
Services connecting back (-r) to one place share a pseudo-listener
without a socket, of any type. After a reload a client moves to one of
the newest services of its type on its listener with a slot free: 26 if
there is none of its type, 27 if none has a slot.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
After a reload, unless noforce, a client of an old service goes by the
rules of the newest service on its listener, the one accepting there
now. It takes one of that service's maxconn slots, which counts it
there and keeps that service and its configuration alive until the
client is done, and is re-authenticated as its client: access lists,
users, authentication, bandwidth limits and traffic counters come from
the new configuration. A later reload moves it on, giving the slot
back. The client stays its own service's child, which keeps its
filters and TLS state.
It is dropped if its service was removed or is of another type now
(26), or if the newest service has no free slot (27); what the
re-authentication refuses ends it as before. A service without a
listener (UDP, connecting back) keeps its clients as they are.
This restores what force did before the configuration was kept per
service: since then a client compared its version with its own
service's configuration, which never changes, and was never checked
again. Services now take their configuration's version as it is: a
reload numbers the new configuration before reading it, and so does
startup now, where services were numbered one ahead.
A listener now stays with its old service until that service's last
client is gone, those clients looking the newest service up through
it; the socket itself is still closed as soon as nobody accepts on it.
Connection limits are given back in the configuration they were taken
in. PSRV() is the service whose rules apply to a client.
The reload case checks that a download of a user the new configuration
keeps finishes under it, and that one of a user it removes is cut.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A service's TCP listening socket is kept as a listener: when a reload
commits, before the old services are told to stop, each new service
takes over the listener of an older configuration's service on the same
address, port, network namespace and device with the same -ol options.
It skips socket() and bind(), calling only listen() again for its own
backlog, and logs "Accepting connections, taken over". The old service
stops accepting at its next poll and lets go of the socket; whoever
lets go last closes it, unlinking a Unix socket's path.
So a service staying where it was no longer leaves a gap: before, the
old one closed its socket and the new one opened another, and a
connection queued on the old socket was reset. UDP services, whose
clients read from the service's socket, and two services of the same
configuration on one address keep their own sockets as before.
A test case makes requests nonstop through three reloads and wants none
refused or reset; against the build before this, connections were
reset. The reload case no longer waits out the switch. SIGUSR1 in
3proxy(8) describes how a reload behaves now.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A reload reads the file into a new configuration, allocated from the
defaults, and prepares its services without starting them. Only if the
whole file is read without an error does it come into force: the old
configuration's services stop accepting, the new ones start. On an
error everything prepared is dropped and the old configuration, which
never stopped, stays in force.
Each service holds its configuration until it is freed, after its last
client; a configuration no longer in force is released when the last
holder lets go (confhold/confput). freeconf() now only releases what a
configuration holds, touching nothing global.
What runs without a service of its own reads the configuration in
force, runconf, rather than conf, which a reload points at the one
being read: the resolver, which holds it while it waits (a client
resolves with its own service's), the log, the thread pool, the admin
pages, which walk the services under config_mutex, and the signal
handlers.
The users table, havelog and the resolver choice are per configuration
now, the resolver applied when the configuration comes into force, so
a refused file leaves the users as they were. The log file, the
configuration file name, the string table, daemon mode and the filter
list belong to the process and are handed from one configuration to
the next. Settings left out of a new file go back to their defaults,
where before they kept the old values.
The gap between the old and the new listening sockets remains: a
connection queued on an old service's socket when it closes is reset.
Handing the socket over is the next phase.
A test case reloads a broken file (nothing changes, alice still gets
in, bob from the refused file does not) and a good one (bob only, the
service on its new port, the old one closed, a download through the
old service finishing whole). 40 reloads, a third refused, under load
show no memory errors under AddressSanitizer.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
-s[N]:MIN,STREAK,SMALL sets where the automatic mode splices, as -g
does for grace delay: a direction is spliced after STREAK reads in a
row of MIN bytes or more (SMALL from a TLS socket), copied again after
more than STREAK splices in a row of less than SMALL, and a body of
known length is spliced if MIN or more of it is left. The defaults,
16384,2,4096, are what was built in; a value left out or 0 keeps its
default.
The man page describes it, and its HTML copy, which still said splice()
was not built by default, now has the current text. A test case relays
answers of every size, chunked ones and a 3 MB upload through every
mode, two of them with heuristics set, and compares them byte for byte.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A service with ssl_ktls keeps its splice() mode. A connection whose TLS
sockets all got kernel TLS both ways may splice, as a plain one does when
no data filters apply; one where the kernel did not take TLS (nosplice)
copies as before.
TLS may hold data for a socket that a splice() from it would not see:
OpenSSL reads more from a kernel TLS socket than it is asked for. sockmap
asks the TLS code (splicetls) and reads that through TLS into the buffer
first, once the pipe has drained, so the order is kept, then splices on.
A read from a TLS socket counts as large at 4K, TLS handing out a record
at a time, and the short read collecting what TLS left from the last one
does not end a run of large ones. A splice() refused because a record is
not data (an alert, a key update, a ticket) falls back to reading it
through TLS instead of ending the connection.
The kernel TLS test compares the bodies with the origin's.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A service keeps the configuration it was started with (srvparam.config)
and its threads and its clients' read it there: limits and counters
(alwaysauth, the limiter), RADIUS, dnspr's name servers, and the checks
for a reload in the accept loop and in sockmap(). The configuration is
still reloaded in place, so it is the current one for now; this is what
lets a reload build a new one while the old one still serves.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
conf is now a pointer to the current configuration rather than the
structure itself: at first confboot, which exists before main() runs, as
plugins and the standalone services use it too; later configurations
will start from confdefaults. Both are made from confinit.h, the one list
of initial values. The timeouts are an array of the configuration
instead of a pointer to a global one. Nothing behaves differently yet.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
What a numbered counter counted now belongs to the process: a counter
rule points to the value of its number, created the first time the
number is met (from the counter file, if one is open) and shared by the
rules of that number in every configuration loaded. A reload no longer
resets it when there is no counter file, and once configurations can
outlive one another, clients of the old and the new one count into the
same value. An unnumbered counter, and those LdapPlugin makes, keep a
value of their own as before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
conf.haveerror was only initialised and srvparam.srvfds only declared;
the extern declarations of stdlog and demon named globals that do not
exist (both are fields of conf); a local in make_ace() and one in the
webadmin XML page were never used.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
They are the state of the process, not configuration: conf.time (now
curtime), timetoexit and needreload are globals of their own. Plugins
reach needreload through a pointer added at the end of struct
pluginlink; LdapPlugin uses it. The extern for a paused global, never
defined, goes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nservers, numservers and authnserver (resolve.c, common.c) and
radiuslist, nradservers and radiussecret (authradius.c) were globals of
their own; they are configuration, set by nserver, authnserver and
radius, and now live in struct extparam with the rest of it.
MAXNSERVERS, MAXRADIUS and struct radserver move to structures.h, which
plugins include without proxy.h.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
dialer ran a program when a name could not be resolved, for dial-up
links; with nscache it did not even run. The command is still accepted
and ignored, so configurations using it keep loading.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Like the timeouts, each service now keeps the values given before it
instead of reading the global ones while its connections run. setopts()
takes the service's maxseg. The TLS handshakes wait the service's
CONNECT_TO, kept in its SSL configuration, rather than the global one.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
-s takes the directions to splice: -s1 what the server sends (downloads),
-s2 what the client sends (uploads), -s3 both. -s alone (-s7) splices both
only where it pays, as -s5 and -s6 do for one direction:
- a body of known length is spliced if at least 16K of it is left to read
once what is already buffered is counted, so an answer read whole with
its headers is just copied;
- a stream of unknown length starts copied, is spliced after 2 reads in a
row of at least 16K, and goes back to copying after 3 splices in a row
of less than 4K, once its pipe is empty.
Pipes are made when a direction first splices and kept with the
connection for its next mappings, so a kept-alive connection no longer
makes and closes two pipes per request; one left holding data by an
error or a timeout is closed. The direction not spliced keeps its buffer.
Before, -s spliced both directions and every mapping made both pipes,
which made small answers slower than without splice: with kept-alive
50K answers 2 pipe2() and 4 close() per request, now none.
Also corrects the default buffer size in the -b help (65536 for TCP).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
pop3p, imapp and smtpp are built with MAILPROXY=true, and ftppr and the
ftp:// scheme of the HTTP proxy with FTP=true; with CMake the switches are
3PROXY_USE_MAILPROXY and 3PROXY_USE_FTP. Neither is in a default build, and
the standalone binaries follow what was built.
A configuration naming one of them is still read either way. The three mail
protocols amount to a STARTTLS negotiation now that mail is carried over
TLS, so without a proxy of their own those names are tlspr speaking the
protocol: the file holds a stand-in which sets the protocol for the
connection and returns tlspr for the caller to run, which serves the
service name and a parent chain alike and leaves conf.c and the redirect
table untouched. FTP has no such fallback, so the service is known, answers
nothing and logs the refusal.
The Linux workflow builds both, so all of it is still compiled and run.
