The proxy accepts the connection expected and joins it with the
client both ways, and refuses one from another address than the one
asked for. socks_bind() and socks_bind_peer() in the harness make the
request and read the second reply.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Once 3proxy runs (after the first configuration is in force, so after a
daemon's fork), each logger has a thread of its own writing from a
queue, and a client never waits for a log: ODBC, RADIUS, a full disk or
a pipe nobody reads hold up only that log's thread.
A queued line is fixed size: the part of struct clientparam logging
reads (everything before the filters: addresses, counters, names and
credentials, which hold no pointer it follows but srv), the time it was
logged, and its text, 256 bytes inline or a copy of its own if longer.
The thread formats it: struct tm is per call now, timechanged() and
dologname() included, and the type's format() takes the time.
minlogqueue (64) slots are allocated as the thread starts and kept,
maxlogqueue (4096) lines are queued at most. A line which finds the
queue full is dropped and counted on the last one queued; once that is
written, "Log X has its queue full, N records dropped" goes to where
errors go. A log which fails is no longer written to the error log
instead: its lines are dropped and counted while it is retried every
minute, and the count is told when it is open again, or when it was
reopened after failed writes.
A service which ended is stopped at once (srvstop(): sockets, listener,
its clients gone), but freed (srvrelease()) through its log's queue,
behind the lines which refer to it and to its configuration: the event
is part of struct srvparam, so a full queue never drops it. logfinish()
writes what is queued, 10 seconds at most, also as 3proxy exits on Unix.
Standalone services write as before, as each line comes.
LOGQUEUEMIN, LOGQUEUEMAX, DEFAULT_MAXCHILD, DEFAULT_POOLMAX,
DEFAULT_POOLTIMEOUT and POOLPERSERVICE (the automatic thread pool cap
for each service, which was 32 in the code) can be set with -D at build
time.
Tests: a log nobody reads (a FIFO) holds no client up, drops past its
queue and says so, and lines queued for a service a reload removed are
written as it formatted them; /dev/full says what it dropped.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The last datagram's time was taken after the sleep following it, half a
second and a slow runner's overshoot late, so a session ending on time
looked early. The test resolver no longer raises when its socket is
closed under it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A logger (struct logger) is identified by the string log or -l gives it:
"" stdout, "/dev/null" none, "@IDENT" syslog, "&DSN" ODBC, "radius",
anything else a file. Each type (struct logtype) has init, formatting of
a connection into a buffer, writing, close, and a periodic tick for
rotation and flushing; ODBC and RADIUS register their types from 3proxy,
which alone has them. Services and configurations hold the logger they
use; the last to let go closes it, except the standard ones (none,
stdout, stderr, syslog), which are opened at startup and kept.
-l takes every log type now, and a file named by several services or by
both log and -l is opened once. rotate and archiver are bound to the
logger of the last log command when its configuration comes into force,
and every file logger is rotated by the main loop.
Errors with no connection go to a new stderr logger; after daemon, or as
a Windows service, stdout logging writes nothing and errors go to the
last log configured. A log whose writes keep failing is closed and
reopened; while it cannot be, its lines go to stderr and it is retried
every minute. ODBC's own reconnection throttle gives way to this.
Formatting (logbuf) is separate from counting traffic and clearing the
statistics, done once by logdo(), the services' log function, so a later
step can format or copy a connection's line and write it from a thread
of its own. dologname() and timechanged() move to log.c, which the
standalone services link as well.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The loop went through every session three times a turn: for the nearest
deadline, to register its sockets with epoll, and for the deadlines come
and the sessions ended. A datagram cost as much as the sessions there
were, idle ones too.
- The deadlines are in a binary heap, the nearest at its top: the wait
is until the top, and the sessions timed out are taken from the top
down to the first not due. A datagram only moves a session's deadline
later, which leaves the heap as it is: the session is looked at again
when its old deadline comes, and put back where its deadline is then,
once in its timeout however many datagrams it had. A deadline set
earlier by a handler goes up the heap.
- A session ended is put on a list and let go at the end of the turn,
and its sockets are registered with epoll after the loop or a handler
had it, not every session every turn.
- Deadlines are milliseconds of monotime(), CLOCK_MONOTONIC (the
performance counter on Windows), which the time of day being set does
not move: before, a clock set back kept sessions for as long, and one
set forward ended them all. The loop reads it once a turn, in u->now.
With epoll, a datagram costs the same with 20000 idle sessions as with
none (38-40 us of the proxy's CPU a round trip, ARM), where it took 59
before. poll() elsewhere still builds its set every turn.
Tests: a session heard from every half second lives through two seconds
of idle timeout, and ends two seconds after the last datagram.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each service resolves names the way the last of defaultresolve (the name
servers in force, configured or the system's with the hosts file),
fakeresolve (127.0.0.2) and sysresolve (the C library) read before it
says; the last one in the configuration also applies outside services.
defaultresolve is the new command, and the default.
The name servers and the hosts file no longer depend on the mode: nserver
lines, or the system's, are always in force, so dnspr keeps forwarding;
with fakeresolve it does not, as before. fakeresolver moves to common.c,
which the standalone services link without resolve.c.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The command keeps the C library's resolution which a configuration
without nserver had before the system's name servers became the
default: no hosts file imported. nserver lines, if any, are still
there for dnspr to forward to. Of sysresolve and fakeresolve the last
given applies.
Tests: sysdns, with sysresolve dnspr has no hosts file names and a
name is resolved through the system routines.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Without nserver (or fakeresolve) names were resolved by the C library.
3proxy's own resolver is now the default, with the system's name
servers: up to 3 from /etc/resolv.conf, on Windows from the registry
(set for the system, set for an interface with an address, Tcpip6
interfaces, then those DHCP gave). With none found, the C library as
before. nserver takes up to 3 as well (MAXNSERVERS), a fourth is
warned about.
The name servers in force belong to the process, not a configuration:
nserver lines are kept aside while a configuration is read and replace
them only once it is committed, so a reload resolves with those in
force until then. Resolutions and dnspr read a copy of them.
The system's are read at start, before the configuration and a chroot
in it, and are in force while it is read, so its host names resolve
with them; read again at a commit, and checked every 5 seconds from
the main loop while the default is in force. What cannot be read then
(within a chroot) stays as last read.
The hosts file, read with them, is imported into tables of its own
while the default is in force: its first address of a family for a
name. A name is looked up as one nsrecord made first, then the hosts
file, then the name cache, then name servers. A changed file replaces
what was imported. No name cache is made for it: without nscache
nothing is cached, as before.
The name cache's keys are host names of any case now: DNS names are.
Tests: sysdns, a name of the hosts file answered by dnspr without
nserver, in any case, and not with one.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A -r service waited up to 60 s for the go-ahead on the connection it
made, without looking at a reload. One replaced by a reload went on
waiting, holding a slot and a thread, and took a client if the far
side's go-ahead came in that time; every reload left another such
connection in the far side's queue. The wait is now a second at a time
(60 s in all, as before): at a reload the service takes a go-ahead
already there, its client going on in the service which replaced it,
and otherwise closes the connection and stops.
The far side (-R) sent its go-ahead to any connection it accepted, and
a closed one cost it a client. A connection the -r side closed, at a
reload or after its wait, is readable before any go-ahead, since that
side sends nothing first: once past the ACL it is passed over, without
counting as one of the tries for as many as the listen queue holds
(1 + maxchild/16), so that connections made to be closed or to send
something cannot keep a client's thread in the loop.
Tests: reattach's far side passes over closed connections as -R does.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
REQMAX is 4 KB: a log line holds 4000 characters, so a longer copy of
the request line was cut by the log before "..." could show.
The headers this proxy adds are written straight after the header
block, HDRRESERVE (2 KB, what they come to at most) asked to be free
there first: in the side's buffer that is the room hdrread() left, so
nothing is copied, and the buffer on the stack they were built in is
gone.
The scheme and authority of a request are no longer kept in a buffer
of their own: req, the line in absolute form as logged, has them, and
where the authority begins and ends in it is kept instead, for the
next request on the connection to be compared with and for a parent
proxy to be given. An authority beyond what req keeps of the line is
refused. A line a request filter rewrote is logged only when the
rewrite is applied.
Tests: a 5 KB line through a parent arrives whole in absolute form.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A request to a parent HTTP proxy went as two writes, the line as the
client sent it from req and then the rest of the headers from buf,
held together with TCP_CORK, which does not help over TLS nor with
headers bigger than a segment. The line now goes back to absolute form
where it is: a request's header block is read HDRFRONT (300) bytes
into clibuf, room for http://, a host name of 256 characters and a
port; the method moves down into it and the scheme and authority go
in after it, and line and headers are sent in one piece. Where they
do not fit, or the headers are in a buffer of their own by then, both
are put together in one. TCP_CORK is gone.
The scheme and authority of a request are kept for that (auth), from
its line or, for a request with a path only, its Host header; the last
one is what the next request on the connection is compared with. A
CONNECT line is kept whole in buf. The request filters are given the
line in absolute form in a buffer of their own, made only when there
are some, as req was. req is the copy logged and nothing else: at most
REQMAX (6 KB) of the line, "..." after what is cut, never a reason to
refuse the request.
The headers this proxy adds are built in a buffer on the stack, of
bounded parts, and put after the header block once there is room for
them, in a standalone proxy too, which never asked for it. HDRRESERVE
is 1 KB.
Tests: through a parent a POST arrives with its line in absolute form,
its headers and body whole; user and password in the URL; headers in a
buffer of their own (a data filter) put together with the line; a 7 KB
request line served whole and logged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The HTTP proxy read each request into a buffer of 64 KB of its own,
cleared it with memset, and read the answer's headers into it again;
the header lines were copied there one by one from clibuf or srvbuf.
Now the header block of a request is read whole into clibuf, and that
of an answer into srvbuf, and worked on where it is (hdrread()):
- the block is moved to one byte into the buffer, and its lines, as
they are kept, are moved down to its start, so the line being worked
on, and the 0 written after it, never reach the next one; headers
taken out are left behind;
- HDRRESERVE is left free after it, for the headers this proxy adds,
before whatever came after it, a body or another request, from
where the connection is read on;
- the buffer is grown only for a block bigger than it, up to the
limits there were (512 KB for a request, 64 KB for an answer);
- the headers are copied to a buffer of their own only when they need
more room than that, when a filter hands back one, when a body is to
be read whole into the same buffer for a data filter, and for an ftp
listing, which is written where the headers were.
Whether buf is the headers' own buffer is kept with it (bufown), not
found from where it points: the side's buffer may be a bigger one by
then let go of. filterswap() leaves alone a buffer in either side's
buffer, which may be one grown by malloc, not only their fixed arrays.
What took buf for a buffer of LINESIZE does no longer: the request is
logged from a copy of its line, and the rest of a request refused
before its headers were all read is drained through a small buffer of
its own, keeping track of where its lines begin.
Tests: a POST whose body comes in the same packet as its headers,
which are rewritten, with a header added and one taken out; 45 KB of
request headers; two requests pipelined in one packet; empty lines
before a request; an answer with a 30 KB header; a 600 KB header block
refused, the proxy answering after it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Answers from the name servers now go into the shared name cache, so
they are checked before they are taken:
- a query goes to a name server under a random id of its own, the
client's put back only in what is relayed to it;
- an answer over UDP is taken only from the address and port of the
name server asked;
- the answer must have that id, be a response, and carry the question
asked, byte for byte, over UDP and TCP alike;
- of its records only one owned by the question's name itself (a
pointer to it) is kept in the cache;
- a name with a label of other characters than a host's, a '.' or a
NUL among them, reads as another name once made text, so its answer
is relayed but not kept.
Tests: an answer with another id is neither relayed nor kept, and a
name with a '.' inside a label is answered but not cached under the
name its text reads as.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
dnspr answered an A or AAAA query missing from the name cache with
udpresolve(), which waited for the name servers in place, and a name
server over TCP was waited for in place too, holding up every query of
the service. Now only the name cache is looked at in place; anything it
does not have goes to the name servers from the loop: over UDP the query
is sent on a socket the loop waits on, over TCP the socket is connected
without waiting, the query written once the loop sees it writable, and
the answer read in as many pieces as it comes. A name server silent for
DNS_TO is followed by the next one, as udpresolve() did; the last ends
the query with 821. An address in an answer is kept in the name cache,
unless -s says not to use it.
The UDP loop lets a session say what it waits for on its socket
(events) and a service handle a session's deadline itself (timeout).
Tests: an A query not in the cache answered by the name server and then
from the cache with the server gone, an A query waiting for a silent
name server while another is answered, a name server over TCP whose
answer comes in pieces, and the second name server asked when the
first is silent.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
udppm: 20 clients at once each answered with their own datagrams, -s
ending a session with its first answer, STRING_LONG of silence ending
it (92), maxconn leaving a client without a slot unanswered, and a
session adopted on a reload and ended when the new rules deny its
client. dnspr: a query sent on to the name server and its answer
relayed, and a query answered while another waits for a silent name
server, which ends at DNS_TO (821).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On Linux the port windows lie within net.ipv4.ip_local_port_range, as
the kernel honours no other, so the ephemeral port of a request the
range does not apply to falls into the 50-port window now and then
(CI saw 34001 in 33968-34017). The case now makes five such requests
and wants one of them outside the window: bound to it, all would be in.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When a reload cannot take a listening Unix socket over (a bind-time
-ol option changed), the new service binds the path anew, and the old
listener, closing after it, removed the file, the new socket's by then,
leaving nothing to connect to. The socket's file (device and inode) is
now noted on binding and the path removed on closing only if it is
still that file.
structures.h includes <stddef.h> for offsetof() in SASIZE(), which
needs it wherever the header is used.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On Linux a proxy listening on a Unix socket relayed only the first
bytes of an answer (result 95): sockmap() read with recvfrom() into the
client's address, which for a Unix stream socket Linux fills with an
empty AF_UNIX one, and then sent with sendto() to that address, which a
connected Unix stream socket refuses. A TCP connection has no use for
addresses: sockmap() now sends and receives with NULL ones, and
connect-back sends its command with socksend() and reads the answer
with sockrecvfrom() without an address, which a NULL sin now allows.
An abstract socket name (-iunix:@NAME) was bound with the whole size of
struct sockaddr_un, padded with zeros, so a client naming it by its
bytes, as most do, reached another name. SASIZE() now gives an abstract
address its exact size; getsockname() calls that took SASIZE() of the
old contents as the buffer size take the buffer's size instead.
tcppm refused port 0 for a unix: target, though the manual gives
"tcppm 8080 unix:/var/run/app.sock 0" as the example: the port of a
Unix socket target is ignored, so 0 is allowed.
A test case covers a proxy on a Unix socket and on an abstract one on
Linux, tcppm to a Unix socket, a parent reached through one, and a
listening Unix socket across reloads: taken over each time, its file
kept, and removed once no service listens on it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The downloads spanning a reload asked httpsrv for 64 KB blocks 60 ms
apart, meant as two seconds; but httpsrv sends at most 8 KB a block, so
they took 245 waits, some 15 seconds, and on the CI runners, whose
timers stretch, more than the 20 the cases waited. They now ask for
400 KB in 8 KB blocks 40 ms apart, two seconds indeed, and are waited
for up to a minute.
The UDP part of the takeover case slept 1.5 s after the reload adding
its udppm and then counted every datagram unanswered as lost; it now
asks until the service answers first, and wants more than 20
datagrams through the reloads rather than 100, what a slow runner can
send. That none is lost is the check.
On a failure, run.py prints every log line with a result code other
than 0, not only the last lines, so the reason a connection ended is
shown wherever it was logged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A listener counts the traffic of its clients as it is logged: bytes from
servers (%I) and from clients (%O). clearstat(), which every log writer
calls once a line is logged, as well as lognone and logradius, adds the
counts to the client's listener before clearing them, through
listenertrafficfunc, set where services have listeners and NULL in the
standalone binaries.
The admin services page shows them as listenertrafin and
listenertrafout. The takeover case checks that they go on through three
reloads.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
S_HTTPSRV, the type of admin and httpsrv services, was added before
S_ZOMBIE without a name in the string table, so its index held
"ZOMBIE": the admin services page, %N in logs and RADIUS named admin
and httpsrv services and their clients ZOMBIE, and a zombie service had
no name at all (NULL). Index 21 is now "HTTPSRV", 22 "ZOMBIE"; 22 was
unused.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A listener counts its clients, whichever of its services they came to,
old or new: those connected now, raised when a client is spawned, and
those done with, raised as one leaves its service. As the listener
outlives a reload, so do the counts.
The services page of the admin interface shows them for each service as
listenerclients and listenerprocessed; services sharing a socket show
the same counts. The takeover case checks that the count goes on
through three reloads and that no client is left counted.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Services of one configuration on the same address, port, network
namespace and device now share one listener and its socket, all
accepting on it, instead of each binding its own: on Linux the second
bind failed unless SO_REUSEPORT was set. A listener keeps a list of the
services accepting on it; the first to start opens the socket under the
listener's open_mutex, as binding may take long, and the others use it
("Accepting connections, shared"). The socket is closed once the last of
them stops.
Such services must be of one type with the same -ol options: otherwise
the configuration is refused ("a service of another type listens
there", "a service with other socket options listens there").
mainclaim() now runs for every service, at startup too: it joins the
listener of its configuration for the address, takes over one of an
older configuration (one of its type first), or makes a new one, under
one hold of listener_mutex so that a listener found is not freed before
it is joined.
Services connecting back (-r) to one place share a pseudo-listener
without a socket, of any type. After a reload a client moves to one of
the newest services of its type on its listener with a slot free: 26 if
there is none of its type, 27 if none has a slot.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Only the -ol options which act when a socket is bound (SO_REUSEADDR,
SO_REUSEPORT, SO_EXCLUSIVEADDRUSE, SO_PORT_SCALABILITY,
SO_REUSE_UNICASTPORT, IP_TRANSPARENT, TCP_FASTOPEN) must be the same
for a socket to be taken over. The others are set on the socket taken
over, those no longer given turned off; TCP_MAXSEG, whose default is
not 0, still needs a new socket when it is dropped.
Several services may still share an address, in one configuration or
across reloads: each new one takes over a listener of its own, one of
an old service of its type first, or opens another.
A service connecting back (-r) has no socket to share, but now a
listener without one for the place it connects to: on a reload the new
service connecting to the same place takes it over, and the old
service's clients are re-authenticated as its clients, as a listening
service's are.
Nothing moves from srvparam to the listener: its socket is what accept
and UDP sessions use, and the rest is the service's configuration,
which the listener only copies to compare.
Tests: a live option changed keeps the socket, a bind-time one makes
another, two services on one address are both taken over, and a
download through a connect-back service goes on or is cut as its user
is kept or removed. 3proxy(8) describes the takeover and
re-authentication.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
UDP sessions check for a reload in udpsockmap() as TCP connections do
in sockmap(): unless noforce, a session goes by the newest service on
its listener, re-authenticated as its client, or is dropped.
A request on a connection that outlives a reload, an HTTP one kept
alive, is authorized by clientauth(): it moves the connection to the
newest service on its listener first if a reload came since, then
authorizes as the service whose rules apply (PSRV). The proxy uses it
for every request, so a request of a user the new configuration
removed is answered 407 rather than authorized by the old rules and
cut while relayed. reattach() takes whether to authenticate: sockmap()
and udpsockmap() move and authenticate at once, clientauth() moves and
then authorizes the request.
A test case covers the drops: with one slot free in the new service one
of two downloads moves on and the other is cut (27; a service keeps one
of its maxconn slots for the next connection it accepts), and a proxy
download is cut when a socks service takes its port (26). The reload
case checks the kept-alive request.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A UDP service's socket is kept as a listener like a TCP one, the
protocol part of what must match: on a reload a new UDP service on the
same address takes it over. It is the way the old service's sessions
answer as well, and they go on answering through it; the new service
reads what comes, a datagram of an old session forwarded to it as
before, sessions being found by the service's and the client's
addresses whichever service made them. The socket stays open while the
old sessions do.
The takeover case keeps a udppm session going through three reloads
and wants every datagram back; with the build before this datagrams
were lost.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
After a reload, unless noforce, a client of an old service goes by the
rules of the newest service on its listener, the one accepting there
now. It takes one of that service's maxconn slots, which counts it
there and keeps that service and its configuration alive until the
client is done, and is re-authenticated as its client: access lists,
users, authentication, bandwidth limits and traffic counters come from
the new configuration. A later reload moves it on, giving the slot
back. The client stays its own service's child, which keeps its
filters and TLS state.
It is dropped if its service was removed or is of another type now
(26), or if the newest service has no free slot (27); what the
re-authentication refuses ends it as before. A service without a
listener (UDP, connecting back) keeps its clients as they are.
This restores what force did before the configuration was kept per
service: since then a client compared its version with its own
service's configuration, which never changes, and was never checked
again. Services now take their configuration's version as it is: a
reload numbers the new configuration before reading it, and so does
startup now, where services were numbered one ahead.
A listener now stays with its old service until that service's last
client is gone, those clients looking the newest service up through
it; the socket itself is still closed as soon as nobody accepts on it.
Connection limits are given back in the configuration they were taken
in. PSRV() is the service whose rules apply to a client.
The reload case checks that a download of a user the new configuration
keeps finishes under it, and that one of a user it removes is cut.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A service's TCP listening socket is kept as a listener: when a reload
commits, before the old services are told to stop, each new service
takes over the listener of an older configuration's service on the same
address, port, network namespace and device with the same -ol options.
It skips socket() and bind(), calling only listen() again for its own
backlog, and logs "Accepting connections, taken over". The old service
stops accepting at its next poll and lets go of the socket; whoever
lets go last closes it, unlinking a Unix socket's path.
So a service staying where it was no longer leaves a gap: before, the
old one closed its socket and the new one opened another, and a
connection queued on the old socket was reset. UDP services, whose
clients read from the service's socket, and two services of the same
configuration on one address keep their own sockets as before.
A test case makes requests nonstop through three reloads and wants none
refused or reset; against the build before this, connections were
reset. The reload case no longer waits out the switch. SIGUSR1 in
3proxy(8) describes how a reload behaves now.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A reload reads the file into a new configuration, allocated from the
defaults, and prepares its services without starting them. Only if the
whole file is read without an error does it come into force: the old
configuration's services stop accepting, the new ones start. On an
error everything prepared is dropped and the old configuration, which
never stopped, stays in force.
Each service holds its configuration until it is freed, after its last
client; a configuration no longer in force is released when the last
holder lets go (confhold/confput). freeconf() now only releases what a
configuration holds, touching nothing global.
What runs without a service of its own reads the configuration in
force, runconf, rather than conf, which a reload points at the one
being read: the resolver, which holds it while it waits (a client
resolves with its own service's), the log, the thread pool, the admin
pages, which walk the services under config_mutex, and the signal
handlers.
The users table, havelog and the resolver choice are per configuration
now, the resolver applied when the configuration comes into force, so
a refused file leaves the users as they were. The log file, the
configuration file name, the string table, daemon mode and the filter
list belong to the process and are handed from one configuration to
the next. Settings left out of a new file go back to their defaults,
where before they kept the old values.
The gap between the old and the new listening sockets remains: a
connection queued on an old service's socket when it closes is reset.
Handing the socket over is the next phase.
A test case reloads a broken file (nothing changes, alice still gets
in, bob from the refused file does not) and a good one (bob only, the
service on its new port, the old one closed, a download through the
old service finishing whole). 40 reloads, a third refused, under load
show no memory errors under AddressSanitizer.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
-s[N]:MIN,STREAK,SMALL sets where the automatic mode splices, as -g
does for grace delay: a direction is spliced after STREAK reads in a
row of MIN bytes or more (SMALL from a TLS socket), copied again after
more than STREAK splices in a row of less than SMALL, and a body of
known length is spliced if MIN or more of it is left. The defaults,
16384,2,4096, are what was built in; a value left out or 0 keeps its
default.
The man page describes it, and its HTML copy, which still said splice()
was not built by default, now has the current text. A test case relays
answers of every size, chunked ones and a 3 MB upload through every
mode, two of them with heuristics set, and compares them byte for byte.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A service with ssl_ktls keeps its splice() mode. A connection whose TLS
sockets all got kernel TLS both ways may splice, as a plain one does when
no data filters apply; one where the kernel did not take TLS (nosplice)
copies as before.
TLS may hold data for a socket that a splice() from it would not see:
OpenSSL reads more from a kernel TLS socket than it is asked for. sockmap
asks the TLS code (splicetls) and reads that through TLS into the buffer
first, once the pipe has drained, so the order is kept, then splices on.
A read from a TLS socket counts as large at 4K, TLS handing out a record
at a time, and the short read collecting what TLS left from the last one
does not end a run of large ones. A splice() refused because a record is
not data (an alert, a key update, a ticket) falls back to reading it
through TLS instead of ending the connection.
The kernel TLS test compares the bodies with the origin's.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ssl_ktls asks OpenSSL to hand TLS to the kernel (SSL_OP_ENABLE_KTLS)
on the connections of the services which follow, both to clients and
to servers; ssl_noktls, the default, keeps it in 3proxy. Where the
library, the kernel and the cipher allow it, the kernel then encrypts
and decrypts, and SSL_read/SSL_write go on working over it. What a
service got is logged on its first TLS connection.
Reading ahead is left off where the kernel took receiving, which reads
a record at a time. A build whose library has no kernel TLS (wolfSSL,
OpenSSL before 3.0 or without it) warns and ignores the command.
splice() is not used for TLS services yet.
Tested on AlmaLinux 10 (OpenSSL 3.5.8, tls module): both directions
of both legs of a TLS chain in the kernel, the suite passing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
What a numbered counter counted now belongs to the process: a counter
rule points to the value of its number, created the first time the
number is met (from the counter file, if one is open) and shared by the
rules of that number in every configuration loaded. A reload no longer
resets it when there is no counter file, and once configurations can
outlive one another, clients of the old and the new one count into the
same value. An unnumbered counter, and those LdapPlugin makes, keep a
value of their own as before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The previous commit lets a half-closed client be written to on Windows
too, so the case left out there is back.
This reverts commit 5d1cdf8.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
WSAPoll reports a hang-up as soon as the peer has half-closed, rather
than once both sides are shut down as poll() does elsewhere, and
sockmap() takes a hang-up on a socket it is writing to for a peer it
can no longer write to: a half-closed client gets nothing there. That
is a separate problem, Windows only; the case added with the fix for
the other platforms is skipped there until it is dealt with.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When the client shuts down its sending side, sockmap() shuts down the
server's in turn, and once the server has answered and closed, poll()
reports a hang-up on its socket. A socket sockmap() had nothing to ask
of at that moment, its buffer being full for a slow client, was still
polled, and a hang-up there ended both directions: whatever of the
answer was still unread in the kernel was dropped and the client got an
end of stream in the middle of it, megabytes short with large answers.
A hang-up on such a socket now only marks it: what is left is read
once the buffer has room, up to the end of the stream as usual. Errors
still end the connection. Since poll() reports a hang-up whatever is
asked, the socket is left out of poll() while nothing is asked of it.
The same holds for the client's side, for an upload cut short after the
server half-closes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The reload asked for through /R happens on the next tick of the main
loop, and between the old services stopping and the new ones starting a
request is refused: on Windows the new listener cannot bind the port
until the old one has closed it, and retries with a growing delay. When
the tick fell inside the case, every request after it failed with
ConnectionRefusedError (MSVC run of 98e8dfb). The case now waits for
both services to have restarted after each /R.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
extport binds a local port out of its range and connects from it. A port
which is free to bind may still be half of a pair which is closing from an
earlier connection to the same destination, and Windows fails the connect
with WSAEADDRINUSE where other systems fail the bind and fall back. The
request then failed with error 13 for a reason another port would have
avoided, which is what the range is searched for in the first place.
Retry the connect with another port from the range. Only for that error, so
a destination which is simply unreachable still costs one attempt, and out of
the same ten tries the bind already spends, so the attempts parentretries
makes are not multiplied. EADDRINUSE joins the errno values proxy.h maps to
their Winsock numbers, without which the test would never match on Windows.
parent_ports gave its three services one window between them, which is how
the case met the collision on the Windows runner. Each has its own now.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
bindwithrange() honours a range by setting IP_LOCAL_PORT_RANGE where it
exists, which leaves the kernel to pick a port inside it and skip the ones
still closing. Everywhere else it binds a port out of the range at random,
ten times, and lets the system choose when all ten are taken.
A port which carried a connection cannot be bound again until that close
completes - four minutes of it on Windows - so a window of fifty ports
shared by three services in one configuration can have too few left, and
the connection then binds outside the range and the case fails. The window
is five hundred ports there instead. The Linux one is unchanged: the
kernel picks from it and needs no room to spare.
op_rewrite checked what it built with targetunsafe(), which describes a
path on this machine: on Windows it must name a drive or a share, so a
rewritten request path was refused and every rewrite rule failed there.
A rewrite produces a request path and is checked as one.
The header loop stopped at HTTPSRV_MAXHDR and answered anyway, leaving
the rest of the request in the stream for the next one to be read out
of. It refuses the request instead. A header longer than the buffer
arrives as several lines, so the count bounds what is read rather than
what a client may send in one header.
The test for the buffer growth after a PCRE rewrite sent its request
through to an httpsrv origin, which stops reading at that same cap; it
uses an origin which reads whatever it is sent, since what is under test
is the proxy in the middle.
