errno is WSAGetLastError() on Windows, but seterrno3() set the C
library's errno there: the EAGAIN of a TLS read or write wanting the
socket again was not seen as one, and the error left was whatever the
socket call before had set. seterrno3() sets the socket error now.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A server which logs in on USER alone (230, anonymous FTP often does) or
refuses it had its answer read but not handed back: ftplogin() gave the
length of what it read only for the answer to PASS. The client, told
331 by ftppr and waiting for the answer to its PASS, waited until the
time out. The answer to USER is the client's now when there is no PASS.
The test case logs in through ftppr to a server answering 230, a
multiline 230, 331 then 230, 530, and 331 then 530.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tlspr splits the ClientHello it passes in the middle of the server name
(SNI) when the service has s_option, set by -s. In builds with splice()
(Linux, the default there) -s is splice for every service but UDP ones,
admin and httpsrv, so the split was never made, and the How-To's SNI
break did nothing on Linux.
-B sets it whatever the build: for TCP services but admin and httpsrv,
where s_option has a meaning of its own. -s still does it where it is
not splice. The How-To, 3proxy.cfg(5) and tlspr(8) give -B.
Fixes#1283
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The socket of a connection to the destination is bound to the external
address before connect(), with port 0 unless a range is given. bind()
then takes an ephemeral port for the socket alone: not one in use
towards another destination, nor one of a connection still closing in
TIME_WAIT, which connect() would reuse. With many connections the range
runs out and bind() fails, result 12, or searches long for a port: 7 ms
a call was seen. The more connections are open at once, the sooner:
with event loops, serving more of them at once, the client of every
tenth connection or so of a saturated relay got its end at once.
Where the system has IP_BIND_ADDRESS_NO_PORT (Linux 4.2), the port is
left to connect() for a socket that is going to connect: chosen for
the pair with the destination, together with IP_LOCAL_PORT_RANGE when
a range is given.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
It was made with socket(), past the socket functions of the service,
which the rest of its life went through.
(cherry picked from commit 7718ffa)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The clients of a -R service all wait for the connection back on the
same listening socket, which was blocking: one connection woke them
all, one took it, and the others were left in accept() until the next,
past their time out and through a reload. The socket is non-blocking,
and an accept() which finds nothing, the connection taken by another
client or gone, is not one of the client's tries. nonblock() makes a
socket non-blocking, acceptagain() tells such an accept().
(cherry picked from commit 126c7d0)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A read TLS could do only once it wrote, or a write once it read, was an
error which ended the connection, as when TLS 1.3 answers a key update
while the socket cannot take more. Each TLS socket remembers what a read
and a write it could not do wait for, and the poll of the TLS layer
waits for it instead of what was asked: whatever that gets is the
caller's chance to try again, revents what it asked for, and its events
are given back as they were.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
socksend() takes the client's parameters, and was passed the socket
functions' state instead: NULL, the global functions used then, as
intended, but on a TLS connection the state of its TLS layer, which
socksend() read as a clientparam.
(cherry picked from commit 718798c)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ubuntu-latest is now 26.04. Its rpm 6 runs the signing command of
.rpmmacros without taking its second word as argv[0], so gpg got the
"gpg" there as a file name and every rpm of 1.0.2 failed to sign. The
OpenWrt 24.10 SDK refuses its install, which is not GNU's. arm64 already
builds on ubuntu-24.04-arm.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A peer gone is then an error of the send rather than SIGPIPE. SIGPIPE
stays ignored for what cannot pass the flag: TLS writes and splice().
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
wolfSSL built with WOLFSSL_QT, as Ubuntu 26.04's 5.9.1 is, returns a
serial number without the ASN.1 header its X509_set_serialNumber()
requires, so every certificate copy failed. The copy then gets a
positive serial number from the hash of the source certificate.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
3proxy ignored SIGPIPE only after readconfig(), which starts the
services: a client gone while written to in the meantime killed the
process (as auto's case saw on macOS). Standalone services never ignored
it. Sends pass no MSG_NOSIGNAL, so this is the only guard.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
sockrecvfrom() polled once and then read in a loop for as long as the
read failed with EAGAIN. Over TLS a socket is readable when only a part
of a record has come, and the read fails with EAGAIN until the rest
does: a client sending half a record and then nothing kept a thread
taking all of a CPU, and the read never timed out. On EAGAIN the socket
is now polled again, the read repeated at once only for EINTR.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
dnspr relayed to the client whatever came first to the socket it asked
the name server from, from any address, under any id. Now:
- the query goes to the name server under a random id of its own, the
client's put back in what is relayed to it;
- an answer over UDP is taken only from the address and port of the
name server asked, anything else read past until DNS_TO runs out;
- the answer must have that id, be a response, and carry the question
asked byte for byte, over UDP and TCP alike (a TCP answer that does
not is error 834).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Read-ahead was off, so OpenSSL read every record in two calls, its
header and then its body. It was off because ssl_poll() asks
ssl_pending(), and SSL_pending() does not count data read ahead and not
yet decrypted: poll() would wait while data waits in OpenSSL.
SSL_has_pending() does count it. With OpenSSL 1.1.0 and later, which
have it, ssl_pending() uses it and records are read ahead; with older
ones and wolfSSL nothing changes.
SSL_has_pending() is also true when only a part of a record has been
read ahead, which is no data yet: poll() would then return at once, over
and over and without its timeout, until the rest came. ssl_pending()
says so only when decrypted data waits, or when what was read ahead
makes a whole record (SSL_peek() of one byte, the sockets not blocking);
otherwise the socket is polled for the rest.
Decrypting TLS streams takes 9-14% less time for the same data (ARM).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
bench/mix/build.sh builds them; a checkout switched from devel, where
bench/mix/.gitignore covers them, keeps them as untracked files.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
proxychild() allocated its 64 KB buffer for headers once and kept it
for the life of the connection: for a CONNECT tunnel all the time data
was relayed, and for a kept-alive connection between its requests.
With 2000 tunnels that was 128 MB, with 20000 mostly idle connections
over a gigabyte.
The buffer is now freed before a tunnel starts relaying, and when a
kept-alive connection has waited a second for its next request; it is
allocated again once the next request starts coming. logurl() takes a
copy of its own for a request it logs after its buffer was freed.
2000 tunnels: 339 to 213 MB; 20000 connections, 3000 exchanges a
second: 2.65 to 1.63 GB through the HTTP proxy, 2.66 to 1.42 GB through
CONNECT; busy kept-alive connections within 1.5% of before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On Linux the port windows lie within net.ipv4.ip_local_port_range, as
the kernel honours no other, so the ephemeral port of a request the
range does not apply to falls into the 50-port window now and then
(CI saw 34001 in 33968-34017). The case now makes five such requests
and wants one of them outside the window: bound to it, all would be in.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On Linux a proxy listening on a Unix socket relayed only the first
bytes of an answer (result 95): sockmap() read with recvfrom() into the
client's address, which for a Unix stream socket Linux fills with an
empty AF_UNIX one, and then sent with sendto() to that address, which a
connected Unix stream socket refuses. A TCP connection has no use for
addresses: sockmap() now sends and receives with NULL ones, and
connect-back sends its command with socksend() and reads the answer
with sockrecvfrom() without an address, which a NULL sin now allows.
An abstract socket name (-iunix:@NAME) was bound with the whole size of
struct sockaddr_un, padded with zeros, so a client naming it by its
bytes, as most do, reached another name. SASIZE() now gives an abstract
address its exact size; getsockname() calls that took SASIZE() of the
old contents as the buffer size take the buffer's size instead.
tcppm refused port 0 for a unix: target, though the manual gives
"tcppm 8080 unix:/var/run/app.sock 0" as the example: the port of a
Unix socket target is ignored, so 0 is allowed.
On a reload the new service bound the path anew and the old one,
closing after it, removed the file, the new service's by then, leaving
nothing to connect to. A service now notes the file it bound (device
and inode) and removes the path on closing only if it is still that
file.
A test case covers a proxy on a Unix socket and on an abstract one on
Linux, tcppm to a Unix socket, a parent reached through one, and a
listening Unix socket across reloads: its file kept and the proxy
answering, and the file removed once no service listens on it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
S_HTTPSRV, the type of admin and httpsrv services, was added before
S_ZOMBIE without a name in the string table, so its index held
"ZOMBIE": the admin services page, %N in logs and RADIUS named admin
and httpsrv services and their clients ZOMBIE, and a zombie service had
no name at all (NULL). Index 21 is now "HTTPSRV", 22 "ZOMBIE"; 22 was
unused.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- readconfig() kept its line buffer and argument array on every error
return, so each refused reload leaked them.
- A filter refusing a connection in filter_client (TLS: a failed
handshake) was not kept, so its filter_clear never ran and the state
it made was lost. makefilters() now clears it, and empties the slot
first: a pooled worker's slot may hold the previous connection's.
- ssl_filter_close() freed the client CA paths but not the server ones,
for every TLS service set up.
- Compiled patterns were never freed: an ACL's (pcre: and regex: host
names), an httpsrv rule's. copyacl() shared them between the
configuration's ACL and each service's copy; each copy now compiles
its own, and freeacl() and freehttprules() free them.
What LeakSanitizer still reports is OpenSSL's per-thread random
generator state of pooled threads, released only by OpenSSL at exit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 796e9f8156)
Open Watcom has no struct _stat64 or _wstat64: its 64-bit stat is
struct _stati64 and _wstati64, so httpsrv.c did not compile there and
the 1.0.1 lite build had no binaries.
widepath() compared and wrote '\\\\', a two-character constant, for a
backslash: a UNC path (\\server\share) never took the \\?\UNC\ form,
and forward slashes became backslashes only because the constant was
truncated to its last byte.
The Watcom workflow passed although nmake failed; it now fails when
the build does or leaves no 3proxy.exe.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ssl_filter_open() copies ssl_client_mode into the service's
configuration, but the filters deciding when to begin TLS towards the
server read the global value instead: the last ssl_client_mode in the
configuration applied to every service, and a reload changed it for the
services still running.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit bab80ef684)
A socket which hung up and of which nothing is asked was given to poll()
as INVALID_SOCKET, so that poll() would not return at once for the
hang-up every time. poll() ignores a negative descriptor, but SOCKET is
unsigned on Windows, and WSAPoll fails the whole call for it: sockmap()
ended the session as soon as a client half-closed. Such a socket is now
not in the array at all.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 98fd0f329f)
sockmap() took a hang-up on a socket it had data to write to for a peer
it could no longer write to. With poll() a hang-up comes once both sides
are shut down, so that held. WSAPoll reports one as soon as the peer has
half-closed, still reading, and a half-closed client got nothing on
Windows: the test added with the previous fix failed there.
The write is now tried, and a peer really gone makes it fail, which
ends the connection as before. WSAPoll keeps reporting the hang-up
whether the socket has room or not, so on Windows room is waited for
with select() first unless poll() reported it too.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit ddefa40affb76de267a8be1cf59a183a8eb77e32)
When the client shuts down its sending side, sockmap() shuts down the
server's in turn, and once the server has answered and closed, poll()
reports a hang-up on its socket. A socket sockmap() had nothing to ask
of at that moment, its buffer being full for a slow client, was still
polled, and a hang-up there ended both directions: whatever of the
answer was still unread in the kernel was dropped and the client got an
end of stream in the middle of it, megabytes short with large answers.
A hang-up on such a socket now only marks it: what is left is read
once the buffer has room, up to the end of the stream as usual. Errors
still end the connection. Since poll() reports a hang-up whatever is
asked, the socket is left out of poll() while nothing is asked of it.
The same holds for the client's side, for an upload cut short after the
server half-closes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 8c87b43a2a)
A kept-alive client connection reuses the connection to the server while
requests go to the same host. The check compared the start of the request
line up to the end of the host, method included, so a POST after a GET to
the same host, or a GET after a POST, closed the server connection and
opened another one. Only the host and port are compared now.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The C library reads /etc/localtime on the first use of local time and
connects to /dev/log on the first syslog() call. With a chroot before
either, every local time became UTC and every syslog message was lost.
The time zone is now loaded before chroot() and openlog() connects at
once. The chroot entry of 3proxy.cfg(5) lists what PAM, LDAP and ODBC
still need inside the chroot.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Without -Ne, the reply to CONNECT and BIND named the local address of the
client's connection to the proxy instead of the socket towards the
destination: the branch meant for UDP ASSOCIATE also took these. A BIND
client was told the proxy's own port and the peer could never reach the
listener. Introduced with -Ne / -Ni (0c8be90).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The benchmarks live on the devel branch only, so link to their results
there; master has no bench/.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
-s takes the directions to splice: -s1 what the server sends (downloads),
-s2 what the client sends (uploads), -s3 both. -s alone (-s7) splices both
only where it pays, as -s5 and -s6 do for one direction:
- a body of known length is spliced if at least 16K of it is left to read
once what is already buffered is counted, so an answer read whole with
its headers is just copied;
- a stream of unknown length starts copied, is spliced after 2 reads in a
row of at least 16K, and goes back to copying after 3 splices in a row
of less than 4K, once its pipe is empty.
Pipes are made when a direction first splices and kept with the
connection for its next mappings, so a kept-alive connection no longer
makes and closes two pipes per request; one left holding data by an
error or a timeout is closed. The direction not spliced keeps its buffer.
Before, -s spliced both directions and every mapping made both pipes,
which made small answers slower than without splice: with kept-alive
50K answers 2 pipe2() and 4 close() per request, now none.
Also corrects the default buffer size in the -b help (65536 for TCP).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The reload asked for through /R happens on the next tick of the main
loop, and between the old services stopping and the new ones starting a
request is refused: on Windows the new listener cannot bind the port
until the old one has closed it, and retries with a growing delay. When
the tick fell inside the case, every request after it failed with
ConnectionRefusedError (MSVC run of 98e8dfb). The case now waits for
both services to have restarted after each /R.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
