A peer gone is then an error of the send rather than SIGPIPE. SIGPIPE
stays ignored for what cannot pass the flag: TLS writes and splice().
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
wolfSSL built with WOLFSSL_QT, as Ubuntu 26.04's 5.9.1 is, returns a
serial number without the ASN.1 header its X509_set_serialNumber()
requires, so every certificate copy failed. The copy then gets a
positive serial number from the hash of the source certificate.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
3proxy ignored SIGPIPE only after readconfig(), which starts the
services: a client gone while written to in the meantime killed the
process (as auto's case saw on macOS). Standalone services never ignored
it. Sends pass no MSG_NOSIGNAL, so this is the only guard.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
sockrecvfrom() polled once and then read in a loop for as long as the
read failed with EAGAIN. Over TLS a socket is readable when only a part
of a record has come, and the read fails with EAGAIN until the rest
does: a client sending half a record and then nothing kept a thread
taking all of a CPU, and the read never timed out. On EAGAIN the socket
is now polled again, the read repeated at once only for EINTR.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
dnspr relayed to the client whatever came first to the socket it asked
the name server from, from any address, under any id. Now:
- the query goes to the name server under a random id of its own, the
client's put back in what is relayed to it;
- an answer over UDP is taken only from the address and port of the
name server asked, anything else read past until DNS_TO runs out;
- the answer must have that id, be a response, and carry the question
asked byte for byte, over UDP and TCP alike (a TCP answer that does
not is error 834).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Read-ahead was off, so OpenSSL read every record in two calls, its
header and then its body. It was off because ssl_poll() asks
ssl_pending(), and SSL_pending() does not count data read ahead and not
yet decrypted: poll() would wait while data waits in OpenSSL.
SSL_has_pending() does count it. With OpenSSL 1.1.0 and later, which
have it, ssl_pending() uses it and records are read ahead; with older
ones and wolfSSL nothing changes.
SSL_has_pending() is also true when only a part of a record has been
read ahead, which is no data yet: poll() would then return at once, over
and over and without its timeout, until the rest came. ssl_pending()
says so only when decrypted data waits, or when what was read ahead
makes a whole record (SSL_peek() of one byte, the sockets not blocking);
otherwise the socket is polled for the rest.
Decrypting TLS streams takes 9-14% less time for the same data (ARM).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
bench/mix/build.sh builds them; a checkout switched from devel, where
bench/mix/.gitignore covers them, keeps them as untracked files.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
proxychild() allocated its 64 KB buffer for headers once and kept it
for the life of the connection: for a CONNECT tunnel all the time data
was relayed, and for a kept-alive connection between its requests.
With 2000 tunnels that was 128 MB, with 20000 mostly idle connections
over a gigabyte.
The buffer is now freed before a tunnel starts relaying, and when a
kept-alive connection has waited a second for its next request; it is
allocated again once the next request starts coming. logurl() takes a
copy of its own for a request it logs after its buffer was freed.
2000 tunnels: 339 to 213 MB; 20000 connections, 3000 exchanges a
second: 2.65 to 1.63 GB through the HTTP proxy, 2.66 to 1.42 GB through
CONNECT; busy kept-alive connections within 1.5% of before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On Linux the port windows lie within net.ipv4.ip_local_port_range, as
the kernel honours no other, so the ephemeral port of a request the
range does not apply to falls into the 50-port window now and then
(CI saw 34001 in 33968-34017). The case now makes five such requests
and wants one of them outside the window: bound to it, all would be in.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On Linux a proxy listening on a Unix socket relayed only the first
bytes of an answer (result 95): sockmap() read with recvfrom() into the
client's address, which for a Unix stream socket Linux fills with an
empty AF_UNIX one, and then sent with sendto() to that address, which a
connected Unix stream socket refuses. A TCP connection has no use for
addresses: sockmap() now sends and receives with NULL ones, and
connect-back sends its command with socksend() and reads the answer
with sockrecvfrom() without an address, which a NULL sin now allows.
An abstract socket name (-iunix:@NAME) was bound with the whole size of
struct sockaddr_un, padded with zeros, so a client naming it by its
bytes, as most do, reached another name. SASIZE() now gives an abstract
address its exact size; getsockname() calls that took SASIZE() of the
old contents as the buffer size take the buffer's size instead.
tcppm refused port 0 for a unix: target, though the manual gives
"tcppm 8080 unix:/var/run/app.sock 0" as the example: the port of a
Unix socket target is ignored, so 0 is allowed.
On a reload the new service bound the path anew and the old one,
closing after it, removed the file, the new service's by then, leaving
nothing to connect to. A service now notes the file it bound (device
and inode) and removes the path on closing only if it is still that
file.
A test case covers a proxy on a Unix socket and on an abstract one on
Linux, tcppm to a Unix socket, a parent reached through one, and a
listening Unix socket across reloads: its file kept and the proxy
answering, and the file removed once no service listens on it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
S_HTTPSRV, the type of admin and httpsrv services, was added before
S_ZOMBIE without a name in the string table, so its index held
"ZOMBIE": the admin services page, %N in logs and RADIUS named admin
and httpsrv services and their clients ZOMBIE, and a zombie service had
no name at all (NULL). Index 21 is now "HTTPSRV", 22 "ZOMBIE"; 22 was
unused.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- readconfig() kept its line buffer and argument array on every error
return, so each refused reload leaked them.
- A filter refusing a connection in filter_client (TLS: a failed
handshake) was not kept, so its filter_clear never ran and the state
it made was lost. makefilters() now clears it, and empties the slot
first: a pooled worker's slot may hold the previous connection's.
- ssl_filter_close() freed the client CA paths but not the server ones,
for every TLS service set up.
- Compiled patterns were never freed: an ACL's (pcre: and regex: host
names), an httpsrv rule's. copyacl() shared them between the
configuration's ACL and each service's copy; each copy now compiles
its own, and freeacl() and freehttprules() free them.
What LeakSanitizer still reports is OpenSSL's per-thread random
generator state of pooled threads, released only by OpenSSL at exit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 796e9f8156)
Open Watcom has no struct _stat64 or _wstat64: its 64-bit stat is
struct _stati64 and _wstati64, so httpsrv.c did not compile there and
the 1.0.1 lite build had no binaries.
widepath() compared and wrote '\\\\', a two-character constant, for a
backslash: a UNC path (\\server\share) never took the \\?\UNC\ form,
and forward slashes became backslashes only because the constant was
truncated to its last byte.
The Watcom workflow passed although nmake failed; it now fails when
the build does or leaves no 3proxy.exe.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ssl_filter_open() copies ssl_client_mode into the service's
configuration, but the filters deciding when to begin TLS towards the
server read the global value instead: the last ssl_client_mode in the
configuration applied to every service, and a reload changed it for the
services still running.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit bab80ef684)
A socket which hung up and of which nothing is asked was given to poll()
as INVALID_SOCKET, so that poll() would not return at once for the
hang-up every time. poll() ignores a negative descriptor, but SOCKET is
unsigned on Windows, and WSAPoll fails the whole call for it: sockmap()
ended the session as soon as a client half-closed. Such a socket is now
not in the array at all.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 98fd0f329f)
sockmap() took a hang-up on a socket it had data to write to for a peer
it could no longer write to. With poll() a hang-up comes once both sides
are shut down, so that held. WSAPoll reports one as soon as the peer has
half-closed, still reading, and a half-closed client got nothing on
Windows: the test added with the previous fix failed there.
The write is now tried, and a peer really gone makes it fail, which
ends the connection as before. WSAPoll keeps reporting the hang-up
whether the socket has room or not, so on Windows room is waited for
with select() first unless poll() reported it too.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit ddefa40affb76de267a8be1cf59a183a8eb77e32)
When the client shuts down its sending side, sockmap() shuts down the
server's in turn, and once the server has answered and closed, poll()
reports a hang-up on its socket. A socket sockmap() had nothing to ask
of at that moment, its buffer being full for a slow client, was still
polled, and a hang-up there ended both directions: whatever of the
answer was still unread in the kernel was dropped and the client got an
end of stream in the middle of it, megabytes short with large answers.
A hang-up on such a socket now only marks it: what is left is read
once the buffer has room, up to the end of the stream as usual. Errors
still end the connection. Since poll() reports a hang-up whatever is
asked, the socket is left out of poll() while nothing is asked of it.
The same holds for the client's side, for an upload cut short after the
server half-closes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 8c87b43a2a)
A kept-alive client connection reuses the connection to the server while
requests go to the same host. The check compared the start of the request
line up to the end of the host, method included, so a POST after a GET to
the same host, or a GET after a POST, closed the server connection and
opened another one. Only the host and port are compared now.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The C library reads /etc/localtime on the first use of local time and
connects to /dev/log on the first syslog() call. With a chroot before
either, every local time became UTC and every syslog message was lost.
The time zone is now loaded before chroot() and openlog() connects at
once. The chroot entry of 3proxy.cfg(5) lists what PAM, LDAP and ODBC
still need inside the chroot.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Without -Ne, the reply to CONNECT and BIND named the local address of the
client's connection to the proxy instead of the socket towards the
destination: the branch meant for UDP ASSOCIATE also took these. A BIND
client was told the proxy's own port and the peer could never reach the
listener. Introduced with -Ne / -Ni (0c8be90).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The benchmarks live on the devel branch only, so link to their results
there; master has no bench/.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
-s takes the directions to splice: -s1 what the server sends (downloads),
-s2 what the client sends (uploads), -s3 both. -s alone (-s7) splices both
only where it pays, as -s5 and -s6 do for one direction:
- a body of known length is spliced if at least 16K of it is left to read
once what is already buffered is counted, so an answer read whole with
its headers is just copied;
- a stream of unknown length starts copied, is spliced after 2 reads in a
row of at least 16K, and goes back to copying after 3 splices in a row
of less than 4K, once its pipe is empty.
Pipes are made when a direction first splices and kept with the
connection for its next mappings, so a kept-alive connection no longer
makes and closes two pipes per request; one left holding data by an
error or a timeout is closed. The direction not spliced keeps its buffer.
Before, -s spliced both directions and every mapping made both pipes,
which made small answers slower than without splice: with kept-alive
50K answers 2 pipe2() and 4 close() per request, now none.
Also corrects the default buffer size in the -b help (65536 for TCP).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The reload asked for through /R happens on the next tick of the main
loop, and between the old services stopping and the new ones starting a
request is refused: on Windows the new listener cannot bind the port
until the old one has closed it, and retries with a growing delay. When
the tick fell inside the case, every request after it failed with
ConnectionRefusedError (MSVC run of 98e8dfb). The case now waits for
both services to have restarted after each /R.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
extport binds a local port out of its range and connects from it. A port
which is free to bind may still be half of a pair which is closing from an
earlier connection to the same destination, and Windows fails the connect
with WSAEADDRINUSE where other systems fail the bind and fall back. The
request then failed with error 13 for a reason another port would have
avoided, which is what the range is searched for in the first place.
Retry the connect with another port from the range. Only for that error, so
a destination which is simply unreachable still costs one attempt, and out of
the same ten tries the bind already spends, so the attempts parentretries
makes are not multiplied. EADDRINUSE joins the errno values proxy.h maps to
their Winsock numbers, without which the test would never match on Windows.
parent_ports gave its three services one window between them, which is how
the case met the collision on the Windows runner. Each has its own now.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
pop3p, imapp and smtpp are built with MAILPROXY=true, and ftppr and the
ftp:// scheme of the HTTP proxy with FTP=true; with CMake the switches are
3PROXY_USE_MAILPROXY and 3PROXY_USE_FTP. Neither is in a default build, and
the standalone binaries follow what was built.
A configuration naming one of them is still read either way. The three mail
protocols amount to a STARTTLS negotiation now that mail is carried over
TLS, so without a proxy of their own those names are tlspr speaking the
protocol: the file holds a stand-in which sets the protocol for the
connection and returns tlspr for the caller to run, which serves the
service name and a parent chain alike and leaves conf.c and the redirect
table untouched. FTP has no such fallback, so the service is known, answers
nothing and logs the refusal.
The Linux workflow builds both, so all of it is still compiled and run.
nserver and authnserver parse their argument with parsehost(), which reads
the first colon as the address/port separator, so a bare IPv6 address picks
up a bogus port number. Document the bracket form in 3proxy.cfg(5) and in
both HOWTOs.
Refs #1275, #460
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Package maintainers, Gentoo among them, build from source and had nothing
signed to verify against: the source archive GitHub generates for a tag is
neither signed nor guaranteed to stay byte-identical.
Add release-tarball.yml. It builds 3proxy-<version>.tar.gz with git archive
from the release tag, signs the tarball and SHA256SUMS-src with the release
key, attests build provenance and uploads all four files to the release.
Also drop the SECURITY.md line about Authenticode signing, left over from the
removal of self-signed Windows binaries.
Closes#1270
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signing the release binaries with a self-signed certificate gives no
trust benefit: Windows does not recognize the issuer, so SmartScreen
warns anyway, and the signature is a frequent trigger for antivirus
false positives.
Drop the Decode Certificate / Extract public certificate / Sign steps
from the MSVC and Watcom workflows, stop shipping the extracted
3proxy.crt in the distribution, and remove the certificate URL from the
build version string.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
bindwithrange() honours a range by setting IP_LOCAL_PORT_RANGE where it
exists, which leaves the kernel to pick a port inside it and skip the ones
still closing. Everywhere else it binds a port out of the range at random,
ten times, and lets the system choose when all ten are taken.
A port which carried a connection cannot be bound again until that close
completes - four minutes of it on Windows - so a window of fifty ports
shared by three services in one configuration can have too few left, and
the connection then binds outside the range and the case fails. The window
is five hundred ports there instead. The Linux one is unchanged: the
kernel picks from it and needs no room to spare.
op_rewrite checked what it built with targetunsafe(), which describes a
path on this machine: on Windows it must name a drive or a share, so a
rewritten request path was refused and every rewrite rule failed there.
A rewrite produces a request path and is checked as one.
The header loop stopped at HTTPSRV_MAXHDR and answered anyway, leaving
the rest of the request in the stream for the next one to be read out
of. It refuses the request instead. A header longer than the buffer
arrives as several lines, so the count bounds what is read rather than
what a client may send in one header.
The test for the buffer growth after a PCRE rewrite sent its request
through to an httpsrv origin, which stops reading at that same cap; it
uses an origin which reads whatever it is sent, since what is under test
is the proxy in the middle.
41 of 112 configuration commands appear in a test. Record the rest as a
plan, ordered by how much of the product each gap covers against how much
of a fixture it needs, so the next person does not have to redo the survey.
Includes the two limitations left deliberately unasserted, and why pinning
the current behaviour there would be wrong.
tcppm and udppm forward a port each, so test both directions of each: a
request through the TCP mapper reaching its target, a datagram through the
UDP one coming back answered, and a mapper whose rules deny the client
answering nothing.
tlspr takes its destination from the name in the handshake, so point that
name at 127.0.0.1 with nsrecord and give the certificate the same name: the
name then both chooses where the request goes and is what the client checks.
dnspr answers from its cache, including the documented 0.0.0.0 record, which
is handed out as an address rather than withheld.
auto is asked to serve an HTTP proxy request, SOCKS4, SOCKS5 and a TLS
handshake on one port, and to make nothing of a request that is none of
them. Its protocols reach different places, so there are two origins.
The SOCKS UDP association was only checked for the port it binds. Send
datagrams through it as well, large and small, and check a second
association gets its own port - and that one bound inside an intport range
still relays.
A UDP service has no socket to connect to, so readiness is found by asking
until it answers rather than racing it.
A rewrite only reached the copy of the request kept for logging. On a
direct connection the server is sent the request line held in the header
buffer, which was parsed and reduced to its path before the filters ran, so
the rewrite showed up in the log and nowhere else. Through an HTTP parent
the rewritten copy is what goes out, which is why it appeared to work
there.
Splice the new path back into that buffer. The destination was chosen, and
the access rules applied to it, before the rewrite happened, so a rewrite
that changes the method or the authority is left alone: acting on it would
send the request somewhere the rules never saw. Say so in the manual.
The copy needed to notice a rewrite is only taken when a request filter
exists, so a proxy without one does no extra work.
The filter walked its list of access rules but tested the first entry each
time round, so anything pcre_extend appended was never consulted and the
command did nothing at all. Test the entry the loop is on.
pcre_extend takes an ACE and no FILTER_ACTION - the rule keeps the action it
was given - so correct the manual, which documented an argument the command
does not read.
The tests cover matching and denial, rule order, client headers, matching
options, the access rule a pcre rule carries, pcre_extend, and rewriting a
reply header and reply data. Request rewriting is covered through an HTTP
parent, which is the path where it reaches the wire.
The CA was created with no extensions, so it is not usable as a CA and
clients report that they cannot get the local issuer certificate. Add
basicConstraints, keyCertSign and a subject key identifier, in a file
rather than through -addext, which LibreSSL - the openssl on macOS and some
BSDs - does not apply the same way.
Ask for the key identifiers on the signed certificates too: OpenSSL 3 adds
them when it signs and LibreSSL does not, and Python has verified strictly
since 3.13, refusing a chain whose certificate carries no
authorityKeyIdentifier. Finish with openssl verify -x509_strict, which is
the check the client will make.
Both recipes were run against OpenSSL 3.6 and LibreSSL 3.3: the old one
fails strict verification, the new one passes on both.
OpenSSL 3 adds a subject and authority key identifier when it signs;
LibreSSL, which is the openssl on a stock macOS, does not. Python has
verified strictly since 3.13 and rejects a chain whose certificate has no
Authority Key Identifier, so the macOS runners refused a certificate the
Linux ones accepted. Ask for both by name, and make the self-check strict
so the next such gap is caught before a handshake.
httpsrv parsed Content-Length and never read what followed. The reply is
followed by a close, and closing a socket that still holds unread data
resets the connection instead of ending it, so a POST could cost the client
the reply it was about to read. Windows does that reliably; the same test
passes on Linux and macOS, which is why it looked flaky. Drain the body,
bounded at a megabyte.
The test CA was built with -addext, which LibreSSL - the openssl on a stock
macOS - does not apply the same way, leaving a certificate that is not
usable as a CA and a client that cannot build a chain to it. Put the
extensions in a file both accept, and verify the generated chain before any
of it is handed to a proxy, so a failure there is not read as a fault in
the proxy.
The MITM case reached its origin by address, so it depended on the
certificate 3proxy spoofs carrying an IP alternative name. It does when the
upstream certificate is copied, which is what happens on Linux and macOS
but not on Windows, where the client then refused the connection. Point a
name at 127.0.0.1 with nsrecord instead, and check the chain rather than
the name: an intercepted certificate names the upstream host, not the one
that was asked for. The log assertions gain from it too, since the name is
better evidence than a port that the request was seen.
nsrecord needs nserver as well as nscache, and has to follow nscache, so
say that in the manual: the record goes into the table nscache allocates,
and the table is only consulted when nserver is set.
The port-range fallback was exercised with a range the same case had
already used, so on a busy machine it could fail to bind for the ordinary
reason rather than the one under test. Use privileged ports, which nothing
can take.
When a case fails, print what its servers wrote: the reason usually goes to
the server's stderr, which was captured and then thrown away.
Three arrangements, with key material generated for the run rather than
kept in the tree: a proxy wrapped in TLS, a proxy that reaches a TLS parent
and verifies it against the CA, and MITM.
The MITM case checks what interception is for: the decrypted request line,
URI and all, reaches the log, where the same request through a plain
CONNECT tunnel leaves only the host and port.
The origin runs in its own process there so the proxy log holds only what
the proxy saw, and log assertions wait, since a record is written when the
connection finishes rather than when the reply arrives.
Verification of the spoofed certificate is deliberately not strict: 3proxy
issues those without an Authority Key Identifier, which Python rejects
under its 3.13 defaults.
The cmake Windows build links vcpkg's pcre2, whose DLL lives in
c:/vcpkg/installed/x64-windows/bin. Running the tests without that on PATH
started a 3proxy that died at load with 0xC0000135 before opening a socket,
which the suite could only report as "never listened".
Report the reason instead of leaving it blank: name the exit code, and say
when a body is empty because the request failed rather than because the
reply carried nothing.
The release carried nothing installable on a router. Build the OpenWrt package
in the SDK for the four architectures that cover most consumer hardware:
mipsel_24kc ramips, MediaTek MT7620/MT7621
mips_24kc ath79, Atheros/QCA
arm_cortex-a7_neon-vfpv4 ipq40xx
aarch64_cortex-a53 mediatek filogic and similar
Linking against the distribution's libraries rather than building static keeps
the package near 100kB and gives it TLS and PCRE support.
The SDK file name carries the toolchain flavour and differs between targets -
ipq40xx is musl_eabi where the others are musl - so it is taken from the
directory listing instead of being assembled from the target name.
The package is built from the checked out tree rather than the published
release archive: the workflow runs when a release is created, and depending on
GitHub having generated that archive already would be a race.
opkg verifies the signature of a feed index and never of a package file, so
the packages are published with checksums and a signature over them, in the
same shape as the other artifacts, and are not signed in any opkg specific
way.
3proxy is not in the OpenWrt feed, and the static binaries built here install
nowhere in particular: they carry no service definition, no configuration and
no way to survive a sysupgrade.
Add the package as it would be submitted to openwrt/packages - a Makefile, a
procd init script and a UCI schema. Built in the 24.10 SDK for mipsel_24kc
against the distribution's libraries, so the result is a 220kB package
depending on libopenssl and libpcre2 rather than a static binary, with TLS and
PCRE support actually present.
3proxy.cfg is order dependent, which is the whole difficulty of generating it
from UCI, where sections are unordered:
- access rules are named sections referenced by a service through an ordered
list, so the reference order decides precedence, and the list is flushed
before each service so rules do not leak into the next one
- parent proxies extend the allow rule they follow, so they hang off the
access rule rather than the service
- the TLS switches apply to every service below them, so the generator
tracks what is in effect and emits a directive only when a service needs a
different state, rather than letting certificate spoofing leak into a
service that did not ask for it
- bandwidth, connection and counter limits are global and match on their own
ACL pattern, so they are a separate ordered list
Nothing is enabled by default: the global switch is off, no service ships
enabled, and a service section that omits the option does not start either, so
installing the package opens no ports.
Values that would make 3proxy reject the whole configuration at boot are
checked while writing it - unknown access actions, limiter types, pcre types
and actions - and reported instead of being passed through.
The wiki pages were copies of doc/html made by a script run by hand outside
this repository, so the wiki drifted from master whenever that step was
skipped.
Copy them from a workflow instead, taking master as the source and committing
only when a page actually changed.
GitHub wikis cannot run Actions, so the workflow lives here and pushes to the
wiki repository rather than the other way round.
It runs on a push touching doc/html, and also after the documentation workflow
finishes: that workflow commits with GITHUB_TOKEN, and GitHub does not raise
push events for those commits, so a push trigger alone would never see
regenerated HTML.
The HTML under doc/html was produced by a script kept outside this repository,
so editing a man page and refreshing its HTML were separate manual steps and
the two could drift apart.
Run the same groff conversion in a workflow triggered by a change under man/.
It rebuilds doc/html/index.html and the man5 and man8 pages, and commits only
when something actually changed.
Output was verified byte for byte against the committed tree: all twelve man
pages and index.html are reproduced exactly.
The build date stored in version.h is now the single time source for a
release. Each package workflow converts it to SOURCE_DATE_EPOCH and passes
that into the build containers, the debian changelog entry is stamped from it
rather than from the build clock, and rpmbuild is told to use it as the build
time and to clamp file mtimes to it.
rpm also records the build host, which is a container id and therefore differs
on every run, so pin it as well - otherwise the timestamps alone do not make
the package reproducible.
Rebuilding the same commit now produces byte identical packages, verified for
both formats by building twice and comparing checksums.
In the version workflow the clock is read exactly once, when a new build date
is minted. YEAR3PROXY is derived from the build date instead of being read
separately, so the two cannot straddle a year boundary and a re-run in a later
year no longer rewrites version.h.
The version files were derived from RELEASE by a script kept outside this
repository, so bumping RELEASE and updating src/version.h, the spec version
and debian/changelog were separate manual steps.
Do it in a workflow triggered by a change to RELEASE. It writes the same
content the external script produced, verified byte for byte against the
current tree.
The build date is now minted once, when RELEASE changes, and stored in
version.h rather than taken from the clock of whichever build is running. The
package workflows read it from there, so rebuilding a commit produces the same
version string and the same package file names. Re-running the workflow
without a RELEASE change keeps the existing build date and makes no commit.
The rpm build runs in a container as root, so the packages arrived in the
workspace owned by root. Renaming them still worked, since that only needs
write permission on the directory, but rpm --addsign rewrites the file in
place and failed:
error: 3proxy-0.9.9.0.el9.x86_64.rpm: open failed: Permission denied
Restore the invoking user's ownership before leaving the container, falling
back to a mode change where the container cannot chown, as under a rootless
container runtime.
dpkg-buildpackage ran on the runner itself, so the packages inherited its ABI.
On ubuntu-latest that means glibc 2.38 and libssl3t64, which restricts the
packages to Ubuntu 24.04 and newer - Debian 12, Debian 13 and Ubuntu 22.04
cannot install them.
Nothing in the source needs glibc 2.38. The floor comes from the build host:
gcc 13 with glibc 2.38 redirects sscanf and strtol to __isoc23_ variants, and
arc4random arrived in glibc 2.36.
Build in an Ubuntu 22.04 container instead. That lowers the dependencies to
libc6 (>= 2.34) and libssl3, and libssl3t64 declares Provides: libssl3, so one
package covers Ubuntu 22.04 and later as well as Debian 12 and later. Verified
by installing on all four.
The armhf cross build moves into the same container, keeping the cross
toolchain on PATH so that install -s uses the ARM strip rather than the host
one.
No published release was ever signed with it - signing was introduced after
0.9.9 - so it verifies nothing and only adds a second key for users to reason
about.
rpmbuild ran directly on the Ubuntu runner, so the packages carried Ubuntu's
ABI: glibc 2.38, libssl.so.3 and libpcre2-8. No RPM distribution can satisfy
that - RHEL 9 has glibc 2.34, RHEL 8 has 2.28 and OpenSSL 1.1 - so the rpms
were installable nowhere.
Build them in AlmaLinux 8, 9 and 10 containers instead, covering supported
AlmaLinux and CentOS Stream releases. Release: 1%{?dist} already in the spec
now expands, so the packages are named el8, el9 and el10 and can coexist.
Two spec changes are needed for a build outside Ubuntu:
debug_package is disabled. Enterprise Linux enables debuginfo extraction by
default while Ubuntu's rpm does not; the build produces no debug sources, so
rpmbuild aborted on an empty debugsourcefiles.list.
/bin/3proxy is excluded from automatic requires. The installed configuration
file is executable and begins with #!/bin/3proxy, so rpm generates a
dependency on it. rpm 4.16 and later canonicalise that to /usr/bin/3proxy
while %files declares /bin/3proxy, leaving a dependency that can never be
satisfied and making the package uninstallable on el9 and el10.
Enterprise Linux has no 32-bit ARM, so the armhf workflow now builds only the
deb package.
The OpenSSL, wolfSSL, PCRE2 and PAM probes built their test program with
echo "...\n...". make runs recipes through /bin/sh, which is dash on Debian
and Ubuntu, where the builtin echo expands \n. On distributions where /bin/sh
is bash - every RPM based distribution, and macOS - it does not, so the probe
compiled
#include <openssl/ssl.h>n int main(){return 0;}
which is not valid C. Every probe therefore failed and the build silently
dropped TLS, PCRE2 and PAM support with no diagnostic.
Use printf, which expands escapes the same way everywhere.
rpm 4.14 (RHEL/CentOS/Rocky 8) cannot import an Ed25519 public key at all:
the import fails and package verification reports SIGNATURES NOT OK. RHEL 8
is supported until 2029 and is a realistic target for a signed package repo,
so the Ed25519 key excludes a large part of the rpm audience.
Publish an RSA-4096 signing key as 3proxy-release-key.asc and keep the old
Ed25519 key as 3proxy-release-key-ed25519.asc so artifacts up to 0.9.9 stay
verifiable. Workflows derive the key id from the imported secret key, so no
workflow change is needed.
The release key is ed25519, so rpm stores the header signature in
DSAHEADER (EdDSA/SHA256) and RSAHEADER is empty. The verification only
looked at RSAHEADER and failed correctly signed packages.
Do not rely on the gpg-agent passphrase cache surviving between steps:
the signing command gets the passphrase from a file. A test signature is
made first, so a wrong passphrase or a key which can not sign fails with
a clear gpg error instead of an rpm which is silently left unsigned.
On failure the rpm version, the sign command and the key capabilities are
printed.
rpm on Debian/Ubuntu defaults %__gpg to /usr/bin/gpg2, which does not exist,
so rpm --addsign failed with 'Could not exec gpg' and packages were left
unsigned.
GPG signing of rpm packages and detached signatures for deb packages and
for the SHA256SUMS files, using GPG_PRIVATE_KEY / GPG_PASSPHRASE secrets.
Checksums are published as release assets.
Build provenance is attested before the assets are uploaded, so a failed
attestation does not leave unattested files published.
Workflow permissions are read-only, write permissions are requested per
job only where they are used.
Docker: single workflow, one job per image per platform instead of one
workflow per registry. Platforms are built in parallel and pushed by
digest, then combined into a manifest list pushed to Docker Hub and GHCR
at once, so both registries get identical digests. arm64 and arm/v7 build
on native arm runners, ppc64le is dropped.
Registry provenance/sbom attestations are disabled (they were shown as
unknown/unknown entries in the registries), build provenance is attested
with actions/attest-build-provenance instead and is verifiable with
'gh attestation verify oci://...'. Release binaries (rpm, deb, zip) are
attested the same way. cosign version is pinned and images are signed by
digest.