bughz/3proxy
1
0
Fork 0
mirror of https://github.com/3proxy/3proxy.git synced 2026-10-16 00:25:48 +08:00
Code Issues Actions 8 Packages Projects Releases Wiki Activity

Make the automatic splice() mode's heuristics a service option

Browse Source 
-s[N]:MIN,STREAK,SMALL sets where the automatic mode splices, as -g
does for grace delay: a direction is spliced after STREAK reads in a
row of MIN bytes or more (SMALL from a TLS socket), copied again after
more than STREAK splices in a row of less than SMALL, and a body of
known length is spliced if MIN or more of it is left. The defaults,
16384,2,4096, are what was built in; a value left out or 0 keeps its
default.

The man page describes it, and its HTML copy, which still said splice()
was not built by default, now has the current text. A test case relays
answers of every size, chunked ones and a 3 MB upload through every
mode, two of them with heuristics set, and compares them byte for byte.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Vladimir Dubrovin 2026-10-14 19:02:23 +03:00
parent cfe52878d0
commit 61041ebe7d
7 changed files with 207 additions and 32 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

33
doc/html/man5/3proxy.cfg.5.html
View File

@ -218,12 +218,33 @@ second. Useful to minimize polling <b><br>
-s</b> <br>
(for admin) secure, allow only secure operations, currently
only traffic counters view without ability to reset. <br>
(for TCP services, Linux) enable splice(). splice() is not
built by default and is disabled even when built, because
current Linux does not implement SPLICE_F_MOVE, so no real
zero-copy takes place and the read/write path is faster for
most traffic. Rebuild with -DWITHSPLICE to make -s
available, -s0 disables it explicitly. <b><br>
(for TCP services, Linux) use splice() to pass data between
the client and the server without copying it through 3proxy.
<b>-s</b> alone splices both directions where it pays: a body
of known length if at least 16K of it is left to read, a
stream once it carries large reads, back to copying when they
turn small. <b>-s1</b> always splices what the server sends
to the client (downloads), <b>-s2</b> what the client sends
(uploads), <b>-s3</b> both; adding 4 (<b>-s5</b>, <b>-s6</b>,
<b>-s7</b>, the last the same as <b>-s</b>, and the default)
splices those directions only where it pays. <b>-s0</b> turns
it off. Where it pays can be tuned with
<b>:</b><i>MIN</i><b>,</b><i>STREAK</i><b>,</b><i>SMALL</i>
after the mode, e.g. <b>-s:16384,2,4096</b> (the default) or
<b>-s6:65536,3,8192</b>: a direction is spliced after
<i>STREAK</i> reads in a row of <i>MIN</i> bytes or more
(<i>SMALL</i> from a TLS socket, which hands out a record at a
time) and copied again after more than <i>STREAK</i> splices
in a row of less than <i>SMALL</i>; a body of known length is
spliced if at least <i>MIN</i> of it is left to read. splice()
suits large transfers: they go faster and use less CPU. A
spliced direction needs a pipe (two file descriptors), made
when first needed and kept for the connection, and more kernel
memory. A service with data filters does not splice, so that
the filters see all of the data, nor does one ending or
beginning TLS, unless the kernel does TLS for it both ways
(<b>ssl_ktls</b>). Available in builds with WITHSPLICE, the
default on Linux. <b><br>
-C</b> <br>
(for TCP services) a connection closed by any of the sides
terminates the session, buffered data is delivered before

17
man/3proxy.cfg.5
View File

@ -209,11 +209,18 @@ once it carries large reads, back to copying when they turn small. \fB-s1\fR alw
splices what the server sends to the client (downloads), \fB-s2\fR what the client
sends (uploads), \fB-s3\fR both; adding 4 (\fB-s5\fR, \fB-s6\fR, \fB-s7\fR, the last the same as
\fB-s\fR, and the default) splices those directions only where it pays. \fB-s0\fR turns it
off. splice() suits large transfers: they go faster and use less CPU; small
requests go slower with it. A spliced direction needs a pipe (two file
descriptors), made when first needed and kept for the connection, and more
kernel memory. A service with data filters does not splice, so that the filters
see all of the data. Available in builds with WITHSPLICE, the default on Linux.
off. Where it pays can be tuned with \fB:\fIMIN\fB,\fISTREAK\fB,\fISMALL\fR after the
mode, e.g. \fB-s:16384,2,4096\fR (the default) or \fB-s6:65536,3,8192\fR: a direction is
spliced after \fISTREAK\fR reads in a row of \fIMIN\fR bytes or more (\fISMALL\fR from a
TLS socket, which hands out a record at a time) and copied again after more
than \fISTREAK\fR splices in a row of less than \fISMALL\fR; a body of known length is
spliced if at least \fIMIN\fR of it is left to read. splice() suits large
transfers: they go faster and use less CPU. A spliced direction needs a pipe
(two file descriptors), made when first needed and kept for the connection,
and more kernel memory. A service with data filters does not splice, so that
the filters see all of the data, nor does one ending or beginning TLS, unless
the kernel does TLS for it both ways (\fBssl_ktls\fR). Available in builds with
WITHSPLICE, the default on Linux.
.br
.B -C
(for TCP services) a connection closed by any of the sides terminates the

5
src/proxy.h
View File

@ -235,6 +235,11 @@ extern struct extparam confboot;
#define SPLICE_DOWNLOAD 1
#define SPLICE_UPLOAD 2
#define SPLICE_AUTO 4
/* where SPLICE_AUTO splices, by default (-s[N]:MIN,STREAK,SMALL): see
sockmap.c */
#define SPLICE_MIN 16384
#define SPLICE_STREAK 2
#define SPLICE_SMALL 4096
int sockmap(struct clientparam * param, int timeo, int usesplice);
int udpsockmap(struct clientparam * param, int timeo);
int udpbind(struct clientparam * param);

23
src/proxymain.c
View File

@ -436,6 +436,10 @@ static int srvprepare(struct srvstart *s, int argc, char **argv){
" -s[N] Use splice(): -s1 server to client, -s2 client to server, -s3 both\n"
" directions; -s (-s7, the default) both, -s5 or -s6 one, only where it\n"
" pays (large transfers); -s0 never. Not with data filters\n"
" :MIN,STREAK,SMALL tune where it pays (-s:16384,2,4096): splice after\n"
" STREAK reads in a row of MIN bytes or more (SMALL from a TLS socket),\n"
" copy again after more than STREAK splices of less than SMALL; a body of\n"
" known length is spliced if MIN or more of it is left\n"
" -C connection closed by any of the sides terminates the session, by default\n"
" the session is kept until both sides close it (TCP half-close)\n"
" -U(0-3) (for socks) what to do when the destination changes within an UDP\n"
@ -716,10 +720,22 @@ static int srvprepare(struct srvstart *s, int argc, char **argv){
#endif
srv->s_option = 1 + atoi(argv[i]+2);
#ifdef WITHSPLICE
else
srv->usesplice = (*(argv[i]+2)? atoi(argv[i]+2) :
else {
/* -s[N][:MIN,STREAK,SMALL] */
char *heur = strchr(argv[i]+2, ':');
srv->usesplice = ((argv[i][2] && argv[i][2] != ':')? atoi(argv[i]+2) :
SPLICE_DOWNLOAD|SPLICE_UPLOAD|SPLICE_AUTO) &
(SPLICE_DOWNLOAD|SPLICE_UPLOAD|SPLICE_AUTO);
if(heur){
int min = 0, streak = 0, small = 0;
sscanf(heur+1, "%d,%d,%d", &min, &streak, &small);
if(min > 0) srv->splicemin = min;
if(streak > 0) srv->splicestreak = streak;
if(small > 0) srv->splicesmall = small;
}
}
#endif
break;
case 'o':
@ -1498,6 +1514,9 @@ void srvinit(struct srvparam * srv, struct clientparam *param){
#ifdef WITHSPLICE
/* where it pays: large transfers, -s0 to never */
srv->usesplice = SPLICE_DOWNLOAD|SPLICE_UPLOAD|SPLICE_AUTO;
srv->splicemin = SPLICE_MIN;
srv->splicestreak = SPLICE_STREAK;
srv->splicesmall = SPLICE_SMALL;
#endif
srv->halfclose = 1;
srv->udpauth = 3;

38
src/sockmap.c
View File

@ -44,18 +44,16 @@ ssize_t splice(int fd_in, loff_t *off_in, int fd_out, loff_t *off_out, size_t le
/* With SPLICE_AUTO, when a direction allowed to splice actually does
(without it, an allowed direction always does).
A body of known length is spliced if at least SPLICE_MIN of it is left
A body of known length is spliced if at least splicemin of it is left
to read once what is already buffered is counted: a smaller one, or one
read whole with its headers, is cheaper to copy.
A stream of unknown length starts copied and is spliced after
SPLICE_STREAK reads in a row of at least SPLICE_MIN each, and goes back to
copying after SPLICE_STREAK+1 splices in a row of less than SPLICE_SMALL. */
#define SPLICE_MIN 16384
#define SPLICE_SMALL 4096
#define SPLICE_STREAK 2
/* a large read from a TLS socket: TLS hands out a record at a time, at most
16K and often less */
#define SPLICE_TLSMIN SPLICE_SMALL
splicestreak reads in a row of at least splicemin each, and goes back to
copying after splicestreak+1 splices in a row of less than splicesmall.
From a TLS socket a read of splicesmall counts as large: TLS hands out a
record at a time, at most 16K and often less. The service sets the three
(-s[N]:MIN,STREAK,SMALL), SPLICE_MIN, SPLICE_STREAK and SPLICE_SMALL by
default. */
#endif
@ -149,6 +147,8 @@ int sockmap(struct clientparam * param, int timeo, int usesplice){
into the buffer first (drain), once the pipe is empty, so as to keep the
order; tail: the last read left some, the next only collects it */
int draincli = 0, drainsrv = 0, tailcli = 0, tailsrv = 0;
int splicemin = param->srv->splicemin, splicestreak = param->srv->splicestreak,
splicesmall = param->srv->splicesmall;
if((!param->nolongdatfilter && (param->ndatfilterscli > 0 || param->ndatfilterssrv)) || param->nosplice) usesplice = 0;
allowcli = (usesplice & SPLICE_UPLOAD) != 0;
@ -190,8 +190,8 @@ int sockmap(struct clientparam * param, int timeo, int usesplice){
if(inclientbuf >= fromclient) FROMCLIENT = 0;
#ifdef WITHSPLICE
/* a body of known length large enough is spliced from the start */
if(autosplice && allowsrv && param->waitserver64 && fromserver > inserverbuf && fromserver - inserverbuf >= SPLICE_MIN) splicesrv = 1;
if(autosplice && allowcli && param->waitclient64 && fromclient > inclientbuf && fromclient - inclientbuf >= SPLICE_MIN) splicecli = 1;
if(autosplice && allowsrv && param->waitserver64 && fromserver > inserverbuf && fromserver - inserverbuf >= (uint64_t)splicemin) splicesrv = 1;
if(autosplice && allowcli && param->waitclient64 && fromclient > inclientbuf && fromclient - inclientbuf >= (uint64_t)splicemin) splicecli = 1;
#endif
if(param->srvinbuf == param->srvoffset) param->srvinbuf =param->srvoffset = 0;
if(param->cliinbuf == param->clioffset) param->cliinbuf =param->clioffset = 0;
@ -504,8 +504,8 @@ log("done read from client to pipe");
graceclitraf += res;
gracesrvnum = gracesrvtraf = 0;
if(autosplice && !param->waitclient64){
if(res >= SPLICE_SMALL) streakcli = 0;
else if(++streakcli > SPLICE_STREAK) wantbufcli = 1;
if(res >= splicesmall) streakcli = 0;
else if(++streakcli > splicestreak) wantbufcli = 1;
}
inclientpipe += res;
if(inclientpipe >= MAXSPLICE) TOCLIENTPIPE = 0;
@ -567,8 +567,8 @@ log("done read from server to pipe\n");
gracesrvtraf += res;
graceclinum = graceclitraf = 0;
if(autosplice && !param->waitserver64){
if(res >= SPLICE_SMALL) streaksrv = 0;
else if(++streaksrv > SPLICE_STREAK) wantbufsrv = 1;
if(res >= splicesmall) streaksrv = 0;
else if(++streaksrv > splicestreak) wantbufsrv = 1;
}
param->nreads++;
param->statssrv64 += res;
@ -620,8 +620,8 @@ log("done read from client to buf");
else if(autosplice && allowcli && !param->waitclient64){
int tls = splicetls? splicetls(param, param->clisock) : 0;
if(res >= (tls? SPLICE_TLSMIN : SPLICE_MIN)){
if(++streakcli >= SPLICE_STREAK){
if(res >= (tls? splicesmall : splicemin)){
if(++streakcli >= splicestreak){
/* a stream: splice from now on, after what is
buffered has been sent */
splicecli = TOCLIENTPIPE = FROMCLIENTPIPE = 1;
@ -675,8 +675,8 @@ log("done read from server to buf");
else if(autosplice && allowsrv && !param->waitserver64){
int tls = splicetls? splicetls(param, param->remsock) : 0;
if(res >= (tls? SPLICE_TLSMIN : SPLICE_MIN)){
if(++streaksrv >= SPLICE_STREAK){
if(res >= (tls? splicesmall : splicemin)){
if(++streaksrv >= splicestreak){
splicesrv = TOSERVERPIPE = FROMSERVERPIPE = 1;
streaksrv = 0;
}

2
src/structures.h
View File

@ -653,6 +653,8 @@ struct srvparam {
unsigned char fakeip6[16];
#ifdef WITHSPLICE
int usesplice;
/* the automatic mode's heuristics, -s[N]:MIN,STREAK,SMALL */
int splicemin, splicestreak, splicesmall;
#endif
int halfclose;
int udpauth;

121
tests/cases/splice.py Normal file
View File

@ -0,0 +1,121 @@
"""splice(): every mode relays what it is given unchanged, small and large,
both ways, the automatic mode's heuristics set or not (-s[N]:MIN,STREAK,SMALL).
Where splice() is not built (anything but Linux) -s is accepted and ignored,
so the same checks then cover copying.
"""
import os
import socket
import threading
MODES = ["-s0", "-s", "-s3", "-s:65536,3,8192", "-s2:32768,2,4096"]
def echo_origin(listener):
"""Sends back whatever each connection sends, until it shuts down its
side."""
def serve(c):
try:
while True:
b = c.recv(65536)
if not b:
break
c.sendall(b)
c.shutdown(socket.SHUT_WR)
except OSError:
pass
c.close()
while True:
try:
c, _ = listener.accept()
except OSError:
return
threading.Thread(target=serve, args=(c,), daemon=True).start()
def echo_through(port, data):
"""data sent through the mapped port while what comes back is read; the
sending side is shut down once all is sent."""
s = socket.create_connection(("127.0.0.1", port), timeout=20)
got = []
def reader():
while True:
try:
b = s.recv(65536)
except OSError:
return
if not b:
return
got.append(b)
th = threading.Thread(target=reader, daemon=True)
th.start()
s.sendall(data)
s.shutdown(socket.SHUT_WR)
th.join(30)
s.close()
return b"".join(got)
def run(t):
origin = t.free_port()
listener = socket.socket()
listener.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
listener.bind(("127.0.0.1", 0))
listener.listen(64)
echo = listener.getsockname()[1]
threading.Thread(target=echo_origin, args=(listener,), daemon=True).start()
proxies, mapped = [], []
conf = f"""
log
auth iponly
allow *
http data * /data
httpsrv -p{origin}
flush
auth iponly
allow *
"""
for mode in MODES:
p, m = t.free_port(), t.free_port()
proxies.append(p)
mapped.append(m)
conf += f" proxy {mode} -p{p}\n tcppm {mode} {m} 127.0.0.1 {echo}\n"
t.start("splice", conf, ports=[origin] + proxies + mapped)
direct = {size: t.http(f"http://127.0.0.1:{origin}/data?size={size}").body
for size in (1000, 100000, 5000000)}
chunked = t.http(f"http://127.0.0.1:{origin}/data?size=1000000&chunked=1").body
up = os.urandom(3000000)
try:
for mode, p, m in zip(MODES, proxies, mapped):
proxy = f"127.0.0.1:{p}"
for size, body in direct.items():
got = t.http(f"http://127.0.0.1:{origin}/data?size={size}", proxy=proxy)
if got.body == body:
t.ok(f"{mode}: a {size} byte answer arrives unchanged")
else:
t.fail(f"{mode}: a {size} byte answer arrives unchanged",
f"{size} bytes", f"{got.length} different bytes")
got = t.http(f"http://127.0.0.1:{origin}/data?size=1000000&chunked=1", proxy=proxy)
if got.body == chunked:
t.ok(f"{mode}: a chunked answer arrives unchanged")
else:
t.fail(f"{mode}: a chunked answer arrives unchanged",
f"{len(chunked)} bytes", f"{got.length} different bytes")
back = echo_through(m, up)
if back == up:
t.ok(f"{mode}: 3 MB sent through tcppm come back unchanged")
else:
t.fail(f"{mode}: 3 MB sent through tcppm come back unchanged",
f"{len(up)} bytes", f"{len(back)} bytes, different")
small = echo_through(m, b"hello")
t.eq(b"hello", small, f"{mode}: a few bytes through tcppm come back")
finally:
listener.close()