Support = in socket options, make TCP_NODELAY default

doc/html/highload.html

@@ -34,7 +34,8 @@ Each connected client requires:
 <ul>
 <li>1 thread (process)
 <li>2 sockets (file descriptors). For FTP, 4 sockets are required.
-<br>Up to 128K of kernel buffer memory. This is the theoretical maximum; actual numbers depend on connection quality and traffic amount.
+<br>With splice() (Linux, -s), 2 more pipes (file descriptors) = 4 file descriptors, 6 for FTP.
+<br>Up to 128K (up to 256K with splice()) of kernel buffer memory. This is the theoretical maximum; actual numbers depend on connection quality and traffic amount.
 <br>1 additional socket (file descriptor) during name resolution for non-cached names
 <br>1 additional socket during authentication or logging for RADIUS authentication or logging.
 <li>1 ephemeral port (3 ephemeral ports for FTP connections).
@@ -279,6 +280,8 @@ tuning anything else.
 
 If 3proxy is used in a VPS environment, there can be additional limitations.
 For example, kernel resources, system CPU usage, and IOCTLs can be limited differently, and this can become a bottleneck.
+splice() (see below) moves work into the kernel, so where the hosting limits kernel resources
+(seen as nearly 100% CPU usage without any real activity of the application) do not enable it.
 
 <h4>Extending the Ephemeral Port Range</h4>
 
@@ -641,17 +644,37 @@ monitor "/some/path/3proxy/3proxy.ver"
 e.g., with "touch /some/path/3proxy/3proxy.ver".
 </ol>
 
-<h4>Use TCP_NODELAY to Speed Up Connections with Small Amounts of Data</h4>
+<h4>TCP_NODELAY for Connections with Small Amounts of Data</h4>
 
-If most requests require an exchange with a small amount of data in both directions
+TCP_NODELAY is set by default on client (oc) and server (os)
-without the need for bandwidth, e.g., messengers or small web requests,
+connections: without it, Nagle's algorithm holds back a short write until the
-you can eliminate Nagle's algorithm delay with the TCP_NODELAY flag. Usage example:
+previous one is acknowledged, and a peer delays its acknowledgement, so a reply
+written in more than one part (e.g. HTTP headers, then a short body) waits tens
+of milliseconds. On slow connections with high delays, where the bandwidth is the
+bottleneck and fewer packets matter more than latency, it can be turned off:
 <pre>
-proxy -osTCP_NODELAY -ocTCP_NODELAY
+proxy -osTCP_NODELAY=0 -ocTCP_NODELAY=0
 </pre>
-sets TCP_NODELAY for client (oc) and server (os) connections.
+
-<p>Do not use TCP_NODELAY on slow connections with high delays when
+<h4>Use Splice to Speed Up Large Data Transfers</h4>
-connection bandwidth is a bottleneck.
+
+On Linux, splice() passes data between the client and the server connections
+through a kernel pipe, without copying it into the 3proxy process. Linux builds
+include it, but a service uses it only when given the -s option:
+<pre>
+proxy -s
+socks -s
+tcppm -s 8080 192.168.1.1 80
+</pre>
+-s0 turns it off explicitly. In builds without it (CMake option 3PROXY_USE_SPLICE=OFF,
+or -DWITHSPLICE removed from the Makefile) -s is accepted and ignored.
+<p>splice() pays off when most of the traffic is large transfers: in the
+<a href="https://github.com/3proxy/3proxy/tree/master/bench">benchmarks</a> it moved
+1.4 to 1.7 times more data than the read/write path at about half the CPU per gigabyte.
+It does not help small requests and short connections, which go slower with it, and
+every spliced connection needs two more pipes (file descriptors) and more kernel
+buffer memory. Data filters are not applied to spliced data.
+<p>TCP_NODELAY and splice() do not conflict and can be combined on high-speed connections.
 
 <h4>Add Grace Delay to Reduce System Calls</h4>
 
@@ -661,7 +684,7 @@ the minimal number of packets in the same direction to apply the algorithm,
 and the last value is the delay added after polling and prior to reading data.
 The example above adds a 10-millisecond delay before reading data if the average
 polling size is below 8000 bytes and 3 read operations have been made in the same
-direction. <pre>logdump 1 1</pre> is useful
+direction. It's especially useful with splice. <pre>logdump 1 1</pre> is useful
 to see how grace delays work; choose a delay value to avoid filling the read
 buffer (typically 64K) but keep the request sizes close to the chosen average
 on large file uploads/downloads.

man/3proxy.cfg.5

@@ -184,8 +184,10 @@ connect to given remote HOST:port instead of listening local connection on -p or
 .br
 .B -oc\fIOPTIONS\fB, -os\fIOPTIONS\fB, -ol\fIOPTIONS\fB, -or\fIOPTIONS\fB, -oR\fIOPTIONS\fR
 options for proxy-to-client (\fB-oc\fR), proxy-to-server (\fB-os\fR), proxy listening (\fB-ol\fR), connect back client (\fB-or\fR), connect back listening (\fB-oR\fR) sockets.
+\fIOPTIONS\fR is a comma separated list of \fINAME\fR or \fINAME\fR=\fIVALUE\fR; a name alone sets the option to 1 (TCP_MAXSEG to the value of \fBmaxseg\fR), and \fINAME\fR=0 turns it off. Options add to the defaults and to the options given earlier for the same socket. An unknown name or a value which is not a number is an error.
+TCP_NODELAY is set by default on proxy-to-client and proxy-to-server sockets, so a reply written in several parts is not delayed by Nagle's algorithm; use \fB-ocTCP_NODELAY=0\fR or \fB-osTCP_NODELAY=0\fR to turn it off, e.g. to send fewer packets on slow links.
 Options like TCP_CORK, TCP_NODELAY, TCP_DEFER_ACCEPT, TCP_QUICKACK, TCP_TIMESTAMPS, TCP_FASTOPEN, SO_REUSEADDR, SO_REUSEPORT, SO_EXCLUSIVEADDRUSE, SO_PORT_SCALABILITY, SO_REUSE_UNICASTPORT, SO_KEEPALIVE, SO_DONTROUTE may be supported depending on OS.
-SO_REUSEADDR and SO_REUSEPORT are set on the listening socket by default on Unix. On Windows SO_REUSEADDR is not set: it is not required to rebind a listening port and it only lets another local process bind the same address and port. Use SO_EXCLUSIVEADDRUSE (Windows) on the listening socket (\fB-ol\fR) to prevent that.
+SO_REUSEADDR and SO_REUSEPORT are set by default on the listening sockets (\fB-ol\fR, \fB-oR\fR) on Unix; \fB-olSO_REUSEPORT=0\fR turns one off. On Windows SO_REUSEADDR is not set: it is not required to rebind a listening port and it only lets another local process bind the same address and port. Use SO_EXCLUSIVEADDRUSE (Windows) on the listening socket (\fB-ol\fR) to prevent that.
 .br
 .B -H
 (for all services) Expect HAProxy PROXY protocol v1 header on incoming connection.
@@ -200,11 +202,13 @@ delay GRACE_DELAY milliseconds before polling if average polling size is below G
  (for admin) secure, allow only secure operations, currently only traffic counters
 view without ability to reset.
 .br
- (for TCP services, Linux) enable splice(). splice() is not built by default and
+ (for TCP services, Linux) use splice() to pass data between the client and the
-is disabled even when built, because current Linux does not implement
+server without copying it through 3proxy. It is off unless -s is given (-s0 turns
-SPLICE_F_MOVE, so no real zero-copy takes place and the read/write path is faster
+it off explicitly). splice() suits large transfers: they go faster and use less
-for most traffic. Rebuild with -DWITHSPLICE to make -s available, -s0 disables it
+CPU. Small requests and short connections go slower with it, every connection
-explicitly.
+needs two more pipes (file descriptors) and more kernel memory, and data filters
+are not applied to spliced data. Available in builds with WITHSPLICE, the default
+on Linux.
 .br
 .B -C
  (for TCP services) a connection closed by any of the sides terminates the

src/common.c

@@ -746,7 +746,7 @@ int doconnect(struct clientparam * param){
 			param->sinsl = param->srv->extsa;
 	}
 	*SAPORT(&param->sinsl) = 0;
-	setopts(param->remsock, param->srv->srvsockopts);
+	setopts(param->remsock, &param->srv->srvsockopts);
 
 #ifndef _WIN32
 	/* not on Windows, see the accepting side in proxymain.c */

src/proxy.h

@@ -501,7 +501,8 @@ extern struct socketoptions {
 	int opt;
 	char * optname;
 } sockopts[];
-void setopts(SOCKET s, int opts);
+int getopts(const char *s, struct sockoptset *opts);
+void setopts(SOCKET s, const struct sockoptset *opts);
 char * printopts(char *sep);
 
 #ifdef _WINCE

src/proxymain.c

@@ -224,27 +224,62 @@ char * printopts(char *sep){
 }
 
 
-int getopts(const char *s){
+/* A comma separated list of options, each NAME or NAME=VALUE; a name alone
-	int i=0, ret=0;
+   stands for 1, or for TCP_MAXSEG for the value of maxseg. Options are added
-	for(; sockopts[i].optname; i++)if(strstr(s,sockopts[i].optname)) ret |= (1<<i);
+   to what is already set, so a default is changed only by naming it, and
-	return ret;
+   turned off with NAME=0. An unknown name or a value which is not a number
+   makes the whole list fail. */
+int getopts(const char *s, struct sockoptset *opts){
+	while(*s){
+		const char *end = strchr(s, ',');
+		const char *eq;
+		size_t len;
+		int i, value = 1;
+
+		if(!end) end = s + strlen(s);
+		eq = memchr(s, '=', (size_t)(end - s));
+		len = (size_t)((eq? eq : end) - s);
+		if(eq){
+			char *num;
+
+			if(!len) return -1;
+			value = (int)strtol(eq + 1, &num, 0);
+			if(num == eq + 1 || num != end) return -1;
+		}
+		else if(len) {
+#ifdef TCP_MAXSEG
+			if(len == 10 && !strncmp(s, "TCP_MAXSEG", 10)) value = 0;
+#endif
+		}
+		if(len){
+			for(i = 0; sockopts[i].optname; i++)
+				if(strlen(sockopts[i].optname) == len && !strncmp(s, sockopts[i].optname, len)) break;
+			if(!sockopts[i].optname || i >= MAXSOCKOPTS) return -1;
+			opts->set |= (1u << i);
+			opts->value[i] = value;
+		}
+		s = *end? end + 1 : end;
+	}
+	return 0;
 }
 
-void setopts(SOCKET s, int opts){
+void setopts(SOCKET s, const struct sockoptset *opts){
-	int i, opt, set;
+	int i, value;
-	for(i = 0; opts >= (opt = (1<<i)); i++){
+
-		set = 1;
+	for(i = 0; sockopts[i].optname && i < MAXSOCKOPTS; i++){
+		if(!(opts->set & (1u << i))) continue;
+		value = opts->value[i];
 #ifdef TCP_MAXSEG
-		if(sockopts[i].opt == TCP_MAXSEG){
+		if(sockopts[i].opt == TCP_MAXSEG && *sockopts[i].optname == 'T'){
-		    if(!conf.maxseg) continue;
+		    if(!value) value = conf.maxseg;
-		    set = conf.maxseg;
+		    if(!value) continue;
 		}
 #endif
-		if(opts & opt) setsockopt(s, *sockopts[i].optname == 'T'? IPPROTO_TCP:
+		setsockopt(s, *sockopts[i].optname == 'T'? IPPROTO_TCP:
 #ifdef SOL_IP
 			*sockopts[i].optname == 'I'? SOL_IP: 
 #endif
-			SOL_SOCKET, sockopts[i].opt, (char *)&set, sizeof(set));
+			SOL_SOCKET, sockopts[i].opt, (char *)&value, sizeof(value));
 	}
 }
 
@@ -356,7 +391,9 @@ int MODULEMAINFUNC (int argc, char** argv){
 	" -64 Prefer IPv6 for outgoing connections, use both IPv4 and IPv6\n"
 	" -ocOPTIONS, -osOPTIONS, -olOPTIONS, -orOPTIONS -oROPTIONS - options for\n"
 	" to-client (oc), to-server (os), listening (ol) socket, connect back client\n"
-	" (or) socket, connect back server (oR) listening socket\n"
+	" (or) socket, connect back server (oR) listening socket, a comma separated\n"
+	" list of NAME or NAME=VALUE (NAME=0 turns an option off); TCP_NODELAY is\n"
+	" set on to-client and to-server sockets by default\n"
 	" where possible options are: ";
 #endif
 
@@ -401,6 +438,34 @@ int MODULEMAINFUNC (int argc, char** argv){
  srvinit(&srv, &defparam);
  srv.pf = childdef.pf;
  isudp = childdef.isudp;
+
+ /* Default socket options, which -oc, -os, -ol and -oR change by naming
+    them, NAME=0 to turn one off.
+
+    Nagle's algorithm holds back a short write until the previous one is
+    acknowledged, and a peer delays its acknowledgement, which stalls every
+    reply written in more than one piece.
+
+    SO_REUSEADDR is not set on Windows: it is not needed to rebind a
+    listening port there, and it only allows another local process to bind
+    the same address and port, with undefined behaviour as to which socket
+    receives the connections. Use -olSO_EXCLUSIVEADDRUSE to prevent that
+    instead. A Windows build which does not share the listening socket is the
+    exception: its UDP services bind a second socket to the same address to
+    answer from, and Windows only allows that when both sockets ask for it.
+ */
+ (void)getopts("TCP_NODELAY", &srv.clisockopts);
+ (void)getopts("TCP_NODELAY", &srv.srvsockopts);
+#ifndef _WIN32
+ (void)getopts("SO_REUSEADDR", &srv.lissockopts);
+ (void)getopts("SO_REUSEADDR", &srv.cbssockopts);
+#elif !defined SHARE_UDP_SOCKET
+ if(isudp) (void)getopts("SO_REUSEADDR", &srv.lissockopts);
+#endif
+#ifdef SO_REUSEPORT
+ (void)getopts("SO_REUSEPORT", &srv.lissockopts);
+ (void)getopts("SO_REUSEPORT", &srv.cbssockopts);
+#endif
 #ifndef NOUDPMAIN
  if(isudp) {
 	if(!udp_table.ihashtable)inithashtable(&udp_table, 64, 256, 65536);
@@ -638,19 +703,19 @@ int MODULEMAINFUNC (int argc, char** argv){
 		 case 'o':
 			switch(argv[i][2]){
 			 case 's':
-				srv.srvsockopts = getopts(argv[i]+3);
+				error = getopts(argv[i]+3, &srv.srvsockopts);
 				break;
 			 case 'c':
-				srv.clisockopts = getopts(argv[i]+3);
+				error = getopts(argv[i]+3, &srv.clisockopts);
 				break;
 			 case 'l':
-				srv.lissockopts = getopts(argv[i]+3);
+				error = getopts(argv[i]+3, &srv.lissockopts);
 				break;
 			 case 'r':
-				srv.cbcsockopts = getopts(argv[i]+3);
+				error = getopts(argv[i]+3, &srv.cbcsockopts);
 				break;
 			 case 'R':
-				srv.cbcsockopts = getopts(argv[i]+3);
+				error = getopts(argv[i]+3, &srv.cbssockopts);
 				break;
 			 default:
 				error = 1;
@@ -829,42 +894,13 @@ int MODULEMAINFUNC (int argc, char** argv){
 			perror("socket()");
 			return -2;
 		}
-		setopts(sock, srv.lissockopts);
+		setopts(sock, &srv.lissockopts);
 #ifdef _WIN32
 		ioctlsocket(sock, FIONBIO, &ul);
 #else
 		fcntl(sock,F_SETFL,O_NONBLOCK | fcntl(sock,F_GETFL));
 #endif
 		srv.srvsock = sock;
-#ifdef WITH_UN
-		if(*SAFAMILY(&srv.intsa) != AF_UNIX)
-#endif
-		{
-/* SO_REUSEADDR is not set on Windows: it is not needed to rebind a listening
-   port there, and it only allows another local process to bind the same
-   address and port, with undefined behaviour as to which socket receives the
-   connections. Use -olSO_EXCLUSIVEADDRUSE to prevent that instead.
-
-   A Windows build which does not share the listening socket is the exception:
-   its UDP services bind a second socket to the same address to answer from,
-   and Windows only allows that when both sockets ask for it.
- */
-#ifndef _WIN32
-		opt = 1;
-		if(srv.so._setsockopt(srv.so.state, sock, SOL_SOCKET, SO_REUSEADDR, (char *)&opt, sizeof(int)))perror("setsockopt()");
-#else
-#ifndef SHARE_UDP_SOCKET
-		if(isudp){
-			opt = 1;
-			if(srv.so._setsockopt(srv.so.state, sock, SOL_SOCKET, SO_REUSEADDR, (char *)&opt, sizeof(int)))perror("setsockopt()");
-		}
-#endif
-#endif
-#ifdef SO_REUSEPORT
-		opt = 1;
-		srv.so._setsockopt(srv.so.state, sock, SOL_SOCKET, SO_REUSEPORT, (char *)&opt, sizeof(int));
-#endif
-		}
 #if defined SO_BINDTODEVICE || defined IP_BOUND_IF
 		if(bindtodevice(&srv.so, srv.so.state, sock, srv.ibindtodevice, *SAFAMILY(&srv.intsa))) {
 		    dolog(&defparam, (unsigned char *)"failed to bind device");
@@ -957,16 +993,7 @@ int MODULEMAINFUNC (int argc, char** argv){
 		freesrvstrings(&srv, cbc_string, cbl_string);
 		return -6;
 	}
-#ifndef _WIN32
+	setopts(srv.cbsock, &srv.cbssockopts);
-	opt = 1;
-	srv.so._setsockopt(srv.so.state, srv.cbsock, SOL_SOCKET, SO_REUSEADDR, (char *)&opt, sizeof(int));
-#endif
-#ifdef SO_REUSEPORT
-	opt = 1;
-	srv.so._setsockopt(srv.so.state, srv.cbsock, SOL_SOCKET, SO_REUSEPORT, (char *)&opt, sizeof(int));
-#endif
-
-	setopts(srv.cbsock, srv.cbssockopts);
 
 	if(srv.so._bind(srv.so.state, srv.cbsock, (struct sockaddr*)&cbsa, SASIZE(&cbsa))==-1) {
 		dolog(&defparam, (unsigned char *)"Failed to bind connect back socket");
@@ -1025,7 +1052,7 @@ int MODULEMAINFUNC (int argc, char** argv){
 		if(iscbc){
 			new_sock=so._socket(so.state, SASOCK(&defparam.sincr), SOCK_STREAM, IPPROTO_TCP);
 			if(new_sock != INVALID_SOCKET){
-				setopts(new_sock, srv.cbcsockopts);
+				setopts(new_sock, &srv.cbcsockopts);
 
 				parsehost(srv.family, cbc_string, (struct sockaddr *)&defparam.sincr);
 #if defined SO_BINDTODEVICE || defined IP_BOUND_IF
@@ -1102,7 +1129,7 @@ int MODULEMAINFUNC (int argc, char** argv){
 				}
 				continue;
 			}
-			setopts(new_sock, srv.clisockopts);
+			setopts(new_sock, &srv.clisockopts);
 		}
 		size = sizeof(defparam.sincl);
 		if(srv.so._getsockname(srv.so.state, new_sock, (struct sockaddr *)&defparam.sincl, &size)){

src/socks.c

@@ -184,7 +184,7 @@ void * sockschild(struct clientparam* param) {
 	 param->operation = command == 2?BIND:UDPASSOC;
 	 if(command == 2){
 		if ((param->remsock=param->srv->so._socket(param->sostate, SASOCK(&param->req), SOCK_STREAM, IPPROTO_TCP)) == INVALID_SOCKET) {RETURN (11);}
-		setopts(param->remsock, param->srv->lissockopts);
+		setopts(param->remsock, &param->srv->lissockopts);
 		if(extbindtodevice(param, param->remsock)) {RETURN (12);}
 	 }
 	 break;

src/structures.h

@@ -529,6 +529,14 @@ struct filterp {
 
 #define MAX_FILTERS 16
 
+/* Socket options for one kind of socket, -oc and the like: which of
+   sockopts[] are given, by their index there, and the value of each. */
+#define MAXSOCKOPTS 32
+struct sockoptset {
+	unsigned set;
+	int value[MAXSOCKOPTS];
+};
+
 
 struct sockfuncs {
 	void * state;
@@ -613,7 +621,7 @@ struct srvparam {
 	int stacksize;
 	int noforce;
 	int anonymous;
-	int clisockopts, srvsockopts, lissockopts, cbcsockopts, cbssockopts;
+	struct sockoptset clisockopts, srvsockopts, lissockopts, cbcsockopts, cbssockopts;
 	int gracetraf, gracenum, gracedelay;
 	int requirecert;
 	int haproxy;