mirror of
https://github.com/3proxy/3proxy.git
synced 2026-09-17 11:35:50 +08:00
parent: support fraction and % syntaxis for weight; support fallbacks with 0 weight
This commit is contained in:
parent
6c4663a045
commit
41a08c11be
@ -870,9 +870,35 @@ build proxy chain. Proxies may be grouped. Proxy inside the
|
|||||||
group is selected randomly. If few groups are specified one proxy
|
group is selected randomly. If few groups are specified one proxy
|
||||||
is randomly picked from each group and chain of proxies is created
|
is randomly picked from each group and chain of proxies is created
|
||||||
(that is second proxy connected through first one and so on).
|
(that is second proxy connected through first one and so on).
|
||||||
Weight is used to group proxies. Weight is a number between 1 and 1000.
|
Weight is used to group proxies. A weight is a share of the whole, written
|
||||||
|
either as a fraction of one, anything beginning with 0 or with a point, or the
|
||||||
|
old way, in thousandths:
|
||||||
|
.br
|
||||||
|
\fB.5\fR and \fB0.5\fR and \fB500\fR are all a half
|
||||||
|
.br
|
||||||
|
\fB.333\fR and \fB333\fR are both 333 thousandths
|
||||||
|
.br
|
||||||
|
\fB1000\fR and \fB1.0\fR and \fB100%\fR are all the whole share
|
||||||
|
.br
|
||||||
|
\fB50.5%\fR and \fB.505\fR and \fB505\fR are all the same share
|
||||||
|
.br
|
||||||
|
\fB0\fR is a fallback, described below
|
||||||
|
.br
|
||||||
|
A fraction takes up to nine digits after the point, \fB.333333333\fR being
|
||||||
|
the finest share there is, and the old notation may now carry further digits
|
||||||
|
after a point of its own, so \fB123.456\fR means the same as \fB.123456\fR.
|
||||||
|
Weights are scanned as integers, nothing is read as a floating point number.
|
||||||
|
\fB1.0\fR, with as many zeroes after the point as you care to write, is the
|
||||||
|
one weight read as it looks rather than in thousandths, so that the whole
|
||||||
|
share can be written as a fraction too: a bare \fB1\fR is still a thousandth
|
||||||
|
of it, and \fB1.5\fR still one and a half of them. A weight ending in
|
||||||
|
\fB%\fR is a percentage, and takes up to seven digits after the point.
|
||||||
|
.br
|
||||||
Weights are summed and proxies are grouped together until the weight of
|
Weights are summed and proxies are grouped together until the weight of
|
||||||
the group is 1000. That is:
|
the group is the whole share. A group which falls short of it by no more than
|
||||||
|
a thousandth, which three weights of \fB333\fR do, is taken for a whole one
|
||||||
|
rather than for a group with a remainder, so a share which cannot be divided
|
||||||
|
evenly needs no adjusting by hand. That is:
|
||||||
.br
|
.br
|
||||||
allow *
|
allow *
|
||||||
.br
|
.br
|
||||||
@ -881,7 +907,8 @@ the group is 1000. That is:
|
|||||||
parent 500 connect 192.168.10.1 3128
|
parent 500 connect 192.168.10.1 3128
|
||||||
.br
|
.br
|
||||||
makes 3proxy to randomly choose between 2 proxies for all outgoing
|
makes 3proxy to randomly choose between 2 proxies for all outgoing
|
||||||
connections. These 2 proxies form 1 group (summarized weight is 1000).
|
connections. These 2 proxies form 1 group (their weights are the whole share
|
||||||
|
between them).
|
||||||
.br
|
.br
|
||||||
allow * * * 80
|
allow * * * 80
|
||||||
.br
|
.br
|
||||||
@ -1014,12 +1041,37 @@ local HTTP proxy parses requests and allows only GET and POST requests.
|
|||||||
.br
|
.br
|
||||||
Optional username and password are used to authenticate on parent
|
Optional username and password are used to authenticate on parent
|
||||||
proxy. Username of \'*\' means username must be supplied by user.
|
proxy. Username of \'*\' means username must be supplied by user.
|
||||||
|
.br
|
||||||
|
A parent which fails is taken out of the choice for the rest of that
|
||||||
|
connection, so the next attempt, see \fBparentretries\fR, goes to another
|
||||||
|
member of the group instead of the same parent again. Its share is spread over
|
||||||
|
the parents of the group which are left, in proportion to their weights.
|
||||||
|
.br
|
||||||
|
Weight 0 marks a fallback parent. Such a parent takes no share of the random
|
||||||
|
choice, and none of the share left over by a parent which failed, and is only
|
||||||
|
used once every weighted parent of its group has failed,
|
||||||
|
which is how a parent used only when another one is down is configured:
|
||||||
|
.br
|
||||||
|
allow *
|
||||||
|
.br
|
||||||
|
parent 1000 socks5 192.168.10.1 1080
|
||||||
|
.br
|
||||||
|
parent 0 socks5 192.168.20.1 1080
|
||||||
|
.br
|
||||||
|
Several fallbacks are tried in the order they are written. Reaching a fallback
|
||||||
|
costs an attempt, so \fBparentretries\fR has to be at least as large as the
|
||||||
|
number of parents to try.
|
||||||
|
.br
|
||||||
|
When every parent of a group has failed the request fails as well, rather
|
||||||
|
than being sent without a parent.
|
||||||
|
|
||||||
.br
|
.br
|
||||||
.BR parentretries
|
.BR parentretries
|
||||||
\fI<number>\fR
|
\fI<number>\fR
|
||||||
.br
|
.br
|
||||||
Number of retries to connect to parent proxy. Default is 1.
|
Number of attempts to reach a parent proxy. Default is 2. Each attempt
|
||||||
|
picks a parent again, leaving out the ones which already failed, so this is
|
||||||
|
also the number of different parents a request may be tried through.
|
||||||
|
|
||||||
|
|
||||||
.br
|
.br
|
||||||
|
|||||||
85
src/conf.c
85
src/conf.c
@ -903,6 +903,82 @@ static int parserange(unsigned char *arg, uint32_t *range)
|
|||||||
return 0;
|
return 0;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/* Scan a parent weight into its share of WEIGHTSCALE, as an integer: there is
|
||||||
|
* no floating point anywhere near a configuration file.
|
||||||
|
*
|
||||||
|
* A weight starting with 0 or . is a fraction of one, so .333 and 0.333 are
|
||||||
|
* both a third. Anything else is the old notation, thousandths, where 1000 is
|
||||||
|
* the whole share, and it may now carry more digits after a dot: 123.456 means
|
||||||
|
* the same as .123456. Either way at most 9 digits are kept, which is the
|
||||||
|
* resolution weights are held at.
|
||||||
|
*
|
||||||
|
* 1 followed by a point and nothing but zeroes is the one weight read as it
|
||||||
|
* looks rather than as thousandths: 1.0 is the whole share, where a bare 1 is
|
||||||
|
* a thousandth of it.
|
||||||
|
*
|
||||||
|
* A weight may also be written as a percentage, which is what a trailing %
|
||||||
|
* makes it: 50.5% is .505 is 505.
|
||||||
|
*/
|
||||||
|
static int parseweight(unsigned char * s, unsigned * weight){
|
||||||
|
static const unsigned pow10[10] = {1, 10, 100, 1000, 10000, 100000,
|
||||||
|
1000000, 10000000, 100000000, 1000000000};
|
||||||
|
unsigned char *p, *end;
|
||||||
|
uint64_t val = 0, res;
|
||||||
|
int ndigits = 0, atpoint = -1, after, percent = 0, fraction;
|
||||||
|
|
||||||
|
if(!s || !*s) return 1;
|
||||||
|
end = s + strlen((char *)s);
|
||||||
|
if(end[-1] == '%'){
|
||||||
|
percent = 1;
|
||||||
|
if(--end == s) return 1;
|
||||||
|
}
|
||||||
|
/* 1.0, with as many zeroes after it as anyone cares to write, is the one
|
||||||
|
weight read as it looks rather than as thousandths: the whole share,
|
||||||
|
where a bare 1 is a thousandth of it */
|
||||||
|
if(!percent && s[0] == '1' && s[1] == '.' && s[2]){
|
||||||
|
for(p = s + 2; *p == '0'; p++);
|
||||||
|
if(!*p){
|
||||||
|
*weight = WEIGHTSCALE;
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
fraction = (*s == '.' || *s == '0');
|
||||||
|
for(p = s; p < end; p++){
|
||||||
|
if(*p == '.'){
|
||||||
|
if(atpoint >= 0) return 1;
|
||||||
|
atpoint = ndigits;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if(*p < '0' || *p > '9') return 1;
|
||||||
|
if(ndigits == 18) return 1;
|
||||||
|
val = (val * 10) + (unsigned)(*p - '0');
|
||||||
|
ndigits++;
|
||||||
|
}
|
||||||
|
if(!ndigits || val > WEIGHTSCALE) return 1;
|
||||||
|
after = (atpoint < 0)? 0 : ndigits - atpoint;
|
||||||
|
if(percent){
|
||||||
|
/* a hundredth of the whole share for every 1% */
|
||||||
|
if(after > 7) return 1;
|
||||||
|
res = val * pow10[7 - after];
|
||||||
|
}
|
||||||
|
else if(fraction){
|
||||||
|
/* the digits before the point are the leading zero and add
|
||||||
|
nothing, so only the ones after it say what the share is */
|
||||||
|
if(atpoint < 0) return val? 1 : (*weight = 0, 0);
|
||||||
|
if(after > 9) return 1;
|
||||||
|
res = val * pow10[9 - after];
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
/* thousandths, with the digits after the point carrying on
|
||||||
|
from them: three digits of a whole share, six more after */
|
||||||
|
if(after > 6) return 1;
|
||||||
|
res = val * pow10[6 - after];
|
||||||
|
}
|
||||||
|
if(res > WEIGHTSCALE) return 1;
|
||||||
|
*weight = (unsigned)res;
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
|
||||||
static int h_parent(int argc, unsigned char **argv){
|
static int h_parent(int argc, unsigned char **argv){
|
||||||
struct ace *acl = NULL;
|
struct ace *acl = NULL;
|
||||||
struct chain *chains;
|
struct chain *chains;
|
||||||
@ -922,9 +998,10 @@ static int h_parent(int argc, unsigned char **argv){
|
|||||||
return(21);
|
return(21);
|
||||||
}
|
}
|
||||||
memset(chains, 0, sizeof(struct chain));
|
memset(chains, 0, sizeof(struct chain));
|
||||||
chains->weight = (unsigned)atoi((char *)argv[1]);
|
/* 0 is the fallback weight: such a parent is only used once every
|
||||||
if(chains->weight == 0 || chains->weight >1000) {
|
weighted parent of its group has failed */
|
||||||
fprintf(stderr, "Chaining error: bad chain weight %u line %d\n", chains->weight, linenum);
|
if(parseweight(argv[1], &chains->weight)) {
|
||||||
|
fprintf(stderr, "Chaining error: bad chain weight %s line %d\n", argv[1], linenum);
|
||||||
free(chains);
|
free(chains);
|
||||||
return(3);
|
return(3);
|
||||||
}
|
}
|
||||||
@ -1497,7 +1574,7 @@ static int h_ace(int argc, unsigned char **argv){
|
|||||||
return 5;
|
return 5;
|
||||||
}
|
}
|
||||||
*SAPORT(&acl->chains->addr) = htons((uint16_t)atoi((char *)argv[2]));
|
*SAPORT(&acl->chains->addr) = htons((uint16_t)atoi((char *)argv[2]));
|
||||||
acl->chains->weight = 1000;
|
acl->chains->weight = WEIGHTSCALE;
|
||||||
case ALLOW:
|
case ALLOW:
|
||||||
case DENY:
|
case DENY:
|
||||||
if(!conf.acl){
|
if(!conf.acl){
|
||||||
|
|||||||
@ -802,7 +802,7 @@ static struct property prop_pwlist[] = {
|
|||||||
static struct property prop_chain[] = {
|
static struct property prop_chain[] = {
|
||||||
{"addr", ef_chain_addr, TYPE_SA, "parent address"},
|
{"addr", ef_chain_addr, TYPE_SA, "parent address"},
|
||||||
{"type", ef_chain_type, TYPE_STRING, "parent type"},
|
{"type", ef_chain_type, TYPE_STRING, "parent type"},
|
||||||
{"weight", ef_chain_weight, TYPE_SHORT, "parent weight 0-1000"},
|
{"weight", ef_chain_weight, TYPE_INTEGER, "parent weight, 1000000000 is the whole share, 0 is a fallback"},
|
||||||
{"user", ef_chain_user, TYPE_STRING, "parent login"},
|
{"user", ef_chain_user, TYPE_STRING, "parent login"},
|
||||||
{"password", ef_chain_password, TYPE_PASSWORD, "parent password"},
|
{"password", ef_chain_password, TYPE_PASSWORD, "parent password"},
|
||||||
{"secure", ef_chain_secure, TYPE_INTEGER, "secure mode"},
|
{"secure", ef_chain_secure, TYPE_INTEGER, "secure mode"},
|
||||||
|
|||||||
111
src/redirect.c
111
src/redirect.c
@ -289,45 +289,105 @@ static void chainaddr(struct chain * cur, PROXYSOCKADDRTYPE * sa){
|
|||||||
*sa = fresh;
|
*sa = fresh;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
static int parentfailed(struct clientparam * param, struct chain * ch){
|
||||||
|
int i;
|
||||||
|
|
||||||
|
for(i = 0; i < param->nfailedparents; i++)
|
||||||
|
if(param->failedparents[i] == ch) return 1;
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Remember a parent which could not be used for this connection, so that a
|
||||||
|
* retry picks another member of its group. The list is per connection: a
|
||||||
|
* parent which is down for one client is not taken away from the others.
|
||||||
|
*/
|
||||||
|
static void parentfail(struct clientparam * param, struct chain * ch){
|
||||||
|
if(!ch || param->nfailedparents >= MAXFAILEDPARENTS) return;
|
||||||
|
if(parentfailed(param, ch)) return;
|
||||||
|
param->failedparents[param->nfailedparents++] = ch;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Pick the parent to use for one group.
|
||||||
|
*
|
||||||
|
* A group is the members whose weights add up to WEIGHTSCALE, together with
|
||||||
|
* the zero weight members among them. *after is left pointing at the group
|
||||||
|
* after this one, or at NULL. A group which lands within WEIGHTFUZZ of the
|
||||||
|
* whole share counts as a whole one, so that 333 three times over is a group
|
||||||
|
* rather than a group and a remainder of a thousandth.
|
||||||
|
*
|
||||||
|
* A member which already failed for this connection is not offered again and
|
||||||
|
* its weight is given to the others, so a retry goes somewhere else. Zero
|
||||||
|
* weight members are the fallback: they are only offered once every weighted
|
||||||
|
* member of the group has failed, and they never take part in the share.
|
||||||
|
* Where the weights add up to plainly less than the whole share the remainder
|
||||||
|
* keeps its meaning of "no parent at all" and is still counted, so such a
|
||||||
|
* group can still leave the connection direct.
|
||||||
|
*
|
||||||
|
* Returns NULL when the group adds no parent. *exhausted tells the two cases
|
||||||
|
* apart: it is set when the group had parents and all of them are gone, which
|
||||||
|
* is a failure rather than a reason to connect directly.
|
||||||
|
*/
|
||||||
|
static struct chain * pickchain(struct clientparam * param, struct chain * group,
|
||||||
|
struct chain ** after, int * exhausted){
|
||||||
|
struct chain *cur;
|
||||||
|
uint64_t total = 0, avail = 0, slack;
|
||||||
|
uint64_t r;
|
||||||
|
|
||||||
|
*exhausted = 0;
|
||||||
|
for(cur = group; cur; cur = cur->next){
|
||||||
|
if(total + WEIGHTFUZZ >= WEIGHTSCALE && cur->weight) break;
|
||||||
|
total += cur->weight;
|
||||||
|
if(cur->weight && !parentfailed(param, cur)) avail += cur->weight;
|
||||||
|
}
|
||||||
|
*after = cur;
|
||||||
|
|
||||||
|
if(avail){
|
||||||
|
slack = (total + WEIGHTFUZZ < WEIGHTSCALE)? WEIGHTSCALE - total : 0;
|
||||||
|
r = ((uint64_t)myrand() << 32 | myrand()) % (avail + slack);
|
||||||
|
for(cur = group; cur != *after; cur = cur->next){
|
||||||
|
if(!cur->weight || parentfailed(param, cur)) continue;
|
||||||
|
if(r < cur->weight) return cur;
|
||||||
|
r -= cur->weight;
|
||||||
|
}
|
||||||
|
return NULL;
|
||||||
|
}
|
||||||
|
for(cur = group; cur != *after; cur = cur->next){
|
||||||
|
if(!cur->weight && !parentfailed(param, cur)) return cur;
|
||||||
|
}
|
||||||
|
if(total) *exhausted = 1;
|
||||||
|
return NULL;
|
||||||
|
}
|
||||||
|
|
||||||
int handleredirect(struct clientparam * param, struct ace * acentry){
|
int handleredirect(struct clientparam * param, struct ace * acentry){
|
||||||
int connected = 0;
|
int connected = 0;
|
||||||
int weight = 1000;
|
|
||||||
int res;
|
int res;
|
||||||
int done = 0;
|
|
||||||
int ha = 0;
|
int ha = 0;
|
||||||
struct chain * cur;
|
struct chain * cur;
|
||||||
|
struct chain * after;
|
||||||
struct chain * redir = NULL;
|
struct chain * redir = NULL;
|
||||||
int r2;
|
|
||||||
int saved = 0;
|
int saved = 0;
|
||||||
|
|
||||||
if((SAISNULL(¶m->req) || !*SAPORT(¶m->req)) && param->operation != UDPASSOC) {
|
if((SAISNULL(¶m->req) || !*SAPORT(¶m->req)) && param->operation != UDPASSOC) {
|
||||||
return 100;
|
return 100;
|
||||||
}
|
}
|
||||||
|
|
||||||
r2 = (myrand()%1000);
|
for(cur = acentry->chains; cur; cur = after){
|
||||||
|
struct chain * sel;
|
||||||
|
int exhausted;
|
||||||
|
|
||||||
for(cur = acentry->chains; cur; cur=cur->next){
|
sel = pickchain(param, cur, &after, &exhausted);
|
||||||
if(((weight = weight - cur->weight) > r2)|| done) {
|
/* every parent of the group is gone: connecting direct instead
|
||||||
if(weight <= 0) {
|
would be a way around the rule, so the request fails */
|
||||||
weight += 1000;
|
if(exhausted) return 13;
|
||||||
done = 0;
|
if(!sel) continue;
|
||||||
r2 = (myrand()%1000);
|
cur = sel;
|
||||||
}
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
if(cur->type != R_EXTIP && cur->type != R_HA &&
|
if(cur->type != R_EXTIP && cur->type != R_HA &&
|
||||||
cur->type != R_EXTPORT && cur->type != R_INTPORT) param->redirected++;
|
cur->type != R_EXTPORT && cur->type != R_INTPORT) param->redirected++;
|
||||||
done = 1;
|
|
||||||
if(weight <= 0) {
|
|
||||||
weight += 1000;
|
|
||||||
done = 0;
|
|
||||||
r2 = (myrand()%1000);
|
|
||||||
}
|
|
||||||
if(!connected){
|
if(!connected){
|
||||||
if(cur->type == R_EXTPORT || cur->type == R_INTPORT){
|
if(cur->type == R_EXTPORT || cur->type == R_INTPORT){
|
||||||
if(cur->type == R_EXTPORT) param->extport = cur->range;
|
if(cur->type == R_EXTPORT) param->extport = cur->range;
|
||||||
else param->intport = cur->range;
|
else param->intport = cur->range;
|
||||||
if(cur->next)continue;
|
if(after)continue;
|
||||||
return 0;
|
return 0;
|
||||||
}
|
}
|
||||||
if(cur->type == R_EXTIP){
|
if(cur->type == R_EXTIP){
|
||||||
@ -349,7 +409,7 @@ int handleredirect(struct clientparam * param, struct ace * acentry){
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
#endif
|
#endif
|
||||||
if(cur->next)continue;
|
if(after)continue;
|
||||||
return 0;
|
return 0;
|
||||||
}
|
}
|
||||||
else if(SAISNULL(&cur->addr) && !*SAPORT(&cur->addr)){
|
else if(SAISNULL(&cur->addr) && !*SAPORT(&cur->addr)){
|
||||||
@ -375,7 +435,7 @@ int handleredirect(struct clientparam * param, struct ace * acentry){
|
|||||||
if(cur->type == R_HA){
|
if(cur->type == R_HA){
|
||||||
ha = 1;
|
ha = 1;
|
||||||
}
|
}
|
||||||
if(cur->next)continue;
|
if(after)continue;
|
||||||
if(!ha) return 0;
|
if(!ha) return 0;
|
||||||
if(param->operation == UDPASSOC) return 0;
|
if(param->operation == UDPASSOC) return 0;
|
||||||
}
|
}
|
||||||
@ -397,6 +457,7 @@ int handleredirect(struct clientparam * param, struct ace * acentry){
|
|||||||
saved = 1;
|
saved = 1;
|
||||||
}
|
}
|
||||||
if((res = alwaysauth(param))){
|
if((res = alwaysauth(param))){
|
||||||
|
parentfail(param, cur);
|
||||||
return (res >= 10)? res : 60+res;
|
return (res >= 10)? res : 60+res;
|
||||||
}
|
}
|
||||||
if(ha) {
|
if(ha) {
|
||||||
@ -420,7 +481,10 @@ int handleredirect(struct clientparam * param, struct ace * acentry){
|
|||||||
|
|
||||||
chainaddr(cur, &next);
|
chainaddr(cur, &next);
|
||||||
res = (redir)?clientnegotiate(redir, param, (struct sockaddr *)&next, cur->exthost):0;
|
res = (redir)?clientnegotiate(redir, param, (struct sockaddr *)&next, cur->exthost):0;
|
||||||
if(res) return res;
|
if(res) {
|
||||||
|
parentfail(param, cur);
|
||||||
|
return res;
|
||||||
|
}
|
||||||
}
|
}
|
||||||
redir = cur;
|
redir = cur;
|
||||||
param->redirtype = redir->type;
|
param->redirtype = redir->type;
|
||||||
@ -444,6 +508,7 @@ int handleredirect(struct clientparam * param, struct ace * acentry){
|
|||||||
|
|
||||||
if(!connected || !redir) return 0;
|
if(!connected || !redir) return 0;
|
||||||
res = clientnegotiate(redir, param, (struct sockaddr *)¶m->req, param->hostname);
|
res = clientnegotiate(redir, param, (struct sockaddr *)¶m->req, param->hostname);
|
||||||
|
if(res) parentfail(param, redir);
|
||||||
if(saved){
|
if(saved){
|
||||||
SOCKET s;
|
SOCKET s;
|
||||||
|
|
||||||
|
|||||||
@ -62,6 +62,13 @@ typedef struct _3proxy_sem_s {
|
|||||||
#endif
|
#endif
|
||||||
#endif
|
#endif
|
||||||
#define MAXBANDLIMS 10
|
#define MAXBANDLIMS 10
|
||||||
|
#define MAXFAILEDPARENTS 16
|
||||||
|
/* Parent weights are kept as a share of WEIGHTSCALE, which is what the
|
||||||
|
weights of a group add up to. WEIGHTFUZZ is how far short of it a group may
|
||||||
|
fall and still be taken for a whole one, a thousandth of the share: three
|
||||||
|
weights of 333, or of .333333333, otherwise leave a remainder. */
|
||||||
|
#define WEIGHTSCALE 1000000000u
|
||||||
|
#define WEIGHTFUZZ 1000000u
|
||||||
|
|
||||||
#ifdef WITH_POLL
|
#ifdef WITH_POLL
|
||||||
#include <poll.h>
|
#include <poll.h>
|
||||||
@ -336,7 +343,7 @@ struct chain {
|
|||||||
unsigned char * exthost;
|
unsigned char * exthost;
|
||||||
unsigned char * extuser;
|
unsigned char * extuser;
|
||||||
unsigned char * extpass;
|
unsigned char * extpass;
|
||||||
unsigned short weight;
|
unsigned weight;
|
||||||
unsigned short cidr;
|
unsigned short cidr;
|
||||||
/* local port range for extport/intport, first in the low half */
|
/* local port range for extport/intport, first in the low half */
|
||||||
uint32_t range;
|
uint32_t range;
|
||||||
@ -763,6 +770,11 @@ struct clientparam {
|
|||||||
one connection rather than for the service, which is how a redirect
|
one connection rather than for the service, which is how a redirect
|
||||||
and a service name standing in for a mail proxy reach tlspr. */
|
and a service name standing in for a mail proxy reach tlspr. */
|
||||||
PROXYSERVICE starttls;
|
PROXYSERVICE starttls;
|
||||||
|
/* Parents which failed for this connection. A retry picks another
|
||||||
|
member of the group instead of the same one again, and a zero weight
|
||||||
|
member is only reached once every weighted one is in here. */
|
||||||
|
struct chain *failedparents[MAXFAILEDPARENTS];
|
||||||
|
int nfailedparents;
|
||||||
};
|
};
|
||||||
|
|
||||||
struct filemon {
|
struct filemon {
|
||||||
|
|||||||
160
tests/cases/parent_failover.py
Normal file
160
tests/cases/parent_failover.py
Normal file
@ -0,0 +1,160 @@
|
|||||||
|
"""parent: what happens to a group when one of its members is down.
|
||||||
|
|
||||||
|
A parent which fails is taken out of the random choice for the rest of the
|
||||||
|
connection, so a retry reaches another member of the group instead of the same
|
||||||
|
dead one again. A parent of weight 0 is the fallback of its group: it is only
|
||||||
|
used once every weighted member has failed.
|
||||||
|
|
||||||
|
The number of attempts is bounded by parentretries, two by default, so each
|
||||||
|
case here needs at most one parent to fail before a working one is reached.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import time
|
||||||
|
|
||||||
|
|
||||||
|
def served(server, needle, since=0):
|
||||||
|
"""How many log lines carrying needle a proxy wrote past offset since."""
|
||||||
|
return sum(1 for line in server.output()[since:].splitlines() if needle in line)
|
||||||
|
|
||||||
|
|
||||||
|
def wait_served(server, needle, count, since=0, timeout=5.0):
|
||||||
|
"""Wait for count such lines: a session is logged once it is over, which
|
||||||
|
is a moment after the client has its answer."""
|
||||||
|
deadline = time.time() + timeout
|
||||||
|
while time.time() < deadline:
|
||||||
|
seen = served(server, needle, since)
|
||||||
|
if seen >= count:
|
||||||
|
return seen
|
||||||
|
time.sleep(0.05)
|
||||||
|
return served(server, needle, since)
|
||||||
|
|
||||||
|
|
||||||
|
def run(t):
|
||||||
|
srv = t.free_port()
|
||||||
|
good = t.free_port()
|
||||||
|
spare = t.free_port()
|
||||||
|
# nothing is ever started here, so connecting to it is refused at once
|
||||||
|
dead = t.free_port()
|
||||||
|
|
||||||
|
origin = t.start("failover_origin", f"""
|
||||||
|
log
|
||||||
|
auth iponly
|
||||||
|
allow *
|
||||||
|
http echo * /echo**
|
||||||
|
httpsrv -p{srv}
|
||||||
|
""", ports=[srv])
|
||||||
|
|
||||||
|
goodp = t.start("failover_good", f"""
|
||||||
|
log
|
||||||
|
auth iponly
|
||||||
|
allow *
|
||||||
|
proxy -p{good}
|
||||||
|
""", ports=[good])
|
||||||
|
|
||||||
|
sparep = t.start("failover_spare", f"""
|
||||||
|
log
|
||||||
|
auth iponly
|
||||||
|
allow *
|
||||||
|
proxy -p{spare}
|
||||||
|
""", ports=[spare])
|
||||||
|
|
||||||
|
fallback = t.free_port()
|
||||||
|
idle = t.free_port()
|
||||||
|
group = t.free_port()
|
||||||
|
allgone = t.free_port()
|
||||||
|
lone = t.free_port()
|
||||||
|
|
||||||
|
t.start("failover_client", f"""
|
||||||
|
log
|
||||||
|
auth iponly
|
||||||
|
|
||||||
|
# the only weighted parent is dead, the fallback has to take over
|
||||||
|
flush
|
||||||
|
allow *
|
||||||
|
parent 1000 connect 127.0.0.1 {dead}
|
||||||
|
parent 0 connect 127.0.0.1 {spare}
|
||||||
|
proxy -p{fallback}
|
||||||
|
|
||||||
|
# the weighted parent works, so the fallback stays untouched
|
||||||
|
flush
|
||||||
|
allow *
|
||||||
|
parent 1000 connect 127.0.0.1 {good}
|
||||||
|
parent 0 connect 127.0.0.1 {spare}
|
||||||
|
proxy -p{idle}
|
||||||
|
|
||||||
|
# one member of a group of two is dead: a retry must not pick it again
|
||||||
|
flush
|
||||||
|
allow *
|
||||||
|
parent 500 connect 127.0.0.1 {dead}
|
||||||
|
parent 500 connect 127.0.0.1 {good}
|
||||||
|
proxy -p{group}
|
||||||
|
|
||||||
|
# nothing left to fall back to
|
||||||
|
flush
|
||||||
|
allow *
|
||||||
|
parent 1000 connect 127.0.0.1 {dead}
|
||||||
|
proxy -p{allgone}
|
||||||
|
|
||||||
|
# a parent of weight 0 on its own is simply the parent to use
|
||||||
|
flush
|
||||||
|
allow *
|
||||||
|
parent 0 connect 127.0.0.1 {good}
|
||||||
|
proxy -p{lone}
|
||||||
|
""", ports=[fallback, idle, group, allgone, lone])
|
||||||
|
|
||||||
|
url = f"http://127.0.0.1:{srv}/echo"
|
||||||
|
needle = f"CONNECT 127.0.0.1:{srv}"
|
||||||
|
|
||||||
|
# --- the fallback takes over --------------------------------------------
|
||||||
|
mark = len(sparep.output())
|
||||||
|
r = t.http(url, proxy=f"127.0.0.1:{fallback}")
|
||||||
|
t.eq(200, r.status, "a dead weighted parent falls back to the parent of weight 0")
|
||||||
|
t.eq(1, wait_served(sparep, needle, 1, mark),
|
||||||
|
"the fallback parent carried the request")
|
||||||
|
|
||||||
|
# --- and only then ------------------------------------------------------
|
||||||
|
mark = len(sparep.output())
|
||||||
|
gmark = len(goodp.output())
|
||||||
|
for _ in range(4):
|
||||||
|
t.eq(200, t.http(url, proxy=f"127.0.0.1:{idle}").status,
|
||||||
|
"a working weighted parent serves the request")
|
||||||
|
t.eq(4, wait_served(goodp, needle, 4, gmark),
|
||||||
|
"every request went through the weighted parent")
|
||||||
|
t.eq(0, served(sparep, needle, mark),
|
||||||
|
"the fallback is left alone while the weighted parent works")
|
||||||
|
|
||||||
|
# --- a dead member of a weighted group ----------------------------------
|
||||||
|
# Whichever of the two the first attempt picks, the request has to end up
|
||||||
|
# at the one that is up: the dead one is not offered to the retry again.
|
||||||
|
gmark = len(goodp.output())
|
||||||
|
statuses = [t.http(url, proxy=f"127.0.0.1:{group}").status for _ in range(8)]
|
||||||
|
t.eq([200] * 8, statuses,
|
||||||
|
"a dead member of a group never fails a request twice over")
|
||||||
|
t.eq(8, wait_served(goodp, needle, 8, gmark),
|
||||||
|
"all of them were carried by the member which is up")
|
||||||
|
|
||||||
|
# --- nothing left -------------------------------------------------------
|
||||||
|
# With every parent of the group gone the request has to fail. Connecting
|
||||||
|
# direct instead would be a way around the rule that asked for a parent.
|
||||||
|
omark = len(origin.output())
|
||||||
|
r = t.http(url, proxy=f"127.0.0.1:{allgone}")
|
||||||
|
t.ne(200, r.status, "a request fails when every parent of the group is down")
|
||||||
|
time.sleep(0.5)
|
||||||
|
t.eq(0, served(origin, "/echo", omark),
|
||||||
|
"and it is not sent direct to the origin instead")
|
||||||
|
|
||||||
|
# --- weight 0 on its own ------------------------------------------------
|
||||||
|
gmark = len(goodp.output())
|
||||||
|
t.eq(200, t.http(url, proxy=f"127.0.0.1:{lone}").status,
|
||||||
|
"a parent of weight 0 alone is used like any other")
|
||||||
|
t.eq(1, wait_served(goodp, needle, 1, gmark),
|
||||||
|
"through the parent it names")
|
||||||
|
|
||||||
|
# --- what the parser still rejects --------------------------------------
|
||||||
|
out = t.run_config("failover_badweight", f"""
|
||||||
|
auth iponly
|
||||||
|
allow *
|
||||||
|
parent 1001 connect 127.0.0.1 {good}
|
||||||
|
proxy -p{t.free_port()}
|
||||||
|
""")
|
||||||
|
t.contains(out, "bad chain weight", "a weight above 1000 is still refused")
|
||||||
210
tests/cases/parent_weights.py
Normal file
210
tests/cases/parent_weights.py
Normal file
@ -0,0 +1,210 @@
|
|||||||
|
"""parent weights: the fraction notation, and what a group adds up to.
|
||||||
|
|
||||||
|
A weight is scanned as an integer into a share of 1000000000. A weight which
|
||||||
|
starts with 0 or . is a fraction of one, .333 being a third; anything else is
|
||||||
|
the old notation, thousandths, which may now carry further digits after a dot,
|
||||||
|
so 123.456 means the same as .123456. 1.0 is the exception, read as it looks,
|
||||||
|
and a trailing % makes a weight a percentage: 50.5% is .505 is 505.
|
||||||
|
|
||||||
|
The values are read back from the admin interface, which dumps the parsed
|
||||||
|
configuration, so what is checked is the number 3proxy holds rather than the
|
||||||
|
behaviour it happens to produce.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import re
|
||||||
|
import time
|
||||||
|
|
||||||
|
CHAIN_WEIGHT = re.compile(
|
||||||
|
r"parent weight[^<]*</description><value><!\[CDATA\[([0-9]+)")
|
||||||
|
|
||||||
|
|
||||||
|
def weights(t, adm):
|
||||||
|
return [int(v) for v in CHAIN_WEIGHT.findall(t.http(f"http://127.0.0.1:{adm}/S").text)]
|
||||||
|
|
||||||
|
|
||||||
|
def served(server, needle, since=0):
|
||||||
|
return sum(1 for line in server.output()[since:].splitlines() if needle in line)
|
||||||
|
|
||||||
|
|
||||||
|
def wait_served(server, needle, count, since=0, timeout=5.0):
|
||||||
|
deadline = time.time() + timeout
|
||||||
|
while time.time() < deadline:
|
||||||
|
seen = served(server, needle, since)
|
||||||
|
if seen >= count:
|
||||||
|
return seen
|
||||||
|
time.sleep(0.05)
|
||||||
|
return served(server, needle, since)
|
||||||
|
|
||||||
|
|
||||||
|
def run(t):
|
||||||
|
adm = t.free_port()
|
||||||
|
prx = t.free_port()
|
||||||
|
dummy = t.free_port()
|
||||||
|
|
||||||
|
t.start("weights_parse", f"""
|
||||||
|
auth iponly
|
||||||
|
allow *
|
||||||
|
admin -p{adm}
|
||||||
|
|
||||||
|
flush
|
||||||
|
auth iponly
|
||||||
|
allow *
|
||||||
|
parent 1000 connect 127.0.0.1 {dummy}
|
||||||
|
parent 1.0 connect 127.0.0.1 {dummy}
|
||||||
|
parent 1.00000000000000 connect 127.0.0.1 {dummy}
|
||||||
|
parent 500 connect 127.0.0.1 {dummy}
|
||||||
|
parent 1 connect 127.0.0.1 {dummy}
|
||||||
|
parent 1.5 connect 127.0.0.1 {dummy}
|
||||||
|
parent 123.456 connect 127.0.0.1 {dummy}
|
||||||
|
parent 12.34 connect 127.0.0.1 {dummy}
|
||||||
|
parent 1.000001 connect 127.0.0.1 {dummy}
|
||||||
|
parent .333 connect 127.0.0.1 {dummy}
|
||||||
|
parent 0.333 connect 127.0.0.1 {dummy}
|
||||||
|
parent .5 connect 127.0.0.1 {dummy}
|
||||||
|
parent .333333333 connect 127.0.0.1 {dummy}
|
||||||
|
parent 0.000000001 connect 127.0.0.1 {dummy}
|
||||||
|
parent 100% connect 127.0.0.1 {dummy}
|
||||||
|
parent 50.5% connect 127.0.0.1 {dummy}
|
||||||
|
parent 50% connect 127.0.0.1 {dummy}
|
||||||
|
parent 33.3333333% connect 127.0.0.1 {dummy}
|
||||||
|
parent 1.0% connect 127.0.0.1 {dummy}
|
||||||
|
parent 0.0000001% connect 127.0.0.1 {dummy}
|
||||||
|
parent 0 connect 127.0.0.1 {dummy}
|
||||||
|
proxy -p{prx}
|
||||||
|
""", ports=[adm, prx])
|
||||||
|
|
||||||
|
t.eq([
|
||||||
|
1000000000, # 1000, the old notation for the whole share
|
||||||
|
1000000000, # 1.0, the same share written as a fraction
|
||||||
|
1000000000, # 1.00000000000000, zeroes past the point change nothing
|
||||||
|
500000000, # 500
|
||||||
|
1000000, # 1, a thousandth
|
||||||
|
1500000, # 1.5, one thousandth and a half of one
|
||||||
|
123456000, # 123.456, the old notation carried further
|
||||||
|
12340000, # 12.34
|
||||||
|
1000001, # 1.000001, six digits past the thousandths
|
||||||
|
333000000, # .333
|
||||||
|
333000000, # 0.333, the same thing written out
|
||||||
|
500000000, # .5
|
||||||
|
333333333, # .333333333, the finest the resolution goes
|
||||||
|
1, # 0.000000001, one part of the whole
|
||||||
|
1000000000, # 100%
|
||||||
|
505000000, # 50.5%, the same as .505 and as 505
|
||||||
|
500000000, # 50%
|
||||||
|
333333333, # 33.3333333%, seven digits past the point
|
||||||
|
10000000, # 1.0%, a percent rather than the whole share
|
||||||
|
1, # 0.0000001%, one part again
|
||||||
|
0, # the fallback weight
|
||||||
|
], weights(t, adm), "every notation is scanned into its share")
|
||||||
|
|
||||||
|
# --- what the parser refuses ------------------------------------------
|
||||||
|
for bad in ("1001", "1000.1", "1.0000001", ".1234567890", "01", "abc",
|
||||||
|
"1.2.3", "-1", "1e9", "101%", "50.55555555%", "%", "5%%",
|
||||||
|
"%5"):
|
||||||
|
out = t.run_config("weights_bad", f"""
|
||||||
|
auth iponly
|
||||||
|
allow *
|
||||||
|
parent {bad} connect 127.0.0.1 {dummy}
|
||||||
|
proxy -p{t.free_port()}
|
||||||
|
""")
|
||||||
|
t.contains(out, "bad chain weight", f"{bad} is refused as a weight")
|
||||||
|
|
||||||
|
# --- a group that all but adds up ---------------------------------------
|
||||||
|
# .999999999 is one part short of the whole share. It still closes its
|
||||||
|
# group, so the parent after it is the next hop of a chain rather than
|
||||||
|
# another member of the same group.
|
||||||
|
srv = t.free_port()
|
||||||
|
first = t.free_port()
|
||||||
|
second = t.free_port()
|
||||||
|
chained = t.free_port()
|
||||||
|
grouped = t.free_port()
|
||||||
|
thirds = t.free_port()
|
||||||
|
|
||||||
|
t.start("weights_origin", f"""
|
||||||
|
log
|
||||||
|
auth iponly
|
||||||
|
allow *
|
||||||
|
http echo * /echo**
|
||||||
|
httpsrv -p{srv}
|
||||||
|
""", ports=[srv])
|
||||||
|
|
||||||
|
firstp = t.start("weights_first", f"""
|
||||||
|
log
|
||||||
|
auth iponly
|
||||||
|
allow *
|
||||||
|
proxy -p{first}
|
||||||
|
""", ports=[first])
|
||||||
|
|
||||||
|
secondp = t.start("weights_second", f"""
|
||||||
|
log
|
||||||
|
auth iponly
|
||||||
|
allow *
|
||||||
|
proxy -p{second}
|
||||||
|
""", ports=[second])
|
||||||
|
|
||||||
|
t.start("weights_client", f"""
|
||||||
|
log
|
||||||
|
auth iponly
|
||||||
|
|
||||||
|
# one part short of the whole share still ends the group
|
||||||
|
flush
|
||||||
|
allow *
|
||||||
|
parent .999999999 connect 127.0.0.1 {first}
|
||||||
|
parent 1000 connect 127.0.0.1 {second}
|
||||||
|
proxy -p{chained}
|
||||||
|
|
||||||
|
# two halves, one written each way, are one group and one hop
|
||||||
|
flush
|
||||||
|
allow *
|
||||||
|
parent 500 connect 127.0.0.1 {first}
|
||||||
|
parent .5 connect 127.0.0.1 {second}
|
||||||
|
proxy -p{grouped}
|
||||||
|
|
||||||
|
# three thirds are a thousandth short of the whole share and still
|
||||||
|
# make a group, so the parent after them is the next hop
|
||||||
|
flush
|
||||||
|
allow *
|
||||||
|
parent 333 connect 127.0.0.1 {first}
|
||||||
|
parent 333 connect 127.0.0.1 {first}
|
||||||
|
parent 333 connect 127.0.0.1 {first}
|
||||||
|
parent 1000 connect 127.0.0.1 {second}
|
||||||
|
proxy -p{thirds}
|
||||||
|
""", ports=[chained, grouped, thirds])
|
||||||
|
|
||||||
|
url = f"http://127.0.0.1:{srv}/echo"
|
||||||
|
toorigin = f"CONNECT 127.0.0.1:{srv}"
|
||||||
|
tosecond = f"CONNECT 127.0.0.1:{second}"
|
||||||
|
|
||||||
|
fmark, smark = len(firstp.output()), len(secondp.output())
|
||||||
|
t.eq(200, t.http(url, proxy=f"127.0.0.1:{chained}").status,
|
||||||
|
"a chain of two groups carries the request")
|
||||||
|
t.eq(1, wait_served(firstp, tosecond, 1, fmark),
|
||||||
|
"the first group's parent was asked for the second one")
|
||||||
|
t.eq(1, wait_served(secondp, toorigin, 1, smark),
|
||||||
|
"and the second group's parent reached the origin")
|
||||||
|
t.eq(0, served(firstp, toorigin, fmark),
|
||||||
|
"the first parent never went to the origin itself")
|
||||||
|
|
||||||
|
# both members of one group talk to the origin, never to each other
|
||||||
|
fmark, smark = len(firstp.output()), len(secondp.output())
|
||||||
|
for _ in range(8):
|
||||||
|
t.eq(200, t.http(url, proxy=f"127.0.0.1:{grouped}").status,
|
||||||
|
"a group of two halves carries the request")
|
||||||
|
t.eq(8, wait_served(firstp, toorigin, 8, fmark, timeout=0.5) +
|
||||||
|
wait_served(secondp, toorigin, 8, smark, timeout=0.5),
|
||||||
|
"each request took one hop, through either half")
|
||||||
|
t.eq(0, served(firstp, tosecond, fmark),
|
||||||
|
"the halves are one group, not a chain")
|
||||||
|
|
||||||
|
# --- three thirds -------------------------------------------------------
|
||||||
|
# 999000000 is within a thousandth of the whole share, so the group closes
|
||||||
|
# there. Were it left open the parent of weight 1000 would join it and
|
||||||
|
# carry about half the requests on its own, without the first hop.
|
||||||
|
fmark, smark = len(firstp.output()), len(secondp.output())
|
||||||
|
for _ in range(8):
|
||||||
|
t.eq(200, t.http(url, proxy=f"127.0.0.1:{thirds}").status,
|
||||||
|
"a group of three thirds carries the request")
|
||||||
|
t.eq(8, wait_served(firstp, tosecond, 8, fmark),
|
||||||
|
"every request took a third as its first hop")
|
||||||
|
t.eq(8, wait_served(secondp, toorigin, 8, smark),
|
||||||
|
"and the parent after them as its second")
|
||||||
Loading…
Reference in New Issue
Block a user