mirror of
https://github.com/joyieldInc/predixy.git
synced 2025-12-24 22:46:41 +08:00
Merge pull request #1 from instacart/MattyIC/add_local_testing_changes
STO-1330
This commit is contained in:
Matty Wright
GitHub
commit
d95f7a794a
18
.gitignore
vendored
Normal file
18
.gitignore
vendored
Normal file
@ -0,0 +1,18 @@
|
||||
# Build artifacts
|
||||
*.o
|
||||
src/predixy
|
||||
|
||||
# IDE and editor files
|
||||
.vscode/
|
||||
.idea/
|
||||
*.swp
|
||||
*~
|
||||
|
||||
# OS generated files
|
||||
.DS_Store
|
||||
.DS_Store?
|
||||
._*
|
||||
.Spotlight-V100
|
||||
.Trashes
|
||||
ehthumbs.db
|
||||
Thumbs.db
|
||||
157
README.md
157
README.md
@ -23,7 +23,7 @@
|
||||
+ Stats info, CPU/Memory/Requests/Responses and so on.
|
||||
+ Latency monitor.
|
||||
|
||||
## Build
|
||||
## Generic Build Instructions
|
||||
|
||||
Predixy can be compiled and used on Linux, OSX, BSD, Windows([Cygwin](http://www.cygwin.com/)). Requires C++11 compiler.
|
||||
|
||||
@ -48,13 +48,13 @@ For examples:
|
||||
$ make MT=false
|
||||
$ make debug MT=false TS=true
|
||||
|
||||
## Install
|
||||
## Generic Install Instructions
|
||||
|
||||
Just copy src/predixy to the install path
|
||||
|
||||
$ cp src/predixy /path/to/bin
|
||||
|
||||
## Configuration
|
||||
## Generic Configuration Instructions
|
||||
|
||||
See below files:
|
||||
+ predixy.conf, basic config, will refrence below config files.
|
||||
@ -117,6 +117,156 @@ Reset all stats and latency monitors, require admin permission.
|
||||
|
||||
redis> CONFIG ResetStat
|
||||
|
||||
## Instacart Local Development Instructions (Cluster mode disabled)
|
||||
|
||||
### Prerequisites
|
||||
- Local Redis instance running on default port (6379)
|
||||
- C++11 compiler
|
||||
- Make
|
||||
|
||||
### Install Redis Locally
|
||||
|
||||
1. Use Homebrew to install Redis in a single instance mode with cluster mode disabled to test your changes locally
|
||||
```bash
|
||||
$ brew install redis
|
||||
```
|
||||
|
||||
2. Start Redis
|
||||
```bash
|
||||
$ brew services start redis
|
||||
```
|
||||
|
||||
3. Verify your local Redis is running:
|
||||
```bash
|
||||
$ redis-cli -h 127.0.0.1 -p 6379 ping
|
||||
```
|
||||
Should return "PONG"
|
||||
|
||||
```bash
|
||||
$ redis-cli -h 127.0.0.1 -p 6379 info
|
||||
```
|
||||
|
||||
Will return info about the Redis instance, if required for debugging
|
||||
|
||||
Note, logs for Redis installed by Brew will appear in `/opt/homebrew/var/log/redis.log` by default.
|
||||
|
||||
### Building and Running Predixy Locally
|
||||
|
||||
1. Compile Predixy:
|
||||
```bash
|
||||
$ make
|
||||
```
|
||||
This will create object files and the executable in the `src` directory.
|
||||
|
||||
2. Verify your local Redis is running:
|
||||
```bash
|
||||
$ redis-cli -h 127.0.0.1 -p 6379 ping
|
||||
```
|
||||
Should return "PONG"
|
||||
|
||||
3. Start Predixy using the local configuration:
|
||||
```bash
|
||||
$ src/predixy conf/predixy_local.conf
|
||||
```
|
||||
You should see output indicating Predixy is listening on 127.0.0.1:7617
|
||||
|
||||
4. Test the connection through Predixy:
|
||||
```bash
|
||||
# Basic connectivity test
|
||||
$ redis-cli -h 127.0.0.1 -p 7617 ping
|
||||
|
||||
# Check Predixy status
|
||||
$ redis-cli -h 127.0.0.1 -p 7617 info
|
||||
|
||||
# Test read/write operations
|
||||
$ redis-cli -h 127.0.0.1 -p 7617 set test "Hello via Predixy"
|
||||
$ redis-cli -h 127.0.0.1 -p 7617 get test
|
||||
```
|
||||
|
||||
5. To stop Predixy:
|
||||
```bash
|
||||
$ pkill -f predixy
|
||||
```
|
||||
## Instacart Local Development Instructions (Cluster mode disabled)
|
||||
|
||||
### Setting up Redis Cluster Locally
|
||||
|
||||
1. Create directories for each Redis node (we'll use 3 nodes):
|
||||
```bash
|
||||
$ mkdir -p redis-cluster/{7000,7001,7002}
|
||||
```
|
||||
|
||||
2. Create Redis configuration for each node. First for port 7000:
|
||||
```bash
|
||||
$ cat > redis-cluster/7000/redis.conf << EOL
|
||||
port 7000
|
||||
cluster-enabled yes
|
||||
cluster-config-file nodes-7000.conf
|
||||
cluster-node-timeout 5000
|
||||
appendonly yes
|
||||
dir ./
|
||||
bind 127.0.0.1
|
||||
daemonize no
|
||||
EOL
|
||||
```
|
||||
|
||||
3. Copy and adjust configuration for other nodes:
|
||||
```bash
|
||||
$ cp redis-cluster/7000/redis.conf redis-cluster/7001/redis.conf
|
||||
$ cp redis-cluster/7000/redis.conf redis-cluster/7002/redis.conf
|
||||
$ sed -i '' 's/7000/7001/g' redis-cluster/7001/redis.conf
|
||||
$ sed -i '' 's/7000/7002/g' redis-cluster/7002/redis.conf
|
||||
```
|
||||
|
||||
4. Start each Redis instance (in separate terminal windows):
|
||||
```bash
|
||||
$ redis-server redis-cluster/7000/redis.conf
|
||||
$ redis-server redis-cluster/7001/redis.conf
|
||||
$ redis-server redis-cluster/7002/redis.conf
|
||||
```
|
||||
|
||||
5. Create the cluster:
|
||||
```bash
|
||||
$ redis-cli --cluster create 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 --cluster-replicas 0
|
||||
```
|
||||
Type 'yes' when prompted to accept the configuration.
|
||||
|
||||
6. Verify cluster status:
|
||||
```bash
|
||||
$ redis-cli -p 7000 cluster nodes
|
||||
```
|
||||
|
||||
### Testing with Predixy
|
||||
|
||||
1. Start Predixy with the cluster configuration:
|
||||
```bash
|
||||
$ src/predixy conf/predixy_cluster.conf
|
||||
```
|
||||
|
||||
2. Test the cluster setup through Predixy:
|
||||
```bash
|
||||
# Basic connectivity test
|
||||
$ redis-cli -p 7617 set test "Hello Cluster"
|
||||
$ redis-cli -p 7617 get test
|
||||
|
||||
# Check Predixy cluster status
|
||||
$ redis-cli -p 7617 info
|
||||
```
|
||||
|
||||
### Cleanup
|
||||
When done testing, you can:
|
||||
1. Stop Predixy (Ctrl+C or `pkill predixy`)
|
||||
2. Stop each Redis instance (Ctrl+C in each terminal)
|
||||
3. Remove cluster files: `rm -rf redis-cluster`
|
||||
|
||||
Note: The cluster setup distributes keys across all nodes using hash slots. Each node in this setup handles approximately 5461 hash slots (16384/3 slots per node).
|
||||
|
||||
### Notes
|
||||
- Predixy will be listening on port 7617 while your Redis instance remains on 6379
|
||||
- The configuration in `predixy_local.conf` is set up for a single local Redis instance
|
||||
- Build artifacts (*.o files) are ignored by git but can be safely kept for development
|
||||
- Logs will appear in stdout by default
|
||||
|
||||
## Benchmark
|
||||
|
||||
predixy is fast, how fast? more than twemproxy, codis, redis-cerberus
|
||||
@ -133,3 +283,4 @@ All rights reserved.
|
||||
License under BSD 3-clause "New" or "Revised" License
|
||||
|
||||
WeChat:cppfan 
|
||||
|
||||
|
||||
119
conf/predixy_cluster.conf
Normal file
119
conf/predixy_cluster.conf
Normal file
@ -0,0 +1,119 @@
|
||||
################################### GENERAL ####################################
|
||||
## Predixy configuration for Instacart local development for cluster mode enabled redis setups
|
||||
|
||||
## Specify a name for this predixy service
|
||||
## redis command INFO can get this
|
||||
Name PredixyLocalCluster
|
||||
|
||||
## Specify listen address, support IPV4, IPV6, Unix socket
|
||||
## Examples:
|
||||
# Bind 127.0.0.1:7617
|
||||
# Bind 0.0.0.0:7617
|
||||
# Bind /tmp/predixy
|
||||
|
||||
## Bind to localhost for local testing
|
||||
Bind 127.0.0.1:7617
|
||||
|
||||
## Worker threads
|
||||
WorkerThreads 1
|
||||
|
||||
## Memory limit, 0 means unlimited
|
||||
MaxMemory 1G
|
||||
|
||||
## Examples:
|
||||
# MaxMemory 100M
|
||||
# MaxMemory 1G
|
||||
# MaxMemory 0
|
||||
|
||||
## MaxMemory can change online by CONFIG SET MaxMemory xxx
|
||||
## Default is 0
|
||||
# MaxMemory 0
|
||||
|
||||
## Close the connection after a client is idle for N seconds (0 to disable)
|
||||
## ClientTimeout can change online by CONFIG SET ClientTimeout N
|
||||
## Default is 0
|
||||
ClientTimeout 300
|
||||
|
||||
|
||||
## IO buffer size
|
||||
## Default is 4096
|
||||
BufSize 4096
|
||||
|
||||
################################### LOG ########################################
|
||||
## Log file path
|
||||
## Unspecify will log to stdout
|
||||
## Default is Unspecified
|
||||
# Log ./predixy.log
|
||||
|
||||
## LogRotate support
|
||||
|
||||
## 1d rotate log every day
|
||||
## nh rotate log every n hours 1 <= n <= 24
|
||||
## nm rotate log every n minutes 1 <= n <= 1440
|
||||
## nG rotate log evenry nG bytes
|
||||
## nM rotate log evenry nM bytes
|
||||
## time rotate and size rotate can combine eg 1h 2G, means 1h or 2G roate a time
|
||||
|
||||
## Examples:
|
||||
# LogRotate 1d 2G
|
||||
# LogRotate 1d
|
||||
|
||||
## Default is disable LogRotate
|
||||
|
||||
|
||||
## In multi-threads, worker thread log need lock,
|
||||
## AllowMissLog can reduce lock time for improve performance
|
||||
## AllowMissLog can change online by CONFIG SET AllowMissLog true|false
|
||||
## Default is true
|
||||
# AllowMissLog false
|
||||
|
||||
## LogLevelSample, output a log every N
|
||||
## all level sample can change online by CONFIG SET LogXXXSample N
|
||||
LogVerbSample 0
|
||||
LogDebugSample 0
|
||||
LogInfoSample 10000
|
||||
LogNoticeSample 1
|
||||
LogWarnSample 1
|
||||
LogErrorSample 1
|
||||
|
||||
|
||||
################################### AUTHORITY ##################################
|
||||
Include auth.conf
|
||||
|
||||
################################### SERVERS ####################################
|
||||
# Include cluster.conf
|
||||
# Include sentinel.conf
|
||||
Include try.conf
|
||||
|
||||
ClusterServerPool {
|
||||
Password ""
|
||||
MasterReadPriority 60
|
||||
StaticSlaveReadPriority 50
|
||||
DynamicSlaveReadPriority 50
|
||||
RefreshInterval 1
|
||||
ServerTimeout 1
|
||||
ServerFailureLimit 10
|
||||
ServerRetryTimeout 1
|
||||
KeepAlive 120
|
||||
Servers {
|
||||
+ 127.0.0.1:7000
|
||||
+ 127.0.0.1:7001
|
||||
+ 127.0.0.1:7002
|
||||
}
|
||||
}
|
||||
|
||||
################################### DATACENTER #################################
|
||||
## LocalDC specify current machine dc
|
||||
# LocalDC bj
|
||||
|
||||
## see dc.conf
|
||||
# Include dc.conf
|
||||
|
||||
|
||||
################################### COMMAND ####################################
|
||||
## Custom command define, see command.conf
|
||||
#Include command.conf
|
||||
|
||||
################################### LATENCY ####################################
|
||||
## Latency monitor define, see latency.conf
|
||||
Include latency.conf
|
||||
104
conf/predixy_local.conf
Normal file
104
conf/predixy_local.conf
Normal file
@ -0,0 +1,104 @@
|
||||
################################### GENERAL ####################################
|
||||
## Predixy configuration for Instacart local development for cluster mode disabled redis setups (Aka Master/Replica)
|
||||
|
||||
## Specify a name for this predixy service
|
||||
## redis command INFO can get this
|
||||
Name PredixyLocalRedis
|
||||
|
||||
## Specify listen address, support IPV4, IPV6, Unix socket
|
||||
## Using a different port than Redis to avoid conflicts
|
||||
Bind 127.0.0.1:7617
|
||||
|
||||
## Worker threads
|
||||
WorkerThreads 1
|
||||
|
||||
## Memory limit, 0 means unlimited
|
||||
MaxMemory 0
|
||||
|
||||
## Close the connection after a client is idle for N seconds (0 to disable)
|
||||
## ClientTimeout can change online by CONFIG SET ClientTimeout N
|
||||
## Default is 0
|
||||
ClientTimeout 300
|
||||
|
||||
## IO buffer size
|
||||
## Default is 4096
|
||||
BufSize 4096
|
||||
|
||||
################################### LOG ########################################
|
||||
## Log file path
|
||||
## Unspecify will log to stdout
|
||||
## Default is Unspecified
|
||||
# Log ./predixy.log
|
||||
|
||||
## LogRotate support
|
||||
|
||||
## 1d rotate log every day
|
||||
## nh rotate log every n hours 1 <= n <= 24
|
||||
## nm rotate log every n minutes 1 <= n <= 1440
|
||||
## nG rotate log evenry nG bytes
|
||||
## nM rotate log evenry nM bytes
|
||||
## time rotate and size rotate can combine eg 1h 2G, means 1h or 2G roate a time
|
||||
|
||||
## Examples:
|
||||
# LogRotate 1d 2G
|
||||
# LogRotate 1d
|
||||
|
||||
## Default is disable LogRotate
|
||||
|
||||
## In multi-threads, worker thread log need lock,
|
||||
## AllowMissLog can reduce lock time for improve performance
|
||||
## AllowMissLog can change online by CONFIG SET AllowMissLog true|false
|
||||
## Default is true
|
||||
# AllowMissLog false
|
||||
|
||||
## LogLevelSample, output a log every N
|
||||
## all level sample can change online by CONFIG SET LogXXXSample N
|
||||
LogVerbSample 0
|
||||
LogDebugSample 0
|
||||
LogInfoSample 10000
|
||||
LogNoticeSample 1
|
||||
LogWarnSample 1
|
||||
LogErrorSample 1
|
||||
|
||||
################################### AUTHORITY ##################################
|
||||
## Simple auth config for local development
|
||||
Authority {
|
||||
Auth {
|
||||
Mode write
|
||||
}
|
||||
}
|
||||
|
||||
################################### SERVERS ####################################
|
||||
## Standalone Redis configuration
|
||||
StandaloneServerPool {
|
||||
Databases 16
|
||||
Hash crc16
|
||||
HashTag "{}"
|
||||
Distribution modula
|
||||
MasterReadPriority 60
|
||||
StaticSlaveReadPriority 50
|
||||
DynamicSlaveReadPriority 50
|
||||
RefreshMethod fixed
|
||||
ServerTimeout 1
|
||||
ServerFailureLimit 10
|
||||
ServerRetryTimeout 1
|
||||
KeepAlive 120
|
||||
Group local {
|
||||
+ 127.0.0.1:6379
|
||||
}
|
||||
}
|
||||
|
||||
################################### DATACENTER #################################
|
||||
## LocalDC specify current machine dc
|
||||
# LocalDC bj
|
||||
|
||||
## see dc.conf
|
||||
# Include dc.conf
|
||||
|
||||
################################### COMMAND ####################################
|
||||
## Custom command define, see command.conf
|
||||
#Include command.conf
|
||||
|
||||
################################### LATENCY ####################################
|
||||
## Latency monitor define, see latency.conf
|
||||
Include latency.conf
|
||||
Loading…
Reference in New Issue
Block a user