diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..ef91b57 --- /dev/null +++ b/.gitignore @@ -0,0 +1,18 @@ +# Build artifacts +*.o +src/predixy + +# IDE and editor files +.vscode/ +.idea/ +*.swp +*~ + +# OS generated files +.DS_Store +.DS_Store? +._* +.Spotlight-V100 +.Trashes +ehthumbs.db +Thumbs.db \ No newline at end of file diff --git a/README.md b/README.md index e0caff1..8baae60 100644 --- a/README.md +++ b/README.md @@ -23,7 +23,7 @@ + Stats info, CPU/Memory/Requests/Responses and so on. + Latency monitor. -## Build +## Generic Build Instructions Predixy can be compiled and used on Linux, OSX, BSD, Windows([Cygwin](http://www.cygwin.com/)). Requires C++11 compiler. @@ -48,13 +48,13 @@ For examples: $ make MT=false $ make debug MT=false TS=true -## Install +## Generic Install Instructions Just copy src/predixy to the install path $ cp src/predixy /path/to/bin -## Configuration +## Generic Configuration Instructions See below files: + predixy.conf, basic config, will refrence below config files. @@ -117,6 +117,156 @@ Reset all stats and latency monitors, require admin permission. redis> CONFIG ResetStat +## Instacart Local Development Instructions (Cluster mode disabled) + +### Prerequisites +- Local Redis instance running on default port (6379) +- C++11 compiler +- Make + +### Install Redis Locally + +1. Use Homebrew to install Redis in a single instance mode with cluster mode disabled to test your changes locally +```bash +$ brew install redis +``` + +2. Start Redis +```bash +$ brew services start redis +``` + +3. Verify your local Redis is running: +```bash +$ redis-cli -h 127.0.0.1 -p 6379 ping +``` +Should return "PONG" + +```bash +$ redis-cli -h 127.0.0.1 -p 6379 info +``` + +Will return info about the Redis instance, if required for debugging + +Note, logs for Redis installed by Brew will appear in `/opt/homebrew/var/log/redis.log` by default. + +### Building and Running Predixy Locally + +1. Compile Predixy: +```bash +$ make +``` +This will create object files and the executable in the `src` directory. + +2. Verify your local Redis is running: +```bash +$ redis-cli -h 127.0.0.1 -p 6379 ping +``` +Should return "PONG" + +3. Start Predixy using the local configuration: +```bash +$ src/predixy conf/predixy_local.conf +``` +You should see output indicating Predixy is listening on 127.0.0.1:7617 + +4. Test the connection through Predixy: +```bash +# Basic connectivity test +$ redis-cli -h 127.0.0.1 -p 7617 ping + +# Check Predixy status +$ redis-cli -h 127.0.0.1 -p 7617 info + +# Test read/write operations +$ redis-cli -h 127.0.0.1 -p 7617 set test "Hello via Predixy" +$ redis-cli -h 127.0.0.1 -p 7617 get test +``` + +5. To stop Predixy: +```bash +$ pkill -f predixy +``` +## Instacart Local Development Instructions (Cluster mode disabled) + +### Setting up Redis Cluster Locally + +1. Create directories for each Redis node (we'll use 3 nodes): +```bash +$ mkdir -p redis-cluster/{7000,7001,7002} +``` + +2. Create Redis configuration for each node. First for port 7000: +```bash +$ cat > redis-cluster/7000/redis.conf << EOL +port 7000 +cluster-enabled yes +cluster-config-file nodes-7000.conf +cluster-node-timeout 5000 +appendonly yes +dir ./ +bind 127.0.0.1 +daemonize no +EOL +``` + +3. Copy and adjust configuration for other nodes: +```bash +$ cp redis-cluster/7000/redis.conf redis-cluster/7001/redis.conf +$ cp redis-cluster/7000/redis.conf redis-cluster/7002/redis.conf +$ sed -i '' 's/7000/7001/g' redis-cluster/7001/redis.conf +$ sed -i '' 's/7000/7002/g' redis-cluster/7002/redis.conf +``` + +4. Start each Redis instance (in separate terminal windows): +```bash +$ redis-server redis-cluster/7000/redis.conf +$ redis-server redis-cluster/7001/redis.conf +$ redis-server redis-cluster/7002/redis.conf +``` + +5. Create the cluster: +```bash +$ redis-cli --cluster create 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 --cluster-replicas 0 +``` +Type 'yes' when prompted to accept the configuration. + +6. Verify cluster status: +```bash +$ redis-cli -p 7000 cluster nodes +``` + +### Testing with Predixy + +1. Start Predixy with the cluster configuration: +```bash +$ src/predixy conf/predixy_cluster.conf +``` + +2. Test the cluster setup through Predixy: +```bash +# Basic connectivity test +$ redis-cli -p 7617 set test "Hello Cluster" +$ redis-cli -p 7617 get test + +# Check Predixy cluster status +$ redis-cli -p 7617 info +``` + +### Cleanup +When done testing, you can: +1. Stop Predixy (Ctrl+C or `pkill predixy`) +2. Stop each Redis instance (Ctrl+C in each terminal) +3. Remove cluster files: `rm -rf redis-cluster` + +Note: The cluster setup distributes keys across all nodes using hash slots. Each node in this setup handles approximately 5461 hash slots (16384/3 slots per node). + +### Notes +- Predixy will be listening on port 7617 while your Redis instance remains on 6379 +- The configuration in `predixy_local.conf` is set up for a single local Redis instance +- Build artifacts (*.o files) are ignored by git but can be safely kept for development +- Logs will appear in stdout by default + ## Benchmark predixy is fast, how fast? more than twemproxy, codis, redis-cerberus @@ -133,3 +283,4 @@ All rights reserved. License under BSD 3-clause "New" or "Revised" License WeChat:cppfan ![wechat](https://github.com/joyieldInc/predixy/blob/master/doc/wechat-cppfan.jpeg) + diff --git a/conf/predixy_cluster.conf b/conf/predixy_cluster.conf new file mode 100644 index 0000000..8bfdde9 --- /dev/null +++ b/conf/predixy_cluster.conf @@ -0,0 +1,119 @@ +################################### GENERAL #################################### +## Predixy configuration for Instacart local development for cluster mode enabled redis setups + +## Specify a name for this predixy service +## redis command INFO can get this +Name PredixyLocalCluster + +## Specify listen address, support IPV4, IPV6, Unix socket +## Examples: +# Bind 127.0.0.1:7617 +# Bind 0.0.0.0:7617 +# Bind /tmp/predixy + +## Bind to localhost for local testing +Bind 127.0.0.1:7617 + +## Worker threads +WorkerThreads 1 + +## Memory limit, 0 means unlimited +MaxMemory 1G + +## Examples: +# MaxMemory 100M +# MaxMemory 1G +# MaxMemory 0 + +## MaxMemory can change online by CONFIG SET MaxMemory xxx +## Default is 0 +# MaxMemory 0 + +## Close the connection after a client is idle for N seconds (0 to disable) +## ClientTimeout can change online by CONFIG SET ClientTimeout N +## Default is 0 +ClientTimeout 300 + + +## IO buffer size +## Default is 4096 +BufSize 4096 + +################################### LOG ######################################## +## Log file path +## Unspecify will log to stdout +## Default is Unspecified +# Log ./predixy.log + +## LogRotate support + +## 1d rotate log every day +## nh rotate log every n hours 1 <= n <= 24 +## nm rotate log every n minutes 1 <= n <= 1440 +## nG rotate log evenry nG bytes +## nM rotate log evenry nM bytes +## time rotate and size rotate can combine eg 1h 2G, means 1h or 2G roate a time + +## Examples: +# LogRotate 1d 2G +# LogRotate 1d + +## Default is disable LogRotate + + +## In multi-threads, worker thread log need lock, +## AllowMissLog can reduce lock time for improve performance +## AllowMissLog can change online by CONFIG SET AllowMissLog true|false +## Default is true +# AllowMissLog false + +## LogLevelSample, output a log every N +## all level sample can change online by CONFIG SET LogXXXSample N +LogVerbSample 0 +LogDebugSample 0 +LogInfoSample 10000 +LogNoticeSample 1 +LogWarnSample 1 +LogErrorSample 1 + + +################################### AUTHORITY ################################## +Include auth.conf + +################################### SERVERS #################################### +# Include cluster.conf +# Include sentinel.conf +Include try.conf + +ClusterServerPool { + Password "" + MasterReadPriority 60 + StaticSlaveReadPriority 50 + DynamicSlaveReadPriority 50 + RefreshInterval 1 + ServerTimeout 1 + ServerFailureLimit 10 + ServerRetryTimeout 1 + KeepAlive 120 + Servers { + + 127.0.0.1:7000 + + 127.0.0.1:7001 + + 127.0.0.1:7002 + } +} + +################################### DATACENTER ################################# +## LocalDC specify current machine dc +# LocalDC bj + +## see dc.conf +# Include dc.conf + + +################################### COMMAND #################################### +## Custom command define, see command.conf +#Include command.conf + +################################### LATENCY #################################### +## Latency monitor define, see latency.conf +Include latency.conf \ No newline at end of file diff --git a/conf/predixy_local.conf b/conf/predixy_local.conf new file mode 100644 index 0000000..7f4cd42 --- /dev/null +++ b/conf/predixy_local.conf @@ -0,0 +1,104 @@ +################################### GENERAL #################################### +## Predixy configuration for Instacart local development for cluster mode disabled redis setups (Aka Master/Replica) + +## Specify a name for this predixy service +## redis command INFO can get this +Name PredixyLocalRedis + +## Specify listen address, support IPV4, IPV6, Unix socket +## Using a different port than Redis to avoid conflicts +Bind 127.0.0.1:7617 + +## Worker threads +WorkerThreads 1 + +## Memory limit, 0 means unlimited +MaxMemory 0 + +## Close the connection after a client is idle for N seconds (0 to disable) +## ClientTimeout can change online by CONFIG SET ClientTimeout N +## Default is 0 +ClientTimeout 300 + +## IO buffer size +## Default is 4096 +BufSize 4096 + +################################### LOG ######################################## +## Log file path +## Unspecify will log to stdout +## Default is Unspecified +# Log ./predixy.log + +## LogRotate support + +## 1d rotate log every day +## nh rotate log every n hours 1 <= n <= 24 +## nm rotate log every n minutes 1 <= n <= 1440 +## nG rotate log evenry nG bytes +## nM rotate log evenry nM bytes +## time rotate and size rotate can combine eg 1h 2G, means 1h or 2G roate a time + +## Examples: +# LogRotate 1d 2G +# LogRotate 1d + +## Default is disable LogRotate + +## In multi-threads, worker thread log need lock, +## AllowMissLog can reduce lock time for improve performance +## AllowMissLog can change online by CONFIG SET AllowMissLog true|false +## Default is true +# AllowMissLog false + +## LogLevelSample, output a log every N +## all level sample can change online by CONFIG SET LogXXXSample N +LogVerbSample 0 +LogDebugSample 0 +LogInfoSample 10000 +LogNoticeSample 1 +LogWarnSample 1 +LogErrorSample 1 + +################################### AUTHORITY ################################## +## Simple auth config for local development +Authority { + Auth { + Mode write + } +} + +################################### SERVERS #################################### +## Standalone Redis configuration +StandaloneServerPool { + Databases 16 + Hash crc16 + HashTag "{}" + Distribution modula + MasterReadPriority 60 + StaticSlaveReadPriority 50 + DynamicSlaveReadPriority 50 + RefreshMethod fixed + ServerTimeout 1 + ServerFailureLimit 10 + ServerRetryTimeout 1 + KeepAlive 120 + Group local { + + 127.0.0.1:6379 + } +} + +################################### DATACENTER ################################# +## LocalDC specify current machine dc +# LocalDC bj + +## see dc.conf +# Include dc.conf + +################################### COMMAND #################################### +## Custom command define, see command.conf +#Include command.conf + +################################### LATENCY #################################### +## Latency monitor define, see latency.conf +Include latency.conf \ No newline at end of file