A highly concurrent, fault-tolerant, and masterless distributed Key-Value store built entirely from scratch in Modern C++, featuring a real-time visual dashboard built in React & Tailwind CSS.
Engineered to handle massive scalability, this system implements core distributed systems concepts inspired by Amazon Dynamo and Apache Cassandra.
- Implemented a Ring Topology using the 64-bit FNV-1a Hash Algorithm to ensure uniform data distribution and eliminate hot spots.
- Configured with 100 Virtual Nodes (vNodes) per physical server to achieve perfect load balancing and seamless scaling.
- Strict Quorum Logic (W=2, R=2): Ensures consistency across cluster nodes. Writes are acknowledged once replica quorums are achieved.
- Write-Ahead Logging (WAL): Disk-backed crash recovery. Every node recovers its exact state from local logs instantly upon reboot.
- Gossip Protocol: Background heartbeat threads monitor cluster health, instantly detecting node failures and dynamically updating the routing topology.
- Architected a thread-safe custom Storage Engine utilizing
std::shared_mutex(Read-Write Locks). - Achieved 33,685 Write Requests Per Second (RPS) with a highly concurrent load of 100 simultaneous threads, maintaining an average latency of just ~2.97ms (Benchmarked via Apache
ab).
- Dynamically supports adding or removing nodes.
- Exposes an
/admin/rebalanceendpoint that intelligently calculates hash boundaries and migrates live data to new nodes without cluster downtime.
- A real-time React/Vite dashboard that visualizes the Consistent Hashing Ring.
- Chaos Engineering Controls: Inject massive workloads or simulate node deaths with a single click to visually demonstrate data rebalancing and fault tolerance in real-time.
- Backend Engine: Modern C++ (C++17)
- Networking/HTTP:
cpp-httplib - Frontend UI: React.js, Vite, Tailwind CSS
- Performance Testing: Apache Bench (
ab) & Custom Bash Scripts
- GCC/G++ (Supports C++17)
- CMake 3.10+
- Apache Bench (
abtool for load testing) - Node.js & npm (For the UI)
# Clone the repository
git clone https://github.com/Tejas-Raj01/distributed-system.git
cd distributed-system
# Create release build with maximum optimizations
cmake -B build -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS="-O3"
cmake --build build# Ensure data directory exists for WAL logging
mkdir -p data
# Start a server node on port 8080
./build/kv_server 8080To run high-concurrency performance benchmarks matching our test setup (100 concurrent threads, 50,000 requests), follow these steps:
- Ubuntu/Debian:
sudo apt install apache2-utils - Fedora/RHEL:
sudo dnf install httpd-tools - Arch Linux:
sudo pacman -S apache
./build/kv_server 8080 > data/server.log 2>&1 &- Create a post payload file:
echo "key=StressTestKey&value=MassiveDataLoad" > payload.txt
- Run Apache Bench for 50,000 POST requests with 100 concurrent connections:
ab -n 50000 -c 100 -p payload.txt -T "application/x-www-form-urlencoded" "http://127.0.0.1:8080/put"
Run Apache Bench for 50,000 GET requests under 100 concurrent connections:
ab -n 50000 -c 100 "http://127.0.0.1:8080/get?key=StressTestKey"Tested on Release build (-O3 optimized) with 50,000 requests under 100 concurrent threads:
| Metric / Parameter | Write (POST /put) |
Read (GET /get) |
|---|---|---|
| Total Requests | 50,000 | 50,000 |
| Concurrency Level | 100 connections | 100 connections |
| Throughput (Requests/sec) | 33,684.94 req/sec | 1,809.14 req/sec |
| Mean Latency (Average) | 2.969 ms | 55.275 ms |
| Concurrent Request Latency | 0.030 ms | 0.553 ms |
| Success / Error Rate | 100% Success (0 Errors) | 100% Success (0 Errors) |
| Percentile | Write Latency (ms) | Read Latency (ms) |
|---|---|---|
| 50% (Median) | 1 ms | 49 ms |
| 66% | 2 ms | 50 ms |
| 75% | 2 ms | 50 ms |
| 80% | 2 ms | 51 ms |
| 90% | 3 ms | 71 ms |
| 95% | 4 ms | 72 ms |
| 98% | 5 ms | 73 ms |
| 99% | 5 ms | 73 ms |