Introduction

Welcome to the Xylem documentation!

Xylem is a high-performance and modular traffic generator and measurement tool designed for RPC workloads. It provides a flexible framework for benchmarking and load testing distributed systems with different combinations of application protocols (e.g., Redis, HTTP, Memcached) and transport protocols (e.g., TCP, UDP, Unix Domain Socket).

Key Features

• Multi-Protocol Support: Built-in support for Redis, HTTP, Memcached, Masstree, and xylem-echo protocols
• Flexible Transport Layer: Support for TCP, UDP, and Unix Domain Sockets
• High Performance: Efficient event-driven architecture for generating high loads
• Reproducible: Configuration-first design using TOML profiles ensures reproducibility
• Detailed Metrics: Latency measurements and statistics using various sketch algorithms
• Multi-Threaded: Support for thread affinity and multi-threaded workload generation

Use Cases

Xylem is ideal for:

• Performance Benchmarking: Measure the performance of your RPC services under various load conditions
• Load Testing: Generate realistic traffic patterns to test system behavior under stress
• Latency Analysis: Collect detailed latency statistics to identify performance bottlenecks
• Protocol Testing: Validate protocol implementations across different transport layers
• Capacity Planning: Determine the maximum throughput and optimal configuration for your services

Project Status

Xylem is actively developed and maintained. The project is licensed under MIT OR Apache-2.0.

Getting Help

• GitHub Issues: Report bugs or request features at github.com/minhuw/xylem/issues
• Source Code: View the source code at github.com/minhuw/xylem

Next Steps

• Follow the Quick Start guide
• Learn about CLI Reference
• Explore the Architecture to understand Xylem’s design

Quick Start

This guide will help you run your first benchmark with Xylem.

Prerequisites

• Rust 1.70 or later
• A target service to benchmark (e.g., Redis, HTTP server, Memcached)

Build Xylem

# Clone the repository
git clone https://github.com/minhuw/xylem.git
cd xylem

# Build in release mode
cargo build --release

# The binary will be at target/release/xylem

Running a Redis Benchmark

1. Start a Redis Server

First, ensure you have a Redis server running locally:

redis-server

By default, Redis listens on localhost:6379.

2. Run the Benchmark

Use one of the included example profiles:

./target/release/xylem -P profiles/redis-get-zipfian.toml

This profile runs a Redis GET benchmark with a Zipfian key distribution.

3. Understanding the Output

Xylem will display statistics about the benchmark, including:

• Latency percentiles (p50, p95, p99, p99.9, etc.)
• Throughput (requests per second)
• Error rates
• Per-thread statistics

Configuration-First Design

Xylem uses TOML configuration files (called “profiles”) to define experiments. This ensures reproducibility and simplifies complex workload specifications.

Basic Syntax

xylem -P <profile.toml>

Example Profiles

Xylem includes example profiles in the profiles/ directory:

# Run a Redis GET benchmark with Zipfian distribution
xylem -P profiles/redis-get-zipfian.toml

# Run an HTTP load test
xylem -P profiles/http-spike.toml

# Run a Memcached benchmark
xylem -P profiles/memcached-ramp.toml

Customizing the Benchmark

You can override configuration values using the --set flag with dot notation:

# Change target address
./target/release/xylem -P profiles/redis-get-zipfian.toml \
  --set target.address=192.168.1.100:6379

# Change experiment duration
./target/release/xylem -P profiles/redis-get-zipfian.toml \
  --set experiment.duration=120s

# Change multiple parameters
./target/release/xylem -P profiles/redis-get-zipfian.toml \
  --set experiment.duration=60s \
  --set experiment.seed=42 \
  --set workload.keys.n=1000000

Creating a Custom Profile

Create your own TOML profile file:

# my-benchmark.toml

[experiment]
duration = "30s"
seed = 123

[target]
protocol = "redis"
transport = "tcp"
address = "localhost:6379"

[workload]
[workload.keys]
strategy = "zipfian"
n = 10000
theta = 0.99
value_size = 100

[[traffic_groups]]
name = "main"
protocol = "redis"
threads = [0, 1, 2, 3]
connections_per_thread = 10
max_pending_per_connection = 1

[traffic_groups.sampling_policy]
type = "unlimited"

[traffic_groups.policy]
type = "closed-loop"

Run with:

./target/release/xylem -P my-benchmark.toml

Profile File Structure

A typical profile file includes:

# Experiment configuration
[experiment]
duration = "60s"
seed = 123

# Target service configuration
[target]
protocol = "redis"
address = "127.0.0.1:6379"

# Workload configuration
[workload]
# ... workload parameters

# Traffic groups (thread assignment and rate control)
[[traffic_groups]]
name = "group-1"
threads = [0, 1, 2, 3]
# ... rate control parameters

Logging

Control logging verbosity:

# Debug level logging
xylem -P profiles/redis.toml --log-level debug

# Using RUST_LOG environment variable
RUST_LOG=debug xylem -P profiles/redis.toml

Next Steps

• Explore CLI Reference for all available options
• Learn about Configuration file format
• Check out example profiles in the profiles/ directory

CLI Reference

Complete reference for Xylem command-line interface.

Xylem uses a config-first design with TOML profile files. This ensures reproducibility and simplifies complex workload specifications.

Basic Usage

xylem -P profiles/redis-get-zipfian.toml

Global Options

--version

Display version information.

xylem --version

--help

Display help information.

xylem --help

-P, --profile <FILE>

Path to TOML profile configuration file.

xylem -P profiles/redis-bench.toml

--set <KEY=VALUE>

Override any configuration value using dot notation. Can be specified multiple times.

xylem -P profiles/redis.toml --set target.address=192.168.1.100:6379
xylem -P profiles/http.toml --set experiment.duration=120s --set experiment.seed=12345

Examples:

• --set target.address=127.0.0.1:6379
• --set experiment.duration=60s
• --set experiment.seed=999
• --set target.protocol=memcached-binary
• --set workload.keys.n=1000000
• --set traffic_groups.0.threads=[0,1,2,3]
• --set output.file=/tmp/results.json

-l, --log-level <LEVEL>

Set log level (trace, debug, info, warn, error).

Default: info

xylem -P profiles/redis.toml --log-level debug

Subcommands

completions <SHELL>

Generate shell completion scripts.

Supported shells:

• bash
• zsh
• fish
• powershell
• elvish

Examples:

# Bash
xylem completions bash > ~/.local/share/bash-completion/completions/xylem

# Zsh
xylem completions zsh > ~/.zsh/completions/_xylem

# Fish
xylem completions fish > ~/.config/fish/completions/xylem.fish

schema

Generate JSON Schema for configuration files.

xylem schema > config-schema.json

Configuration Overrides

The --set flag uses dot notation to override any configuration value:

# Override target address
xylem -P profiles/redis.toml --set target.address=localhost:6379

# Override experiment parameters
xylem -P profiles/bench.toml --set experiment.duration=300s --set experiment.seed=42

# Override workload settings
xylem -P profiles/redis.toml --set workload.keys.n=1000000

# Override traffic group settings
xylem -P profiles/multi.toml --set traffic_groups.0.threads=[0,1,2,3]

# Add new traffic group
xylem -P profiles/base.toml --set 'traffic_groups.+={name="new-group",threads=[4,5]}'

Profile Files

Xylem requires a TOML profile file that defines the experiment configuration. See the profiles/ directory for example configurations.

Example profile structure:

[experiment]
duration = "60s"
seed = 123

[target]
protocol = "redis"
address = "127.0.0.1:6379"

[workload]
# Workload configuration...

[[traffic_groups]]
name = "group-1"
threads = [0, 1, 2, 3]

Environment Variables

Xylem respects the following environment variables:

• RUST_LOG - Set logging level (e.g., debug, info, warn, error)

RUST_LOG=debug xylem -P profiles/redis.toml

See Also

• Configuration - Detailed configuration file format
• Configuration Schema - Configuration schema reference

Configuration

Xylem uses TOML configuration files (called “profiles”) for defining experiments.

Configuration File Structure

A profile is a TOML document with the following top-level sections:

[experiment]
duration = "60s"
seed = 123

[target]
protocol = "redis"
address = "127.0.0.1:6379"

[workload]
# Workload parameters

[[traffic_groups]]
name = "main"
threads = [0, 1, 2, 3]
# Rate control parameters

Sections

The configuration is divided into several sections:

• Workload Configuration - Define workload patterns, duration, and key distributions
• Transport Configuration - Configure network transport options
• Protocol Configuration - Protocol-specific settings

Basic Example

[experiment]
duration = "60s"
seed = 42

[target]
protocol = "redis"
transport = "tcp"
address = "localhost:6379"

[workload.keys]
strategy = "zipfian"
n = 10000
theta = 0.99
value_size = 100

[[traffic_groups]]
name = "group-1"
threads = [0, 1, 2, 3]
connections_per_thread = 10
max_pending_per_connection = 1

[traffic_groups.sampling_policy]
type = "unlimited"

[traffic_groups.policy]
type = "closed-loop"

[output]
format = "json"
file = "results.json"

Configuration Overrides

You can override any configuration value using the --set flag with dot notation:

# Override target address
xylem -P profile.toml --set target.address=192.168.1.100:6379

# Override experiment duration
xylem -P profile.toml --set experiment.duration=120s

# Override multiple parameters
xylem -P profile.toml --set experiment.duration=60s --set experiment.seed=999

Loading Configuration

Use the -P or --profile flag to load a profile file:

xylem -P profiles/redis-get-zipfian.toml

See Also

• Workload Configuration
• Transport Configuration
• Protocol Configuration
• Configuration Schema

Workload Configuration

The workload configuration is where you define how Xylem generates load against your system. This is where theory meets practice - where you translate your understanding of production traffic into a reproducible benchmark. A well-crafted workload configuration captures the essential characteristics of real user behavior: which keys get accessed, how often different operations occur, and how data sizes vary across your application.

The Structure of a Workload

Every workload in Xylem consists of three fundamental components that work together to simulate realistic traffic:

Key distribution determines which keys your benchmark accesses. Will you hit every key equally (uniform random), focus on a hot set (Zipfian), or model temporal locality where recent keys are hot (Gaussian)? This choice fundamentally shapes your cache hit rates and system behavior.

Operations configuration controls what your benchmark does with those keys. Real applications don’t just perform one operation - they mix reads and writes, batch operations, and specialized commands in patterns that reflect business logic. You can model a read-heavy cache (90% GET), a balanced session store (70% GET, 30% SET), or even custom workloads using any Redis command.

Value sizes determine how much data each operation handles. Production systems rarely use fixed-size values - small cache keys, medium session data, large objects all coexist. Xylem lets you model this complexity, even configuring different sizes for different operations to match your actual data patterns.

Together, these three components create a workload that mirrors reality. Let’s explore each in depth.

Understanding Key Distributions

The way your application accesses keys has profound implications for performance. A cache with perfect hit rates under sequential access might struggle with random patterns. A system optimized for uniform load might behave differently when faced with hot keys. Xylem provides several distribution strategies, each modeling distinct real-world patterns.

Sequential Keys: The Baseline

Sequential access is the simplest pattern - keys are accessed in order:

[workload.keys]
strategy = "sequential"
start = 0
max = 100000
value_size = 128

Every benchmark accesses keys 0, 1, 2, and so on up to 99,999. This pattern represents the best-case scenario for most systems: perfect predictability, excellent cache locality, and minimal memory pressure. While unrealistic for production, sequential access gives you a performance ceiling - the best your system can possibly do.

Use sequential keys when establishing baseline performance, comparing different configurations, or debugging issues. The predictability makes problems easier to isolate.

Uniform Random: Pure Chaos

At the opposite extreme, uniform random access treats every key equally:

[workload.keys]
strategy = "random"
max = 1000000
value_size = 128

Each request has an equal chance of accessing any key from 0 to 999,999. This represents the worst case for caching: no locality, maximum memory pressure, and frequent cache misses. If sequential access shows you your ceiling, random access shows you your floor.

Random distributions are perfect for stress testing. They reveal capacity limits, expose memory management issues, and help you understand behavior under the worst possible access patterns. Use this when you need to establish how your system degrades under pressure.

Round-Robin: Predictable Cycling

Round-robin access cycles through keys in order, wrapping back to the beginning:

[workload.keys]
strategy = "round-robin"
max = 100000
value_size = 128

This pattern accesses keys 0, 1, 2, … 99,999, then immediately jumps back to 0. It’s less common in production but useful when you need predictable, repeating patterns - perhaps for testing cache warm-up behavior or validating eviction policies.

Zipfian Distribution: The Power Law

Real-world access patterns rarely distribute evenly. Some keys - popular content, active user accounts, frequently accessed configuration - receive far more traffic than others. Zipfian distribution models this fundamental characteristic of production systems:

[workload.keys]
strategy = "zipfian"
exponent = 0.99
max = 1000000
value_size = 128

The exponent controls how skewed the distribution is. At 0.99 (typical for real workloads), the pattern is dramatically skewed: the top 1% of keys might receive 80% of requests, while the bottom 50% collectively receive only a few percent. This mirrors what you see in production - think of popular YouTube videos, trending tweets, or frequently accessed products.

Zipfian is your go-to distribution for realistic cache testing. It reveals how well your system handles hot keys, how effective your caching strategy is, and whether you can maintain performance when traffic concentrates on a small key set. The pattern remains stable over time - if key 1000 is popular now, it stays popular throughout the benchmark.

Gaussian Distribution: Temporal Locality

Sometimes hotness isn’t about persistent popularity - it’s about recency. Session stores, time-series databases, and log aggregators often exhibit temporal locality: recent data is hot, older data is cold. Gaussian distribution captures this pattern beautifully:

[workload.keys]
strategy = "gaussian"
mean_pct = 0.5      # Center of the hot zone
std_dev_pct = 0.1   # How concentrated
max = 10000
value_size = 128

The percentages make this intuitive. With mean_pct = 0.5, the hot spot centers at 50% of your keyspace - key 5000 in this example. The std_dev_pct = 0.1 means the standard deviation is 10% of the keyspace (1000 keys). Using the standard rules for normal distributions:

• Roughly 68% of accesses hit the center ±1000 keys (4000-6000)
• About 95% hit the center ±2000 keys (3000-7000)
• Nearly all requests fall within ±3000 keys (2000-8000)

This creates a “warm zone” that moves through your keyspace. It’s perfect for modeling recently written data being immediately read (think recent log entries), active user sessions clustering in a time window, or rolling analytics windows where you repeatedly access recent periods.

The key difference from Zipfian: Gaussian models temporal hotness (recent data is hot), while Zipfian models persistent hotness (popular data stays popular). Choose based on whether your hot set is stable (Zipfian) or shifts over time (Gaussian).

Configuring Operations: What Your Benchmark Does

Real applications don’t perform just one operation. They mix reads and writes in patterns that reflect business logic. A content delivery network might be 95% reads, while a logging system might be 80% writes. Xylem lets you model these patterns precisely.

Single Operation for Simplicity

When you’re establishing baselines or focusing on a specific operation’s performance, use a fixed operation:

[workload.operations]
strategy = "fixed"
operation = "get"

This configuration generates only GET operations. It’s simple, predictable, and useful when you want to isolate and measure one aspect of your system. Change to "set" for write testing, or "incr" for counter workloads.

Weighted Operations for Realism

Production systems mix operations in characteristic ratios. Model these patterns with weighted configurations:

[workload.operations]
strategy = "weighted"

[[workload.operations.commands]]
name = "get"
weight = 0.7  # 70% reads

[[workload.operations.commands]]
name = "set"
weight = 0.3  # 30% writes

The weights define the probability of each operation. Xylem normalizes them automatically, so weight = 0.7 and weight = 70 work identically. Choose whatever feels natural - percentages often read more clearly.

Common patterns you’ll encounter:

Heavy cache workloads (CDN, API response caching):

GET: 0.9-0.95, SET: 0.05-0.1

Session stores (authentication, user state):

GET: 0.7, SET: 0.3

Counter systems (rate limiting, metrics):

GET: 0.5, INCR: 0.5

Write-heavy systems (logging, event streams):

GET: 0.2, SET: 0.8

Batch Operations with MGET

Many applications fetch multiple keys atomically for efficiency. Model this with MGET operations:

[[workload.operations.commands]]
name = "mget"
weight = 0.2
count = 10  # Ten keys per operation

Each MGET request fetches 10 consecutive keys, simulating how your application might retrieve a user’s session along with related data, or fetch a time series of recent measurements. This helps you understand pipeline efficiency and batch operation performance.

Testing Replication with WAIT

Running Redis with replication? The WAIT command helps you measure replication lag under load:

[[workload.operations.commands]]
name = "wait"
weight = 0.1
num_replicas = 2      # Wait for 2 replicas
timeout_ms = 1000     # Give up after 1 second

Include WAIT in your mix to understand consistency guarantees. How often does replication keep up? What’s the latency cost of waiting for replicas? WAIT operations answer these questions.

Custom Commands for Advanced Use Cases

Redis supports hundreds of commands for lists, sets, sorted sets, hashes, and more. Custom templates let you benchmark any operation:

[[workload.operations.commands]]
name = "custom"
weight = 0.1
template = "HSET sessions:__key__ data __data__"

The template system provides three placeholders:

• __key__ - Current key from your distribution
• __data__ - Random bytes sized per your value_size config
• __value_size__ - Numeric size, useful for scores or counts

Examples that model real workloads:

Add timestamped events to a sorted set:

template = "ZADD events __value_size__ event:__key__"

Update session data with TTL:

template = "SETEX session:__key__ 1800 __data__"

Push to an activity log:

template = "LPUSH user:__key__:activity __data__"

Track active users:

template = "SADD active_now user:__key__"

Modeling Value Size Variation

Real data doesn’t come in uniform chunks. Cache keys might be 64 bytes, session tokens 256 bytes, and user profiles 2KB. Different operations handle different sizes. Xylem gives you sophisticated tools to model this reality.

Fixed Sizes: The Known Quantity

When every value has the same size - or when you want consistency for benchmarking - use fixed sizes:

[workload.value_size]
strategy = "fixed"
size = 128

Every operation uses exactly 128 bytes. This is your baseline: consistent, repeatable, and predictable. Use fixed sizes when comparing configurations or establishing capacity limits.

Uniform Distribution: Exploring a Range

Don’t know your exact size distribution but want to test across a range? Uniform distribution gives every size equal probability:

[workload.value_size]
strategy = "uniform"
min = 64
max = 4096

Requests will use sizes anywhere from 64 bytes to 4KB, spread evenly. A request might use 64 bytes, the next 3876 bytes, another 512 bytes - every size in the range has equal probability. This is useful for stress testing memory allocation and fragmentation.

Normal Distribution: Natural Clustering

Most real data clusters around a typical size. Session data averages 512 bytes with variation. Cache entries typically run 256 bytes but occasionally spike. Normal distribution models this natural clustering:

[workload.value_size]
strategy = "normal"
mean = 512.0
std_dev = 128.0
min = 64      # Floor
max = 4096    # Ceiling

The mean (512 bytes) is your typical value size. The standard deviation (128 bytes) controls spread. Using standard normal distribution rules:

• 68% of values fall within 384-640 bytes (±1σ)
• 95% fall within 256-768 bytes (±2σ)
• 99.7% fall within 128-896 bytes (±3σ)

The min/max bounds prevent extremes. Without them, the normal distribution could occasionally generate tiny (2-byte) or huge (50KB) values. The clamping keeps values practical while preserving the clustering behavior.

Per-Command Sizes: Maximum Realism

Your most powerful tool is configuring sizes per operation type. Reads often fetch small keys or IDs, while writes store larger data:

[workload.value_size]
strategy = "per_command"
default = { strategy = "fixed", size = 256 }

[workload.value_size.commands.get]
distribution = "fixed"
size = 64  # GETs fetch small cache keys

[workload.value_size.commands.set]
distribution = "uniform"
min = 128
max = 1024  # SETs write varied data

Notice the terminology shift: per-command configs use distribution to distinguish them from the top-level strategy. Each command can have its own distribution with its own parameters.

The default configuration handles operations you haven’t explicitly configured. If your workload includes INCR but you haven’t specified a size, it uses the default.

Real-world examples:

Session store (small IDs, larger session data):

GET: fixed 64 bytes
SET: normal mean=512, σ=128

Content cache (tiny keys, variable content):

GET: fixed 32 bytes
SET: uniform 1KB-100KB

Analytics (small queries, large result sets):

GET: normal mean=2KB, σ=512
SET: uniform 10KB-1MB

Time-Varying Load Patterns

Beyond key distributions, operations, and sizes, you can vary the overall request rate over time. This is your macro-level control, letting you model daily cycles, traffic spikes, or gradual ramp-ups.

Constant Load

The simplest pattern maintains steady throughput:

[workload.pattern]
type = "constant"
rate = 10000.0  # Requests per second

Use constant patterns for baseline testing and capacity planning.

Gradual Ramp-Up

Simulate scaling up or warming caches:

[workload.pattern]
type = "ramp"
start_rate = 1000.0
end_rate = 10000.0

Traffic increases linearly from 1K to 10K requests per second over your experiment duration.

Traffic Spikes

Model sudden load increases:

[workload.pattern]
type = "spike"
base_rate = 5000.0
spike_rate = 20000.0
spike_start = "10s"
spike_duration = "5s"

Normal traffic at 5K RPS suddenly jumps to 20K RPS at the 10-second mark, holds for 5 seconds, then returns to baseline. Perfect for testing autoscaling, circuit breakers, and graceful degradation.

Cyclical Patterns

Model daily traffic cycles:

[workload.pattern]
type = "sinusoidal"
min_rate = 5000.0
max_rate = 15000.0
period = "60s"

Traffic oscillates smoothly between 5K and 15K RPS, completing one full cycle every 60 seconds.

Bringing It All Together

Let’s build a complete workload that demonstrates these concepts. Imagine you’re testing a session store that exhibits temporal locality (recent sessions are hot), mixed operations (reads dominate but writes are significant), and varied data sizes (small lookups, larger session data):

[workload]
# Temporal locality - recently created sessions get most traffic
[workload.keys]
strategy = "gaussian"
mean_pct = 0.6       # Hot spot toward recent sessions
std_dev_pct = 0.15   # Moderately concentrated
max = 10000
value_size = 512

# Read-heavy but significant writes
[workload.operations]
strategy = "weighted"

[[workload.operations.commands]]
name = "get"
weight = 0.7

[[workload.operations.commands]]
name = "set"
weight = 0.3

# Small reads (session ID lookups), varied writes (full session data)
[workload.value_size]
strategy = "per_command"
default = { strategy = "fixed", size = 256 }

[workload.value_size.commands.get]
distribution = "fixed"
size = 64

[workload.value_size.commands.set]
distribution = "uniform"
min = 128
max = 1024

# Constant baseline load
[workload.pattern]
type = "constant"
rate = 10000.0

This configuration captures essential characteristics: temporal hotness (Gaussian keys), realistic operation mix (70/30 read-write), and varied sizes matching actual data patterns. Running this gives you insights you can trust.

Practical Wisdom

Ensuring Reproducible Results

Randomness is essential for realistic workloads, but you need reproducible results. Set a seed:

[experiment]
seed = 42  # Any consistent value works

With the same seed, Xylem generates identical sequences of keys and sizes across runs. You can share configurations with colleagues knowing they’ll see exactly the same workload pattern.

Choosing Distributions Wisely

Your distribution choice fundamentally shapes results:

Gaussian when you have temporal locality:

• Session stores (recent sessions hot)
• Time-series databases (recent data accessed)
• Rolling analytics (current window active)

Zipfian when you have persistent hot keys:

• Content caching (popular items stay popular)
• User data (power users dominate traffic)
• E-commerce (bestsellers get most views)

Random when you need worst-case analysis:

• Capacity planning (maximum memory pressure)
• Stress testing (minimum cache effectiveness)
• Establishing performance floors

Sequential when you need predictability:

• Baseline measurements
• Configuration comparison
• Debugging and development

Matching Real Traffic Patterns

Study your production metrics before configuring:

1. Measure your read-write ratio from application logs
2. Sample actual value sizes from your database
3. Identify hot keys through cache hit rate analysis
4. Understand temporal patterns in your access logs

Then translate these observations into configuration. A workload that mirrors production gives you results you can trust.

Starting Simple, Adding Complexity

Begin with simple configurations:

• Fixed sizes
• Single operation type
• Sequential keys

Establish baselines, then add complexity:

• Add operation variety
• Introduce size variation
• Switch to realistic distributions

Each addition reveals new insights. Compare results to understand what each aspect contributes.

See Also

• Redis Protocol - Detailed Redis operation guide
• Transport Configuration - Network settings
• Protocol Configuration - Protocol selection
• Configuration Schema - Complete reference

Transport Configuration

Transport is explicitly specified in the [target] section along with the connection address.

Available Transports

• tcp - TCP/IP connections
• udp - UDP connections
• unix - Unix domain sockets

Configuration

TCP (hostname:port)

[target]
protocol = "redis"
transport = "tcp"
address = "localhost:6379"

Unix Domain Socket (filesystem path)

[target]
protocol = "redis"
transport = "unix"
address = "/var/run/redis/redis.sock"

UDP (hostname:port)

UDP support depends on the protocol implementation.

[target]
protocol = "memcached-binary"
transport = "udp"
address = "localhost:11211"

See the Architecture - Transport Layer for implementation details.

Protocol Configuration

Configure the protocol in the [target] section of your TOML profile.

Specifying Protocol

[target]
protocol = "redis"  # or "http", "memcached-binary", "memcached-ascii", "masstree", "xylem-echo"
address = "localhost:6379"

Supported Protocols

• redis - Redis Serialization Protocol (RESP)
• http - HTTP/1.1
• memcached-binary - Memcached binary protocol
• memcached-ascii - Memcached text protocol
• masstree - Masstree protocol
• xylem-echo - Echo protocol (testing/development only)

See the Architecture - Protocol Layer for implementation details.

Protocols

Xylem supports multiple application protocols for different types of RPC workloads.

Supported Protocols

• Redis - Redis protocol for key-value operations
• HTTP - HTTP/1.1 protocol for web services
• Memcached - Memcached protocol for caching

Protocol Selection

Specify the protocol in your TOML profile configuration:

[target]
protocol = "redis"
address = "localhost:6379"

Protocol Architecture

All protocols in Xylem implement a common interface that provides:

1. Request Generation - Convert logical operations to wire format
2. Response Parsing - Parse responses from the server
3. State Management - Track request-response correlation
4. Error Handling - Detect and report protocol errors

See Also

• Configuration

Redis Protocol

Redis uses the RESP (Redis Serialization Protocol) for client-server communication. This document explains the protocol format, common commands, and how Xylem supports benchmarking Redis workloads.

What Xylem Supports

Xylem implements RESP and supports:

• Standard Redis commands (GET, SET, INCR, MGET, WAIT)
• Custom command templates for any Redis operation
• Request pipelining for high throughput
• Configurable key distributions and value sizes

Xylem does not currently support:

• RESP3 protocol (only RESP2) - TODO: RESP3 adds semantic types (maps, sets, doubles, booleans), push messages, and streaming data. While RESP2 covers all benchmarking needs and works with all Redis versions, RESP3 support could be added for testing RESP3-specific features or comparing protocol performance.
• Pub/Sub commands
• Cluster mode (supports single-instance and simple replication)
• Transactions (MULTI/EXEC)

Protocol Overview

Redis commands are sent as text-based protocol messages. The key characteristics:

Text-based - Human-readable format using printable ASCII characters and CRLF line endings

Binary-safe - Bulk strings can contain any binary data with explicit length encoding

Request-response - Client sends command, server sends response (pipelining allows multiple pending requests)

Stateless - Each command is independent (except for transactions and pub/sub)

RESP Wire Format

RESP (REdis Serialization Protocol) is Redis’s wire protocol. All commands and responses use this format.

RESP Data Types

RESP has five data types, each identified by its first byte:

Simple Strings - Start with +, terminated by \r\n

+OK\r\n

Used for status replies.

Errors - Start with -, terminated by \r\n

-ERR unknown command\r\n

Used for error messages.

Integers - Start with :, terminated by \r\n

:42\r\n

Used for numeric results (INCR, DECR, counter values).

Bulk Strings - Start with $, followed by length and data

$5\r\nhello\r\n

• $5 - Length in bytes
• hello - Actual data
• \r\n - Terminator

Null bulk string (key not found):

$-1\r\n

Arrays - Start with *, followed by element count

*3\r\n$5\r\nhello\r\n$5\r\nworld\r\n:42\r\n

• *3 - Array with 3 elements
• Three elements: two bulk strings and one integer

Request Format

All Redis commands are sent as RESP arrays of bulk strings.

GET command:

*2\r\n$3\r\nGET\r\n$8\r\nkey:1234\r\n

• *2 - Array with 2 elements
• $3\r\nGET\r\n - Command name (3 bytes)
• $8\r\nkey:1234\r\n - Key name (8 bytes)

SET command:

*3\r\n$3\r\nSET\r\n$8\r\nkey:1234\r\n$128\r\n<128 bytes>\r\n

• *3 - Array with 3 elements
• Command: SET
• Key: key:1234
• Value: 128 bytes of data

MGET command (multiple keys):

*11\r\n$4\r\nMGET\r\n$8\r\nkey:1000\r\n$8\r\nkey:1001\r\n...<8 more keys>...\r\n

• *11 - Command + 10 keys = 11 elements
• MGET followed by 10 bulk strings

INCR command:

*2\r\n$4\r\nINCR\r\n$7\r\ncounter\r\n

Response Format

Response type depends on the command:

GET (successful):

$128\r\n<128 bytes of data>\r\n

Returns bulk string with value.

GET (key not found):

$-1\r\n

Returns null bulk string.

SET:

+OK\r\n

Returns simple string for success.

INCR:

:43\r\n

Returns integer (new counter value).

MGET:

*3\r\n$5\r\nvalue1\r\n$5\r\nvalue2\r\n$-1\r\n

Returns array with:

• Two values (bulk strings)
• One null (missing key)

WAIT:

:2\r\n

Returns integer (number of replicas that acknowledged).

Common Redis Commands

GET key

Retrieves the value of a key.

Request: GET key:1234 Response: Bulk string with value, or null if key doesn’t exist Use case: Read operations, cache lookups

SET key value [options]

Sets a key to hold a string value.

Request: SET key:1234 <data> Response: +OK on success Options: Can include EX/PX for expiration, NX/XX for conditional sets Use case: Write operations, cache updates

INCR key

Increments the integer value of a key by one.

Request: INCR counter:visits Response: Integer (new value after increment) Behavior: Creates key with value 0 if it doesn’t exist, then increments Use case: Counters, rate limiting, analytics

DECR key

Decrements the integer value of a key by one.

Request: DECR counter:stock Response: Integer (new value after decrement) Use case: Inventory tracking, quota management

MGET key [key …]

Returns values of all specified keys.

Request: MGET key:1 key:2 key:3 Response: Array of bulk strings (null for missing keys) Efficiency: Single round-trip for multiple keys Use case: Batch reads, fetching related data

WAIT numreplicas timeout

Blocks until all previous write commands are replicated to at least numreplicas.

Request: WAIT 2 1000 (wait for 2 replicas, timeout 1000ms) Response: Integer (number of replicas that acknowledged) Use case: Testing replication lag, consistency verification Note: Only meaningful in replicated Redis setups

Custom Commands

Redis supports hundreds of commands for different data structures:

Lists: LPUSH, RPUSH, LPOP, RPOP, LRANGE Sets: SADD, SREM, SMEMBERS, SINTER Sorted Sets: ZADD, ZREM, ZRANGE, ZRANGEBYSCORE Hashes: HSET, HGET, HMGET, HGETALL Strings: APPEND, STRLEN, GETRANGE, SETRANGE

Xylem supports these through custom command templates (see Command Selection section below).

Pipelining

Redis supports pipelining: sending multiple commands without waiting for responses.

Benefits:

• Reduces round-trip latency
• Increases throughput
• Efficient use of network bandwidth

How it works:

Client sends:    GET key1    GET key2    GET key3
                 ─────────────────────────────────>
Server responds: <value1>    <value2>    <value3>
                 <─────────────────────────────────

Xylem support: Configure max_pending_per_connection to control pipeline depth.

Benchmarking with Xylem

Xylem allows you to configure various aspects of Redis workloads:

Basic Configuration

[target]
protocol = "redis"
address = "127.0.0.1:6379"
transport = "tcp"

Command Selection

Single command:

[workload.operations]
strategy = "fixed"
operation = "get"  # Options: get, set, incr

Mixed workload with weighted distribution:

[workload.operations]
strategy = "weighted"

[[workload.operations.commands]]
name = "get"
weight = 0.7  # 70% reads

[[workload.operations.commands]]
name = "set"
weight = 0.3  # 30% writes

Batch operations (MGET):

[[workload.operations.commands]]
name = "mget"
weight = 0.2
count = 10  # Fetch 10 keys per request

Replication testing (WAIT):

[[workload.operations.commands]]
name = "wait"
weight = 0.1
num_replicas = 2
timeout_ms = 1000

Custom commands with templates:

[[workload.operations.commands]]
name = "custom"
weight = 0.1
template = "ZADD leaderboard __value_size__ player:__key__"

Template variables: __key__, __data__, __value_size__

Key Distribution

Sequential access:

[workload.keys]
strategy = "sequential"
start = 0
max = 100000

Uniform random:

[workload.keys]
strategy = "random"
max = 1000000

Zipfian (hot-key patterns):

[workload.keys]
strategy = "zipfian"
exponent = 0.99
max = 1000000

Gaussian (temporal locality):

[workload.keys]
strategy = "gaussian"
mean_pct = 0.5
std_dev_pct = 0.1
max = 10000

Value Size Distribution

Fixed size:

[workload.value_size]
strategy = "fixed"
size = 128

Uniform distribution:

[workload.value_size]
strategy = "uniform"
min = 64
max = 4096

Normal distribution:

[workload.value_size]
strategy = "normal"
mean = 512.0
std_dev = 128.0
min = 64
max = 4096

Per-command sizes:

[workload.value_size]
strategy = "per_command"
default = { strategy = "fixed", size = 256 }

[workload.value_size.commands.get]
distribution = "fixed"
size = 64

[workload.value_size.commands.set]
distribution = "uniform"
min = 128
max = 1024

Complete Example

[experiment]
name = "redis-realistic-workload"
description = "Mixed operations with hot keys and varied sizes"
duration = "60s"
seed = 42  # Reproducible results

[target]
address = "127.0.0.1:6379"
protocol = "redis"
transport = "tcp"

[workload]
# Hot-key pattern: Zipfian distribution
[workload.keys]
strategy = "zipfian"
exponent = 0.99
max = 1000000
value_size = 512

# 70% reads, 30% writes
[workload.operations]
strategy = "weighted"

[[workload.operations.commands]]
name = "get"
weight = 0.7

[[workload.operations.commands]]
name = "set"
weight = 0.3

# Per-command value sizes
[workload.value_size]
strategy = "per_command"
default = { strategy = "fixed", size = 256 }

[workload.value_size.commands.get]
distribution = "fixed"
size = 64

[workload.value_size.commands.set]
distribution = "normal"
mean = 512.0
std_dev = 128.0
min = 128
max = 2048

[workload.pattern]
type = "constant"
rate = 10000.0

[[traffic_groups]]
name = "redis-benchmark"
threads = [0]
connections_per_thread = 20
max_pending_per_connection = 10  # Pipelining depth

[traffic_groups.policy]
type = "closed-loop"

[traffic_groups.sampling_policy]
type = "limited"
rate = 1.0
max_samples = 100000

[output]
format = "json"
file = "results/redis-benchmark.json"

Common Workload Patterns

Cache workload (CDN, API responses):

• Commands: 90-95% GET, 5-10% SET
• Keys: Zipfian (persistent hot content)
• Sizes: Small reads (64B), larger writes (256-1024B)

Session store:

• Commands: 70% GET, 30% SET
• Keys: Gaussian (temporal locality)
• Sizes: Small lookups (64B), varied session data (128-1024B)

Counter system (rate limiting, analytics):

• Commands: 50% GET, 50% INCR
• Keys: Random or uniform
• Sizes: Small fixed (64B)

Write-heavy (logging, time-series):

• Commands: 20-30% GET, 70-80% SET
• Keys: Sequential or temporal
• Sizes: Uniform or normal distribution

See Also

• Workload Configuration - Detailed configuration guide
• Configuration Schema - Complete reference
• Redis Documentation - Official Redis protocol documentation

HTTP Protocol

TODO: This section is under development. HTTP protocol documentation will be added once the configuration format is validated.

Overview

Xylem supports HTTP/1.1 protocol for benchmarking web services and APIs.

Supported Methods

• GET
• POST
• PUT
• DELETE
• HEAD
• PATCH

Check back later for detailed configuration examples and usage guidance.

Memcached Protocol

Note: This page is TODO. Configuration details need validation against actual implementation.

Masstree Protocol

TODO: This page requires validation of TOML configuration format and protocol-specific details.

The Masstree protocol implementation will be documented here once the TOML configuration format is validated.

See Also

• Protocol Configuration
• Architecture: Protocol Layer

Xylem Echo Protocol

Note: This is a testing and development protocol, not intended for production use.

TODO: This page requires validation of TOML configuration format and protocol-specific details.

The xylem-echo protocol is used for testing and development purposes. Full documentation will be added once the TOML configuration format is validated.

See Also

• Protocol Configuration
• Architecture: Protocol Layer

Transports

Xylem supports multiple transport layers for network communication.

Supported Transports

• TCP - Reliable byte stream transport
• UDP - Unreliable datagram transport
• Unix Domain Sockets - Local inter-process communication

Transport Selection

Specify the transport explicitly in your TOML profile configuration:

[target]
protocol = "redis"
transport = "tcp"
address = "localhost:6379"

# or, for a Unix socket:
[target]
protocol = "redis"
transport = "unix"
address = "/var/run/redis.sock"

See Also

• Transport Configuration
• Architecture

TCP Transport

Note: This page is TODO. Configuration details need validation against actual implementation.

UDP Transport

Note: This page is TODO. Configuration details need validation against actual implementation.

Unix Domain Sockets

Note: This page is TODO. Configuration details need validation against actual implementation.

Output Formats

TODO: This section is under development. Output format configuration and examples will be added once the implementation is validated.

Planned Support

Xylem will support multiple output formats for results and metrics:

• Text (human-readable)
• JSON (machine-readable)
• CSV (for analysis)

Check back later for detailed documentation on output configuration.

Architecture Overview

Xylem is designed with a modular, layered architecture that separates concerns and enables extensibility.

High-Level Architecture

┌─────────────────────────────────────────┐
│           CLI Interface                 │
│            (xylem-cli)                  │
└─────────────────────────────────────────┘
                     │
                     ▼
┌─────────────────────────────────────────┐
│         Core Engine                     │
│       (xylem-core)                      │
│  - Workload Management                  │
│  - Statistics Collection                │
│  - Threading & Event Loop               │
└─────────────────────────────────────────┘
         │                    │
         ▼                    ▼
┌─────────────────┐  ┌──────────────────┐
│  Protocols - X  │  │  Transports - Y  │
│(xylem-protocols)│  │ (xylem-transport)│
│  - Redis        │  │   - TCP          │
│  - HTTP         │  │   - UDP          │
│  - Memcached    │  │   - Unix Socket  │
│  - Masstree     │  │                  │
│  - xylem-echo   │  │                  │
└─────────────────┘  └──────────────────┘

Design Principles

Xylem’s architecture follows several core design principles. The system employs a modular design where each component has a well-defined responsibility: the CLI layer handles user interaction and TOML configuration parsing, the core engine orchestrates workload generation and statistics collection, protocol implementations handle message encoding/decoding, and the transport layer manages network communication.

The architecture enables composability through clean interfaces. Protocols and transports can be combined freely - Redis can run over TCP, UDP, or Unix domain sockets; HTTP over TCP or Unix sockets; and Memcached over any supported transport. This flexibility allows Xylem to adapt to different testing scenarios without code modifications.

Key Components

Core Engine

The core engine handles workload generation, connection management, statistics collection, and event loop orchestration.

Protocol Layer

Implements application protocols through request encoding, response parsing, and protocol state management.

Transport Layer

Handles network communication including connection establishment, data transmission, and error handling.

Data Flow

The system processes requests through a defined pipeline. Users provide workload configuration via TOML profile files. The core engine reads the configuration during initialization to instantiate the appropriate protocols and transports. During execution, the event loop generates requests according to the configured pattern and collects responses as they arrive. Statistics are gathered throughout the run, tracking latency, throughput, and error rates. Upon completion, results are formatted according to the output configuration and presented to the user.

See Also

• Protocol Layer
• Transport Layer

Protocol Layer

The protocol layer (xylem-protocols) implements application-level protocols for RPC workload generation and measurement.

Protocol Trait

All protocol implementations conform to the Protocol trait interface:

#![allow(unused)]
fn main() {
pub trait Protocol: Send {
    type RequestId: Eq + Hash + Clone + Copy + Debug;
    
    /// Generate a request with an ID
    fn generate_request(
        &mut self,
        conn_id: usize,
        key: u64,
        value_size: usize,
    ) -> (Vec<u8>, Self::RequestId);
    
    /// Parse a response and return the request ID
    fn parse_response(
        &mut self,
        conn_id: usize,
        data: &[u8],
    ) -> Result<(usize, Option<Self::RequestId>)>;
    
    /// Protocol name
    fn name(&self) -> &'static str;
    
    /// Reset protocol state
    fn reset(&mut self);
}
}

The trait provides a uniform interface for protocol implementations, enabling the core engine to interact with any protocol through a consistent API.

Protocol Design

Protocol implementations are designed to minimize state and memory overhead. Each implementation encodes requests in the appropriate wire format and parses responses into a form suitable for latency measurement. The RequestId mechanism enables request-response correlation, which is required for accurate latency measurement in scenarios involving pipelining or out-of-order responses.

Supported Protocols

The following protocol implementations are available:

• Redis - RESP (Redis Serialization Protocol) for key-value operations
• HTTP - HTTP/1.1 for web service testing
• Memcached - Binary and text protocol variants for cache testing

Detailed documentation for each protocol is available in the Protocols section of the User Guide.

Request-Response Correlation

The protocol layer maintains state necessary for correlating responses with their originating requests. This is required for protocols supporting pipelining or scenarios where multiple requests are in flight concurrently. Each protocol defines an appropriate RequestId type for its correlation semantics.

See Also

• Protocol implementations in User Guide
• Architecture Overview
• Transport Layer

Transport Layer

The transport layer (xylem-transport) provides network communication primitives for Xylem.

Transport Trait

All transport implementations conform to the Transport trait interface:

#![allow(unused)]
fn main() {
pub trait Transport: Send {
    /// Connect to the target
    fn connect(&mut self) -> Result<()>;
    
    /// Send data
    fn send(&mut self, data: &[u8]) -> Result<usize>;
    
    /// Receive data
    fn recv(&mut self, buf: &mut [u8]) -> Result<usize>;
    
    /// Close the connection
    fn close(&mut self) -> Result<()>;
    
    /// Check if connected
    fn is_connected(&self) -> bool;
}
}

The trait provides a uniform interface for transport mechanisms, enabling the core engine to perform network operations independently of the underlying transport implementation.

Transport Design

Transport implementations are lightweight wrappers around operating system network primitives. Each implementation handles protocol-specific communication details while presenting a consistent interface to higher layers. The design prioritizes efficiency and correctness, with explicit error handling and resource management.

Supported Transports

The following transport implementations are available:

• TCP - Connection-oriented byte streams using TCP sockets
• UDP - Connectionless datagram communication
• Unix Domain Sockets - Local inter-process communication

Detailed documentation for each transport is available in the Transports section of the User Guide.

Non-Blocking I/O

All transport implementations support non-blocking operation for integration with the event-driven architecture. The core engine uses mio for I/O multiplexing, enabling a single thread to manage multiple concurrent connections. Transports register file descriptors with the event loop and respond to readiness notifications.

Connection Management

The transport layer provides primitives for connection lifecycle management. The core engine implements higher-level functionality such as connection pooling and reconnection strategies. This separation maintains transport implementation simplicity while allowing sophisticated connection management policies in the core.

See Also

• Transport implementations in User Guide
• Architecture Overview
• Protocol Layer

Configuration Schema

Xylem uses TOML configuration files to define experiments. This reference documents the complete schema for experiment profiles.

Overview

A complete Xylem configuration file contains the following sections:

[experiment]      # Experiment metadata and duration
[target]          # Target service address, protocol, and transport
[workload]        # Workload generation parameters
[[traffic_groups]] # One or more traffic generation groups
[output]          # Output format and destination

Experiment Section

The [experiment] section defines metadata and experiment-wide settings.

[experiment]
name = "redis-bench"
description = "Redis benchmark with latency/throughput agent separation"
duration = "30s"
seed = 42

Fields

• name (string, required): Name of the experiment
• description (string, optional): Description of the experiment
• duration (string, required): Duration in format “Ns”, “Nm”, “Nh” (seconds, minutes, hours)
• seed (integer, optional): Random seed for reproducibility

Target Section

The [target] section specifies the target service to benchmark.

[target]
address = "127.0.0.1:6379"
protocol = "redis"
transport = "tcp"

Fields

• address (string, required): Target address. Format depends on transport:
  • TCP/UDP: "host:port" or "ip:port"
  • Unix socket: "/path/to/socket"
• protocol (string, required): Application protocol. Supported values:
  • "redis"
  • "http"
  • "memcached-binary"
  • "memcached-ascii"
  • "masstree"
  • "xylem-echo" (testing only)
• transport (string, required): Transport layer. Supported values:
  • "tcp"
  • "udp"
  • "unix"

Workload Section

The [workload] section defines the workload pattern and key distribution.

[workload.keys]
strategy = "zipfian"
n = 1000000
theta = 0.99
value_size = 64

[workload.pattern]
type = "constant"
rate = 50000.0

Fields

[workload.keys] - Key distribution parameters:

• strategy (string, required): Distribution strategy
  • "uniform": Uniform distribution
  • "zipfian": Zipfian distribution (power law)
  • "random": Random distribution
• n (integer, required for uniform/zipfian): Key space size (total number of keys)
• max (integer, required for random): Maximum key value
• theta (float, required for zipfian): Skew parameter (0.0 to 1.0)
  • 0.99 = high skew (typical for caches)
  • 0.5 = moderate skew
• value_size (integer, required): Size of values in bytes

[workload.pattern] - Traffic pattern:

• type (string, required): Pattern type
  • "constant": Constant rate
  • (other types may be supported)
• rate (float, required): Target request rate in requests/second

Traffic Groups

Traffic groups define how workload is distributed across threads and connections. You can define multiple [[traffic_groups]] sections.

[[traffic_groups]]
name = "latency-agent"
protocol = "redis"
threads = [0]
connections_per_thread = 10
max_pending_per_connection = 1

[traffic_groups.sampling_policy]
type = "unlimited"

[traffic_groups.policy]
type = "poisson"
rate = 100.0

Fields

• name (string, required): Name for this traffic group
• protocol (string, optional): Override protocol for this group (defaults to [target] protocol)
• threads (array of integers, required): Thread IDs to use for this group
• connections_per_thread (integer, required): Number of connections per thread
• max_pending_per_connection (integer, required): Maximum pending requests per connection
  • 1 = no pipelining (accurate latency measurement)
  • Higher values = pipelining enabled

Sampling Policy

[traffic_groups.sampling_policy]:

• type (string, required):
  • "unlimited": Sample every request (100% sampling)
  • "limited": Sample a fraction of requests
• rate (float, required for limited): Sampling rate (0.0 to 1.0)
  • 0.01 = 1% sampling
  • 0.1 = 10% sampling

Traffic Policy

[traffic_groups.policy]:

• type (string, required):
  • "poisson": Poisson arrival process (open-loop)
  • "closed-loop": Closed-loop (send as fast as possible)
• rate (float, required for poisson): Request rate per connection in requests/second

Output Section

The [output] section configures where and how results are written.

[output]
format = "json"
file = "/tmp/results.json"

Fields

• format (string, required): Output format
  • "json": JSON format
  • (other formats may be supported)
• file (string, required): Output file path

Complete Example

[experiment]
name = "redis-bench"
description = "Redis benchmark"
duration = "30s"
seed = 42

[target]
address = "127.0.0.1:6379"
protocol = "redis"
transport = "tcp"

[workload.keys]
strategy = "zipfian"
n = 1000000
theta = 0.99
value_size = 64

[workload.pattern]
type = "constant"
rate = 50000.0

[[traffic_groups]]
name = "latency-agent"
protocol = "redis"
threads = [0]
connections_per_thread = 10
max_pending_per_connection = 1

[traffic_groups.sampling_policy]
type = "unlimited"

[traffic_groups.policy]
type = "poisson"
rate = 100.0

[[traffic_groups]]
name = "throughput-agent"
protocol = "redis"
threads = [1, 2, 3]
connections_per_thread = 25
max_pending_per_connection = 32

[traffic_groups.sampling_policy]
type = "limited"
rate = 0.01

[traffic_groups.policy]
type = "closed-loop"

[output]
format = "json"
file = "/tmp/redis-bench-results.json"

See Also

• Configuration Guide
• CLI Reference
• Example Profiles