Cbench

Scalable Cluster Benchmarking and Testing Toolkit

Cbench is a framework for building, running, and parsing HPC benchmarks across Linux parallel compute clusters. It automates the full lifecycle: generating job scripts, submitting them to a batch scheduler, and parsing the resulting output into structured results.

Originally developed at Sandia National Laboratories (v1.3.0, 2013). Version 2.0.0 adds a modern Python toolchain alongside the original Perl scripts.

Repository layout

cbench/
├── cbench.pl               # Core Perl library (job launchers, schedulers, template engine)
├── cluster.def             # Perl cluster configuration (legacy)
├── cluster.yaml            # YAML cluster configuration (Python toolchain)
├── tools/                  # Perl scripts: gen_jobs, start_jobs, output_parse, ...
├── sbin/                   # Orchestration wrappers
├── templates/              # Job script templates (*.in)
├── perllib/
│   ├── output_parse/       # Benchmark-specific Perl parsers (31 benchmarks)
│   ├── parse_filter/       # Custom error-detection filter modules
│   ├── gen_jobs/           # Job-generation hook modules
│   └── hw_test/            # Single-node hardware test modules
├── python/                 # ← Modern Python toolchain (see below)
│   ├── pyproject.toml
│   ├── cbench/             # Installable Python package
│   └── tests/
├── doc/                    # Full documentation (see doc/README)
└── opensource/             # Bundled benchmark sources

openapps/                   # Scientific application sources (AMG, LAMMPS, Trilinos, ...)
fixit123/                   # Node breakfix and qualification framework

Quick start — Python toolchain

The Python toolchain (cbench/python/) is a pip-installable package that replaces the core Perl scripts with a single cbench CLI. The original Perl scripts remain intact and fully functional.

Requirements

Python 3.9+
click, pyyaml, rich, jinja2, jsonschema (installed automatically)

Install from RPM or DEB (recommended for clusters)

Download the package and GPG public key from the GitHub Releases page.

RHEL / Rocky / Fedora — with GPG verification (recommended):

# Import the signing key once, then install
sudo rpm --import RPM-GPG-KEY-cbench
sudo dnf install ./cbench-<version>-1.noarch.rpm

RHEL / Rocky / Fedora — skip GPG check (air-gapped or quick install):

sudo dnf install --nogpgcheck ./cbench-<version>-1.noarch.rpm

Ubuntu / Debian:

sudo apt install ./cbench_<version>-1_all.deb

Install from source (development)

pip install -e cbench/python/

Configure

Copy and edit the cluster configuration:

cp cbench/cluster.yaml my_cluster.yaml
# Edit: max_nodes, procs_per_node, batch_method, joblaunch_method, etc.
export CBENCHOME=/path/to/cbench/cbench
export CBENCHTEST=/path/to/your/test/tree

Build benchmark software

# List available builders
cbench build list

# Build a single benchmark (downloads source, compiles, installs to $CBENCHTEST/bin/)
cbench build run stream
cbench build run imb --mpicc mpicc
cbench build run ior --jobs 8
cbench build run hpl --blas-lib "-lopenblas"
cbench build run npb --extra class=C

# Build everything
cbench build all --mpicc mpicc --jobs 8 --blas-lib "-lopenblas"

# Build everything using 4 parallel workers
cbench build all --mpicc mpicc --jobs 8 --parallel 4

# Preview without downloading or compiling
cbench build run ior --dry-run

# Show build status (cached / not cached) for all builders
cbench build list --prefix $CBENCHTEST

# Pull upstream changes for git-cloned sources and rebuild only if updated
cbench build update
cbench build update imb   # update a single benchmark

# Report newer upstream versions without downloading anything: new commits on
# git sources (plus the latest release tag) and newer pinned tarballs
cbench build check-updates
cbench build check-updates npb osu

Note: cbench build update only updates git-cloned sources (most builders); a tarball or single-file source (stream) always reports "source unchanged". Use cbench build run <name> --force to re-download and rebuild one. Tarball versions are pinned in the builders: cbench build check-updates shows when a newer one is out, and moving to it is a code change.

Available builders: stream, imb (Intel MPI Benchmarks), osu (OSU MPI Micro-Benchmarks), ior (IOR + mdtest), hpl (HPL Linpack — requires BLAS), hpcc (HPC Challenge — requires BLAS), npb (NAS Parallel Benchmarks), amg (LLNL AMG), hpccg (Mantevo HPCCG), mpibench (LLNL mpiBench), mpigraph (LLNL mpiGraph), graph500, bonnie (Bonnie++), iozone, fio, io500 (IO500 — needs network, MPI and autotools), gpfsperf (optional — built from the GPFS samples, the single-node binary ships with GPFS, so the build is needed for gpfsperf-mpi (built when mpicc is found) or a custom variant; GPFS RDMA needs no special build; build all skips it on hosts without GPFS).

Sources are cloned/downloaded to $CBENCHTEST/src/ and binaries are installed to $CBENCHTEST/bin/.

Probe the compute nodes (node facts)

gen-jobs runs on the submit host, so it learns the compute nodes' CPU count, memory and IO-target filesystems from a facts file written by nodecheck:

# probe every node (pdsh -R ssh|exec, or ssh), check the pool is homogeneous,
# write $CBENCHTEST/nodefacts/<name>.json
cbench nodecheck --nodelist 'n[1-64]' --name compute
cbench nodecheck --partition batch --allow-heterogeneous

nodecheck also records each node's interconnect and GPFS transport, without needing root:

Interconnect: the link layers of the active RDMA ports in /sys/class/infiniband (ib, roce), else tcp.
GPFS transport: rdma when /var/mmfs/gen/mmfs.cfg enables verbsRdma and one of its verbsPorts is active, else tcp. This is the configured setting, not a live check.

A node that differs from the rest (e.g. one fell back to TCP) fails the check unless you pass --allow-heterogeneous. Jobs generated with --nodefacts print CBENCH LABEL: interconnect=ib gpfs_transport=rdma, and cbench parse keeps that in each result's status_detail.

IO tests write to named targets set in cluster.yaml:

io_targets:
  parallel: /gpfs/scratch     # IOR, mdtest, io500 (and gpfsperf if it is GPFS)
  node-local: /tmp            # fio, bonnie, iozone
  # gpfs: /gpfs/other         # optional explicit gpfsperf target

Generate job scripts

cbench gen-jobs --testset bandwidth --ident run1
cbench gen-jobs --testset linpack   --ident run1 --ppn 4,8 --maxprocs 256 --nodefacts compute
cbench gen-jobs --testset bandwidth --ident run1 --dry-run   # preview without writing
cbench gen-jobs --testset iometadata --ident run1 --match '^fio-'   # only matching job names

Job scripts are written to $CBENCHTEST/<testset>/<ident>/<jobname>/. IO and HPL testsets size themselves from --nodefacts (or from values set explicitly in cluster.yaml). Without either, they stop before writing anything.

IO profiles

A profile bundles IO benchmarks into one testset directory ($CBENCHTEST/<profile>/<ident>/), so start-jobs and parse take --testset <profile>:

cbench gen-jobs --profile io-default --ident io1 --nodefacts compute            # node-local: fio, bonnie, iozone
cbench gen-jobs --profile io-default --ident io1 --group parallel --group gpfs  # ior + mdtest; gpfsperf
cbench gen-jobs --profile io-default --ident io1 --group gpfs-mpi --nodefacts compute  # gpfsperf-mpi, one job per node count
cbench gen-jobs --profile io-default --ident io1 --group gpfs-node --nodefacts compute # gpfsperf -dio on each node, one at a time
cbench gen-jobs --profile io-default --ident io1 --group all
cbench gen-jobs --profile io500      --ident io1 --io500-stonewall 300          # one job per node count
cbench gen-jobs --profile io-default --ident quick --fio-runtime 30             # short fio runs

You can define your own profiles in cluster.yaml. In a custom group, the group's target decides where every member writes. Sizing and free-space caps follow that target. So you can run fio against GPFS, or point benchmarks at an extra target such as a local NVMe:

io_targets:
  parallel: /gpfs/scratch
  nvme: /local/nvme
io_profiles:
  fast-local:
    description: fio and IOR on the NVMe scratch; fio on GPFS on request
    default_groups: [nvme]                        # default: every group
    groups:
      nvme:
        target: nvme                              # an io_targets key
        members: [iometadata_fio, io_ior1mNtoN]   # <testset>_<benchmark> templates
      gpfs-small:
        target: parallel
        suffix: gsmall                            # job names fio-gsmall-...; default: group name
        members: [iometadata_fio]

cbench gen-jobs --profile fast-local --group all generates both groups. A custom profile can't reuse a built-in name (io-default, io500).

The gpfs-node group runs gpfsperf on each node in turn: the nodes from --nodefacts, or --nodelist 'n[1-8]'. It uses direct I/O (-dio) on a bounded file (gpfsperf_node_size_mib, default 4 GiB), so each node takes minutes rather than the 2×-RAM run's half hour. cbench parse prints a per-node table and flags any node more than --outlier-pct (default 10%) worse than the median. Jobs run one at a time, by Slurm --dependency=singleton or the interactive loop, unless you use gen-jobs --concurrent (batch) or start-jobs --interactive --concurrent. Without a scheduler, each job runs gpfsperf on its node over ssh.

Sizing rules:

Data size: buffered tests write 2× RAM so the page cache can't hold the data, capped to 90% of the target's free space with a CBENCH CAVEAT.
Concurrency: every IO benchmark uses all CPUs. Set io_threads_max to cap it, or io_threads_basis: physical to count cores instead of logical CPUs.
Sequential sizes: fio's sequential block size, iozone and gpfsperf record sizes, and IOR -t follow io_profile: ai 1m, general 4m, hpc 8m, streaming 16m. The default auto picks hpc on parallel filesystems and general elsewhere. When the filesystem's block size (or Lustre stripe size) is known, auto moves up to the smallest profile whose size is a whole multiple of it, so transfers never split a block: e.g. 16 MiB GPFS blocks get streaming (16m). cbench nodecheck prints what auto picks for each target and warns if an explicit io_profile/io_seq_bs doesn't fit; gen-jobs prints a NOTE when auto moved up. An explicit io_profile or io_seq_bs is never changed.
IOPS: random/IOPS tests always use 4k transfers.

Single-node jobs are named for their real concurrency (e.g. fio-local-16ppn-16).

Submit jobs

cbench start-jobs --testset bandwidth --ident run1 --batch
cbench start-jobs --testset bandwidth --ident run1 --throttledbatch 20  # keep 20 jobs active
cbench start-jobs --testset bandwidth --ident run1 --interactive
cbench start-jobs --testset bandwidth --ident run1 --interactive --echo-output  # stream job output live

start-jobs fails if a batch submission fails, or if the scheduler queue query fails in throttled mode. With --interactive, jobs that exit nonzero are listed at the end and the command exits 1; the remaining jobs still run.

To run without a batch scheduler (e.g. on a single workstation), set batch_method: local in cluster.yaml. Generated scripts will be plain .sh files and start-jobs runs them directly with bash, one at a time:

# cluster.yaml
batch_method: local
max_nodes: 1
procs_per_node: 8

cbench gen-jobs  --testset bandwidth --ident run1   # writes .sh scripts
cbench start-jobs --testset bandwidth --ident run1 --batch  # runs each with bash

Multi-node MPI without a scheduler

Without a scheduler, mpirun doesn't know which nodes to use, so pass a hostfile and the Open MPI settings in joblaunch_extraargs. This setup ran gpfsperf-mpi on two RHEL 9 nodes with Open MPI 4.1.1 from the distro (/usr/lib64/openmpi) over plain TCP:

# cluster.yaml
max_nodes: 2
procs_per_node: 8
joblaunch_method: openmpi
joblaunch_extraargs: >-
  --hostfile /shared/cbtest/hosts
  --prefix /usr/lib64/openmpi -x LD_LIBRARY_PATH
  --mca oob_tcp_if_include 172.16.0.0/16
  --mca pml ob1 --mca btl self,vader,tcp --mca btl_tcp_if_include 172.16.0.0/16

# /shared/cbtest/hosts
node1 slots=8
node2 slots=8

export PATH=/usr/lib64/openmpi/bin:$PATH LD_LIBRARY_PATH=/usr/lib64/openmpi/lib
cbench gen-jobs   --profile io-default --group gpfs-mpi --ident mpi1 --run-type interactive --nodefacts compute
cbench start-jobs --testset io-default --ident mpi1 --interactive

What each setting does, and the failure it avoids:

Hostfile: under a scheduler, the job's allocation tells mpirun which nodes to use; without one, the hostfile does. Open MPI starts a daemon on every host in it, even for a 1-node job, so every listed node must be reachable.
--prefix and -x LD_LIBRARY_PATH: the remote node finds orted and the MPI libraries without changes to its login PATH.
oob_tcp_if_include: keeps Open MPI's daemons on the network the nodes share. Without it, a node with several interfaces can advertise one the other node can't reach, and the job fails with ORTE was unable to reliably start one or more daemons. cbench parse --customparse openmpi reports that line.
pml ob1, btl self,vader,tcp, btl_tcp_if_include: use Open MPI's own TCP transport on that network. The default UCX transport picks its own interfaces and can fail across nodes with ucp_ep_create ... Destination is unreachable. Leave these out on InfiniBand or RoCE, where UCX is the transport you want.
mpirun starts the remote ranks over ssh, so the first node needs key-based ssh to the others, and the job's files (binaries, data dir) must be on a filesystem every node mounts.

Watching jobs

cbench watch shows each job's state from its .heartbeat file:

cbench watch --testset io-default --ident run1            # one snapshot
cbench watch --testset io-default --ident run1 --follow   # redraw every 10 s until done

running: elapsed time and when the file was last updated.
stale: still marked running, but no update for 3 heartbeat intervals. It was probably killed with SIGKILL, or its node died. --stale-after SECONDS changes the window.
finished / failed: the exit code the job recorded.
not started: no heartbeat file yet.

--follow exits when nothing is running or waiting to start: 0 when no job failed or went stale, else 1, so scripts can wait on it.

Stopping a running job

Job scripts clean up after themselves when they stop:

Data removed: IO scratch data (e.g. /tmp/job<ID>, or a gpfsperf file on GPFS) is removed on a normal exit, on exit, and on SIGINT, SIGTERM or SIGHUP.
Exit recorded: <jobdir>/<job>.heartbeat records exited rc=N, 130 for Ctrl-C and 143 for SIGTERM.
How jobs get stopped: Ctrl-C on start-jobs --interactive sends SIGINT; a scheduler's scancel or time limit sends SIGTERM.

Two limits:

SIGKILL can't be caught. A job killed outright leaves its data, and its .heartbeat still says still running (cbench watch shows it as stale). Some benchmarks' worker processes outlive it: fio's workers run in their own process group and keep writing until you stop them.
A job started with SIGINT ignored never sees Ctrl-C. nohup, or & from a non-interactive shell script, start it that way, and a shell can't catch a signal that was ignored when it started. Stop such a job with SIGTERM (kill <pid>), which still runs the cleanup.

Parse results

cbench parse --testset bandwidth --ident run1
cbench parse --testset bandwidth --ident run1 --output json

Generate a skeleton job script

# Write setvars-1ppn-1.slurm + setvars-1ppn-1.sh to the current directory
cbench make-skel

# Choose a different skeleton template (skeleton_hello.in)
cbench make-skel --skelname hello --ppn 4 --numprocs 16

# Preview without writing
cbench make-skel --skelname setvars --dry-run

# Write to a specific directory
cbench make-skel --skelname setvars --outdir /path/to/newbench/

Available skeletons are the skeleton_*.in files in $CBENCHOME/templates/ (hello, setvars, snb). All TOKEN_HERE substitutions are expanded exactly as they would be in a real cbench gen-jobs run.

Remove failed jobs

# Preview which ERROR job directories would be removed
cbench rm-failed --testset bandwidth --ident run1

# Actually delete them
cbench rm-failed --testset bandwidth --ident run1 --force

# Restrict to a subset of jobs
cbench rm-failed --testset bandwidth --ident run1 --match "xhpl" --force

Results are written to:

Terminal: colored pass/fail table
$CBENCHTEST/<testset>/<ident>/results.json
$CBENCHTEST/cbench_results.db (SQLite)

Query the results database

cbench query --benchmark xhpl
cbench query --cluster mycluster --status PASSED --since 2025-01-01
cbench query --testset bandwidth --output json

# Date range filter
cbench query --since 2025-01-01 --until 2025-06-30

# CSV export (one row per metric per run — easy to pipe into spreadsheets)
cbench query --testset bandwidth --output csv > results.csv

# Aggregate: mean/min/max per metric grouped by benchmark
cbench query --testset bandwidth --aggregate
cbench query --aggregate --output json

Node hardware testing

# Create a test identifier for a set of nodes
cbench nodehwtest gen-jobs --nodelist n[1-10] --ident run1

# Submit one batch job per node (or run via pdsh)
cbench nodehwtest start-jobs --ident run1 --nodebatch
cbench nodehwtest start-jobs --ident run1 --remote

# Parse node_hw_test output files and report statistical outliers
cbench nodehwtest parse --ident run1 --characterize --save-targets baseline
cbench nodehwtest parse --ident run1 --load-targets baseline

# Store parsed metrics to the SQLite DB (queryable via cbench query --testset nodehwtest)
cbench nodehwtest parse --ident run1 --store

Output files are named <node>.node_hw_test.run<NNNN> in $CBENCHTEST/nodehwtest/<ident>/.

Sizing utilities

# Run sizes filtered by various criteria
cbench utils run-sizes --maxprocs 1024 --pof2
cbench utils run-sizes --maxprocs 512  --square
cbench utils run-sizes --maxprocs 2048 --mult 256
cbench utils run-sizes --maxprocs 100  --addr 5    # arithmetic sequence: 5,10,15,...

# Find HPL grid decompositions for N MPI ranks
cbench utils find-pq --nprocs 512 --decent-only

# Compute HPL problem size N from memory and utilization
cbench utils find-n --nprocs 512 --ppn 16 --memory 64000 --util 0.5,0.6,0.7

# Find valid NPB processor counts (power-of-2 and perfect-square)
cbench utils npb-procs --nprocs 500

Diagnose output files

# Scan output files for known error patterns and aggregate by error type
cbench diag file1.out file2.out --filters openmpi,slurm

# Scan an entire testset/ident tree
cbench diag --testset bandwidth --ident run1 --filters slurm,misc

# Show only sources that appeared ≥ 3 times; emit JSON
cbench diag --testset bandwidth --ident run1 --threshold 3 --output json

# Filter by node role (for OMPI src→dst errors)
cbench diag file.out --source-only
cbench diag file.out --source-dest-only

Available filter modules: openmpi, slurm, torque, mvapich, mpiexec, cray, misc.

Single-node benchmarking

# Run the default single-node suite (stream, cachebench, dgemm, mpistreams, linpack, npb, hpcc)
cbench snb run --ident run1 --destdir /scratch/snb

# fio is opt-in and needs explicit targets (O_DIRECT probed per target; IO profile
# sets the sequential block size; data jobs time-capped)
cbench snb run --ident run1 --tests fio --fs-target /tmp --fs-target /gpfs/scratch
cbench snb run --ident run1 --tests fio --fs-target /tmp --io-profile streaming --fio-runtime 60

# Run and store results directly to the SQLite DB
cbench snb run --ident run1 --destdir /scratch/snb --store

# Run a subset of tests
cbench snb run --ident run1 --tests "stream|dgemm" --numcores 32

# Preview commands without executing
cbench snb run --ident run1 --dry-run

# Display a results summary table from saved output files
cbench snb report --ident run1 --destdir /scratch/snb
cbench snb report --ident run1 --node n042   # report for a specific node

# JSON output (suitable for scripts and monitoring)
cbench snb report --ident run1 --output json

# Store results from existing output files into the SQLite DB
cbench snb store --ident run1 --destdir /scratch/snb --node n042

# Compare two runs and flag regressions (reads from DB)
cbench snb compare --ident run2 --baseline run1 --node n042
cbench snb compare --ident run2 --baseline run1 --threshold 10.0

# Dispatch an SNB run to a remote node via ssh/pdsh (shared filesystem required)
cbench snb run --ident run1 --destdir /scratch/snb --remote n042
cbench snb run --ident run1 --destdir /scratch/snb --remote n042 --remote-cbench /opt/cbench/bin/cbench

Output files are written to <destdir>/<ident>/<hostname>.snb.<test>.out. Results are stored in $CBENCHTEST/cbench_results.db (same DB as cbench parse).

Web dashboard

# Install the optional web extra
pip install "cbench[web]"

# Start the dashboard (defaults to http://127.0.0.1:8080/)
cbench serve

# Expose to the network
cbench serve --host 0.0.0.0 --port 9090

# Air-gapped clusters: avoid CDN requests
cbench serve --no-cdn

# Air-gapped with local Bootstrap + Chart.js assets
cbench serve --no-cdn --assets-dir /path/to/assets/
# (assets dir must contain bootstrap.min.css and chart.umd.min.js)

Endpoints: / HTML dashboard, /api/results /api/summary /api/trend JSON API, /metrics Prometheus text exposition format.

Drive cbench from an AI assistant (MCP server)

cbench mcp is a Model Context Protocol server on stdio, so an MCP client such as Claude Code or Claude Desktop can run the whole workflow: probe nodes, edit cluster.yaml, check for missing software, generate, start, watch and parse jobs, and query results.

# Install the optional mcp extra (needs Python 3.10+)
pip install "cbench[mcp]"

# Register it with Claude Code
claude mcp add cbench -e CBENCHTEST=/shared/cbtest -- cbench mcp

Tools:

Kind	Tools
Read only	`status`, `get_config`, `list_testsets`, `node_facts`, `watch_jobs`, `query_results`, `task_status`, `check_deps`, `check_updates` (network lookups)
Need `confirm=true`	`set_config`, `run_nodecheck`, `gen_jobs`, `start_jobs`, `build_benchmark`
Re-derives data	`parse_results` (idempotent: re-parsing overwrites the run's rows)

Consent. A tool that changes anything does nothing unless it's called with confirm=true. Without it, the tool returns the exact cbench command it would run, or the cluster.yaml diff. The server tells the client to show that plan and to confirm only after you agree.
set_config schema-validates the new file before writing it and keeps the old one as cluster.yaml.bak. Comments in the file are not preserved.
check_deps never installs system packages. It lists what's missing (scheduler client, MPI, pdsh, fio, compilers) and suggests the package to install. The only thing it can install is cbench's own benchmarks, through build_benchmark.
Long operations (interactive start_jobs, build_benchmark) run detached and return a task id. task_status reports their state and log tail; their files are under $CBENCHTEST/.cbench-mcp/tasks/.
Commands. Side-effecting tools run the cbench CLI (python -m cbench), so they behave exactly as they do from a shell.

Supported benchmarks

Benchmark	Parser	Key metrics
HPL / Linpack	`xhpl`, `xhpl2`	GigaFlops, runtime
HPCC Suite	`hpcc`	HPL, DGEMM, PTRANS, STREAM, FFT, RandomAccess
NAS Parallel Benchmarks	`npb`	Mop/s
Intel MPI Benchmarks	`imb`	latency µs, bandwidth MB/s
IOR	`ior`, `io`, `iosanity`	write/read MB/s
IO500	`io500`	score, bandwidth GiB/s, IOPS kIOPS
elbencho	`elbencho`	throughput MiB/s, IOPS, latency µs per phase
gpfsperf	`gpfsperf`	throughput MB/s, IOPS, avg latency ms
fio	`fio`	read/write IOPS, BW MiB/s, lat avg/p99 µs
MLPerf Training	`mlperf`, `mlperf-training`	time_to_train s, accuracy, epochs
MLPerf Inference	`mlperf`, `mlperf-inference`	samples/s, latency p90/mean/p99 ns
OSU MPI	`osu`, `mpioverhead`	bandwidth MB/s, latency µs
AMG	`amg`	solver FOM
b_eff	`beff`	bidir bandwidth MB/s
Bonnie++	`bonnie`	sequential/random I/O
COM	`com`	unidir/bidir bandwidth
Graph500	`graph500`	TEPS
HPCCG	`hpccg`	MFlops
IRS	`irs`	FOM, zone time
LAMMPS	`lammps`	ns/day
mdtest	`mdtest`	file/dir metadata ops/s
Miranda	`miranda`	transfer rate MiB/s
mpiBench	`mpibench`, `collective`	collective latency µs
mpiGraph	`mpigraph`	all-pairs send/recv MB/s
phdMesh	`phdmesh`	search/rebalance time
rotate / rotlat	`rotate`, `rotlat`	ring bandwidth/latency
routecheck	`routecheck`	routing validation time
sPPM	`sppm`	HYD/IO cpu+wall time
SQMR	`sqmr`	message rate msgs/s
stress	`stress`, `longstress`	all-to-all aggregate MB/s
SWEEP3D	`sweep3d`	CPU time, grind time
Trilinos Epetra	`trilinos`	SpMV, SpMM, BLAS MFlops
fileop	`fileop`	pass/fail
laten	`laten`	MPI latency µs

Supported schedulers and launchers

Batch schedulers: SLURM, Torque/PBS, PBS Pro, LSF, Moab, Cray CLE Torque, local (no scheduler — runs scripts directly with bash)

MPI launchers: Open MPI (orterun on 4.x and earlier, mpirun on 5.x; picked on the compute node at run time — set CBENCH_OMPI_RUN or joblaunch_cmd to force one), mpiexec, SLURM (srun), yod, ALPS (aprun)

Original Perl toolchain

The original Perl tools remain fully functional. Set CBENCHOME and CBENCHTEST, then:

# Generate jobs (Perl)
$CBENCHOME/sbin/gen_jobs.pl --testset bandwidth --ident run1

# Submit jobs
$CBENCHOME/sbin/start_jobs.pl --testset bandwidth --ident run1 --batch

# Parse output
$CBENCHOME/tools/cbench_output_parse.pl --testset bandwidth --ident run1

For full Perl toolchain documentation see cbench/doc/README and the HOWTOs in cbench/doc/.

Running tests

pip install -e "cbench/python[dev]"
cd cbench/python
python -m pytest tests/ -v

CI runs automatically on Python 3.9–3.12 for every push touching cbench/python/.

Configuration reference

cluster.yaml (or the legacy cluster.def) controls all cluster-specific parameters:

Parameter	Default	Description
`cluster_name`	`test`	Cluster identifier (override with `$CBENCHCLUSTER`)
`max_nodes`	`1`	Total nodes available
`procs_per_node`	`8`	Cores per node
`max_ppn_procs`	auto	Max processes at each ppn level
`memory_per_node_mb`	`2048`	RAM per node in MB
`default_walltime`	`04:00:00`	Default job walltime (HH:MM:SS)
`walltime_method`	`0`	`0`=constant, `1`=stepped by run size
`walltime_steptime`	`10`	Minutes added per run-size step (method 1)
`joblaunch_method`	`openmpi`	MPI launcher: `openmpi`, `mpiexec`, `slurm`, `alps`, `yod`
`batch_method`	`slurm`	Scheduler: `slurm`, `torque`, `pbspro`, `lsf`, `moab`, `local`
`memory_util_factors`	`[0.25, 0.80, 0.85]`	Memory utilization fractions (Linpack)
`parse_filter_include`	see yaml	Error-detection filter modules to load
`job_heartbeat_s`	`60`	Seconds between a job's "still running" updates in `<jobdir>/<job>.heartbeat` (and on the terminal for interactive runs); `0` = off. `gen-jobs --heartbeat` / env `CBENCH_HEARTBEAT` override

License

Cbench is free software licensed under the GNU General Public License v2. Copyright (2005) Sandia Corporation. Under the terms of Contract DE-AC04-94AL85000 with Sandia Corporation, the U.S. Government retains certain rights in this software.

Name		Name	Last commit message	Last commit date
.github/workflows		.github/workflows
cbench		cbench
fixit123		fixit123
openapps		openapps
.gitignore		.gitignore
.pre-commit-config.yaml		.pre-commit-config.yaml
CHANGELOG.md		CHANGELOG.md
CLAUDE.md		CLAUDE.md
README.md		README.md

Navigation Menu

Repository navigation

About

Repository files navigation

Cbench

Repository layout

Quick start — Python toolchain

Requirements

Install from RPM or DEB (recommended for clusters)

Install from source (development)

Configure

Build benchmark software

Probe the compute nodes (node facts)

Generate job scripts

IO profiles

Submit jobs

Multi-node MPI without a scheduler

Watching jobs

Stopping a running job

Parse results

Generate a skeleton job script

Remove failed jobs

Query the results database

Node hardware testing

Sizing utilities

Diagnose output files

Single-node benchmarking

Web dashboard

Drive cbench from an AI assistant (MCP server)

Supported benchmarks

Supported schedulers and launchers

Original Perl toolchain

Running tests

Configuration reference

License

About

Releases

Packages

Contributors

Languages

Latest commit

History

Folders and files

Repository files navigation

Cbench

Repository layout

Quick start — Python toolchain

Requirements

Install from RPM or DEB (recommended for clusters)

Install from source (development)

Configure

Build benchmark software

Probe the compute nodes (node facts)

Generate job scripts

IO profiles

Submit jobs

Multi-node MPI without a scheduler

Watching jobs

Stopping a running job

Parse results

Generate a skeleton job script

Remove failed jobs

Query the results database

Node hardware testing

Sizing utilities

Diagnose output files

Single-node benchmarking

Web dashboard

Drive cbench from an AI assistant (MCP server)

Supported benchmarks

Supported schedulers and launchers

Original Perl toolchain

Running tests

Configuration reference

License

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages