AI synthetic developers that test the developer experience of your software. Give it a git repo URL — it spins up isolated agent containers (one per persona), each clones the repo and evaluates it like a real developer. Findings are deduplicated, scored GREEN/YELLOW/RED, and viewable in the built-in UI. Pick any model your provider serves for each run, and give each persona its own machine image when you want to test on, say, Fedora with Go.
- Define personas — each with an identity, perspective, and constraints (e.g., "backend developer who has never used AI" or "platform engineer evaluating for OpenShift deployment")
- Define a journey — ordered phases like "Read the docs", "Set up locally", "Test the API", "Try deploying"
- Point at a repo — provide a git repository URL, and optionally pick the model and a machine image (environment) for each persona
- Get a report — each persona independently clones the repo, follows the journey phases, and reports findings with evidence. All tools (file reading, command execution, HTTP requests) are always available.
Findings are scored GREEN / YELLOW / RED and stored in a database. A built-in UI shows run history, findings, and reports.
- Docker with Docker Compose v2 (Docker Desktop, or Docker Engine on Linux)
- An LLM key. The easiest is a free
NVIDIA_API_KEYfrom build.nvidia.com.MODELS_CORP_API_KEY+MODELS_CORP_URL,ANTHROPIC_API_KEY, and Vertex AI also work.
git clone https://github.com/mpk-droid/SimCrew.git && cd SimCrew
echo "NVIDIA_API_KEY=nvapi-..." > .env # your key; .env is gitignored
docker compose up --buildOpen http://localhost:8000. The first build takes a few minutes.
Port already taken? Set SIMCREW_PORT=8001 (UI) or SIMCREW_DB_PORT=5433 (Postgres) in .env.
For Vertex AI, run gcloud auth application-default login, put ANTHROPIC_VERTEX_PROJECT_ID and CLOUD_ML_REGION in .env, and start with docker compose -f docker-compose.yml -f docker-compose.vertex.yml up --build.
git clone https://github.com/mpk-droid/SimCrew.git && cd SimCrew
oc new-project simcrew
helm install simcrew ./chart --set secrets.nvidiaApiKey=nvapi-...
oc get route simcrew -o jsonpath='https://{.spec.host}{"\n"}'The chart deploys the orchestrator, a Postgres database, and the RBAC that lets the orchestrator start one agent Job per persona. Agent pods run code from the repos they evaluate, so a NetworkPolicy keeps them off the cluster network. They can reach the orchestrator, DNS, and the public internet only. If your LLM endpoint has a private IP, add it with --set networkPolicy.extraEgressCIDRs={10.1.2.0/24}.
On plain Kubernetes, add --set route.enabled=false and expose the simcrew service with your own Ingress.
Choosing a model. Each run can name any model your provider serves: type it in the Model field on New Run (it suggests what the provider lists) or pass "model" to POST /api/runs. Leave it blank to use the server's default (NVIDIA_NIM_MODEL, or ANTHROPIC_MODEL for Anthropic/Vertex). GET /api/models shows the default and the available models.
Via the UI:
- Open http://localhost:8000. Built-in personas and journeys (DX evaluation and smoke test) load on startup.
- Check your setup with a quick run: New Run → repo
https://github.com/mpk-droid/synthetic-users-smoke-test.git, journey Smoke Test Journey, personas Alex (Test), Blake (Test), Casey (Test). - Then evaluate your own repo with the DX Evaluation journey and the Priya, Sam, Dana, and Kai personas.
Via the API:
# List available personas
curl http://localhost:8000/api/personas
# List available journeys
curl http://localhost:8000/api/journeys
# Create a run (replace IDs from the responses above)
curl -X POST http://localhost:8000/api/runs \
-H "Content-Type: application/json" \
-d '{
"name": "Evaluate my-service",
"repo_url": "https://github.com/org/my-service.git",
"persona_environments": [{"persona_id": "<persona-uuid>", "environment_id": null}],
"journey_id": "<journey-uuid>",
"model": "nvidia/nemotron-3-ultra-550b-a55b"
}'
# Check run detail
curl http://localhost:8000/api/runs/<run-id>
# Delete a run
curl -X DELETE http://localhost:8000/api/runs/<run-id>Each persona runs in its own Docker container. The orchestrator manages the lifecycle:
repo_url ──▶ Orchestrator (FastAPI + Postgres)
│
┌──────────┼──────────┐
│ │ │
Agent: Priya Agent: Sam Agent: Dana ...
(container) (container) (container)
│ │ │
git clone git clone git clone
read docs follow setup read source
run commands try to run test edges
│ │ │
└────▶ Findings ◀────┘
│
Deduplicate & Score
(GREEN / YELLOW / RED)
The same Docker image serves both roles via SU_ROLE environment variable:
orchestrator(default) — runs the FastAPI app with UI, DB, and container managementagent— runs a lightweight server that clones repos and executes the LLM tool-use loop
| File | Purpose |
|---|---|
AGENTS.md |
Project context for any coding agent |
.cursor/rules/ |
Scoped rules for Cursor (project context, backend Python, frontend React) |
.vscode/tasks.json |
Editor tasks: docker: up (full stack) and dev: full stack (hot reload) |
For hot-reload during development, run the backend and frontend separately:
# Start Postgres
docker compose up db -d
# Backend (FastAPI with hot reload)
cd backend
uv sync
DATABASE_URL="postgresql+asyncpg://synthetic:synthetic@localhost:5432/synthetic_users" \
uv run uvicorn app.main:app --port 8000 --reload
# Frontend (Vite dev server with API proxy)
cd frontend
npm install
npm run dev # opens on :5174, proxies /api/* to :8000Personas are defined by structured fields, not raw prompts. The service generates the system prompt for you.
Via the UI:
- Go to Personas → Create Persona
- Fill in: Name, Identity, Perspective, Constraints, Expertise Level
- Click "Preview Prompt" to see the generated system prompt
- Adjust fields if needed, then create
- Review the system prompt and click "Approve"
Via the API:
# Preview a prompt without saving
curl -X POST http://localhost:8000/api/prompts/generate \
-H "Content-Type: application/json" \
-d '{
"name": "Principal Engineer",
"identity": "Staff engineer with 15 years experience evaluating SDKs",
"perspective": "API ergonomics, error handling, integration complexity",
"constraints": "Knows distributed systems but has not used this product before"
}'
# Create the persona
curl -X POST http://localhost:8000/api/personas \
-H "Content-Type: application/json" \
-d '{
"name": "Principal Engineer",
"identity": "Staff engineer with 15 years experience evaluating SDKs",
"perspective": "API ergonomics, error handling, integration complexity",
"constraints": "Knows distributed systems but has not used this product before",
"role_label": "Staff engineer"
}'
# Approve the generated prompt
curl -X POST http://localhost:8000/api/personas/<id>/approve-promptShips with 4 personas designed for evaluating developer tools and templates:
| Persona | Role | Catches |
|---|---|---|
| Priya | Engineering Director | Unclear value props, jargon-heavy docs, missing business context |
| Sam | AI Novice (backend dev) | Unexplained AI terminology, missing setup guidance, assumed knowledge |
| Dana | Production Engineer | Poor error handling, tight coupling, production anti-patterns |
| Kai | Platform Engineer | Missing resource limits, deployment issues, operational gaps |
And a 5-phase journey: First Impressions → Setup → Running Locally → Using the Target → Deployment.
Each persona's agent runs inside a container image. Default is the standard SimCrew agent image. To simulate a different machine (another OS, a Go or Java toolchain, missing tools), publish a variant of the base image, register it as an Environment, and pick it per persona on a run.
Base image:
ghcr.io/mpk-droid/simcrew:latest
This image includes the SimCrew agent server (SU_ROLE=agent). Custom environment images must extend it — do not use an unrelated container image.
# Start from the example
cp examples/environment/Dockerfile ./Dockerfile.env
# Edit: add packages, change OS tooling, etc.
docker build -f Dockerfile.env -t quay.io/<you>/simcrew-fedora:latest .
docker push quay.io/<you>/simcrew-fedora:latestMinimal Dockerfile:
FROM ghcr.io/mpk-droid/simcrew:latest
USER 0
RUN dnf install -y --nodocs <your-packages> && dnf clean all
USER 1001- Open Environments → New Environment
- Name, e.g.
Fedora + Go - Container image: your pushed image, e.g.
quay.io/<you>/simcrew-fedora:latest - Description (optional): a note for whoever reviews the results, e.g.
Fresh laptop, no oc installed. Agents never see it: they find out what's on their machine by using it, like a real developer
Or via the API: POST /api/environments with {"name": ..., "image": ..., "description": ...}.
On New Run, select personas and pick an environment per persona. Leave Default to use the base agent image.
If an environment's agent never becomes healthy, the run fails with a message naming the persona and image. Usually the image isn't built FROM the base image, or it overrides the entrypoint.
The Docker host or cluster running agents must be able to pull the image. Public images work out of the box. For private images:
- Locally: run
docker pull <image>on your machine first. SimCrew's own container doesn't have your registry login, but it uses images already present on the host. - On a cluster: agent pods use the namespace's
defaultservice account, so link a pull secret to it:oc create secret docker-registry my-pull --docker-server=quay.io --docker-username=... --docker-password=...thenoc secrets link default my-pull --for=pull.
Environments run whatever image they point to. On an instance other people can reach, limit them to registries you trust with SU_ALLOWED_ENV_IMAGE_PREFIXES (comma-separated, e.g. ghcr.io/mpk-droid/,quay.io/my-team/), or the Helm value environments.allowedImagePrefixes. Empty (the default) allows any image. The check applies when environments are created or edited and again when a run starts.
| Variable | Required | Description |
|---|---|---|
DATABASE_URL |
Yes (orchestrator) | PostgreSQL connection string (asyncpg) |
NVIDIA_API_KEY |
One LLM option | NVIDIA NIM key from build.nvidia.com |
MODELS_CORP_API_KEY, MODELS_CORP_URL |
One LLM option | Models.Corp MaaS key and endpoint |
ANTHROPIC_API_KEY |
One LLM option | Direct Anthropic API key |
NVIDIA_NIM_MODEL |
No | Default model when NVIDIA_API_KEY is set (default nvidia/nemotron-3-ultra-550b-a55b) |
NVIDIA_NIM_BASE_URL |
No | Any OpenAI-compatible endpoint; NVIDIA_API_KEY is sent as its bearer token (default NVIDIA's API) |
ANTHROPIC_MODEL |
No | Default model for Anthropic or Vertex (default claude-sonnet-5-5) |
ANTHROPIC_VERTEX_PROJECT_ID, CLOUD_ML_REGION |
One LLM option | Vertex AI project and region |
SU_ROLE |
No | orchestrator (default) or agent |
SU_AGENT_IMAGE |
No | Image for agent containers (default simcrew:latest) |
SU_DOCKER_NETWORK |
No | Docker network for agent containers (default simcrew) |
SU_ALLOWED_ENV_IMAGE_PREFIXES |
No | Comma-separated image prefixes environments may use (default: any image) |
SU_AGENT_RUN_TIMEOUT |
No | Seconds to wait for all personas in a run before failing it (default 1800; raise it for long journeys or rate-limited keys) |
SU_ALLOWED_REPO_SCHEMES |
No | Comma-separated URL schemes accepted for repo_url (default https; git@host:path is always accepted) |
See Run on your cluster for the basic install. The chart uses the public image ghcr.io/mpk-droid/simcrew, which is rebuilt for amd64 and arm64 on every push to main. Common overrides:
| Value | Purpose |
|---|---|
secrets.nvidiaApiKey, secrets.modelsCorpApiKey + secrets.modelsCorpUrl, secrets.anthropicApiKey |
LLM credentials |
image.repository, image.tag |
Use your own build |
route.host |
Custom hostname for the Route |
postgresql.persistence.size, postgresql.persistence.storageClass |
Database volume |
postgresql.enabled=false + externalDatabaseUrl |
Use an existing database |
networkPolicy.extraEgressCIDRs |
Private ranges agents may reach (e.g. an internal LLM endpoint) |
| Endpoint | Method | Description |
|---|---|---|
/health |
GET | Health check |
/api/personas |
GET, POST | List / create personas |
/api/personas/{id} |
GET, PUT, DELETE | Get / update / delete persona |
/api/personas/{id}/generate-prompt |
POST | Regenerate system prompt from fields |
/api/personas/{id}/approve-prompt |
POST | Mark prompt as approved |
/api/journeys |
GET, POST | List / create journeys |
/api/journeys/{id} |
GET, PUT, DELETE | Get / update / delete journey |
/api/journeys/{id}/phases |
POST | Add phase to journey |
/api/journeys/{id}/phases/{pid} |
PUT, DELETE | Update / delete phase |
/api/runs |
GET, POST | List / create runs |
/api/runs/{id} |
GET, DELETE | Run detail / delete run |
/api/runs/{id}/findings |
GET | All findings for a run |
/api/prompts/generate |
POST | Preview prompt from structured fields |
/api/models |
GET | Default model and the models the provider lists |
Full OpenAPI docs at http://localhost:8000/docs.
Contributions are welcome. See CONTRIBUTING.md to get started, and SECURITY.md to report a vulnerability. Everyone taking part follows the Code of Conduct.
SimCrew was created by and is maintained by Muthukumaran PK (@mpk-droid). See MAINTAINERS.md.
SimCrew is licensed under the Apache License 2.0.