Skip to content

TERMINAL_QUICKSTART.md

Latest commit

 

History

History
97 lines (67 loc) · 5.29 KB

TERMINAL_QUICKSTART.md

File metadata and controls

97 lines (67 loc) · 5.29 KB
Raw

Terminal quick start

New here? strata launch sets up a model, serves it and opens the monitor in one guided flow: TERMINAL_LAUNCH.md.

These commands use the strata command that the one-line Terminal install puts on your PATH (how to install). The Terminal carries its own Python, so there is nothing else to install. In a source checkout, type python strata.py (or python3 strata.py) in place of strata, from the repository root. The commands below read, prepare and rerun a model you already have; they do not download one. The complete options, permissions and limits are in TERMINAL.md.

Read a running server

With Strata already serving locally, get one machine-readable snapshot:

strata status --url http://127.0.0.1:8080 --json

The interactive monitor comes with the installed Terminal. Open the one-screen Overview:

strata monitor --url http://127.0.0.1:8080

The Overview puts serving state, prompt and answer speed, batch/queue pressure, four selected GPU rows, requests and activity on one screen. Press 1–6 for detail (1–7 in the monitor that strata launch opens, where 7 is the Connect view), Space to freeze the display, r to refresh, and q to detach. Detach leaves the server running. On a machine without a terminal, use status --json or monitor --once --json.

For a remote server, use HTTPS or a local SSH tunnel. A saved connection stores only the credential environment variable's name:

strata connection create work --url https://example.invalid --credential-env STRATA_TELEMETRY_KEY
strata monitor --connection work

Set STRATA_TELEMETRY_KEY in the environment before connecting. Replace example.invalid with your own server root. The monitor client never needs a management credential.

Prepare and rerun a local model

Use the run config written by setup. Start by reviewing its components and effective launch; neither command starts an engine:

strata doctor --profile /absolute/path/to/strata-model.json --components --output json
strata serve --profile /absolute/path/to/strata-model.json --plan --output json

For repeatable local management, create a named profile from that config. Review changes before saving them:

strata profile create four-gpu --from-config /absolute/path/to/strata-model.json
strata profile set four-gpu parallel=8 batch_groups=auto max_queue=32 queue_timeout=15 batch_telemetry=true --plan
strata profile set four-gpu parallel=8 batch_groups=auto max_queue=32 queue_timeout=15 batch_telemetry=true
strata profile show four-gpu --effective --output json
strata runtime start --profile four-gpu --plan --output json
strata runtime start --profile four-gpu
strata runtime verify --profile four-gpu --output json

Choose slot and queue settings for the actual model and card memory. A requested parallel=8 does not prove the engine loaded eight effective slots. Check the reported values after start. To change a running profile, save the edit, review the restart, then execute it:

strata profile set four-gpu parallel=4 --plan
strata profile set four-gpu parallel=4
strata runtime restart --profile four-gpu --plan --output json
strata runtime restart --profile four-gpu

The restart is an explicit operation. profile set alone does not restart or retune a running model. runtime stop --profile four-gpu stops only the owned instance; q in Monitor does not. The desktop app starts its models through the same runtime host on every system.

Check cards, calibrate and benchmark

Read installed components without launching the engine. The --live forms also probe host GPUs:

strata components verify --profile four-gpu --output json
strata doctor --profile four-gpu --live --output json

Calibration is separate from serving. Review the plan and measure a decode sweep only when the selected devices are available for exclusive use. Measuring writes a report but does not apply settings:

strata engine calibrate --profile four-gpu --plan --output json
strata engine calibrate --profile four-gpu --measure --exclusive --out /absolute/path/to/calibration.json
strata engine calibrate --profile four-gpu --review-result /absolute/path/to/calibration.json --output json

After reviewing the report, applying and rolling back remain separate explicit actions. Both require exclusive device access:

strata engine calibrate --profile four-gpu --apply-result /absolute/path/to/calibration.json --exclusive
strata engine calibrate --profile four-gpu --review-rollback /absolute/path/to/calibration.json --output json
strata engine calibrate --profile four-gpu --rollback-result /absolute/path/to/calibration.json --exclusive

Check the candidate report and owner state before either action. The full preconditions and rollback behavior are in TERMINAL.md.

After the model is loaded, save an explicit serving benchmark report:

strata bench run --profile four-gpu --suite serving-smoke --concurrency 4 --out /absolute/path/to/new-bench-report

The benchmark writes into a new output directory and does not publish its result. Synthetic screenshots and local fake-engine tests do not establish performance on a four-card host. Capture the exact GPU generation, model, driver and workload before comparing real runs.