Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/RELEASE_CHECKLIST.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ time.
- [ ] Run `npm ci`.
- [ ] Run `npm run typecheck`.
- [ ] Run `npm run test:functional`.
- [ ] Run `npm run test:onboarding`.
- [ ] Run `npm run test:security` and review every aggregate regression result.
- [ ] Run `npm run test:performance` and compare the reported p95/RSS values with the fixed budgets and the 20-percent release regression gates.
- [ ] Run `npm ls --all`; investigate missing, invalid, or unexpected dependencies.
Expand Down
6 changes: 6 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,9 @@ jobs:
run: npm run build && npm run typecheck
- name: Test
run: npm run test:functional
- name: Contributor onboarding
timeout-minutes: 16
run: npm run test:onboarding
- name: Shell syntax
run: sh -n scripts/validate.sh && node --check templates/pre-commit
- name: Check and doctor worked example
Expand Down Expand Up @@ -76,6 +79,9 @@ jobs:
run: npm run build && npm run typecheck
- name: Test in Git Bash
run: npm run test:functional
- name: Contributor onboarding
timeout-minutes: 16
run: npm run test:onboarding
- name: Shell syntax
run: sh -n scripts/validate.sh && node --check templates/pre-commit
- name: Check and doctor worked example
Expand Down
1 change: 1 addition & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@ npm run typecheck
npm run test:unit
npm run test:integration
npm run test:e2e
npm run test:onboarding
npm run test:security
npm run test:performance
npm test
Expand Down
12 changes: 12 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,18 @@
All notable changes to GraphKeeper are documented here. GraphKeeper follows semantic
versioning; while the package is below 1.0, minor releases may change public behavior.

## [Unreleased]

### Added

- `graphkeeper close run`, which atomically closes one existing open run with an end
timestamp and verdict while preserving its accumulated provenance.

### Changed

- The shipped agent skill and command documentation now direct agents to create runs
with `append run`, add claims with `append claim`, and close runs with `close run`.

## [0.5.0] - 2026-08-22

### Added
Expand Down
5 changes: 4 additions & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,7 @@ Run these before opening a pull request:
npm run build
npm run typecheck
npm run test:functional
npm run test:onboarding
npm test
npm run test:security
npm run test:performance
Expand All @@ -141,7 +142,9 @@ Run these before opening a pull request:
node --check templates/pre-commit
git diff --check

`npm test` includes all compiled tests and benchmarks for a complete local check.
`npm run test:onboarding` isolates the clean-clone install journey because it depends
on package-registry and cache performance. `npm test` includes all compiled tests and
benchmarks for a complete local check.
`npm run release:verify` deliberately runs the deterministic functional, security,
and package gates; performance budgets run separately in dedicated Ubuntu and
Windows CI jobs. This keeps performance regressions visible without making
Expand Down
9 changes: 5 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,7 @@ Run this at the root of the repository whose memory you want to protect:

Review the displayed plan and confirm it. `init` creates the JSON graph, `evidence/`, the canonical validator, a pre-commit hook, and the repository-scoped skill. `--integrate codex` also adds the Codex reminder to `AGENTS.md`.

Now ask your agent to record a verified finding (with `$graphkeeper` in Codex, `/graphkeeper` in Claude Code, or your agent's invocation from the table above). The agent appends claims and runs through `graphkeeper append claim ...` and `graphkeeper append run ...`, which serialize concurrent writers so parallel sessions never overwrite each other. After it writes the claim and evidence, validate and retrieve the result:
Now ask your agent to record a verified finding (with `$graphkeeper` in Codex, `/graphkeeper` in Claude Code, or your agent's invocation from the table above). The agent creates a run with `graphkeeper append run ...`, records claims with `graphkeeper append claim ...`, and finishes with `graphkeeper close run ...`. These commands serialize concurrent writers so parallel sessions never overwrite each other. After it writes the claim and evidence, validate and retrieve the result:

npx graphkeeper check
npx graphkeeper query test_payments_flaky
Expand Down Expand Up @@ -206,8 +206,9 @@ An agent writes to `graph/` and `evidence/`, and `graphkeeper check` (and the in
| `graphkeeper check` | Run the same fast schema, append-only history, and committed-evidence checks used by the Git hook. |
| `graphkeeper query <subject>` | Resolve an exact ID or unique alias and print active claims with provenance. It does not read evidence contents. |
| `graphkeeper doctor` | Run fast validation plus file existence, containment, line-range, dangling-reference, and unused-entity checks. |
| `graphkeeper append claim ...` | Concurrency-serially append a validating claim and link it into its producing run. See `graphkeeper --help` for the required/optional flags. |
| `graphkeeper append run ...` | Concurrency-serially append a validating run record. See `graphkeeper --help` for the required/optional flags. |
| `graphkeeper append claim ...` | Concurrency-serially append a validating claim and link it into its producing run. See the [append command reference](docs/append-commands.md) for flags and lifecycle constraints. |
| `graphkeeper append run ...` | Concurrency-serially create a validating run record. It remains create-only; see the [append command reference](docs/append-commands.md). |
| `graphkeeper close run --id <id> --ended <timestamp> --verdict <value>` | Concurrency-serially close one existing open run without replacing its accumulated provenance. |
| `graphkeeper update` | Check npm's stable `latest` release and globally install one exact newer version. Repository files are never changed. |
| `graphkeeper --help` | Print the supported command grammar and options. |
| `graphkeeper --version` (`-v`) | Print the installed GraphKeeper version. |
Expand Down Expand Up @@ -298,4 +299,4 @@ Use [GitHub Issues](https://github.com/anusbutt/Graph_Keeper/issues) for reprodu

See [`CONTRIBUTING.md`](CONTRIBUTING.md) for the development workflow, extension points, and boundaries, [prepared contribution issues](docs/contributor-issues.md), [`SUPPORT.md`](.github/SUPPORT.md) for usage help, and [`SECURITY.md`](.github/SECURITY.md) for private vulnerability reporting. Version `0.5.0` is pre-1.0 and its API may change. Release ownership and the target version must be verified immediately before publishing; completing the repository release gates does not publish anything.

GraphKeeper is available under the [MIT License](LICENSE).
GraphKeeper is available under the [MIT License](LICENSE).
13 changes: 11 additions & 2 deletions benchmarks/graphkeeper-memory-bench-v0.1.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,8 @@

This document defines the first four scenarios for evaluating durable project memory in coding agents.

The benchmark defines the scenarios. Test A also has manually recorded pre-fix and post-fix executions; reusable scoring methodology, automated evaluation, and datasets remain future work.
The benchmark defines the scenarios. Tests A and B have manually recorded executions;
reusable scoring methodology, automated evaluation, and datasets remain future work.

## Test A — Repeated Investigation

Expand Down Expand Up @@ -71,4 +72,12 @@ GraphKeeper Memory Bench v0.1 currently defines only these four scenarios:
3. Provenance
4. Correction history

Test A has recorded [pre-fix FAIL](results/repeated-investigation-pre-fix-2026-08-14/README.md) and [post-fix PASS](results/repeated-investigation-post-fix-pass-2026-08-14/README.md) results. These are manual evidence records, not an automated benchmark harness. Tests B-D, reusable scoring methodology, automated evaluation, and datasets remain future work.
Test A has recorded [pre-fix FAIL](results/repeated-investigation-pre-fix-2026-08-14/README.md)
and [post-fix PASS](results/repeated-investigation-post-fix-pass-2026-08-14/README.md)
results. Test B has recorded a
[pre-fix FAIL](results/stale-memory-cursor-adapter-2026-09-03/README.md) and a
[post-fix PASS](results/stale-memory-authentication-post-fix-2026-09-03/README.md).
The separate open-run lifecycle defect observed in that pass has a subsequent
[close-command PASS](results/stale-memory-authentication-close-run-2026-09-04/README.md).
These are manual evidence records, not an automated benchmark harness. Tests C-D,
reusable scoring methodology, automated evaluation, and datasets remain future work.
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# Stale-memory benchmark: explicit run closure

Date: 2026-09-04
Scenario: GraphKeeper Memory Bench v0.1, Test B -- Stale Memory
Result: **PASS, including the previously failing run lifecycle**

## Result

Session 1 started in a fresh isolated repository where authentication lived in
`src/auth.ts`. A fresh ephemeral Codex session recorded two grounded claims, closed
its producing run with `graphkeeper close run`, passed `graphkeeper check` and
`graphkeeper doctor`, and committed only memory files.

The fixture then moved that file unchanged to `src/security/auth.ts` and committed the
move without updating GraphKeeper memory. Immediately before Session 2, the active
`implemented_in` claim still returned `src/auth.ts`.

Session 2 was a separate ephemeral Codex invocation with the same neutral prompt. It
retrieved the existing memory first, confirmed the contradiction, captured fresh
evidence, and appended `claim_16332d0e` for `src/security/auth.ts` with
`supersedes: claim_c9cd4a1e`. It preserved the old location claim and the independently
valid token-comparison claim. It then closed its distinct producing run with
`graphkeeper close run` and committed only graph and evidence files.

The final query returns `src/security/auth.ts`, not the stale location. Both producing
runs contain `ended` and `verdict: passed`; final check and doctor results are clean
with zero warnings. This passes Test B and resolves the separate lifecycle defect
recorded by the prior benchmark.

## Scoring

- [x] Session 1 committed grounded memory for the original location.
- [x] The repository changed without updating the memory store.
- [x] Session 2 retrieved the stale active claim before current-state investigation.
- [x] Session 2 reported `src/security/auth.ts` as current.
- [x] Fresh line-addressable evidence was captured.
- [x] The replacement location claim explicitly supersedes the old location claim.
- [x] The old claim remains preserved in `graph/claims.json`.
- [x] The stale location is absent from active query results.
- [x] Session 1 and Session 2 used distinct runs and distinct Codex threads.
- [x] Both sessions closed their run through `graphkeeper close run` before commit.
- [x] Both memory commits exclude application files.
- [x] The final graph passes `check` and `doctor` with zero warnings.

## Secondary observation

Session 2 correctly began with retrieved memory and bounded its code inspection to the
authentication surface, but after finding `src/auth.ts` missing it also inspected Git
history. That was unnecessary for proving the current file move and is stricter than
Test B requires; it does not affect stale-claim correction or run closure. A future
prompt-efficiency benchmark can score this separately.

## Reproducibility

- Session 1 thread: `01a06944-59cd-74e3-b003-091bb11e4493`
- Session 2 thread: `01a06946-e152-7252-8bd9-149059cb9e54`
- Both were separate `codex exec --ephemeral --ignore-user-config --json` invocations.
- Both received the exact prompt in `prompts.md`.
- Candidate package SHA-256:
`23a50807101ca76a04d488ebca092892f319674c4bdb2369b516f37bc5e88788`
- Candidate and installed skill SHA-256:
`e94bd49275de040b8571fd9a0074553b18c7dca0ebcf8e9901eeaacad7130829`

`fixture.bundle` contains the complete inspectable Git history. `graphkeeper/` is the
final memory snapshot; `fixture/` contains the installed guidance and current source;
the two `session-*-final.md` files retain each agent's final report.
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Benchmark environment and identities

- Date: 2026-09-04 (Asia/Karachi; behavioral commands ran before midnight UTC)
- GraphKeeper source base: `0b40633`
- GraphKeeper package version: `0.5.0` plus the uncommitted close-command candidate
- Candidate package SHA-256: `23a50807101ca76a04d488ebca092892f319674c4bdb2369b516f37bc5e88788`
- Candidate template SHA-256: `e94bd49275de040b8571fd9a0074553b18c7dca0ebcf8e9901eeaacad7130829`
- Installed fixture skill SHA-256: `e94bd49275de040b8571fd9a0074553b18c7dca0ebcf8e9901eeaacad7130829`
- Codex CLI: `codex-cli 0.153.0`
- Codex model: CLI default with user configuration ignored
- Node.js: `v22.21.0`
- npm: `10.9.4`
- Session 1 thread: `01a06944-59cd-74e3-b003-091bb11e4493`
- Session 2 thread: `01a06946-e152-7252-8bd9-149059cb9e54`

Fixture commits:

```text
5cd07b8 initial implementation
861982c chore: initialize repository memory
7eade35 docs(memory): record authentication implementation
bd15a26 refactor module layout
452fd50 memory: update authentication module location
```

The candidate was packed from the working tree after all repository quality gates
passed. The complete fixture history is retained in `fixture.bundle`.
Binary file not shown.
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
<!-- graphkeeper:codex:start -->
## GraphKeeper memory

Before repeating repository investigation, invoke `$graphkeeper` to check
existing durable findings. Record new durable, evidence-backed findings through
that skill.
<!-- graphkeeper:codex:end -->
Loading