diff --git a/docs/DocsIndex.md b/docs/DocsIndex.md index 9a19ec81d..59d85ae30 100644 --- a/docs/DocsIndex.md +++ b/docs/DocsIndex.md @@ -37,6 +37,7 @@ The trust loop is how "certified" is a checkable claim: analyzer fence → witne - `samples/autonomy-objectives/README.md` — a complete tracked objective + witness + recorded model proposal, and how to feed it to the loop. - `spikes/README.md` — what each spike under `spikes/` is, which ledger rows cite it, and why none of them is a supported entry point; `spikes/autonomy-first-flight/run-first-flight.ps1` flies one real iteration (Docker + Ollama). - `scripts/run-cert-gate.sh` + `scripts/cert-gate-config.sh` — reproduce the CI `cert-gate` locally with the same filter. +- **`docs/LearningHarness.md`** — **long-term design sketch (M7, post-M6):** autonomous experience → verified adaptation; four-loop harness (record → reflect → evaluate → promote), learning ladder (episodic memory through trust-kernel changes), evidence-chain requirements, fleet learning, and autonomy HOLD until dogfood unlock. Roadmap feature, not yet implemented. ## Operator / Production Readiness diff --git a/docs/LearningHarness.md b/docs/LearningHarness.md new file mode 100644 index 000000000..91691a312 --- /dev/null +++ b/docs/LearningHarness.md @@ -0,0 +1,404 @@ +# Learning Harness: Autonomous Experience → Verified Adaptation + +**Status:** Design sketch (long-term roadmap — not yet implemented) +**Date:** 2026-09-19 +**Milestone:** M7 (post-M6) + +--- + +## Principle + +**Agents can learn freely. They cannot trust their own learning freely.** + +This distinction is the foundation of the Learning Harness architecture. Learning — extracting patterns from experience, forming hypotheses, proposing adaptations — is cheap, safe, and happens continuously. Trust — promoting those adaptations into production scope — is expensive, gated, and happens only after evidence meets the promotion bar. + +--- + +## What "Learn" Means + +In Ashlar, **learning** is NOT: +- Neural network weight updates +- Gradient descent or backpropagation +- Training loops that modify model parameters + +In Ashlar, **learning** IS: +- Experience → Hypothesis: "When tasks shaped like X fail with Y, pattern Z is present" +- Hypothesis → Experiment: "Try adaptation A on tasks matching profile B" +- Experiment → Evidence: "A succeeded in N/M trials; confidence score C" +- Evidence → Promotion: Pass Ashlar certification gates → activate under policy + +Learning is structural, explicit, and auditable. Every learned rule carries its derivation chain: which experiences it came from, what experiments validated it, and which gates approved its promotion. + +--- + +## Four-Loop Harness Diagram + +``` +┌─────────────────┐ +│ Experience │ Every task run: objective, context, plan, tools used, +│ Record │ output, success/failure, validator results, cost, duration +└────────┬────────┘ + │ + ▼ +┌─────────────────┐ +│ Reflect │ Analyze experience windows: detect patterns, form hypotheses, +│ (Pattern │ propose adaptations (strategy change, new tool, routing rule) +│ Detection) │ +└────────┬────────┘ + │ + ▼ +┌─────────────────┐ +│ Propose │ Package adaptation as a learning candidate: +│ Adaptation │ scope, confidence, evidence chain, risk tier +└────────┬────────┘ + │ + ▼ +┌─────────────────┐ +│ Ashlar Gates │ Certification gate (witness, mutation, determinism) +│ (Evaluate) │ + Tier-based admission (Tier 0 auto, Tier 1 hold, Tier 2 human) +│ │ + Replay validation (strategy candidates run against test corpus) +└────────┬────────┘ + │ + ├─ PASS ──▶ ┌────────────────┐ + │ │ Promote │ Install under scope restrictions: + │ │ (Activate) │ - canary rollout + │ │ │ - watch window + │ │ │ - auto-rollback on breach + │ └────────────────┘ + │ + └─ FAIL ──▶ ┌────────────────┐ + │ Store Failure │ Log rejection reason, fence catalog update, + │ (Quarantine) │ quarantine hash (never re-propose bit-identical) + └────────────────┘ +``` + +--- + +## Learning Ladder + +The **learning ladder** orders adaptations by blast radius and required promotion bar. Higher rungs touch more authority perimeter and require stronger evidence + human gates. + +| Rung | Adaptation Type | Risk | Promotion Bar | Authority Change | +|------|----------------|------|---------------|------------------| +| **1** | Episodic memory | Lowest | Automatic (no gate) | None — recall only | +| **2** | Semantic memory | Very Low | Automatic (hash check) | None — knowledge extraction | +| **3** | Strategy (heuristic) | Low | Replay gate + Tier 0 cert | Task planning only | +| **4** | Routing rule | Low | Replay gate + Tier 0 cert | Execution path selection | +| **5** | Prompt policy | Medium | Replay gate + Tier 1 (human admit) | Agent instruction template | +| **6** | Workflow template | Medium | Replay gate + Tier 1 + multi-task corpus | Orchestration structure | +| **7** | New tool (external) | Medium-High | Full cert + Tier 1 + capability policy check | Expands capability surface | +| **8** | New brick/code | High | Full cert + Tier 1 + canary + adversarial tests | Code generation + admission | +| **9** | Trust-kernel change | Highest | Tier 2 (human objective required) | Changes gates themselves | + +**Autonomy ceiling:** Rungs 1–4 are **designed to allow** Tier 0 autonomy **when M7 ships and dogfood unlock criteria are met** (see `docs/dogfood-scorecard.md`). Rungs 5–8 require human admission (Tier 1). Rung 9 requires human-authored objective (Tier 2). Authority is never autonomously learnable. + +--- + +## Separation of Concerns + +The Learning Harness distinguishes four orthogonal dimensions: + +| Dimension | Definition | Promotion Cost | Blast Radius | +|-----------|-----------|----------------|--------------| +| **Knowledge** | Facts, patterns, task-context associations (episodic/semantic memory) | Cheap (hash-verified storage) | Narrow (recall only) | +| **Strategies** | Task-planning heuristics, routing rules, retry policies | Low (replay validation) | Medium (task execution) | +| **Capabilities** | New tools, bricks, external API integrations | High (full certification + canary) | High (expands surface) | +| **Authority** | Gate rules, tier boundaries, admission policies | Highest (human objective required) | Critical (trust kernel) | + +**Cost and frequency:** Knowledge and strategies are cheap to update and change often. Capabilities are expensive and change rarely. Authority updates are human-gated and versioned with the trust kernel. + +--- + +## Core Port Sketch + +```csharp +public interface ILearningHarness +{ + /// + /// Record a completed task execution as an experience entry. + /// Captures: objective, context, plan, tools, output, success, cost, duration. + /// Failures are first-class and carry validator results. + /// + Task RecordExperienceAsync(AgentExperience experience, CancellationToken ct = default); + + /// + /// Retrieve learned context (rules, patterns, prior experiences) relevant to a task. + /// Used by agents during planning: "What have I learned about tasks like this?" + /// + Task> RecallAsync(TaskContext task, CancellationToken ct = default); + + /// + /// Analyze a window of experiences and propose learning candidates. + /// Reflection window may be: last N tasks, tasks in time range, tasks matching filter. + /// Returns: pattern hypotheses, strategy adaptations, capability proposals. + /// + Task> ReflectAsync( + ReflectionWindow window, + CancellationToken ct = default); + + /// + /// Evaluate a learning candidate: run replay tests, check tier placement, + /// estimate blast radius, compute confidence from evidence chain. + /// + Task EvaluateAsync( + LearningCandidate candidate, + CancellationToken ct = default); + + /// + /// Promote a candidate that passed evaluation: install under policy restrictions, + /// begin canary rollout, activate watch window, set up auto-rollback. + /// Returns: promotion record with canary plan and rollback trigger. + /// + Task PromoteAsync( + LearningCandidate candidate, + CancellationToken ct = default); +} +``` + +--- + +## AgentExperience Schema + +Every task execution is recorded as an `AgentExperience` with these fields: + +```csharp +public sealed record AgentExperience +{ + public required Guid ExperienceId { get; init; } + public required Guid TaskId { get; init; } + public required string AgentId { get; init; } + public required string AgentVersion { get; init; } + + public required string Objective { get; init; } + public required string ContextHash { get; init; } // SHA-256 of context snapshot + + public required IReadOnlyList ToolsAvailable { get; init; } + public required IReadOnlyList ToolsUsed { get; init; } + + public required TaskPlan Plan { get; init; } // Serialized plan structure + public required TaskOutput Output { get; init; } + + public required bool Success { get; init; } + public required IReadOnlyList ValidatorResults { get; init; } + public string? UserFeedback { get; init; } + + public required TimeSpan Duration { get; init; } + public required long TokenCost { get; init; } + public required long ToolCost { get; init; } // API calls, etc. + + public required string PolicyVersion { get; init; } + public required string EnvironmentVersion { get; init; } + public required DateTimeOffset Timestamp { get; init; } +} +``` + +**Failures are first-class:** A failed task is a complete experience record. Failure patterns are the primary source of useful learning signals (what contexts predict failure, what adaptations reduce failure rate). + +--- + +## LearnedRule Schema + +Every promoted rule must carry its evidence chain: + +```csharp +public sealed record LearnedRule +{ + public required Guid RuleId { get; init; } + public required string RuleType { get; init; } // "Strategy" | "Routing" | "Tool" | ... + public required string Scope { get; init; } // Task filter: which contexts this applies to + + public required IReadOnlyList DerivedFromExperiences { get; init; } + public required DateTimeOffset LearnedAt { get; init; } + + public required double SuccessRateBefore { get; init; } + public required double SuccessRateAfter { get; init; } + public required int TrialCount { get; init; } + public required double Confidence { get; init; } // 0.0–1.0 + + public DateTimeOffset? ExpiresAt { get; init; } // Optional: time-boxed rules + public required string PromotionGateVersion { get; init; } + + public required string EvidenceChain { get; init; } // Serialized derivation: experiences → hypothesis → experiments → gate verdicts +} +``` + +**Lineage is mandatory:** No rule is promoted without a documented trail from raw experiences through evaluation to admission. This enables: +- Tracing a rule back to the experiences that justified it +- Revoking a rule when its source experiences are invalidated +- Re-evaluating rules when gate versions change + +--- + +## Replay as Promotion Gate + +**Strategy and routing candidates** (ladder rungs 3–4) must pass replay validation before promotion: + +1. **Build test corpus:** Select N prior successful tasks matching the candidate's scope +2. **Replay with candidate:** Re-run those tasks using the proposed adaptation +3. **Compare outcomes:** Success rate, cost, duration vs. baseline +4. **Verdict:** Promote if: `success_rate >= baseline` AND `cost_increase <= threshold` + +Replay is fast (no external API calls if mocked) and cheap (no human in loop). It prevents regressions: a strategy that would have broken prior successful tasks is rejected before reaching production. + +--- + +## Unifying Existing Pieces + +Ashlar already has components that belong in the Learning Harness. The M7 milestone unifies them under a single subsystem: + +| Existing Component | Learning Harness Role | +|-------------------|----------------------| +| Task execution records | `IExperienceStore` (subtype of existing stores) | +| Failure pattern detector | `IReflector` (pattern detection step) | +| Self-improver / self-context | `ReflectAsync` / `RecallAsync` implementations | +| Test failure → adaptation proposals | `LearningCandidate` generation from test results | +| Certification gate | `EvaluateAsync` gate leg | +| Tier-based admission | `PromoteAsync` admission logic | +| Canary rollout + watch window | `PromotionResult` orchestration | +| Quarantine / rollback | `PromoteAsync` failure path | +| Federation (`.ashpkg` signing) | Distributed learning: signed rule packages | + +**New subsystem boundary:** `Ashlar.Learning` namespace with ports in core, adapters at edges, DIP (Dependency Inversion Principle) throughout. No Forge UI code in the Learning Harness kernel — product adapters consume the ports. + +--- + +## Self-Extension: Highest Rung + +**Self-extension** (generating a new brick or tool) is the highest learnable rung (ladder rung 8): + +1. Agent reflects: "Tasks matching profile X consistently fail; no tool exists for capability Y" +2. Agent proposes: New brick implementing interface Z +3. Harness evaluates: + - Full certification gate (witness, mutation, determinism, analyzer) + - Tier 1 admission (human reviews proposal) + - Capability policy check (does this expand surface safely?) +4. On admit: Canary rollout + - Install brick for 10% of matching tasks + - Watch window: 50 invocations or 7 days + - Auto-rollback on: error rate > 5%, latency breach, capability violation +5. On canary success: Promote to general availability + +Self-extension is **not special-cased** — it follows the same four-loop harness as any other adaptation, just with the highest promotion bar. + +--- + +## Fleet Learning (Distributed) + +The Learning Harness supports **federated learning** without requiring universal trust: + +1. **Node A learns a strategy** (rungs 3–4) and promotes it locally +2. **Node A packages the rule** as a signed `.ashpkg` with evidence chain: + - `LearnedRule` record (includes `DerivedFromExperiences` references) + - Anonymized experience summaries (no PII, no proprietary context) + - Promotion gate verdict (certificate + tier + timestamp) +3. **Node A publishes to fleet catalog** (federation, per `docs/Federation.md`) +4. **Node B discovers the package** via peer sync or LAN multicast +5. **Node B re-verifies under local policy:** + - Check signature (trust Node A's signing key?) + - Check evidence chain (sufficient trials? acceptable confidence?) + - Check compatibility (same policy version? same environment constraints?) +6. **If Node B accepts:** Install rule under canary (even if Node A flew it successfully, Node B's environment may differ) + +**Distributed learning without universal trust:** Each node decides independently whether to accept a peer's learned rule. Signing proves provenance; local gates enforce safety. + +--- + +## Commercial Positioning (HOLD until Dogfood Unlock) + +**Marketing one-liner (when unlocked):** + +> "A runtime where AI agents improve from experience without gaining uncontrolled authority." + +**Unlock criteria (from `docs/dogfood-scorecard.md`):** + +- Last-N=10 ≥80% pass rate (Strict mode, Ed25519 signing) +- Mean time-to-admit ≤15 minutes (single-objective canary) +- Strict rejection rate ≤70% (≥30% of proposals certified) +- ≥7 consecutive days with all thresholds met +- **At least 7 landed dated Strict PASS rows on master** in `docs/dogfood-ledger.md` (not CI artifacts alone) +- Real hygiene-via-Ashlar PR proof (production-quality Ashlar PR via full autonomy loop) + +**Until unlock:** Learning Harness is roadmap/design only. Do not claim "autonomous learning in production" until the thresholds are met and continuous dogfood proof is operational. + +--- + +## Honesty Gate + +**Autonomy claims HOLD** until: + +1. **Dogfood scorecard unlocked** (see `docs/dogfood-scorecard.md`) +2. **Dated Strict production runs** in public ledger (see `docs/dogfood-ledger.md`) +3. **M1–M6 complete** (see `docs/audits/2026-09-completion-roadmap.md`) +4. **Adversarial validation pack green** (16/16 tests, per M2) + +The Learning Harness design exists to prove the path is known. The honesty gate exists to prove the path is flown. + +--- + +## Architectural Constraints + +**Ports in core, adapters at edges:** + +- `ILearningHarness`, `IExperienceStore`, `IReflector` are ports (interfaces in `Ashlar.Core.Contracts`) +- SQLite/Postgres adapters, Ollama/OpenAI reflection backends, Forge UI are edge adapters +- DIP: Core never references edge infrastructure + +**No Forge UI in kernel:** + +- Learning Harness lives in `src/Ashlar.Learning.*` (framework tier) +- Forge consumes `ILearningHarness` via DI (product tier) +- Dashboard, visualizations, approval workflows: product concerns, not kernel concerns + +**Testability:** + +- Experience replay: replay gate must be fast (mock tool calls, no network) +- Reflection: must run in CI (no human, no external API keys) +- Promotion: canary rollout testable via time-acceleration (not wall-clock dependent) + +--- + +## Related Documents + +- `docs/audits/2026-09-completion-roadmap.md` — M1–M6 path to autonomy claims; M7 (this doc) is post-M6 +- `docs/certification-evidence.md` — Current certification proof ledger (what's live today) +- `docs/dogfood-scorecard.md` — Autonomy marketing unlock thresholds +- `docs/dogfood-ledger.md` — Dated production evidence (autonomy HOLD until green) +- `docs/SELF-EXTEND-AUDIT.md` — Self-extension safety invariants (enforced today) +- `docs/Federation.md` — Signed package distribution (used for fleet learning) +- `docs/trust-loop/ashlar-trust-loop-spec.md` — Trust-loop normative spec (gates, tiers, admission) +- `docs/RunningASelfExtendingNode.md` — Operator guide (dials, canary, rollback) + +--- + +## Implementation Notes (M7 Roadmap Sketch) + +**Not a schedule** — just the known subsystem pieces: + +1. **Experience store** (adapt existing task ledger) +2. **Reflector port + first adapter** (pattern detection from experience windows) +3. **Replay gate** (test corpus selection + re-run orchestration) +4. **Learning candidate schema** (standardize proposal format) +5. **Promotion orchestrator** (tier check → cert gate → canary → watch → rollback) +6. **Ladder enforcer** (mapping candidate type → required tier + gates) +7. **Evidence chain serializer** (JSON schema for `LearnedRule` provenance) +8. **Fleet sync adapter** (federated rule discovery + local re-verification) + +**Exit criteria for M7:** + +- [ ] Design doc accepted (this document) +- [ ] `ILearningHarness` port in `Ashlar.Core.Contracts` +- [ ] `IExperienceStore` implemented (SQLite adapter minimum) +- [ ] `IReflector` port + one concrete implementation (pattern detector) +- [ ] Replay gate proven in CI (strategy candidate → test corpus → verdict) +- [ ] Learning ladder enforced (candidate type → tier mapping) +- [ ] Linked from `docs/DocsIndex.md` (under Trust loop or Additional Material) + +**Dependencies:** + +- M1 (disarm/cert honesty) — canary/rollback must be proven before learning relies on it +- Dogfood continuous proof operational — autonomy HOLD until scorecard unlocked +- M6 complete — no learning-based autonomy claims until M1–M6 closed + +--- + +**END OF DESIGN SKETCH** diff --git a/docs/audits/2026-09-completion-roadmap.md b/docs/audits/2026-09-completion-roadmap.md index 76c609ae7..4dde81602 100644 --- a/docs/audits/2026-09-completion-roadmap.md +++ b/docs/audits/2026-09-completion-roadmap.md @@ -26,7 +26,7 @@ Ashlar has **proven the technical foundation** for autonomous self-extension: ce - ✅ Autonomous iteration harness proven through P6/S5 campaigns - ✅ Tier 0 autonomous admission with rollback/quarantine cycle - ✅ Product split architecture documented and boundary-gated -- ⚠️ Certification composition signer vulnerability remains (limitation 9; limitations 7–8 closed by PR #523) +- ✅ Certification composition signer vulnerability CLOSED (limitation 9; limitations 7–8 closed by PR #523) - ⚠️ Adversarial validation coverage incomplete - ⚠️ Product repos not extracted; Forge product surface undefined - ⚠️ Public ledger for autonomous runs not implemented @@ -35,7 +35,9 @@ Ashlar has **proven the technical foundation** for autonomous self-extension: ce Six milestones (M0–M6) progress from honesty baseline to commercial claims. Each milestone has clear **exit criteria**, **dependencies**, and **ownership**. Conservative estimate: **M0–M3 can close in the current development cycle**; M4–M6 require product-team coordination and sustained dogfood operation. -**Key Risk:** Certification composition signer vulnerability (limitation 9) is **publicly documented**; composition records are minted under the committed dev key because no production registration supplies one (corrected 2026-09-13: the signer itself now honours explicitly supplied keys. Limitations 7–8 (signature stripping, schema downgrade) were closed by PR #523. M1 MUST close limitation 9 before any commercial claim. +**Post-M6 roadmap:** M7 (Learning Harness) is the long-term feature for autonomous experience-driven improvement. Design sketched in `docs/LearningHarness.md`; implementation deferred until M1–M6 complete and dogfood scorecard unlocked. + +**Key Risk:** Certification composition signer vulnerability (limitation 9) **CLOSED 2026-09-13** (the signer itself now honours explicitly supplied keys; the injected brick signer IS the composition lane's key holder). Limitations 7–8 (signature stripping, schema downgrade) were closed by PR #523. **M1.1b has been met** — composition records are now minted under operator-supplied keys, not the committed dev constant. --- @@ -48,7 +50,7 @@ Six milestones (M0–M6) progress from honesty baseline to commercial claims. Ea | Capability | Evidence | Limitations | |-----------|----------|-------------| | **Autonomous brick generation & certification** | P6 live model proposals; S5 3/5 certified at qwen3.8:27b; cert-gate CI runs 27918340788, 27918244198 | Single-task family (damage→health compositions); recorded/replay model only in CI; live models local-only | -| **Safety invariants enforced** | SELF-EXTEND-AUDIT verdicts A–D all ENFORCED; SelfExtendInvariant*Tests pass | Certification composition signer vulnerability remains (limitation 9; limitations 7–8 closed); no adversarial campaign yet | +| **Safety invariants enforced** | SELF-EXTEND-AUDIT verdicts A–D all ENFORCED; SelfExtendInvariant*Tests pass | Certification composition signer vulnerability CLOSED 2026-09-13 (limitations 7–8–9 all closed); no adversarial campaign yet | | **Tier-based admission control** | Trust-loop spec sections 3, 7; Tier 0/1/2 classifier structural | Tier definitions not adversarially validated; kernel-touch smuggling untested | | **Session containment** | P3-P5 flights: in-session build + execution; Docker backend hardening M26 | Host retains orchestration, mutant compilation, judgment (limitation 4); read-only rootfs unflown on live daemon | | **Rollback & quarantine** | Trust-loop spec R5.1–R5.4; swap-host generation retention | Watch-window breach → auto-rollback path unflown (mechanism exists, not proven) | @@ -64,7 +66,7 @@ Six milestones (M0–M6) progress from honesty baseline to commercial claims. Ea **NOT Claimable:** - ❌ "Adversarially hardened autonomous extension" (no adversarial campaign) -- ❌ "Production-ready unattended operation" (limitation 9 remains exploitable; no public ledger yet) +- ❌ "Production-ready unattended operation" (no public ledger yet; adversarial validation incomplete) - ❌ "Public audit trail" (no ledger yet) - ❌ "Forge product" (not extracted, not integrated) @@ -89,27 +91,23 @@ Six milestones (M0–M6) progress from honesty baseline to commercial claims. Ea #### M1.1: Certification Schema Hardening -**Problem:** Limitation 9 allows composition signer key injection bypass. (Limitations 7–8 already closed by PR #523.) +**Problem:** ~~Limitation 9 allows composition signer key injection bypass.~~ **STATUS: CLOSED 2026-09-13.** (Limitations 7–8 already closed by PR #523, commit 966e6bf4.) -**Status of Limitations 7–8 (CLOSED by PR #523, commit 966e6bf4):** +**Status of All Limitations (7–8–9 CLOSED):** - ✅ Signature stripping (limitation 7): `CertificationVerifyOptions.Default` and `.Strict` now set `RequireEd25519Signature = true` - ✅ Schema downgrade (limitation 8): `MinimumSchemaVersion = 2` enforced fail-closed by default +- ✅ Composition signer key injection (limitation 9): `CompositionCertificationRecordSigner` now honours explicitly supplied `hmacKey`; the injected brick signer IS the composition lane's key holder; operator keys reach composition records -**Remaining Fix Required:** - -1. **Fix composition signer key injection (limitation 9)** - - `CompositionCertificationRecordSigner` constructor honors explicitly supplied `brickSigner` instead of discarding it - - Remove `_ = brickSigner;` discard; thread through key resolution - - **Exit:** Host passing real key via `brickSigner` mints compositions under that key, not committed constant +**M1.1b Exit Criterion Met:** +- Host passing real key via `brickSigner` mints compositions under that key, not committed constant ✅ +- Limitation 9 section in certification-evidence.md marked CLOSED with fix commit ref ✅ **Acceptance:** -- Limitation 9 fix compiled, tested, merged -- Limitation 9 section in certification-evidence.md marked CLOSED with fix commit ref +- All three limitations (7–8–9) closed and documented in certification-evidence.md **Owner:** Runtime team (cryptography/certification subsystem) -**Dependencies:** None -**Risk:** Composition path consumers must supply real keys; migration guidance needed -**Effort:** 1–2 days (only limitation 9 remains; 7–8 already closed) +**Dependencies:** None (already complete) +**Status:** ✅ COMPLETE (as of 2026-09-13) #### M1.2: Hold-Admit-Swap CI Proof @@ -125,7 +123,7 @@ Six milestones (M0–M6) progress from honesty baseline to commercial claims. Ea - **Exit:** CI green; test demonstrates Tier 1 hold → admit → swap with no false admits **Owner:** CI/platform team -**Dependencies:** M1.1 (limitation 9 closure) +**Dependencies:** M1.1 complete (all certification limitations closed) **Effort:** 2–3 days #### M1.3: Canary Tier 0 Swap in CI @@ -177,7 +175,7 @@ Six milestones (M0–M6) progress from honesty baseline to commercial claims. Ea ### M1 Exit Criteria - [x] **M1.1a** Limitations 7–8 closed (PR #523, commit 966e6bf4) -- [ ] **M1.1b** Limitation 9 closed; certification-evidence.md updated +- [x] **M1.1b** Limitation 9 closed (2026-09-13); certification-evidence.md updated - [ ] **M1.2** Tier 1 hold → admit → swap proven in CI - [ ] **M1.3** Tier 0 autonomous swap proven in CI (canary) - [ ] **M1.4** Pause, rollback, quarantine proven in CI @@ -639,6 +637,212 @@ ledger/ --- +## M7: Learning Harness (Long-term) + +**Goal:** Unify experience recording, reflection, evaluation, and promotion under a single `ILearningHarness` subsystem that enables agents to improve from experience without gaining uncontrolled authority. + +**Status:** Design sketch — roadmap feature, not yet implemented + +### Concept + +The **Learning Harness** is Ashlar's long-term answer to "How do agents learn from experience?" It establishes a four-loop architecture: + +1. **Experience Record:** Every task execution → structured experience entry (success/failure, context, cost, duration) +2. **Reflect:** Analyze experience windows → detect patterns → propose adaptations (strategies, routing rules, new tools) +3. **Evaluate:** Run candidates through Ashlar gates (certification, replay, tier classification, blast radius) +4. **Promote:** Admit under policy restrictions (canary rollout, watch window, auto-rollback) + +**Core principle:** Agents can learn freely (form hypotheses, propose adaptations). They cannot trust their own learning freely (promotion requires gates). + +### Learning Ladder + +Adaptations are ranked by blast radius and required promotion bar: + +| Rung | Type | Risk | Promotion Bar | +|------|------|------|---------------| +| 1–2 | Episodic/semantic memory | Lowest | Automatic | +| 3–4 | Strategy/routing rules | Low | Replay gate + Tier 0 cert | +| 5–6 | Prompt policy/workflow | Medium | Replay + Tier 1 (human admit) | +| 7–8 | New tool/brick | High | Full cert + Tier 1 + canary | +| 9 | Trust-kernel change | Highest | Tier 2 (human objective) | + +**Autonomy ceiling:** Rungs 1–4 are **designed to allow** Tier 0 autonomy **when M7 ships and dogfood unlock criteria are met** (per `docs/dogfood-scorecard.md`). Rungs 5–8 require human admission. Rung 9 requires human-authored objective. Authority is never autonomously learnable. + +### M7 Work Streams + +#### M7.1: Core Port Design + +**Deliverable:** `ILearningHarness` interface in `Ashlar.Core.Contracts` + +**Methods:** +- `RecordExperienceAsync(AgentExperience)` — log task execution +- `RecallAsync(TaskContext)` — retrieve learned context for planning +- `ReflectAsync(ReflectionWindow)` — analyze experiences → propose candidates +- `EvaluateAsync(LearningCandidate)` — run through gates +- `PromoteAsync(LearningCandidate)` — install under canary/watch + +**Acceptance:** +- Port interface defined +- `AgentExperience` and `LearnedRule` schemas documented +- Design doc published: `docs/LearningHarness.md` + +**Effort:** Design phase (no code yet; M7 is post-M6) + +#### M7.2: Experience Store + +**Deliverable:** `IExperienceStore` adapter (subtype of existing task ledger) + +**Schema:** Every task → `AgentExperience` record with: +- ExperienceId, TaskId, AgentId, Objective, ContextHash +- ToolsAvailable, ToolsUsed, Plan, Output +- Success, ValidatorResults, UserFeedback +- Duration, TokenCost, ToolCost +- PolicyVersion, EnvironmentVersion, Timestamp + +**Acceptance:** +- SQLite adapter implemented +- Store can record and query experiences by: agent, time range, success/failure, context hash +- Failures are first-class (failure patterns drive learning) + +**Effort:** 5–7 days (schema + adapter + tests) + +#### M7.3: Reflector Port + Pattern Detector + +**Deliverable:** `IReflector` port + one concrete implementation + +**Function:** Analyze experience windows → detect patterns → generate `LearningCandidate` proposals + +**Example patterns:** +- "Tasks with context hash prefix X fail 80% of the time" → propose new strategy +- "Tool Y unused in 500 consecutive successes" → propose removal from available set +- "Tasks matching profile Z have 2x token cost vs. baseline" → propose prompt policy change + +**Acceptance:** +- `IReflector` interface defined +- One pattern detector implemented (e.g., failure-rate-by-context) +- Detector runs in CI (no external dependencies) + +**Effort:** 8–10 days (port + first detector + tests) + +#### M7.4: Replay Gate + +**Deliverable:** Replay validation for strategy and routing candidates + +**Process:** +1. Select N prior successful tasks matching candidate scope +2. Re-run with proposed adaptation (mocked tool calls, no network) +3. Compare: success rate, cost, duration vs. baseline +4. Verdict: admit if `success_rate >= baseline` AND `cost_increase <= threshold` + +**Acceptance:** +- Replay gate runs in CI +- Strategy candidate: one green (admitted), one red (rejected) +- Test corpus selection configurable (time range, agent, context filter) + +**Effort:** 10–12 days (corpus selection + replay orchestration + gate integration) + +#### M7.5: Promotion Orchestrator + +**Deliverable:** `PromoteAsync` implementation with canary/watch/rollback + +**Flow:** +1. Check tier (Tier 0 auto, Tier 1 hold for human, Tier 2 reject) +2. Run certification gate (witness, mutation, determinism) +3. Begin canary: install for 10% of matching tasks +4. Watch window: 50 invocations or 7 days +5. On breach: auto-rollback + quarantine +6. On success: promote to general availability + +**Acceptance:** +- Canary rollout proven in CI (time-accelerated, not wall-clock) +- Watch window breach triggers rollback +- Promotion record includes evidence chain (experiences → evaluation → gate verdicts) + +**Effort:** 12–15 days (orchestration + watch + rollback + tests) + +#### M7.6: Ladder Enforcement + +**Deliverable:** Mapping from candidate type → required tier + gates + +**Table:** +- Episodic/semantic → automatic +- Strategy/routing → Replay + Tier 0 cert +- Prompt/workflow → Replay + Tier 1 +- Tool/brick → Full cert + Tier 1 + canary +- Kernel → Tier 2 (reject autonomous) + +**Acceptance:** +- Ladder enforcer rejects out-of-tier proposals +- CI test: kernel-touch candidate → Tier 2 rejection (no autonomous admit) +- CI test: strategy candidate → Replay + Tier 0 path + +**Effort:** 3–5 days (tier mapper + tests) + +### M7 Exit Criteria + +- [ ] **M7.1** `ILearningHarness` port defined; `docs/LearningHarness.md` published +- [ ] **M7.2** `IExperienceStore` implemented (SQLite minimum) +- [ ] **M7.3** `IReflector` port + one pattern detector +- [ ] **M7.4** Replay gate proven in CI (strategy candidates) +- [ ] **M7.5** Promotion orchestrator with canary/watch/rollback +- [ ] **M7.6** Learning ladder enforced (tier mapping) +- [ ] **M7.7** Linked from `docs/DocsIndex.md` (under Trust loop or Additional Material) +- [ ] **M7.8** Unify existing pieces: Observe/Adapt/Improve, ITestFailureStore, self-improver, self-context, self-extension under `Ashlar.Learning` namespace + +**Owner:** Framework team (runtime + autonomy) +**Dependencies:** +- M1 complete (disarm/cert honesty — canary/rollback must be proven before learning relies on it) +- Dogfood continuous proof operational (autonomy HOLD until scorecard unlocked) +- **Autonomy marketing HOLD:** No "autonomous learning in production" claims until M1–M6 closed AND dogfood scorecard met + +**Completion Target:** Post-M6 (long-term roadmap) + +**Commercial Positioning (HOLD):** + +> "A runtime where AI agents improve from experience without gaining uncontrolled authority." + +This claim is **blocked** until: +1. M1–M6 complete +2. Dogfood scorecard unlocked (per `docs/dogfood-scorecard.md`): + - Last-N=10 ≥80% pass rate + - Mean time-to-admit ≤15 minutes + - Strict rejection rate ≤70% + - ≥7 consecutive days with all thresholds met + - **At least 7 landed dated Strict PASS rows on master** in `docs/dogfood-ledger.md` + - Real hygiene-via-Ashlar PR proof +3. Dated Strict production runs in public ledger +4. Adversarial validation pack green (16/16 tests, per M2) + +### Fleet Learning (Federated) + +The Learning Harness supports **distributed learning without universal trust:** + +1. Node A learns a rule and packages it as signed `.ashpkg` with evidence chain +2. Node A publishes to fleet catalog (per `docs/Federation.md`) +3. Node B discovers package via peer sync / LAN multicast +4. Node B re-verifies under local policy (signature, evidence, compatibility) +5. If accepted: Node B installs under canary (even if Node A flew it successfully) + +**Key:** Signing proves provenance; local gates enforce safety. Each node decides independently whether to trust a peer's learned rule. + +### Architectural Notes + +**Ports in core, adapters at edges:** +- `ILearningHarness`, `IExperienceStore`, `IReflector` are ports (interfaces in `Ashlar.Core.Contracts`) +- SQLite/Postgres adapters, Ollama/OpenAI reflection backends are infrastructure +- Forge UI, dashboards, approval workflows are product concerns (not kernel) + +**DIP (Dependency Inversion Principle):** +- Core never references edge infrastructure +- Learning Harness lives in `src/Ashlar.Learning.*` (framework tier) +- Products consume `ILearningHarness` via DI + +**No Forge UI in kernel:** +- Dashboard, visualizations, approval UIs belong in `products/Ashlar.Forge.*` or commercial satellites +- Kernel provides ports only + +--- + ## Dependencies & Critical Path ### Dependency Graph @@ -668,7 +872,7 @@ M0 (baseline) → M1 (harden) → M2 (adversarial) → M3 (Forge scaffold) | **Legal blocks autonomous claims (M6.2)** | Launch delayed | Start legal review early (during M4); prepare fallback: "Autonomous with operator oversight" (Tier 1 only) | | **Forge team unavailable (M3–M4)** | Product demo blocked | Core team can stub Forge scaffold; defer full Forge.Verify to post-launch | | **Public ledger attracts adversarial scrutiny** | Reputation risk | Expect it; have incident-response plan; ledger shows we patch findings (not hide them) | -| **Composition signer vulnerability (limitation 9) exploited before M1 close** | **Medium** | Fix composition key injection immediately; limitations 7–8 already closed (signature/schema hardening) | +| **M1.2–M1.4 gate complexity higher than estimated** | Timeline slip | Limitation 9 already closed (2026-09-13); remaining M1 work is CI proof, not core fixes | --- @@ -679,7 +883,7 @@ M0 (baseline) → M1 (harden) → M2 (adversarial) → M3 (Forge scaffold) **Owns:** M1 (hardening), M2 (adversarial validation) **Responsibilities:** -- Close remaining certification vulnerability (limitation 9; 7–8 already closed) +- ~~Close remaining certification vulnerability (limitation 9; 7–8 already closed)~~ **COMPLETE: limitations 7–8–9 all closed** - Implement disarm mechanisms (pause, rollback, quarantine) - Write adversarial test pack (16 tests) - Maintain cert-gate, kernel-gate, adversarial-gate CI @@ -763,7 +967,7 @@ M0 (baseline) → M1 (harden) → M2 (adversarial) → M3 (Forge scaffold) ### M1–M2: Technical Hardening -- **Zero** exploitable certification vulnerabilities (limitation 9 closed; 7–8 already closed by PR #523) +- **Zero** exploitable certification vulnerabilities (limitations 7–8–9 all closed; 7–8 by PR #523, 9 closed 2026-09-13) - **Zero** false certificates in adversarial campaign (16/16 tests green) - **100%** disarm mechanism coverage (pause, rollback, quarantine CI-proven) @@ -790,12 +994,13 @@ M0 (baseline) → M1 (harden) → M2 (adversarial) → M3 (Forge scaffold) 1. **CEO sign-off on M0 baseline** (this document) 2. **Assign owners to M1–M6** (framework, product, CI, legal teams) -3. **Begin M1.1b** (close limitation 9) — composition signer vulnerability is public and remains exploitable -4. **Schedule legal kick-off** for M6.2 (2-week lead time needed) +3. ~~**Begin M1.1b** (close limitation 9)~~ **COMPLETE: limitation 9 closed 2026-09-13** +4. **Begin M1.2–M1.4** (CI proof of hold→admit→swap, canary, disarm mechanisms) +5. **Schedule legal kick-off** for M6.2 (2-week lead time needed) ### Strategic Priorities -**Priority 1: M1 (harden)** — Closes remaining certification vulnerability (limitation 9); must complete before any commercial claim. +**Priority 1: M1 (harden)** — ~~Closes remaining certification vulnerability (limitation 9)~~ **Limitations 7–8–9 all closed;** M1 now focuses on CI proof of disarm mechanisms (pause, rollback, quarantine) and admission gates (hold→admit→swap, canary). **Priority 2: M2 (adversarial)** — Proves safety model; required for "adversarially validated" claim. @@ -863,7 +1068,7 @@ M0 (baseline) → M1 (harden) → M2 (adversarial) → M3 (Forge scaffold) |----------|----------|-----------| | **Architecture.md** | `docs/Architecture.md` | Framework layers, trust architecture | | **SELF-EXTEND-AUDIT.md** | `docs/SELF-EXTEND-AUDIT.md` | Invariants A–D enforcement proof | -| **certification-evidence.md** | `docs/certification-evidence.md` | Current certification proof ledger; limitations 7–8 closed, limitation 9 open | +| **certification-evidence.md** | `docs/certification-evidence.md` | Current certification proof ledger; limitations 7–8–9 all closed | | **trust-loop-spec.md** | `docs/trust-loop/ashlar-trust-loop-spec.md` | Trust-loop normative spec (§8 adversarial) | | **self-extend-spec.md** | `docs/trust-loop/trust-loop-ext-autonomous-self-extension.md` | Autonomous-specific spec (tier model, recursion discipline) | | **product-split.md** | `docs/architecture/product-split.md` | Framework vs. product boundary |