Skip to content

docs(#5579): add harness field integration checklist - #5581

Closed
fullsend-ai-coder[bot] wants to merge 1 commit into
mainfrom
agent/5579-harness-field-checklist
Closed

fullsend-ai-coder[bot] wants to merge 1 commit into
mainfrom
agent/5579-harness-field-checklist

Conversation

@fullsend-ai-coder

Copy link
Copy Markdown
Contributor

Summary

Add docs/contributing/harness-fields.md — a contributor checklist covering the four implicit integration points that new harness schema struct fields must be wired into. This addresses the gap identified in retro analysis of PR #5192, where 4 of 6 review findings traced to undocumented conventions visible only by tracing sibling fields through the codebase.

Related Issue

Closes #5579

Changes

  • New file: docs/contributing/harness-fields.md — covers:
    • Expansion pipeline — os.Expand routing in run.go and checkVarRefs registration in ValidateRunnerEnvWith, with a table of all currently wired fields
    • Environment construction — RunnerEnv merge via envToList/childScriptEnv with a table of all call sites
    • Composition carry-forward — all 3 merge sites (mergeBaseIntoChild, mergeForgeConfigInto, mergeForgeConfig) with their per-type merge rules
    • Security pipeline — resolveBaseScripts, resolveBaseResources, resolveBaseHostFiles, validateBaseRelPath, and auditBaseFetch
    • Quick reference — grep pattern for tracing a sibling field through all 4 stages
  • Updated AGENTS.md — added table entry with trigger: Adding or modifying fields in the harness schema structs

Testing

  • docs/contributing/harness-fields.md exists with all 4 sections
  • Each section references concrete functions and current field lists verified against the codebase
  • AGENTS.md table entry exists with correct trigger phrase
  • Website sidebar uses getMarkdownFiles("contributing", ...) — auto-discovers the new file, no config change needed

Checklist

  • PR title follows Conventional Commits (correct type, ! for breaking changes)
  • Documentation-only change — no Go code, no tests needed

Closes #5579

Post-script verification

  • Branch is not main/master (agent/5579-harness-field-checklist)
  • Secret scan passed (gitleaks — f473cd512233268c5da72773be7e1e21042b3408..HEAD)
  • PR body secret scan passed (gitleaks — no-git)
  • Pre-commit hooks passed (authoritative run on runner)
  • Tests ran inside sandbox

Add docs/contributing/harness-fields.md covering the four pipeline
stages that new harness schema fields must be wired into: expansion
(os.Expand + checkVarRefs), environment construction (RunnerEnv
merge via envToList/childScriptEnv), composition carry-forward (all
three merge sites in compose.go and forge.go), and security pipeline
(resolveBaseScripts/resolveBaseResources/resolveBaseHostFiles with
validateBaseRelPath and auditBaseFetch). Each section lists the
current fields wired through that stage and provides guidance on
when and how to add new fields.

Update AGENTS.md topic-specific guidance table with the trigger
"Adding or modifying fields in the harness schema structs."

Closes #5579
@fullsend-ai-coder
fullsend-ai-coder Bot requested a review from a team July 24, 2026 16:44
@fullsend-ai-coder fullsend-ai-coder Bot added the ready-for-review Triggers review agent dispatch label Jul 24, 2026
@github-actions

Copy link
Copy Markdown

Site preview

Preview: https://2867a65a-site.fullsend-ai.workers.dev

Commit: c84b33ce658058fbbc358ff5bb1b88a36dd2ceea

@fullsend-ai-review

fullsend-ai-review Bot commented Jul 24, 2026 •

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 4:48 PM UTC · Completed 5:00 PM UTC
Commit: c84b33c · View workflow run →

@codecov

codecov Bot commented Jul 24, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@fullsend-ai-review

Copy link
Copy Markdown

Review

Findings

Medium

  • [protected-path] AGENTS.md — This PR modifies AGENTS.md, which is a protected governance file. The change adds a table entry pointing to the new docs/contributing/harness-fields.md contributor guide. The linked issue (Document harness field integration pipeline as a contributor checklist #5579) provides clear context and authorization for this modification. Human approval is always required for protected-path changes, regardless of context.

Labels: PR adds a new contributor guide under docs/contributing/ covering harness schema field integration, and updates AGENTS.md with a table entry for it.

@fullsend-ai-review fullsend-ai-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See the review comment for full details.

Comment thread AGENTS.md
| [Design Decisions](docs/contributing/design-decisions.md) | Understanding architectural principles and key decisions |
| [Vouch System](docs/contributing/vouch-system.md) | Working with the contributor vouch gate or PR workflows |
| [Tier Conventions](docs/contributing/tier-conventions.md) | Using the term "tier" in code or docs — covers the three distinct tier contexts |
| [Harness Fields](docs/contributing/harness-fields.md) | Adding or modifying fields in the harness schema structs — covers expansion, environment, composition, and security pipelines |

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[medium] protected-path

This PR modifies AGENTS.md, which is a protected governance file. The change adds a table entry pointing to the new docs/contributing/harness-fields.md contributor guide. The linked issue (#5579) provides clear context and authorization for this modification. Human approval is always required for protected-path changes, regardless of context.

@fullsend-ai-review fullsend-ai-review Bot added requires-manual-review Review requires human judgment component/docs User-facing documentation component/harness Agent harness, config, and skills loading labels Jul 24, 2026
@github-actions

Copy link
Copy Markdown

This pull request has been automatically marked as stale because it has not had any activity in the last month. It will be closed in 2 weeks if no further activity occurs. Remove the stale label to reset the inactivity timer.

@github-actions github-actions Bot added the stale label Aug 24, 2026
@rh-hemartin

Copy link
Copy Markdown
Member

No one taking care of this, closing.

@rh-hemartin
rh-hemartin deleted the agent/5579-harness-field-checklist branch August 26, 2026 07:08
@fullsend-ai-retro

fullsend-ai-retro Bot commented Aug 26, 2026 •

Copy link
Copy Markdown

🤖 Finished Retro · ✅ Success · Started 7:09 AM UTC · Completed 7:29 AM UTC

Commit: c84b33c · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $3.28

@fullsend-ai-retro

Copy link
Copy Markdown

Retro: PR #5581 — docs(#5579): add harness field integration checklist

Timeline

  1. 2026-07-24 16:09 UTC — Issue #5579 created by fullsend-ai-retro from a retro on PR #5192, proposing a contributor checklist for harness field integration pipelines.
  2. 16:32–16:37 — Triage agent categorized the issue (documentation/medium), added labels, and applied ready-to-code.
  3. 16:37–16:44 — Code agent created PR #5581 with a new 172-line docs/contributing/harness-fields.md and a one-line AGENTS.md table entry.
  4. 16:45–17:00 — Review agent posted one medium-severity protected-path finding for the AGENTS.md modification. No content defects identified.
  5. All CI checks passed — commit-lint, tests, codecov, e2e, functional, vouch, build.
  6. No human reviewed the PR. Labels ready-for-review and requires-manual-review were applied, but no human reviewer engaged.
  7. 2026-08-24 — Stale bot marked the PR after 31 days of inactivity.
  8. 2026-08-26 07:07 — rh-hemartin closed the PR: "No one taking care of this, closing." PR was never merged.

Workflow quality assessment

Agent pipeline performed well. The full triage → code → review chain completed in under 30 minutes. The code agent's output was high quality — it covered all 4 required categories (expansion, environment, composition, security) with concrete function references, tables of currently wired fields, and a bonus quick-reference grep section. The review agent correctly identified the only procedural concern (protected-path on AGENTS.md). All CI passed. The failure was entirely in the human review stage.

This is a retro-of-a-retro. Issue #5579 was itself created by the retro agent analyzing PR #5192, where 4/6 review gaps traced to the absence of this exact documentation. The code agent faithfully implemented the retro's proposal, but the resulting PR was wasted because no human reviewed it. The original retro's validation criteria ("next 3 harness-field PRs should have review coverage ≥3/4 categories") remain unevaluable because the document was never merged.

Evidence for existing issues (no new proposals needed)

All candidate improvement areas are already well-covered by existing open issues:

  • Re-dispatch cycle risk — Issue Document harness field integration pipeline as a contributor checklist #5579 remains open with ready-to-code label despite the implementing PR being closed. This creates a risk of the code agent re-dispatching and recreating the same PR. Existing issues #1969 (code agent should detect identical re-creation), #3050 and #3818 (remove ready-to-code on maintainer rejection), and #5315 (auto-close retro-created issues when PR closes without merge) cover this directly. This PR provides concrete new evidence: a retro-originated issue left in a dispatchable state after its sole implementing PR was closed.

  • Human review bottleneck — The PR sat 33 days without any human review. Existing issues #6152 (throttle dispatch on unreviewed backlog), #1339 (WIP limits), and #4807 (61-day unreviewed PR superseded by human) track this problem. This PR adds another data point: a well-executed docs PR with all CI green, wasted because no human triaged the review queue.

  • Protected-path friction on mechanical changes — The AGENTS.md change was a single table-row addition pointing to a new doc, not a governance rule change. The review agent correctly flagged it as protected-path, which contributed to the requires-manual-review label and the subsequent bottleneck. Existing issues #1392, #1551, #2745, and 15+ related issues track protected-path severity calibration.

  • Retro proposals touching protected paths — The retro agent's proposal (issue Document harness field integration pipeline as a contributor checklist #5579) inherently required modifying AGENTS.md, which the retro agent could have flagged as a merge-friction risk. Existing issue #2290 (retro/triage should detect when proposed fix requires protected paths) covers this.

Conclusion

The agent pipeline executed correctly at every stage. The failure mode — a high-quality PR dying from lack of human review — is a known systemic issue with extensive tracking. No new proposals are warranted; the evidence from this PR strengthens the case for the existing issues listed above, particularly #5315 (auto-close retro issues on PR close) and #6152 (throttle on unreviewed backlog).

This branch was successfully deployed

1 active deployment
site-preview — c84b33ce Deployed Jul 24, 2026 by github-actions[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

component/docs User-facing documentation component/harness Agent harness, config, and skills loading ready-for-review Triggers review agent dispatch requires-manual-review Review requires human judgment stale

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Document harness field integration pipeline as a contributor checklist

1 participant