Skip to content

fix(harness): warn when URL-base skill is fetched with only SKILL.md - #2697

Merged
ggallen merged 1 commit into
fullsend-ai:mainfrom
ggallen:worktree-fix-pr-2690-review
Jun 26, 2026
Merged

ggallen merged 1 commit into
fullsend-ai:mainfrom
ggallen:worktree-fix-pr-2690-review

Conversation

@ggallen

@ggallen ggallen commented Jun 26, 2026

Copy link
Copy Markdown
Member

Summary

Addresses review feedback on #2690:

  • [important] fetchBaseSkill only fetches SKILL.md, silently losing companion files (sub-agents, scripts, meta-prompts). Now attaches a warning to the Dependency so CLI emits it during fullsend lock and fullsend run, guiding users to use forge-format URLs for multi-file skills.
  • [minor] lock_test.go asserted only deps[0] after bumping count to 3 — now verifies Field, URL, and Type for all 3 deps (base, agent, skill).
  • Propagates the Warning field through harness.Dependency → resolve.Dependency → CLI output.

Test plan

  • All internal/harness tests pass (including new warning assertions on TestFetchBaseSkill_OnlySKILLMDFetched, TestFetchBaseSkill_PartialIndexHit, cache-hit path)
  • All internal/cli tests pass (including strengthened TestRunLock_URLBaseOnlyDeps)
  • Pre-commit hooks pass

🤖 Generated with Claude Code

@qodo-code-review

Copy link
Copy Markdown

PR Summary by Qodo

Warn when URL-based skill fetch only downloads SKILL.md
🐞 Bug fix ✨ Enhancement 🧪 Tests 🕐 20-40 Minutes

Grey Divider

Description

• Attach a non-fatal warning when URL-based skills can only fetch SKILL.md.
• Propagate dependency warnings through harness → resolver → CLI lock/run output.
• Strengthen lock and harness tests to assert dependency types and warnings.
Diagram

graph TD
A["harness: fetchBaseSkill"] --> B["harness.Dependency\n(Warning)"] --> C["resolve.Dependency\n(Warning)"] --> D["CLI: lock/run"] --> E["Printer.StepWarn"]
F["Tests"] --> A
F --> D
Loading
High-Level Assessment

The following are alternative approaches to this PR:

1. Fail hard on URL-based multi-file skills
  • ➕ Prevents silent runtime breakage by forcing users onto forge-format URLs immediately
  • ➕ Simplifies semantics: URL-based skill fetch is explicitly unsupported beyond SKILL.md
  • ➖ Breaking change for existing users relying on URL bases
  • ➖ May be too strict if users intentionally use single-file skills
2. Support archive-based URL fetching for skills (zip/tar)
  • ➕ Enables true multi-file skill retrieval over plain HTTP without directory listing
  • ➕ Avoids requiring forge-format URLs in some environments
  • ➖ Adds complexity: packaging format, security checks, size limits, extraction safety
  • ➖ Requires new conventions and documentation for authors/publishers
3. Introduce a manifest/index endpoint for URL bases
  • ➕ Keeps HTTP model but enables enumerating companion files deterministically
  • ➕ Can evolve to include hashes/metadata for reproducible locking
  • ➖ Requires server-side support and spec changes
  • ➖ More involved rollout/migration than a warning

Recommendation: The PR’s approach (attach a non-fatal warning and surface it in lock/run) is the best near-term fix: it preserves backward compatibility while explicitly informing users about missing companion files and the recommended forge-format URL path. Consider an archive-based or manifest-based enhancement later if multi-file URL bases become a common requirement.

Files changed (6) +30 / -2

Enhancement (3) +8 / -0
lock.goSurface dependency warnings during lock and persist them in lock entries +4/-0

Surface dependency warnings during lock and persist them in lock entries

• Prints a StepWarn line when a resolved dependency includes a non-empty Warning. Also propagates Warning when converting resolved dependencies into CLI lockfile dependency entries.

internal/cli/lock.go

run.goEmit base dependency warnings during run +3/-0

Emit base dependency warnings during run

• When a base dependency contains Warning text, prints it via StepWarn so users see partial-fetch limitations at runtime.

internal/cli/run.go

resolve.goPropagate dependency warnings through resolver model +1/-0

Propagate dependency warnings through resolver model

• Adds a Warning field to resolve.Dependency to carry non-fatal dependency warnings from harness resolution into CLI consumers.

internal/resolve/resolve.go

Bug fix (1) +9 / -2
compose.goAdd Warning to dependencies and warn on URL-based skill partial fetch +9/-2

Add Warning to dependencies and warn on URL-based skill partial fetch

• Extends harness.Dependency with a Warning field. In fetchBaseSkill, constructs and attaches a warning indicating that only SKILL.md is fetched from plain HTTP URL bases, and that companion files will be missing.

internal/harness/compose.go

Tests (2) +13 / -0
lock_test.goStrengthen URL-base dependency assertions in lock tests +9/-0

Strengthen URL-base dependency assertions in lock tests

• Expands TestRunLock_URLBaseOnlyDeps to validate Field, URL, and Type across all expected dependencies (base, agent, skill) instead of only the first entry.

internal/cli/lock_test.go

compose_test.goAssert warning presence for URL-base skill fetch scenarios +4/-0

Assert warning presence for URL-base skill fetch scenarios

• Updates multiple tests to verify that skill dependencies include the expected 'only SKILL.md' warning (including cache-hit and partial-index-hit paths).

internal/harness/compose_test.go

@ggallen
ggallen force-pushed the worktree-fix-pr-2690-review branch from 6a50f82 to b738b25 Compare June 26, 2026 14:52
@github-actions

github-actions Bot commented Jun 26, 2026 •

Copy link
Copy Markdown

Site preview

Preview: https://42a8cb2b-site.fullsend-ai.workers.dev

Commit: 6392001b156e6fe4e9bbfaa50ff6952160c670b2

@fullsend-ai-review

fullsend-ai-review Bot commented Jun 26, 2026 •

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 2:56 PM UTC · Completed 3:07 PM UTC
Commit: b738b25 · View workflow run →

@qodo-code-review

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (1) 📘 Rule violations (0) 📜 Skill insights (0)

Context used
✅ Compliance rules (platform): 51 rules

Grey Divider


Remediation recommended

1. No warnings in lock --all 🐞 Bug ◔ Observability
Description
runLock() now prints dep.Warning, but runLockAll() never iterates over lockOneAgent()
results to emit dependency warnings, so fullsend lock --all remains silent for the
partial-skill-fetch warning. Users running bulk locks (common in CI) won’t see the new warning even
though the underlying condition still applies.
Code

internal/cli/lock.go[R131-133]

+		if dep.Warning != "" {
+			printer.StepWarn(fmt.Sprintf("  %s: %s", dep.Field, dep.Warning))
+		}
Relevance

⭐⭐⭐ High

Team favors surfacing warnings in CLI flows (lint warnings in lock/--all in #2362; consistency focus
in #2262).

PR-#2362
PR-#2262
PR-#1333

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The warning is emitted only in the single-harness lock flow; the bulk lock flow never prints
dependency details (and thus never prints warnings), even when lockOneAgent() returns dependencies
that may contain warnings.

internal/cli/lock.go[125-134]
internal/cli/lock.go[367-435]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`fullsend lock --all` does not surface the new `Dependency.Warning` messages. The warning is only printed in `runLock()` (single harness flow), while `runLockAll()` calls `lockOneAgent()` and persists the lock entry but never prints `result.deps` (and therefore never prints warnings).

### Issue Context
The PR adds warning attachment in `harness.fetchBaseSkill()` and propagates it through `Dependency` structs to be printed by the CLI. This works for `fullsend lock <agent>` and `fullsend run`, but not for the bulk lock pathway.

### Fix Focus Areas
- internal/cli/lock.go[125-134]
- internal/cli/lock.go[367-435]

### Suggested approach
- Option A (smallest change): In `runLockAll()`, after a successful `lockOneAgent()` call where `result != nil`, iterate `result.deps` and print the same info/warn lines as `runLock()`.
- Option B (cleaner): Extract a helper (e.g., `printResolvedDeps(printer, deps)`) used by both `runLock()` and `runLockAll()` so warning rendering stays consistent across lock modes.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Qodo Logo

@codecov

codecov Bot commented Jun 26, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@fullsend-ai-review

fullsend-ai-review Bot commented Jun 26, 2026 •

Copy link
Copy Markdown

Looks good to me


Labels: PR modifies skill fetching behavior in the harness composition layer

Previous run

Looks good to me

Previous run (2)

Looks good to me


Labels: PR modifies harness composition and CLI lock/run commands

@fullsend-ai-review fullsend-ai-review Bot added ready-for-merge All reviewers approved — ready to merge component/harness Agent harness, config, and skills loading labels Jun 26, 2026
@ggallen
ggallen force-pushed the worktree-fix-pr-2690-review branch from b738b25 to bcad7c7 Compare June 26, 2026 15:15
@fullsend-ai-review

fullsend-ai-review Bot commented Jun 26, 2026 •

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 3:18 PM UTC · Completed 3:26 PM UTC
Commit: bcad7c7 · View workflow run →

@fullsend-ai-review fullsend-ai-review Bot added ready-for-merge All reviewers approved — ready to merge and removed ready-for-merge All reviewers approved — ready to merge labels Jun 26, 2026

@ralphbean ralphbean left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. One minor note inline.

Comment thread internal/harness/compose.go Resolved
Address PR fullsend-ai#2690 review feedback: fetchBaseSkill only fetches SKILL.md,
silently losing companion files (sub-agents, scripts, meta-prompts).
Emit a warning on the Dependency so CLI surfaces it during lock/run.
Also strengthen lock_test.go assertions to verify all 3 deps (base,
agent, skill) instead of only deps[0].

Signed-off-by: Greg Allen <gallen@redhat.com>
Signed-off-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Greg Allen <gallen@redhat.com>
@fullsend-ai-review

fullsend-ai-review Bot commented Jun 26, 2026 •

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 4:07 PM UTC · Completed 4:16 PM UTC
Commit: 6392001 · View workflow run →

@fullsend-ai-review fullsend-ai-review Bot added ready-for-merge All reviewers approved — ready to merge component/skills and removed ready-for-merge All reviewers approved — ready to merge labels Jun 26, 2026
@ggallen
ggallen added this pull request to the merge queue Jun 26, 2026
Merged via the queue into fullsend-ai:main with commit b674ebf Jun 26, 2026
21 of 23 checks passed
@ggallen
ggallen deleted the worktree-fix-pr-2690-review branch June 26, 2026 16:32
@fullsend-ai-retro

fullsend-ai-retro Bot commented Jun 26, 2026 •

Copy link
Copy Markdown

🤖 Finished Retro · ✅ Success · Started 4:36 PM UTC · Completed 4:43 PM UTC
Commit: 6392001 · View workflow run →

@fullsend-ai-retro

Copy link
Copy Markdown

Retro: PR #2697 — warn when URL-based skill fetch only downloads SKILL.md

Verdict: Workflow went well. No new proposals — existing issues already cover the one identified inefficiency.

Timeline

  1. ggallen opened PR fix(harness): warn when URL-base skill is fetched with only SKILL.md #2697 (single commit, addressing review feedback from PR feat(harness): resolve declarative resources from URL-referenced bases #2690). The change adds a Warning field to Dependency, surfaces it through lock/run CLI output, and includes test coverage.
  2. qodo-code-review[bot] posted a summary and flagged one "bug": claimed runLockAll() doesn't surface dependency warnings. This was a false positive — the actual code shows runLockAll() calls printResolvedDeps() identically to runLock().
  3. fullsend-ai-review[bot] ran 3 successful reviews (on SHAs b738b25, bcad7c7, 6392001) plus 1 failed run, approving each time. The review agent correctly identified this as a clean, well-scoped change.
  4. ralphbean (human) approved with one inline suggestion: soften the warning wording from "are not included" to "may be missing" since single-file skills are legitimate.
  5. ggallen addressed the wording feedback and force-pushed.
  6. Review agent ran again on the updated SHA and approved.

What went well

  • Review agent was correct. It approved a genuinely clean change without flagging false positives. The Qodo bot's finding about runLockAll() was incorrect — the review agent's approval was the right call.
  • Good test coverage. Codecov confirmed all modified lines are covered.
  • Human review added value. The wording improvement from ralphbean was a subjective UX refinement — the kind of nuance that's hard for automated review to catch consistently.

Inefficiency noted (already tracked)

Autonomy readiness note

The review agent's approval aligned with the human reviewer's verdict. The only human-added value was a subjective wording improvement on a user-facing warning message — not a correctness or architecture issue. This class of change (adding a new field + propagating it through existing patterns + test coverage) is a good candidate for higher review autonomy.

This branch was successfully deployed

1 active deployment
site-preview — 6392001b Deployed Jun 26, 2026 by github-actions[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

component/harness Agent harness, config, and skills loading component/skills ready-for-merge All reviewers approved — ready to merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants