Agent Persona Exploration - 2026-03-07

[github-actions[bot]]

This report summarizes findings from systematic testing of gh-aw workflow configuration quality across 5 software worker personas and 7 representative scenarios. Note: the agentic-workflows custom agent was not available in the task runner environment; scenarios were evaluated against gh-aw best practices using a general-purpose agent with full codebase context.

Persona Overview

• Agent evaluated: gh-aw workflow configuration quality
• Scenarios Tested: 7 (across 5 personas)
• Average Quality Score: 4.1 / 5.0
• Run: §22788815727

---

Key Findings

• 🏆 PM-1 (Weekly Feature Digest) scored highest (4.8/5) — no external infrastructure dependencies, pure GitHub API, textbook schedule trigger
• ⚠️ workflow_run trigger is correct for post-CI scenarios but requires an explicit if: github.event.workflow_run.conclusion == 'failure' guard — this is easily missed
• 🔧 actions toolset + bash: true are frequently needed for log-based workflows but commonly omitted in first attempts
• 📦 Artifact-dependent workflows (QA coverage, flaky test digest) carry hidden external prerequisites that must be documented upfront
• 🔒 Security posture was consistently strong — pull-requests: write via safe-outputs delegation, minimal permissions, and contents: read were correctly applied across scenarios

---

Top Patterns

1. Most common triggers: pull_request + paths filter (PR automation), workflow_run + if-guard (event-driven), schedule + workflow_dispatch (digests)
2. Most recommended tools: GitHub MCP (pull_requests + repos toolsets), actions toolset for log access, playwright for visual workflows, cache-memory for deduplication
3. Security practices observed: safe-outputs write delegation (never declare pull-requests: write directly), minimal permission scopes, update-comment/close-older-issues to prevent output sprawl

---

View High Quality Responses (Score ≥ 4.4)

PM-1: Weekly Feature Digest (4.8/5)

• Trigger: schedule (Friday 8 AM) + workflow_dispatch — perfect fit
• Tools: GitHub MCP pull_requests + issues + discussions only — no bash, no artifacts
• Output: create-discussion with close-older-discussions: true keeps only current week visible
• Why excellent: zero infrastructure prerequisites, self-contained, label-based grouping with fallback logic

BE-1: DB Schema Migration Safety (4.6/5)

• Trigger: pull_request + paths: ["db/migrate/**", "**/*.sql"] — precise activation
• Tools: pull_requests + repos toolsets only
• Key insight: use COMMENT not REQUEST_CHANGES to avoid blocking merges on AI misfires
• One gap: SQL dialect specificity (PostgreSQL vs MySQL lock semantics differ)

DO-1: Deployment Failure Incident (4.4/5)

• Trigger: workflow_run + if: conclusion == 'failure' — mirrors existing ci-doctor.md pattern exactly
• Tools: actions toolset + cache-memory for deduplication
• Insight: steps: pre-run block should pre-filter logs to failed jobs only before passing to AI

View Areas for Improvement (Score ≤ 3.6)

BE-2: Deployment Failure Log Triage (3.6/5)

• Tool gap: bash: true is required to decompress GitHub's gzip log archives — commonly omitted
• Security concern: deployment logs may contain partially-masked secrets; prompt must include redaction instructions
• Trigger gap: workflow_run always fires on completion; the if: conclusion == 'failure' guard must be manually specified and documented

QA-2: Weekly Flaky Test Digest (3.6/5)

• Hidden prerequisite: GitHub Actions API exposes job-level conclusions only, not per-test results. True per-test flakiness detection requires CI to emit JUnit XML artifacts under a consistent artifact name
• Prompt complexity: defining "flaky" operationally (same commit, mixed pass/fail across ≥3 runs, excluding code-change windows) is semantically hard — this is the highest-risk prompt to write vaguely
• Recommendation: Document CI artifact prerequisite explicitly and provide a job-level fallback for repos without JUnit artifacts

---

Score Summary

Scenario	Persona	Trigger	Tools	Security	Clarity	Complete	Avg
PM-1: Feature Digest	Product Manager	5	5	5	4	5	4.8
BE-1: Schema Review	Backend	5	5	4	4	5	4.6
DO-1: Deploy Incident	DevOps	5	5	4	4	4	4.4
FE-1: Visual Regression	Frontend	5	4	5	3	3	4.0
QA-1: Coverage Analysis	QA	4	4	5	3	4	4.0
BE-2: Log Triage	Backend	4	3	3	4	4	3.6
QA-2: Flaky Test Digest	QA	5	3	5	2	3	3.6

---

Recommendations

1. Document the workflow_run failure-filter pattern prominently — the if: github.event.workflow_run.conclusion == 'failure' guard is non-obvious, easy to omit, and causes workflows to run on every CI completion. Add it as a required pattern in workflow templates for event-driven scenarios.

2. Create an "artifact-dependent workflow" checklist — workflows that rely on CI-emitted artifacts (coverage reports, JUnit XML) should document their prerequisites upfront. Consider a warning in the compiler or documentation when actions toolset is used without a corresponding artifact-name constant in the prompt.

3. Promote cache-memory deduplication as a standard practice — every workflow that creates issues or discussions should use cache-memory to avoid duplicate outputs on retry runs. This pattern (already used in ci-doctor.md) should be highlighted in the getting-started guide for all event-driven and scheduled workflows.

---

References:

• §22788815727

AI generated by Agent Persona Explorer · history

[pelikhan]

/plan also extract cache deduplication as shared agentic workflow

[github-actions[bot]]

This discussion has been marked as outdated by Agent Persona Explorer.

A newer discussion is available at Discussion #21155.