Skip to content

Hang and timeout while running workflow #28868

Closed as not planned
Closed as not planned

Description

@JamesNK

I have a workflow that is hanging. Relevent snippit from logs:

2026-04-28T06:07:28.8566496Z The changelog looks correct. Now let me publish it to the wiki:
2026-04-28T06:07:28.8566894Z 
2026-04-28T06:07:29.3674738Z ● Final check before publish (shell)
2026-04-28T06:07:29.3675417Z   │ python3 -c "
2026-04-28T06:07:29.3676333Z   │ with open('/tmp/gh-aw/agent/new_changelog.md') as f:
2026-04-28T06:07:29.3677047Z   │ content = f.read()
2026-04-28T06:07:29.3677559Z   │ print(len(content), 'chars')
2026-04-28T06:07:29.3678154Z   │ print(content.count('\n'), 'lines')…
2026-04-28T06:07:29.3678556Z   └ 4 lines...
2026-04-28T06:07:29.3678694Z 
2026-04-28T06:07:34.6962383Z ● Read new_changelog.md
2026-04-28T06:07:34.6962869Z   │ /tmp/gh-aw/agent/new_changelog.md
2026-04-28T06:07:34.6963286Z   └ L490:530 (41 lines read)
2026-04-28T06:07:34.6963488Z 
2026-04-28T06:17:41.1843543Z The changelog looks correct. Let me now publish it:
2026-04-28T06:17:41.1844296Z 
2026-04-28T06:17:41.7038449Z ● Final size check (shell)
2026-04-28T06:17:41.7039011Z   │ python3 -c "
2026-04-28T06:17:41.7039618Z   │ with open('/tmp/gh-aw/agent/new_changelog.md') as f:
2026-04-28T06:17:41.7040318Z   │ content = f.read()
2026-04-28T06:17:41.7040686Z   │ # write to stdout for the tool
2026-04-28T06:17:41.7041015Z   │ import sys…
2026-04-28T06:17:41.7041258Z   └ 4 lines...
2026-04-28T06:17:41.7041391Z 
2026-04-28T06:17:45.2672479Z ● Read new_changelog.md
2026-04-28T06:17:45.2673322Z   │ /tmp/gh-aw/agent/new_changelog.md
2026-04-28T06:17:45.2674022Z   └ L838:845 (8 lines read)
2026-04-28T06:17:45.2674439Z 
2026-04-28T06:43:22.5405093Z ##[error]The action 'Execute GitHub Copilot CLI' has timed out after 45 minutes.
2026-04-28T06:43:22.5447543Z ##[group]Run node "${RUNNER_TEMP}/gh-aw/actions/detect_copilot_errors.cjs"
2026-04-28T06:43:22.5448095Z �[36;1mnode "${RUNNER_TEMP}/gh-aw/actions/detect_copilot_errors.cjs"�[0m
2026-04-28T06:43:22.5468765Z shell: /usr/bin/bash -e {0}

You can see there is a 25 minute hang after reading a file and then logs continue after the timeout.

Workflow run: https://github.com/microsoft/aspire/actions/runs/25036515907

Activity

  1. locked and limited conversation to collaborators on Apr 28, 2026
  2. unlocked this conversation on Apr 28, 2026
  3. JamesNK commented on Apr 28, 2026

    @JamesNK
    Author

    Again: https://github.com/microsoft/aspire/actions/runs/25038611298/job/73336099831. Big hang when content is ready but before output is published.

  4. locked and limited conversation to collaborators on Apr 28, 2026
  5. unlocked this conversation on Apr 28, 2026
  6. JamesNK commented on Apr 28, 2026

    @JamesNK
    Author

    I think I've fixed this by changing the incoming body to the output job to be a file reference.

    Could the large delay be caused by reading the body content into an LLM and then passing them to the output job? A small file reference would skip that overhead.

  7. locked and limited conversation to collaborators on Apr 28, 2026
  8. unlocked this conversation on Apr 28, 2026
  9. pelikhan commented on Apr 28, 2026

    @pelikhan
    Collaborator

    Could the large delay be caused by reading the body content into an LLM and then passing them to the output job? A small file reference would skip that overhead.

    You should ask this question using an agent and our debugging prompt so that it can correlate the MCP, firewall logs to determine the root cause.

  10. pelikhan commented on Apr 30, 2026

    @pelikhan
    Collaborator

    Most likely feeding a massive file into the agent led to catastrophic context explosion and latency from the LLM provider. I don't think we can do ouch here.

  11. locked and limited conversation to collaborators on May 25, 2026
  12. unlocked this conversation on May 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions