Skip to content

Pushing is slower than expected (> 2min) #386

Closed
#423
Closed
Pushing is slower than expected (> 2min)#386
#423
Assignees
EndBug
Labels
status: pinnedShould not be labeled as staletype: featureNew feature or feature request

Description

@melink14
melink14
opened on Apr 3, 2022

Describe the bug
I created a workflow which pushes updated visual regression screenshots when those tests fail, pushing the images takes twice as long as running the entire presubmit lint/test suite.

Workflow used
https://github.com/melink14/rikaikun/blob/23fc0df1f33bd32456a73e09ddc16e19c52a3087/.github/workflows/presubmit.yml

Expected behavior
Based on pushing from my computer and pushing that happens during other Actions (like semantic release) I expect it to be quite fast to push a single commit to git.

Logs
https://github.com/melink14/rikaikun/runs/5803997253?check_suite_focus=true

Activity

  1. melink14
    added
    status: pendingMore info is needed before deciding what to do
    on Apr 3, 2022

  2. stale commented on Apr 23, 2022

    @stale
    stalebot
    on Apr 23, 2022

    This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

  3. stale
    added
    status: staleInactive issues and PRs
    on Apr 23, 2022
  4. EndBug
    added
    status: pinnedShould not be labeled as stale
    and removed
    status: staleInactive issues and PRs
    on Apr 23, 2022

  5. kachkaev commented on Aug 11, 2022

    @kachkaev
    kachkaev
    on Aug 11, 2022

    Same here in a private repo. I am pushing to a lightweight branch in a repo that has a lot of heavy branches. It feels as if the command pulls all branches first before pushing. Just judging by its time – it takes 6+ minutes.

    UPD I removed the pull option but the action did not become faster. It was originally pull: "--rebase --autostash". There must be a way of pulling just a single branch, but performance is not a high priority in my case. It’d be great if the action was as fast as running git commit & git push though.

  6. EndBug commented on Aug 18, 2022

    @EndBug
    EndBug
    on Aug 18, 2022
    Owner

    Hi @kachkaev, thanks for writing in
    Would you be able to provide some logs? I'm trying to replicate this behavior since I should have some time to look into it, but the logs from the issue author have expired.

  7. kachkaev commented on Aug 18, 2022

    @kachkaev
    kachkaev
    on Aug 18, 2022

    👋 @EndBug! Here are my logs:

    2022-08-05T05:57:16.6131225Z ##[group]Run EndBug/add-and-commit@v9
2022-08-05T05:57:16.6131487Z with:
2022-08-05T05:57:16.6131694Z   author_name: GitHub Actions
2022-08-05T05:57:16.6132032Z   author_email: github.actions@my-project.example.org
2022-08-05T05:57:16.6132372Z   message: Update seed data
2022-08-05T05:57:16.6132626Z   pull: --rebase --autostash
2022-08-05T05:57:16.6132856Z   push: true
2022-08-05T05:57:16.6133332Z   cwd: some-cwd
2022-08-05T05:57:16.6133559Z   add: .
2022-08-05T05:57:16.6133782Z   default_author: github_actor
2022-08-05T05:57:16.6134041Z   pathspec_error_handling: ignore
2022-08-05T05:57:16.6134593Z   github_token: ***
2022-08-05T05:57:16.6134799Z ##[endgroup]
2022-08-05T05:57:19.9912054Z 
2022-08-05T05:57:19.9917541Z 
2022-08-05T05:57:19.9918073Z 
2022-08-05T05:57:19.9918542Z 
2022-08-05T05:57:19.9919006Z 
2022-08-05T05:57:19.9919470Z 
2022-08-05T05:57:20.0064685Z Running in /home/runner/work/repo-name/repo-name/some-cwd
2022-08-05T05:57:20.0087082Z Add input parsed as single string, running 1 git add command.
2022-08-05T05:57:20.0088165Z > Using 'GitHub Actions <github.actions@my-project.example.org>' as author.
2022-08-05T05:57:20.0090565Z > Using "Update seed data" as commit message.
2022-08-05T05:57:20.0092763Z ##[group]Internal logs
2022-08-05T05:57:20.0097351Z > Staging files...
2022-08-05T05:57:20.0098326Z > Adding files...
2022-08-05T05:57:20.1205370Z > No files to remove.
2022-08-05T05:57:20.1205889Z > Checking for uncommitted changes in the git working tree...
2022-08-05T05:57:20.1341700Z > Found 2 changed files.
2022-08-05T06:03:26.9984275Z {
2022-08-05T06:03:26.9985182Z   raw: '',
2022-08-05T06:03:26.9986195Z   remote: 'https://github.com/org-name/repo-name',
2022-08-05T06:03:26.9986697Z   branches: [
2022-08-05T06:03:26.9987775Z     { name: 'another-branch', tracking: 'origin/another-branch' },
2022-08-05T06:03:26.9988218Z     {
2022-08-05T06:03:26.9988706Z       name: 'some-other-branch-01',
2022-08-05T06:03:27.0058350Z       tracking: 'origin/some-other-branch-01'
2022-08-05T06:03:27.0058647Z     },
2022-08-05T06:03:27.0058842Z     {
2022-08-05T06:03:27.0059165Z       name: 'some-other-branch-02',
2022-08-05T06:03:27.0059557Z       tracking: 'origin/some-other-branch-02'
2022-08-05T06:03:27.0059799Z     },
2022-08-05T06:03:27.0059988Z     {
2022-08-05T06:03:27.0060306Z       name: 'some-other-branch-03',
2022-08-05T06:03:27.0060708Z       tracking: 'origin/some-other-branch-03'
2022-08-05T06:03:27.0060961Z     }

============================ ≈ 350 lines trimmed (≈ 95 more branches) ============================

2022-08-05T06:03:27.0174817Z     {
2022-08-05T06:03:27.0175135Z       name: 'some-other-branch-98',
2022-08-05T06:03:27.0175531Z       tracking: 'origin/some-other-branch-98'
2022-08-05T06:03:27.0175783Z     },
2022-08-05T06:03:27.0175967Z     {
2022-08-05T06:03:27.0176294Z       name: 'some-other-branch-99',
2022-08-05T06:03:27.0176720Z       tracking: 'origin/some-other-branch-99'
2022-08-05T06:03:27.0176975Z     },
2022-08-05T06:03:27.0177162Z     ... 10 more items
2022-08-05T06:03:27.0177367Z   ],
2022-08-05T06:03:27.0177561Z   tags: []
2022-08-05T06:03:27.0177812Z }
2022-08-05T06:03:27.0178066Z > Pulling from remote...
2022-08-05T06:03:34.4661967Z {
2022-08-05T06:03:34.4663005Z   raw: '',
2022-08-05T06:03:34.4663820Z   remote: 'https://github.com/org-name/repo-name',
2022-08-05T06:03:34.4709698Z   branches: [],
2022-08-05T06:03:34.4709925Z   tags: []
2022-08-05T06:03:34.4711665Z }
2022-08-05T06:03:34.8203641Z xt {
2022-08-05T06:03:34.8204521Z   remoteMessages: It { all: [] },
2022-08-05T06:03:34.8205425Z   created: [],
2022-08-05T06:03:34.8205835Z   deleted: [],
2022-08-05T06:03:34.8206162Z   files: [],
2022-08-05T06:03:34.8206500Z   deletions: {},
2022-08-05T06:03:34.8206907Z   insertions: {},
2022-08-05T06:03:34.8212049Z   summary: { changes: 0, deletions: 0, insertions: 0 }
2022-08-05T06:03:34.8212361Z }
2022-08-05T06:03:34.8212633Z > Checking for conflicts...
2022-08-05T06:03:34.8273839Z nr {
2022-08-05T06:03:34.8274410Z   not_added: [],
2022-08-05T06:03:34.8274803Z   conflicted: [],
2022-08-05T06:03:34.8275184Z   created: [],
2022-08-05T06:03:34.8275547Z   deleted: [],
2022-08-05T06:03:34.8275941Z   ignored: undefined,
2022-08-05T06:03:34.8276853Z   modified: [ 'file1.txt', 'file2.txt' ],
2022-08-05T06:03:34.8278160Z   renamed: [],
2022-08-05T06:03:34.8278557Z   files: [
2022-08-05T06:03:34.8278994Z     sr {
2022-08-05T06:03:34.8279632Z       path: 'file1.txt',
2022-08-05T06:03:34.8280111Z       index: 'M',
2022-08-05T06:03:34.8280603Z       working_dir: ' '
2022-08-05T06:03:34.8281244Z     },
2022-08-05T06:03:34.8281833Z     sr { path: 'file2.txt', index: 'M', working_dir: ' ' }
2022-08-05T06:03:34.8282098Z   ],
2022-08-05T06:03:34.8282570Z   staged: [ 'file1.txt', 'file2.txt' ],
2022-08-05T06:03:34.8282870Z   ahead: 0,
2022-08-05T06:03:34.8283078Z   behind: 0,
2022-08-05T06:03:34.8283385Z   current: 'selected-branch',
2022-08-05T06:03:34.8283767Z   tracking: 'origin/selected-branch',
2022-08-05T06:03:34.8284035Z   detached: false,
2022-08-05T06:03:34.8284270Z   isClean: [Function]
2022-08-05T06:03:34.8284499Z }
2022-08-05T06:03:34.8284730Z > No conflicts found.
2022-08-05T06:03:34.8285034Z > Re-staging files...
2022-08-05T06:03:34.8830473Z > Creating commit...
2022-08-05T06:03:34.8966565Z {
2022-08-05T06:03:34.8968001Z   author: null,
2022-08-05T06:03:34.8968763Z   branch: 'selected-branch',
2022-08-05T06:03:34.8969683Z   commit: '9c3d66591e8861166029c5549f7778755321b50c',
2022-08-05T06:03:34.8970167Z   root: false,
2022-08-05T06:03:34.8972559Z   summary: { changes: 2, insertions: 2, deletions: 0 }
2022-08-05T06:03:34.8973215Z }
2022-08-05T06:03:34.8974278Z 
2022-08-05T06:03:34.8974977Z 
2022-08-05T06:03:34.9009007Z 
2022-08-05T06:03:34.9012440Z > No tag info provided.
2022-08-05T06:03:34.9014303Z > Pushing commit to repo...
2022-08-05T06:03:42.3433112Z 
2022-08-05T06:03:42.3434666Z {
2022-08-05T06:03:42.3434945Z   pushed: [],
2022-08-05T06:03:42.3435157Z   update: {
2022-08-05T06:03:42.3435364Z     head: {
2022-08-05T06:03:42.3435749Z       local: 'refs/heads/selected-branch',
2022-08-05T06:03:42.3436150Z       remote: 'refs/heads/selected-branch'
2022-08-05T06:03:42.3436399Z     },
2022-08-05T06:03:42.3436777Z     hash: { from: 'd3adbe864c9eb', to: '9c3d66591e886' }
2022-08-05T06:03:42.3437026Z   },
2022-08-05T06:03:42.3437466Z   repo: 'https://github.com/org-name/repo-name',
2022-08-05T06:03:42.3437988Z   ref: { local: 'refs/remotes/origin/selected-branch' },
2022-08-05T06:03:42.3438315Z   remoteMessages: It { all: [] }
2022-08-05T06:03:42.3438536Z }
2022-08-05T06:03:42.3438756Z > No tags to push.
2022-08-05T06:03:42.3443077Z ##[endgroup]
2022-08-05T06:03:42.3443378Z > Task completed.
2022-08-05T06:03:42.3445184Z ##[group]Outputs
2022-08-05T06:03:42.3445440Z committed: true
2022-08-05T06:03:42.3445802Z commit_long_sha: 9c3d66591e8861166029c5549f7778755321b50c
2022-08-05T06:03:42.3446181Z commit_sha: 9c3d66591e8861166029c5549f7778755321b50c
2022-08-05T06:03:42.3446443Z pushed: true
2022-08-05T06:03:42.3446663Z tagged: false
2022-08-05T06:03:42.3446901Z tag_pushed: false
2022-08-05T06:03:42.3447287Z ##[endgroup]

    Total task duration was 6:26 in this case. As you can see from the logs, the problem is likely to do with syncing all branches, not just one. I have a rather ‘wide’ repo, i.e. 100+ branches with unrelated stuff in them. Syncing everything is what probably takes a while.

    In the meantime, I’ve temporarily switched to:

          - name: Commit and push
        run: |
          git add --all \
           && git commit --message "Update seed data" \
           && git push \
           || true
        env:
          GIT_AUTHOR_EMAIL: github.actions@my-project.example.org
          GIT_AUTHOR_NAME: GitHub Actions
          GIT_COMMITTER_EMAIL: github.actions@my-project.example.org
          GIT_COMMITTER_NAME: GitHub Actions
        working-directory: some-cwd

    Hope this info helps! 🙌

  8. EndBug commented on Aug 20, 2022

    @EndBug
    EndBug
    on Aug 20, 2022
    Owner

    Thanks for the logs! Yeah, it's definitely the fetch that's slowing down the action: I was able to reproduce this on a fork of the repo the author was having problems too.

    I think I'll add an option to disable fetching so that large repos can use the action too. Disabling the fetch step will impact branch and tag creation, so I'll make sure to add some documentation in the README

  9. EndBug
    added a commit that references this issue on Aug 20, 2022

    feat: add `fetch` input

    a6c0d89
  10. EndBug
    added
    type: featureNew feature or feature request
    and removed
    status: pendingMore info is needed before deciding what to do
    on Aug 22, 2022
  11. EndBug
    self-assigned this
    on Aug 22, 2022

  12. EndBug commented on Aug 22, 2022

    @EndBug
    EndBug
    on Aug 22, 2022
    Owner

    Alright, I've opened a PR (#423) with a new input that would let you customize or skip the git fetch command.
    @kachkaev could you review it?

  13. kachkaev commented on Aug 22, 2022

    @kachkaev
    kachkaev
    on Aug 22, 2022

    Great, thank @EndBug! The diff makes sense 👍

  14. EndBug
    added a commit that references this issue on Aug 22, 2022

    feat: add `fetch` input (#423)

    f7edeca
  15. EndBug
    closed this as completedin #423on Aug 22, 2022

  16. EndBug commented on Aug 22, 2022

    @EndBug
    EndBug
    on Aug 22, 2022
    Owner

    @all-contributors please add @melink14 and @kachkaev for their bug report

  17. allcontributors commented on Aug 22, 2022

    @allcontributors
    allcontributorsbot
    on Aug 22, 2022
    Contributor

    @EndBug

    I've put up a pull request to add @melink14! 🎉

  18. EndBug commented on Aug 22, 2022

    @EndBug
    EndBug
    on Aug 22, 2022
    Owner

    @all-contributors please add @kachkaev for their bug report

  19. allcontributors commented on Aug 22, 2022

    @allcontributors
    allcontributorsbot
    on Aug 22, 2022
    Contributor

    @EndBug

    I've put up a pull request to add @kachkaev! 🎉

  20. EndBug commented on Aug 22, 2022

    @EndBug
    EndBug
    on Aug 22, 2022
    Owner

    Alright, I've just published the feature in v9.1.0 (also v9 and latest)
    Thanks everyone 🎉

  21. melink14
    added a commit that references this issue on Dec 4, 2022

    ci(presubmit): Fixes several quality of life issues in presubmit work…

    f434b81
  22. mergify
    added a commit that references this issue on Dec 5, 2022

    ci(presubmit): Fixes several quality of life issues in presubmit work…

    51ea7e4
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

status: pinnedShould not be labeled as staletype: featureNew feature or feature request

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions