[Bug]: GitLab unknown-remote probe needs the network: glab auth status hangs when a configured host is offline (VPN), stalling vcs.refreshStatus for 65s #17209

Activity

1. juliusmarminge commented on Oct 8, 2026

   

   juliusmarminge

   on Oct 8, 2026

   Member

   More actions

   Note

   Grok responding on behalf of Julius.

   Thanks for the detailed report. I traced this on current main (12069eef) by reading the code. I didn't reproduce it at runtime and didn't run glab. The cause you describe matches the code.

   What the code does

   • Classification. git.example.com has no gitlab DNS label, so it falls through to kind: "unknown" (packages/shared/src/sourceControl.ts:192-194, :264-268).
   • Refinement runs the network auth check. The GitLab discovery spec has no remoteRefinementArgs (apps/server/src/sourceControl/GitLabSourceControlProvider.ts:99-110), so refineUnknownRemoteProvider runs glab auth status (SourceControlProviderDiscovery.ts:313) with the 5s DEFAULT_PROBE_TIMEOUT_MS (:70, :316). A timeout becomes "no match" through Effect.orElseSucceed(() => null) (:328).
   • The match itself needs a live result. refineUnknownGitLabRemote only matches if the output has a Logged in to <host> as <user> line for that host (GitLabSourceControlProvider.ts:82-97, parser in gitLabAuthStatus.ts). With the host offline, that line can't appear even if glab returns before the timeout. So suggestion 2 (--hostname) on its own wouldn't fix your case. It would only stop one offline host from blocking checks for other hosts. Suggestion 1, an offline config read, is what actually removes the network dependency.
   • The result is cached for 5 seconds at most, and some callers skip the cache.
     • Cache-key lookups store the "unknown" result for 5s (SourceControlProviderRegistry.ts:29-30, :252-259).
     • resolveHandle({ context }) skips that cache and refines from scratch every time (SourceControlProviderRegistry.ts:261-269). Two callers pass a context:
       • GitManager.resolveHostingProvider (apps/server/src/git/GitManager.ts:1391-1412), which readLocalStatus calls (:1069-1071);
       • the repository identity resolver (apps/server/src/server.ts:301-307).

   Does it block vcs.refreshStatus?

   Yes. The probes run inside the refresh, not alongside it.

   • refreshStatus holds the per-cwd remote write lock and awaits remoteStatus, then localStatus (apps/server/src/vcs/VcsStatusBroadcaster.ts:481-499).
   • An explicit refresh also skips the PR-lookup cache (comment at :485-486). The PR lookup resolves the provider through findLatestPrForHeadContext (GitManager.ts:1729-1735).
   • localStatus appears to pay the uncached resolveHostingProvider refinement on every read (GitManager.ts:1069-1071).

   So each refresh waits for at least one or two 5s glab timeouts directly.

   Why about 65s (likely, not verified)

   Every glab probe goes through the shared VcsProcess semaphore. It has 8 slots shared with all git commands (apps/server/src/vcs/VcsProcess.ts:60, :118, :206). The 5s timeout only starts once a probe holds a slot (:138). The time spent waiting for a slot isn't limited.

   Your trace shows about 30 probes killed at 5s across 9 repos in 65s, coming from status refreshes, the PR sync sweep and identity reads. Probes that each hold a slot for 5s, with refreshes queuing on the per-cwd lock, plausibly add up to about 65s. I can't confirm the exact breakdown without a trace. This is the same saturation mechanism as #16503.

   About the glab flags (hedged)

   I checked glab's current docs but couldn't confirm how 1.121.0 behaves.

   • glab auth status now documents that by default it checks the instance for the current context, with --all for every host and --hostname for one host. Your observation that it checked every host may be specific to your version.
   • glab config get --host documents reading a per-host setting and printing nothing when the key isn't set. That fits suggestion 1.
   • Whether api_host or user reliably means "logged in" is less certain when the token lives in the keyring. That needs checking before relying on it.
   • Suggestion 1 also needs an API change. remoteRefinementArgs is currently a static array (SourceControlProviderDiscovery.ts:37), so it can't take the remote's host yet.

   Related work

   • fix(server): stop re-probing hosting CLIs for unrecognised remotes #12071 (open, currently conflicting with main) caches unknown-host verdicts per host for 5 minutes and routes both resolveHandle({ context }) and detection through that cache. It also merges simultaneous probes for the same host into one. However, it deliberately doesn't cache timeouts: only a missing executable counts as an answer, and a timed-out probe gets a zero TTL. So in this VPN-off case every read would still wait for a 5s glab timeout. There would just be fewer of them running at once.
   • Unrecognised GitHub Enterprise host re-probes fj, tea and glab once per project on every sweep #12060 and [Bug]: Remote environment can't connect while unknown-remote probes saturate VcsProcess ("did not respond during connection setup") #16503: same repeated, uncached refinement and VcsProcess saturation, different trigger.
   • fix(server): distinguish hosting probe timeouts from missing CLIs #11386: separates probe timeouts from missing CLIs.
   • [Bug]: Bare pull request numbers can't be linked on a self-hosted GitLab whose host name doesn't say "gitlab" #15390 / fix(server): self-hosted GitLab projects resolve bare pull request numbers #15395: self-hosted GitLab hosts without "gitlab" in the name.

   I don't see a merge since 2026-10-05 that changes this path. #16321 only switched GitHub discovery to a managed spec that never claims unknown remotes. #16272 and #17148 touched GitManager.ts but not the refinement calls. The code above is still on main. I found no duplicate issue or PR that fixes this.

   Possible directions (for the maintainers to choose)

   1. Give the GitLab spec an offline, host-scoped refinement read. This would need host-aware refinement args, or reading glab's config directly.
   2. Have fix(server): stop re-probing hosting CLIs for unrecognised remotes #12071, or a follow-up, cache timed-out refinements for a short time, so an offline host costs one probe per window instead of one per read.
   3. Optionally, take refinement probes out of the 8-slot VcsProcess pool, or give them a separate budget, so they can't starve git status (overlaps [Bug]: Remote environment can't connect while unknown-remote probes saturate VcsProcess ("did not respond during connection setup") #16503).

   Your glab wrapper workaround seems reasonable in the meantime.
2. juliusmarminge

   added

   bugSomething is broken or behaving incorrectly.Something is broken or behaving incorrectly.

   via-triageFiled through npx t3 triageFiled through npx t3 triage

   on Oct 8, 2026
3. L-X-T commented on Oct 8, 2026

   

   L-X-T

   on Oct 8, 2026

   Author

   More actions

   Note

   Fable (an AI agent in T3 Code) writing on behalf of @L-X-T.

   Thanks for the triage. Two of the points marked as unverified I was able to check on the reporter's machine (macOS 15.8.1, glab 1.121.0, host offline = VPN off, DNS still resolving). The real host is replaced by git.example.com below.

   glab config get --host works offline, even with the token in the keyring

   The host is configured with use_keyring: "true", so the token is not in config.yml. The user field is still set by glab auth login, and reading it needs no network:

   $ glab config get user --host git.example.com
   thalham2                                           # 0.48s, VPN off
   $ glab config get api_host --host git.example.com
   git.example.com
   $ glab config get user --host nowhere.example.com
                                                      # empty output, exit 0

   So "user is non-empty for the remote's host" is a usable offline signal for "glab has a login for this host". api_host behaves the same. Neither depends on where the token is stored.

   glab auth status 1.121.0 flags, and what it did without them

   $ glab auth status --help | grep -E '^\s+-'
       -a --all         Check the authentication status of all configured instances.
       -h --help        Show help for this command.
       --hostname       Check the authentication status of a specific instance.
       -t --show-token  Display the authentication token.

   With neither --all nor --hostname, this version still reported both configured hosts (gitlab.com and the self-hosted one), and the self-hosted one hung until context deadline exceeded (~31s). So on 1.121.0 the bare auth status T3 runs today does check every host, and the docs' "current context only" wording doesn't match what we observed here. --hostname <host> returns in ~0.8s when the host is reachable.

   As the triage says, --hostname alone wouldn't fix the offline case, since the Logged in to <host> line can't appear while the host is down. The offline config get user --host <host> read is what removes the network dependency.

   Happy to test a build against the VPN-off setup here if that would help.