Skip to content

HTTP/2: requests dispatched around a graceful GOAWAY (lastStreamId = 2^31-1) stall or fail with "frameError" and are never requeued #6064

Open

Description

@mohamed-rdr

Bug Description

When an HTTP/2 server retires a session with a graceful GOAWAY (NO_ERROR, lastStreamId = 2^31-1, which is what hyper/h2, Go and nghttp2 send to start a graceful shutdown), requests the client dispatches around that GOAWAY are neither sent nor requeued on a new connection. Most stall until the caller's own timeout or abort; a few reject immediately with:

InformationalError: HTTP/2: "frameError" received - type 1, code 2   (code: UND_ERR_INFO)

None of the failed requests reach the server, so all of them were safe to dispatch on a new connection. While stalled, the pool reports connected: false with requests still pending/running, and no new connection is opened for them.

The same scenario with lastStreamId = 0 fully recovers, thanks to #5473 (fix for #5468). The graceful variant seems to take a different path in onHttp2SessionGoAway: with lastStreamId = 2^31-1, every in-flight request counts as accepted, so nothing is requeued, including requests assigned to the session that Node then refuses to send. Also possibly relevant: kReceivedGoAway makes undici ignore any later GOAWAY on the session, so the final GOAWAY a graceful server sends with the real last stream ID is never used to requeue refused streams.

Reproduction

Standalone reproduction script:

'use strict'

const { test } = require('node:test')
const assert = require('node:assert')
const { createServer, constants } = require('node:http2')
const { once } = require('node:events')
const { Agent, request } = require('undici')

// The server retires each session with GOAWAY every 8 streams while the client keeps 4 requests in flight.
// Requests that fail never reached the server, so they could have been replayed on a new connection.
async function run (lastStreamId) {
  const seen = new Set()
  let streams = 0
  const server = createServer()
  server.on('stream', (stream, headers) => {
    stream.on('error', () => {})
    seen.add(headers[':path'])
    streams += 1
    stream.respond({ ':status': 200 })
    stream.end('ok')
    if (streams % 8 === 0) stream.session.goaway(constants.NGHTTP2_NO_ERROR, lastStreamId)
  })
  server.listen(0, '127.0.0.1')
  await once(server, 'listening')
  const origin = `http://127.0.0.1:${server.address().port}`
  const agent = new Agent({ connections: 1, useH2c: true, h2Options: { maxConcurrentStreams: 4 } })

  const failures = []
  let next = 0
  const worker = async () => {
    while (next < 400) {
      const path = `/r${next++}`
      try {
        const res = await request(origin + path, { dispatcher: agent, signal: AbortSignal.timeout(1000) })
        await res.body.text()
      } catch (err) {
        failures.push(`${err.code ?? err.name}: ${err.message} (reached server: ${seen.has(path)})`)
      }
    }
  }
  await Promise.all(Array.from({ length: 4 }, worker))
  await agent.destroy()
  server.close()
  return failures
}

test('requests survive a GOAWAY with lastStreamId = 0', async () => {
  assert.deepStrictEqual(await run(0), [])
})

test('requests survive a graceful GOAWAY with lastStreamId = 2^31-1', async () => {
  const failures = await run(2 ** 31 - 1)
  assert.deepStrictEqual(failures, [], `${failures.length} of 400 failed, e.g.\n${[...new Set(failures)].slice(0, 3).join('\n')}`)
})

node --test repro-graceful-goaway.cjs:

ok 1 - requests survive a GOAWAY with lastStreamId = 0
not ok 2 - requests survive a graceful GOAWAY with lastStreamId = 2^31-1
  AssertionError [ERR_ASSERTION]: 41 of 400 failed, e.g.
  23: The operation was aborted due to timeout (reached server: false)
  UND_ERR_INFO: HTTP/2: "frameError" received - type 1, code 2 (reached server: false)

Repeated runs of 500 requests with lastStreamId = 2^31-1 failed 46–53 each (mostly stalls, 0–2 frameError); lastStreamId = 0 succeeded 500/500 every time.

Expected Behavior

Requests that never reached the server are dispatched on a new connection, as they are for lastStreamId = 0.

Logs & Screenshots

Under production-like load against a hyper (Rust) server that retires connections every ~60s with graceful_shutdown() (h2c, 1,660 single-connection pools, 4 streams each, ~2,800 req/s), about 1 in 270 retirements failed a request: UND_ERR_INFO: HTTP/2: "frameError" received - type 1, code 2 (73%) and UND_ERR_SOCKET: other side closed (27%). undici 7.x fails all in-flight requests on any GOAWAY, so this is an improvement over 7.x, but the graceful path still drops requests.

Environment

  • undici 8.12.2 and 8.12.3
  • Node.js v24.14.1 on macOS (repro); the frameError failures also show up on Linux / Node 24 in Kubernetes
  • h2c via useH2c: true

Additional context

Related: #5468 / #5473 (same symptom, lastStreamId = 0 case, fixed).

Activity

  1. mohamed-rdr commented on Oct 20, 2026

    @mohamed-rdr
    ContributorAuthor

    Opened #6065. It requeues requests whose HEADERS nghttp2 refused to send after the GOAWAY (the frameError failures). The stalls with the node:http2 server come from nghttp2 ignoring all new streams once it has sent a GOAWAY, with no follow-up GOAWAY, so the client gets no signal for them. A server that sends a final GOAWAY with the real last stream ID (hyper, Go) recovers fully with this change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions