Skip to content

Check in checked builds that GetEventSize covers every debugger IPC event - #135462

Open
pcshrosbree wants to merge 1 commit into
dotnet:mainfrom
pcshrosbree:fix/geteventsize-coverage-check
Open

pcshrosbree wants to merge 1 commit into
dotnet:mainfrom
pcshrosbree:fix/geteventsize-coverage-check

Conversation

@pcshrosbree

Copy link
Copy Markdown
Contributor

Problem: DbgTransportSession::GetEventSize maps each DB_IPCE_* event type to its wire size, and each new event is added to its switch by hand. A type missing from the switch falls to default:. That asserts in checked builds, but in release builds it silently sends only the event header; for an event with a payload, the receiver then reads stale bytes in its place. This has happened at least three times: #97476 (DISABLE_OPTS, after #97461), #110122 (SET_ENABLE_CUSTOM_NOTIFICATION_RESULT) and #134714 (the four RESOLVE_UPDATE_METADATA_* events, fixed by #134717). Nothing checked the switch against dbgipceventtypes.h, and no test calls GetEventSize.

Change: a _DEBUG-only startup check that every declared event type has a case in GetEventSize, as proposed in #134715.

Details

  • A non-asserting helper. The body of GetEventSize moves unchanged into a private helper, TryGetEventAdditionalSize, which returns false for an unknown type. GetEventSize keeps its signature, callers and behaviour: it calls the helper, and for an unknown type it still logs, asserts and returns the header size.
  • The check. A new _DEBUG-only CheckGetEventSizeCoverage expands IPC_EVENT_TYPE1 and IPC_EVENT_TYPE2 from dbgipceventtypes.h, in the style of DoCompileTimeCheckOnDbgIpcEventTypes. TYPE0 entries are range markers and are skipped. It probes the helper with a zeroed DebuggerIPCEvent of each type (the helper reads only type, plus StepData.rangeCount for STEP/STEP_RESULT, which is 0). On a type with no case it asserts with a message that names the type and says what to do, for example: DB_IPCE_RESOLVE_UPDATE_METADATA_1 has no case in DbgTransportSession::TryGetEventAdditionalSize, the switch behind GetEventSize. Add its size there, or list it as never sent in DbgTransportSession::CheckGetEventSizeCoverage.
  • Where it runs. Once, from the runtime side's DbgTransportSession::Init, which Debugger::Startup reaches whenever the debugger transport is enabled. It also compiles on the debugger side, where nothing calls it.
  • A pointer for new events. A comment in dbgipceventtypes.h, after the rules for adding an event, points at the switch and the check.

No new tests are added: the check runs in every checked CoreCLR process on Unix that has the debugger transport enabled, so the existing checked test runs exercise it.

Classification

Eight declared types have no case in GetEventSize today. The check lists them as deliberately unsized, and that list is the only place a type can be exempted; removing a type from the list makes the check cover it.

Type Why it is not sized
UNHANDLED_EXCEPTION, BREAKPOINT_REMOVE_RESULT, LIST_THREADS, SUSPEND_THREAD, RESUME_THREAD, CONTINUE_EXCEPTION Never sent: nothing in the tree uses them beyond the declaration.
METADATA_UPDATE Never sent: debug/di/process.cpp still has a receiver, but the only sender was removed in #103921.
DEBUGGER_INVALID Not an event: the debugger side's sentinel (initialised, asserted against, restored and tested in debug/di/process.cpp).

If you prefer, any of the seven never-sent types can instead be sized in GetEventSize, or have its declaration removed, here or in a follow-up. DEBUGGER_INVALID stays because it is a live sentinel.

Validation

On linux-x64: ./build.sh clr+libs -c Release -rc Checked (which builds both libcoreclr.so and libmscordbi.so), then ./src/tests/build.sh Checked generatelayoutonly, then a small IL program assembled with Core_Root/ilasm and run under Core_Root/corerun.

  • Clean tree, default environment: runs to completion, no assert.
  • DOTNET_EnableDiagnostics_Debugger=0: runs to completion; the check is not reached (confirmed with a planted build as well).
  • CoreCLR Release: the check is compiled out.

Each plant below was applied alone, followed by a CoreCLR rebuild and the same run; after restoring, the tree was rebuilt and runs clean again.

Plant Result
delete case DB_IPCE_RESOLVE_UPDATE_METADATA_1: asserts at startup naming DB_IPCE_RESOLVE_UPDATE_METADATA_1
delete case DB_IPCE_RESOLVE_UPDATE_METADATA_1_RESULT: asserts naming DB_IPCE_RESOLVE_UPDATE_METADATA_1_RESULT
delete case DB_IPCE_RESOLVE_UPDATE_METADATA_2: asserts naming DB_IPCE_RESOLVE_UPDATE_METADATA_2
delete case DB_IPCE_RESOLVE_UPDATE_METADATA_2_RESULT: asserts naming DB_IPCE_RESOLVE_UPDATE_METADATA_2_RESULT
remove DB_IPCE_METADATA_UPDATE from the never-sent list asserts naming DB_IPCE_METADATA_UPDATE

Not run: Windows, macOS, arm64 and the Debug configuration; DOTNET_EnableDiagnostics=0. On Windows FEATURE_DBGIPC is off, so the transport source is not compiled there.

Limits

  • The check catches an event type that is missing from GetEventSize. It does not catch a case that is present but sized wrongly.
  • It runs only where the debugger transport is enabled (Unix, and not under DOTNET_EnableDiagnostics=0 or DOTNET_EnableDiagnostics_Debugger=0), and only in checked and debug builds.

Fixes #134715

Note

This change and description were prepared with AI assistance (Anthropic Claude and OpenAI Codex), under my direction. I reviewed them before posting.

DbgTransportSession::GetEventSize maps each DB_IPCE_* event type to its
wire size. A type missing from its switch falls to the default case,
which asserts in checked builds but in release builds sends only the
event header, so the receiver reads stale bytes in place of the payload.
This has happened at least three times (dotnet#97476, dotnet#110122, dotnet#134714), and
nothing checked the switch against dbgipceventtypes.h.

Add a checked-build check, CheckGetEventSizeCoverage, that expands
IPC_EVENT_TYPE1 and IPC_EVENT_TYPE2 from dbgipceventtypes.h (TYPE0
entries are range markers and are skipped) and asserts, naming the type,
on any declared type that GetEventSize does not size. So that the probe
does not trip GetEventSize's own assert, the switch moves into a
non-asserting helper, TryGetEventAdditionalSize, which reads only the
event type (plus StepData.rangeCount, zero in the probe event).
GetEventSize calls the helper and keeps its signature, its assert and
its result. The check runs once, from the runtime side's
DbgTransportSession::Init, which Debugger::Startup reaches whenever the
debugger transport is enabled. It also compiles on the debugger side,
where nothing calls it.

Eight declared types have no case today. The check lists them as
deliberately unsized, and that list is the only place a type can be
exempted:
- never sent, no sender anywhere in the tree: UNHANDLED_EXCEPTION,
  BREAKPOINT_REMOVE_RESULT, LIST_THREADS, SUSPEND_THREAD, RESUME_THREAD
  and CONTINUE_EXCEPTION;
- never sent, receiver only: METADATA_UPDATE, whose only sender was
  removed in dotnet#103921;
- DEBUGGER_INVALID, the debugger side's sentinel value, not an event.

A comment in dbgipceventtypes.h, beside the rules for adding an event,
points at GetEventSize and the check.

The check catches a type that is missing from the switch. It does not
catch a case that is present but sized wrongly.

Fixes dotnet#134715
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 3 pipeline(s).
13 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @steveisok, @tommcdon, @dotnet/dotnet-diag
See info in area-owners.md if you want to be subscribed.

@dotnet-policy-service dotnet-policy-service Bot added the community-contribution Indicates that the PR has been added by a community member label Oct 9, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-Diagnostics-coreclr community-contribution Indicates that the PR has been added by a community member

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Check that every debugger IPC event type is sized by DbgTransportSession::GetEventSize

1 participant