Skip to content

Add HybridCache GetAsync extension that reads without creating - #7876

Draft
Eng3bbas wants to merge 1 commit into
dotnet:mainfrom
Eng3bbas:hybridcache-getasync
Draft

Eng3bbas wants to merge 1 commit into
dotnet:mainfrom
Eng3bbas:hybridcache-getasync

Conversation

@Eng3bbas

@Eng3bbas Eng3bbas commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Implements the API requested in #5688. The public API still needs API review on that issue before it can merge.

What it adds

HybridCacheExtensions.GetAsync<T>(key, options, cancellationToken) returns ValueTask<(bool Found, T? Value)>.

  • It reads L1, then L2. It never invokes a factory, and it never writes a miss to either tier.
  • It is marked experimental (EXTEXP0018, the existing HybridCache experiment ID). A constant for that ID is added to DiagnosticIds.Experiments.

DefaultHybridCache serves it through the existing stampede pipeline with DisableUnderlyingData:

  • A new IsDefaultResult marker on the shared StampedeState<TState, T> lets every joined caller tell a miss apart from a cached 0 or null.
  • The stampede uses the same TState as the stateless GetOrCreateAsync overload, so it can share work with those callers instead of throwing the wrong-type error.

Other HybridCache implementations fall back to GetOrCreateAsync, with both writes disabled and a factory that records the miss. The XML remarks document that fallback's limitation.

Bug fix found along the way

On the DisableUnderlyingData miss path, SetDefaultResult completed every joined caller with ImmutableCacheItem<T>.GetReservedShared(). That shared item holds a single reservation, but each joined caller releases once when it unwraps the result. With N callers on one stampede, the item was released N times:

  • in debug builds, this trips Debug.Assert(newCount >= 0, "over-release detected");
  • in release builds, the static counter drifts.

The miss now completes with the stampede's own cache item, set to default. That item already holds one reservation per caller. GetReservedShared() has no other callers, so it is removed. JoinedGetOrCreateMissesWithoutUnderlyingDataBalanceReservations covers this through the existing GetOrCreateAsync API.

Tests

GetAsyncTests (16 cases) covers:

  • miss and hit, and a cached default value;
  • defensive copies of mutable types;
  • invalid keys and cancellation;
  • reading from L2, with and without copying the value into L1;
  • a miss never being written to either tier;
  • concurrent misses and hits, cancellable and not;
  • the fallback for non-default implementations.

The full Microsoft.Extensions.Caching.Hybrid.Tests suite passes on net8.0, net9.0 and net10.0 (macOS arm64). The API baseline was regenerated with ApiChief, with the existing assembly version line kept. The README gains a short usage example.

🤖 Generated with Claude Code

Microsoft Reviewers: Open in CodeFlow

Adds HybridCacheExtensions.GetAsync<T>, which reads L1 then L2 and
reports whether a value was found, without invoking a factory or
writing a miss to either tier. It is marked experimental (EXTEXP0018).

DefaultHybridCache serves it through the existing stampede pipeline
with DisableUnderlyingData, and a new IsDefaultResult marker on the
shared stampede state lets every joined caller tell a miss apart
from a cached default value. Other HybridCache implementations fall
back to GetOrCreateAsync with both writes disabled.

This also fixes a reference-count bug on the DisableUnderlyingData
miss path: every joined caller was completed with one shared item
that held a single reservation, so each extra caller over-released
it (a Debug.Assert in debug builds, a drifting counter in release).
The miss now completes with the stampede's own cache item, which
already holds one reservation per caller.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Eng3bbas
Eng3bbas marked this pull request as ready for review October 22, 2026 21:05
@Eng3bbas
Eng3bbas requested review from a team as code owners October 22, 2026 21:05
Copilot AI balanced review requested due to automatic review settings October 22, 2026 21:05

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@Eng3bbas

Copy link
Copy Markdown
Contributor Author

@dotnet-policy-service agree

@dotnet-comment-bot

Copy link
Copy Markdown
Collaborator

‼️ Found issues ‼️

Project Coverage Type Expected Actual
Microsoft.Extensions.Diagnostics.Testing Line 99 98.65 🔻
Microsoft.Extensions.Telemetry Line 93 92.75 🔻
Microsoft.Extensions.AI Line 89 88.4 🔻
Microsoft.Extensions.AI.OpenAI Line 75 68.24 🔻
Microsoft.Extensions.AI.OpenAI Branch 75 57.54 🔻
Microsoft.Extensions.DataIngestion.MarkItDown Line 75 4.46 🔻
Microsoft.Extensions.DataIngestion.MarkItDown Branch 75 0 🔻
Microsoft.Extensions.Diagnostics.ResourceMonitoring Line 99 96.07 🔻
Microsoft.Extensions.Diagnostics.ResourceMonitoring Branch 99 92.76 🔻
Microsoft.Extensions.Diagnostics.ResourceMonitoring.Kubernetes Line 99 97.73 🔻
Microsoft.Extensions.ServiceDiscovery.Abstractions Line 75 42.11 🔻
Microsoft.Extensions.ServiceDiscovery.Abstractions Branch 75 42.86 🔻
Microsoft.Extensions.ServiceDiscovery Line 75 68.11 🔻
Microsoft.Extensions.ServiceDiscovery Branch 75 71.43 🔻
Microsoft.Extensions.ServiceDiscovery.Dns Line 75 71.61 🔻
Microsoft.Extensions.ServiceDiscovery.Yarp Line 75 73.85 🔻
Microsoft.Extensions.ServiceDiscovery.Yarp Branch 75 70 🔻
Microsoft.Extensions.VectorData.Abstractions Line 75 37.39 🔻
Microsoft.Extensions.VectorData.Abstractions Branch 75 22.73 🔻

🎉 Good job! The coverage increased 🎉
Update MinCodeCoverage in the project files.

Project Expected Actual
Microsoft.Extensions.Http.Diagnostics 94 95
Microsoft.Gen.BuildMetadata 97 100
Microsoft.Gen.MetadataExtractor 57 73
Microsoft.Gen.MetricsReports 67 69
Microsoft.Extensions.AI.Abstractions 82 86
Microsoft.Extensions.AI.Evaluation.NLP 0 78
Microsoft.Extensions.Caching.Hybrid 82 85
Microsoft.Extensions.DataIngestion 75 89
Microsoft.Extensions.DataIngestion.Markdig 75 90
Microsoft.Extensions.Http.Resilience 97 100

Full code coverage report: https://dev.azure.com/dnceng-public/public/_build/results?buildId=1645965&view=codecoverage-tab

@svick

svick commented Oct 23, 2026

Copy link
Copy Markdown
Member

Marking as draft for now, see #5688 (comment).

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants