[API Proposal]: Add GetAsync to HybridCache

@nibdev

Background and motivation

There are instances, such as session validation, where a consumer only wants to confirm that an item exists in the cache and not create it. This is not currently (directly) possible with the HybridCache implementation.

API Proposal

namespace Microsoft.Extensions.Caching.Hybrid;

public abstract class HybridCache
{
    public abstract ValueTask<T?> GetAsync<T>(string key, HybridCacheEntryOptions? options = null, CancellationToken cancellationToken = default);
}

API Usage

public async ValueTask<bool> ValidateSession(string sessionId, HybridCache cache) 
{
    var session = cache.GetAsync<Session>(sessionId);
    return (session is not null);
}

Alternative Designs

I can make this work using the existing API but I'm not a fan of my solution so if anyone has a better workaround let me know.

public async ValueTask<T?> GetAsync<T>(string key, HybridCacheEntryOptions? options = null, CancellationToken cancellationToken = default) where T : notnull
{
    try
    {
        return await GetOrCreateAsync<T>(key, _ => throw new CacheMissException(), options, null, cancellationToken);
    }
    catch (CacheMissException)
    {
        return default;
    }
}

More efficient option suggested by @nibdev

public static async Task<(bool Found, T? Result)> TryGetAsync<T>(this HybridCache cache, string key) where T : class
{
    var factoryCalled = false;
    var result = await cache.GetOrCreateAsync(
        key,
        _ =>
        {
            factoryCalled = true;
            return ValueTask.FromResult(default(T));
        },
        new HybridCacheEntryOptions
        {
            Expiration = TimeSpan.Zero,
            LocalCacheExpiration = TimeSpan.Zero,
            Flags = HybridCacheEntryFlags.DisableLocalCacheWrite | HybridCacheEntryFlags.DisableDistributedCacheWrite
        });

    return (!factoryCalled, result);
}

Risks

No response

I have another type of issue with not having a simple Get method. I would like to cache the access token so I don't need to obtain it every time I call some API. The problem is that I want to configure the absolute expiration of the cache entry based on the expiration of the token provided by the factory method.
The solution to my problem would be either

having Get method, or
having an overload of the GetOrCreate method accepting the delegate to create HybridCacheEntryOptions, something like this Func<CancellationToken, T, HybridCacheEntryOptions>

I can make this work using the existing API but I'm not a fan of my solution so if anyone has a better workaround let me know.

I don't know if this is better for you but it probably is a bit less expensive as it does not use exceptions. And it could cache null and still would tell you if not found or null was cached

public static async Task<(bool Found, T? Result)> TryGetAsync<T>(this HybridCache cache, string key)
    where T : class
{
    var factoryCalled = false;
    var result = await cache.GetOrCreateAsync(
       key,
       _ =>
       {
           factoryCalled = true;
           return ValueTask.FromResult(default(T));
       },
       new HybridCacheEntryOptions
       {
           Expiration = TimeSpan.Zero,
           LocalCacheExpiration = TimeSpan.Zero,
           Flags = HybridCacheEntryFlags.DisableLocalCacheWrite | HybridCacheEntryFlags.DisableDistributedCacheWrite
       });

    return (!factoryCalled, result);
}

I don't know if this is better for you but it probably is a bit less expensive as it does not use exceptions. And it could cache null and still would tell you if not found or null was cached

Thanks for the suggestion.

I'd support the first API suggestion. I just replaced my use of IDistributedCache with HybridCache in my solution and found the lack of that API a bit odd, to be honest. While it is possible to implement it ourselves, that shouldn't be necessary.

Apart from the options already discussed in this thread, I also tried to use the HybridCacheEntryFlags.DisableUnderlyingData flag. Based on its documentation, it "Only fetches the value from cache; does not attempt to access the underlying data store." However, if the cache entry does not exist, the following line of code returns null, which is unexpected:

HybridCacheEntryOptions options = new() { Flags = HybridCacheEntryFlags.DisableUnderlyingData };
SomeClass value = await cache.GetOrCreateAsync(someKey, _ => new ValueTask(new SomeClass()), options, ...);

The compiler strongly believes value is non-nullable, so it will complain about null checks, for example. Thus, the proposed API, which returns a ValueTask<T?>, would indeed be very helpful.

This has come up a few times; considering API for net10

@nibdev

Combining the solutions from @nibdev and @ThomasBarnekow :

 public async static Task<T> GetAsync<T>(
     this HybridCache cache, string key)
 {

     return await cache.GetOrCreateAsync(
        key,
        _ =>  ValueTask.FromResult(default(T)),
        new HybridCacheEntryOptions
        {
            Flags = HybridCacheEntryFlags.DisableUnderlyingData
        });

 }

(I haven't checked whether it would compile, but can that be Task<T?>? NRT syntax is fun ... Also: best to throw the options instance into a static readonly field that you can reuse, rather than allocating a new one per-call; I'd also be tempted to lose the async/await and return the VT directly, but that's subjective; adding static to the lambda won't change anything in this case, but is a good idea to ensure you aren't accidentally capturing anything)

Perhaps:

public static ValueTask<T?> GetAsync<T>(
     this HybridCache cache, string key)
     => cache.GetOrCreateAsync(
        key,
        static _ =>  new(default(T)!),
        s_DisableUnderlying);

private static readonly HybridCacheEntryOptions s_DisableUnderlying
    = new() { Flags = HybridCacheEntryFlags.DisableUnderlyingData };

(I haven't checked whether it would compile, but can that be Task<T?>? NRT syntax is fun...

As mentioned elsewhere, the problem with a GetAsync method like this is that there's no way to differentiate between "the value in the cache was null" and "there was nothing in the cache".

In case this limitation is ok, I would be more explicit with the name (naming is hard etc): ValueTask<T?> GetOrDefaultAsync<T>.

Also, why returning a Task instead of a ValueTask, like the GetOrSetAsync already does?

Also: best to throw the options instance into a static readonly field that you can reuse, rather than allocating a new one per-call;

Good catch!

adding static to the lambda won't change anything in this case, but is a good idea to ensure you aren't accidentally capturing anything)

Apropos of this: why not just pass null! as the factory? It will not be called anyway (I tried it and it does work).

Also, why returning a Task instead of a ValueTask, like the GetOrSetAsync already does?

Typo

(I tried it and it does work).

Not at a PC to check, and couldn't remember whether we added a null fault in this case (I agree it makes sense not to, when not enabled)

@nibdev

If we need to know if the cached value was found, @nibdev solution is the way to go.

 public static async ValueTask<(bool Found, T? Value)> TryGetAsync<T>(
    this HybridCache cache, string key)
 {
     var found = true;
     var value = await cache.GetOrCreateAsync(
        key,
        _ => {
            found = false;
            return ValueTask.FromResult(default(T));
        },
        s_DisableWrite);


     return (found, value);
 }


  private static readonly HybridCacheEntryOptions s_DisableWrite
      = new() {
          Flags = HybridCacheEntryFlags.DisableLocalCacheWrite | HybridCacheEntryFlags.DisableDistributedCacheWrite
      };

Usage:

  var v = await _cache.TryGetAsync<int>("key");

  if (v.Found)
  {
     Debug.WriteLine(v.Value);
  }

I think the big point here is that, imho, it's simply not possible to add GetOrDefaultAsync and TryGetAsync as extension methods: they must be native implementations in HybridCache itself. Why? Because of a subtle but very important detail.

Let me explain (and please correct me if I'm wrong).

Both methods are explicitly NOT designed to set a new value in the cache, only to get what's there, right? I mean, for example the GetOrDefault method does not mean "get the value from the cache and if not there SAVE the default value" but "get the value from the cache and if not there RETURN the default value". This will be even more important when the multi-node notifications part will be implemented. There's a big difference (I've been bitten by this multiple times).

To make it formally correct and achieve this behaviour we'd need to use DisableLocalCacheWrite + DisableDistributedCacheWrite, to avoid writing to the cache when nothing was there at all.

But, and this is important, if we use those flags we are now losing on the automatic L1 write in case of an L1 MISS + L2 HIT, which is very important to avoid bombarding L2 every at single call, and basically the whole point of having an L1+L2 cache in the first place.

In theory we may call the method without the 2 flags (to get the auto-L1-save-from-L2) and add them in the factory (like with adaptive caching) but this is not possible because A) there's no way to pass the entry options to the factory and B) the entry options class is intended to be immutable anyway (see here, sealed class + init-only members).

So again, imho there's no way to achieve GetOrDefaultAsync/TryGetAsync with what we currently have (and this is, FWIW, the reasons why in FusionCache I made both methods first class).

Here's an example of both GetOrDefaultAsync and TryGetAsync which are (again, imho) formally correct in behaviour but without the L1-auto-save-from-L2 feature:

public static class HybridCacheExtMethods
{
	private static readonly HybridCacheEntryOptions s_TryGetAsync_EntryOptions = new()
	{
		Flags = HybridCacheEntryFlags.DisableLocalCacheWrite | HybridCacheEntryFlags.DisableDistributedCacheWrite
	};

	public static async ValueTask<(bool Found, T? Value)> TryGetAsync<T>(this HybridCache cache, string key)
	{
		var found = true;
		var value = await cache.GetOrCreateAsync(
			key,
			_ =>
			{
				found = false;
				return ValueTask.FromResult(default(T));
			},
			s_TryGetAsync_EntryOptions
		);

		return (found, value);
	}

	private static readonly HybridCacheEntryOptions s_GetOrDefaultAsync_EntryOptions = new()
	{
		Flags = HybridCacheEntryFlags.DisableLocalCacheWrite | HybridCacheEntryFlags.DisableDistributedCacheWrite | HybridCacheEntryFlags.DisableUnderlyingData
	};

	public static async ValueTask<T?> GetOrDefaultAsync<T>(this HybridCache cache, string key)
	{
		return await cache.GetOrCreateAsync<T?>(
			key,
			null!,
			s_GetOrDefaultAsync_EntryOptions
		);
	}

	public static async ValueTask<T?> GetOrDefaultAlternativeAsync<T>(this HybridCache cache, string key)
	{
		var tmp = await cache.TryGetAsync<T>(key);

		return tmp.Found ? tmp.Value : default;
	}
}

NOTE: I also put an alternative version of the GetOrDefaultAsync method, as an example.

Another smaller point to be made is that, as multiple people pointed out, to have a TryGetAsync method where we can differentiate between "null was in the cache" and "there was nothing in the cache" is to not use a simple T? result type, because it's not enough. Because of this we saw the use of a tuple like (bool Found, T? Value). This is great, except that these tuples have limitations on the fields names (eg: Found and Value) being carried around between different compilation units, so I'm not sure what will happen if the ext methods (and returning tuple) are defined in a package (but maybe I'm wrong and these limits are not valid anymore?). This, btw, is the reason why in FusionCache I created the MaybeValue<T> type, so it was clearly defined and would not suffer these limitations.

Thoughts?

@nibdev

hello @nibdev @jodydonetti

I’ve tested your implementation with the following concurrent scenario, but the results were not as expected:

public async Task TestAsync()
{
    var tasks = new List<Task<(bool, bool?)>>();
    for (var i = 0; i < 100; i++)
    {
        tasks.Add(Task.Run(async () =>
        {
            var found = true;
            var value = await _cacheService.GetOrCreateAsync("test_key",
                _ =>
                {
                    found = false;
                    return ValueTask.FromResult(default(bool?));
                },
                new HybridCacheEntryOptions
                {
                    Flags = HybridCacheEntryFlags.DisableLocalCacheWrite | HybridCacheEntryFlags.DisableDistributedCacheWrite
                });

            return (found, value);
        }));
    }

    var resultsArray = await Task.WhenAll(tasks);
    var results = resultsArray.ToList();
}

Some results have the found flag set to true, which should not be possible.
Somehow, it seems that factory isn't properly executed?

@bezaou

Hi @bezaou

I’ve tested your implementation with the following concurrent scenario, but the results were not as expected:

Just to be on the same page: you mean just my implementation above or also with my 3rd party HybridCache implementation?

Some results have the found flag set to true, which should not be possible. Somehow, it seems that factory isn't properly executed?

The behaviour is strange, I agree, but I'd have to check the code of the Microsoft default HybridCache implementation (see DefaultHybridCache here).

But if I have to guess without looking at the code I think it has to do with some low level impl detail related to the cache stampede protection, like directly sharing an inner task/result or something like that. It may be a case of Leaky Abstraction, but I may be wrong.

I'll try to take a deeper look tomorrow.

@bezaou

Hi @bezaou , I've been able to make some more tests, in particular this (using the ext methods I posted above):

using ConsoleAppHybridStampedeBehaviour;
using Microsoft.Extensions.Caching.Hybrid;
using Microsoft.Extensions.DependencyInjection;

var services = new ServiceCollection();
services.AddHybridCache();
var sp = services.BuildServiceProvider();

var hc = sp.GetRequiredService<HybridCache>();

Console.WriteLine("PARALLEL:");
await Parallel.ForAsync(0, 10, async (_, _) =>
{
	var foo = await hc.TryGetAsync<int>("foo");
	Console.WriteLine($"foo: {foo.Found}, {foo.Value}");
});

Console.WriteLine("SEQUENTIAL:");
for (var i = 0; i < 10; i++)
{
	var foo = await hc.TryGetAsync<int>("foo");
	Console.WriteLine($"foo: {foo.Found}, {foo.Value}");
}

Running this gives me variations of this:

PARALLEL:
foo: False, 0
foo: True, 0
foo: True, 0
foo: True, 0
foo: True, 0
foo: True, 0
foo: True, 0
foo: True, 0
foo: True, 0
foo: True, 0
SEQUENTIAL:
foo: False, 0
foo: False, 0
foo: False, 0
foo: False, 0
foo: False, 0
foo: False, 0
foo: False, 0
foo: False, 0
foo: False, 0
foo: False, 0

or this:

PARALLEL:
foo: False, 0
foo: False, 0
foo: False, 0
foo: False, 0
foo: True, 0
foo: True, 0
foo: True, 0
foo: False, 0
foo: False, 0
foo: True, 0
SEQUENTIAL:
foo: False, 0
foo: False, 0
foo: False, 0
foo: False, 0
foo: False, 0
foo: False, 0
foo: False, 0
foo: False, 0
foo: False, 0
foo: False, 0

So yes, I think I can confirm that the current behaviour for the Microsoft's implementation of HybridCache is "inconsistent" or "non deterministic".

To put it simply, what I mean is that currently you cannot count on the factory being executed when the value is not in the cache (or is skipped via DisableLocalCache/DisableLocalCacheRead).

My supposition for why is still that this is happening because of some internal optimization, like sharing an underlying task or something like that.

Maybe @mgravell can confirm this or shed a light.

I also tried the same code with the FusionCache adapter (see here).

By simply adding a reference to FusionCache and changing this line:

services.AddHybridCache();

with this line:

services.AddFusionCache().AsHybridCache();

the result is consistently this:

PARALLEL:
foo: False, 0
foo: False, 0
foo: False, 0
foo: False, 0
foo: False, 0
foo: False, 0
foo: False, 0
foo: False, 0
foo: False, 0
foo: False, 0
SEQUENTIAL:
foo: False, 0
foo: False, 0
foo: False, 0
foo: False, 0
foo: False, 0
foo: False, 0
foo: False, 0
foo: False, 0
foo: False, 0
foo: False, 0

Hope this helps.

Stumbled upon this issue right now, just before .NET 11 release, are there any plans to implement this?

@DontEatRice

@DontEatRice This won't be included in .Net 11. We'll see what we can do for .Net 12.

@bezaou

I've opened a PR for this: #7876.

It adds GetAsync<T>, which returns ValueTask<(bool Found, T? Value)>. It's an experimental (EXTEXP0018) extension method in Microsoft.Extensions.Caching.Hybrid. It reads L1, then L2, never runs a factory, and never writes a miss to either tier.

About the concurrency problem raised above (@bezaou, @jodydonetti): the factory-flag workaround reports Found = true for callers that join another caller's lookup, because only the first caller's factory runs. For the default implementation, the PR doesn't rely on the factory. It reuses the stampede pipeline with DisableUnderlyingData, and marks a miss on the shared stampede state, so every joined caller sees it. ConcurrentMissesAreAllReportedAsNotFound covers this. Other HybridCache implementations get the factory-based fallback, and its limitation is documented.

While working on it, I also found and fixed a reference-count bug on the existing GetOrCreateAsync + DisableUnderlyingData miss path: joined callers over-released a shared item. The PR description has the details.

This is an extension method rather than a virtual on the abstract HybridCache class only because that class lives in dotnet/runtime. If API review prefers a virtual method, the implementation can move behind it unchanged.

@Eng3bbas

@Eng3bbas Thank you for your contribution, but we won't be able to accept the PR right now. GetAsync shouldn't be an extension method that is specialized for DefaultHybridCache, it should be a virtual method on HybridCache. But implementing an API across dotnet/runtime and dotnet/extensions currently doesn't work well.

So, the current plan is:

I will figure out a sensible way to add a new API to HybridCache and implement it in DefaultHybridCache. I want to get this done before the end of the year, but no promises.
We will agree on the API design and I will get the API through API review.
I will reopen your PR and we will review it.

I will let you know when there's more information. Feel free to ping me in 2027 if I haven't done so.

For the bug, I have opened #7880 to track it. Feel free to create a separate PR for the bugfix, there's nothing blocking that.

Sorry for the complications.

Navigation Menu

Repository navigation

[API Proposal]: Add `GetAsync` to `HybridCache` #5688

Background and motivation

API Proposal

API Usage

Alternative Designs

Risks

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[API Proposal]: Add GetAsync to HybridCache #5688

Description

Background and motivation

API Proposal

API Usage

Alternative Designs

Risks

Activity

ondrej-zoo commented on Dec 4, 2024

nibdev commented on Dec 11, 2024

CoryCharlton commented on Dec 11, 2024

ThomasBarnekow commented on Jan 6, 2025

mgravell commented on Feb 27, 2025

mvxproject commented on Feb 28, 2025

mgravell commented on Feb 28, 2025

jodydonetti commented on Feb 28, 2025

mgravell commented on Feb 28, 2025

mvxproject commented on Mar 1, 2025

jodydonetti commented on Mar 1, 2025

bezaou commented on Apr 17, 2025

jodydonetti commented on Apr 19, 2025

jodydonetti commented on Apr 25, 2025

jodydonetti commented on Apr 25, 2025

DontEatRice commented on Sep 11, 2026

svick commented on Sep 24, 2026

Eng3bbas commented on Oct 22, 2026

svick commented on Oct 23, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

[API Proposal]: Add `GetAsync` to `HybridCache` #5688