Adding Cross-Cutting Memory Caching to an HttpClientFactory in ASP.NET Core with Polly

Couple days ago I Added Resilience and Transient Fault handling to your .NET Core HttpClient with Polly. Polly provides a way to pre-configure instances of HttpClient which apply Polly policies to every outgoing call. That means I can say things like "Call this API and try 3 times if there's any issues before you panic," as an example. It lets me move the cross-cutting concerns - the policies - out of the business part of the code and over to a central location or module, or even configuration if I wanted.

I've been upgrading my podcast of late at https://www.hanselminutes.com to ASP.NET Core 2.1 and Razor Pages with SimpleCast for the back end. Since I've removed SQL Server as my previous data store and I'm now sitting entirely on top of a third party API I want to think about how often I call this API. As a rule, there's no reason to call it any more often that a change might occur.

I publish a new show every Thursday at 5pm PST, so I suppose I could cache the feed for 7 days, but sometimes I make description changes, add links, update titles, etc. The show gets many tens of thousands of hits per episode so I definitely don't want to abuse the SimpleCast API, so I decided that caching for 4 hours seemed reasonable.

I went and wrote a bunch of caching code on my own. This is fine and it works and has been in production for a few months without any issues.

A few random notes:

• Stuff is being passed into the Constructor by the IoC system built into ASP.NET Core
  • That means the HttpClient, Logger, and MemoryCache are handed to this little abstraction. I don't new them up myself
• All my "Show Database" is, is a GetShows()
  • That means I have TestDatabase that implements IShowDatabase that I use for some Unit Tests. And I could have multiple implementations if I liked.
• Caching here is interesting.
  • Sure I could do the caching in just a line or two, but a caching double check is more needed that one often realizes.
  • I check the cache, and if I hit it, I am done and I bail. Yay!
  • If not, Let's wait on a semaphoreSlim. This a great, simple way to manage waiting around a limited resource. I don't want to accidentally have two threads call out to the SimpleCast API if I'm literally in the middle of doing it already.
    • "The SemaphoreSlim class represents a lightweight, fast semaphore that can be used for waiting within a single process when wait times are expected to be very short."
  • So I check again inside that block to see if it showed up in the cache in the space between there and the previous check. Doesn't hurt to be paranoid.
  • Got it? Cool. Store it away and release as we finally the try.

Don't copy paste this. My GOAL is to NOT have to do any of this, even though it's valid.

public class ShowDatabase : IShowDatabase

{

    private readonly IMemoryCache _cache;

    private readonly ILogger _logger;

    private SimpleCastClient _client;


    public ShowDatabase(IMemoryCache memoryCache,

            ILogger<ShowDatabase> logger,

            SimpleCastClient client)

    {

        _client = client;

        _logger = logger;

        _cache = memoryCache;

    }


    static SemaphoreSlim semaphoreSlim = new SemaphoreSlim(1);


    static HttpClient client = new HttpClient();


    public async Task<List<Show>> GetShows()

    {

        Func<Show, bool> whereClause = c => c.PublishedAt < DateTime.UtcNow;


        var cacheKey = "showsList";

        List<Show> shows = null;


        //CHECK and BAIL - optimistic

        if (_cache.TryGetValue(cacheKey, out shows))

        {

            _logger.LogDebug($"Cache HIT: Found {cacheKey}");

            return shows.Where(whereClause).ToList();

        }


        await semaphoreSlim.WaitAsync();

        try

        {

            //RARE BUT NEEDED DOUBLE PARANOID CHECK - pessimistic

            if (_cache.TryGetValue(cacheKey, out shows))

            {

                _logger.LogDebug($"Amazing Speed Cache HIT: Found {cacheKey}");

                return shows.Where(whereClause).ToList();

            }


            _logger.LogWarning($"Cache MISS: Loading new shows");

            shows = await _client.GetShows();

            _logger.LogWarning($"Cache MISS: Loaded {shows.Count} shows");

            _logger.LogWarning($"Cache MISS: Loaded {shows.Where(whereClause).ToList().Count} PUBLISHED shows");


            var cacheExpirationOptions = new MemoryCacheEntryOptions();

            cacheExpirationOptions.AbsoluteExpiration = DateTime.Now.AddHours(4);

            cacheExpirationOptions.Priority = CacheItemPriority.Normal;


            _cache.Set(cacheKey, shows, cacheExpirationOptions);

            return shows.Where(whereClause).ToList(); ;

        }

        catch (Exception e)

        {

            _logger.LogCritical("Error getting episodes!");

            _logger.LogCritical(e.ToString());

            _logger.LogCritical(e?.InnerException?.ToString());

            throw;

        }

        finally

        {

            semaphoreSlim.Release();

        }

    }

}


public interface IShowDatabase

{

    Task<List<Show>> GetShows();

}

Again, this is great and it works fine. But the BUSINESS is in _client.GetShows() and the rest is all CEREMONY. Can this be broken up? Sure, I could put stuff in a base class, or make an extension method and bury it in there, so use Akavache or make a GetOrFetch and start passing around Funcs of "do this but check here first":

IObservable<T> GetOrFetchObject<T>(string key, Func<Task<T>> fetchFunc, DateTimeOffset? absoluteExpiration = null);

Could I use Polly and refactor via subtraction?

Per the Polly docs:

The Polly CachePolicy is an implementation of read-through cache, also known as the cache-aside pattern. Providing results from cache where possible reduces overall call duration and can reduce network traffic.

First, I'll remove all my own caching code and just make the call on every page view. Yes, I could write the Linq a few ways. Yes, this could all be one line. Yes, I like Logging.

public async Task<List<Show>> GetShows()

{

    _logger.LogInformation($"Loading new shows");

    List<Show> shows = await _client.GetShows();

    _logger.LogInformation($"Loaded {shows.Count} shows");

    return shows.Where(c => c.PublishedAt < DateTime.UtcNow).ToList(); ;

}

No caching, I'm doing The Least.

Polly supports both the .NET MemoryCache that is per process/per node, an also .NET Core's IDistributedCache for having one cache that lives somewhere shared like Redis or SQL Server. Since my podcast is just one node, one web server, and it's low-CPU, I'm not super worried about it. If Azure WebSites does end up auto-scaling it, sure, this cache strategy will happen n times. I'll switch to Distributed if that becomes a problem.

I'll add a reference to Polly.Caching.MemoryCache in my project.

I ensure I have the .NET Memory Cache in my list of services in ConfigureServices in Startup.cs:

services.AddMemoryCache();

STUCK...for now!

AND...here is where I'm stuck. I got this far into the process and now I'm either confused OR I'm in a Chicken and the Egg Situation.

Forgive me, friends, and Polly authors, as this Blog Post will temporarily turn into a GitHub Issue. Once I work through it, I'll update this so others can benefit. And I still love you; no disrespect intended.

The Polly.Caching.MemoryCache stuff is several months old, and existed (and worked) well before the new HttpClientFactory stuff I blogged about earlier.

I would LIKE to add my Polly Caching Policy chained after my Transient Error Policy:

services.AddHttpClient<SimpleCastClient>().

    AddTransientHttpErrorPolicy(policyBuilder => policyBuilder.CircuitBreakerAsync(

            handledEventsAllowedBeforeBreaking: 2,

            durationOfBreak: TimeSpan.FromMinutes(1)

    )).

    AddPolicyHandlerFromRegistry("myCachingPolicy"); //WHAT I WANT?

However, that policy hasn't been added to the Policy Registry yet. It doesn't exist! This makes me feel like some of the work that is happening in ConfigureServices() is a little premature. ConfigureServices() is READY, AIM and Configure() is FIRE/DO-IT, in my mind.

If I set up a Memory Cache in Configure, I need to use the Dependency System to get the stuff I want, like the .NET Core IMemoryCache that I put in services just now.

public void Configure(IApplicationBuilder app, IPolicyRegistry<string> policyRegistry, IMemoryCache memoryCache)

{

    MemoryCacheProvider memoryCacheProvider = new MemoryCacheProvider(memoryCache);

    var cachePolicy = Policy.CacheAsync(memoryCacheProvider, TimeSpan.FromMinutes(5));

    policyRegistry.Add("cachePolicy", cachePolicy);

    ...

But at this point, it's too late! I need this policy up earlier...or I need to figure a way to tell the HttpClientFactory to use this policy...but I've been using extension methods in ConfigureServices to do that so far. Perhaps some exception methods are needed like AddCachingPolicy(). However, from what I can see:

• This code can't work with the ASP.NET Core 2.1's HttpClientFactory pattern...yet. https://github.com/App-vNext/Polly.Caching.MemoryCache
• I could manually new things up, but I'm already deep into Dependency Injection...I don't want to start newing things and get into scoping issues.
• There appear to be changes between  v5.4.0 and 5.8.0. Still looking at this.
• Bringing in the Microsoft.Extensions.Http.Polly package brings in Polly-Signed 5.8.0...
  • But bringing in Polly.Caching.MemoryCache brings in an unsigned 5.4.0
  • All I need is https://github.com/App-vNext/Polly.Caching.MemoryCache/blob/master/src/Polly.Caching.MemoryCache.Shared/MemoryCacheProvider.cs so I could bring that in an compile it myself (to unblock myself) but then I have to #define PORTABLE. I shouldn't need to do any of this.
  • So Polly.Caching.MemoryCache needs to be updated for 5.8 (signed?) or not...not sure. This signed/unsigned thing is always bad.

I'm likely bumping into a point in time thing. I will head to bed and return to this post in a day and see if I (or you, Dear Reader) have solved the problem in my sleep.

"code making and code breaking" by earthlightbooks - Licensed under CC-BY 2.0 - Original source via Flickr

Sponsor: Check out JetBrains Rider: a cross-platform .NET IDE. Edit, refactor, test and debug ASP.NET, .NET Framework, .NET Core, Xamarin or Unity applications. Learn more and download a 30-day trial!