Stopping a cache stampede is less about the hit rate and more
IMemoryCache is a trap because GetOrCreateAsync isn't actually atomic. It doesn't lock. If 50 requests miss the cache at the same time, they all trigger the factory method, all hit the database, and all try to write the result.I put together a real-world scenario to test this using a fake DB that simulates a 200ms latency. Here is the typical (and problematic) implementation:
app.MapGet("/products/memory/{id}", async (string id, IMemoryCache cache, FakeDb db) =>
await cache.GetOrCreateAsync($"mem:product:{id}", entry =>
{
entry.AbsoluteExpirationRelativeToNow = TimeSpan.FromMinutes(5);
return db.LoadProductAsync(id, flavor: "memory");
}));To fix this, you need to move to HybridCache (stable since .NET 9). It handles request deduplication per key, meaning only the first request hits the DB while the others wait for that specific task to complete.
Deployment Steps
1. Register the service in your DI container:
builder.Services.AddHybridCache();2. Update your endpoint to use the new provider:
app.MapGet("/products/hybrid/{id}", async (string id, HybridCache cache, FakeDb db, CancellationToken ct) =>
await cache.GetOrCreateAsync(
$"hyb:product:{id}",
async token => await db.LoadProductAsync(id, flavor: "hybrid"),
new HybridCacheEntryOptions { Expiration = TimeSpan.FromMinutes(5) },
cancellationToken: ct));Performance Deep Dive
After a warmup pass to eliminate JIT overhead, the results were stark:
- IMemoryCache cold burst: 50 db calls, 215 ms wall time
- HybridCache cold burst: 1 db calls, 208 ms wall time
The wall time is nearly identical because the waiting requests still have to wait for the single DB query to finish. However, the load on the database drops from 50 queries to 1. In a production environment, this is the difference between a stable system and a database spiral where concurrent queries slow each other down, widening the miss window and letting even more requests pile in.
Once the cache is warm, both methods perform equally. The real win here is surviving the expiry window. For anyone building a high-throughput AI workflow or data-heavy API, switching to HybridCache is a low-effort, high-impact optimization.