Cache & cache-stampede problem
Cache
When aiming to improve application performance, caching is often the first solution that comes to mind. It’s especially powerful for systems serving static data or those with heavy read operations, as it reduces the load on databases and downstream systems, thereby improving service latency. A simple caching solution can be implemented directly on the application instance using an LRU hash map.
Commonly, this problem is addressed using in-memory data stores like Redis, Oracle Coherence, and Memcached. However, using these solutions introduces additional serialization/deserialization and network overhead. To balance this, some companies employ a multi-layered caching strategy: in-process cache as the L1 cache, in-memory stores as the L2 cache, and the database as the master datastore.
Sounds great!! so what’s the problem 🤔
Cache stampede
Cache stampede problem occurs when multiple requests try to fetch the same data from the backend simultaneously after the cached data has expired, leading to a surge of requests that can overwhelm the backend system. This could create significant negative impact on system performance and availability. Here are some of the consequences —
Redundant Computations
Many requests will end up performing the same data retrieval operation or expensive computation simultaneously, which leads to wasteful use of resources and increased processing overhead
Backend or Database Overload
This sudden surge of requests overwhelm the resources, causing increased latency, timeouts, and potential failures which degrades performance or the system becomes unavailable
Cascading Failures
In distributed systems, a cache stampede in one component can propagate and trigger cascading failures in other dependent components
Solutions
We can mitigate this issue with the following solutions —
comments, claps and feedback are welcome. Happy coding 👨💻
