Error: redis hotkey problem

What's Causing This Error

A HotKey refers to a key in Redis that is accessed much more frequently than other keys, leading to increased CPU usage and node load. This can negatively impact the performance of other requests and decrease the success rate of the service.

The impacts of the HotKey issue are outlined as follows:

1. Traffic becomes centralized and may reach the limit of physical network interface cards (NICs).
2. Excessive requests can overload the cache service, potentially causing it to fail.
3. The database can experience a breakdown, triggering a service disruption.

Solution - Here's How To Resolve It

To address the HotKey problem, the following steps are suggested:

1. Consider HotKeys during the design phase to prevent their generation in the database.
2. Implement caches on the service side to alleviate HotKey issues. It is recommended to use a multi-level cache approach, such as combining Redis with a local level-2 cache.
3. Disable hotspot keys by configuring a whitelist for HotKeys on the service side. This allows specific HotKeys to be shielded as required.