The delay mechanism of Redis lock, commonly known as the "watchdog" mechanism, is to deal with abnormal situations such as crashes or network partitions when executing tasks by clients holding the lock, which causes the lock to be unreleased, thereby avoiding the occurrence of deadlocks.
1. What is a "watchdog"
The main function of the watchdog mechanism is to automatically renew the lock, ensuring that the lock will not expire before the node completes the task. Specifically, when a node acquires a lock, the watchdog will periodically check the expiration time of the lock and extend the expiration time of the lock if necessary to ensure that the node can complete the task smoothly.
2. Analysis
Here is the core code snippet of the Redisson watchdog mechanism:
// Initialize the watchdog threadprivate void startWatchdog() {
// Check the lock status every 10 seconds long delay = 10 * 1000;
watchdogFuture = (() -> {
try {
// Check the lock currently held checkAndExtendLocks();
} catch (Exception e) {
// Handle exceptions handleWatchdogException(e);
}
}, delay, delay, );
}
// Check and extend the expiration time of the lockprivate void checkAndExtendLocks() {
for (RLock lock : locks) {
if (()) {
// Update the expiration time of the lock ();
}
}
}
// Update the expiration time of the lockprivate void extendLeaseTime() {
String script = "if ('exists', KEYS[1]) == 1 then " +
"('pexpire', KEYS[1], ARGV[1]); " +
"return 1; " +
"else " +
"return 0; " +
"end";
// Execute Redis script to update the expiration time of the lock (new DefaultRedisScript<>(script, ), (lockKey), leaseTime);
}
In the above code, the startWatchdog method starts a timing task, checks the currently held lock every 10 seconds, and calls the checkAndExtendLocks method to extend the expiration time of the lock. The extendLeaseTime method updates the expiration time of the lock by executing the Redis script, ensuring that the lock does not expire before the task is completed.
Source code interpretation of tryLock method
The tryLock method in Redisson is one of the core methods of acquiring a lock, providing the function of non-blocking attempt to acquire a lock. The following is the core implementation of the tryLock method and its source code interpretation.
The core code of the tryLock method
@Override
public boolean tryLock(long waitTime, long leaseTime, TimeUnit unit) throws InterruptedException {
long time = (waitTime);
long leaseTimeInMillis = (leaseTime);
long currentTime = ();
long lockExpireTime = currentTime + leaseTimeInMillis;
String lockValue = ().toString();
boolean acquired = tryAcquireLock(lockValue, leaseTimeInMillis);
if (!acquired && time > 0) {
long endTime = currentTime + time;
while (() < endTime) {
acquired = tryAcquireLock(lockValue, leaseTimeInMillis);
if (acquired) {
break;
}
(10); // Retry interval
}
}
if (acquired) {
scheduleExpirationRenewal(lockValue, leaseTimeInMillis);
}
return acquired;
}
In this code, the tryLock method tries to acquire the lock within the specified waiting time and sets the expiration time of the lock. Method parameters include:
waitTime: Waiting time, that is, the time when the lock is continuously attempted to acquire before timeout.
leaseTime: Expiration time of lock.
unit：Time unit.
tryAcquireLock method:The tryAcquireLock method tries to actually acquire the lock, and returns true if it succeeds.
private boolean tryAcquireLock(String lockValue, long leaseTimeInMillis) {
String script = "if ('setnx', KEYS[1], ARGV[1]) == 1 then " +
"('pexpire', KEYS[1], ARGV[2]); " +
"return 1; " +
"else " +
"return 0; " +
"end";
List<Object> keys = (lockKey);
List<Object> args = (lockValue, leaseTimeInMillis);
Long result = (new DefaultRedisScript<>(script, ), keys, args);
return result != null && result == 1;
}
This method executes the Lua script:
Use the setnx command to try to set the key-value pair of the lock, and return 1 if it succeeds.
Use the pexpire command to set the expiration time of the lock.
scheduleExpirationRenewal method
If the lock is successfully acquired, the scheduleExpirationRenewal method will start a watchdog thread to automatically extend the expiration time of the lock.
private void scheduleExpirationRenewal(String lockValue, long leaseTimeInMillis) {
long delay = leaseTimeInMillis / 3;
(() -> {
String script = "if ('get', KEYS[1]) == ARGV[1] then " +
"('pexpire', KEYS[1], ARGV[2]); " +
"end";
List<Object> keys = (lockKey);
List<Object> args = (lockValue, leaseTimeInMillis);
(new DefaultRedisScript<>(script, ), keys, args);
}, delay, delay, );
}
This method starts a timed task, every time the leaseTimeInMillis/3 interval, checks whether the lock is still held by the current thread, and if so, extends its expiration time.
Summary of key pointsThe tryLock method provides the function of non-blocking attempts to acquire locks, allowing multiple attempts to acquire locks within a specified time.
The tryAcquireLock method executes Lua scripts and uses the Redis commands setnx and pexpire to achieve the lock acquisition and expiration time setting.
scheduleExpirationRenewal method starts a timed task, which automatically extends the expiration time of the lock through the Lua script to prevent the lock from expired before the task is completed.
Through the above code and parsing, we can understand more clearly how the tryLock method in Redisson works and its role in distributed lock management.
Advantages and disadvantages of watchdog mechanism
advantage:
Automatic renewal: The watchdog mechanism can automatically renew the lock, ensuring that the lock will not expire before completion.
High reliability: By regularly checking the status of the lock, the watchdog mechanism can ensure the holding status of the lock, thereby improving the reliability of the system.
shortcoming:
Resource consumption: The watchdog mechanism requires background threads to regularly check the status of the lock, which will consume a certain amount of system resources.
Increased complexity: The introduction of watchdog mechanisms increases the complexity of the system and may require additional debugging and maintenance efforts.
Optimization of watchdog mechanism
When using Redisson's watchdog mechanism, the following optimizations can be made for specific application scenarios and system requirements: Reasonably set the inspection frequency: reasonably set the inspection frequency of the watchdog thread according to the task execution time and the load condition of the system, which not only ensures the holding status of the lock, but also reduces the consumption of system resources. Optimize Redis scripts: Use efficient Redis scripts to update the expiration time of locks and reduce the load on the Redis server. Monitoring and alarm: Establish a complete monitoring and alarm mechanism to promptly detect and handle abnormal situations in the watchdog mechanism, and ensure the stability of the system.
III. Case practice
Case 1: Watchdog mechanism optimization in order processing system
Scene description
In an order processing system of a large e-commerce platform, order processing may take a long time. To ensure that the distributed lock does not expire during processing, the system enables Redisson's watchdog mechanism. However, due to the high system load, frequent inspections of watchdog threads lead to system resource consumption problems.
Solution
Set the check frequency reasonably: By adjusting the Redisson configuration, reduce the inspection frequency of watchdog threads to reduce system resource consumption.
Config config = new Config();
()
.setAddress("redis://127.0.0.1:6379")
.setWatchdogTimeout(30000); // Set the watchdog timeout to 30 secondsRedissonClient redissonClient = (config);
2. Optimize Redis scripts: Use Lua scripts to update the expiration time of the lock and reduce the load on the Redis server.
String script = "if ('get', KEYS[1]) == ARGV[1] then " +
"return ('expire', KEYS[1], ARGV[2]) " +
"else return 0 end";
(script, ("myLock"), ("lockValue", "30"));
3. Monitoring and alarm: Establish monitoring and alarm mechanisms to promptly detect and handle abnormal situations in the watchdog mechanism.
// Example: Use Prometheus for monitoring@Autowired
private MeterRegistry meterRegistry;
public void monitorWatchdog() {
("redisson_watchdog_status", ("myLock").isLocked() ? 1 : 0);
}
Case 2: Watchdog mechanism optimization in data processing tasks
Scene description
In a data processing system, each task can take several minutes or even longer to complete. If the watchdog thread stops working for some reason, it may cause the lock to expire and cause data inconsistency.
Solution
Introduce alternate threads: Add a backup thread to monitor the status of the watchdog thread. If it is found that the watchdog thread stops working, start the backup thread immediately for processing.
public class WatchdogBackup implements Runnable {
private final RedissonClient redissonClient;
public WatchdogBackup(RedissonClient redissonClient) {
= redissonClient;
}
@Override
public void run() {
RLock lock = ("myLock");
while (true) {
if (!()) {
("Watchdog stopped, acquiring lock...");
();
}
try {
(5000);
} catch (InterruptedException e) {
().interrupt();
}
}
}
}
public void startBackupWatchdog(RedissonClient redissonClient) {
Thread backupThread = new Thread(new WatchdogBackup(redissonClient));
(true);
();
}
2. Redundancy mechanism: Set up multiple watchdog threads to improve system reliability.
public void startRedundantWatchdogs(RedissonClient redissonClient) {
for (int i = 0; i < 3; i++) {
Thread watchdogThread = new Thread(new WatchdogBackup(redissonClient));
(true);
();
}
}
Through these optimization measures, we can effectively improve the reliability and efficiency of the watchdog mechanism, ensuring that the locks will not expire during long-term task execution, thereby avoiding data inconsistency and system resource consumption problems.
This is the end of this article about in-depth understanding of Redis’s watchdog mechanism. For more related content on Redis watchdog mechanism, please search for my previous articles or continue browsing the related articles below. I hope everyone will support me in the future!