1. Background introduction:
When we store data, we sometimes add three-level structures of local cache, distributed cache and database storage. When we take values, we often have a process like the following:
1. Get the local cache first, and return it directly if the value exists.
2. The local cache does not exist, obtain the distributed cache, directly return it when it exists, and update the local cache
3. The distributed cache does not exist. Query the database, update the distributed cache, update the local cache, and finally return it.
But if for some scenarios, there may be only local cache, only distributed cache, or several combinations of the above three, how can we deal with such changes? How can we abstract a set of methods to deal with changes caused by various different data storage methods?
2. Design ideas:
First, let’s analyze the model of the above process and abstract 5 methods:
In fact, different scenarios are nothing more than a combination of these methods, but the contents are different. Speaking of this, we should have already had ideas and can be implemented using delegation.
3. Detailed design:
① Define a class that contains the delegation of the above five methods;
public class DataOperateInput<T>
{
public Func<T> GetDataFromLocalCache { get; set; } = null; //Get local cached data public Func<T> GetDataFromDistributeCache { get; set; } = null; //Get distributed cache data public Func<T> GetDataFromDb { get; set; } = null; //Get DB data public Action<T> SetDataTolocalCache { get; set; } = null; //Set local cache data public Action<T> SetDataToDistributeCache { get; set; } = null; //Set distributed cache data}② Implement one method and combine these five methods.
public class DataOperate
{
/// <summary>
/// Get the data entry /// </summary>
/// <typeparam name="T"></typeparam>
/// <param name="input"></param>
/// <returns></returns>
public T GetData<T>(DataOperateInput<T> input) where T : class, new()
{
T result = null;
// Need to get from local cache if ( != null)
{
//Calling the local cache delegate method to get the value result = ();
if (result != null)
{
return result;
}
}
//Get the value is empty or not obtained from the local cache. Call the following method to obtain data from the distributed cache and Db return GetDataFromDistributeAndDB(input);
}
/// <summary>
/// Get data from distributed cache and Db /// </summary>
/// <typeparam name="T"></typeparam>
/// <param name="input"></param>
/// <returns></returns>
private T GetDataFromDistributeAndDB<T>(DataOperateInput<T> input) where T : class, new()
{
T result = null;
if ( != null)
{
//Get the value from the cache result = ();
//If you need to set it to cache locally, then set it if (result != null)
{
//If the delegate to set the local cache exists, call it to set the local cache ?.Invoke(result);
}
}
//The value is empty or not obtained from the distributed cache, call the following method to obtain data from Db return GetDataFromDB(input);
}
/// <summary>
/// Get data from the database /// </summary>
/// <typeparam name="T"></typeparam>
/// <param name="input"></param>
/// <returns></returns>
private T GetDataFromDB<T>(DataOperateInput<T> input) where T : class, new()
{
T result = null;
if ( != null)
{
//Take the value from DB result = ();
//If you need to set distributed cache and local cache, then set if (result != null)
{
?.Invoke(result);
?.Invoke(result);
}
}
return result;
}
}③ Detailed implementation of a service class and various GetData and SetData methods;
A. Define an enumeration class, through which data sources can be freely combined
/// <summary>
/// Data source category/// </summary>
[Flags]
public enum DataSourceKind
{
/// <summary>
/// Local cache /// </summary>
LocalCache = 1,
/// <summary>
/// Distributed Cache /// </summary>
DistributeCache = 2,
/// <summary>
/// database /// </summary>
DataBase = 4
}B. Define a specific entity class. For example, I define a User class here
public class User : IUser
{
public long UserId { get; set; }
public string Name { get; set; }
public int Age { get; set; }
public int Sex { get; set; }
}C. Implement a method to obtain user information
/// <summary>
/// Get user data /// </summary>
/// <param name="userId">UserId (can be your own related business code)</param> /// <param name="dataSources">Data source type (the caller can combine it by himself)</param> /// <param name="needUpdatelocal">Is the local cache required to be updated</param> /// <param name="needUpdateDistribute">Is it necessary to update the distributed cache?</param> /// <returns></returns>
public User GetUserInfo(long userId,
DataSourceKind dataSources = ,
bool needUpdatelocal = false,
bool needUpdateDistribute = false)
{
($"======Data Source:{()} Whether to update locally:{needUpdatelocal} Whether to updateRedis:{needUpdateDistribute}======");
//Initialize an input parameter class var input = new DataOperateInput<User>();
//If the value is included from the local cache if (())
{
= () =>
{
//! ! Here you can write specific processing logic for obtaining local cache return GetUserFromLocalCache(userId);
};
}
//If the value is included from the distributed cache if (())
{
= () =>
{
//! ! Here you can write specific processing logic for obtaining distributed caches return GetUserFromRedisCache(userId);
};
if (needUpdatelocal)
{
= (value) =>
{
//! ! Here you can write specific processing logic for setting local cache SetUserToLocalCache(value);
};
}
}
//If the value is included to cache the database if (())
{
= () =>
{
//! ! Here you can write specific processing logic for obtaining database data return GetUserFromDB(userId);
};
if (needUpdateDistribute)
{
//! ! Here you can write specific processing logic for setting distributed cache = (value) =>
{
SetUserToRedisCache(value);
};
}
if (needUpdatelocal)
{
//! ! Here you can write specific processing logic for setting local cache = (value) =>
{
SetUserToLocalCache(value);
};
}
}
//Execute the input we combined var result = new DataOperate().GetData(input);
("=============================================\n");
return result;
}The above code describes the use of the method encapsulated GetData, some of which delegated methods need to be implemented in detail, and I did not write them in detail here. The following lists the codes for GetUserFromLocalCache, GetUserFromRedisCache, GetUserFromDB, SetUserToLocalCache, and SetUserToRedisCache for testing.
/// <summary>
/// Get user information from local cache/// </summary>
/// <param name="userId"></param>
/// <returns></returns>
private User GetUserFromLocalCache(long userId)
{
User user = null;
if (userId == 1 )
{
user = new User
{
UserId = userId,
Age = 10,
Name = $"BigOrange_{userId}",
Sex = 1
};
}
if (user == null)
{
($"Getting values from local cache Not found UserId={userId}");
}
else
{
($"Getting values from local cache UserId={} Name={} ");
}
return user;
}
/// <summary>
/// Get user information from Redis cache/// </summary>
/// <param name="userId"></param>
/// <returns></returns>
private User GetUserFromRedisCache(long userId )
{
User user = null;
if (userId == 1 || userId == 2 )
{
user = new User
{
UserId = userId,
Age = 10,
Name = $"BigOrange_{userId}",
Sex = 1
};
}
if (user == null)
{
($"fromRedisCache access value Not found UserId={userId}");
}
else
{
($"fromRedisCache access value UserId={} Name={}");
}
return user;
}
/// <summary>
/// Get user information from DB/// </summary>
/// <param name="userId"></param>
/// <returns></returns>
private User GetUserFromDB(long userId)
{
("Take value from database");
User user = null;
if (userId == 1 || userId == 2 || userId == 3)
{
user = new User
{
UserId = userId,
Age = 10,
Name = $"BigOrange_{userId}",
Sex = 1
};
}
if (user == null)
{
($"fromDBGet the value Not found UserId={userId}");
}
else
{
($"fromDBGet the value UserId={} Name={}");
}
return user;
}
/// <summary>
/// Set user information to local cache/// </summary>
/// <param name="userInfo"></param>
/// <returns></returns>
private bool SetUserToLocalCache(User userInfo)
{
($"Set the value to the local cache:useId = {}");
return true;
}
/// <summary>
/// Set user information to Redis cache/// </summary>
/// <param name="userInfo"></param>
/// <returns></returns>
private bool SetUserToRedisCache(User userInfo)
{
($"Set the value toRediscache:useId = {}");
return true;
}④Test
According to the above code, some test entries were written:
static void Main(string[] args)
{
var userInfoService = new UserInfoService();
/*
* Test cases
User1, User2, User3 exists in the database
Distributed cache User1, User2
Local cache User1
*/
//1. Only get values from local cache (1, );
(2, );
//2. Only get value from Redis cache (2, );
(3, );
//3. Only get values from DB (3, );
(4, );
//4. Get values from local cache and Redis (1, | );
//Not updated to local (2, | , false);
//Update to local (2, | , true);
//5. Take values from Redis and DB (2, | );
(3, | , false, false);
(3, | , false, true);
//6. Take values from local and DB (1, | );
(3, | , false,false);
(3, | , true, false);
//7. Use all three (1, | | ,false,false);
(2, | | ,false,false);
(2, | | , true,false);
(3, | | ,false,false);
(3, | | , true, false);
(3, | | , false, true);
(3, | | , true,true);
();
}Execution results:
======Data source: LocalCache whether to update locally: False whether to update Redis: False======
Get the value from the local cache UserId=1 Name=BigOrange_1
===================================================Data source: LocalCache whether to update locally: False whether to update Redis: False======
Getting the value from local cache not found UserId=2
===================================================Data source:DistributeCache Whether to update locally:False Whether to update Redis:False======
Get the value from Redis cache UserId=2 Name=BigOrange_2
===================================================Data source:DistributeCache Whether to update locally:False Whether to update Redis:False======
Getting the value from Redis cache not found UserId=3
===================================================Data source:DataBase Whether to update locally:False Whether to update Redis:False======
Get the value from DB UserId=3 Name=BigOrange_3
===================================================Data source:DataBase Whether to update locally:False Whether to update Redis:False======
Get value from DB No query UserId=4
===================================================Data source: LocalCache, DistributeCache Whether to update locally: False Whether to update Redis: False======
Get the value from the local cache UserId=1 Name=BigOrange_1
===================================================Data source: LocalCache, DistributeCache Whether to update locally: False Whether to update Redis: False======
Getting the value from local cache not found UserId=2
Get the value from Redis cache UserId=2 Name=BigOrange_2
===================================================Data source: LocalCache, DistributeCache Whether to update locally: True Whether to update Redis:False======
Getting the value from local cache not found UserId=2
Get the value from Redis cache UserId=2 Name=BigOrange_2
Set the value to local cache: useId = 2
===================================================Data source:DistributeCache, DataBase Whether to update locally:False Whether to update Redis:False======
Get the value from Redis cache UserId=2 Name=BigOrange_2
Get the value from DB UserId=2 Name=BigOrange_2
===================================================Data source:DistributeCache, DataBase Whether to update locally:False Whether to update Redis:False======
Getting the value from Redis cache not found UserId=3
Get the value from DB UserId=3 Name=BigOrange_3
===================================================Data source:DistributeCache, DataBase Whether to update locally:False Whether to update Redis:True======
Getting the value from Redis cache not found UserId=3
Get the value from DB UserId=3 Name=BigOrange_3
Set the value to Redis cache: useId = 3
===================================================Data source: LocalCache, DataBase Whether to update locally: False Whether to update Redis: False======
Get the value from the local cache UserId=1 Name=BigOrange_1
===================================================Data source: LocalCache, DataBase Whether to update locally: False Whether to update Redis: False======
Getting the value from local cache not found UserId=3
Get the value from DB UserId=3 Name=BigOrange_3
===================================================Data source: LocalCache, DataBase Whether to update locally: True Whether to update Redis:False======
Getting the value from local cache not found UserId=3
Get the value from DB UserId=3 Name=BigOrange_3
Set the value to local cache: useId = 3
===================================================Data source: LocalCache, DistributeCache, DataBase Whether to update locally: False Whether to update Redis: False======
Get the value from the local cache UserId=1 Name=BigOrange_1
===================================================Data source: LocalCache, DistributeCache, DataBase Whether to update locally: False Whether to update Redis: False======
Getting the value from local cache not found UserId=2
Get the value from Redis cache UserId=2 Name=BigOrange_2
Get the value from DB UserId=2 Name=BigOrange_2
===================================================Data source: LocalCache, DistributeCache, DataBase Whether to update locally: True whether to update Redis:False======
Getting the value from local cache not found UserId=2
Get the value from Redis cache UserId=2 Name=BigOrange_2
Set the value to local cache: useId = 2
Get the value from DB UserId=2 Name=BigOrange_2
Set the value to local cache: useId = 2
===================================================Data source: LocalCache, DistributeCache, DataBase Whether to update locally: False Whether to update Redis: False======
Getting the value from local cache not found UserId=3
Getting the value from Redis cache not found UserId=3
Get the value from DB UserId=3 Name=BigOrange_3
===================================================Data source: LocalCache, DistributeCache, DataBase Whether to update locally: True whether to update Redis:False======
Getting the value from local cache not found UserId=3
Getting the value from Redis cache not found UserId=3
Get the value from DB UserId=3 Name=BigOrange_3
Set the value to local cache: useId = 3
===================================================Data source: LocalCache, DistributeCache, DataBase Whether to update locally: False Whether to update Redis:True======
Getting the value from local cache not found UserId=3
Getting the value from Redis cache not found UserId=3
Get the value from DB UserId=3 Name=BigOrange_3
Set the value to Redis cache: useId = 3
===================================================Data source: LocalCache, DistributeCache, DataBase Whether to update locally: True Whether to update Redis: True======
Getting the value from local cache not found UserId=3
Getting the value from Redis cache not found UserId=3
Get the value from DB UserId=3 Name=BigOrange_3
Set the value to Redis cache: useId = 3
Set the value to local cache: useId = 3
=============================================
4. Let's summarize
Similar to the user information above, the acquisition method may be different for different systems and different performance requirements.
For example: For a background management system, the acquisition of user information is a low-frequency operation, which may only need to be obtained from the database. At this time, the background system generally does not set up local cache and distributed cache, while for an interface system, there may be millions of visits per day. If you only obtain it from the database, it is difficult to bear, so you need to use distributed cache and local cache. The more levels there are, the more changes and combinations there will be. However, if each entity implements its own way, it is wasteful. Therefore, if you can abstract a set of methods, you only need to tell the method of accessing the methods and then get the data you want. Perhaps this is a better way, and how to get and save it will be given by the caller, so that you can deal with complex rules. This is also the reason why so many delegations are used. Since there are many ways to obtain and set User caches, doing so can open up specific operations of obtaining and setting caches to users. In terms of system reconstruction, some general methods can be abstracted, which are relatively low in cost and have better scalability.
5. Off topic
In the above code, there is no thread-safe processing for updating data. Multiple processes update the distributed cache, and multiple threads of the same process update the local cache may all require lock operations.
The above is the entire content of this article. I hope that the content of this article has certain reference value for your study or work. Thank you for your support. If you want to know more about it, please see the following links