1. When will multiple data sources be used?
In Java development, "multi-data source" refers to configuring and using multiple different database connections in one application. Typically, a Java application connects to a single database. However, in some complex application scenarios, multiple different databases may be required, and multiple data sources need to be configured.
1.1. Specific meaning of multiple data sources
- Multiple database instances: Multiple data sources can refer to multiple database instances of the same type (such as MySQL). For example, an application may need to connect to two different MySQL databases simultaneously, one for storing user information and the other for storing order information.
- Different types of databases: Multiple data sources can also refer to different types of databases. For example, an application may need to connect to a MySQL database to store user information, and also to an Oracle database to store financial data.
- Different access policies: Multi-data source configuration may also be used to implement different database access policies, such as read-write separation (one data source is used for write operations and the other is used for read operations), or different tenants use different data sources under a multi-tenant architecture.
1.2. Why do you need multiple data sources?
- Business Requirements: Different business modules may need to be stored in different databases. For example, financial data and user data may be stored in two separate databases separately for better management and security control.
- System architecture: In large distributed systems, sub-stores and tables are usually used to deal with the challenge of massive data, and different database instances may be distributed on different servers.
- Read and write separation: In order to improve the performance of the system, especially in high concurrency scenarios, it is common to separate the read and write operations of the database. Read operations can obtain data from multiple read-only slave libraries (Slaves), while write operations are written to the master library (Master).
- Migration or compatibility: During the system migration or upgrade process, you may need to access both new and old database systems at the same time, or you may need to be compatible with different versions of the database.
- Multi-tenant support: In SaaS applications, each tenant's data may be isolated and stored in a different database to ensure the independence and security of the data.
2. Three ways to implement multiple data sources in Java
2.1 Based on AbstractRoutingDataSource provided by Spring
Implementing multi-data source configuration using AbstractRoutingDataSource provided by Spring is an effective method for dynamic data source management. By inheriting AbstractRoutingDataSource, you can dynamically select the data source to use based on certain conditions (such as request context, current thread information, etc.). This method is very suitable for scenarios where data sources need to be switched dynamically according to business logic, such as read and write separation, library division and table division, etc.
The detailed steps for implementing multi-data source configuration based on AbstractRoutingDataSource are as follows:
2.1.1 Create a DynamicDataSource class
First, create a class that inherits AbstractRoutingDataSource to implement the dynamic switching logic of the data source.
@Component
@Primary
public class DynamicDataSource extends AbstractRoutingDataSource {
// The data source identifier currently used public static ThreadLocal<String> name=new ThreadLocal<>();
// Write data source @Autowired
DataSource dataSource1;
// Read data source @Autowired
DataSource dataSource2;
@Override
protected Object determineCurrentLookupKey() {
return ();
}
@Override
public void afterPropertiesSet() {
// Initialize all data sources for targetDataSources Map<Object, Object> targetDataSources=new HashMap<>();
("W",dataSource1);
("R",dataSource2);
(targetDataSources);
// Set the default data source for defaultTargetDataSource (dataSource1);
();
}
}
2.1.2 Configuring the data source
Next, we need to configure multiple data sources.
@Configuration
public class DataSourceConfig {
/**
database1Configuration
/
@Bean
@ConfigurationProperties(prefix = ".datasource1")
public DataSource dataSource1() {
// The underlying layer will automatically get the configuration and create a DruidDataSource return ().build();
}
/**
database2Configuration
/
@Bean
@ConfigurationProperties(prefix = ".datasource2")
public DataSource dataSource2() {
// The underlying layer will automatically get the configuration and create a DruidDataSource return ().build();
}
@Bean
public DataSourceTransactionManager transactionManager1(DynamicDataSource dataSource){
DataSourceTransactionManager dataSourceTransactionManager = new DataSourceTransactionManager();
(dataSource);
return dataSourceTransactionManager;
}
@Bean
public DataSourceTransactionManager transactionManager2(DynamicDataSource dataSource){
DataSourceTransactionManager dataSourceTransactionManager = new DataSourceTransactionManager();
(dataSource);
return dataSourceTransactionManager;
}
}
2.1.3 Custom annotations to facilitate the distinction between different data sources
At the business layer, we can set the identifier of the current data source to dynamically switch the data source by annotating.
@Target({,})
@Retention()
public @interface WR {
String value() default "W";
}
At the business layer, we can set the identifier of the current data source to dynamically switch the data source by annotating. After registering the DynamicDataSource you implemented as the default DataSource instance, you only need to change the name ID in advance every time you use DataSource to quickly switch the data source.
@Component
@Aspect
public class DynamicDataSourceAspect implements Ordered {
// Front @Before("within(.dynamic_datasource..*) && @annotation(wr)")
public void before(JoinPoint point, WR wr){
String name = ();
(name);
}
@Override
public int getOrder() {
return 0;
}
}
2.1.4 Service sets data source through annotations
Add annotations to the specific usage methods of the business layer to realize data source switching
@Service
public class UserImplService implements UserService {
@Autowired
UserMapper userMapper;
@Override
@WR("R") // Library 1 public List<User> list() {
return ();
}
@Override
@WR("W") // Library 2 public void save(User friend) {
(friend);
}
}
2.2 Register multiple SqlSessionFactory using MyBatis
In a Spring environment, if there are multiple data sources and you need to configure separate SqlSessionFactory and SqlSessionTemplate for each data source, you can do this by configuring multiple SqlSessionFactory. If you want to register multiple data sources using the MyBatis framework, you need to manually register the core objects such as DataSource, SqlSessionFactory, and DataSourceTransactionManager under MyBatis.
The detailed steps for implementing multi-data source configuration based on MyBatis are as follows:
2.2.1 Registration of different data sources
Data source 1:
@Configuration
// Inherit mybatis:// 1. Specify the scanned mapper interface package (main library)// 2. Specify which sqlSessionFactory is used (main library)@MapperScan(basePackages = ".dynamic_datasource_mybatis.",
sqlSessionFactoryRef="wSqlSessionFactory")
public class WMyBatisConfig {
@Bean
@ConfigurationProperties(prefix = ".datasource1")
public DataSource dataSource1() {
// The underlying layer will automatically get the configuration and create a DruidDataSource return ().build();
}
@Bean
@Primary
public SqlSessionFactory wSqlSessionFactory()
throws Exception {
final SqlSessionFactoryBean sessionFactory = new SqlSessionFactoryBean();
// Specify the main library (dataSource1());
// You can manually specify the corresponding files of the main library /*(new PathMatchingResourcePatternResolver()
.getResources("classpath:mapper/order/*.xml"));*/
return ();
}
@Bean
@Primary
public DataSourceTransactionManager wTransactionManager(){
DataSourceTransactionManager dataSourceTransactionManager = new DataSourceTransactionManager();
(dataSource1());
return dataSourceTransactionManager;
}
@Bean
public TransactionTemplate wTransactionTemplate(){
return new TransactionTemplate(wTransactionManager());
}
}
Data Source 2:
@Configuration
// Inherit mybatis:// 1. Specify the scanned mapper interface package (main library)// 2. Specify which sqlSessionFactory is used (main library)@MapperScan(basePackages = ".dynamic_datasource_mybatis.",
sqlSessionFactoryRef="rSqlSessionFactory")
public class RMyBatisConfig {
@Bean
@ConfigurationProperties(prefix = ".datasource2")
public DataSource dataSource2() {
// The underlying layer will automatically get the configuration and create a DruidDataSource return ().build();
}
@Bean
@Primary
public SqlSessionFactory rSqlSessionFactory()
throws Exception {
final SqlSessionFactoryBean sessionFactory = new SqlSessionFactoryBean();
// Specify the main library (dataSource2());
// You can manually specify the corresponding files of the main library /*(new PathMatchingResourcePatternResolver()
.getResources("classpath:mapper/r/*.xml"));*/
return ();
}
@Bean
public DataSourceTransactionManager rTransactionManager(){
DataSourceTransactionManager dataSourceTransactionManager = new DataSourceTransactionManager();
(dataSource2());
return dataSourceTransactionManager;
}
@Bean
public TransactionTemplate rTransactionTemplate(){
return new TransactionTemplate(rTransactionManager());
}
2.2.2 Specific use of the business layer
In an application, when the specific DAO layer uses SqlSessionFactory and SqlSessionTemplate, MyBatis will use the corresponding data source according to the package path specified by @MapperScan.
@Service
public class UserImplService implements UserService {
@Autowired
private RUserMapper rFriendMapper;
@Autowired
private WUserMapper wFriendMapper;
// Read -- Read library @Override
public List<User> list() {
return ();
}
// Save -- Write library @Override
public void save(User user) {
(user);
}
// Save -- Write library @Override
public void saveW(User user) {
("xman11");
(user);
}
// Save -- Read library @Override
public void saveR(User user) {
("xman");
(user);
}
}
2.3 Using dynamic-datasource framework
Multi-datasource configuration in Java projects is very convenient to use the dynamic-datasource-spring-boot-starter framework. This framework can support automatic switching of multiple data sources, configuration management of dynamic data sources and other functions.
The detailed steps for implementing multi-data source configuration based on the dynamic-datasource-spring-boot-starter framework are as follows:
2.3.1 Add dependencies
First, you need to add the dependency of dynamic-datasource-spring-boot-starter in the file.
<dependency>
<groupId></groupId>
<artifactId>dynamic-datasource-spring-boot-starter</artifactId>
<version>3.5.0</version> <!-- Please adjust the version according to the project needs -->
</dependency>
2.3.2 Configuring the data source
Configure multiple data sources in the file. Each data source can be given a unique name (such as master and slave)
spring:
datasource:
dynamic:
primary: master # Default data source or data source group, which is used when no data source is specified strict: false # Set to true to check whether the data source in the configuration file is valid datasource:
master: # Data Source 1 driver-class-name:
url: jdbc:mysql://localhost:3306/master_db
username: master_user
password: master_pass
slave: # Data Source Two driver-class-name:
url: jdbc:mysql://localhost:3306/slave_db
username: slave_user
password: slave_pass
2.3.3 Configuring the data source
Use the @DS annotation to specify that a method or class uses a specific data source.
@Service
public class UserImplService implements UserService {
@Autowired
UserMapper userMapper;
@Override
@DS("master")
public void save(User user) {
(user);
}
@Override
@DS("slave_1") // From the library, if multiple configurations are configured according to the underscore naming method, you can specify the prefix (group name) public List<User> list() {
return ();
}
}
For MyBatis, dynamic-datasource also applies. Just configure MyBatis normally and use the @DS annotation on the Mapper method.
import ;
import ;
import ;
@Mapper
public interface UserMapper {
@DS("master")
@Select("SELECT * FROM user WHERE id = #{id}")
User getUserFromMaster(int id);
@DS("slave")
@Select("SELECT * FROM user WHERE id = #{id}")
User getUserFromSlave(int id);
}
The dynamic-datasource framework provides a simple and efficient way to implement multi-data source configuration and dynamic switching. Through annotation, you can easily switch data sources to meet the needs of multiple data sources in your business.
3 Summary
These three ways to implement multiple data sources have their own advantages and disadvantages and are suitable for different scenarios. The specific method to choose must be judged based on actual needs. In addition, the way to configure multiple data sources is more than that. For example, when using Spring Data JPA, you can also implement multiple data sources by configuring multiple EntityManagers. Each EntityManager can be associated with a specific DataSource, enabling operations on multiple databases. Therefore, in multi-data source configuration, it is crucial to flexibly choose the appropriate solution.
Source code address:/arkhamYJ/
This is the end of this article about the three implementation methods of Java multi-data source. For more related Java multi-data source content, please search for my previous articles or continue browsing the related articles below. I hope everyone will support me in the future!