SoFunction
Updated on 2025-04-11

Springboot multi-data source implementation method

1. Introduction: Necessity and application scenarios of multi-data sources

In modern software development, as business complexity increases and data volumes expand rapidly, an application may need to access multiple databases simultaneously to meet different data processing needs.

For example, an e-commerce system may require a database to process transaction data and another to analyze user behavior. This is the necessity of multi-data source configuration.

Why do you need multiple data sources?

  1. Business logic separation: Different databases can carry different business modules. For example, order processing and user analysis can be carried out in different databases separately, making the business clearer and improving the maintainability of the system.
  2. Performance optimization: By distributing requests to a special database, a single database can be avoided from becoming a performance bottleneck due to excessive requests, thereby improving the performance of the entire application.
  3. Data security and isolation: Sensitive data can be stored separately in a more secure database, while ordinary data can be processed in a conventional security-level database, thereby improving data security.
  4. Technology diversity: Different database technologies (such as relational and non-relational databases) have their specific advantages, and multi-data source configuration allows leveraging the advantages of various database technologies in a single project.

Application scenarios

  1. Big data analysis and real-time business system parallel: In many enterprise-level applications, real-time business databases and big data analysis databases are usually needed to manage separately to avoid complex data analysis operations affecting the performance of core business systems.
  2. Microservice architecture: In a microservice architecture, individual microservices may require independent database instances to maintain the autonomy and decoupling of services.
  3. Legacy system integration: During the integration of new and old systems, it may be necessary to access the database of new and old systems at the same time to ensure the integrity and consistency of data.
  4. Cross-regional data processing: Globalized businesses may require the deployment of databases in different regions to reduce data access latency and comply with geographic data regulations.

Through these scenarios, we can see that the configuration of multi-data sources is not only a technical requirement, but also part of business development and data management strategies.

2. Data source configuration in Spring Boot

Spring Boot greatly simplifies database connections and operations of Java applications. It supports quick up and running with automatic configuration, but requires some additional configuration when dealing with multiple data sources.

Here are detailed instructions on how to configure a single default data source in Spring Boot and how to scale to multiple data sources.

2.1 Introduction to the default data source configuration

In Spring Boot, configuring a default data source is very simple. You just need toorAdd relevant database connection configuration to the file. Spring Boot uses these properties to automatically configure data sources and JPA or JDBC templates.

Example -Configuration:

=jdbc:mysql://localhost:3306/mydatabase
=myuser
=mypass
-class-name=

These configurations tell Spring Boot how to connect to the database and which JDBC driver to use. Spring Boot supports most mainstream databases and automatically configures connection pools (default isHikariCP）。

2.2 How to configure multiple data sources in Spring Boot

Configuring multiple data sources requires more manual settings. You need to define the configuration for each data source and make sure Spring Boot can correctly distinguish and manage them. Here are the steps to configure multiple data sources:

Step 1: Define the data source configuration

First, you need to define a different prefix for each data source in the configuration file. For example:

#Main Data Source=jdbc:mysql://localhost:3306/primary_db
=root
=123456
-class-name=

# Auxiliary data source=jdbc:mysql://localhost:3306/secondary_db
=root
=123456
-class-name=

Step 2: Create a data source configuration class

You need to create configuration classes to load and distinguish these data sources. Each data source should have its own configuration class and @Bean definition.

@Configuration
public class DataSourceConfig {

    @Bean(name = "primaryDataSource")
    @Primary
    @ConfigurationProperties(prefix = "")
    public DataSource primaryDataSource() {
        return ().build();
    }

    @Bean(name = "secondaryDataSource")
    @ConfigurationProperties(prefix = "")
    public DataSource secondaryDataSource() {
        return ().build();
    }
}

In this configuration,@PrimaryThe annotation marks the primary data source, which means that if multiple data sources are available, Spring Boot will default to use the@Primarydata source.

Step 3: Configure JdbcTemplate or EntityManager

For each data source, you may need to configure oneJdbcTemplateorEntityManagerEasy to operate the database.

@Bean
public JdbcTemplate primaryJdbcTemplate(@Qualifier("primaryDataSource") DataSource dataSource) {
    return new JdbcTemplate(dataSource);
}

@Bean
public JdbcTemplate secondaryJdbcTemplate(@Qualifier("secondaryDataSource") DataSource dataSource) {
    return new JdbcTemplate(dataSource);
}

Through the above steps, you can configure and use multiple data sources in your Spring Boot application. This configuration method can not only meet complex business needs, but also help you better manage and maintain different data environments.

3. Integrate MyBatis with Multiple Data Sources

MyBatis is a popular persistence layer framework that supports customized SQL, stored procedures, and advanced mapping. Integrating MyBatis for multi-data source management in Spring Boot applications requires some specific configuration to ensure that each data source is correctly bound to the corresponding SQL map and transaction manager.

Configuring MyBatis to use multiple data sources

To configure MyBatis to use multiple data sources, you need to define theSqlSessionFactoryandTransactionManager. This ensures that MyBatis can provide independent session and transaction control for each database connection.

  1. Define the data source: Define multiple data sources as mentioned earlier.
  2. Create SqlSessionFactory: Create one for each data sourceSqlSessionFactory, which is the core component of MyBatis, used to manage all SQL operations of MyBatis.
  3. Configuring TransactionManager: Configure a transaction manager for each data source to ensure the correct management of transactions.
@Configuration
public class MyBatisConfig {

    @Bean
    @Primary
    public SqlSessionFactory primarySqlSessionFactory(@Qualifier("primaryDataSource") DataSource dataSource) throws Exception {
        SqlSessionFactoryBean sessionFactory = new SqlSessionFactoryBean();
        (dataSource);
        return ();
    }

    @Bean
    public SqlSessionFactory secondarySqlSessionFactory(@Qualifier("secondaryDataSource") DataSource dataSource) throws Exception {
        SqlSessionFactoryBean sessionFactory = new SqlSessionFactoryBean();
        (dataSource);
        return ();
    }

    @Bean
    @Primary
    public DataSourceTransactionManager primaryTransactionManager(@Qualifier("primaryDataSource") DataSource dataSource) {
        return new DataSourceTransactionManager(dataSource);
    }

    @Bean
    public DataSourceTransactionManager secondaryTransactionManager(@Qualifier("secondaryDataSource") DataSource dataSource) {
        return new DataSourceTransactionManager(dataSource);
    }
}

Data source specification of the Mapper interface

Defining differentSqlSessionFactoryAfter that, you need to specify which data source should be used for each Mapper interface or XML mapping file. This is usually achieved by using specific annotations on the Mapper interface or by configuration.

  1. use@MapperScanAnnotation Specify the data source: Can be used in the configuration class@MapperScanAnnotation specifies different packagesSqlSessionFactory。
@Configuration
@MapperScan(basePackages = "", sqlSessionFactoryRef = "primarySqlSessionFactory")
public class PrimaryDataSourceConfig {
    // Primary data source configuration
}

@Configuration
@MapperScan(basePackages = "", sqlSessionFactoryRef = "secondarySqlSessionFactory")
public class SecondaryDataSourceConfig {
    // Secondary data source configuration
}

In this way, you can ensure that the Mapper interface for each data source will only be associated with the specifiedSqlSessionFactoryinteraction, thereby realizing data source isolation. Such configuration makes managing multiple data sources more flexible and efficient in large projects.

4. Dynamic routing of data sources

In a multi-data source environment, dynamic data source routing becomes a powerful strategy that allows applications to select different databases based on specific logic or conditions at runtime. This is especially useful for applications that require dynamic switching of data sources based on user requests, transaction characteristics, or other business logic.

4.1 What is data source routing?

Data source routing is a mechanism that dynamically determines which data source to go to based on certain rules. For example, in a multi-tenant system, each tenant may have its own database, and the system needs to dynamically switch to the corresponding database based on the current user's tenant information.

4.2 Implementing a simple dynamic data source routing

In order to implement dynamic data source routing, we usually need to create a dynamic data source router, which inherits fromAbstractRoutingDataSource, and rewritedetermineCurrentLookupKey()Method to decide which data source key to use.

Here is a simple implementation example:

1. Define dynamic data source classes

import ;

public class DynamicDataSource extends AbstractRoutingDataSource {
    @Override
    protected Object determineCurrentLookupKey() {
        // DynamicDataSourceContextHolder is used to hold the data source identifier used by the current thread        return ();
    }
}

2. Configure dynamic data sources

In Spring configuration file, you need to configureDynamicDataSourceAs the data source and specify the actual data source map.

@Configuration
public class DataSourceConfig {

    @Bean
    public DataSource dataSource() {
        DynamicDataSource dynamicDataSource = new DynamicDataSource();
        // Set the default data source        (primaryDataSource());
        // Configure multiple data sources        Map<Object, Object> dataSourceMap = new HashMap<>();
        ("primary", primaryDataSource());
        ("secondary", secondaryDataSource());
        (dataSourceMap);
        
        return dynamicDataSource;
    }

    @Bean
    public DataSource primaryDataSource() {
        return new HikariDataSource(); // Configure the main data source    }

    @Bean
    public DataSource secondaryDataSource() {
        return new HikariDataSource(); // Configure secondary data sources    }
}

3. Use@TransactionalSpecify the data source

In the service or data access layer, you can specify@TransactionalAnnotatedvalueProperties to select a specific data source.

@Service
public class UserService {

    @Transactional(value = "primaryTransactionManager")
    public void addUser(User user) {
        // Add user using the primary data source    }

    @Transactional(value = "secondaryTransactionManager")
    public User findUser(String username) {
        // Find users using secondary data sources    }
}

Such a setting allows applications to flexibly select appropriate data sources according to business needs at runtime, thereby achieving more complex data operation strategies and higher data operation flexibility.

5. Transaction management configuration

Transaction management becomes particularly important when dealing with multiple data sources, as improper transaction processing can lead to data inconsistency and other serious problems. In Spring Boot applications, correctly configuring and managing transactions under multiple data sources is the key to ensuring data integrity and consistency.

How to manage transactions under multiple data sources

Local Affairs：

  • Local Affairsis the simplest transaction type, it only involves a single data source.
  • In Spring Boot, you can configure a transaction manager for each data source.
  • Then, you can use it in the service layer@TransactionalAnnotation to specify which transaction manager to use.

Example configuration transaction manager:

@Bean
public DataSourceTransactionManager transactionManager1(DataSource dataSource1) {
    return new DataSourceTransactionManager(dataSource1);
}

@Bean
public DataSourceTransactionManager transactionManager2(DataSource dataSource2) {
    return new DataSourceTransactionManager(dataSource2);
}

Use the specified transaction manager:

@Transactional(transactionManager = "transactionManager1")
public void someDataServiceMethod() {
    // Business logic}

Global transactions：

  • Global transactions(also known as distributed transactions) involve multiple data sources or services.
  • In Spring Boot, you can use JTA (Java Transaction API) to configure a global transaction manager such as Atomikos or Bitronix.

Configure the global transaction manager (using Atomikos as an example):

@Bean(initMethod = "init", destroyMethod = "close")
public UserTransactionManager atomikosTransactionManager() throws Throwable {
    UserTransactionManager userTransactionManager = new UserTransactionManager();
    (false);
    return userTransactionManager;
}

@Bean
public JtaTransactionManager transactionManager(UserTransactionManager userTransactionManager) {
    JtaTransactionManager transactionManager = new JtaTransactionManager();
    (userTransactionManager);
    (userTransactionManager);
    return transactionManager;
}

Use the global transaction manager at the service layer:

@Transactional
public void someDataServiceMethod() {
    // Business logic involves multiple data sources}

Configure global and local transactions

When configuring transaction management, you need to decide whether each business scenario is suitable for local transactions or global transactions. Local transactions are simple and have good performance, suitable for single data source operations. Global transactions are suitable for operations that require multiple databases or services, but may result in higher performance overhead.

Make sure to clearly distinguish different transaction managers when configuring and using transactions, especially when using@TransactionalSpecify the correct manager when annotating to avoid data problems caused by transaction management chaos.

6. Practical example: Complete multi-data source configuration example

In this section, we will show you how to configure and use multiple data sources in a Spring Boot application with a practical example. This example will include configuration files, data source configuration, MyBatis integration, and dynamic data source implementations.

Example Overview

Suppose we have an application that needs to access two databases at the same time: one is the main database (for daily business operations), and the other is the audit database (for recording audit logs). We will use MySQL as the database.

Step 1: Add dependencies

First, inAdd necessary dependencies to:

<dependencies>
    <dependency>
        <groupId></groupId>
        <artifactId>spring-boot-starter-web</artifactId>
    </dependency>
    <dependency>
        <groupId></groupId>
        <artifactId>mybatis-spring-boot-starter</artifactId>
        <version>2.1.4</version>
    </dependency>
    <dependency>
        <groupId>mysql</groupId>
        <artifactId>mysql-connector-java</artifactId>
        <scope>runtime</scope>
    </dependency>
    <dependency>
        <groupId></groupId>
        <artifactId>spring-boot-starter-jdbc</artifactId>
    </dependency>
</dependencies>

Step 2: Configure the data source

existConfigure two data sources:

spring:
  datasource:
    primary:
      jdbc-url: jdbc:mysql://localhost:3306/business_db
      username: user1
      password: pass1
      driver-class-name: 
    audit:
      jdbc-url: jdbc:mysql://localhost:3306/audit_db
      username: user2
      password: pass2
      driver-class-name:

Step 3: Configure the data source Bean and MyBatis

Configure two data sources and correspondingSqlSessionFactory：

@Configuration
public class DataSourceConfig {

    @Bean(name = "primaryDataSource")
    @ConfigurationProperties(prefix = "")
    public DataSource primaryDataSource() {
        return ().build();
    }

    @Bean(name = "auditDataSource")
    @ConfigurationProperties(prefix = "")
    public DataSource auditDataSource() {
        return ().build();
    }

    @Bean(name = "primarySqlSessionFactory")
    public SqlSessionFactory primarySqlSessionFactory(@Qualifier("primaryDataSource") DataSource dataSource)
            throws Exception {
        SqlSessionFactoryBean sessionFactory = new SqlSessionFactoryBean();
        (dataSource);
        return ();
    }

    @Bean(name = "auditSqlSessionFactory")
    public SqlSessionFactory auditSqlSessionFactory(@Qualifier("auditDataSource") DataSource dataSource)
            throws Exception {
        SqlSessionFactoryBean sessionFactory = new SqlSessionFactoryBean();
        (dataSource);
        return ();
    }
}

Step 4: Dynamic Data Source Routing

Implementing a simple dynamic data source routing can be extendedAbstractRoutingDataSource：

public class DynamicDataSource extends AbstractRoutingDataSource {
    @Override
    protected Object determineCurrentLookupKey() {
        return ();
    }
}

HereDataSourceContextIt is a tool class that manages data source keys (such as thread-local variables) that determines which data source the current thread should use.

Step 5: Configure and use

Finally, you need to switch the data source as needed in the business logic. This is usually done by using annotations on the service layer method or by calling it directlyDataSourceContextTo set up the data source.

This example provides a basic framework for multi-data source configuration, you can adjust and extend it according to actual needs, such as adding transaction manager configuration, optimizing connection pool settings, etc.

7. Simplify multi-data source management-dynamic

dynamic-datasource-spring-boot-starteris a data source management framework for Spring Boot applications. It is not only a simple component, but a complete solution designed to simplify the configuration and management of multiple data sources. This framework provides the ability to switch data sources dynamically at runtime, which is very useful for applications that need to connect to multiple databases, especially in scenarios where read and write separation or connection to different types of databases are required.

Functions and features

dynamic-datasource-spring-boot-starterThe main functions and features include:

  1. Simplify multi-data source configuration: Make configuration simple and intuitive by declaring multiple data sources in the configuration file of your Spring Boot application.
  2. Dynamic data source switching: Provides method or class-based annotations (e.g.@DS), allowing developers to easily specify which data source should be used for specific operations.
  3. Flexible data source routing: Supports dynamic selection of data sources based on business logic, such as switching data sources based on user request parameters or other logic.
  4. Supports master-slave replication: Very suitable for database read and write separation, improving database operation efficiency and performance.
  5. Integration with the Spring ecosystem: Good integration with other Spring components such as Spring Boot and Spring Data JPA, making it seamlessly work with existing Spring applications.

Application scenarios

  • Read and write separation: In applications that need to deal with a large number of read operations and relatively few write operations, read operations can be pointed to the slave database and write operations can be pointed to the master database.
  • Performance and usability: By distributing requests to multiple data sources, the response time and overall performance of the application can be improved.
  • Data isolation: In a multi-tenant system, each tenant may need a separate database. usedynamic-datasource-spring-boot-starterEach tenant's data source can be easily managed.

Anyway,dynamic-datasource-spring-boot-starterIt is a powerful multi-data source management framework that is ideal for complex applications that require high flexibility and powerful data source management capabilities. It greatly improves development efficiency and application performance by simplifying configuration and increasing the ability to switch data sources at runtime.

Introduce dependencies

First, make sure that your Spring Boot project has been addeddynamic-datasource-spring-boot-starterdependency. If you haven't added it yet, you canAdd the following dependencies:

&lt;dependency&gt;
    &lt;groupId&gt;&lt;/groupId&gt;
    &lt;artifactId&gt;dynamic-datasource-spring-boot-starter&lt;/artifactId&gt;
    &lt;version&gt;Version&lt;/version&gt;
&lt;/dependency&gt;

Version selection based on the dependent version of the actual project

Configure multiple data sources

existorConfigure multiple data sources in the file. For example, use the YAML format to configure it as follows:

spring:
  datasource:
    dynamic:
      primary: master # Set the main data source      datasource:
        master:
          url: jdbc:mysql://localhost:3306/master_db
          username: root
          password: password
          driver-class-name: 
        slave:
          url: jdbc:mysql://localhost:3306/slave_db
          username: root
          password: password
          driver-class-name:

Use @DS annotation

@DSis an annotation for multiple data sources separation, providing the ability to use multiple data sources simultaneously in the same application. This annotation belongs todynamic-datasource-spring-boot-starterLibrary for dynamically managing multiple data sources in Spring Boot applications. Here is how to use it@DSSome basic information and examples of annotations.

Use in method

import ;

@Service
public class SomeService {

    @DS("master")
    public void writeToMaster() {
        // The operation here will use the master data source in the configuration    }

    @DS("slave")
    public void readFromSlave() {
        // The operation here will use the slave data source in the configuration    }
}

Use on the class

If all methods in a class should use the same data source, you can@DSAnnotations are placed at the class level:

import ;

@DS("slave")
@Service
public class ReadOnlyService {
    // All methods will use slave data source    public void queryData() {
        // ...
    }
}

Things to note

  • Ensure the keys of the data source (e.g.masterandslave) matches the name you defined in the configuration file.
  • use@DSWhen annotating, the dynamic data source will switch to the specified data source before the method is called, and switch back to the original data source after the method is executed.
  • Consider transaction management policies, especially when using multiple data sources, multiple transaction managers may need to be configured.

In this way,@DSAnnotations provide a very flexible and powerful mechanism for handling multiple data sources, making it simple and direct to selecting different data sources according to business needs in the same application.

Summarize

The above is personal experience. I hope you can give you a reference and I hope you can support me more.