With the development of business, more and more configurations will be made in the application system, and there are also different business characteristics between configurations, such as the database configuration of business dependencies, cache information configuration, index storage configuration, etc. This type of configuration is generally relatively stable and will not change frequently. It is usually placed in the project as a configuration file and released with the application.
In addition to these configurations, some configurations are often modified, such as the current limit downgrade switch configuration, whitelist configuration in the service, etc. In addition to frequent changes, these configuration items also require real-time performance. If they are released together with the application, then the service will be re-released for each change, which is very inconvenient.
In order to solve this type of configuration problem, a distributed configuration management platform has emerged. In this lesson, we will learn about the content related to distributed configuration management.
1. Application scenarios of configuration management
In project development, database information and other configuration management are generally launched with the project, such as Java's web system, and it is accustomed to putting database configuration information into this configuration file.
In distributed scenarios, the application scope of configuration management is more extensive. For example, the current limit and downgrade configuration mentioned above, when e-commerce websites hold large-scale promotions, due to the surge in the number of visitors, in order to ensure the stability of the core transaction link, some less important businesses will be downgraded. So how to close non-core services? A distributed configuration management system is needed to be able to manage downgraded services in real time and ensure system security.
In some asynchronous business scenarios, configuration management is also widely used, such as data synchronization is often found in work, and the speed of synchronization needs to be controlled; in some timing tasks, it is necessary to control the timing task triggering and execution time, etc., which can all be achieved through configuration management.
2. How to implement configuration management
The essence of distributed configuration management is the application of push-subscribe mode. The configured application is the subscriber, and the configuration management service is the pusher. The client publishes data to the configuration center, and the configuration center pushes the configuration data to the subscriber.
Configuration management services often encapsulate a client, and the application interacts with the configuration management service based on the client. In actual implementation, the client can actively pull data, or it can be implemented based on event notification.
To implement the configuration management center, the following steps are generally required:
- Extract configuration information and store it in a public place, such as file system, database, and Redis;
- Use the publish/subscribe mode to allow the subsystem to subscribe to these configuration information;
- Open a visual configuration management center to the outside world to operate and maintain configuration information.
3. Characteristic requirements for distributed configuration management
A qualified distributed configuration management system, in addition to configuration release and push, also needs to meet the following characteristics:
- With high availability, the server cluster should be free of single point of failure, and services can be provided as long as there are still surviving nodes in the cluster;
- Fault tolerance ensures that the configuration platform is not available and does not affect the normal operation of the client;
- High performance, for configuration platforms, should be as low as possible performance overhead, and should not bring unacceptable performance losses to the application because of obtaining configuration;
- Reliable storage, including data backup, disaster recovery, consistency, etc., to ensure that configuration data is not lost as much as possible;
- Effective in real time, and client applications can promptly perceive configuration changes.
It can be seen that a good configuration management system not only provides configuration release and push, but also has many requirements for advanced features.
4. Distributed configuration center selection
The distributed configuration management system can be self-developed or open source components, such as Ctrip's open source Apollo, Taobao's Diamond, Baidu's Disconf, etc.
1、Diamond
Taobao's Diamond is a relatively early configuration management component in China. It has a simple design and the configuration information will be persisted into the MySQL database and local disk, and disaster recovery is carried out by adding local files to the database.
The client and the server interact through Http requests and perceive data changes by comparing the MD5 values of the data. During operation, the client will check regularly whether the configuration has changed. Each time the client passes MD5 to the server, and the server will compare whether the MD5 transmitted from the server is the same as the MD5 in its own memory. If the same is true, it means that the data has not changed, and a string indicating that the data remains unchanged is returned to the client; if it is different, it means that the data has changed, and the relevant information of the changed data is returned to the client, and the client will re-request the updated configuration file.
Diamond open source version has not been updated for a long time. It is more suitable for configuration management of small business systems. The source code scale is also relatively small. You can download the corresponding source code to view it. The download address is: github-diamond.
2、Disconf
Disconf is a distributed configuration management platform of Baidu, with the code repository address: knightliao-disconf.
The implementation of Disconf is based on ZooKeeper. Application installation needs to rely on the ZooKeeper environment and dynamic updates are implemented with the ZooKeeper watch mechanism. During the initialization process, the configuration file will be registered. In this way, when the configuration file is updated, ZooKeeper will notify the client, and then the client obtains the latest configuration from the Disconf server and updates it to the local area, thus completing the dynamic configuration update.
For details about Disconf, you can view the design documentation provided by the author:/zh_CN/latest/design/。
3、Apollo
Apollo is Ctrip's open source distributed configuration center. The official description is: Apollo can centrally manage the configuration of different environments and clusters of applications. After configuration modification, it can be pushed to the application side in real time, and has standardized permissions, process governance and other characteristics, and is suitable for microservice configuration management scenarios.
The Apollo server is developed based on Spring Boot and Spring Cloud. It can be run directly after packaging, without the need to install additional application containers such as Tomcat. Apollo supports clients in multiple languages, including Java and .Net clients. Clients do not need to rely on other frameworks to operate, and they have less intrusion into the system.
Compared with Diamond and Disconf, Apollo has always maintained a stable version update, the open source community is also relatively active, the management interface is friendly, suitable for large business systems, and is more recommended. You can learn more about Apollo's repository ctricorp-apollo.
In addition to the above components, ZooKeeper, which is familiar to everyone, is often used as distributed configuration management. Similar to the implementation of Disconf, it relies on the publish subscription function of ZooKeeper and is implemented based on the watch mechanism.
The configuration management selections introduced in the content all provide configuration management functions separately. In fact, in most business systems, configuration management is not a separate function, and is generally integrated with service governance or gateways. For example, Spring Cloud Nacos not only supports service discovery, but also provides configuration management functions. Dubbo’s console, Dubbo Admin, also has a built-in service configuration push function.
Summarize
The above is personal experience. I hope you can give you a reference and I hope you can support me more.