Install
Install Docker environment in Ubuntu
If you have not installed Docker on your Ubuntu system, you can install it as follows:
Update the package list of the system:
sudo apt-get update
Install Docker-related dependency packages:
sudo apt-get install -y apt-transport-https ca-certificates curl software-properties-common
Add Docker's official GPG key:
curl -fsSL /linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt//
Add Docker's official software source:
sudo add-apt-repository "deb [arch=$(dpkg --print-architecture signed-by=/etc/apt//)] /linux/ubuntu $(lsb_release -cs) stable"
Update the package list and install Docker:
sudo apt-get update sudo apt-get install -y docker-ce docker-ce-cli
Start the Docker service and set up the power-on self-start:
sudo systemctl start docker sudo systemctl enable docker
Install PostgreSQL
To install pg_rman in Docker, you first need to pull the Docker image of PostgreSQL. You can use the following command to pull the official PostgreSQL image from the Docker Hub:
docker pull postgres
Create a PostgreSQL container Next, create a PostgreSQL container named "my-postgres" and set a username and password for connection use. The following commands can be used:
docker run -d --name my-postgres -e POSTGRES_USER=myuser -e POSTGRES_PASSWORD=mypassword postgres
in,myuserandmypasswordThe username and password to be set are respectively, which you can modify as needed.
Enter the PostgreSQL container Run the following command to enter the shell of the PostgreSQL container:
docker exec -it my-postgres bash
Install and configure pg_rman
Install git in container
If not in the containergitCommand, you can install it through the following steps:
1. Execute the following commands in the container to update the package management tool:
apt-get update
2. Use the following command to installgit：
apt-get install -y git
This will installgitTools that enable you to download the pg_rman source code and perform subsequent compilation and installation steps. After the installation is completed, you can follow the previous steps to continue the installation of pg_rman.
Install pg_rman in container
In the shell of the PostgreSQL container, you can use the following command to install the dependencies of pg_rman and compile and install pg_rman:
1. Install the compilation tools and dependencies:
apt-get update apt-get install -y git build-essential libpq-dev libssl-dev
2.Clone the pg_rman repository:
git clone /ossc-db/pg_rman.git
3. Enter the pg_rman directory:
cd pg_rman
4. Compile and install pg_rman:
make make install
Configure PostgreSQL
After installing pg_rman, you need to configure pg_rman in PostgreSQL for backup and recovery operations. You can edit PostgreSQL configuration files in the shell of the PostgreSQL container using the following command:
1. Edit PostgreSQL configuration file:
vi /var/lib/postgresql/data/
2. Create a directory in the container to store backup files
mkdir -p /var/lib/postgresql/pg_rman_backup
3. Add the following to the configuration file and enable pg_rman:
# Enable the pg_rman pluginshared_preload_libraries = 'pg_rman' # Specify the backup directory for pg_rmanpg_rman.backup_path = '/var/lib/postgresql/pg_rman_backup'
4. Save and exit the configuration file.
5. Restart the PostgreSQL service in the container to make the configuration take effect:
service postgresql restart
Use pg_rman for backup and recovery, now you can use pg_rman for backup and recovery of PostgreSQL databases in the Docker container. Here are some commonly used pg_rman command examples:
test
Create a full backup
pg_rman backup -B /var/lib/postgresql/pg_rman_backup -b my_backup
Create incremental backups
pg_rman backup -B /var/lib/postgresql/pg_rman_backup -b my_backup --incremental
Restore backup
pg_rman restore -B /var/lib/postgresql/pg_rman_backup -i my_backup
Parameter explanation:
pg_rman: This is runningpg_rmanTool commands.backup: This is the operation you want to perform, in this case, the backup operation.-B /var/lib/postgresql/pg_rman_backup:This is-BThe option is followed by a directory path. It specifies the directory in which the backup will be stored, and in this case the backup will be placed in the directory./var/lib/postgresql/pg_rman_backupmiddle.-b my_backup:This is-bThe option is followed by a label or name that identifies the backup. Here, the backup tag is set to "my_backup". This tag helps you identify and manage different backups.- These parameters are used to tell
pg_rmanWhere to store backups and provide an easy-to-identify tag for backups. Typically, you will include additional options and parameters to specify detailed information about the PostgreSQL database and backup configuration.
Configure pg_rman
1. Create configuration files in PostgreSQL container
touch /var/lib/pgsql/data/pg_rman.conf
2. Edit the configuration file
Use a text editor (e.g.viornano) Edit the configuration file, add the following content, and configure it according to your needs:
BACKUP_PATH = '/var/lib/postgresql/pg_rman_backup' BACKUP_MODE = FULL LOG_DIR = '/var/log/pg_rman'
In the above configuration,/var/lib/postgresql/pg_rman_backupReplace with the actual directory path you want to store the backup,/var/log/pg_rmanIt is the directory where the pg_rman backup logs are backed up. These paths refer to the file and directory paths in the PostgreSQL container, and the corresponding folder needs to be created in advance
1. Map the backup path to the host path:Suppose you want to add the backup path inside the container/var/lib/postgresql/pg_rman_backupMap to the host/host/backupDirectory, can be used when running containers-vParameters:
-v /host/backup:/var/lib/postgresql/pg_rman_backup
In this way, the backup file will be in the container/var/lib/postgresql/pg_rman_backupDirectory created, but will also be on the host/host/backupVisible in the directory.
2. Map the log path to the host path:Suppose you want to add the log path inside the container/var/log/pg_rmanMap to the host/host/logDirectory, can be used when running containers-vParameters:
-v /host/log:/var/log/pg_rman
In this way, the pg_rman backup log will be in the container/var/log/pg_rmanDirectory creation, but will also be on the host/host/logVisible in the directory.
This way, you can easily share backup files and backup logs within containers and between hosts. Make sure to replace the path in the command with the actual path you want to use.
3. Restart the PostgreSQL container (container data volumes can be added according to actual conditions)
docker restart my_postgres
This will make the pg_rman configuration take effect.
This step can be ignored by configuring PostgreSQL
For the steps to enable the pg_rman extension within the PostgreSQL container, it depends on your PostgreSQL database configuration and requirements. I will further explain the situation of this step below:
1. Start a PostgreSQL container in Docker. You can create a PostgreSQL-based Docker container using the following command:
docker run -d --name my_postgres -p 5432:5432 -e POSTGRES_USER=myuser -e POSTGRES_PASSWORD=mypassword postgres
in,my_postgresIt is the container name,myuserandmypasswordIt is the username and password of PostgreSQL.
2. Enter the shell of the PostgreSQL container:
docker exec -it my_postgres psql -U myuser
in,myuserIt is the PostgreSQL username set before.
3. Install the pg_rman extension in the PostgreSQL container:
If you want to use it in PostgreSQL containerpg_rmanExtend to perform backup and restore operations, and your PostgreSQL database is not enabled yetpg_rmanExtensions, then you need to execute the following SQL commands inside the PostgreSQL container to enable the extension:
CREATE EXTENSION pg_rman;
This will install the pg_rman extension in PostgreSQL.
- This is usually used to enable it at the database level
pg_rmanExtensions so that you can run directly within the PostgreSQL databasepg_rmanOrder.
- This is usually used to enable it at the database level
-
No need to enable the pg_rman extension in PostgreSQL container(No such step):
- If your PostgreSQL container already contains
pg_rmanExtensions, or it is already enabled in your PostgreSQL databasepg_rmanExtend, then you don't need to enable it inside the container again. - This may be because the PostgreSQL container image you are using already contains
pg_rmanExtensions, or you have configured it in the PostgreSQL database outside the container.
- If your PostgreSQL container already contains
1. Create a backup directory for pg_rman
mkdir /pg_rman_backup
This will create a directory outside the container for storing backups.
2. Configure pg_rman in PostgreSQL container
ALTER SYSTEM SET pg_rman.config_file_directory = '/pg_rman_backup';
This will set the configuration file directory of pg_rman to the backup directory that was created previously.
3. Restart the PostgreSQL container
docker restart my_postgres
This will make the pg_rman configuration take effect.
Set up timed backups
1. Create a script file named pg_rman_backup.sh and place it in the backup directory, such as /pg_rman_backup.
2. Edit the pg_rman_backup.sh script and set the parameters and options for backup. For example, the following is a sample script:
#!/bin/bash # Define backup directory and file nameBACKUP_DIR="/pg_rman_backup" BACKUP_FILE="backup_%Y%m%d_%H%M%S" # Get the current date and timeCURRENT_DATE=$(date "+%Y-%m-%d") CURRENT_TIME=$(date "+%H:%M:%S") # Determine whether the current date is Sunday or Monday to Saturdayif [ "$(date +%u)" == "7" ]; then # If it is Sunday, perform full backup echo "[$CURRENT_DATE $CURRENT_TIME] Performing full backup..." pg_rman backup --backup-path=$BACKUP_DIR --backup-mode=full --backup-name-format=$BACKUP_FILE else # If it is Monday to Saturday, perform incremental backups echo "[$CURRENT_DATE $CURRENT_TIME] Performing incremental backup..." pg_rman backup --backup-path=$BACKUP_DIR --backup-mode=incremental --backup-name-format=$BACKUP_FILE fi
3. Set up the timing tasks, use cron to set up full backups in the container every Sunday at 1 a.m., and perform incremental backups every Monday to Saturday at 1 a.m. For example, the following is a cron configuration example:
# Enter the PostgreSQL containerdocker exec -it my_postgres bash # Edit crontab configuration filecrontab -e # Add the following timing tasks in the crontab configuration file0 1 * * 7 /pg_rman_backup/pg_rman_backup.sh # Perform full backup every Sunday at 1 a.m.0 1 * * 1-6 /pg_rman_backup/pg_rman_backup.sh # Every Monday to Saturday morning1Point to perform incremental backup
This will perform a full backup every Sunday at 1 a.m. and incremental backup every Monday to Saturday at 1 a.m., with a regular backup based on your needs. Note that the time settings for crontab are based on your system time zone, and the above example assumes that your system time zone is UTC. If your system time zone is different, you need to adjust it according to the actual situation.
Stop timing tasks
1. Open the terminal or SSH to connect to your server.
2. Run the following command to edit your crontab configuration file:
crontab -e
In edit mode, find the timed task lines you added before, and delete or comment out the lines. You can add it before the line#Symbols, so that they will be treated as comments and will no longer be executed:
# 0 1 * * 7 /pg_rman_backup/pg_rman_backup.sh # 0 1 * * 1-6 /pg_rman_backup/pg_rman_backup.sh
Save and exit the editor.
This will stop the previously set timing tasks. Crontab will no longer perform these tasks.
Please note that you need to run with a user with appropriate permissionscrontab -eTo edit the crontab configuration file. If you userootThe user sets these tasks, you should userootRun the above commands in the user's identity.
Delete backup
In a Docker environment, to delete the backup file created by pg_rman, you can do it in the following steps:
Go to the host running the PostgreSQL container.
Locate the storage path to the pg_rman backup file on the host. This is usually in a certain mount directory in the container, and can be accessed through the mount option.-vor--volumeSpecified when starting the container. For example, if the following command is used when starting the container:
docker run -v /my/backup/folder:/var/lib/postgresql/pg_rman_backup postgres
Then the backup file will be stored on the host/my/backup/folderin the directory.
Use the command line or file management tool on the host to delete backup files that are no longer needed. For example, you can usermCommands to delete files, for example:
rm /my/backup/folder/my_backup
Note that you should be careful when deleting backup files to ensure that you only delete backup files that you no longer need and that it will not affect the integrity and availability of the database you are using.
In addition, you can also set the pg_rman.backup_path parameter with the -e or --env parameter when running the PostgreSQL container, store the backup file to other directories in the container, and use the corresponding permissions to delete the backup file in the container. For example, you can start a container with the following command and store the backup file in the /my/backup/folder directory:
docker run -e pg_rman.backup_path=/my/backup/folder postgres
Then use the corresponding permissions in the container to delete the backup file. Note that this approach requires sufficient permissions within the container to delete files and requires care to avoid accidental deletion of important data.
How do pg_rman knows the data file location of postgres
pg_rman gets the location of the data file by connecting to the PostgreSQL database because PostgreSQL maintains metadata information about the data file. You do not need to manually specify the data file location, pg_rman automatically detects the data directory in the database.
Typically, pg_rman looks at the PostgreSQL data directory and learns the location of the data file from the database's metadata. This includes the main data directory, table space, and other related information. pg_rman then uses this information to back up and restore the database.
In PostgreSQL, data files are usually stored in the database data directory.basein the directory. pg_rman looks for these data files to ensure that the backup contains all the necessary data.
While pg_rman can automatically identify the location of a data file in most cases, in some special cases you may need to manually specify the location of the data file, especially if your database configuration is different from the standard settings. To manually specify the location of the data file, you can use it in the configuration file of pg_rmanDATABASE_PATHParameters to define the database data directory path. For example:
DATABASE_PATH = '/path/to/postgresql/data'
Replace /path/to/postgresql/data with the actual path to your PostgreSQL data directory.
In short, pg_rman is usually able to automatically detect the location of PostgreSQL data files, but you can also specify them manually if needed. This way, pg_rman can correctly back up and restore the database.
The above is the detailed process steps for implementing PostgreSQL automatic backup in Docker environment. For more information about Docker PostgreSQL automatic backup, please follow my other related articles!