当前位置：首页 > news >正文

PostgreSQL 备份恢复：pg_probackup

news 2025/7/5 13:50:55

文章目录

- 前言
- 1. 安装备份工具
- - 1.1 环境介绍
  - 1.2 RPM 安装
  - 1.3 验证
- 2. 配置备份工具
- - 2.1 初始化设置
  - 2.2 创建备份用户
  - 2.3 配置自动归档
- 3. 工具使用介绍
- - 3.1 init
  - 3.2 add-instance
  - 3.3 del-instance
  - 3.4 set-config
  - 3.5 show-config
  - 3.6 set-backup
  - 3.7 backup
  - 3.8 show
  - 3.9 delete
  - 3.10 restore
  - 3.11 catchup
  - 3.12 archive-push
  - 3.13 archive-get
  - 3.14 checkdb
  - 3.15 validate
  - 3.16 merge

前言

pg_probackup 是一款功能强大的开源 PostgreSQL 备份工具，和 MySQL 社区 XtraBackup 类似，本篇文章将和各位共同探索学习。

开源地址：
https://github.com/postgrespro/pg_probackup
文档地址：
https://postgrespro.github.io/pg_probackup

1. 安装备份工具

1.1 环境介绍

环境介绍：CentOS Linux release 7.8.2003 (Core)
数据库版本：PostgreSQL - 12.2

1.2 RPM 安装

# RPM Centos Packages
rpm -ivh https://repo.postgrespro.ru/pg_probackup/keys/pg_probackup-repo-centos.noarch.rpm# pg_probackup-这里写 PG 对应的大版本
yum install pg_probackup-{15,14,13,12,11}
yum install pg_probackup-{15,14,13,12,11}-debuginfo

官方 Github 有安装说明，可以根据环境安装指引安装，上面是 CentOS 环境安装方法。

1.3 验证

RPM 安装完成后会自动配置环境变量，这里我安装的是 12 版本，所以使用 pg_probackup-12 命令操作工具：

pg_probackup-12 --help

2. 配置备份工具

2.1 初始化设置

初始化备份目录：

pg_probackup-12 init -B ${backup_dir}

pg_probackup-12 init -B /data/pgsql12/backup
INFO: Backup catalog ‘/data/pgsql12/backup’ successfully initialized

添加新的备份实例：

# 本地实例
pg_probackup-12  add-instance -B ${backup_dir} -D ${PGDATA} --instance ${instance_name}# 添加远程实例
pg_probackup-12  add-instance -B ${backup_dir} -D ${PGDATA} --instance ${instance_name} --remote-prot=ssh --remote-host=${remote_ip} --remote-port=${remote_ssh_port} --remote-user=${remote_ssh_user} --remote-path=${pg_probackup_dir}

pg_probackup-12 add-instance -B /data/pgsql12/backup/ -D /data/pgsql12/data/ --instance test01
INFO: Instance ‘test01’ successfully initialized

2.2 创建备份用户

PostgreSQL versions 10 - 14 备份用户创建语句：

BEGIN;
CREATE ROLE backup WITH LOGIN;
GRANT USAGE ON SCHEMA pg_catalog TO backup;
GRANT EXECUTE ON FUNCTION pg_catalog.current_setting(text) TO backup;
GRANT EXECUTE ON FUNCTION pg_catalog.set_config(text, text, boolean) TO backup;
GRANT EXECUTE ON FUNCTION pg_catalog.pg_is_in_recovery() TO backup;
GRANT EXECUTE ON FUNCTION pg_catalog.pg_start_backup(text, boolean, boolean) TO backup;
GRANT EXECUTE ON FUNCTION pg_catalog.pg_stop_backup(boolean, boolean) TO backup;
GRANT EXECUTE ON FUNCTION pg_catalog.pg_create_restore_point(text) TO backup;
GRANT EXECUTE ON FUNCTION pg_catalog.pg_switch_wal() TO backup;
GRANT EXECUTE ON FUNCTION pg_catalog.pg_last_wal_replay_lsn() TO backup;
GRANT EXECUTE ON FUNCTION pg_catalog.txid_current() TO backup;
GRANT EXECUTE ON FUNCTION pg_catalog.txid_current_snapshot() TO backup;
GRANT EXECUTE ON FUNCTION pg_catalog.txid_snapshot_xmax(txid_snapshot) TO backup;
GRANT EXECUTE ON FUNCTION pg_catalog.pg_control_checkpoint() TO backup;
COMMIT;

PostgreSQL versions 15 备份用户创建语句：

BEGIN;
CREATE ROLE backup WITH LOGIN;
GRANT USAGE ON SCHEMA pg_catalog TO backup;
GRANT EXECUTE ON FUNCTION pg_catalog.current_setting(text) TO backup;
GRANT EXECUTE ON FUNCTION pg_catalog.set_config(text, text, boolean) TO backup;
GRANT EXECUTE ON FUNCTION pg_catalog.pg_is_in_recovery() TO backup;
GRANT EXECUTE ON FUNCTION pg_catalog.pg_backup_start(text, boolean) TO backup;
GRANT EXECUTE ON FUNCTION pg_catalog.pg_backup_stop(boolean) TO backup;
GRANT EXECUTE ON FUNCTION pg_catalog.pg_create_restore_point(text) TO backup;
GRANT EXECUTE ON FUNCTION pg_catalog.pg_switch_wal() TO backup;
GRANT EXECUTE ON FUNCTION pg_catalog.pg_last_wal_replay_lsn() TO backup;
GRANT EXECUTE ON FUNCTION pg_catalog.txid_current() TO backup;
GRANT EXECUTE ON FUNCTION pg_catalog.txid_current_snapshot() TO backup;
GRANT EXECUTE ON FUNCTION pg_catalog.txid_snapshot_xmax(txid_snapshot) TO backup;
GRANT EXECUTE ON FUNCTION pg_catalog.pg_control_checkpoint() TO backup;
COMMIT;

用户创建完成后，注意 pg_hba.conf 放开 backup 用户。

2.3 配置自动归档

配置 wal 自动归档需要调整如下参数：

# 自动归档的时间，单位为秒，可酌情设置，建议 1 分钟
max_wal_senders = 60
# 开启归档
archive_mode = 'on'
# WAL 格式，归档必须是 replica 及更高级别
wal_level = 'replica'
# 配置归档命令
archive_command = 'pg_probackup-12 archive-push -B /data/pgsql12/backup --instance test01 --wal-file-path=%p --wal-file-name=%f'

配置完成后，需要重启数据库，可以使用下方命令查看归档信息：

pg_probackup-12 show -B /data/pgsql12/backup --instance test01 --archive

3. 工具使用介绍

使用 pg_probackup-12 --help 可以看到该工具主要分为几块功能，本小节会详细介绍。

pg_probackup-12 - utility to manage backup/recovery of PostgreSQL database.pg_probackup-12 help [COMMAND]pg_probackup-12 versionpg_probackup-12 init -B backup-pathpg_probackup-12 set-backup -B backup-path --instance=instance_name-i backup-id [--ttl=interval] [--expire-time=timestamp][--note=text][--help]pg_probackup-12 show-config -B backup-path --instance=instance_name[--format=format][--help]....................

3.1 init

pg_probackup-12 init -B backup-path

初始化备份目录，安装完 pg_probackup 需要做的事情，这里相当于给 pg_probackup-12 创建一个工作目录，管理备份文件和归档文件 pg_probackup 可以用于管理多个实例的备份。

例如：

pg_probackup-12 init -B /pg_data/backup

将 /pg_data/backup 设置为备份工具的 home 目录。

3.2 add-instance

pg_probackup-12 add-instance -B backup-path -D pgdata-path--instance=instance_name[--external-dirs=external-directories-paths][--remote-proto] [--remote-host][--remote-port] [--remote-path] [--remote-user][--ssh-options]

添加需要备份的实例信息 pg_probackup 可以管理多个实例的备份，可以作为备份管理中心。

添加本地备份实例，例如：

pg_probackup-12  add-instance -B /pg_data/backup -D /data/pgsql12/data --instance node01

将本地数据目录为 /pg_data/backup 的 PostgreSQL 添加到备份管理中心。如果要添加远程备份实例，需要先配置互信。

## 在远程备份实例主机上
# su - postgres
$ ssh-keygen            
$ ssh-copy-id postgres@${备份机_ip}## 在备份机上
# su - postgres
$ ssh-keygen            
$ ssh-copy-id postgres@${备份实例主机_ip}## 测试互信
$ ssh postgres@${对方IP}

添加远程实例：

pg_probackup-12 add-instance -B /data/pgsql12/backup -D /data/pgsql/data --instance test02 --remote-prot=ssh --remote-host=172.16.104.55 --remote-port=22 --remote-user=postgres

INFO: Instance ‘test02’ successfully initialized

3.3 del-instance

pg_probackup-12 del-instance -B backup-path--instance=instance_name[--help]

从备份元数据中心里删除实例信息。例如：

pg_probackup-12  del-instance -B /data/pgsql12/backup/ --instance test02

INFO: Delete: RZXNFS 2023-08-25 14:02:16+08
INFO: Delete: RZXNEX 2023-08-25 14:01:45+08
INFO: Delete: RZXND9 2023-08-25 14:00:45+08
INFO: Delete: RZXNCL 2023-08-25 14:00:21+08
INFO: Delete: RZXNA5 2023-08-25 13:58:53+08
INFO: Delete: RZXN9V 2023-08-25 13:58:43+08
INFO: Instance ‘test02’ successfully deleted

3.4 set-config

pg_probackup-12 set-config -B backup-path --instance=instance_name[-D pgdata-path][--external-dirs=external-directories-paths][--log-level-console=log-level-console][--log-level-file=log-level-file][--log-format-file=log-format-file][--log-filename=log-filename][--error-log-filename=error-log-filename][--log-directory=log-directory][--log-rotation-size=log-rotation-size][--log-rotation-age=log-rotation-age][--retention-redundancy=retention-redundancy][--retention-window=retention-window][--wal-depth=wal-depth][--compress-algorithm=compress-algorithm][--compress-level=compress-level][--archive-timeout=timeout][-d dbname] [-h host] [-p port] [-U username][--remote-proto] [--remote-host][--remote-port] [--remote-path] [--remote-user][--ssh-options][--restore-command=cmdline] [--archive-host=destination][--archive-port=port] [--archive-user=username][--help]

可以配置备份保留策略，及一些配置参数，例如配置备份保留策略：

pg_probackup-12 set-config -B /pg_data/backup --instance node01 --retention-redundancy 7 --retention-window 7

3.5 show-config

pg_probackup-12 show-config -B backup-path --instance=instance_name[--format=format][--help]

查看备份服务的参数配置信息。

3.6 set-backup

pg_probackup-12 set-backup -B backup-path --instance=instance_name-i backup-id [--ttl=interval] [--expire-time=timestamp][--note=text][--help]

修改备份文件的元数据信息。

3.7 backup

pg_probackup-12 backup -B backup-path -b backup-mode --instance=instance_name[-D pgdata-path] [-C][--stream [-S slot-name] [--temp-slot]][--backup-pg-log] [-j num-threads] [--progress][--no-validate] [--skip-block-validation][--external-dirs=external-directories-paths][--no-sync][--log-level-console=log-level-console][--log-level-file=log-level-file][--log-format-console=log-format-console][--log-format-file=log-format-file][--log-filename=log-filename][--error-log-filename=error-log-filename][--log-directory=log-directory][--log-rotation-size=log-rotation-size][--log-rotation-age=log-rotation-age] [--no-color][--delete-expired] [--delete-wal] [--merge-expired][--retention-redundancy=retention-redundancy][--retention-window=retention-window][--wal-depth=wal-depth][--compress][--compress-algorithm=compress-algorithm][--compress-level=compress-level][--archive-timeout=archive-timeout][-d dbname] [-h host] [-p port] [-U username][-w --no-password] [-W --password][--remote-proto] [--remote-host][--remote-port] [--remote-path] [--remote-user][--ssh-options][--ttl=interval] [--expire-time=timestamp] [--note=text][--help]

启动备份的方法，下面详细介绍该功能参数：

-B, --backup-path=backup-path：备份工具 init 的目录。
-b, --backup-mode=backup-mode：备份的模式，有 FULL、PAGE、DELTA、PTRACK 四种模式。
- FULL：创建一个完整备份，其中包含要恢复的集群的所有数据文件。
- PAGE：根据自上次完整或增量备份以来生成的 WAL 文件创建增量备份。仅从数据文件中读取更改的块。
- DELTA：读取数据目录中的所有数据文件，并为自上次备份以来已更改的页面创建增量备份。
- PTRACK：动态创建增量备份跟踪页面更改。
-C, --smooth-checkpoint：将检查点分散在一段时间内。默认情况下， pg_probackup 尝试尽快完成检查点。
–instance=instance_name：实例名，通过指定实例名，备份工具会根据元数据查到该实例的信息。
–stream：如果在备份命令中加入该参数，表示使用流模式备份 WAL 日志。
-S, --slot=SLOTNAME：指定 WAL 流的复制槽。该选项只能与 --stream 标志一起使用。
-j, --threads=NUM：并发几个线程来进行备份。

使用案例，在本地发起备份：

pg_probackup-12 backup -B /data/pgsql12/backup/ --instance test01 -b full

INFO: Database backup start
INFO: wait for pg_start_backup()
INFO: Wait for WAL segment /data/pgsql12/backup/wal/test01/00000002000000020000007E to be archived
INFO: PGDATA size: 2397MB
INFO: Current Start LSN: 2/7E000028, TLI: 2
INFO: Start transferring data files
INFO: Data files are transferred, time elapsed: 39s
INFO: wait for pg_stop_backup()
INFO: pg_stop backup() successfully executed
INFO: stop_lsn: 2/7F0000F0
INFO: Getting the Recovery Time from WAL
INFO: Syncing backup files to disk
INFO: Backup files are synced, time elapsed: 1s
INFO: Validating backup RZXNYO
INFO: Backup RZXNYO data files are valid
INFO: Backup RZXNYO resident size: 2400MB
INFO: Backup RZXNYO completed

使用案例，发起远程备份：

pg_probackup-12 backup -B /data/pg_backup --instance test02 --remote-user='postgres' --remote-host='172.16.104.7' --remote-proto=ssh --stream --remote-port=22 -b full

INFO: Database backup start
INFO: wait for pg_start_backup()
INFO: Wait for WAL segment /data/pg_backup/backups/test02/S04Q23/database/pg_wal/000000020000000200000092 to be streamed
INFO: PGDATA size: 2405MB
INFO: Current Start LSN: 2/92000028, TLI: 2
INFO: Start transferring data files
INFO: Data files are transferred, time elapsed: 40s
INFO: wait for pg_stop_backup()
INFO: pg_stop backup() successfully executed
INFO: stop_lsn: 2/920001A0
INFO: Getting the Recovery Time from WAL
INFO: Syncing backup files to disk
INFO: Backup files are synced, time elapsed: 1s
INFO: Validating backup S04Q23
INFO: Backup S04Q23 data files are valid
INFO: Backup S04Q23 resident size: 2439MB
INFO: Backup S04Q23 completed

接下来测试增量备份：

# 先发起一个物理全量备份
pg_probackup-12 backup -B /pg_data/backup --instance node01 -b full

查看备份信息：

======================================================================================================================================Instance  Version  ID      Recovery Time           Mode  WAL Mode  TLI  Time    Data    WAL  Zratio  Start LSN   Stop LSN    Status  
======================================================================================================================================node01    12       S0GD3F  2023-09-04 16:33:57+08  FULL  ARCHIVE   3/0   44s  2446MB   16MB    1.00  4/E9000028  4/EA000128  OK

测试制造数据变动：

update pgbench_accounts set bid = 6;

基于上次的全备，进行增量备份：

pg_probackup-12 backup -B /pg_data/backup --instance node01 -b page

======================================================================================================================================Instance  Version  ID      Recovery Time           Mode  WAL Mode  TLI  Time    Data    WAL  Zratio  Start LSN   Stop LSN    Status  
======================================================================================================================================node01    12       S0GDJI  2023-09-04 16:43:24+08  PAGE  ARCHIVE   3/3   31s  1090MB   16MB    1.00  5/6D000110  5/6E0000F0  OK      node01    12       S0GD3F  2023-09-04 16:33:57+08  FULL  ARCHIVE   3/0   44s  2446MB   16MB    1.00  4/E9000028  4/EA000128  OK

3.8 show

pg_probackup-12 show -B backup-path[--instance=instance_name [-i backup-id]][--format=format] [--archive][--no-color] [--help]

该方法用于查看备份列表信息和归档信息。使用案例，查看备份信息：

pg_probackup-12 show -B /data/pg_backup/

BACKUP INSTANCE 'test02'
======================================================================================================================================Instance  Version  ID      Recovery Time           Mode  WAL Mode  TLI   Time    Data   WAL  Zratio  Start LSN   Stop LSN    Status  
======================================================================================================================================test02    12       S04Q23  2023-08-29 09:42:50+08  FULL  STREAM    2/0    54s  2407MB  32MB    1.00  2/92000028  2/920001A0  OK

使用案例，查看归档信息：

pg_probackup-12 show -B /data/pgsql12/backup/ --archive

ARCHIVE INSTANCE 'test01'
==================================================================================================================================TLI  Parent TLI  Switchpoint  Min Segno                 Max Segno                 N segments  Size   Zratio  N backups  Status   
==================================================================================================================================2    0           0/0          000000020000000200000072  000000020000000200000092  32          512MB  1.00    0          DEGRADED

3.9 delete

 pg_probackup-12 delete -B backup-path --instance=instance_name[-j num-threads] [--progress][--retention-redundancy=retention-redundancy][--retention-window=retention-window][--wal-depth=wal-depth][-i backup-id | --delete-expired | --merge-expired | --status=backup_status][--delete-wal][--dry-run] [--no-validate] [--no-sync][--help]

该方法用于删除备份，或删除过期的归档日志。例如：

pg_probackup-12 delete -B /pg_data/backup/ --instance node01 -i S0G7IN
# INFO: Delete: S0G7IN 2023-09-04 14:32:47+08

删除过期的备份和 WAL 日志：

pg_probackup-12 delete -B /pg_data/backup --instance node01 --delete-expired --delete-wal

3.10 restore

pg_probackup-12 restore -B backup-path --instance=instance_name[-D pgdata-path] [-i backup-id] [-j num-threads][--recovery-target-time=time|--recovery-target-xid=xid|--recovery-target-lsn=lsn [--recovery-target-inclusive=boolean]][--recovery-target-timeline=timeline][--recovery-target=immediate|latest][--recovery-target-name=target-name][--recovery-target-action=pause|promote|shutdown][--restore-command=cmdline][-R | --restore-as-replica] [--force][--primary-conninfo=primary_conninfo][-S | --primary-slot-name=slotname][--no-validate] [--skip-block-validation][-T OLDDIR=NEWDIR] [--progress][--external-mapping=OLDDIR=NEWDIR][--skip-external-dirs] [--no-sync][-X WALDIR | --waldir=WALDIR][-I | --incremental-mode=none|checksum|lsn][--db-include | --db-exclude][--remote-proto] [--remote-host][--remote-port] [--remote-path] [--remote-user][--ssh-options][--archive-host=hostname][--archive-port=port] [--archive-user=username][--help]

该方法用于将备份恢复到新的 PostgreSQL 实例，如果指定恢复目标参数，pg_probackup 会查找最近的备份并将其恢复到指定的恢复目标，如果没有提供备份 ID 也没有提供恢复目标选项， pg_probackup 使用最新的备份来执行恢复。

全量备份恢复案例：

# 停掉 PostgreSQL
pg_ctl -D /data/pgsql12/data/ -l /data/pgsql12/logs/start.log stop# 删掉数据目录
rm -rf /data/pgsql12/data# 使用备份恢复，这里用的是远程备份恢复
pg_probackup-12 restore -B /data/pg_backup --instance test02 --remote-user='postgres' --remote-host='172.16.104.7' --remote-proto=ssh --stream --remote-port=22# 使用备份恢复，这里是本地备份
pg_probackup-12 restore -B /data/pgsql12/backup/ --instance test01 -i S08V98# 恢复后启动 PostgreSQL
pg_ctl -D /data/pgsql12/data/ -l /data/pgsql12/logs/start.log start

INFO: Validating backup S04Q23
INFO: Backup S04Q23 data files are valid
INFO: Backup S04Q23 WAL segments are valid
INFO: Backup S04Q23 is valid.
INFO: Restoring the database from backup at 2023-08-29 09:42:03+08
INFO: Start restoring backup files. PGDATA size: 2437MB
INFO: Backup files are restored. Transfered bytes: 2437MB, time elapsed: 52s
INFO: Restore incremental ratio (less is better): 100% (2437MB/2437MB)
INFO: Syncing restored files to disk
INFO: Restored backup files are synced, time elapsed: 3s
INFO: Restore of backup S04Q23 completed.

3.11 catchup

pg_probackup-12 catchup  -b catchup-mode--source-pgdata=path_to_pgdata_on_remote_server--destination-pgdata=path_to_local_dir[--stream [-S slot-name] [--temp-slot | --perm-slot]][-j num-threads][-T OLDDIR=NEWDIR][--exclude-path=path_prefix][-d dbname] [-h host] [-p port] [-U username][-w --no-password] [-W --password][--remote-proto] [--remote-host][--remote-port] [--remote-path] [--remote-user][--ssh-options][--dry-run][--help]

3.12 archive-push

pg_probackup-12 archive-push -B backup-path --instance=instance_name--wal-file-name=wal-file-name[--wal-file-path=wal-file-path][-j num-threads] [--batch-size=batch_size][--archive-timeout=timeout][--no-ready-rename] [--no-sync][--overwrite] [--compress][--compress-algorithm=compress-algorithm][--compress-level=compress-level][--remote-proto] [--remote-host][--remote-port] [--remote-path] [--remote-user][--ssh-options][--help]

该方法用于备份归档文件，可将该命令添加到 PostgreSQL 参数文件中。

# 自动归档的时间，单位为秒，可酌情设置，建议 1 分钟
max_wal_senders = 60
# 开启归档
archive_mode = 'on'
# WAL 格式，归档必须是 replica 及更高级别
wal_level = 'replica'
# 配置归档命令
archive_command = 'pg_probackup-12 archive-push -B /data/pgsql12/backup --instance test01 --wal-file-path=%p --wal-file-name=%f'

3.13 archive-get

pg_probackup-12 archive-get -B backup-path --instance=instance_name--wal-file-path=wal-file-path--wal-file-name=wal-file-name[-j num-threads] [--batch-size=batch_size][--no-validate-wal][--remote-proto] [--remote-host][--remote-port] [--remote-path] [--remote-user][--ssh-options][--help]

该方法用于将 WAL 文件从备份目录拷贝到 PostgreSQL WAL 日志目录，用于 PITR。该命令由 pg_probackup 自动设置 Restore_command 用户可不用关心如何设置。

3.14 checkdb

pg_probackup-12 checkdb [-B backup-path] [--instance=instance_name][-D pgdata-path] [--progress] [-j num-threads][--amcheck] [--skip-block-validation][--heapallindexed] [--checkunique][--help]

通过检测物理和逻辑损坏来验证 PostgreSQL 数据库集群的正确性。

3.15 validate

pg_probackup-12 validate -B backup-path [--instance=instance_name][-i backup-id] [--progress] [-j num-threads][--recovery-target-time=time|--recovery-target-xid=xid|--recovery-target-lsn=lsn [--recovery-target-inclusive=boolean]][--recovery-target-timeline=timeline][--recovery-target-name=target-name][--skip-block-validation][--help]

该方法用于验证备份的正确性。使用案例，验证 test02 实例编号为 S04Q23 的备份。

pg_probackup-12 validate -B /data/pg_backup -i S04Q23 --instance test02

INFO: Validating backup S04Q23
INFO: Backup S04Q23 data files are valid
INFO: Backup S04Q23 WAL segments are valid
INFO: Backup S04Q23 is valid.
INFO: Validate of backup S04Q23 completed.

3.16 merge

pg_probackup-12 merge -B backup-path --instance=instance_name-i backup-id [--progress] [-j num-threads][--no-validate] [--no-sync][--help]

该方法用于合并增量备份，合并备份文件。