A backup is a consistent copy of a database cluster that can be restored to recover from a hardware failure, to perform Point-In-Time Recovery, or to bring up a new standby.

Full Backup: copies the entire contents of the database cluster to the backup server. The first backup of the database cluster is always a Full Backup. is always able to restore a full backup directly. The full backup does not depend on any files outside of the full backup for consistency.

Differential Backup: copies only those database cluster files that have changed since the last full backup. restores a differential backup by copying all of the files in the chosen differential backup and the appropriate unchanged files from the previous full backup. The advantage of a differential backup is that it requires less disk space than a full backup, however, the differential backup and the full backup must both be valid to restore the differential backup.

Incremental Backup: copies only those database cluster files that have changed since the last backup (which can be another incremental backup, a differential backup, or a full backup). As an incremental backup only includes those files changed since the prior backup, they are generally much smaller than full or differential backups. As with the differential backup, the incremental backup depends on other backups to be valid to restore the incremental backup. Since the incremental backup includes only those files since the last backup, all prior incremental backups back to the prior differential, the prior differential backup, and the prior full backup must all be valid to perform a restore of the incremental backup. If no differential backup exists then all prior incremental backups back to the prior full backup, which must exist, and the full backup itself must be valid to restore the incremental backup.

A restore is the act of copying a backup to a system where it will be started as a live database cluster. A restore requires the backup files and one or more WAL segments in order to work correctly.

WAL is the mechanism that uses to ensure that no committed changes are lost. Transactions are written sequentially to the WAL and a transaction is considered to be committed when those writes are flushed to disk. Afterwards, a background process writes the changes into the main database cluster files (also known as the heap). In the event of a crash, the WAL is replayed to make the database consistent.

WAL is conceptually infinite but in practice is broken up into individual 16MB files called segments. WAL segments follow the naming convention 0000000100000A1E000000FE where the first 8 hexadecimal digits represent the timeline and the next 16 digits are the logical sequence number (LSN).

A valid backup will always include at least one WAL segment even if explicit writes were made to the database between backups.

Creating the demo cluster is optional but commands in the user guide may not work in other environments without significant modification. The cluster won't be started immediately because there is still some configuration to do.

By default will only accept local connections. The examples in this guide will require connections from other servers so listen_addresses is configured to listen on all interfaces. This may not be appropriate for secure installations.

For demonstration purposes the log_line_prefix setting will be minimally configured. This keeps the log output as brief as possible to better illustrate important information.

By default CentOS/RHEL includes the day of the week in the log filename. This makes automating the user guide a bit more complicated so the log_filename is set to a constant.

Backing up a running cluster requires WAL archiving to be enabled.

The wal_level setting must be set to archive at a minimum but hot_standby and logical also work fine for backups. Setting wal_level to hot_standy and increasing max_wal_senders is a good idea even if you do not currently run a hot standby as this will allow them to be added later without restarting the master cluster.

The cluster must be restarted after making these changes and before performing a backup.

To perform a backup of the cluster run with the backup command.

By default will attempt to perform an incremental backup. However, an incremental backup must be based on a full backup and since no full backup existed ran a full backup instead.

The type option can be used to specify a full or differential backup.

This time there was no warning because a full backup already existed. While incremental backups can be based on a full or differential backup, differential backups must be based on a full backup. A full backup can be performed by running the backup command with {[dash]}-type=full.

Use the info command to get information about backups.

The oldest and newest backups are shown in the info output. The oldest backup will always be a full backup (indicated by an F at the end of the label) but the newest backup can be full, differential (ends with D), or incremental (ends with I).

More information about the backup command can be found in the Backup section.

Backups can protect you from a number of disaster scenarios, the most common of which are hardware failure and data corruption. The easiest way to simulate data corruption is to remove an important cluster file.

Starting the cluster without this important file will result in an error.

To restore a backup of the cluster run with the restore command. The cluster needs to be stopped (in this case it is already stopped) and all files must be removed from the data directory.

This time the cluster started successfully since the restore replaced the missing pg_control file.

More information about the restore command can be found in the Restore section.

By default will wait for the next regularly scheduled checkpoint before starting a backup. Depending on the checkpoint_timeout and checkpoint_segments settings in it may be quite some time before a checkpoint completes and the backup can begin.

By setting start-fast on the command-line or in {[backrest-config-demo]} an immediate checkpoint is requested and the backup will start more quickly. This is convenient for testing and for ad-hoc backups. For instance, if a backup is being taken at the beginning of a release window it makes no sense to wait for a checkpoint. Since regularly scheduled backups generally only happen once per day it is unlikely that enabling the start-fast in {[backrest-config-demo]} will negatively affect performance, however for high-volume transactional systems you may want to pass {[dash]}-start-fast on the command-line instead.

Sometimes will exit unexpectedly and the backup in progress on the cluster will not be properly stopped. exits as quickly as possible when an error occurs so that the cause can be reported accurately and is not masked by another problem that might happen during a more extensive cleanup.

Here an error in intentionally caused by removing repository permissions.

Even when the permissions are fixed will still be unable to perform a backup because the cluster is stuck in backup mode.

Enabling the stop-auto option allows to stop the current backup if it detects that no other backup process is running.

Now will stop the old backup and start a new one so the process completes successfully.

Although useful this feature may not be appropriate when another third-party backup solution is being used to take online backups as will not recognize that the other software is running and may terminate a backup started by that software. However, it would be unusual to run more than one third-party backup solution at the same time so this is not likely to be a problem.

Note that pg_dump and pg_base_backup do not take online backups so are not affected. It is safe to run them in conjunction with .

Set full-retention to the number of full backups required. New backups must be completed before expiration will occur — that means if retention-full=2 then there will be three full backups stored before the oldest one is expired.

Backup retention-full=2 but currently there is only one full backup so the next full backup to run will not expire any full backups.

Archive is expired because WAL segments were generated before the oldest backup. These are not useful for recovery — only WAL segments generated after a backup can be used to recover that backup.

The {[backup-full-first]} full backup is expired and archive retention is based on the {[backup-full-second]} which is now the oldest full backup.

Set retention-diff to the number of differential backups required. Differentials only rely on the prior full backup so it is possible to create a rolling set of differentials for the last day or more. This allows quick restores to recent points-in-time but reduces overall space consumption.

Backup retention-diff=2 so two differentials will need to be performed before one is expired. An incremental backup is added to demonstrate incremental expiration. Incremental backups cannot be expired independently — they are always expired with their related full or differential backup.

Now performing a differential backup will expire the previous differential and incremental backups leaving only one differential backup.

Restore a Backup in Quick Start required the database cluster directory to be cleaned before the restore could be performed. The delta allows to automatically determine which files in the database cluster directory can be preserved and which ones need to be restored from the backup. This is accomplished by calculating a SHA-1 cryptographic hash for each file in the database cluster directory. If the SHA-1 hash does not match the hash stored in the backup then that file will be restored. This operation is very efficient when combined with the thread-max option. Since the process is shut down during the restore, a larger number of threads can be used than might be desirable during a backup when the process is running.

For this example a new host named backup has been created to store the cluster backups. Follow the instructions in Installation to install and Create the Repository to create the repository. The backup host must also be configured with the database host/user and database path.

The database host must be configured with the backup host/user.

Commands are run the same as on a single host configuration except that the backup and expire command are run from the backup host and all other commands are run from the database host.

To perform a backup of the cluster run with the backup command on the backup host.

Since a new repository was created on the backup host the warning about the incremental backup changing to a full backup was emitted.

To perform a restore of the cluster run with the restore command on the database host.

A new backup must be performed to due to the timeline switch.

A hot standby performs replication using the WAL archive and allows read-only queries.

A new host named db-standby will be created to run the standby. Follow the instructions in Installation to install , Setup Demo Cluster to setup the demo cluster, and Create the Repository to create the repository on the db-standby host.

configuration is very similar to db-master except that the standby_mode setting will be enabled to keep the cluster in recovery mode when the end of the WAL stream has been reached.

Now the standby can be created with the restore command.

Note that the standby_mode setting has been written into the recovery.conf file. Configuring recovery settings in means that the recovery.conf file does not need to be stored elsewhere since it will be properly recreated with each restore. The --type=preserve option can be used with the restore to leave the existing recovery.conf file in place if that behavior is preferred.

The hot_standby setting must be enabled before starting to allow read-only connections on db-standby. Otherwise, connection attempts will be refused.

The log gives valuable information about the recovery. Note especially that the cluster has entered standby mode and is ready to accept read-only connections.

An easy way to test that replication is properly configured is to create a table on db-master.

And then query the same table on db-standby.

So, what went wrong? Since is pulling WAL segments from the archive to perform replication, changes won't be seen on the standby until the WAL segment that contains those changes is pushed from db-master.

This can be done manually by calling pg_switch_xlog() which pushes the current WAL segment to the archive (a new WAL segment is created to contain further changes).

Now after a short delay the table will appear on db-standby.

Instead of relying solely on the WAL archive, streaming replication makes a direct connection to the master and applies changes as soon as they are made on the master. This results in much less lag between the master and standby.

Streaming replication requires a user with the replication privilege.

The pg_hba.conf file must be updated to allow the standby to connection as the replication user. Be sure to replace the IP address below with the actual IP address of your db-master. A reload will be required after modifying the pg_hba.conf file.

The standby needs to know how to contact the master so the primary_conninfo setting will be configured in .

It is possible to configure a password in the primary_conninfo setting but using a .pgpass file is more flexible and secure.

Now the standby can be created with the restore command.

By default CentOS/RHEL stores the postgresql.conf file in the data directory. That means the change made to postgresql.conf was overwritten by the last restore and the hot_standby setting must be enabled again. Other solutions to this problem are to store the postgresql.conf file elsewhere or to enable the hot_standby setting on the db-master host where it will be ignored.

The log will confirm that streaming replication has started.

Now when a table is created on db-master it will appear on db-standby quickly and without the need to call pg_switch_xlog().