A backup is a consistent copy of a database cluster that can be restored to recover from a hardware failure, to perform Point-In-Time Recovery, or to bring up a new standby.

Full Backup: copies the entire contents of the database cluster to the backup. The first backup of the database cluster is always a Full Backup. is always able to restore a full backup directly. The full backup does not depend on any files outside of the full backup for consistency.

Differential Backup: copies only those database cluster files that have changed since the last full backup. restores a differential backup by copying all of the files in the chosen differential backup and the appropriate unchanged files from the previous full backup. The advantage of a differential backup is that it requires less disk space than a full backup, however, the differential backup and the full backup must both be valid to restore the differential backup.

Incremental Backup: copies only those database cluster files that have changed since the last backup (which can be another incremental backup, a differential backup, or a full backup). As an incremental backup only includes those files changed since the prior backup, they are generally much smaller than full or differential backups. As with the differential backup, the incremental backup depends on other backups to be valid to restore the incremental backup. Since the incremental backup includes only those files since the last backup, all prior incremental backups back to the prior differential, the prior differential backup, and the prior full backup must all be valid to perform a restore of the incremental backup. If no differential backup exists then all prior incremental backups back to the prior full backup, which must exist, and the full backup itself must be valid to restore the incremental backup.

A restore is the act of copying a backup to a system where it will be started as a live database cluster. A restore requires the backup files and one or more WAL segments in order to work correctly.

WAL is the mechanism that uses to ensure that no committed changes are lost. Transactions are written sequentially to the WAL and a transaction is considered to be committed when those writes are flushed to disk. Afterwards, a background process writes the changes into the main database cluster files (also known as the heap). In the event of a crash, the WAL is replayed to make the database consistent.

WAL is conceptually infinite but in practice is broken up into individual 16MB files called segments. WAL segments follow the naming convention 0000000100000A1E000000FE where the first 8 hexadecimal digits represent the timeline and the next 16 digits are the logical sequence number (LSN).

Encryption is the process of converting data into a format that is unrecognizable unless the appropriate password (also referred to as passphrase) is provided.

will encrypt the repository based on a user-provided password, thereby preventing unauthorized access to data stored within the repository.

Upgrading from v1 to v2 is fairly straight-forward. The repository format has not changed and all non-deprecated options from v1 are accepted, so for most installations it is simply a matter of installing the new version.

However, there are a few caveats:

Many option names have changed to improve consistency although the old names from v1 are still accepted. In general, db-* options have been renamed to pg-* and backup-*/retention-* options have been renamed to repo-* when appropriate.

and repository options must be indexed when using the new names introduced in v2, e.g. pg1-host, pg1-path, repo1-path, repo1-type, etc.

Creating the demo cluster is optional but is strongly recommended, especially for new users, since the example commands in the user guide reference the demo cluster; the examples assume the demo cluster is running on the default port (i.e. 5432). The cluster will not be started until a later section because there is still some configuration to do.

By default {[user-guide-os]} includes the day of the week in the log filename. This makes the user guide a bit more complicated so the log_filename is set to a constant.

Backing up a running cluster requires WAL archiving to be enabled. Note that at least one WAL segment will be created during the backup process even if no explicit writes are made to the cluster.

%p is how specifies the location of the WAL segment to be archived. Setting wal_level to at least {[wal-level]} and increasing max_wal_senders is a good idea even if there are currently no replicas as this will allow them to be added later without restarting the primary cluster.

The cluster must be restarted after making these changes and before performing a backup.

When archiving a WAL segment is expected to take more than 60 seconds (the default) to reach the repository, then the archive-timeout option should be increased. Note that this option is not the same as the archive_timeout option which is used to force a WAL segment switch; useful for databases where there are long periods of inactivity. For more information on the archive_timeout option, see Write Ahead Log.

The archive-push command can be configured with its own options. For example, a lower compression level may be set to speed archiving without affecting the compression used for backups.

This configuration technique can be used for any command and can even target a specific stanza, e.g. demo:archive-push.

expires backups based on retention options.

More information about retention can be found in the Retention section.

The repository will be configured with a cipher type and key to demonstrate encryption. Encryption is always performed client-side even if the repository type (e.g. S3 or other object store) supports encryption.

It is important to use a long, random passphrase for the cipher key. A good way to generate one is to run: openssl rand -base64 48.

Once the repository has been configured and the stanza created and checked, the repository encryption settings cannot be changed.

The stanza-create command must be run to initialize the stanza. It is recommended that the check command be run after stanza-create to ensure archiving and backups are properly configured.

By default will wait for the next regularly scheduled checkpoint before starting a backup. Depending on the checkpoint_timeout and checkpoint_segments settings in it may be quite some time before a checkpoint completes and the backup can begin. Generally, it is best to set start-fast=y so that the backup starts immediately. This forces a checkpoint, but since backups are usually run once a day an additional checkpoint should not have a noticeable impact on performance. However, on very busy clusters it may be best to pass {[dash]}-start-fast on the command-line as needed.

To perform a backup of the cluster run with the backup command.

By default will attempt to perform an incremental backup. However, an incremental backup must be based on a full backup and since no full backup existed ran a full backup instead.

The type option can be used to specify a full or differential backup.

This time there was no warning because a full backup already existed. While incremental backups can be based on a full or differential backup, differential backups must be based on a full backup. A full backup can be performed by running the backup command with {[dash]}-type=full.

During an online backup waits for WAL segments that are required for backup consistency to be archived. This wait time is governed by the archive-timeout option which defaults to 60 seconds. If archiving an individual segment is known to take longer then this option should be increased.

Backups can be scheduled with utilities such as cron.

In the following example, two cron jobs are configured to run; full backups are scheduled for 6:30 AM every Sunday with differential backups scheduled for 6:30 AM Monday through Saturday. If this crontab is installed for the first time mid-week, then pgBackRest will run a full backup the first time the differential job is executed, followed the next day by a differential backup.

Once backups are scheduled it's important to configure retention so backups are expired on a regular schedule, see Retention.

Use the info command to get information about backups.

Backups can protect you from a number of disaster scenarios, the most common of which are hardware failure and data corruption. The easiest way to simulate data corruption is to remove an important cluster file.

Starting the cluster without this important file will result in an error.

To restore a backup of the cluster run with the restore command. The cluster needs to be stopped (in this case it is already stopped) and all files must be removed from the data directory.

This time the cluster started successfully since the restore replaced the missing pg_control file.

More information about the restore command can be found in the Restore section.

The COPY command allows info to be loaded into a table. The following example wraps that logic in a function that can be used to perform real-time queries.

Now the monitor.pgbackrest_info() function can be used to determine the last successful backup time and archived WAL for a stanza.

jq is a command-line utility that can easily extract data from JSON.

Now jq can be used to query the last successful backup time for a stanza.

Or the last archived WAL.

A new host named repository is created to store the cluster backups.

can use TLS with client certificates to enable communication between the hosts. It is also possible to use SSH, see Setup SSH.

expects client/server certificates to be generated in the same way as . See Secure TCP/IP Connections with TLS for detailed instructions on generating certificates.

The repository host must be configured with the {[host-pg1]} host/user and database path. The primary will be configured as pg1 to allow a standby to be added later.

The database host must be configured with the repository host/user. The default for the repo1-host-user option is pgbackrest. If the postgres user does restores on the repository host it is best not to also allow the postgres user to perform backups. However, the postgres user can read the repository directly if it is in the same group as the pgbackrest user.

configuration may be found in the Configure Archiving section.

Commands are run the same as on a single host configuration except that some commands such as backup and expire are run from the repository host instead of the database host.

Configure Azure-compatible object store if required.

Configure GCS-compatible object store if required.

Configure S3-compatible object store if required.

The TLS server must be configured and started on each host.

Create the stanza in the new repository.

Check that the configuration is correct on both the database and repository hosts. More information about the check command can be found in Check the Configuration.

To perform a backup of the cluster run with the backup command on the repository host.

Since a new repository was created on the repository host the warning about the incremental backup changing to a full backup was emitted.

To perform a restore of the cluster run with the restore command on the database host.

A new host named {[host-pg2]} is created to run the standby.

A hot standby performs replication using the WAL archive and allows read-only queries.

configuration is very similar to {[host-pg1]} except that the standby recovery type will be used to keep the cluster in recovery mode when the end of the WAL stream has been reached.

The demo cluster must be created (even though it will be overwritten on restore) in order to create the configuration files.

Create the path where will be restored.

Now the standby can be created with the restore command.

The hot_standby setting must be enabled before starting to allow read-only connections on {[host-pg2]}. Otherwise, connection attempts will be refused. The rest of the configuration is in case the standby is promoted to a primary.

The log gives valuable information about the recovery. Note especially that the cluster has entered standby mode and is ready to accept read-only connections.

An easy way to test that replication is properly configured is to create a table on {[host-pg1]}.

And then query the same table on {[host-pg2]}.

So, what went wrong? Since is pulling WAL segments from the archive to perform replication, changes won't be seen on the standby until the WAL segment that contains those changes is pushed from {[host-pg1]}.

This can be done manually by calling {[pg-switch-wal]}() which pushes the current WAL segment to the archive (a new WAL segment is created to contain further changes).

Now after a short delay the table will appear on {[host-pg2]}.

Check the standby configuration for access to the repository.

Instead of relying solely on the WAL archive, streaming replication makes a direct connection to the primary and applies changes as soon as they are made on the primary. This results in much less lag between the primary and standby.

Streaming replication requires a user with the replication privilege.

The pg_hba.conf file must be updated to allow the standby to connect as the replication user. Be sure to replace the IP address below with the actual IP address of your {[host-pg2]}. A reload will be required after modifying the pg_hba.conf file.

The standby needs to know how to contact the primary so the primary_conninfo setting will be configured in .

It is possible to configure a password in the primary_conninfo setting but using a .pgpass file is more flexible and secure.

Now the standby can be created with the restore command.

By default {[user-guide-os]} stores the postgresql.conf file in the data directory. That means the change made to postgresql.conf was overwritten by the last restore and the hot_standby setting must be enabled again. Other solutions to this problem are to store the postgresql.conf file elsewhere or to enable the hot_standby setting on the {[host-pg1]} host where it will be ignored.

The log will confirm that streaming replication has started.

Now when a table is created on {[host-pg1]} it will appear on {[host-pg2]} quickly and without the need to call {[pg-switch-wal]}().

The asynchronous archive-push command offloads WAL archiving to a separate process (or processes) to improve throughput. It works by looking ahead to see which WAL segments are ready to be archived beyond the request that is currently making via the archive_command. WAL segments are transferred to the archive directly from the pg_xlog/pg_wal directory and success is only returned by the archive_command when the WAL segment has been safely stored in the archive.

The spool path holds the current status of WAL archiving. Status files written into the spool directory are typically zero length and should consume a minimal amount of space (a few MB at most) and very little IO. All the information in this directory can be recreated so it is not necessary to preserve the spool directory if the cluster is moved to new hardware.

The [stanza]-archive-push-async.log file can be used to monitor the activity of the asynchronous process. A good way to test this is to quickly push a number of WAL segments.

Now the log file will contain parallel, asynchronous activity.

The asynchronous archive-get command maintains a local queue of WAL to improve throughput. If a WAL segment is not found in the queue it is fetched from the repository along with enough consecutive WAL to fill the queue. The maximum size of the queue is defined by archive-get-queue-max. Whenever the queue is less than half full more WAL will be fetched to fill it.

Asynchronous operation is most useful in environments that generate a lot of WAL or have a high latency connection to the repository storage (i.e., S3 or other object stores). In the case of a high latency connection it may be a good idea to increase process-max.

The [stanza]-archive-get-async.log file can be used to monitor the activity of the asynchronous process.