This allows specific options in pgbackrest.conf to be ignored (and set to default) which reduces the need to write new configuration files for specific needs.
Note that boolean, non-command-line options are already negatable.
When a backup host is present, backups should only be allowed on the backup host and restores should only be allowed on the database host unless an alternate configuration is created that ignores the remote host.
Reported by Lardière Sébastien.
Required to test restores on the backup server, a fairly common scenario.
Improve the restore function to accept optional parameters rather than a long list of parameters. In passing, clean up extraneous use of strType and strComment variables.
Bug Fixes:
* Fixed an issue that could cause copying large manifests to fail during restore. (Reported by Craig A. James.)
* Fixed incorrect WAL offset for 32-bit architectures. (Fixed by Javier Wilson.)
* Fixed an issue retrieving WAL for old database versions. After a stanza-upgrade it should still be possible to restore backups from the previous version and perform recovery with archive-get. However, archive-get only checked the most recent db version/id and failed. Also clean up some issues when the same db version/id appears multiple times in the history. (Fixed by Cynthia Shang. Reported by Clinton Adams.)
* Fixed an issue with invalid backup groups being set correctly on restore. If the backup cannot map a group to a name it stores the group in the manifest as false then uses either the owner of $PGDATA to set the group during restore or failing that the group of the current user. This logic was not working correctly because the selected group was overwriting the user on restore leaving the group undefined and the user incorrectly set to the group. (Reported by Jeff McCormick.)
* Fixed an issue passing parameters to remotes. When more than one db was specified the path, port, and socket path would for db1 were passed no matter which db was actually being addressed. (Reported by Uspen.)
Features:
* Repository encryption support. (Contributed by Cynthia Shang, David Steele.)
When more than one db was specified the path, port, and socket path would for db1 were passed no matter which db was actually being addressed.
Reported by Uspen.
If the backup cannot map a group to a name it stores the group in the manifest as false then uses either the owner of $PGDATA to set the group during restore or failing that the group of the current user. This logic was not working correctly because the selected group was overwriting the user on restore leaving the group undefined and the user incorrectly set to the group. (Reported by Jeff McCormick.)
Different encoded strings could be generated based on compiler optimizations. Even though decoding was still successful the encoded strings did not match the standard.
The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
Running coverage testing on multiple distros takes time but doesn't add significant value. Also ensure that the distro designated to run coverage tests is one of the default test distros.
After a stanza-upgrade it should still be possible to restore backups from the previous version and perform recovery with archive-get. However, archive-get only checked the most recent db version/id and failed.
Also clean up some issues when the same db version/id appears multiple times in the history.
Fixed by Cynthia Shang.
Reported by Clinton Adams.
db-path was the only option with a hint so the feature seemed wasteful. All missing stanza options now output the same hint without needing configuration.
Bug Fixes:
* Fix custom settings for compress-level option being ignored. (Reported by Jens Wilke.)
* Remove error when overlapping timelines are detected. Overlapping timelines are valid in many Point-in-Time-Recovery (PITR) scenarios. (Reported by blogh.)
* Fix instances where database-id was not rendered as an integer in JSON info output. (Fixed by Cynthia Shang. Reported by Jason O'Donnell.)
Features:
* Improve performance of list requests on S3. Any beginning literal portion of a filter expression is used to generate a search prefix which often helps keep the request small enough to avoid rate limiting. (Suggested by Mihail Shvein.)
Previously, functions with sensitive options had to be logged at trace level to avoid exposing them. Trace level logging may still expose secrets so use with caution.
Any beginning literal portion of a filter expression is used to generate a search prefix which often helps keep the request small enough to avoid rate limiting.
Suggested by Mihail Shvein.
Bug Fixes:
* Fixed an issue where warnings were being emitted in place of lower priority log messages during backup from standby initialization. (Reported by Uspen.)
* Fixed an issue where some db-* options (e.g. db-port) were not being passed to remotes. (Reported by Uspen.)
Features:
* Exclude contents of pg_snapshots, pg_serial, pg_notify, and pg_dynshmem from backup since they are rebuilt on startup.
* Exclude pg_internal.init files from backup since they are rebuilt on startup.
* Exclude contents of pg_snapshots, pg_serial, pg_notify, and pg_dynshmem from backup since they are rebuilt on startup.
* Exclude pg_internal.init files from backup since they are rebuilt on startup.
Bug Fixes:
* Fixed an issue that could cause compression to abort on growing files. (Reported by Jesper St John, Aleksandr Rogozin.)
* Fixed an issue with keep-alives not being sent to the remote from the local process. (Reported by William Cox.)
Features:
* Up to seven standbys can be configured for backup from standby. (Contributed by Cynthia Shang.)
* PostgreSQL 10 support.
* Allow content-length (in addition to chunked encoding) when reading XML data to improve compatibility with third-party S3 gateways. (Suggested by Victor Gdalevich.)
Refactoring:
* Configuration rules are now pulled from the C library when present.
* Increase HTTP timeout for S3.
* Add HTTP retries to harden against transient S3 network errors.
Bug Fixes:
* The archive_status directory is now recreated on restore to support PostgreSQL 8.3 which does not recreate it automatically like more recent versions do. (Reported by Stephen Frost.)
* Fixed an issue that could cause the empty archive directory for an old PostgreSQL version to be left behind after a stanza-upgrade. (Fixed by Cynthia Shang.)
Features:
* Modified the info command (both text and JSON output) to display the archive ID and minimum/maximum WAL currently present in the archive for the current and prior, if any, database cluster version. (Contributed by Cynthia Shang.)
* Added --backup-ssh-port and --db-ssh-port options to support non-default SSH ports. (Contributed by Cynthia Shang.)
Refactoring:
* Retry when S3 returns an internal error (500).
* Add bIgnoreMissing parameter to Local->manifest().
* Move repository options into a separate section in command/command-line help. (Suggested by Stephen Frost.)
* Fix description of --online based on the command context.
* Remove vestigial repository options from backup command.
The archive_status directory is now recreated on restore to support PostgreSQL 8.3 which does not recreate it automatically like more recent versions do.
Also fixed log checking after PostgreSQL shuts down to include FATAL messages and disallow immediate shutdowns which can throw FATAL errors in the log.
Reported by Stephen Frost.
Modified the info command (both text and JSON output) to display the archive ID and minimum/maximum WAL currently present in the archive for the current and prior, if any, database cluster version.
Contributed by Cynthia Shang.
Bug Fixes:
* Fixed an issue that prevented tablespaces from being backed up on PostgreSQL ≤ 8.4.
* Fixed missing flag in C library build that resulted in a mismatched binary on 32-bit systems. (Reported by Adrian Vondendriesch.)
Features:
* Add s3-repo-ca-path and s3-repo-ca-file options to accommodate systems where CAs are not automatically found by IO::Socket::SSL, i.e. RHEL7, or to load custom CAs. (Suggested by Scott Frazer.)
Refactoring:
* Harden protocol handshake to handle race conditions.
* Fixed misleading error message when a file was opened for write in a missing directory.
* Change log level of hardlink logging to detail.
* Cast size in S3 manifest to integer.
* Rename Archive modules to remove redundancy.
* Improve S3 error reporting.
* Minor optimizations to package loads and ordering for archive-get and archive-push commands.
The integration tests that were supposed to prevent this regression did not work as intended. They verified the contents of a table in the (supposedly) restored tablespace, deleted the table, and then deleted the tablespace. All of this was deemed sufficient to prove that the tablespace had been restored correctly and was valid.
However, PostgreSQL will happily recreate a tablespace on the basis of a single full-page write, at least in the affected versions. Since writes to the test table were replayed from WAL with each recovery, all the tests passed even though the tablespace was missing after the restore.
The tests have been updated to include direct comparisons against the file system and a new table that is not replayed after a restore because it is created before the backup and never modified again.
Versions ≥ 9.0 were not affected due to numerous synthetic integration tests that verify backups and restores file by file.
