1
0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2025-06-14 23:44:58 +02:00
Commit Graph

348 Commits

Author SHA1 Message Date
7cf955425e The C library is now required.
This eliminates conditional loading and eases development of new library features.
2017-11-26 17:45:00 -05:00
c164fc5fb9 Add memGrowRaw() to memory context module. 2017-11-26 12:22:50 -05:00
ba6b49ecb7 v1.26: Repository Encryption
Bug Fixes:

* Fixed an issue that could cause copying large manifests to fail during restore. (Reported by Craig A. James.)
* Fixed incorrect WAL offset for 32-bit architectures. (Fixed by Javier Wilson.)
* Fixed an issue retrieving WAL for old database versions. After a stanza-upgrade it should still be possible to restore backups from the previous version and perform recovery with archive-get. However, archive-get only checked the most recent db version/id and failed. Also clean up some issues when the same db version/id appears multiple times in the history. (Fixed by Cynthia Shang. Reported by Clinton Adams.)
* Fixed an issue with invalid backup groups being set correctly on restore. If the backup cannot map a group to a name it stores the group in the manifest as false then uses either the owner of $PGDATA to set the group during restore or failing that the group of the current user. This logic was not working correctly because the selected group was overwriting the user on restore leaving the group undefined and the user incorrectly set to the group. (Reported by Jeff McCormick.)
* Fixed an issue passing parameters to remotes. When more than one db was specified the path, port, and socket path would for db1 were passed no matter which db was actually being addressed. (Reported by Uspen.)

Features:

* Repository encryption support. (Contributed by Cynthia Shang, David Steele.)
2017-11-21 18:31:54 -05:00
ac54aac448 Fixed an issue passing parameters to remotes.
When more than one db was specified the path, port, and socket path would for db1 were passed no matter which db was actually being addressed.

Reported by Uspen.
2017-11-21 13:17:44 -05:00
c77fc1fa61 Refactor protocol param generation into a new function.
This allows the code to be tested more precisely and doesn't require executing a remote process.
2017-11-21 12:57:00 -05:00
062e714307 Disable gzip filter when --compress-level-network=0.
The filter was used with compress level set to 0 which added overhead without any benefit.
2017-11-21 09:31:15 -05:00
1da63aff07 Fixed an issue with invalid backup groups being set correctly on restore.
If the backup cannot map a group to a name it stores the group in the manifest as false then uses either the owner of $PGDATA to set the group during restore or failing that the group of the current user. This logic was not working correctly because the selected group was overwriting the user on restore leaving the group undefined and the user incorrectly set to the group. (Reported by Jeff McCormick.)
2017-11-20 12:46:35 -05:00
6ef2167e97 Fixed an issue retrieving WAL for old database versions.
After a stanza-upgrade it should still be possible to restore backups from the previous version and perform recovery with archive-get. However, archive-get only checked the most recent db version/id and failed.

Also clean up some issues when the same db version/id appears multiple times in the history.

Fixed by Cynthia Shang.
Reported by Clinton Adams.
2017-11-16 17:18:51 -05:00
550d4fb9c5 Fix incorrect WAL offset for 32-bit architectures.
Fixed by Javier Wilson.
2017-11-16 16:53:49 -05:00
b8746f368d Inflate performance improvement for gzip filter and full unit test coverage. 2017-11-14 15:12:31 -05:00
04e55fe71b Fixed an issue that could cause copying large manifests to fail during restore.
Reported by Craig A. James.
2017-11-14 14:48:44 -05:00
b03c26968a Repository encryption support.
Contributed by Cynthia Shang.
2017-11-06 12:51:12 -05:00
f07d2a3d0d Add eof to S3 file driver (required for encryption support). 2017-11-06 12:40:54 -05:00
8d6a08a32b Library code for repository encryption support. 2017-11-03 13:57:58 -04:00
ac542788da Convert configuration definitions from auto-generated functions to auto-generated data structures. 2017-11-02 08:14:13 -04:00
55f68ef013 Remove configurable option hints.
db-path was the only option with a hint so the feature seemed wasteful. All missing stanza options now output the same hint without needing configuration.
2017-10-30 10:58:04 -04:00
1ef27ec8c2 Add list type for options.
The hash type was being used for lists with an additional flag (`value-hash`) to indicate that it was not really a hash.
2017-10-30 10:50:35 -04:00
85b35f30a4 v1.25: S3 Performance Improvements
Bug Fixes:

* Fix custom settings for compress-level option being ignored. (Reported by Jens Wilke.)
* Remove error when overlapping timelines are detected. Overlapping timelines are valid in many Point-in-Time-Recovery (PITR) scenarios. (Reported by blogh.)
* Fix instances where database-id was not rendered as an integer in JSON info output. (Fixed by Cynthia Shang. Reported by Jason O'Donnell.)

Features:

* Improve performance of list requests on S3. Any beginning literal portion of a filter expression is used to generate a search prefix which often helps keep the request small enough to avoid rate limiting. (Suggested by Mihail Shvein.)
2017-10-24 17:36:21 -04:00
8674a4f7ae Allow functions with sensitive options to be logged at debug level with redactions.
Previously, functions with sensitive options had to be logged at trace level to avoid exposing them. Trace level logging may still expose secrets so use with caution.
2017-10-24 12:35:36 -04:00
d989cf8ac2 Replace dynamically built class hierarchies in I/O layer with fixed parent() calls. 2017-10-22 19:07:17 -04:00
d82fe88031 Fix custom settings for compress-level option being ignored.
Reported by Jens Wilke.
2017-10-22 18:05:46 -04:00
e1cb9ae86a Improve labeling for errors in helper processes. 2017-10-21 14:40:14 -04:00
44c7c0475a Improve protocol error handling.
In particular, "stop" errors are no longer reported as "unexpected".
2017-10-21 14:11:33 -04:00
1f120f3fce Improve performance of list requests on S3.
Any beginning literal portion of a filter expression is used to generate a search prefix which often helps keep the request small enough to avoid rate limiting.

Suggested by Mihail Shvein.
2017-10-20 14:10:16 -04:00
2d56de00fc Fix instances where database-id was not rendered as an integer in JSON info output.
Fixed by Cynthia Shang.
Reported by Jason O'Donnell.
2017-10-18 08:42:32 -04:00
c94e52f697 Remove error when overlapping timelines are detected.
Overlapping timelines are valid in many Point-in-Time-Recovery (PITR) scenarios.

Reported by blogh.
2017-10-16 14:34:44 -04:00
904b83747c Perl error handler recognizes errors thrown from the C library. 2017-10-16 10:47:31 -04:00
f1e739b4aa Add C error handler.
Basic try ... catch ... finally pattern to simplify error handling in C.
2017-10-16 10:09:56 -04:00
8efcc38304 Improvements in C codebase:
* Update C naming conventions.
* Use int datatype wherever possible.
* Better separation of C source from Perl interface.
2017-09-30 10:44:03 -04:00
d0bf946857 v1.24: New Backup Exclusions
Bug Fixes:

* Fixed an issue where warnings were being emitted in place of lower priority log messages during backup from standby initialization. (Reported by Uspen.)
* Fixed an issue where some db-* options (e.g. db-port) were not being passed to remotes. (Reported by Uspen.)

Features:

* Exclude contents of pg_snapshots, pg_serial, pg_notify, and pg_dynshmem from backup since they are rebuilt on startup.
* Exclude pg_internal.init files from backup since they are rebuilt on startup.
2017-09-28 19:06:41 -04:00
4d6c56b4d7 Open log file after async process is completely separated from the main process.
This prevents the main process from also logging to the file.

Suggested by Jens Wilke.
2017-09-20 08:08:36 -04:00
266c9ddcc1 Fixed an issue where some db-* options (e.g. db-port) were not being passed to remotes. 2017-09-19 10:14:18 -04:00
c857015a38 Fixed an issue where warnings were being emitted in place of lower priority log messages during backup from standby initialization. 2017-09-19 10:03:53 -04:00
6343fdd584 Additional backup exclusions.
* Exclude contents of pg_snapshots, pg_serial, pg_notify, and pg_dynshmem from backup since they are rebuilt on startup.
* Exclude pg_internal.init files from backup since they are rebuilt on startup.
2017-09-04 08:26:57 -04:00
df1ec0aaeb v1.23: Multiple Standbys and PostgreSQL 10 Support
Bug Fixes:

* Fixed an issue that could cause compression to abort on growing files. (Reported by Jesper St John, Aleksandr Rogozin.)
* Fixed an issue with keep-alives not being sent to the remote from the local process. (Reported by William Cox.)

Features:

* Up to seven standbys can be configured for backup from standby. (Contributed by Cynthia Shang.)
* PostgreSQL 10 support.
* Allow content-length (in addition to chunked encoding) when reading XML data to improve compatibility with third-party S3 gateways. (Suggested by Victor Gdalevich.)

Refactoring:

* Configuration rules are now pulled from the C library when present.
* Increase HTTP timeout for S3.
* Add HTTP retries to harden against transient S3 network errors.
2017-09-03 17:42:00 -04:00
eea2ccc3ab Add HTTP retries to harden against transient S3 network errors. 2017-09-03 16:48:41 -04:00
80e5e8db01 Increase HTTP timeout for S3. 2017-09-01 19:16:57 -04:00
75511d2481 Allow content-length (in addition to chunked encoding) when reading XML data to improve compatibility with third-party S3 gateways.
Suggested by Victor Gdalevich.
2017-09-01 15:07:20 -04:00
fcb7c6fd1d PostgreSQL 10 support. 2017-09-01 12:29:34 -04:00
36e576b483 Up to seven standbys can be configured for backup from standby.`
Contributed by Cynthia Shang.
2017-08-31 19:15:44 -04:00
206415d4c7 Fixed an issue that could cause compression to abort on growing files.
Reported by Jesper St John, Aleksandr Rogozin.
2017-08-30 16:34:05 -04:00
81ffd09445 Fixed an issue with keep-alives not being sent to the remote from the local process.
Reported by William Cox.
2017-08-30 08:41:46 -04:00
1e0ed07455 Configuration rules are now pulled from the C library when present. 2017-08-25 16:47:47 -04:00
84caca0f34 v1.22: Fixed S3 Retry
Bug Fixes:

* Fixed authentication issue in S3 retry.
2017-08-09 11:50:02 -04:00
61c38f5808 Fixed authentication issue in S3 retry. 2017-08-09 11:27:09 -04:00
cbc026418c v1.21: Improved Info Output and SSH Port Option
Bug Fixes:

* The archive_status directory is now recreated on restore to support PostgreSQL 8.3 which does not recreate it automatically like more recent versions do. (Reported by Stephen Frost.)
* Fixed an issue that could cause the empty archive directory for an old PostgreSQL version to be left behind after a stanza-upgrade. (Fixed by Cynthia Shang.)

Features:

* Modified the info command (both text and JSON output) to display the archive ID and minimum/maximum WAL currently present in the archive for the current and prior, if any, database cluster version. (Contributed by Cynthia Shang.)
* Added --backup-ssh-port and --db-ssh-port options to support non-default SSH ports. (Contributed by Cynthia Shang.)

Refactoring:

* Retry when S3 returns an internal error (500).
* Add bIgnoreMissing parameter to Local->manifest().
2017-08-08 18:09:07 -04:00
038d47bcc0 Retry when S3 returns an internal error (500). 2017-08-08 17:15:01 -04:00
f9bdc092ba Fixed an issue that could cause the empty archive directory for an old PostgreSQL version to be left behind after a stanza-upgrade.
Fixed by Cynthia Shang.
2017-08-04 16:03:55 -04:00
ce2b18c5b8 Added --backup-ssh-port and --db-ssh-port options to support non-default SSH ports.
Contributed by Cynthia Shang.
2017-08-01 10:34:53 -04:00
df7d633683 Improvements to command/command-line help:
* Move repository options into a separate section in command/command-line help. (Suggested by Stephen Frost.)
* Fix description of --online based on the command context.
* Remove vestigial repository options from backup command.
2017-07-26 09:33:10 -04:00