1
0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2024-12-14 10:13:05 +02:00
Commit Graph

145 Commits

Author SHA1 Message Date
David Steele
526acca5bd Config parsing implemented in C. 2017-12-22 23:27:49 -05:00
David Steele
ef3ef85c33 Fixed an issue where WAL was not expired on PostgreSQL 10.
This was caused by a faulty regex that expected all PostgreSQL major versions to be X.X.

Reported by Adam Brusselback.
2017-12-18 18:51:19 -05:00
Cynthia Shang
e5511d0f82 Improve check command to verify that the backup manifest can be built.
Contributed by Cynthia Shang.
2017-12-13 11:16:27 -05:00
David Steele
99a0721631 Allow any non-boolean, non-command-line option to be negated.
This allows specific options in pgbackrest.conf to be ignored (and set to default) which reduces the need to write new configuration files for specific needs.

Note that boolean, non-command-line options are already negatable.
2017-12-05 15:23:32 -05:00
David Steele
c5fc88d340 Fixed an issue that suppressed locality errors for backup and restore.
When a backup host is present, backups should only be allowed on the backup host and restores should only be allowed on the database host unless an alternate configuration is created that ignores the remote host.

Reported by Lardière Sébastien.
2017-12-05 14:59:09 -05:00
David Steele
c77fc1fa61 Refactor protocol param generation into a new function.
This allows the code to be tested more precisely and doesn't require executing a remote process.
2017-11-21 12:57:00 -05:00
David Steele
062e714307 Disable gzip filter when --compress-level-network=0.
The filter was used with compress level set to 0 which added overhead without any benefit.
2017-11-21 09:31:15 -05:00
David Steele
1da63aff07 Fixed an issue with invalid backup groups being set correctly on restore.
If the backup cannot map a group to a name it stores the group in the manifest as false then uses either the owner of $PGDATA to set the group during restore or failing that the group of the current user. This logic was not working correctly because the selected group was overwriting the user on restore leaving the group undefined and the user incorrectly set to the group. (Reported by Jeff McCormick.)
2017-11-20 12:46:35 -05:00
David Steele
8a3425ec04 Automate generation of WAL and pg_control test files.
The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-18 20:02:54 -05:00
David Steele
6ef2167e97 Fixed an issue retrieving WAL for old database versions.
After a stanza-upgrade it should still be possible to restore backups from the previous version and perform recovery with archive-get. However, archive-get only checked the most recent db version/id and failed.

Also clean up some issues when the same db version/id appears multiple times in the history.

Fixed by Cynthia Shang.
Reported by Clinton Adams.
2017-11-16 17:18:51 -05:00
David Steele
1cb11a582d Fix flapping protocol timeout test.
It only matters that the correct error code is returned, so disable logging to prevent message ordering from failing the expect test.
2017-11-15 13:17:57 -05:00
David Steele
04e55fe71b Fixed an issue that could cause copying large manifests to fail during restore.
Reported by Craig A. James.
2017-11-14 14:48:44 -05:00
Cynthia Shang
b03c26968a Repository encryption support.
Contributed by Cynthia Shang.
2017-11-06 12:51:12 -05:00
David Steele
ac542788da Convert configuration definitions from auto-generated functions to auto-generated data structures. 2017-11-02 08:14:13 -04:00
David Steele
8674a4f7ae Allow functions with sensitive options to be logged at debug level with redactions.
Previously, functions with sensitive options had to be logged at trace level to avoid exposing them. Trace level logging may still expose secrets so use with caution.
2017-10-24 12:35:36 -04:00
David Steele
d82fe88031 Fix custom settings for compress-level option being ignored.
Reported by Jens Wilke.
2017-10-22 18:05:46 -04:00
David Steele
e1cb9ae86a Improve labeling for errors in helper processes. 2017-10-21 14:40:14 -04:00
David Steele
44c7c0475a Improve protocol error handling.
In particular, "stop" errors are no longer reported as "unexpected".
2017-10-21 14:11:33 -04:00
Cynthia Shang
2d56de00fc Fix instances where database-id was not rendered as an integer in JSON info output.
Fixed by Cynthia Shang.
Reported by Jason O'Donnell.
2017-10-18 08:42:32 -04:00
David Steele
266c9ddcc1 Fixed an issue where some db-* options (e.g. db-port) were not being passed to remotes. 2017-09-19 10:14:18 -04:00
David Steele
6343fdd584 Additional backup exclusions.
* Exclude contents of pg_snapshots, pg_serial, pg_notify, and pg_dynshmem from backup since they are rebuilt on startup.
* Exclude pg_internal.init files from backup since they are rebuilt on startup.
2017-09-04 08:26:57 -04:00
David Steele
fcb7c6fd1d PostgreSQL 10 support. 2017-09-01 12:29:34 -04:00
Cynthia Shang
36e576b483 Up to seven standbys can be configured for backup from standby.`
Contributed by Cynthia Shang.
2017-08-31 19:15:44 -04:00
David Steele
1e0ed07455 Configuration rules are now pulled from the C library when present. 2017-08-25 16:47:47 -04:00
David Steele
cbc026418c v1.21: Improved Info Output and SSH Port Option
Bug Fixes:

* The archive_status directory is now recreated on restore to support PostgreSQL 8.3 which does not recreate it automatically like more recent versions do. (Reported by Stephen Frost.)
* Fixed an issue that could cause the empty archive directory for an old PostgreSQL version to be left behind after a stanza-upgrade. (Fixed by Cynthia Shang.)

Features:

* Modified the info command (both text and JSON output) to display the archive ID and minimum/maximum WAL currently present in the archive for the current and prior, if any, database cluster version. (Contributed by Cynthia Shang.)
* Added --backup-ssh-port and --db-ssh-port options to support non-default SSH ports. (Contributed by Cynthia Shang.)

Refactoring:

* Retry when S3 returns an internal error (500).
* Add bIgnoreMissing parameter to Local->manifest().
2017-08-08 18:09:07 -04:00
Cynthia Shang
b6da509c9f Consolidate stanza-create and stanza-upgrade tests into new stanza test.
Contributed by Cynthia Shang.
2017-08-04 16:10:51 -04:00
Cynthia Shang
ce2b18c5b8 Added --backup-ssh-port and --db-ssh-port options to support non-default SSH ports.
Contributed by Cynthia Shang.
2017-08-01 10:34:53 -04:00
David Steele
df7d633683 Improvements to command/command-line help:
* Move repository options into a separate section in command/command-line help. (Suggested by Stephen Frost.)
* Fix description of --online based on the command context.
* Remove vestigial repository options from backup command.
2017-07-26 09:33:10 -04:00
David Steele
156fd4d54d Add bIgnoreMissing parameter to Local->manifest(). 2017-07-25 12:44:38 -04:00
Cynthia Shang
cda3b9821b Improved WAL data returned by info command.
Modified the info command (both text and JSON output) to display the archive ID and minimum/maximum WAL currently present in the archive for the current and prior, if any, database cluster version.

Contributed by Cynthia Shang.
2017-07-21 17:32:41 -04:00
David Steele
a10a22127d Consolidate archive-get test into new archive test. 2017-06-22 18:51:39 -04:00
David Steele
918c1c6f49 Add s3-repo-ca-path and s3-repo-ca-file options.
The options accommodate systems where CAs are not automatically found by IO::Socket::SSL, i.e. RHEL7, or to load custom CAs.

Suggested by Scott Frazer.
2017-06-22 18:22:49 -04:00
David Steele
2348693389 Reduce process redundancy in mock tests. 2017-06-21 18:47:25 -04:00
David Steele
64ec7eb490 Eliminate redundancy in real tests. 2017-06-21 16:07:13 -04:00
David Steele
2337be8636 Consolidate archive-push tests into new archive test. 2017-06-21 08:38:53 -04:00
David Steele
11ee528e0e Rename Archive modules to remove redundancy. 2017-06-21 08:02:21 -04:00
David Steele
692b1e274a Split full module into mock and real to allow better test combinations and save time in CI. 2017-06-19 18:55:00 -04:00
David Steele
f6d4457d58 Full/Synthetic test refactor.
* Combine hardlink and non/compressed in synthetic tests to reduce test time and improve coverage.
* Change log level of hardlink logging to detail.
* Cast size in S3 manifest to integer.
2017-06-15 15:32:10 -04:00
David Steele
051c961151 S3 repository support. 2017-06-12 10:52:32 -04:00
David Steele
de7fc37f88 Storage and IO layer refactor:
Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.)
Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 17:51:41 -04:00
Cynthia Shang
7e982f05f5 Fixed config warnings raised from a remote process causing errors in the master process.
Fixed by Cynthia Shang.
2017-05-16 11:28:51 -04:00
David Steele
5296747271 Move backup modules into Backup directory. 2017-05-15 16:01:00 -04:00
David Steele
2a23157bf2 Move modules in Protocol directory in subdirectories. 2017-05-15 11:12:14 -04:00
David Steele
33c87b2a1c Remove flapping protocol timeout test that will be replaced in the upcoming storage patch. 2017-05-15 10:24:46 -04:00
Cynthia Shang
7d0c50fcd6 Fixed the backup command so the backup-standby option is reset (and the backup proceeds on the master) if the standby is not configured and/or reachable.
Fixed by Cynthia Shang.
2017-05-12 16:05:44 -04:00
Cynthia Shang
5bf4f52e3d Fixed the info command so the WAL archive min/max displayed is for the current database version.
Fixed by Cynthia Shang.
2017-05-12 15:49:14 -04:00
David Steele
f207dc7123 Renumber return codes between 25 and 125 to avoid PostgreSQL interpreting some as fatal signal exceptions.
Suggested by Yogesh Sharma.
2017-04-12 12:41:34 -04:00
David Steele
f5f8d51366 Improved error message when the restore command detects the presence of postmaster.pid.
Suggested by Yogesh Sharma.
2017-04-12 09:53:07 -04:00
David Steele
fe02b40293 Backup and restore commands no longer copy via temp files.
In both cases the files are checksummed on resume so there's no danger of partial copies.
2017-04-12 09:05:57 -04:00
David Steele
3d84f2ce5e Improvements to Ini.pm.
* Refactor Ini.pm to facilitate testing.
* Complete statement/branch coverage for Ini.pm.
* Improved functions used to test/munge manifest and info files.
2017-04-10 13:24:45 -04:00