1
0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2024-12-14 10:13:05 +02:00
Commit Graph

56 Commits

Author SHA1 Message Date
David Steele
be90028100 Rename db-* options to pg-* and backup-* options to repo-* to improve consistency.
* repo-* options are now indexed although only one is allowed.
* List deprecated option names in documentation and command-line help.
2018-02-03 18:27:38 -05:00
David Steele
bffc6c49b3 Add perl-bin option to specify the Perl binary location when /usr/bin/env perl won't work. 2018-01-16 13:23:08 -05:00
Cynthia Shang
0e9ba98a50 Add stanza-delete command to cleanup unused stanzas.
Contributed by Cynthia Shang.
2018-01-03 12:23:33 -05:00
David Steele
526acca5bd Config parsing implemented in C. 2017-12-22 23:27:49 -05:00
David Steele
5ea1263d8e Run valgrind on all C unit tests.
Also refactor code to make valgrind happy.
2017-12-22 18:36:36 -05:00
David Steele
4f1725dc36 Replace cfgCommandTotal()/cfgOptionTotal() functions with constants.
The constants are applicable in more cases and allow the compiler to optimize certain loops more efficiently.
2017-12-05 09:45:51 -05:00
David Steele
2ed426a70b Remove deprecated archive-max-mb option. 2017-11-27 20:43:44 -05:00
David Steele
7cf955425e The C library is now required.
This eliminates conditional loading and eases development of new library features.
2017-11-26 17:45:00 -05:00
David Steele
c164fc5fb9 Add memGrowRaw() to memory context module. 2017-11-26 12:22:50 -05:00
David Steele
ba6b49ecb7 v1.26: Repository Encryption
Bug Fixes:

* Fixed an issue that could cause copying large manifests to fail during restore. (Reported by Craig A. James.)
* Fixed incorrect WAL offset for 32-bit architectures. (Fixed by Javier Wilson.)
* Fixed an issue retrieving WAL for old database versions. After a stanza-upgrade it should still be possible to restore backups from the previous version and perform recovery with archive-get. However, archive-get only checked the most recent db version/id and failed. Also clean up some issues when the same db version/id appears multiple times in the history. (Fixed by Cynthia Shang. Reported by Clinton Adams.)
* Fixed an issue with invalid backup groups being set correctly on restore. If the backup cannot map a group to a name it stores the group in the manifest as false then uses either the owner of $PGDATA to set the group during restore or failing that the group of the current user. This logic was not working correctly because the selected group was overwriting the user on restore leaving the group undefined and the user incorrectly set to the group. (Reported by Jeff McCormick.)
* Fixed an issue passing parameters to remotes. When more than one db was specified the path, port, and socket path would for db1 were passed no matter which db was actually being addressed. (Reported by Uspen.)

Features:

* Repository encryption support. (Contributed by Cynthia Shang, David Steele.)
2017-11-21 18:31:54 -05:00
David Steele
3b43047e0d Disable -Wclobber compiler warning.
...because it is mostly useless but keep the rest of of -Wextra.
2017-11-19 16:44:33 -05:00
David Steele
dc1a5c18ac Wrap try in a do...while loop to make sure that no random else is attached to the main if block. 2017-11-19 16:30:23 -05:00
David Steele
cf7da546a3 Simplify try..catch..finally names. 2017-11-13 21:22:13 -05:00
David Steele
bcdfc7d0b5 Enable additional warnings for C builds. 2017-11-06 22:55:34 -05:00
Cynthia Shang
b03c26968a Repository encryption support.
Contributed by Cynthia Shang.
2017-11-06 12:51:12 -05:00
David Steele
8d6a08a32b Library code for repository encryption support. 2017-11-03 13:57:58 -04:00
David Steele
ac542788da Convert configuration definitions from auto-generated functions to auto-generated data structures. 2017-11-02 08:14:13 -04:00
David Steele
55f68ef013 Remove configurable option hints.
db-path was the only option with a hint so the feature seemed wasteful. All missing stanza options now output the same hint without needing configuration.
2017-10-30 10:58:04 -04:00
David Steele
1ef27ec8c2 Add list type for options.
The hash type was being used for lists with an additional flag (`value-hash`) to indicate that it was not really a hash.
2017-10-30 10:50:35 -04:00
David Steele
f4524aeaa9 Add base64 encode/decode. 2017-10-18 11:14:26 -04:00
David Steele
6f5186f9e6 Convert config tests into C unit tests. 2017-10-16 11:13:51 -04:00
David Steele
9d78948a14 Improvements to page checksum module and tests:
* Page checksum module uses new C error handler.
* Convert page checksum tests into C unit tests.
2017-10-16 11:03:06 -04:00
David Steele
904b83747c Perl error handler recognizes errors thrown from the C library. 2017-10-16 10:47:31 -04:00
David Steele
f1e739b4aa Add C error handler.
Basic try ... catch ... finally pattern to simplify error handling in C.
2017-10-16 10:09:56 -04:00
David Steele
10dfbd90b5 Add C unit test infrastructure. 2017-10-12 12:55:48 -04:00
David Steele
08d6f14603 Add LibC.template.pm to simplify LibC module generation. 2017-10-10 18:21:44 -04:00
David Steele
8efcc38304 Improvements in C codebase:
* Update C naming conventions.
* Use int datatype wherever possible.
* Better separation of C source from Perl interface.
2017-09-30 10:44:03 -04:00
David Steele
d36c52271f Break up LibC.xs into separate module files. 2017-09-17 15:52:19 -04:00
David Steele
20af8ecf50 Dynamically generate list of files for C library build. 2017-09-17 14:49:23 -04:00
Cynthia Shang
36e576b483 Up to seven standbys can be configured for backup from standby.`
Contributed by Cynthia Shang.
2017-08-31 19:15:44 -04:00
David Steele
1e0ed07455 Configuration rules are now pulled from the C library when present. 2017-08-25 16:47:47 -04:00
David Steele
84caca0f34 v1.22: Fixed S3 Retry
Bug Fixes:

* Fixed authentication issue in S3 retry.
2017-08-09 11:50:02 -04:00
David Steele
61c38f5808 Fixed authentication issue in S3 retry. 2017-08-09 11:27:09 -04:00
David Steele
cbc026418c v1.21: Improved Info Output and SSH Port Option
Bug Fixes:

* The archive_status directory is now recreated on restore to support PostgreSQL 8.3 which does not recreate it automatically like more recent versions do. (Reported by Stephen Frost.)
* Fixed an issue that could cause the empty archive directory for an old PostgreSQL version to be left behind after a stanza-upgrade. (Fixed by Cynthia Shang.)

Features:

* Modified the info command (both text and JSON output) to display the archive ID and minimum/maximum WAL currently present in the archive for the current and prior, if any, database cluster version. (Contributed by Cynthia Shang.)
* Added --backup-ssh-port and --db-ssh-port options to support non-default SSH ports. (Contributed by Cynthia Shang.)

Refactoring:

* Retry when S3 returns an internal error (500).
* Add bIgnoreMissing parameter to Local->manifest().
2017-08-08 18:09:07 -04:00
Cynthia Shang
cda3b9821b Improved WAL data returned by info command.
Modified the info command (both text and JSON output) to display the archive ID and minimum/maximum WAL currently present in the archive for the current and prior, if any, database cluster version.

Contributed by Cynthia Shang.
2017-07-21 17:32:41 -04:00
David Steele
33d769e147 v1.20: Critical 8.3/8.4 Bug Fix
Bug Fixes:

* Fixed an issue that prevented tablespaces from being backed up on PostgreSQL ≤ 8.4.
* Fixed missing flag in C library build that resulted in a mismatched binary on 32-bit systems. (Reported by Adrian Vondendriesch.)

Features:

* Add s3-repo-ca-path and s3-repo-ca-file options to accommodate systems where CAs are not automatically found by IO::Socket::SSL, i.e. RHEL7, or to load custom CAs. (Suggested by Scott Frazer.)

Refactoring:

* Harden protocol handshake to handle race conditions.
* Fixed misleading error message when a file was opened for write in a missing directory.
* Change log level of hardlink logging to detail.
* Cast size in S3 manifest to integer.
* Rename Archive modules to remove redundancy.
* Improve S3 error reporting.
* Minor optimizations to package loads and ordering for archive-get and archive-push commands.
2017-06-27 22:42:31 -04:00
David Steele
d3054d919b Fixed missing flag in C library build.
The missing flag resulted in a mismatched binary on 32-bit systems.

Reported by Adrian Vondendriesch.
2017-06-22 12:57:10 -04:00
David Steele
6e29f616b3 Harden protocol handshake to handle race conditions. 2017-06-15 15:12:28 -04:00
David Steele
c10c5bb0ef v1.19: S3 Support
Bug Fixes:

* Fixed the info command so the WAL archive min/max displayed is for the current database version. (Fixed by Cynthia Shang.)
* Fixed the backup command so the backup-standby option is reset (and the backup proceeds on the master) if the standby is not configured and/or reachable. (Fixed by Cynthia Shang.)
* Fixed config warnings raised from a remote process causing errors in the master process. (Fixed by Cynthia Shang.)

Features:

* Amazon S3 repository support. (Reviewed by Cynthia Shang.)

Refactoring:

* Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.)
* Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
* Move modules in Protocol directory in subdirectories.
* Move backup modules into Backup directory.
2017-06-12 11:36:00 -04:00
David Steele
2a23157bf2 Move modules in Protocol directory in subdirectories. 2017-05-15 11:12:14 -04:00
David Steele
76161bd845 v1.18: Stanza Upgrade, Refactoring, and Locking Improvements
Bug Fixes:

* Fixed an issue where read-only operations that used local worker processes (i.e. restore) were creating write locks that could interfere with parallel archive-push. (Reported by Jens Wilke.)

Features:

* Added the stanza-upgrade command to provide a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. (Contributed by Cynthia Shang.)
* Added validation of pgbackrest.conf to display warnings if options are not valid or are not in the correct section. (Contributed by Cynthia Shang.)

Refactoring:

* Simplify locking scheme. Now, only the master process will hold write locks (for archive-push and backup commands) and not all local and remote worker processes as before.
* Refactor Ini.pm to facilitate testing.
* Do not set timestamps of files in the backup directories to match timestamps in the cluster directory. This was originally done to enable backup resume, but that process is now implemented with checksums.
* Improved error message when the restore command detects the presence of postmaster.pid. (Suggested by Yogesh Sharma.)
* Renumber return codes between 25 and 125 to avoid PostgreSQL interpreting some as fatal signal exceptions. (Suggested by Yogesh Sharma.)
* The backup and restore commands no longer copy via temp files. In both cases the files are checksummed on resume so there's no danger of partial copies.
* Allow functions to accept optional parameters as a hash.
* Refactor File->list() and fileList() to accept optional parameters.
* Refactor backupLabel() and add unit tests.
* Silence some perl critic warnings. (Contributed by Cynthia Shang.)
2017-04-12 19:17:39 -04:00
Cynthia Shang
af7e4f4b4b Added validation of pgbackrest.conf to display warnings if options are not valid or are not in the correct section.
Contributed by Cynthia Shang.
2017-03-22 19:03:28 -04:00
David Steele
cc9f8fc4b6 v1.17: Page Checksum Bug Fix
Bug Fixes:

* Fixed an issue where newly initialized (but unused) pages would cause page checksum warnings. (Reported by Stephen Frost.)
2017-03-13 16:54:51 -04:00
David Steele
b0947fdaa8 Fixed an issue where newly initialized (but unused) pages would cause page checksum warnings.
Reported by Stephen Frost.
2017-03-13 15:50:13 -04:00
David Steele
3cec6702e8 v1.16: Page Checksum Improvements, CI, and Package Testing
Bug Fixes:

* Fixed an issue where tables over 1GB would report page checksum warnings after the first segment. (Reported by Stephen Frost.)
* Fixed an issue where databases created with a non-default tablespace would raise bogus warnings about pg_filenode.map and pg_internal.init not being page aligned. (Reported by blogh.)

Refactoring:

* Improved the code and tests for fileManifest() to prevent a possible race condition when files are removed by the database while the manifest is being built.
2017-03-02 20:15:14 -05:00
David Steele
14ca4dac6f Improved the code and tests for fileManifest().
Done to prevent a possible race condition when files are removed by the database while the manifest is being built.
2017-02-21 11:56:55 -05:00
David Steele
2d7d9626bb v1.15: Refactoring and Bug Fixes
Bug Fixes:

* Fixed a regression introduced in v1.13 that could cause backups to fail if files were removed (e.g. tables dropped) while the manifest was being built. (Reported by Navid Golpayegani.)

Refactoring:

* Refactor FileCommon::fileManifest() and FileCommon::fileStat to be more modular to allow complete branch/statement level coverage testing.
2017-02-13 20:00:30 -05:00
David Steele
e2ac7e1ea6 Fixed a regression introduced in v1.13 that could cause backups to fail.
This happened if files were removed (e.g. tables dropped) while the manifest was being built.

Reported by Navid Golpayegani.
2017-02-13 19:59:14 -05:00
David Steele
845c6112bf v1.14: Bug Fixes
Bug Fixes:

* Fixed an issue where an archive-push error would not be retried and would instead return errors to PostgreSQL indefinitely (unless the .error file was manually deleted). (Reported by Jens Wilke.)
* Fixed a race condition in parallel archiving where creation of new paths generated an error when multiple processes attempted to do so at the same time. (Reported by Jens Wilke.)

Refactoring:

* Improved performance of wal archive min/max provided by the info command. (Suggested by Jens Wilke.)
2017-02-13 10:07:04 -05:00
David Steele
bc2bb040c5 Fixed an issue where an archive-push error would not be retried.
It would instead return errors to PostgreSQL indefinitely (unless the .error file was manually deleted).

Reported by Jens Wilke.
2017-02-13 09:57:10 -05:00