1
0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2024-12-14 10:13:05 +02:00
Commit Graph

184 Commits

Author SHA1 Message Date
David Steele
6b99d770b5 Reduced the likelihood of torn pages causing a false positive in page checksums by filtering on start backup LSN. 2017-01-30 13:59:00 -05:00
David Steele
43ada9fb33 Improved, multi-process implementation of asynchronous archiving. 2017-01-27 11:02:27 -05:00
David Steele
2a9ba877e1 Refactor process IO and process master/minion code out from the common protocol code. 2017-01-27 10:06:16 -05:00
David Steele
0163d3b3e0 Allow empty files to be created with FileCommon::fileStringWrite() and use temp files to avoid partial reads. 2017-01-27 10:04:41 -05:00
David Steele
2553c7a998 Represent .gz extension with a constant. 2017-01-27 10:02:00 -05:00
David Steele
f3ccf3b897 Added Exception::exceptionCode() and Exception::exceptionMessage() to simplify error handling logic. 2017-01-27 09:56:26 -05:00
David Steele
6620594b77 Various improvements in Lock.pm:
* Allow for locks to be taken more than once in the same process without error.
* Lock directories can be created when more than one directory level is required.
* Clean up optionValid()/optionTest() logic in Lock.pm.
2017-01-27 09:53:46 -05:00
David Steele
18db178ac3 Various improvements to the test suite:
* Allow logging to be suppressed via logDisable() and logEnable().
* Added more flexibility in initializing and cleaning up after modules and tests.
* testResult() suppresses logging and reports exceptions.
* testException() allows messages to be matched with regular expressions.
* Refactor name/locations of common modules that setup test environments.
2017-01-27 09:42:30 -05:00
David Steele
e876a222fc Fixed an issue where protocol mismatch errors did not output the expected value. 2017-01-24 14:58:55 -05:00
Benoit Lobréau (blogh)
b6fa146f9b Added warning to update pg_tablespace.spclocation when remapping tablespaces in PostgreSQL < 9.2.
Contributed by blogh.
2017-01-22 19:29:56 -05:00
David Steele
7d1347cdec Split the archive-get and archive-push commands into separate modules. 2017-01-10 19:59:32 -05:00
David Steele
155c808163 Split the check command out of the Archive.pm module. 2017-01-10 19:54:51 -05:00
David Steele
b58045f575 Moved the Archive modules to the Archive directory. 2017-01-10 19:51:20 -05:00
David Steele
2ab63f27b2 C Library builds only run when C library has actually changed. 2017-01-10 13:12:53 -05:00
David Steele
0b4055e588 Moved File->manifest() into the FileCommon.pm module. 2017-01-10 13:08:03 -05:00
David Steele
0e4f51c271 Added integration for testing coverage with Devel::Cover. 2017-01-09 20:49:04 -05:00
David Steele
39744a4f1d Fixed an issue where a missing user/group on restore could cause an "uninitialized value" error in File->owner().
Reported by Leonardo Avellar.
2017-01-05 15:19:56 -05:00
David Steele
4ff2714bf0 Return proper error code when unable to convert a relative path to an absolute path.
Suggested by Yogesh Sharma.
2017-01-04 18:39:50 -05:00
David Steele
e3d63c0252 Allow multiple --module, --test, and --run options to be used for test.pl. 2017-01-04 18:29:13 -05:00
David Steele
b91ca11329 Added expect log expression to replace year subdirectories in backup.history. 2017-01-04 10:09:21 -05:00
David Steele
314ef2c8b3 Update LICENSE.txt for 2017. 2017-01-04 10:05:43 -05:00
Cynthia Shang
d5ee7d3535 Added documentation to the User Guide for the process-max option.
Contributed by Cynthia Shang.
2016-12-23 16:50:02 -05:00
David Steele
83beab7ec3 Refactor File module to improve test coverage.
Added unit tests for low-level functions in the BackupCommon module.
2016-12-23 11:43:26 -05:00
David Steele
5d3473b52d Refactor File module to improve test coverage. 2016-12-23 08:30:34 -05:00
David Steele
6b2666a9d7 Split test modules into separate files.
Makes the code more maintainable. Tests are dynamically loaded by name rather than requiring an if-else block.
2016-12-23 08:22:59 -05:00
Cynthia Shang
56144c99c0 Improved check command to run on a standby.
Only basic checks are done because pg_switch_xlog() cannot be executed on a replica.

Contributed by Cynthia Shang.
2016-12-21 14:35:20 -05:00
Cynthia Shang
974a02dbe4 Improved stanza-create command so that it can repair broken repositories in most cases and is robust enough to be made mandatory.
Contributed by Cynthia Shang.
2016-12-20 16:52:20 -05:00
David Steele
aecc0f737e Fixed a few directory syncs that were missed for the --repo-sync option. 2016-12-19 14:40:22 -05:00
Adrian Vondendriesch
4e1b74688d Fixed const assignment giving compiler warning in C library.
Fixed by Adrian Vondendriesch.
2016-12-13 08:22:36 -05:00
David Steele
8e3464da80 v1.12: Page Checksums, Configuration, and Bug Fixes
IMPORTANT NOTE: In prior releases it was possible to specify options on the command-line that were invalid for the current command without getting an error. An error will now be generated for invalid options so it is important to carefully check command-line options in your environment to prevent disruption.

Bug Fixes:

* Fixed an issue where options that were invalid for the specified command could be provided on the command-line without generating an error. The options were ignored and did not cause any change in behavior, but it did lead to some confusion. Invalid options will now generate an error. (Reported by Nikhilchandra Kulkarni.)
* Fixed an issue where internal symlinks were not being created for tablespaces in the repository. This issue was only apparent when trying to bring up clusters in-place manually using filesystem snapshots and did not affect normal backup and restore.
* Fixed an issue that prevented errors from being output to the console before the logging system was initialized, i.e. while parsing options. Error codes were still being returned accurately so this would not have made a process look like it succeeded when it did not. (Reported by Adrian Vondendriesch.)
* Fixed an issue where the db-port option specified on the backup server would not be properly passed to the remote unless it was from the first configured database. (Reported by Michael Vitale.)

Features:

* Added the --checksum-page option to allow pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster. (Suggested by Stephen Frost.)
* Added the --repo-link option to allow internal symlinks to be suppressed when the repository is located on a filesystem that does not support symlinks. This does not affect any pgBackRest functionality, but the convenience link latest will not be created and neither will internal tablespace symlinks, which will affect the ability to bring up clusters in-place manually using filesystem snapshots.
* Added the --repo-sync option to allow directory syncs in the repository to be disabled for file systems that do not support them, e.g. NTFS.
* Added a predictable log entry to signal that a command has completed successfully. For example a backup ends successfully with: INFO: backup command end: completed successfully. (Suggested by Jens Wilke.)

Refactoring:

* Abstracted code to determine which database cluster is the master and which are standbys. (Contributed by Cynthia Shang.)
* Improved consistency and flexibility of the protocol layer by using JSON for all messages.
* File copy protocol now accepts a function that can do additional processing on the copy buffers and return a result to the calling process.
* Improved IO->bufferRead to always return requested number of bytes until EOF.
* For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
* Simplified the result hash of File->manifest(), Db->tablespaceMapGet(), and Db->databaseMapGet().
* Improved errors returned from child processes by removing redundant error level and code.
* Code cleanup in preparation for improved stanza-create command. (Contributed by Cynthia Shang.)
* Improved parameter/result logging in debug/trace functions.
2016-12-12 19:18:27 -05:00
David Steele
36a5349b1c Added the --checksum-page option.
This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-12 18:54:07 -05:00
David Steele
6dd0829033 File copy protocol now accepts a function that can do additional processing on the copy buffers and return a result to the calling process. 2016-12-12 18:47:17 -05:00
David Steele
b4884e5a0b Fixed an issue where the db-port option specified on the backup server would not be properly passed to the remote unless it was from the first configured database.
Reported by Michael Vitale.
2016-12-10 14:05:28 -05:00
David Steele
1a5fa920e6 Improved parameter/result logging in debug/trace functions. 2016-12-10 09:15:20 -05:00
David Steele
e61ef6ebce Add more realistic data files to synthetic backup and restore tests. 2016-12-10 09:13:55 -05:00
David Steele
eff7b46eb1 Various improvements to validation of backup and restore. 2016-12-10 09:11:12 -05:00
David Steele
f818901c0e Added the --repo-sync option to allow directory syncs in the repository to be disabled.
This is for file systems that do not support them, e.g. NTFS
2016-12-10 09:06:45 -05:00
David Steele
36378c47bd Added Retention to QuickStart section. 2016-12-04 18:11:23 -05:00
David Steele
896984848b Improved errors returned from child processes by removing redundant error level and code. 2016-12-04 17:51:00 -05:00
David Steele
817592837c Added a predictable log entry to signal that a command has completed successfully.
For example a backup ends successfully with: INFO: backup command end: completed successfully.
2016-12-04 14:44:53 -05:00
David Steele
f823a25ac2 Fixed an issue where options that were invalid for the specified command could be provided on the command-line without generating an error.
* The options were ignored and did not cause any change in behavior, but it did lead to some confusion. Invalid options will now generate an error.
* Removed erroneous --no-config option in help test module.
* Changed the --no-fork test option to --fork with negation to match all other boolean parameters.
2016-12-04 09:25:40 -05:00
David Steele
d18cd7ad0f Fixed an issue that prevented errors from being output to the console before the logging system was initialized.
That is, while parsing options. Error codes were still being returned accurately so this would not have made a process look like it succeeded when it did not.
2016-12-03 18:01:17 -05:00
David Steele
2e01e6b9e1 Improved consistency and flexibility of the protocol layer by using JSON for all messages. 2016-12-03 17:34:51 -05:00
Cynthia Shang
b7f1738200 Code cleanup in preparation for improved stanza-create command.
Contributed by Cynthia Shang.
2016-12-03 10:28:08 -05:00
David Steele
005fc08795 Added the --repo-link option.
Allow internal symlinks to be suppressed when the repository is located on a filesystem that does not support symlinks. This does not affect any pgBackRest functionality, but the convenience link latest will not be created and neither will internal tablespace symlinks, which will affect the ability to bring up clusters in-place manually using filesystem snapshots.
2016-12-03 10:23:26 -05:00
David Steele
6ae4d7de81 Fixed a bug where internal symlinks were not being created for tablespaces in the repository.
This issue was only apparent when trying to bring up clusters in-place with filesystem snapshots and did not affect normal backup and restore.
2016-12-03 09:56:05 -05:00
Cynthia Shang
22cd29771f Update control and wal test files to 9.4 with matching system identifiers.
Contributed by Cynthia Shang.
2016-12-01 09:33:14 -05:00
David Steele
a850335015 Simplified the result hash of File->manifest(), Db->tablespaceMapGet(), and Db->databaseMapGet(). 2016-11-30 14:36:39 -05:00
David Steele
09df07efb7 For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process.
The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 14:23:05 -05:00
David Steele
4e9f8da9a6 Improved IO->bufferRead to always return requested number of bytes until EOF. 2016-11-30 14:18:12 -05:00