1
0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2024-12-14 10:13:05 +02:00
Commit Graph

1265 Commits

Author SHA1 Message Date
David Steele
14ca4dac6f Improved the code and tests for fileManifest().
Done to prevent a possible race condition when files are removed by the database while the manifest is being built.
2017-02-21 11:56:55 -05:00
David Steele
01312f6260 Improved formatting of testResult() output. 2017-02-21 11:47:45 -05:00
David Steele
86fee3da18 Continuous integration using travis-ci. 2017-02-21 08:59:23 -05:00
David Steele
56543f7a99 Added package builds to test suite and other improvements:
* Automated builds of Debian packages for all supported distributions.
* Added --dev option to aggregate commonly used dev options.
* Added --no-package option to skip package builds.
* C library and packages are built by default, added -smart option to rebuild only when file changes are detected.
* The --libc-only option has been changed to --build-only now that packages builds have been added.
2017-02-17 22:31:16 -05:00
David Steele
ac7ff755b3 Container executions now load the user's environment. 2017-02-16 17:18:48 -05:00
David Steele
2d7d9626bb v1.15: Refactoring and Bug Fixes
Bug Fixes:

* Fixed a regression introduced in v1.13 that could cause backups to fail if files were removed (e.g. tables dropped) while the manifest was being built. (Reported by Navid Golpayegani.)

Refactoring:

* Refactor FileCommon::fileManifest() and FileCommon::fileStat to be more modular to allow complete branch/statement level coverage testing.
2017-02-13 20:00:30 -05:00
David Steele
e2ac7e1ea6 Fixed a regression introduced in v1.13 that could cause backups to fail.
This happened if files were removed (e.g. tables dropped) while the manifest was being built.

Reported by Navid Golpayegani.
2017-02-13 19:59:14 -05:00
David Steele
845c6112bf v1.14: Bug Fixes
Bug Fixes:

* Fixed an issue where an archive-push error would not be retried and would instead return errors to PostgreSQL indefinitely (unless the .error file was manually deleted). (Reported by Jens Wilke.)
* Fixed a race condition in parallel archiving where creation of new paths generated an error when multiple processes attempted to do so at the same time. (Reported by Jens Wilke.)

Refactoring:

* Improved performance of wal archive min/max provided by the info command. (Suggested by Jens Wilke.)
2017-02-13 10:07:04 -05:00
David Steele
a54018d573 Improved performance of wal archive min/max provided by the info command.
Suggested by Jens Wilke.
2017-02-13 10:03:02 -05:00
David Steele
adcc7cb954 Fixed a race condition in parallel archiving.
Creation of new paths generated an error when multiple processes attempted to do so at the same time.

Reported by Jens Wilke.
2017-02-13 09:58:50 -05:00
David Steele
bc2bb040c5 Fixed an issue where an archive-push error would not be retried.
It would instead return errors to PostgreSQL indefinitely (unless the .error file was manually deleted).

Reported by Jens Wilke.
2017-02-13 09:57:10 -05:00
David Steele
12041f73e2 Fixed syntax from d68b48d that was not backwards compatible to Perl 5.10. 2017-02-12 21:09:34 -05:00
David Steele
d68b48d19b Improved support for --require option and section depends now default to the previous section. 2017-02-11 10:26:54 -05:00
David Steele
498f52da09 Improvements to documentation engine:
* Documentation can now be built with reusable blocks to reduce duplication.
* Added ability to pass options to containers within the documentation.
* Add proper tag to slightly emphasize proper nouns.
2017-02-10 10:22:05 -05:00
David Steele
aab763d558 Additional notes on the async spool directory. 2017-02-08 09:37:17 -05:00
David Steele
b813bfaff6 Updated async archiving documentation.
More accurately describe how the new method works and how it differs from the old method.

Suggested by Jens Wilke.
2017-02-08 08:41:43 -05:00
David Steele
41eba95155 v1.13: Parallel Archiving, Stanza Create, Improved Info and Check
IMPORTANT NOTE: The new implementation of asynchronous archiving no longer copies WAL to a separate queue. If there is any WAL left over in the old queue after upgrading to 1.13, it will be abandoned and not pushed to the repository.

To prevent this outcome, stop archiving by setting archive_command = false. Next, drain the async queue by running pgbackrest --stanza=[stanza-name] archive-push and wait for the process to complete. Check that the queue in [spool-path]/archive/[stanza-name]/out is empty. Finally, install 1.13 and restore the original archive_command.

IMPORTANT NOTE: The stanza-create command is not longer optional and must be executed before backup or archiving can be performed on a new stanza. Pre-existing stanzas do not require stanza-create to be executed.

Bug Fixes:

* Fixed const assignment giving compiler warning in C library. (Fixed by Adrian Vondendriesch.)
* Fixed a few directory syncs that were missed for the --repo-sync option.
* Fixed an issue where a missing user/group on restore could cause an "uninitialized value" error in File->owner(). (Reported by Leonardo Avellar.)
* Fixed an issue where protocol mismatch errors did not output the expected value.
* Fixed a spurious archive-get log message that indicated an exit code of 1 was an abnormal termination.

Features:

* Improved, multi-process implementation of asynchronous archiving.
* Improved stanza-create command so that it can repair broken repositories in most cases and is robust enough to be made mandatory. (Contributed by Cynthia Shang.)
* Improved check command to run on a standby, though only basic checks are done because pg_switch_xlog() cannot be executed on a replica. (Contributed by Cynthia Shang.)
* Added archive and backup WAL ranges to the info command.
* Added warning to update pg_tablespace.spclocation when remapping tablespaces in PostgreSQL < 9.2. (Contributed by blogh.)
* Remove remote lock requirements for the archive-get, restore, info, and check commands since they are read-only operations. (Suggested by Michael Vitale.)

Refactoring:

* Refactor File and BackupCommon modules to improve test coverage.
* Return proper error code when unable to convert a relative path to an absolute path. (Suggested by Yogesh Sharma.)
* Log file banner is not output until the first log entry is written. (Suggested by Jens Wilke.)
* Moved File->manifest() into the FileCommon.pm module.
* Moved the Archive modules to the Archive directory and split the archive-get and archive-push commands into separate modules.
* Split the check command out of the Archive.pm module.
* Allow logging to be suppressed via logDisable() and logEnable().
* Allow for locks to be taken more than once in the same process without error.
* Lock directories can be created when more than one directory level is required.
* Clean up optionValid()/optionTest() logic in Lock.pm.
* Added Exception::exceptionCode() and Exception::exceptionMessage() to simplify error handling logic.
* Represent .gz extension with a constant.
* Allow empty files to be created with FileCommon::fileStringWrite() and use temp files to avoid partial reads.
* Refactor process IO and process master/minion code out from the common protocol code.
* Reduced the likelihood of torn pages causing a false positive in page checksums by filtering on start backup LSN.
* Remove Intel-specific optimization from C library build flags. (Contributed by Adrian Vondendriesch.)
* Removed --lock option. This option was introduced before the lock directory could be located outside the repository and is now obsolete.
* Added --log-timestamp option to allow timestamps to be suppressed in logging. This is primarily used to avoid filters in the automated documentation.
* Fixed alignment issues with multiline logging.
2017-02-05 20:23:03 -05:00
David Steele
0daeace1cd Fixed alignment issues with multiline logging. 2017-02-05 19:58:52 -05:00
David Steele
cded35791d Added --log-timestamp option.
Allow timestamps to be suppressed in logging. This is primarily used to avoid filters in the automated documentation.
2017-02-05 10:59:49 -05:00
David Steele
55feae645e Log file banner is not output until the first log entry is written.
Suggested by Jens Wilke.
2017-02-05 10:37:54 -05:00
David Steele
2237c3fc1b Added documentation for parallel archive-push. 2017-02-05 10:35:53 -05:00
David Steele
8b5f37c2a8 Added archive and backup WAL ranges to the info command. 2017-02-05 10:31:29 -05:00
David Steele
1666ee177f Fixed --no-online tests to suppress expected errors. 2017-02-02 20:44:42 -05:00
David Steele
efdf64506a Remove remote lock requirements for the archive-get, restore, info, and check commands since they are read-only operations.
Suggested by Michael Vitale.
2017-02-02 20:42:45 -05:00
David Steele
3e54aad88f Removed --lock option.
This option was introduced before the lock directory could be located outside the repository and is now obsolete.
2017-01-31 10:38:45 -05:00
David Steele
7418559a67 Fixed a spurious archive-get log message that indicated an exit code of 1 was an abnormal termination. 2017-01-31 10:36:59 -05:00
Adrian Vondendriesch
9b92edddbe Remove Intel-specific optimization from C library build flags.
Contributed by Adrian Vondendriesch.
2017-01-30 17:06:35 -05:00
David Steele
eb1a1e5308 Missed expect logs from commit 455c604. 2017-01-30 16:22:12 -05:00
David Steele
34e4cfaeec Fixed minor typos. 2017-01-30 15:48:10 -05:00
David Steele
455c6048ee C Library builds immune to clock differences between VM and host. 2017-01-30 14:00:40 -05:00
David Steele
6b99d770b5 Reduced the likelihood of torn pages causing a false positive in page checksums by filtering on start backup LSN. 2017-01-30 13:59:00 -05:00
David Steele
43ada9fb33 Improved, multi-process implementation of asynchronous archiving. 2017-01-27 11:02:27 -05:00
David Steele
2a9ba877e1 Refactor process IO and process master/minion code out from the common protocol code. 2017-01-27 10:06:16 -05:00
David Steele
0163d3b3e0 Allow empty files to be created with FileCommon::fileStringWrite() and use temp files to avoid partial reads. 2017-01-27 10:04:41 -05:00
David Steele
2553c7a998 Represent .gz extension with a constant. 2017-01-27 10:02:00 -05:00
David Steele
f3ccf3b897 Added Exception::exceptionCode() and Exception::exceptionMessage() to simplify error handling logic. 2017-01-27 09:56:26 -05:00
David Steele
6620594b77 Various improvements in Lock.pm:
* Allow for locks to be taken more than once in the same process without error.
* Lock directories can be created when more than one directory level is required.
* Clean up optionValid()/optionTest() logic in Lock.pm.
2017-01-27 09:53:46 -05:00
David Steele
18db178ac3 Various improvements to the test suite:
* Allow logging to be suppressed via logDisable() and logEnable().
* Added more flexibility in initializing and cleaning up after modules and tests.
* testResult() suppresses logging and reports exceptions.
* testException() allows messages to be matched with regular expressions.
* Refactor name/locations of common modules that setup test environments.
2017-01-27 09:42:30 -05:00
David Steele
e876a222fc Fixed an issue where protocol mismatch errors did not output the expected value. 2017-01-24 14:58:55 -05:00
Benoit Lobréau (blogh)
b6fa146f9b Added warning to update pg_tablespace.spclocation when remapping tablespaces in PostgreSQL < 9.2.
Contributed by blogh.
2017-01-22 19:29:56 -05:00
David Steele
7d1347cdec Split the archive-get and archive-push commands into separate modules. 2017-01-10 19:59:32 -05:00
David Steele
155c808163 Split the check command out of the Archive.pm module. 2017-01-10 19:54:51 -05:00
David Steele
b58045f575 Moved the Archive modules to the Archive directory. 2017-01-10 19:51:20 -05:00
David Steele
2ab63f27b2 C Library builds only run when C library has actually changed. 2017-01-10 13:12:53 -05:00
David Steele
0b4055e588 Moved File->manifest() into the FileCommon.pm module. 2017-01-10 13:08:03 -05:00
David Steele
0e4f51c271 Added integration for testing coverage with Devel::Cover. 2017-01-09 20:49:04 -05:00
David Steele
39744a4f1d Fixed an issue where a missing user/group on restore could cause an "uninitialized value" error in File->owner().
Reported by Leonardo Avellar.
2017-01-05 15:19:56 -05:00
David Steele
4ff2714bf0 Return proper error code when unable to convert a relative path to an absolute path.
Suggested by Yogesh Sharma.
2017-01-04 18:39:50 -05:00
David Steele
c8dfc673b8 Moved tests into new modules to better reflect actual usage. 2017-01-04 18:31:16 -05:00
David Steele
e3d63c0252 Allow multiple --module, --test, and --run options to be used for test.pl. 2017-01-04 18:29:13 -05:00