David Steele
2337be8636
Consolidate archive-push tests into new archive test.
2017-06-21 08:38:53 -04:00
David Steele
11ee528e0e
Rename Archive modules to remove redundancy.
2017-06-21 08:02:21 -04:00
David Steele
692b1e274a
Split full module into mock and real to allow better test combinations and save time in CI.
2017-06-19 18:55:00 -04:00
David Steele
f6d4457d58
Full/Synthetic test refactor.
...
* Combine hardlink and non/compressed in synthetic tests to reduce test time and improve coverage.
* Change log level of hardlink logging to detail.
* Cast size in S3 manifest to integer.
2017-06-15 15:32:10 -04:00
David Steele
051c961151
S3 repository support.
2017-06-12 10:52:32 -04:00
David Steele
de7fc37f88
Storage and IO layer refactor:
...
Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.)
Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 17:51:41 -04:00
Cynthia Shang
7e982f05f5
Fixed config warnings raised from a remote process causing errors in the master process.
...
Fixed by Cynthia Shang.
2017-05-16 11:28:51 -04:00
David Steele
5296747271
Move backup modules into Backup directory.
2017-05-15 16:01:00 -04:00
David Steele
2a23157bf2
Move modules in Protocol directory in subdirectories.
2017-05-15 11:12:14 -04:00
David Steele
33c87b2a1c
Remove flapping protocol timeout test that will be replaced in the upcoming storage patch.
2017-05-15 10:24:46 -04:00
Cynthia Shang
7d0c50fcd6
Fixed the backup command so the backup-standby option is reset (and the backup proceeds on the master) if the standby is not configured and/or reachable.
...
Fixed by Cynthia Shang.
2017-05-12 16:05:44 -04:00
Cynthia Shang
5bf4f52e3d
Fixed the info command so the WAL archive min/max displayed is for the current database version.
...
Fixed by Cynthia Shang.
2017-05-12 15:49:14 -04:00
David Steele
f207dc7123
Renumber return codes between 25 and 125 to avoid PostgreSQL interpreting some as fatal signal exceptions.
...
Suggested by Yogesh Sharma.
2017-04-12 12:41:34 -04:00
David Steele
f5f8d51366
Improved error message when the restore command detects the presence of postmaster.pid.
...
Suggested by Yogesh Sharma.
2017-04-12 09:53:07 -04:00
David Steele
fe02b40293
Backup and restore commands no longer copy via temp files.
...
In both cases the files are checksummed on resume so there's no danger of partial copies.
2017-04-12 09:05:57 -04:00
David Steele
3d84f2ce5e
Improvements to Ini.pm.
...
* Refactor Ini.pm to facilitate testing.
* Complete statement/branch coverage for Ini.pm.
* Improved functions used to test/munge manifest and info files.
2017-04-10 13:24:45 -04:00
David Steele
eb4918d018
Fixed locking bug and refactoring.
...
* Fixed an issue where read-only operations that used local worker processes (i.e. restore) were creating write locks that could interfere with parallel archive-push. (Reported by Jens Wilke.)
* Simplify locking scheme. Now, only the master process will hold write locks (archive-push, backup) and not all the local and remote worker processes as before.
2017-04-07 10:31:49 -04:00
Cynthia Shang
b620e6b282
Improvements and additional tests for stanza-upgrade.
...
Contributed by Cynthia Shang.
2017-04-06 19:36:37 -04:00
Cynthia Shang
beead043ac
Added stanza-upgrade command.
...
The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL.
Contributed by Cynthia Shang.
2017-04-03 10:42:55 -04:00
David Steele
b0947fdaa8
Fixed an issue where newly initialized (but unused) pages would cause page checksum warnings.
...
Reported by Stephen Frost.
2017-03-13 15:50:13 -04:00
David Steele
02730526fc
Fixed an issue where databases created with a non-default tablespace would raise bogus warnings about pg_filenode.map and pg_internal.init not being page aligned.
...
Reported by blogh.
2017-03-02 13:50:29 -05:00
David Steele
5957f8566f
Archive test improvements:
...
* Fixed flapping archive-stop test with testResult() retries.
* Added final test of archive contents to archive-push test.
2017-03-02 11:47:23 -05:00
David Steele
c2666cc7f3
Fixed an issue where tables over 1GB would report page checksum errors after the first segment.
...
Reported by Stephen Frost.
2017-03-01 14:15:46 -05:00
David Steele
86fee3da18
Continuous integration using travis-ci.
2017-02-21 08:59:23 -05:00
David Steele
e2ac7e1ea6
Fixed a regression introduced in v1.13 that could cause backups to fail.
...
This happened if files were removed (e.g. tables dropped) while the manifest was being built.
Reported by Navid Golpayegani.
2017-02-13 19:59:14 -05:00
David Steele
0daeace1cd
Fixed alignment issues with multiline logging.
2017-02-05 19:58:52 -05:00
David Steele
cded35791d
Added --log-timestamp option.
...
Allow timestamps to be suppressed in logging. This is primarily used to avoid filters in the automated documentation.
2017-02-05 10:59:49 -05:00
David Steele
8b5f37c2a8
Added archive and backup WAL ranges to the info command.
2017-02-05 10:31:29 -05:00
David Steele
7418559a67
Fixed a spurious archive-get log message that indicated an exit code of 1 was an abnormal termination.
2017-01-31 10:36:59 -05:00
David Steele
eb1a1e5308
Missed expect logs from commit 455c604
.
2017-01-30 16:22:12 -05:00
David Steele
6b99d770b5
Reduced the likelihood of torn pages causing a false positive in page checksums by filtering on start backup LSN.
2017-01-30 13:59:00 -05:00
David Steele
43ada9fb33
Improved, multi-process implementation of asynchronous archiving.
2017-01-27 11:02:27 -05:00
David Steele
2a9ba877e1
Refactor process IO and process master/minion code out from the common protocol code.
2017-01-27 10:06:16 -05:00
David Steele
18db178ac3
Various improvements to the test suite:
...
* Allow logging to be suppressed via logDisable() and logEnable().
* Added more flexibility in initializing and cleaning up after modules and tests.
* testResult() suppresses logging and reports exceptions.
* testException() allows messages to be matched with regular expressions.
* Refactor name/locations of common modules that setup test environments.
2017-01-27 09:42:30 -05:00
David Steele
7d1347cdec
Split the archive-get and archive-push commands into separate modules.
2017-01-10 19:59:32 -05:00
David Steele
b58045f575
Moved the Archive modules to the Archive directory.
2017-01-10 19:51:20 -05:00
David Steele
0b4055e588
Moved File->manifest() into the FileCommon.pm module.
2017-01-10 13:08:03 -05:00
David Steele
c8dfc673b8
Moved tests into new modules to better reflect actual usage.
2017-01-04 18:31:16 -05:00
Cynthia Shang
be337c951a
Improvements to stanza-create:
...
* Force only required when a change is needed for backup.info or archive.info.
* Identical files are not overwritten.
2017-01-04 10:12:29 -05:00
David Steele
b91ca11329
Added expect log expression to replace year subdirectories in backup.history.
2017-01-04 10:09:21 -05:00
David Steele
5d3473b52d
Refactor File module to improve test coverage.
2016-12-23 08:30:34 -05:00
Cynthia Shang
56144c99c0
Improved check command to run on a standby.
...
Only basic checks are done because pg_switch_xlog() cannot be executed on a replica.
Contributed by Cynthia Shang.
2016-12-21 14:35:20 -05:00
Cynthia Shang
974a02dbe4
Improved stanza-create command so that it can repair broken repositories in most cases and is robust enough to be made mandatory.
...
Contributed by Cynthia Shang.
2016-12-20 16:52:20 -05:00
David Steele
36a5349b1c
Added the --checksum-page option.
...
This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-12 18:54:07 -05:00
David Steele
6dd0829033
File copy protocol now accepts a function that can do additional processing on the copy buffers and return a result to the calling process.
2016-12-12 18:47:17 -05:00
David Steele
b4884e5a0b
Fixed an issue where the db-port option specified on the backup server would not be properly passed to the remote unless it was from the first configured database.
...
Reported by Michael Vitale.
2016-12-10 14:05:28 -05:00
David Steele
e61ef6ebce
Add more realistic data files to synthetic backup and restore tests.
2016-12-10 09:13:55 -05:00
David Steele
f818901c0e
Added the --repo-sync option to allow directory syncs in the repository to be disabled.
...
This is for file systems that do not support them, e.g. NTFS
2016-12-10 09:06:45 -05:00
David Steele
896984848b
Improved errors returned from child processes by removing redundant error level and code.
2016-12-04 17:51:00 -05:00
David Steele
817592837c
Added a predictable log entry to signal that a command has completed successfully.
...
For example a backup ends successfully with: INFO: backup command end: completed successfully.
2016-12-04 14:44:53 -05:00