1
0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2024-12-14 10:13:05 +02:00
Commit Graph

810 Commits

Author SHA1 Message Date
David Steele
5ada7fb5ad Use JSON::PP instead of JSON since it is shipped with core Perl. 2015-05-12 15:44:10 -04:00
David Steele
ab4efce5d1 Merge branch 'master' into dev after release 0.65 2015-05-11 18:36:02 -04:00
David Steele
49fe40850f v0.65: Improved resume and restore logging, compact restores
* Better resume support.  Resumed files are checked to be sure they have not been modified and the manifest is saved more often to preserve checksums as the backup progresses.  More unit tests to verify each resume case.

* Resume is now optional.  Use the `resume` setting or `--no-resume` from the command line to disable.

* More info messages during restore.  Previously, most of the restore messages were debug level so not a lot was output in the log.

* Fixed an issue where an absolute path was not written into recovery.conf when the restore was run with a relative path.

* Added `tablespace` setting to allow tablespaces to be restored into the `pg_tblspc` path.  This produces compact restores that are convenient for development, staging, etc.  Currently these restores cannot be backed up as PgBackRest expects only links in the `pg_tblspc` path.
2015-05-11 18:29:40 -04:00
David Steele
a7d6382cf2 Unit test fix for absolute path change in a0d985b7ad. 2015-05-08 19:33:34 -04:00
David Steele
9aa41ee76b Added tablespace setting to allow compact database restores for development, staging, etc. 2015-05-08 18:34:27 -04:00
David Steele
a0d985b7ad Fixed an issue where an absolute path was not written into recovery.conf when the restore was run with a relative path. 2015-05-07 17:37:25 -05:00
David Steele
1ac4b781fd Better info logging for restore. Most of the messages were debug before and some important ones were missing. 2015-05-07 15:56:56 -06:00
David Steele
095a9a0b83 Added resume setting to allow disabling resume feature. 2015-05-07 10:29:30 -06:00
David Steele
328c2ca5c0 Much better resume:
1) Re-checksums files that have checksums in the manifest
2) Recopies files that do not have a checksum
3) Saves the manifest at regular intervals to preserve checksums
4) Unit tests for all cases (that I can think of)
2015-05-06 18:24:34 -06:00
David Steele
56588f6fdd Log testing can now be enabled for certain deterministic tests. This works by comparing the generated logs against a previous copy. Currently only enabled for the backup/synthetic tests. 2015-05-05 11:08:48 -06:00
David Steele
1d1c7e47d1 Doc fixes for archive-max-mb setting. 2015-05-01 19:13:12 -04:00
David Steele
b0db7c0497 Fixed case error in Perl module name. 2015-04-29 17:43:28 -04:00
David Steele
174896e797 Centralized wait functions. 2015-04-23 20:52:41 -04:00
David Steele
2dac171642 More modular locking - this is the first part of a series of improvements. 2015-04-22 16:39:53 -04:00
David Steele
87f69508e3 Set version to 0.75 2015-04-21 10:48:01 -04:00
David Steele
6987cfaa53 Merge branch 'master' into dev after release 0.61 2015-04-21 09:46:28 -04:00
David Steele
a9b9ec2492 v0.61: bug fix for uncompressed remote destination
* Fixed a buffering error that could occur on large, highly-compressible files when copying to an uncompressed remote destination.  The error was detected in the decompression code and resulted in a failed backup rather than corruption so it should not affect successful backups made with previous versions.
2015-04-21 09:45:03 -04:00
David Steele
4b51f5dc50 Removed unused font. 2015-04-19 18:00:51 -04:00
David Steele
4bc9ee9b8a Merge branch 'master' into dev after release v0.60. 2015-04-19 17:53:45 -04:00
David Steele
5185d49873 v0.60: better version support and WAL improvements
* Pushing duplicate WAL now generates an error.  This worked before only if checksums were disabled.

* Database System IDs are used to make sure that all WAL in an archive matches up.  This should help prevent misconfigurations that send WAL from multiple clusters to the same archive.

* Regression tests working back to PostgreSQL 8.3.

* Improved threading model by starting threads early and terminating them late.
2015-04-19 17:53:29 -04:00
David Steele
279bd07623 Each option is assigned a source to designate where it came from (param, config, default).
operationWrite() created to easily pass parameters on to a new process.
2015-04-19 17:29:52 -04:00
David Steele
e20a3c991d Added tests for set = latest. 2015-04-19 17:28:19 -04:00
David Steele
cb153f7242 Docs and comment fixes. 2015-04-19 17:27:40 -04:00
David Steele
6646089f9a wait_remainder() now waits until .05 seconds after the second to resolve some resolution problems. 2015-04-19 17:27:01 -04:00
David Steele
f1cc1d6123 Refactored backup manifest update by centralizing the code. 2015-04-11 15:02:04 -04:00
David Steele
b8ce891603 Altered rsync comparison test to use a specially crafted table that has a 70% compression ratio.
Improved the output of file_size_format().
2015-04-11 12:31:39 -04:00
David Steele
9fe7f87e3a Allow any boolean option that is not command-line only to be negated. 2015-04-07 19:21:48 -04:00
David Steele
43d86e64a4 First pass at tests comparing rsync to backrest. Decent results, but room for improvement.
All tests local over SSH with rsync default compression, 4 threads and default compression on backrest.  Backrest default is gzip = 6, assuming rsync is the same.

On a 1GB DB:

rsync time = 32.82
backrest time = 19.48

backrest is 171% faster.

On a 5GB DB:

rsync time = 171.16
backrest time = 86.97

backrest is 196% faster.
2015-04-07 18:36:59 -04:00
David Steele
7081c8b867 New model where threads are created early and destroyed late.
Backups now work like restores in terms of how jobs are queued.
Split out BackupFile and RestoreFile for easier multi-threading/processing.
2015-04-07 07:34:37 -04:00
David Steele
808c7863b6 Duplicate WAL issue in 9.3 was fixed in 9.3.6. 2015-04-03 09:56:03 -04:00
David Steele
3f651a8ce8 Unit tests will now work across all installed versions of Postgres.
Created a function to list all supported versions.  Now used for all version checking.
2015-04-02 22:07:23 -04:00
David Steele
0b27c74999 Increased version to 0.60 and added release notes. 2015-04-01 16:28:16 -04:00
David Steele
984fceea67 Moved archive functions from pg_backrest.pl and Backup.pl to Archive.pm.
Moved Remote code from pg_backrest.pl to Config.pm.
Added version specific code to regression tests and Db.pm.
archive-push checks for duplicate WAL in the archive.
archive-push reads the db sys id to match up WAL to the correct archive.
2015-04-01 15:58:33 -04:00
David Steele
9caccc2ce2 Doc fixes. 2015-03-29 13:35:36 -04:00
David Steele
92a4d648ac Fixed issues in collision unit tests. 2015-03-26 17:42:45 -04:00
David Steele
16cc2110c5 Merge branch 'master' into dev 2015-03-25 15:17:59 -04:00
David Steele
b37d59832f v0.50: restore and much more
* Added restore functionality.

* All options can now be set on the command-line making pg_backrest.conf optional.

* De/compression is now performed without threads and checksum/size is calculated in stream.  That means file checksums are no longer optional.

* Added option `--no-start-stop` to allow backups when Postgres is shut down.  If `postmaster.pid` is present then `--force` is required to make the backup run (though if Postgres is running an inconsistent backup will likely be created).  This option was added primarily for the purpose of unit testing, but there may be applications in the real world as well.

* Fixed broken checksums and now they work with normal and resumed backups.  Finally realized that checksums and checksum deltas should be functionally separated and this simplied a number of things.  Issue #28 has been created for checksum deltas.

* Fixed an issue where a backup could be resumed from an aborted backup that didn't have the same type and prior backup.

* Removed dependency on Moose.  It wasn't being used extensively and makes for longer startup times.

* Checksum for backup.manifest to detect corrupted/modified manifest.

* Link `latest` always points to the last backup.  This has been added for convenience and to make restores simpler.

* More comprehensive unit tests in all areas.
2015-03-25 15:15:55 -04:00
David Steele
82fa497511 Last doc fixes. 2015-03-25 14:43:21 -04:00
David Steele
4a6f4e2340 Getting docs in shape for the next release. 2015-03-24 18:50:54 -04:00
David Steele
a998119cba Various option and unit test fixes. 2015-03-24 18:50:14 -04:00
David Steele
a3604bc512 Changed retention section to expire. 2015-03-24 11:43:37 -04:00
David Steele
58d6c37f4a Changed permission to mode in manifest. 2015-03-23 17:17:43 -04:00
David Steele
8a3c6388b2 Removed WAL from some option names - still waffling on this naming thing. 2015-03-23 16:01:15 -04:00
David Steele
66c474abe2 A few unit test fixes. 2015-03-23 15:45:25 -04:00
David Steele
1729fa1c51 Added unit tests for expiration. 2015-03-17 18:31:05 -04:00
David Steele
f4f31302f0 Broke archive-required into two options: backup-wal-check and backup-wal-store. 2015-03-16 14:45:53 -04:00
David Steele
258fb9c6e2 More work on automated docs. Merging this to go back to some feature/bug work for a while. 2015-03-16 14:01:01 -04:00
David Steele
882f068254 All config options can now be provided on the command-line.
Lots of name changes, but more thought needed there.
Still needs cleanup.
2015-03-12 12:15:19 -04:00
David Steele
7675a11ded First pass at building automated docs for markdown/html. This works pretty well, but the config sections of doc.xml still require too much maintenance. With the new Config code, it should be possible to generate those sections automatically. 2015-03-08 14:05:41 -04:00
David Steele
ae6bdecfaf Split command-line parameter processing out into a separate file. This is in preparation allowing all parameters to be specified/overridden on the command line, with pg_backrest.conf being option. 2015-03-08 13:26:09 -04:00