David Steele
f1cc1d6123
Refactored backup manifest update by centralizing the code.
2015-04-11 15:02:04 -04:00
David Steele
b8ce891603
Altered rsync comparison test to use a specially crafted table that has a 70% compression ratio.
...
Improved the output of file_size_format().
2015-04-11 12:31:39 -04:00
David Steele
9fe7f87e3a
Allow any boolean option that is not command-line only to be negated.
2015-04-07 19:21:48 -04:00
David Steele
43d86e64a4
First pass at tests comparing rsync to backrest. Decent results, but room for improvement.
...
All tests local over SSH with rsync default compression, 4 threads and default compression on backrest. Backrest default is gzip = 6, assuming rsync is the same.
On a 1GB DB:
rsync time = 32.82
backrest time = 19.48
backrest is 171% faster.
On a 5GB DB:
rsync time = 171.16
backrest time = 86.97
backrest is 196% faster.
2015-04-07 18:36:59 -04:00
David Steele
7081c8b867
New model where threads are created early and destroyed late.
...
Backups now work like restores in terms of how jobs are queued.
Split out BackupFile and RestoreFile for easier multi-threading/processing.
2015-04-07 07:34:37 -04:00
David Steele
808c7863b6
Duplicate WAL issue in 9.3 was fixed in 9.3.6.
2015-04-03 09:56:03 -04:00
David Steele
3f651a8ce8
Unit tests will now work across all installed versions of Postgres.
...
Created a function to list all supported versions. Now used for all version checking.
2015-04-02 22:07:23 -04:00
David Steele
0b27c74999
Increased version to 0.60 and added release notes.
2015-04-01 16:28:16 -04:00
David Steele
984fceea67
Moved archive functions from pg_backrest.pl and Backup.pl to Archive.pm.
...
Moved Remote code from pg_backrest.pl to Config.pm.
Added version specific code to regression tests and Db.pm.
archive-push checks for duplicate WAL in the archive.
archive-push reads the db sys id to match up WAL to the correct archive.
2015-04-01 15:58:33 -04:00
David Steele
9caccc2ce2
Doc fixes.
2015-03-29 13:35:36 -04:00
David Steele
92a4d648ac
Fixed issues in collision unit tests.
2015-03-26 17:42:45 -04:00
David Steele
16cc2110c5
Merge branch 'master' into dev
2015-03-25 15:17:59 -04:00
David Steele
b37d59832f
v0.50: restore and much more
...
* Added restore functionality.
* All options can now be set on the command-line making pg_backrest.conf optional.
* De/compression is now performed without threads and checksum/size is calculated in stream. That means file checksums are no longer optional.
* Added option `--no-start-stop` to allow backups when Postgres is shut down. If `postmaster.pid` is present then `--force` is required to make the backup run (though if Postgres is running an inconsistent backup will likely be created). This option was added primarily for the purpose of unit testing, but there may be applications in the real world as well.
* Fixed broken checksums and now they work with normal and resumed backups. Finally realized that checksums and checksum deltas should be functionally separated and this simplied a number of things. Issue #28 has been created for checksum deltas.
* Fixed an issue where a backup could be resumed from an aborted backup that didn't have the same type and prior backup.
* Removed dependency on Moose. It wasn't being used extensively and makes for longer startup times.
* Checksum for backup.manifest to detect corrupted/modified manifest.
* Link `latest` always points to the last backup. This has been added for convenience and to make restores simpler.
* More comprehensive unit tests in all areas.
2015-03-25 15:15:55 -04:00
David Steele
82fa497511
Last doc fixes.
2015-03-25 14:43:21 -04:00
David Steele
4a6f4e2340
Getting docs in shape for the next release.
2015-03-24 18:50:54 -04:00
David Steele
a998119cba
Various option and unit test fixes.
2015-03-24 18:50:14 -04:00
David Steele
a3604bc512
Changed retention section to expire.
2015-03-24 11:43:37 -04:00
David Steele
58d6c37f4a
Changed permission to mode in manifest.
2015-03-23 17:17:43 -04:00
David Steele
8a3c6388b2
Removed WAL from some option names - still waffling on this naming thing.
2015-03-23 16:01:15 -04:00
David Steele
66c474abe2
A few unit test fixes.
2015-03-23 15:45:25 -04:00
David Steele
1729fa1c51
Added unit tests for expiration.
2015-03-17 18:31:05 -04:00
David Steele
f4f31302f0
Broke archive-required into two options: backup-wal-check and backup-wal-store.
2015-03-16 14:45:53 -04:00
David Steele
258fb9c6e2
More work on automated docs. Merging this to go back to some feature/bug work for a while.
2015-03-16 14:01:01 -04:00
David Steele
882f068254
All config options can now be provided on the command-line.
...
Lots of name changes, but more thought needed there.
Still needs cleanup.
2015-03-12 12:15:19 -04:00
David Steele
7675a11ded
First pass at building automated docs for markdown/html. This works pretty well, but the config sections of doc.xml still require too much maintenance. With the new Config code, it should be possible to generate those sections automatically.
2015-03-08 14:05:41 -04:00
David Steele
ae6bdecfaf
Split command-line parameter processing out into a separate file. This is in preparation allowing all parameters to be specified/overridden on the command line, with pg_backrest.conf being option.
2015-03-08 13:26:09 -04:00
David Steele
f115e01b71
More work on docs.
2015-03-04 22:38:27 -05:00
David Steele
8dc3ba184d
More work on docs.
2015-03-04 20:09:26 -05:00
David Steele
163f898262
HTML docs look to be in a workable format.
2015-03-04 18:50:02 -05:00
David Steele
bfb2c05357
Started on new HTML docs. The markdown is not cutting it.
2015-03-04 00:36:54 -05:00
David Steele
942b29d5b4
Added unit test for recovery type=preserve.
2015-03-03 23:15:40 -05:00
David Steele
d19baefdb9
Removed db-timestamp-start and stop from manifest. Better to get these values from
2015-03-03 22:33:41 -05:00
David Steele
7509b01e22
Added format to manifest to identify a manifest/dir structure version.
2015-03-03 21:21:07 -05:00
David Steele
f5761a41c1
Modules cleanup.
2015-03-03 10:03:55 -05:00
David Steele
3d0d308a98
Made all warnings FATAL.
...
Fixed one bug that was exposed.
2015-03-03 00:57:20 -05:00
David Steele
c0ec62b131
Fixed bug in File->hash() and removed improper last statements from do loops.
2015-03-02 23:42:55 -05:00
David Steele
7dbb11e8b8
Added buffer-size param and added config validation improvements.
2015-03-02 22:58:32 -05:00
David Steele
8e8f2f3c77
New config options compress-level and compress-level-network to control compression levels.
2015-03-02 20:36:12 -05:00
David Steele
49112eb6d3
Better compression window settings.
2015-03-02 09:46:11 -05:00
David Steele
d874e122aa
Optimized compression to disable headers/crc when the source and destination are both uncompressed.
...
Set different compression levels based on usage.
2015-03-01 23:41:01 -05:00
David Steele
0df1b7c473
Changed file sizes are now detected and stored in the manifest.
...
Remove thread file minimums - they are unrealistic for a real db and hinder unit tests.
2015-03-01 22:43:32 -05:00
David Steele
23102f19e5
All copy() cases generate checksums during copy.
...
Unit tests for bAppendChecksum option for copy().
2015-03-01 22:23:52 -05:00
David Steele
83658a4778
Docs for checksum option being removed.
2015-03-01 16:43:54 -05:00
David Steele
76391dde90
Archive checksums are not calculated in stream.
2015-03-01 16:42:27 -05:00
David Steele
393525d280
Full refactor of binary_xfer.
...
Compression now uses Compress::Raw::Zlib.
2015-03-01 15:17:57 -05:00
David Steele
77bc4238dc
ZLib stuff starting to look good. All references removed from File and using binary_xfer for all de/compression.
2015-03-01 13:41:35 -05:00
David Steele
7ede058b45
Backup checksums are no longer optional.
2015-02-28 20:31:56 -05:00
David Steele
28326d6b4c
File->copy now returns hash and size in all cases, though the local copies are not optimal. They just call hash_size().
2015-02-28 19:07:29 -05:00
David Steele
260a6cb8f1
Working on calculating hash and file size in binary_xfer().
2015-02-28 15:54:49 -05:00
David Steele
0f94476a1f
Changed buffer sizes.
2015-02-28 12:10:50 -05:00
