David Steele
d5335b40e8
Fix for issue #80 : enabling archive-copy causes failing differential & incremental backups
2015-05-22 14:49:14 -04:00
David Steele
7081c8b867
New model where threads are created early and destroyed late.
...
Backups now work like restores in terms of how jobs are queued.
Split out BackupFile and RestoreFile for easier multi-threading/processing.
2015-04-07 07:34:37 -04:00
David Steele
3f651a8ce8
Unit tests will now work across all installed versions of Postgres.
...
Created a function to list all supported versions. Now used for all version checking.
2015-04-02 22:07:23 -04:00
David Steele
984fceea67
Moved archive functions from pg_backrest.pl and Backup.pl to Archive.pm.
...
Moved Remote code from pg_backrest.pl to Config.pm.
Added version specific code to regression tests and Db.pm.
archive-push checks for duplicate WAL in the archive.
archive-push reads the db sys id to match up WAL to the correct archive.
2015-04-01 15:58:33 -04:00
David Steele
58d6c37f4a
Changed permission to mode in manifest.
2015-03-23 17:17:43 -04:00
David Steele
882f068254
All config options can now be provided on the command-line.
...
Lots of name changes, but more thought needed there.
Still needs cleanup.
2015-03-12 12:15:19 -04:00
David Steele
3d0d308a98
Made all warnings FATAL.
...
Fixed one bug that was exposed.
2015-03-03 00:57:20 -05:00
David Steele
c0ec62b131
Fixed bug in File->hash() and removed improper last statements from do loops.
2015-03-02 23:42:55 -05:00
David Steele
8e8f2f3c77
New config options compress-level and compress-level-network to control compression levels.
2015-03-02 20:36:12 -05:00
David Steele
d874e122aa
Optimized compression to disable headers/crc when the source and destination are both uncompressed.
...
Set different compression levels based on usage.
2015-03-01 23:41:01 -05:00
David Steele
23102f19e5
All copy() cases generate checksums during copy.
...
Unit tests for bAppendChecksum option for copy().
2015-03-01 22:23:52 -05:00
David Steele
76391dde90
Archive checksums are not calculated in stream.
2015-03-01 16:42:27 -05:00
David Steele
77bc4238dc
ZLib stuff starting to look good. All references removed from File and using binary_xfer for all de/compression.
2015-03-01 13:41:35 -05:00
David Steele
28326d6b4c
File->copy now returns hash and size in all cases, though the local copies are not optimal. They just call hash_size().
2015-02-28 19:07:29 -05:00
David Steele
260a6cb8f1
Working on calculating hash and file size in binary_xfer().
2015-02-28 15:54:49 -05:00
David Steele
0f94476a1f
Changed buffer sizes.
2015-02-28 12:10:50 -05:00
David Steele
d6205d9501
Looks like all unit tests pass - now for a long test run to see if that is really true. And to see if the old lockup is gone.
2015-02-27 23:31:39 -05:00
David Steele
25442655c8
Hash of compressed file is working.
...
Something still broken in binary_xfer because some 0 length archive files are showing up. Investigating.
2015-02-27 18:42:28 -05:00
David Steele
b9378d94a2
Working on removing threads from instream compress/decompression.
2015-02-26 09:22:05 -05:00
David Steele
50e015a838
Revert "Abortive attempt at cleaning up some thread issues - I realized the issue is in mixing threads and objects too liberally. Trying another approach but want to keep this code for historical and reference purposes."
...
This reverts commit e95631f82a
2015-01-30 18:58:49 -05:00
David Steele
e95631f82a
Abortive attempt at cleaning up some thread issues - I realized the issue is in mixing threads and objects too liberally. Trying another approach but want to keep this code for historical and reference purposes.
2015-01-30 14:55:55 -05:00
David Steele
13544d51bf
Recovery is working for none and default, but unit tests are not complete.
2015-01-27 11:44:23 -05:00
David Steele
425e9485aa
Re-added File-hash() - accidentally deleted it when working on owner().
2015-01-08 16:36:45 -05:00
David Steele
2e09df2cec
More work on new manifest object.
2015-01-08 12:04:56 -05:00
David Steele
0085e88966
Added File-owner()
2015-01-07 12:58:21 -05:00
David Steele
d6d57e654e
Fixed the way wait was done after the manifest is created. Previously, waits were done for base and each tablespace which is not very efficient. Now one wait is done after the entire manifest is built. Also storing the exact time that copy began.
2015-01-06 13:08:56 -05:00
David Steele
43098086af
Implemented timestamp last modified to record the time of the last modified file in the backup. Also added timestamp-db-start and timestamp-db-stop to for more info. timestamp-db-start can be used for PITR.
2015-01-03 16:49:26 -05:00
David Steele
91b06bef47
File->manifest detects files that are modified in the future.
2015-01-03 14:22:59 -05:00
David Steele
2e080eedb8
Added an optional delay after manifest build so that files are not copied in the same second that the manifest is built. This can result in (admittedly unlikely) race conditions that can produce an invalid backup. I was also able to reduce the sleep types when waiting for thread termination - so unit test times are improved by almost 100%.
2015-01-02 14:18:07 -05:00
David Steele
fb948de559
Fixed some minor issues in ownership code found by unit tests.
2014-12-31 12:28:16 -05:00
David Steele
7e083e8749
Restore as root is now working.
2014-12-31 11:20:46 -05:00
David Steele
0805055d4b
More work on restore ownership code.
2014-12-30 15:41:43 -05:00
David Steele
46ba3a18df
Ownership code runs through - may not work for all cases.
2014-12-30 11:59:57 -05:00
David Steele
9f8f33f957
Working on restore options and integration with unit tests.
2014-12-23 18:52:38 -05:00
David Steele
9086728b73
Restore removes paths/files/links and recreated paths/links.
2014-12-21 10:11:17 -05:00
David Steele
2c173ba53e
Removed moose from File object.
2014-10-10 15:13:28 -04:00
David Steele
4cbc6c650a
File->hash now works on compressed files.
2014-10-02 13:54:26 -04:00
David Steele
991afe3b16
Merge from dev-0.30.
...
Squashed commit of the following:
commit f3a012eb86702297ac06c9ae7c3da6a2eae41136
Author: David Steele <david@pgmasters.net>
Date: Wed Oct 1 19:05:57 2014 -0400
Fixed an issue where archive-get was not returning 1 if the archive directory did not exist. This happens when it is trying to retrieve archive log 00 and that has not been pushed from the master yet.
2014-10-02 10:55:38 -04:00
David Steele
168398e58a
Merge from dev-0.30.
...
Squashed commit of the following:
commit 2e90e165f5717ffef932996e00227dc78d47f23a
Author: David Steele <david@pgmasters.net>
Date: Tue Sep 30 15:50:07 2014 -0400
Removed outdated conf sample.
commit d32f7690ca99c98bc8cef5011de29f4b3a05bf90
Author: David Steele <david@pgmasters.net>
Date: Tue Sep 30 15:44:25 2014 -0400
Suppress error message from being logged when a file is removed by the db during the backup. The exception was being caught and the backup was successful, but the error message was being logged to stdout and the log file which is confusing.
commit 9ea52d9703da833aa3c215dbadfee60472ffad2a
Author: David Steele <david@pgmasters.net>
Date: Tue Sep 30 11:13:55 2014 -0400
File->manifest now ignores missing files and paths while running.
2014-09-30 15:51:08 -04:00
David Steele
bdbdaf39d3
Work in progress - new hash method.
2014-09-29 19:08:08 -04:00
David Steele
7b1039dbe2
Fixed error message.
2014-09-18 17:39:24 -04:00
David Steele
535f32c827
Removed old unit tests.
2014-09-18 17:24:40 -04:00
David Steele
48b0e0d461
Using ' instead of " where appropriate.
2014-09-16 08:15:49 -04:00
David Steele
0298e8dd16
Created unit tests for files deleted by the db during backup.
2014-08-10 15:02:14 -04:00
David Steele
d0ca69d433
Integrating IPC::Run - IPC::Open3 is out.
2014-06-30 18:35:05 -04:00
David Steele
934b7dbe9b
De/compression is now done in threads instead of forked processes.
2014-06-29 17:23:34 -04:00
David Steele
c85413ec68
Lots of improvements to unit tests. A few bug fixes.
2014-06-29 10:53:39 -04:00
David Steele
f9ec149ffe
Working on unit tests.
2014-06-28 14:32:34 -04:00
David Steele
97b9560e5c
Fixed binary_xfer() issue. Now seems to work in all cases.
2014-06-28 11:47:21 -04:00
David Steele
e1829bb2d2
First fully working backup in a while.
2014-06-23 18:54:00 -04:00
