David Steele
7509b01e22
Added format to manifest to identify a manifest/dir structure version.
2015-03-03 21:21:07 -05:00
David Steele
3d0d308a98
Made all warnings FATAL.
...
Fixed one bug that was exposed.
2015-03-03 00:57:20 -05:00
David Steele
7dbb11e8b8
Added buffer-size param and added config validation improvements.
2015-03-02 22:58:32 -05:00
David Steele
8e8f2f3c77
New config options compress-level and compress-level-network to control compression levels.
2015-03-02 20:36:12 -05:00
David Steele
0df1b7c473
Changed file sizes are now detected and stored in the manifest.
...
Remove thread file minimums - they are unrealistic for a real db and hinder unit tests.
2015-03-01 22:43:32 -05:00
David Steele
76391dde90
Archive checksums are not calculated in stream.
2015-03-01 16:42:27 -05:00
David Steele
7ede058b45
Backup checksums are no longer optional.
2015-02-28 20:31:56 -05:00
David Steele
5d10a18b25
Added thread-max parameter to test.pl to allow backup threads to be specified.
2015-02-28 10:21:36 -05:00
David Steele
d2602a5c07
Tracking down a lockup in the restore threads. It doesn't happen in backup - they are the same except that restore uses the ThreadGroup object. I'm beginning to think that threads and objects don't play together very nicely. Objects in threads seems OK, but threads in objects, not so much.
2015-02-03 20:33:33 -05:00
David Steele
7bee43372d
Increase timeout.
2015-02-02 21:10:44 -05:00
David Steele
850c978240
All restore compares no-start-stop for now.
2015-02-02 19:56:48 -05:00
David Steele
bde8943517
Fixes to restore compare unit tests.
2015-02-02 18:33:11 -05:00
David Steele
8abcf6caf1
For no just ignore sizes in the tests since they are normal and do not indicate a bad backup.
2015-02-02 08:28:37 -05:00
David Steele
a6d3b7e1a9
Working on checking restores against the manifest. Current issue is that the manifest does not always record the final size of the file - it may change while the file is being copied. This is fine in principal but makes testing a pain.
2015-01-31 23:04:24 -05:00
David Steele
267efbd71d
Fixed restore::thread-max
2015-01-31 15:37:59 -05:00
David Steele
e524c4dd1a
Timeline unit tests are working.
...
Options from config file are being written to recovery.conf.
Fixed issue with .history files not being picked up by archive-xfer.
2015-01-31 13:48:09 -05:00
David Steele
11c257296a
In the end it was a single non-undefed reference holding up the show. The Backup file should be split into Archive, Backup, Expire, and made into objects. That would cut down on this kind of nastiness.
2015-01-30 20:16:21 -05:00
David Steele
fb934ecce9
Allow immediate stops when discarding data at end of unit test. Makes the shutdowns faster.
2015-01-30 14:54:08 -05:00
David Steele
19e455afc1
Some fixes in unit tests - more needed.
2015-01-28 16:25:38 -05:00
David Steele
499d6c8422
Added tests for all --delta --force combinations.
...
Added error check for postmaster running.
Added error check for path not empty.
2015-01-28 13:14:46 -05:00
David Steele
16dd271794
Added an incremental backup to vary the unit tests.
2015-01-28 11:12:21 -05:00
David Steele
139b1cf872
Fixed small race condition in cleanup - the archiver was recreating paths after they had been deleted. Put in a loop to make sure it gets done.
2015-01-28 10:29:29 -05:00
David Steele
60550cd45b
Archive async now works local, but there is a pending bug to make sure archive::path != backup::path.
...
Added code to be sure that restore will not try to log into the backup path unless it is local.
2015-01-28 00:28:21 -05:00
David Steele
a59bd8c328
Restores except for type=none are mostly working. There are some failing unit tests to fix.
2015-01-27 22:59:59 -05:00
David Steele
13544d51bf
Recovery is working for none and default, but unit tests are not complete.
2015-01-27 11:44:23 -05:00
David Steele
f59aae101d
Ready to start writing recovery unit tests.
2015-01-26 14:59:58 -05:00
David Steele
ac22c314a9
Working on plumbing for recovery options.
2015-01-26 11:01:24 -05:00
David Steele
6db49e1083
Cleaned up some time functions.
2015-01-25 12:20:12 -05:00
David Steele
9069c34e34
Added more tablespace changes to backup unit tests.
2015-01-24 11:05:26 -05:00
David Steele
e1c5f7f8ed
Now able to check for specific errors in unit tests - as long as they are raised with an error code.
2015-01-23 18:28:39 -05:00
David Steele
c6dad12748
Using usleep for more resolution,
2015-01-23 14:12:53 -05:00
David Steele
d3763908e7
Added longer sleep to show how big an issue this can be.
2015-01-22 21:20:49 -05:00
David Steele
a3b3f98f66
Fixed some issues with remote backups after restore (unit test bugs only).
2015-01-22 20:55:16 -05:00
David Steele
21c715b25c
Restore tablespace remaps are working.
2015-01-22 20:11:33 -05:00
David Steele
df4b50b977
Restore with base path remap working.
2015-01-22 19:04:55 -05:00
David Steele
82969eb497
Wrote some helper functions to make backup unit tests simpler.
...
Fixed an issue with checksum generation in resume.
2015-01-22 17:43:19 -05:00
David Steele
b53cb9f64e
Unit tests for backup resume.
...
Improved warning message when a backup cannot be resumed.
2015-01-22 10:54:02 -05:00
David Steele
fc2838d60f
All references to old-style manifest eliminated (hopefully).
2015-01-21 18:37:49 -05:00
David Steele
c33bc1e899
More migration to manifest object.
2015-01-20 18:00:03 -05:00
David Steele
92c9ed8963
Added checksum to manifest to detect corruption.
2015-01-20 16:05:15 -05:00
David Steele
84abcd7d61
Minor fixes.
2015-01-11 11:53:03 -05:00
David Steele
131d910906
New structure for --delta and --force params.
2015-01-08 15:43:43 -05:00
David Steele
d6d57e654e
Fixed the way wait was done after the manifest is created. Previously, waits were done for base and each tablespace which is not very efficient. Now one wait is done after the entire manifest is built. Also storing the exact time that copy began.
2015-01-06 13:08:56 -05:00
David Steele
43098086af
Implemented timestamp last modified to record the time of the last modified file in the backup. Also added timestamp-db-start and timestamp-db-stop to for more info. timestamp-db-start can be used for PITR.
2015-01-03 16:49:26 -05:00
David Steele
2449d7384c
Wrote a new test to check rsync collisions.
2015-01-03 12:56:44 -05:00
David Steele
2e080eedb8
Added an optional delay after manifest build so that files are not copied in the same second that the manifest is built. This can result in (admittedly unlikely) race conditions that can produce an invalid backup. I was also able to reduce the sleep types when waiting for thread termination - so unit test times are improved by almost 100%.
2015-01-02 14:18:07 -05:00
David Steele
32b37335a1
Trying to find realistic conditions where a file can be changed without the timestamp changing between backups. So far, this is the only case I can make work - it looks like adding a 1 second pause after creation of the manifest would cover this case.
2014-12-31 19:03:03 -05:00
David Steele
59e901684d
Working on improved backup/restore unit tests.
2014-12-31 14:59:33 -05:00
David Steele
9f8f33f957
Working on restore options and integration with unit tests.
2014-12-23 18:52:38 -05:00
David Steele
2db71e91ad
Basic restore functionality is working.
2014-12-22 11:24:32 -05:00
David Steele
9086728b73
Restore removes paths/files/links and recreated paths/links.
2014-12-21 10:11:17 -05:00
David Steele
1557327d2b
Restore cleans out missing files and some basic unit tests.
2014-12-19 17:49:56 +00:00
David Steele
1dc00c9d4b
Starting on restore functionality, finally.
2014-12-18 16:42:54 +00:00
David Steele
1a3c5cd9db
Added latest link so it is easy to get to the last backup.
2014-12-18 16:14:30 +00:00
David Steele
7d19504d3a
Removed some tests in full that were redundant with the tests in backup.
2014-12-18 14:58:07 +00:00
David Steele
6124558347
Moved config functions to a new file.
2014-12-15 22:20:42 +00:00
David Steele
6dd9e881dc
Added reminder.
2014-11-27 14:07:09 +00:00
David Steele
602db70d6a
More backup unit tests.
2014-10-20 11:44:36 -04:00
David Steele
9765b4e20b
Backup unit test - more work on tablespaces and references.
2014-10-19 17:41:11 -04:00
David Steele
66a5da6fd1
More work on backup unit tests - added tablespace tests.
2014-10-19 16:30:16 -04:00
David Steele
4de1d9aa5a
Backup unit tests are working.
2014-10-19 11:46:01 -04:00
David Steele
1d43c63326
Working on better backup unit tests.
2014-10-18 13:25:20 -04:00
David Steele
ed5777101f
Changes in how paths are removed.
2014-10-16 13:22:34 -04:00
David Steele
b6e0bb2ae5
Small unit test changes.
2014-10-16 08:58:11 -04:00
David Steele
5c216c93d5
Working on backup unit tests.
2014-10-15 19:11:09 -04:00
David Steele
b19dd9b552
Changes to the aborted backups code.
2014-10-14 15:44:50 -04:00
David Steele
3a54eaf9de
Last backup type is now checked - still need a unit test for it.
2014-10-14 08:27:50 -04:00
David Steele
2c173ba53e
Removed moose from File object.
2014-10-10 15:13:28 -04:00
David Steele
3d86adadb5
Remote object now using ProcessAsync for de/compression.
...
Removed dependency on Moose from Remote.
2014-10-08 13:54:31 -04:00
David Steele
122dc65351
Added compress and checksum settings to backup manifest.
2014-10-03 09:31:23 -04:00
David Steele
991afe3b16
Merge from dev-0.30.
...
Squashed commit of the following:
commit f3a012eb86702297ac06c9ae7c3da6a2eae41136
Author: David Steele <david@pgmasters.net>
Date: Wed Oct 1 19:05:57 2014 -0400
Fixed an issue where archive-get was not returning 1 if the archive directory did not exist. This happens when it is trying to retrieve archive log 00 and that has not been pushed from the master yet.
2014-10-02 10:55:38 -04:00
David Steele
7ed6587c42
ISSUE-11: Added --no-start-stop and --force options.
2014-09-19 17:51:51 -04:00
David Steele
06ad947102
Using ' instead of " where appropriate.
2014-09-16 11:22:55 -04:00
David Steele
4551a2d089
Docs and unit test fix.
2014-09-14 13:12:37 -04:00
David Steele
679005e537
Some cleanup of the diff/incr code.
2014-08-14 09:40:20 -04:00
David Steele
34cb57f925
Using checkpoint command instead of pg_switch_xlog.
2014-08-12 18:36:38 -04:00
David Steele
6b4a8fbc21
Unit tests now working on Ubuntu 12.04
2014-08-10 21:22:17 -04:00
David Steele
0298e8dd16
Created unit tests for files deleted by the db during backup.
2014-08-10 15:02:14 -04:00
David Steele
09e2351ae8
Test log mode is working.
2014-08-09 18:35:55 -04:00
David Steele
2d9f027e3a
Cleanup, work on test framework.
2014-07-27 18:13:23 -04:00
David Steele
f6e01de7ce
Improved backup::full unit test.
2014-07-16 20:28:47 -04:00
David Steele
5d39b4f646
Local backups are now completely local.
2014-07-16 20:07:50 -04:00
David Steele
9ad1d79f00
Unit tests for archive-get.
2014-07-16 19:15:56 -04:00
David Steele
9d9dae418b
Added archive-put unit tests.
...
Lots of other unit test improvements.
2014-07-15 22:32:41 -04:00
David Steele
8d398e4bff
Starting on archive unit tests.
2014-07-13 10:37:16 -04:00
David Steele
353cdb6b31
Local archiving is working again.
2014-07-12 19:03:39 -04:00
David Steele
e36a785b1b
Fixed execute function - now only displays output when there is an error.
2014-07-12 10:26:51 -04:00
David Steele
7f4c4e13d2
Back to Open3.
2014-07-02 19:15:21 -04:00
David Steele
fead7aa5f1
Not loving IPC::Run, but saving my work just in case.
2014-07-02 18:58:38 -04:00
David Steele
c85413ec68
Lots of improvements to unit tests. A few bug fixes.
2014-06-29 10:53:39 -04:00
David Steele
f9ec149ffe
Working on unit tests.
2014-06-28 14:32:34 -04:00
David Steele
97b9560e5c
Fixed binary_xfer() issue. Now seems to work in all cases.
2014-06-28 11:47:21 -04:00
David Steele
e1829bb2d2
First fully working backup in a while.
2014-06-23 18:54:00 -04:00
David Steele
88bbbb4222
Fighting with a relative path issue in archive logging.
2014-06-22 20:19:13 -04:00
David Steele
5d01aac92e
Working on backup - mostly working except for the last archive step.
2014-06-22 16:05:00 -04:00
David Steele
f91dce259f
Working on backup unit tests.
2014-06-22 14:51:28 -04:00
David Steele
8fff74ccfb
Backup unit test able to create and drop clusters.
2014-06-22 11:56:01 -04:00
David Steele
0b597d8da9
Working on getting backup running again.
2014-06-22 10:30:17 -04:00
