1
0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2024-12-12 10:04:14 +02:00
Commit Graph

3672 Commits

Author SHA1 Message Date
David Steele
85f591f801 More unit tests for file functions. 2014-05-27 09:00:24 -04:00
David Steele
3e12f9230b Working on unit tests for file_copy. Still need to add specific error tests, timestamp, and permissions. 2014-05-14 15:07:37 -04:00
David Steele
db40553434 Changed version. 2014-05-13 16:41:20 -04:00
David Steele
1fa8dbb778 v0.19: Improved error reporting/handling
* Working on improving error handling in the file object.  This is not complete, but works well enough to find a few errors that have been causing us problems (notably, find is occasionally failing building the archive async manifest when system is under load).

* Found and squashed a nasty bug where file_copy was defaulted to ignore errors.  There was also an issue in file_exists that was causing the test to fail when the file actually did exist.  Together they could have resulted in a corrupt backup with no errors, though it is very unlikely.
2014-05-13 11:47:14 -04:00
David Steele
d234aeea64 More work on error-handling. 2014-05-13 11:23:15 -04:00
David Steele
a2b28623d4 Working on better error handling. 2014-04-28 09:13:25 -04:00
David Steele
53fa574d7e Merge branch 'master' of github.com:dwsteele/pg_backrest into dev 2014-04-13 11:58:35 -04:00
David Steele
9acbba173e v0.18: Return soft error from archive-get when file is missing
* The archive-get function returns a 1 when the archive file is missing to differentiate from hard errors (ssh connection failure, file copy error, etc.)  This lets Postgres know that that the archive stream has terminated normally.  However, this does not take into account possible holes in the archive stream.
2014-04-13 09:51:51 -04:00
David Steele
d8c488f73f Added the feature backlog. 2014-04-11 14:15:56 -04:00
David Steele
06b8eefd19 Fixed up README 2014-04-03 18:20:46 -04:00
David Steele
7aa6c8ecaf v0.17: Warn when archive directories cannot be deleted
* If an archive directory which should be empty could not be deleted backrest was throwing an error.  There's a good fix for that coming, but for the time being it has been changed to a warning so processing can continue.  This was impacting backups as sometimes the final archive file would not get pushed if the first archive file had been in a different directory (plus some bad luck).
2014-04-03 18:17:40 -04:00
David Steele
49536498e6 Warning for non-empty dirs instead of error 2014-04-03 16:52:39 -04:00
David Steele
194696f674 Working on unit tests. 2014-04-02 17:25:37 -04:00
David Steele
9a5bb36c0f Added notes. 2014-04-01 21:41:34 -04:00
David Steele
be1163b948 v0.16: RequestTTY=yes for SSH sessions
* Added RequestTTY=yes to ssh sesssions.  Hoping this will prevent random lockups.
2014-04-01 10:49:37 -04:00
David Steele
235c75b1b0 Added ; 2014-04-01 09:01:34 -04:00
David Steele
7624d65edf Added TTY code. 2014-04-01 08:59:09 -04:00
David Steele
2b8972936f Updates to README. 2014-03-30 21:02:17 -04:00
David Steele
765af10c88 Merge branch 'master' of github.com:dwsteele/pg_backrest into dev 2014-03-29 18:18:03 -04:00
David Steele
dafaeb0b82 v0.15: Added archive-get
* Added archive-get functionality to aid in restores.

* Added option to force a checkpoint when starting the backup (start_fast=y).
2014-03-29 18:16:08 -04:00
David Steele
ba2590dc8e archive-get function is working.
Added option to force checkpoint at backup start.
2014-03-28 15:20:36 -04:00
David Steele
8ae6c2719d Better code in file_list_get(). 2014-03-28 11:59:44 -04:00
David Steele
23df43e781 file_list_get now works locally and remotely
Conflicts:
	README.md
	pg_backrest.pl
	pg_backrest_db.pm
	pg_backrest_file.pm
2014-03-28 11:40:37 -04:00
David Steele
9d949b7ad6 v0.11: Minor fixes
Tweaking a few settings after running backups for about a month.

* Removed master_stderr_discard option on database SSH connections.  There have been occasional lockups and they could be related issues originally seen in the file code.

* Changed lock file conflicts on backup and expire commands to ERROR.  They were set to DEBUG due to a copy-and-paste from the archive locks.
2014-03-26 10:53:49 -04:00
David Steele
126b39855d Working on archive-get 2014-03-25 18:56:05 -04:00
David Steele
e68f836b2e Documented enhancements planned for v0.15 2014-03-08 09:12:53 -05:00
David Steele
bbb3b48cf8 Added comments. 2014-03-05 20:06:13 -05:00
David Steele
587bd1f8d9 v0.10: Backup and archiving are functional
This version has been put into production at Resonate, so it does work, but there are a number of major caveats.

* No restore functionality, but the backup directories are consistent Postgres data directories.  You'll need to either uncompress the files or turn off compression in the backup.  Uncompressed backups on a ZFS (or similar) filesystem are a good option because backups can be restored locally via a snapshot to create logical backups or do spot data recovery.

* Archiving is single-threaded.  This has not posed an issue on our multi-terabyte databases with heavy write volume.  Recommend a large WAL volume or to use the async option with a large volume nearby.

* Backups are multi-threaded, but the Net::OpenSSH library does not appear to be 100% threadsafe so it will very occasionally lock up on a thread.  There is an overall process timeout that resolves this issue by killing the process.  Yes, very ugly.

* Checksums are lost on any resumed backup. Only the final backup will record checksum on multiple resumes.  Checksums from previous backups are correctly recorded and a full backup will reset everything.

* The backup.manifest is being written as Storable because Config::IniFile does not seem to handle large files well.  Would definitely like to save these as human-readable text.

* Absolutely no documentation (outside the code).  Well, excepting these release notes.

* Lots of other little things and not so little things.  Much refactoring to follow.
2014-03-05 19:53:13 -05:00
David Steele
1617afb704 Initial commit. 2014-03-05 19:51:03 -05:00
David Steele
1a4f2527e3 Fixed version number. 2014-03-05 19:49:05 -05:00
David Steele
ae30ec98e9 First release merge from master. 2014-03-05 19:47:05 -05:00
David Steele
81616e19bd v0.01: Backup and archiving are functional
This version has been put into production at Resonate, so it does work, but there are a number of major caveats.

* No restore functionality, but the backup directories are consistent Postgres data directories.  You'll need to either uncompress the files or turn off compression in the backup.  Uncompressed backups on a ZFS (or similar) filesystem are a good option because backups can be restored locally via a snapshot to create logical backups or do spot data recovery.

* Archiving is single-threaded.  This has not posed an issue on our multi-terabyte databases with heavy write volume.  Recommend a large WAL volume or to use the async option with a large volume nearby.

* Backups are multi-threaded, but the Net::OpenSSH library does not appear to be 100% threadsafe so it will very occasionally lock up on a thread.  There is an overall process timeout that resolves this issue by killing the process.  Yes, very ugly.

* Checksums are lost on any resumed backup. Only the final backup will record checksum on multiple resumes.  Checksums from previous backups are correctly recorded and a full backup will reset everything.

* The backup.manifest is being written as Storable because Config::IniFile does not seem to handle large files well.  Would definitely like to save these as human-readable text.

* Absolutely no documentation (outside the code).  Well, excepting these release notes.

* Lots of other little things and not so little things.  Much refactoring to follow.
2014-03-05 19:45:01 -05:00
David Steele
951419178b Initial commit. 2014-03-05 19:43:23 -05:00
David Steele
2b55b8ca5e Merge branch 'master' of github.com:dwsteele/pg_backrest into dev 2014-03-05 19:10:23 -05:00
David Steele
1eb8e1be2d v0.01: Backup and archiving are functional
This version has been put into production at Resonate, so it does work, but there are a number of major caveats.

* No restore functionality, but the backup directories are consistent Postgres data directories.  You'll need to either uncompress the files or turn off compression in the backup.  Uncompressed backups on a ZFS (or similar) filesystem are a good option because backups can be restored locally via a snapshot to create logical backups or do spot data recovery.

* Archiving is single-threaded.  This has not posed an issue on our multi-terabyte databases with heavy write volume.  Recommend a large WAL volume or to use the async option with a large volume nearby.

* Backups are multi-threaded, but the Net::OpenSSH library does not appear to be 100% threadsafe so it will very occasionally lock up on a thread.  There is an overall process timeout that resolves this issue by killing the process.  Yes, very ugly.

* Checksums are lost on any resumed backup. Only the final backup will record checksum on multiple resumes.  Checksums from previous backups are correctly recorded and a full backup will reset everything.

* The backup.manifest is being written as Storable because Config::IniFile does not seem to handle large files well.  Would definitely like to save these as human-readable text.

* Absolutely no documentation (outside the code).  Well, excepting these release notes.

* Lots of other little things and not so little things.  Much refactoring to follow.
2014-03-05 18:56:15 -05:00
David Steele
80d5332fa0 Squashed commit of the following:
commit f0d82151b9
Author: David Steele <github@thelabyrinth.net>
Date:   Mon Mar 3 16:38:01 2014 -0500

    Changed thread timeout from WARN to ERROR

commit 571d449717
Author: David Steele <github@thelabyrinth.net>
Date:   Thu Feb 27 19:15:00 2014 -0500

    Redirect find error output to /dev/null.  Sometimes files are removed from the db while find is running.  We only want to error if the find process errors.

commit 84c4cec257
Author: David Steele <github@thelabyrinth.net>
Date:   Thu Feb 27 13:00:41 2014 -0500

    Put redirects in common places where errors are be output but we don't want to see them.

commit ae782d0884
Author: David Steele <github@thelabyrinth.net>
Date:   Tue Feb 25 07:30:45 2014 -0500

    Made archiving single-threaded so it will work

commit a84478045e
Author: David Steele <github@thelabyrinth.net>
Date:   Mon Feb 24 18:04:42 2014 -0500

    No attempt to kill threads - just abort

commit da41eb3883
Author: David Steele <github@thelabyrinth.net>
Date:   Mon Feb 24 17:59:42 2014 -0500

    Added more warnings

commit 21b15b7cf6
Author: David Steele <github@thelabyrinth.net>
Date:   Mon Feb 24 09:33:47 2014 -0500

    Added thread timeout.

commit 5de8f275f0
Author: David Steele <github@thelabyrinth.net>
Date:   Sat Feb 22 11:14:01 2014 -0500

    Removed stderr redirects from the master ssh connection

commit 0e4f0f1308
Author: David Steele <github@thelabyrinth.net>
Date:   Sat Feb 22 10:54:02 2014 -0500

    Fixed issue with archive path expiration

commit e8c7d7aca6
Author: David Steele <github@thelabyrinth.net>
Date:   Sat Feb 22 09:05:58 2014 -0500

    When archive_retention_type is full, use the oldest backup if there are fewer than required

commit 9270a67892
Author: David Steele <github@thelabyrinth.net>
Date:   Fri Feb 21 16:09:34 2014 -0500

    Removed some more stderr redirects

commit 6d975d2830
Author: David Steele <github@thelabyrinth.net>
Date:   Fri Feb 21 08:49:27 2014 -0500

    No longer discarding stderr

commit 0387a8ee09
Author: David Steele <github@thelabyrinth.net>
Date:   Fri Feb 21 07:34:17 2014 -0500

    replaced process id with thread id.  Added use thread to all modules.

commit ac3ce81621
Author: David Steele <github@thelabyrinth.net>
Date:   Fri Feb 21 06:42:42 2014 -0500

    Add process id to logging, don't generate checksum for 0 length files

commit 3d8e933343
Author: David Steele <github@thelabyrinth.net>
Date:   Wed Feb 19 16:06:21 2014 -0500

    More threads

commit 857e177429
Author: David Steele <github@thelabyrinth.net>
Date:   Wed Feb 19 15:13:07 2014 -0500

    Using separate master queues

commit 800cd89831
Author: David Steele <github@thelabyrinth.net>
Date:   Wed Feb 19 13:37:18 2014 -0500

    Thread files are no longer in an array

commit 81433c0fdd
Author: David Steele <github@thelabyrinth.net>
Date:   Wed Feb 19 13:03:52 2014 -0500

    Now using one control master for threading.

commit b4ac9552d2
Author: David Steele <github@thelabyrinth.net>
Date:   Tue Feb 18 19:29:57 2014 -0500

    Skipped files removed from manifest, checksums stored

commit 27b820cfb4
Author: David Steele <github@thelabyrinth.net>
Date:   Tue Feb 18 15:42:51 2014 -0500

    Backup handles files that were removed by the db

commit 187fdaf49b
Author: David Steele <github@thelabyrinth.net>
Date:   Sun Feb 16 19:01:06 2014 -0500

    Better compression choices.

commit 5c695a6522
Author: David Steele <github@thelabyrinth.net>
Date:   Sun Feb 16 18:20:55 2014 -0500

    Improved archive logging and put in max limit

commit 13086e9626
Author: David Steele <github@thelabyrinth.net>
Date:   Sun Feb 16 17:02:25 2014 -0500

    Renamed processes.

commit eb25dbdb9d
Author: David Steele <github@thelabyrinth.net>
Date:   Sun Feb 16 16:58:25 2014 -0500

    Added stanza to command string

commit b45d9c946e
Author: David Steele <github@thelabyrinth.net>
Date:   Sun Feb 16 16:51:05 2014 -0500

    Improved command string

commit 6d7f89fc98
Author: David Steele <github@thelabyrinth.net>
Date:   Sun Feb 16 01:31:24 2014 -0500

    change process string

commit 577c5af33f
Author: David Steele <github@thelabyrinth.net>
Date:   Sun Feb 16 01:24:54 2014 -0500

    testing

commit b514058cee
Author: David Steele <github@thelabyrinth.net>
Date:   Sat Feb 15 16:46:13 2014 -0500

    Changed to perl storable - at least it works

commit 392da91620
Author: David Steele <github@thelabyrinth.net>
Date:   Sat Feb 15 15:28:22 2014 -0500

    Remove old local archive dirs.

commit 0bc40d206c
Author: David Steele <github@thelabyrinth.net>
Date:   Sat Feb 15 14:33:31 2014 -0500

    Some fixes for config

commit 0979841f1a
Author: David Steele <github@thelabyrinth.net>
Date:   Sat Feb 15 14:18:15 2014 -0500

    Async compress, thread kill improvements

commit bd00fb7f0c
Author: David Steele <github@thelabyrinth.net>
Date:   Sat Feb 15 10:45:01 2014 -0500

    Still working on thread kill

commit 3b2dd68aac
Author: David Steele <github@thelabyrinth.net>
Date:   Sat Feb 15 10:31:57 2014 -0500

    More robust join code.

commit 27bf9b6572
Author: David Steele <github@thelabyrinth.net>
Date:   Sat Feb 15 10:09:47 2014 -0500

    First SSH session is shared between objects and thread 0

commit c7aebf05c4
Author: David Steele <github@thelabyrinth.net>
Date:   Fri Feb 14 21:39:18 2014 -0500

    Handle term signals gracefully.

commit d100294894
Author: David Steele <github@thelabyrinth.net>
Date:   Fri Feb 14 19:56:28 2014 -0500

    Ability to resume failed backups, better locking

commit 308652cc65
Author: David Steele <github@thelabyrinth.net>
Date:   Fri Feb 14 09:05:14 2014 -0500

    Remove a lot of path creates - percent logging.

commit e14d52c42a
Author: David Steele <github@thelabyrinth.net>
Date:   Fri Feb 14 08:23:25 2014 -0500

    Improved backup threading

commit e93a444a41
Author: David Steele <github@thelabyrinth.net>
Date:   Thu Feb 13 23:37:52 2014 -0500

    Changes to manifest.

commit 5c393a0df9
Author: David Steele <github@thelabyrinth.net>
Date:   Thu Feb 13 21:44:19 2014 -0500

    Increased time to wait for log files

commit dccdec2ffa
Author: David Steele <github@thelabyrinth.net>
Date:   Thu Feb 13 21:41:20 2014 -0500

    trying to make labels stick

commit ae3dcc57f0
Author: David Steele <github@thelabyrinth.net>
Date:   Thu Feb 13 21:31:57 2014 -0500

    Thread errors now exit

commit 6d9e7db1c6
Author: David Steele <github@thelabyrinth.net>
Date:   Thu Feb 13 20:54:41 2014 -0500

    Increased thread total.

commit 405d36497d
Author: David Steele <github@thelabyrinth.net>
Date:   Thu Feb 13 20:33:05 2014 -0500

    Less logging

commit a78d677b1e
Author: David Steele <github@thelabyrinth.net>
Date:   Thu Feb 13 20:29:42 2014 -0500

    Wait for archive files.

commit d4ba078183
Author: David Steele <github@thelabyrinth.net>
Date:   Thu Feb 13 18:14:15 2014 -0500

    Only save manifest once

commit 1bec36b65b
Author: David Steele <github@thelabyrinth.net>
Date:   Thu Feb 13 18:07:58 2014 -0500

    Better thread logging.

commit 2bafa5cf26
Author: David Steele <github@thelabyrinth.net>
Date:   Thu Feb 13 17:52:38 2014 -0500

    Logging tweak

commit 8453681ccd
Author: David Steele <github@thelabyrinth.net>
Date:   Thu Feb 13 17:50:14 2014 -0500

    Logging tweaks

commit 4349f0caff
Author: David Steele <github@thelabyrinth.net>
Date:   Thu Feb 13 17:47:53 2014 -0500

    Fixes to log level and more logging

commit 7d55ce865b
Author: David Steele <github@thelabyrinth.net>
Date:   Thu Feb 13 17:11:53 2014 -0500

    Work on log levels

commit 133e27d813
Author: David Steele <github@thelabyrinth.net>
Date:   Thu Feb 13 15:26:07 2014 -0500

    Archive tmp file fix.

commit 19f9a5193f
Author: David Steele <github@thelabyrinth.net>
Date:   Thu Feb 13 14:54:28 2014 -0500

    Minor config changes.

commit 13a1f0cf24
Author: David Steele <github@thelabyrinth.net>
Date:   Thu Feb 13 13:54:43 2014 -0500

    Lots and lots of logging improvements

commit 14b13d4840
Author: David Steele <github@thelabyrinth.net>
Date:   Thu Feb 13 12:02:45 2014 -0500

    Async archive log backup working.

commit f0df759bb9
Author: David Steele <github@thelabyrinth.net>
Date:   Thu Feb 13 00:21:25 2014 -0500

    Small debug fix

commit 7c4d463b1d
Author: David Steele <github@thelabyrinth.net>
Date:   Wed Feb 12 23:49:40 2014 -0500

    More balanced backup.

commit b770c8ebec
Author: David Steele <github@thelabyrinth.net>
Date:   Wed Feb 12 22:33:09 2014 -0500

    More work on archive-pull

commit 8e60a3111f
Author: David Steele <github@thelabyrinth.net>
Date:   Wed Feb 12 20:28:27 2014 -0500

    Lots of archive-pull plumbing

commit 16df430b73
Author: David Steele <github@thelabyrinth.net>
Date:   Wed Feb 12 17:34:34 2014 -0500

    New constants and basic fork.

commit 564bd65b3e
Author: David Steele <github@thelabyrinth.net>
Date:   Wed Feb 12 16:01:48 2014 -0500

    Basic plumbing for archive-pull function

commit 5181282b74
Author: David Steele <github@thelabyrinth.net>
Date:   Wed Feb 12 08:04:42 2014 -0500

    Command line parameter checks

commit 729b07fed5
Author: David Steele <github@thelabyrinth.net>
Date:   Wed Feb 12 07:45:29 2014 -0500

    Hardlink now a conf option.

commit 219120ba81
Author: David Steele <github@thelabyrinth.net>
Date:   Wed Feb 12 07:39:42 2014 -0500

    Archiving and flag changes.

commit b9a2e7093e
Author: David Steele <github@thelabyrinth.net>
Date:   Tue Feb 11 21:17:30 2014 -0500

    Some more file debug info

commit 1a9f73b781
Author: David Steele <github@thelabyrinth.net>
Date:   Tue Feb 11 17:07:49 2014 -0500

    Added timestamp to log

commit 49dbc897a6
Author: David Steele <github@thelabyrinth.net>
Date:   Tue Feb 11 16:57:51 2014 -0500

    Small file allocation improvements.

commit 9a1de327a1
Author: David Steele <github@thelabyrinth.net>
Date:   Tue Feb 11 16:37:20 2014 -0500

    Turned ssh compression on

commit 71812fca99
Author: David Steele <github@thelabyrinth.net>
Date:   Tue Feb 11 16:21:33 2014 -0500

    Thread total is defined in the conf file.

commit 171ca709d6
Author: David Steele <github@thelabyrinth.net>
Date:   Tue Feb 11 16:08:55 2014 -0500

    Better file ordering for threads.

commit c58ccb588e
Author: David Steele <github@thelabyrinth.net>
Date:   Tue Feb 11 15:31:16 2014 -0500

    Parallel backup now working

commit a55b37efc3
Author: David Steele <github@thelabyrinth.net>
Date:   Fri Feb 7 19:34:14 2014 -0500

    Working on parallel backup

commit 820791cf92
Author: David Steele <github@thelabyrinth.net>
Date:   Thu Feb 6 17:31:33 2014 -0500

    FF log is skipped for pre-9.3 databases.

commit 60b2be58ac
Author: David Steele <github@thelabyrinth.net>
Date:   Thu Feb 6 16:37:37 2014 -0500

    Finished db object abstraction

commit f20f3b3ee6
Author: David Steele <github@thelabyrinth.net>
Date:   Thu Feb 6 15:54:22 2014 -0500

    Started db object abstraction.

commit 9d08f2a644
Author: David Steele <github@thelabyrinth.net>
Date:   Thu Feb 6 12:49:54 2014 -0500

    More robust config.  Retention is read from config.

commit a2b0d7a674
Author: David Steele <github@thelabyrinth.net>
Date:   Wed Feb 5 22:26:10 2014 -0500

    Backup expiration working again.  Other changes.

commit 27986e0c10
Author: David Steele <github@thelabyrinth.net>
Date:   Wed Feb 5 21:39:08 2014 -0500

    File is now an object.

commit 3e7faf0e5f
Author: David Steele <github@thelabyrinth.net>
Date:   Wed Feb 5 16:53:25 2014 -0500

    Some path fixes.

commit 7bee0f3141
Author: David Steele <github@thelabyrinth.net>
Date:   Wed Feb 5 15:56:05 2014 -0500

    Moved backup location.

commit a15fcb8c23
Author: David Steele <github@thelabyrinth.net>
Date:   Wed Feb 5 13:10:36 2014 -0500

    Moved tmp path.

commit 7ffade2453
Author: David Steele <github@thelabyrinth.net>
Date:   Wed Feb 5 11:35:09 2014 -0500

    Fixed file mod time and permissions

commit c86a61ca3e
Author: David Steele <github@thelabyrinth.net>
Date:   Wed Feb 5 10:40:49 2014 -0500

    Relative links working for tablespaces.

commit 12a960f91a
Author: David Steele <github@thelabyrinth.net>
Date:   Wed Feb 5 08:21:27 2014 -0500

    File copy supports decompression

commit a810499958
Author: David Steele <github@thelabyrinth.net>
Date:   Tue Feb 4 18:48:39 2014 -0500

    Remote backup fully working.

commit e38c482a26
Author: David Steele <github@thelabyrinth.net>
Date:   Mon Feb 3 20:48:02 2014 -0500

    Cleanup

commit 946b8261cf
Author: David Steele <github@thelabyrinth.net>
Date:   Mon Feb 3 20:29:03 2014 -0500

    Moved archive_push function

commit d5832a70c0
Author: David Steele <github@thelabyrinth.net>
Date:   Mon Feb 3 20:23:04 2014 -0500

    Moved last backup function.

commit aef6730267
Author: David Steele <github@thelabyrinth.net>
Date:   Mon Feb 3 19:57:21 2014 -0500

    Moved backup functions.

commit 316a5b5598
Author: David Steele <github@thelabyrinth.net>
Date:   Mon Feb 3 19:03:17 2014 -0500

    Working on remote

commit fa668d5007
Author: David Steele <github@thelabyrinth.net>
Date:   Mon Feb 3 14:50:23 2014 -0500

    Move file init code to file module

commit aef4adf1f3
Author: David Steele <github@thelabyrinth.net>
Date:   Sun Feb 2 19:03:05 2014 -0500

    Splitting main perl file

commit 4c2e4f7e49
Author: David Steele <github@thelabyrinth.net>
Date:   Sun Feb 2 12:45:59 2014 -0500

    Lots more work on remote functionality

commit bf8ac7abe6
Author: David Steele <github@thelabyrinth.net>
Date:   Sun Feb 2 11:12:55 2014 -0500

    manifest_get() now works remotely

commit 4556de4722
Author: David Steele <github@thelabyrinth.net>
Date:   Sun Feb 2 10:40:05 2014 -0500

    Source and destination can now be remote.

commit de972ef5d3
Author: David Steele <github@thelabyrinth.net>
Date:   Sat Feb 1 12:57:27 2014 -0500

    More work on file_copy

commit 52ba8801ef
Author: David Steele <github@thelabyrinth.net>
Date:   Sat Feb 1 11:04:33 2014 -0500

    Working on remote file copy

commit 2ad7a3d815
Author: David Steele <github@thelabyrinth.net>
Date:   Fri Jan 31 00:04:08 2014 -0500

    Formatting.

commit c5de8fa6e6
Author: David Steele <github@thelabyrinth.net>
Date:   Thu Jan 30 23:58:26 2014 -0500

    Created variable for compression extension

commit 151ff8ba60
Author: David Steele <github@thelabyrinth.net>
Date:   Thu Jan 30 23:24:34 2014 -0500

    Removed unused strPath param from path_get()

commit 52004d3c84
Author: David Steele <github@thelabyrinth.net>
Date:   Thu Jan 30 23:16:19 2014 -0500

    Comments

commit 447d83fcdc
Author: David Steele <github@thelabyrinth.net>
Date:   Thu Jan 30 23:09:34 2014 -0500

    Removing more copies.

commit f86618377c
Author: David Steele <github@thelabyrinth.net>
Date:   Thu Jan 30 11:35:34 2014 -0500

    Lots of backup refactoring.

commit e13a706c0a
Author: David Steele <github@thelabyrinth.net>
Date:   Wed Jan 29 15:38:57 2014 -0500

    More moving.  Links now also get references.

commit 54c73b3813
Author: David Steele <github@thelabyrinth.net>
Date:   Wed Jan 29 15:03:57 2014 -0500

    Moving stuff around

commit c26b47d4c7
Author: David Steele <github@thelabyrinth.net>
Date:   Wed Jan 29 12:06:30 2014 -0500

    Abstracting file functions.

commit 47c3c82540
Author: David Steele <github@thelabyrinth.net>
Date:   Wed Jan 29 06:40:15 2014 -0500

    Fixed checksum function.

commit 12f6d889de
Author: David Steele <github@thelabyrinth.net>
Date:   Tue Jan 28 17:16:07 2014 -0500

    More modular file copy.

commit 6625ea1405
Author: David Steele <github@thelabyrinth.net>
Date:   Thu Jan 23 19:39:51 2014 -0500

    Coded paths needed for archive logging

commit cdd1ef7306
Author: David Steele <github@thelabyrinth.net>
Date:   Wed Jan 22 21:40:56 2014 -0500

    Added general path function

commit 161de23c3b
Author: David Steele <github@thelabyrinth.net>
Date:   Wed Jan 22 19:03:20 2014 -0500

    Starting new structure

commit 89b96954f6
Author: David Steele <github@thelabyrinth.net>
Date:   Tue Jan 21 22:02:14 2014 -0500

    Allow any archive file start starts with 24 hex characters.

commit 7e73cf22cf
Author: David Steele <github@thelabyrinth.net>
Date:   Tue Jan 21 21:59:54 2014 -0500

    Changed where .backup files were copied

commit 62ae46e213
Author: David Steele <github@thelabyrinth.net>
Date:   Tue Jan 21 21:12:24 2014 -0500

    Archive expiration working.

commit a83e94ba27
Author: David Steele <github@thelabyrinth.net>
Date:   Tue Jan 21 20:38:58 2014 -0500

    Fixed error in the backup regexp

commit faaaa5690f
Author: David Steele <github@thelabyrinth.net>
Date:   Mon Jan 20 20:09:50 2014 -0500

    Working on archive retention.

commit df7ada85c2
Author: David Steele <github@thelabyrinth.net>
Date:   Sat Jan 18 10:14:35 2014 -0500

    Working on expiration

commit 1e0f2485db
Author: David Steele <github@thelabyrinth.net>
Date:   Sat Jan 18 00:03:32 2014 -0500

    Working on expiration

commit 3719d54cc9
Author: David Steele <github@thelabyrinth.net>
Date:   Wed Jan 15 20:41:29 2014 -0500

    Some notes. removed dead .backup move

commit a7de734105
Author: David Steele <github@thelabyrinth.net>
Date:   Sun Jan 12 20:12:49 2014 -0500

    Some cleanup - fixed tablespace path regression.

commit 05dcbc2246
Author: David Steele <github@thelabyrinth.net>
Date:   Sun Jan 12 15:28:19 2014 -0500

    Working on making backups consistent

commit c1e5904880
Author: David Steele <github@thelabyrinth.net>
Date:   Sun Jan 12 01:46:27 2014 -0500

    Fixed xlog copy - working on correctness

commit 3ed83e78f2
Author: David Steele <github@thelabyrinth.net>
Date:   Sat Jan 11 02:16:14 2014 -0500

    Working on archive log copy

commit 387439ac8e
Author: David Steele <github@thelabyrinth.net>
Date:   Fri Jan 10 16:01:25 2014 -0500

    Working on processing archive logs

commit e23820098d
Author: David Steele <github@thelabyrinth.net>
Date:   Fri Jan 10 14:02:56 2014 -0500

    Notices are not emitted on start and stop backup

commit 8b73a26b84
Author: David Steele <github@thelabyrinth.net>
Date:   Fri Jan 10 00:49:19 2014 -0500

    Some notes for things to do.

commit 49c1fcd3eb
Author: David Steele <github@thelabyrinth.net>
Date:   Fri Jan 10 00:43:26 2014 -0500

    Backup file is being copied to pg_xlog

commit 8e966c50dd
Author: David Steele <github@thelabyrinth.net>
Date:   Fri Jan 10 00:22:51 2014 -0500

    Removed commented code

commit b5f965942b
Author: David Steele <github@thelabyrinth.net>
Date:   Fri Jan 10 00:21:46 2014 -0500

    Added optional hard-linking.

commit cceac444ad
Author: David Steele <github@thelabyrinth.net>
Date:   Thu Jan 9 21:55:09 2014 -0500

    Added comment

commit c47ec3d260
Author: David Steele <github@thelabyrinth.net>
Date:   Thu Jan 9 21:40:33 2014 -0500

    All backup references are now stored.

commit ed5f8226a4
Author: David Steele <github@thelabyrinth.net>
Date:   Thu Jan 9 20:12:46 2014 -0500

    References are from oldest backup

commit bb8fb5b54f
Author: David Steele <github@thelabyrinth.net>
Date:   Thu Jan 9 20:00:34 2014 -0500

    Sparse backup dirs implemented

commit 854972dde6
Author: David Steele <github@thelabyrinth.net>
Date:   Thu Jan 9 18:23:50 2014 -0500

    Basic deltas work.

commit c89984cd59
Author: David Steele <github@thelabyrinth.net>
Date:   Thu Jan 9 18:02:42 2014 -0500

    Revert 2914a6d..e31b21f

    This rolls back to commit 2914a6dcc6.

commit e31b21f95a
Author: David Steele <github@thelabyrinth.net>
Date:   Thu Jan 9 18:01:34 2014 -0500

    Revert 52d8697..e0be213

    This rolls back to commit 52d8697118.

commit e0be213206
Author: David Steele <github@thelabyrinth.net>
Date:   Thu Jan 9 18:01:12 2014 -0500

    Revert 2914a6d..9633e17

    This rolls back to commit 2914a6dcc6.

commit 9633e17d04
Author: David Steele <github@thelabyrinth.net>
Date:   Thu Jan 9 17:58:40 2014 -0500

    Revert "Removed tmp file."

    This reverts commit 52d8697118.

commit 2914a6dcc6
Author: David Steele <github@thelabyrinth.net>
Date:   Thu Jan 9 17:58:01 2014 -0500

    Working on delta

commit 52d8697118
Author: David Steele <github@thelabyrinth.net>
Date:   Fri Jan 3 21:24:13 2014 -0500

    Removed tmp file.

commit ba5075a3e0
Author: David Steele <github@thelabyrinth.net>
Date:   Fri Jan 3 21:19:41 2014 -0500

    New implementation for archive logging

commit 6379b723de
Author: David Steele <github@thelabyrinth.net>
Date:   Fri Jan 3 20:10:53 2014 -0500

    Set the backup label

commit 78457a2f42
Author: David Steele <github@thelabyrinth.net>
Date:   Fri Jan 3 19:59:47 2014 -0500

    Simplified incremental naming

commit ef846ac8db
Author: David Steele <github@thelabyrinth.net>
Date:   Fri Jan 3 19:28:49 2014 -0500

    Working on backup rotation.

commit e6e995ec19
Author: David Steele <gitlab@thelabyrinth.net>
Date:   Wed Dec 18 22:08:49 2013 -0500

    Fixed typo in config.

commit ba72eda700
Author: David Steele <gitlab@thelabyrinth.net>
Date:   Wed Dec 18 21:30:51 2013 -0500

    Commented out dir mod time variable.

commit 9907c8b7b7
Author: David Steele <gitlab@thelabyrinth.net>
Date:   Tue Dec 17 07:47:24 2013 -0500

    Set file mtime.

commit b1ffa94190
Author: David Steele <gitlab@thelabyrinth.net>
Date:   Mon Dec 16 21:59:22 2013 -0500

    New storage structure, saved archive start/stop location.

commit 6d992855d2
Author: David Steele <gitlab@thelabyrinth.net>
Date:   Sun Dec 15 20:56:42 2013 -0500

    Cleaning up start stop backup code.

commit 0d3fa39117
Author: David Steele <gitlab@thelabyrinth.net>
Date:   Sun Dec 15 20:04:07 2013 -0500

    Added start and stop backup.

commit 1d21f4f9fa
Author: David Steele <gitlab@thelabyrinth.net>
Date:   Sun Dec 15 18:18:54 2013 -0500

    Added diff to commands.

commit 31fca50eae
Author: David Steele <gitlab@thelabyrinth.net>
Date:   Sun Dec 15 17:18:50 2013 -0500

    Backup more or less works - no start/stop backup or archiving.

commit 21b7a3f27a
Author: David Steele <gitlab@thelabyrinth.net>
Date:   Sat Dec 14 15:02:47 2013 -0500

    Added manifest load and save.

commit 8b40ea7d59
Author: David Steele <gitlab@thelabyrinth.net>
Date:   Sat Dec 14 14:03:08 2013 -0500

    Tablespace names are now recorded

commit 2ba26ffc26
Author: David Steele <gitlab@thelabyrinth.net>
Date:   Sat Dec 14 13:14:59 2013 -0500

    More refactoring.

commit d12773ca85
Author: David Steele <gitlab@thelabyrinth.net>
Date:   Wed Dec 11 20:39:07 2013 -0500

    Added generic config command

commit 4aa480b858
Author: David Steele <gitlab@thelabyrinth.net>
Date:   Wed Dec 11 20:13:39 2013 -0500

    Cleaned up some var names.

commit 299d402209
Author: David Steele <gitlab@thelabyrinth.net>
Date:   Wed Dec 11 19:57:54 2013 -0500

    More cleanup

commit aa6e72bb9c
Author: David Steele <gitlab@thelabyrinth.net>
Date:   Wed Dec 11 19:22:41 2013 -0500

    Refactoring

commit 0dba2ab8c7
Author: David Steele <gitlab@thelabyrinth.net>
Date:   Tue Dec 10 20:16:43 2013 -0500

    Added backup command

commit 0bdccd2982
Author: David Steele <gitlab@thelabyrinth.net>
Date:   Tue Dec 10 19:18:41 2013 -0500

    Reversed order of tests.

commit f09505f60e
Author: David Steele <gitlab@thelabyrinth.net>
Date:   Tue Dec 10 19:11:54 2013 -0500

    Working on unit tests for archiving.

commit 39989dd01c
Author: David Steele <gitlab@thelabyrinth.net>
Date:   Mon Dec 9 17:26:47 2013 -0500

    Working on manifest.

commit 02a9a693b8
Author: David Steele <gitlab@thelabyrinth.net>
Date:   Fri Dec 6 20:24:14 2013 -0500

    Working on backup.

commit 4d42d448e9
Author: David Steele <gitlab@thelabyrinth.net>
Date:   Thu Dec 5 23:29:05 2013 -0500

    Working on backup.

commit 12518e7296
Author: David Steele <gitlab@thelabyrinth.net>
Date:   Thu Dec 5 08:59:39 2013 -0500

    Working on backup.

commit ae6886c8e0
Author: David Steele <gitlab@thelabyrinth.net>
Date:   Wed Dec 4 22:30:26 2013 -0500

    Started backup command.

commit c9c7e340a7
Author: David Steele <gitlab@thelabyrinth.net>
Date:   Wed Dec 4 21:37:45 2013 -0500

    Added config file.

commit 53b2aae106
Author: David Steele <gitlab@thelabyrinth.net>
Date:   Mon Dec 2 21:37:01 2013 -0500

    Added note.

commit 4a2b009c2e
Author: David Steele <gitlab@thelabyrinth.net>
Date:   Mon Dec 2 21:34:25 2013 -0500

    Removed unused include.

commit 1e114cdc9d
Author: David Steele <gitlab@thelabyrinth.net>
Date:   Mon Dec 2 21:26:32 2013 -0500

    Fix comments.

commit 9553baf540
Author: David Steele <gitlab@thelabyrinth.net>
Date:   Mon Dec 2 21:23:10 2013 -0500

    Basic error checking and options for archive-local command

commit c7c049902f
Author: David Steele <gitlab@thelabyrinth.net>
Date:   Mon Dec 2 16:21:40 2013 -0500

    Added warnings, comments, removed dead code.

commit 5761832df5
Author: David Steele <gitlab@thelabyrinth.net>
Date:   Mon Dec 2 15:10:18 2013 -0500

    Implemented strict.

commit a2c98391df
Author: David Steele <gitlab@thelabyrinth.net>
Date:   Mon Dec 2 14:34:37 2013 -0500

    Starting on configuration.

commit 2be8c5d9e7
Author: David Steele <gitlab@thelabyrinth.net>
Date:   Sat Nov 23 20:05:04 2013 -0500

    Working on unit tests.

commit 5d0251be2f
Author: David Steele <gitlab@thelabyrinth.net>
Date:   Sat Nov 23 19:16:09 2013 -0500

    Working on archive-local and beefing up unit tests.

commit 8b37786e5b
Author: David Steele <gitlab@thelabyrinth.net>
Date:   Fri Nov 22 23:24:37 2013 -0500

    Now connecting to db.

commit 8db52c115a
Author: David Steele <gitlab@thelabyrinth.net>
Date:   Fri Nov 22 17:29:01 2013 -0500

    Fixed whitespace.

commit d86e7eb627
Author: David Steele <dsteele@Davids-MacBook-Pro.local>
Date:   Wed Nov 20 22:30:26 2013 -0500

    Added checksums.

commit bc46aefe61
Author: David Steele <dsteele@Davids-MacBook-Pro.local>
Date:   Wed Nov 20 22:24:30 2013 -0500

    Fixed for OSX.  Do not every use TextEditor on code!

commit e67821a230
Author: David Steele <dsteele@Davids-MacBook-Pro.local>
Date:   Wed Nov 20 21:49:07 2013 -0500

    Working on OSX port.

commit 9ce37308b5
Author: David Steele <github@thelabyrinth.net>
Date:   Sun Nov 17 21:48:53 2013 -0500

    Working on archive-local command.

commit 548578c8c9
Author: David Steele <dsteele@laptop-dev.(none)>
Date:   Sun Nov 17 13:58:21 2013 -0500

    Working on unit test.
2014-03-04 08:00:51 -05:00
David Steele
f0d82151b9 Changed thread timeout from WARN to ERROR 2014-03-03 16:38:01 -05:00
David Steele
571d449717 Redirect find error output to /dev/null. Sometimes files are removed from the db while find is running. We only want to error if the find process errors. 2014-02-27 19:15:00 -05:00
David Steele
84c4cec257 Put redirects in common places where errors are be output but we don't want to see them. 2014-02-27 13:00:41 -05:00
David Steele
ae782d0884 Made archiving single-threaded so it will work 2014-02-25 07:30:45 -05:00
David Steele
a84478045e No attempt to kill threads - just abort 2014-02-24 18:04:42 -05:00
David Steele
da41eb3883 Added more warnings 2014-02-24 17:59:42 -05:00
David Steele
21b15b7cf6 Added thread timeout. 2014-02-24 09:33:47 -05:00
David Steele
5de8f275f0 Removed stderr redirects from the master ssh connection 2014-02-22 11:14:01 -05:00
David Steele
0e4f0f1308 Fixed issue with archive path expiration 2014-02-22 10:54:02 -05:00
David Steele
e8c7d7aca6 When archive_retention_type is full, use the oldest backup if there are fewer than required 2014-02-22 09:05:58 -05:00
David Steele
9270a67892 Removed some more stderr redirects 2014-02-21 16:09:34 -05:00
David Steele
6d975d2830 No longer discarding stderr 2014-02-21 08:49:27 -05:00
David Steele
0387a8ee09 replaced process id with thread id. Added use thread to all modules. 2014-02-21 07:34:17 -05:00
David Steele
ac3ce81621 Add process id to logging, don't generate checksum for 0 length files 2014-02-21 06:42:42 -05:00