David Steele
e44cd6c1ca
More paths through copy() working.
2014-06-05 22:42:47 -04:00
David Steele
ca6b435c4f
June Hackathon: copy_in and copy_out functions written, some paths through File->copy() working, basic unit tests setup.
2014-06-05 16:40:05 -04:00
David Steele
82814aebdd
Whitespace cleaning.
2014-06-04 11:58:30 -04:00
David Steele
1793b7a5c4
Comments and formatting.
2014-06-03 22:02:56 -04:00
David Steele
1b9782cb9e
Converted path_create(), improved unit tests.
2014-06-03 21:22:07 -04:00
David Steele
ffa45d07bf
New version of move().
2014-06-03 19:03:03 -04:00
David Steele
816c67edfd
Converted File::compress
2014-06-02 17:48:07 -04:00
David Steele
0249d3516b
Manifest is working (unit tests at least).
2014-06-02 16:26:37 -04:00
David Steele
361c2c24bf
Working on manifest.
2014-06-01 18:39:35 -04:00
David Steele
0338369193
Working on more robust file functions.
2014-06-01 17:23:33 -04:00
David Steele
85f591f801
More unit tests for file functions.
2014-05-27 09:00:24 -04:00
David Steele
3e12f9230b
Working on unit tests for file_copy. Still need to add specific error tests, timestamp, and permissions.
2014-05-14 15:07:37 -04:00
David Steele
d234aeea64
More work on error-handling.
2014-05-13 11:23:15 -04:00
David Steele
a2b28623d4
Working on better error handling.
2014-04-28 09:13:25 -04:00
David Steele
7624d65edf
Added TTY code.
2014-04-01 08:59:09 -04:00
David Steele
ba2590dc8e
archive-get function is working.
...
Added option to force checkpoint at backup start.
2014-03-28 15:20:36 -04:00
David Steele
8ae6c2719d
Better code in file_list_get().
2014-03-28 11:59:44 -04:00
David Steele
23df43e781
file_list_get now works locally and remotely
...
Conflicts:
README.md
pg_backrest.pl
pg_backrest_db.pm
pg_backrest_file.pm
2014-03-28 11:40:37 -04:00
David Steele
126b39855d
Working on archive-get
2014-03-25 18:56:05 -04:00
David Steele
587bd1f8d9
v0.10: Backup and archiving are functional
...
This version has been put into production at Resonate, so it does work, but there are a number of major caveats.
* No restore functionality, but the backup directories are consistent Postgres data directories. You'll need to either uncompress the files or turn off compression in the backup. Uncompressed backups on a ZFS (or similar) filesystem are a good option because backups can be restored locally via a snapshot to create logical backups or do spot data recovery.
* Archiving is single-threaded. This has not posed an issue on our multi-terabyte databases with heavy write volume. Recommend a large WAL volume or to use the async option with a large volume nearby.
* Backups are multi-threaded, but the Net::OpenSSH library does not appear to be 100% threadsafe so it will very occasionally lock up on a thread. There is an overall process timeout that resolves this issue by killing the process. Yes, very ugly.
* Checksums are lost on any resumed backup. Only the final backup will record checksum on multiple resumes. Checksums from previous backups are correctly recorded and a full backup will reset everything.
* The backup.manifest is being written as Storable because Config::IniFile does not seem to handle large files well. Would definitely like to save these as human-readable text.
* Absolutely no documentation (outside the code). Well, excepting these release notes.
* Lots of other little things and not so little things. Much refactoring to follow.
2014-03-05 19:53:13 -05:00
David Steele
571d449717
Redirect find error output to /dev/null. Sometimes files are removed from the db while find is running. We only want to error if the find process errors.
2014-02-27 19:15:00 -05:00
David Steele
84c4cec257
Put redirects in common places where errors are be output but we don't want to see them.
2014-02-27 13:00:41 -05:00
David Steele
5de8f275f0
Removed stderr redirects from the master ssh connection
2014-02-22 11:14:01 -05:00
David Steele
9270a67892
Removed some more stderr redirects
2014-02-21 16:09:34 -05:00
David Steele
6d975d2830
No longer discarding stderr
2014-02-21 08:49:27 -05:00
David Steele
0387a8ee09
replaced process id with thread id. Added use thread to all modules.
2014-02-21 07:34:17 -05:00
David Steele
800cd89831
Thread files are no longer in an array
2014-02-19 13:37:18 -05:00
David Steele
81433c0fdd
Now using one control master for threading.
2014-02-19 13:03:52 -05:00
David Steele
27b820cfb4
Backup handles files that were removed by the db
2014-02-18 15:42:51 -05:00
David Steele
187fdaf49b
Better compression choices.
2014-02-16 19:01:06 -05:00
David Steele
0979841f1a
Async compress, thread kill improvements
2014-02-15 14:18:15 -05:00
David Steele
308652cc65
Remove a lot of path creates - percent logging.
2014-02-14 09:05:14 -05:00
David Steele
7d55ce865b
Work on log levels
2014-02-13 17:11:53 -05:00
David Steele
133e27d813
Archive tmp file fix.
2014-02-13 15:26:07 -05:00
David Steele
13a1f0cf24
Lots and lots of logging improvements
2014-02-13 13:54:43 -05:00
David Steele
9a1de327a1
Turned ssh compression on
2014-02-11 16:37:20 -05:00
David Steele
c58ccb588e
Parallel backup now working
2014-02-11 15:31:16 -05:00
David Steele
60b2be58ac
Finished db object abstraction
2014-02-06 16:37:37 -05:00
David Steele
f20f3b3ee6
Started db object abstraction.
2014-02-06 15:54:22 -05:00
David Steele
9d08f2a644
More robust config. Retention is read from config.
2014-02-06 12:49:54 -05:00
David Steele
a2b0d7a674
Backup expiration working again. Other changes.
2014-02-05 22:26:10 -05:00
David Steele
27986e0c10
File is now an object.
2014-02-05 21:39:08 -05:00
David Steele
3e7faf0e5f
Some path fixes.
2014-02-05 16:53:25 -05:00
David Steele
7bee0f3141
Moved backup location.
2014-02-05 15:56:05 -05:00
David Steele
a15fcb8c23
Moved tmp path.
2014-02-05 13:10:36 -05:00
David Steele
7ffade2453
Fixed file mod time and permissions
2014-02-05 11:35:09 -05:00
David Steele
c86a61ca3e
Relative links working for tablespaces.
2014-02-05 10:40:49 -05:00
David Steele
12a960f91a
File copy supports decompression
2014-02-05 08:21:27 -05:00
David Steele
a810499958
Remote backup fully working.
2014-02-04 18:48:39 -05:00
David Steele
e38c482a26
Cleanup
2014-02-03 20:48:02 -05:00
