db/pgbackrest
1
0
Fork 0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2024-12-14 10:13:05 +02:00
Code Issues Releases Activity
1,423 Commits 4 Branches 113 Tags 103 MiB
6c3b40152f
Commit Graph

570 Commits

Author SHA1 Message Date
David Steele
6c3b40152f Improve module branch coverage. 
Add full branch coverage for cipher/block, command/command, common/wait, and postgres/pageChecksum modules.
 2018-05-04 09:21:32 -04:00
David Steele
c3a8fbe706 Add _FMT variants for all THROW macros so format types are checked by the compiler. 2018-05-03 11:24:29 -04:00
Yogesh Sharma
6a40c916d4 Add repo-s3-token option to allow temporary credentials tokens to be configured. 
pgBackRest currently has no way to request new credentials so the entire command (e.g. backup, restore) must complete before the credentials expire.

Contributed by Yogesh Sharma.
 2018-05-02 14:06:40 -04:00
David Steele
5e090ba305 Fix failure in manifest build when two or more files in PGDATA are linked to the same directory. 
Reported by Vitaliy Kukharik.
 2018-05-02 12:19:54 -04:00
David Steele
fd1984239e Fix rendering of key/value and list options in help. 
Reported by Clinton Adams.
 2018-05-01 14:07:08 -04:00
David Steele
1a1ed8d6b9 Update help test to use restore command to provide better coverage. 2018-05-01 13:20:48 -04:00
David Steele
8dbbf2de92 Fix archive-copy throwing "path not found" error for incr/diff backups. 
Reported by yummyliu, Vitaliy Kukharik.
 2018-05-01 12:37:06 -04:00
David Steele
71ba08f579 Use path list in the backup manifest to do restore path syncs. 
Remove recursive path sync functionality since it is no longer used.
 2018-05-01 11:05:37 -04:00
David Steele
54dd6f3ed4 Add asynchronous, parallel archive-get. 
This feature maintains a queue of WAL segments to help reduce latency when PostgreSQL requests a WAL segment with restore_command.
 2018-04-30 17:27:39 -04:00
David Steele
321a28f6b0 Add walSegmentNext() and walSegmentRange(). 2018-04-29 11:47:50 -04:00
David Steele
be02c67503 Add pgControlInfo() to read pg_control and determine the PostgreSQL version. 2018-04-29 11:20:51 -04:00
David Steele
8c6e2bdbc7 Add storageInfo() and track size in read objects. 2018-04-29 11:02:21 -04:00
David Steele
d74c167420 Add bufNewC() and only reallocate buffer when the size has changed. 2018-04-29 10:41:05 -04:00
David Steele
d44848baa0 Add strLstExists(), strLstExistsZ(), strSub(), and strSubN() to String and StringList objects. 2018-04-29 10:32:46 -04:00
David Steele
89d3476e32 Refactor archive common functions in preparation for parallel async archive-get. 2018-04-29 10:16:59 -04:00
Cynthia Shang
721432f6ae Update the archive-push-queue-max, manifest-save-threshold, and buffer-size options to accept values in KB, MB, GB, TB, or PB where the multiplier is a power of 1024. 
Contributed by Cynthia Shang.
 2018-04-25 15:46:05 -04:00
Cynthia Shang
8eabc52f3b Add strUpper, strLower, strChr, and strTrunc to String object. 
Contributed by Cynthia Shang.
 2018-04-25 15:16:42 -04:00
David Steele
88a633da17 Divide tests into three types (unit, integration, performance). 
Many options that were set per test can instead be inferred from the types, i.e. container, c, expect, and individual.

Also finish renaming Perl unit tests with the -perl suffix.
 2018-04-24 09:12:25 -04:00
David Steele
bb8c315cad Storage object improvements. 
* Add storageCopy(), storageMove(), and storagePathSync().
* Separate StorageFile object into separate read and write objects.
* Abstract out Posix file read/write objects.
 2018-04-23 17:26:27 -04:00
David Steele
18882cb882 Add bufEq and bufCat to Buffer object. 2018-04-23 10:19:45 -04:00
David Steele
f146502eae Add TEST_ERROR_FMT macro to simplify testing of formatted error messages. 2018-04-23 10:10:55 -04:00
Cynthia Shang
27aa5f756e Add support for additional pgBackRest configuration files. 
Configuration files are loaded from the directory specified by the --config-include-path option.

Add --config-path option for overriding the default base path of the --config and --config-include-path option.

Contributed by Cynthia Shang.
 2018-04-19 11:09:39 -04:00
David Steele
79b7552940 Rename archive-queue-max option to archive-push-queue-max. 
This avoids confusion with the archive-get-queue-max option. The old option name will continue to be accepted.
 2018-04-18 19:37:35 -04:00
David Steele
ea4046a607 Try tweaking time sync settings to prevent clock drift rather than restarting VBoxService on every test run. 2018-04-18 18:47:43 -04:00
David Steele
0381945caa Show command parameters as well as command options in initial info message. 2018-04-17 18:47:14 -04:00
David Steele
e9790fd3b1 Update out-of-date description for the spool-path option. 2018-04-17 17:38:25 -04:00
David Steele
cf0ec44ede Rename Perl tests so they don't conflict with their C counterparts. 2018-04-17 12:03:03 -04:00
David Steele
dce11ce3cf Generate code counts for all source files. 
The source files are also classified by type and purpose.
 2018-04-16 15:52:17 -04:00
David Steele
3e22977484 Use pip 9.03 in test VMs. 
pip 10 drops support for Python 2.6 which is still used by the older test VMs.
 2018-04-16 15:25:50 -04:00
David Steele
2a4ff2f904 Split cfgLoad() into multiple functions to make testing easier. 
Mainly this helps with unit tests that need to do log expect testing. Add harnessCfgLoad() test function, which allows a new config to be loaded for unit testing without resetting log functions, opening a log file, or taking locks.
 2018-04-13 16:05:52 -04:00
David Steele
f0250dab4b Move async forking and more error handling to C. 
The Perl process was exiting directly when called but that interfered with proper locking for the forked async process. Now Perl returns results to the C process which handles all errors, including signals.
 2018-04-12 20:42:26 -04:00
David Steele
6fd0c3dcaa Improved lock implementation written in C. 
Now only two types of locks can be taken: archive and backup. Most commands use one or the other but the stanza-* commands acquire both locks. This provides better protection than the old command-based locking scheme.
 2018-04-11 09:36:12 -04:00
David Steele
4744eb9387 Add storagePathRemove() and use it in the Perl Posix driver. 
This implementation should be faster because it does not stat each file.  It simply assumes that most directory entries are files so attempts an unlink() first.  If the entry is reported by error codes to be a directory then it attempts an rmdir().
 2018-04-11 08:21:09 -04:00
David Steele
c9ce20d41a Skip version checks when testing in --dev mode. 2018-04-09 16:46:36 -04:00
David Steele
3bcf637aee Move archive-stop and expire tests to the mock module. 
These are mock integration tests so they should be grouped with the other mock integration tests.
 2018-04-08 12:57:02 -04:00
David Steele
3b661f4ad5 Move help/version integration tests to mock/all. 
Help and version are covered by unit tests, so we really just to need to make sure there is output when called from the command line.
 2018-04-08 12:16:32 -04:00
David Steele
741acfd779 Move test definitions to test/define.yaml. 
The location is better because it is no longer buried in the Perl test libs. Also, the data can be easily accessed from C.
 2018-04-08 10:19:24 -04:00
David Steele
b77b425096 Allow -DDEBUG_UNIT to be suppressed to test how debug macros behave. 2018-04-07 19:44:00 -04:00
David Steele
82751b3b51 Split debug and assert code into separate headers. 
Assert can be used earlier because it only depends on the error-handler and not logging.
 2018-04-07 13:12:35 -04:00
David Steele
e00f2dd4ad More C type cleanup. 
* Remove typec.h entirely.
* Order all typdefs above local includes.
 2018-04-06 21:46:45 -04:00
David Steele
f3b56cf8f3 Allow MemContext objects to be copied to a new parent. 
This makes it easier to create objects and then copy them to another context when they are complete without having to worry about freeing them on error. Update List, StringList, and Buffer to allow moves. Update Ini and Storage to take advantage of moves.
 2018-04-06 17:04:52 -04:00
David Steele
b9f636cab4 Cleanup C types. 
* Remove pgBackRest defined types included with the C types.
* Remove redefinition of standard C types.
 2018-04-05 22:38:11 -04:00
David Steele
bd25248df0 Abstract Posix storage driver code into a separate module. 2018-04-05 14:48:16 -04:00
David Steele
348278bb68 Make backup directory sync more efficient. 
Scanning the entire backup directory can be very expensive if there are a lot of small tables. The backup manifest contains the backup directory list so use it to perform syncs instead of scanning the backup directory.
 2018-04-03 21:30:15 -04:00
David Steele
0dce8ba2fa Make backup.history sync more efficient. 
Only the backup.history/[year] directory was being synced, so check if the backup.history is newly created and sync it as well.
 2018-04-03 20:54:18 -04:00
David Steele
93fdb98d15 Storage object improvements. 
* Convert all functions to variadic functions.
* Enforce read-only storage.
* Add storageLocalWrite() helper function. Add storageExists(), storagePathCreate(), storageRemove(), and storageStat().
* Add StorageFile object and storageOpenRead()/storageOpenWrite().
 2018-04-03 12:25:21 -04:00
David Steele
90f980fe91 Remove documentation describing info --output=json as experimental. 2018-03-30 21:33:30 -04:00
David Steele
635caff573 Add ASSERT() that is preserved in production builds. 2018-03-30 19:10:34 -04:00
David Steele
a8721ffe11 Improve logic for smart builds to include version changes. 2018-03-27 22:48:22 -04:00
David Steele
5518a2ac51 Full branch coverage for storage/storage module. 2018-03-27 20:40:33 -04:00