1
0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2024-12-14 10:13:05 +02:00
Commit Graph

752 Commits

Author SHA1 Message Date
David Steele
d74c167420 Add bufNewC() and only reallocate buffer when the size has changed. 2018-04-29 10:41:05 -04:00
David Steele
d44848baa0 Add strLstExists(), strLstExistsZ(), strSub(), and strSubN() to String and StringList objects. 2018-04-29 10:32:46 -04:00
Cynthia Shang
721432f6ae Update the archive-push-queue-max, manifest-save-threshold, and buffer-size options to accept values in KB, MB, GB, TB, or PB where the multiplier is a power of 1024.
Contributed by Cynthia Shang.
2018-04-25 15:46:05 -04:00
Cynthia Shang
8eabc52f3b Add strUpper, strLower, strChr, and strTrunc to String object.
Contributed by Cynthia Shang.
2018-04-25 15:16:42 -04:00
David Steele
88a633da17 Divide tests into three types (unit, integration, performance).
Many options that were set per test can instead be inferred from the types, i.e. container, c, expect, and individual.

Also finish renaming Perl unit tests with the -perl suffix.
2018-04-24 09:12:25 -04:00
David Steele
bb8c315cad Storage object improvements.
* Add storageCopy(), storageMove(), and storagePathSync().
* Separate StorageFile object into separate read and write objects.
* Abstract out Posix file read/write objects.
2018-04-23 17:26:27 -04:00
David Steele
02cc8ccbd4 Add THROWP_* macro variants for error handling.
These macros allow an ErrorType pointer to be passed and are required for functions that may return different errors based on a parameter.
2018-04-23 10:24:04 -04:00
David Steele
18882cb882 Add bufEq and bufCat to Buffer object. 2018-04-23 10:19:45 -04:00
David Steele
f146502eae Add TEST_ERROR_FMT macro to simplify testing of formatted error messages. 2018-04-23 10:10:55 -04:00
Cynthia Shang
27aa5f756e Add support for additional pgBackRest configuration files.
Configuration files are loaded from the directory specified by the --config-include-path option.

Add --config-path option for overriding the default base path of the --config and --config-include-path option.

Contributed by Cynthia Shang.
2018-04-19 11:09:39 -04:00
David Steele
79b7552940 Rename archive-queue-max option to archive-push-queue-max.
This avoids confusion with the archive-get-queue-max option. The old option name will continue to be accepted.
2018-04-18 19:37:35 -04:00
David Steele
ab26c765d4 Don't copy CFGDEF_NAME_ALT or CFGDEF_INHERIT when processing config option inheritance. 2018-04-18 18:57:45 -04:00
David Steele
ea4046a607 Try tweaking time sync settings to prevent clock drift rather than restarting VBoxService on every test run. 2018-04-18 18:47:43 -04:00
David Steele
0381945caa Show command parameters as well as command options in initial info message. 2018-04-17 18:47:14 -04:00
David Steele
e9790fd3b1 Update out-of-date description for the spool-path option. 2018-04-17 17:38:25 -04:00
David Steele
cf0ec44ede Rename Perl tests so they don't conflict with their C counterparts. 2018-04-17 12:03:03 -04:00
David Steele
dce11ce3cf Generate code counts for all source files.
The source files are also classified by type and purpose.
2018-04-16 15:52:17 -04:00
David Steele
3e22977484 Use pip 9.03 in test VMs.
pip 10 drops support for Python 2.6 which is still used by the older test VMs.
2018-04-16 15:25:50 -04:00
David Steele
2a4ff2f904 Split cfgLoad() into multiple functions to make testing easier.
Mainly this helps with unit tests that need to do log expect testing. Add harnessCfgLoad() test function, which allows a new config to be loaded for unit testing without resetting log functions, opening a log file, or taking locks.
2018-04-13 16:05:52 -04:00
David Steele
49fc737cd0 Add monitoring examples using PostgreSQL and jq.
Suggested by Stephen Frost, Brian Faherty.
2018-04-13 14:31:33 -04:00
David Steele
d924f3d50a Add logic to find the real oid of the test1 database during restore testing. 2018-04-13 12:00:22 -04:00
David Steele
f0250dab4b Move async forking and more error handling to C.
The Perl process was exiting directly when called but that interfered with proper locking for the forked async process. Now Perl returns results to the C process which handles all errors, including signals.
2018-04-12 20:42:26 -04:00
David Steele
6fd0c3dcaa Improved lock implementation written in C.
Now only two types of locks can be taken: archive and backup. Most commands use one or the other but the stanza-* commands acquire both locks. This provides better protection than the old command-based locking scheme.
2018-04-11 09:36:12 -04:00
David Steele
4744eb9387 Add storagePathRemove() and use it in the Perl Posix driver.
This implementation should be faster because it does not stat each file.  It simply assumes that most directory entries are files so attempts an unlink() first.  If the entry is reported by error codes to be a directory then it attempts an rmdir().
2018-04-11 08:21:09 -04:00
David Steele
c9ce20d41a Skip version checks when testing in --dev mode. 2018-04-09 16:46:36 -04:00
David Steele
3bcf637aee Move archive-stop and expire tests to the mock module.
These are mock integration tests so they should be grouped with the other mock integration tests.
2018-04-08 12:57:02 -04:00
David Steele
3b661f4ad5 Move help/version integration tests to mock/all.
Help and version are covered by unit tests, so we really just to need to make sure there is output when called from the command line.
2018-04-08 12:16:32 -04:00
David Steele
741acfd779 Move test definitions to test/define.yaml.
The location is better because it is no longer buried in the Perl test libs. Also, the data can be easily accessed from C.
2018-04-08 10:19:24 -04:00
David Steele
b77b425096 Allow -DDEBUG_UNIT to be suppressed to test how debug macros behave. 2018-04-07 19:44:00 -04:00
David Steele
82751b3b51 Split debug and assert code into separate headers.
Assert can be used earlier because it only depends on the error-handler and not logging.
2018-04-07 13:12:35 -04:00
David Steele
e00f2dd4ad More C type cleanup.
* Remove typec.h entirely.
* Order all typdefs above local includes.
2018-04-06 21:46:45 -04:00
David Steele
f3b56cf8f3 Allow MemContext objects to be copied to a new parent.
This makes it easier to create objects and then copy them to another context when they are complete without having to worry about freeing them on error. Update List, StringList, and Buffer to allow moves. Update Ini and Storage to take advantage of moves.
2018-04-06 17:04:52 -04:00
David Steele
b9f636cab4 Cleanup C types.
* Remove pgBackRest defined types included with the C types.
* Remove redefinition of standard C types.
2018-04-05 22:38:11 -04:00
David Steele
bd25248df0 Abstract Posix storage driver code into a separate module. 2018-04-05 14:48:16 -04:00
David Steele
348278bb68 Make backup directory sync more efficient.
Scanning the entire backup directory can be very expensive if there are a lot of small tables. The backup manifest contains the backup directory list so use it to perform syncs instead of scanning the backup directory.
2018-04-03 21:30:15 -04:00
David Steele
0dce8ba2fa Make backup.history sync more efficient.
Only the backup.history/[year] directory was being synced, so check if the backup.history is newly created and sync it as well.
2018-04-03 20:54:18 -04:00
David Steele
5890272247 Fix directory syncs running recursively when only the specified directory should be synced.
Reported by Craig A. James.
2018-04-03 18:12:03 -04:00
David Steele
46b6be6d72 Fix header exclusion defines that do not match the general pattern. 2018-04-03 13:35:56 -04:00
David Steele
93fdb98d15 Storage object improvements.
* Convert all functions to variadic functions.
* Enforce read-only storage.
* Add storageLocalWrite() helper function. Add storageExists(), storagePathCreate(), storageRemove(), and storageStat().
* Add StorageFile object and storageOpenRead()/storageOpenWrite().
2018-04-03 12:25:21 -04:00
David Steele
90f980fe91 Remove documentation describing info --output=json as experimental. 2018-03-30 21:33:30 -04:00
David Steele
635caff573 Add ASSERT() that is preserved in production builds. 2018-03-30 19:10:34 -04:00
David Steele
a8721ffe11 Improve logic for smart builds to include version changes. 2018-03-27 22:48:22 -04:00
David Steele
c1ab7b3c98 Start work on C handle io object and use it to output help. 2018-03-25 11:39:24 -04:00
David Steele
9001b9b957 Replace THROW_ON_SYS_ERROR() with THROW_SYS_ERROR().
The former macro was hiding missing branch coverage for critical error handling.
2018-03-24 14:11:29 -04:00
David Steele
c7a6e9d2f2 Document build improvements.
* Perform apt-get update to ensure packages are up to date before installing.
* Add -p to the repository mkdir so it won't fail if the directory already exists, handy for testing packages.
2018-03-23 16:08:32 -04:00
David Steele
dabf727d91 Include VM type in gcov path to avoid conflicts between VMs with different architectures. 2018-03-21 16:08:08 -04:00
David Steele
c6457b070f Full branch coverage for config/parse module. 2018-03-21 13:46:08 -04:00
David Steele
b234f43c9d Refactor usec to msec in common/time.c.
The implementation provides usec resolution but this is not needed in practice and it makes the interface more complicated due to the extra zeros.
2018-03-21 09:18:48 -04:00
David Steele
07f38f584a Use lcov for C unit test coverage reporting.
Switch from Devel::Cover because it would not report on branch coverage for reports converted from gcov.

Branch coverage is not complete, so for the time being errors will only be generated when statement coverage is not complete. Coverage of unit tests is not displayed in the report unless they are incomplete for either statement or branch coverage.
2018-03-19 23:33:28 -04:00
David Steele
2a3d6ecde8 v2.01: Minor Bug Fixes and Improvements
Bug Fixes:

* Fix --target-action and --recovery-option options being reported as invalid when restoring with --type=immediate. (Reported by Brad Nicholson.)
* Immediately error when a secure option (e.g. repo1-s3-key) is passed on the command line. Since pgBackRest would not pass secure options on to sub-processes an obscure error was thrown. The new error is much clearer and provides hints about how to fix the problem. Update command documentation to omit secure options that cannot be specified on the command-line. (Reported by Brad Nicholson.)
* Fix issue passing --no-config to embedded Perl. (Reported by Ibrahim Edib Kokdemir.)
* Fix issue where specifying log-level-stderr > warn would cause a local/remote process to error on exit due to output found on stderr when none was expected. The max value for a local/remote process is now error since there is no reason for these processes to emit warnings. (Reported by Clinton Adams.)
* Fix manifest test in the check command when tablespaces are present. (Fixed by Cynthia Shang. Reported by Thomas Flatley.)

Improvements:

* Error when multiple arguments are set in the config file for an option that does not accept multiple arguments. (Contributed by Cynthia Shang.)
* Remove extraneous sudo commands from src/Makefile. (Contributed by Adrian Vondendriesch.)
2018-03-19 21:49:41 -04:00