1
0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2024-12-14 10:13:05 +02:00
Commit Graph

324 Commits

Author SHA1 Message Date
David Steele
91be372e6a Set log-timestamp=n for integration tests.
This means less filtering of logs needs to be done and new timestamps can be added without adding new filters.
2018-05-11 11:24:38 -04:00
David Steele
a1be8e16cd Update SSL error message test on CentOS 7. 2018-05-11 10:39:36 -04:00
David Steele
790f7c7312 Enforce full branch coverage in C code. 2018-05-05 11:50:11 -04:00
David Steele
c3a8fbe706 Add _FMT variants for all THROW macros so format types are checked by the compiler. 2018-05-03 11:24:29 -04:00
Yogesh Sharma
6a40c916d4 Add repo-s3-token option to allow temporary credentials tokens to be configured.
pgBackRest currently has no way to request new credentials so the entire command (e.g. backup, restore) must complete before the credentials expire.

Contributed by Yogesh Sharma.
2018-05-02 14:06:40 -04:00
David Steele
5e090ba305 Fix failure in manifest build when two or more files in PGDATA are linked to the same directory.
Reported by Vitaliy Kukharik.
2018-05-02 12:19:54 -04:00
David Steele
8dbbf2de92 Fix archive-copy throwing "path not found" error for incr/diff backups.
Reported by yummyliu, Vitaliy Kukharik.
2018-05-01 12:37:06 -04:00
David Steele
71ba08f579 Use path list in the backup manifest to do restore path syncs.
Remove recursive path sync functionality since it is no longer used.
2018-05-01 11:05:37 -04:00
David Steele
54dd6f3ed4 Add asynchronous, parallel archive-get.
This feature maintains a queue of WAL segments to help reduce latency when PostgreSQL requests a WAL segment with restore_command.
2018-04-30 17:27:39 -04:00
David Steele
89d3476e32 Refactor archive common functions in preparation for parallel async archive-get. 2018-04-29 10:16:59 -04:00
David Steele
88a633da17 Divide tests into three types (unit, integration, performance).
Many options that were set per test can instead be inferred from the types, i.e. container, c, expect, and individual.

Also finish renaming Perl unit tests with the -perl suffix.
2018-04-24 09:12:25 -04:00
Cynthia Shang
27aa5f756e Add support for additional pgBackRest configuration files.
Configuration files are loaded from the directory specified by the --config-include-path option.

Add --config-path option for overriding the default base path of the --config and --config-include-path option.

Contributed by Cynthia Shang.
2018-04-19 11:09:39 -04:00
David Steele
79b7552940 Rename archive-queue-max option to archive-push-queue-max.
This avoids confusion with the archive-get-queue-max option. The old option name will continue to be accepted.
2018-04-18 19:37:35 -04:00
David Steele
cf0ec44ede Rename Perl tests so they don't conflict with their C counterparts. 2018-04-17 12:03:03 -04:00
David Steele
dce11ce3cf Generate code counts for all source files.
The source files are also classified by type and purpose.
2018-04-16 15:52:17 -04:00
David Steele
3e22977484 Use pip 9.03 in test VMs.
pip 10 drops support for Python 2.6 which is still used by the older test VMs.
2018-04-16 15:25:50 -04:00
David Steele
2a4ff2f904 Split cfgLoad() into multiple functions to make testing easier.
Mainly this helps with unit tests that need to do log expect testing. Add harnessCfgLoad() test function, which allows a new config to be loaded for unit testing without resetting log functions, opening a log file, or taking locks.
2018-04-13 16:05:52 -04:00
David Steele
f0250dab4b Move async forking and more error handling to C.
The Perl process was exiting directly when called but that interfered with proper locking for the forked async process. Now Perl returns results to the C process which handles all errors, including signals.
2018-04-12 20:42:26 -04:00
David Steele
6fd0c3dcaa Improved lock implementation written in C.
Now only two types of locks can be taken: archive and backup. Most commands use one or the other but the stanza-* commands acquire both locks. This provides better protection than the old command-based locking scheme.
2018-04-11 09:36:12 -04:00
David Steele
3bcf637aee Move archive-stop and expire tests to the mock module.
These are mock integration tests so they should be grouped with the other mock integration tests.
2018-04-08 12:57:02 -04:00
David Steele
3b661f4ad5 Move help/version integration tests to mock/all.
Help and version are covered by unit tests, so we really just to need to make sure there is output when called from the command line.
2018-04-08 12:16:32 -04:00
David Steele
741acfd779 Move test definitions to test/define.yaml.
The location is better because it is no longer buried in the Perl test libs. Also, the data can be easily accessed from C.
2018-04-08 10:19:24 -04:00
David Steele
b77b425096 Allow -DDEBUG_UNIT to be suppressed to test how debug macros behave. 2018-04-07 19:44:00 -04:00
David Steele
82751b3b51 Split debug and assert code into separate headers.
Assert can be used earlier because it only depends on the error-handler and not logging.
2018-04-07 13:12:35 -04:00
David Steele
e00f2dd4ad More C type cleanup.
* Remove typec.h entirely.
* Order all typdefs above local includes.
2018-04-06 21:46:45 -04:00
David Steele
f3b56cf8f3 Allow MemContext objects to be copied to a new parent.
This makes it easier to create objects and then copy them to another context when they are complete without having to worry about freeing them on error. Update List, StringList, and Buffer to allow moves. Update Ini and Storage to take advantage of moves.
2018-04-06 17:04:52 -04:00
David Steele
b9f636cab4 Cleanup C types.
* Remove pgBackRest defined types included with the C types.
* Remove redefinition of standard C types.
2018-04-05 22:38:11 -04:00
David Steele
bd25248df0 Abstract Posix storage driver code into a separate module. 2018-04-05 14:48:16 -04:00
David Steele
93fdb98d15 Storage object improvements.
* Convert all functions to variadic functions.
* Enforce read-only storage.
* Add storageLocalWrite() helper function. Add storageExists(), storagePathCreate(), storageRemove(), and storageStat().
* Add StorageFile object and storageOpenRead()/storageOpenWrite().
2018-04-03 12:25:21 -04:00
David Steele
635caff573 Add ASSERT() that is preserved in production builds. 2018-03-30 19:10:34 -04:00
David Steele
c1ab7b3c98 Start work on C handle io object and use it to output help. 2018-03-25 11:39:24 -04:00
David Steele
dabf727d91 Include VM type in gcov path to avoid conflicts between VMs with different architectures. 2018-03-21 16:08:08 -04:00
David Steele
07f38f584a Use lcov for C unit test coverage reporting.
Switch from Devel::Cover because it would not report on branch coverage for reports converted from gcov.

Branch coverage is not complete, so for the time being errors will only be generated when statement coverage is not complete. Coverage of unit tests is not displayed in the report unless they are incomplete for either statement or branch coverage.
2018-03-19 23:33:28 -04:00
Cynthia Shang
f0a4264f77 Fix manifest test in the check command when tablespaces are present.
Fixed by Cynthia Shang.
Reported by Thomas Flatley.
2018-03-19 19:54:03 -04:00
David Steele
0c313713b1 Use clang for static code analysis during lint testing.
Nothing found except for some functions that should have been marked __noreturn__.
2018-03-18 13:32:19 -04:00
David Steele
f0451c1494 Enable -Wformat=2 and -Wformat-nonliteral and silence new warnings. 2018-03-15 13:53:03 -04:00
David Steele
dd3ce70810 Config test code writes secure options to a file instead of passing on the command-line. 2018-03-15 11:09:46 -04:00
David Steele
4b6cd92a4c Improve debugging.
* Replace remaining NDEBUG blocks with the more granular DEBUG_UNIT.
* Remove some debug memset() calls in MemContext since valgrind is more useful for these checks.
2018-03-12 17:15:14 -04:00
David Steele
cced6ec03a Add ASSERT_DEBUG() macro for debugging.
Replace all current assert() calls except in tests that can't use the debug code.
2018-03-12 14:31:22 -04:00
David Steele
03f1082e86 Improve logging.
Move command begin to C except when it must be called after another command in Perl (e.g. expire after backup). Command begin logs correctly for complex data types like hash and list. Specify which commands will log to file immediately and set the default log level for log messages that are common to all commands. File logging is initiated from C.
2018-03-12 11:03:38 -04:00
David Steele
94494a632f Update pip before installing awscli. 2018-03-12 11:02:46 -04:00
David Steele
f047cd0dfd Improve code generation performance. 2018-03-09 23:00:20 -05:00
David Steele
919635b6f4 Set config before Main::main() call to avoid secrets being exposed in a stack trace. 2018-03-08 16:24:16 -05:00
David Steele
e331564c90 Add new test for Common::Io::Process.
Show that output on stderr will raise an exception on close() even if the exit code is 0.
2018-03-08 12:36:55 -05:00
Cynthia Shang
45ffe37e4f Add cfgOptionTest().
Update cfgOption() calls that are better implemented as cfgOptionTest().

Contributed by Cynthia Shang.
2018-03-05 18:51:48 -05:00
David Steele
406acf9cba Enable -Wconversion and silence new warnings. 2018-03-02 12:07:12 -05:00
David Steele
5b491b3035 Add CentOS/RHEL package builds. 2018-02-26 19:32:27 -05:00
David Steele
64a7196cb6 Rename retention-* options to repo-retention-*. 2018-02-19 15:07:24 -05:00
David Steele
9070325760 Improve performance of HTTPS client.
Buffering now takes the pending bytes on the socket into account (when present) rather than relying entirely on select(). In some instances the final bytes would not be flushed until the connection was closed.
2018-02-18 14:53:27 -05:00
David Steele
72cbb9a9d4 Call Perl with built-in interpreter instead of execvp().
Exec'ing Perl worked fine but generated a very large command line in the process list and potentially exposed secrets.
2018-02-14 17:19:54 -05:00