1
0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2024-12-14 10:13:05 +02:00
Commit Graph

106 Commits

Author SHA1 Message Date
David Steele
e55d733041 Add -ftree-coalesce-vars option to unit test compilation.
This is a workaround for inefficient handling of many setjmps in gcc >= 4.9.  Setjmp is used in all error handling, but in the unit tests each test macro contains an error handling block so they add up pretty quickly for large unit tests.

Enabling -ftree-coalesce-vars in affected versions reduces build time and memory requirements by nearly an order of magnitude.  Even so, compiles are much slower than gcc <= 4.8.

We submitted a bug for this at: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=87316
Which was marked as a duplicate of: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63155
2018-09-17 11:38:10 -04:00
David Steele
4119ce208d Move test expect log out of the regular test directory.
Storing the expect log (created by common/harnessLog) in the regular test directory was not ideal.  It showed up in tests and made it difficult to clear the test directory between each run.

Move the expect log to a purpose-built directory one level up so it does not interfere with regular testing.
2018-09-16 15:58:46 -04:00
David Steele
f0ed89f21f Allow C or Perl coverage to run on more than one VM.
C or Perl coverage tests can now be run on any VM provided a recent enough version of Devel::Cover or lcov is available.

For now, leave u18 as the only VM to run coverage tests due to some issues with older versions of lcov.
2018-09-15 13:27:06 -04:00
David Steele
31cdd9d20b Remove compiler warnings that are not valid for u16. 2018-09-15 08:23:55 -04:00
David Steele
aeb1fa3dfb Don't perform valgrind when requested.
The --no-valgrind flag was not being honored.  It's not clear if this flag ever worked, but it does now.
2018-09-13 19:12:40 -04:00
David Steele
f7fc8422f7 Make Valgrind return an error even when a non-fatal issue is detected.
By default Valgrind does not exit with an error code when a non-fatal error is detected, e.g. unfreed memory.  Use the --error-exitcode option to enabled this behavior.

Update some minor issues discovered in the tests as a result.  Luckily, no issues were missed in the core code.
2018-09-07 16:50:01 -07:00
David Steele
8a8738308c Enable -Wvla. 2018-08-22 14:48:37 -04:00
David Steele
6643afe9a8 Add gzip compression/decompression filters for C. 2018-08-14 14:56:59 -04:00
David Steele
31167d8f98 Enable -Wduplicated-branches, and -Wduplicated-cond. 2018-08-05 08:35:18 -04:00
David Steele
14f21a8f6c Enable -Wpointer-arith. 2018-08-04 19:37:28 -04:00
David Steele
429a356e33 Enable -Wstrict-prototypes and update all void functions to conform. 2018-08-03 19:19:14 -04:00
David Steele
8568622a6f Add --log-level-test option.
This allows setting the test log level independently from the general test harness setting, but current only works for the C tests. It is useful for seeing log output from functions on the console while a test is running.
2018-07-20 19:03:46 -04:00
David Steele
5dc8a2ec08 storageFileRead() accepts a buffer for output rather than creating one.
This is more efficient overall and allows the caller to specify how many bytes will be read on each call. Reads are appended if the buffer already contains data but the buffer size will never increase.

Allow Buffer object "used size" to be different than "allocated size". Add functions to manage used size and remaining size and update automatically when possible.
2018-07-17 19:01:54 -04:00
David Steele
a385cb520b Update primary test environment (Vagrant and Docker) to Ubuntu 18.04. 2018-06-06 15:52:28 -04:00
David Steele
40093f160c Improve efficiency of C library builds now that they are used only for testing. 2018-05-24 14:01:24 -04:00
David Steele
4c7408ea52 Embed exported C functions and Perl modules directly into the pgBackRest executable. 2018-05-22 12:53:08 -04:00
David Steele
52bc073234 Add stack trace macros to all functions.
Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 11:57:32 -04:00
David Steele
abb9651f4c Test harness improvements.
* Allow more than one test to provide coverage for the same module.
* Add option to disable valgrind.
* Add option to disabled coverage.
* Add option to disable debug build.
* Add option to disable compiler optimization.
* Add --dev-test mode.
2018-05-18 06:45:14 -04:00
David Steele
790f7c7312 Enforce full branch coverage in C code. 2018-05-05 11:50:11 -04:00
David Steele
f0250dab4b Move async forking and more error handling to C.
The Perl process was exiting directly when called but that interfered with proper locking for the forked async process. Now Perl returns results to the C process which handles all errors, including signals.
2018-04-12 20:42:26 -04:00
David Steele
b77b425096 Allow -DDEBUG_UNIT to be suppressed to test how debug macros behave. 2018-04-07 19:44:00 -04:00
David Steele
93fdb98d15 Storage object improvements.
* Convert all functions to variadic functions.
* Enforce read-only storage.
* Add storageLocalWrite() helper function. Add storageExists(), storagePathCreate(), storageRemove(), and storageStat().
* Add StorageFile object and storageOpenRead()/storageOpenWrite().
2018-04-03 12:25:21 -04:00
David Steele
dabf727d91 Include VM type in gcov path to avoid conflicts between VMs with different architectures. 2018-03-21 16:08:08 -04:00
David Steele
07f38f584a Use lcov for C unit test coverage reporting.
Switch from Devel::Cover because it would not report on branch coverage for reports converted from gcov.

Branch coverage is not complete, so for the time being errors will only be generated when statement coverage is not complete. Coverage of unit tests is not displayed in the report unless they are incomplete for either statement or branch coverage.
2018-03-19 23:33:28 -04:00
David Steele
f0451c1494 Enable -Wformat=2 and -Wformat-nonliteral and silence new warnings. 2018-03-15 13:53:03 -04:00
David Steele
4b6cd92a4c Improve debugging.
* Replace remaining NDEBUG blocks with the more granular DEBUG_UNIT.
* Remove some debug memset() calls in MemContext since valgrind is more useful for these checks.
2018-03-12 17:15:14 -04:00
David Steele
03f1082e86 Improve logging.
Move command begin to C except when it must be called after another command in Perl (e.g. expire after backup). Command begin logs correctly for complex data types like hash and list. Specify which commands will log to file immediately and set the default log level for log messages that are common to all commands. File logging is initiated from C.
2018-03-12 11:03:38 -04:00
David Steele
406acf9cba Enable -Wconversion and silence new warnings. 2018-03-02 12:07:12 -05:00
David Steele
72cbb9a9d4 Call Perl with built-in interpreter instead of execvp().
Exec'ing Perl worked fine but generated a very large command line in the process list and potentially exposed secrets.
2018-02-14 17:19:54 -05:00
David Steele
a907fd7d2d Build C binary/library for C unit tests that require Perl. 2018-02-14 10:33:21 -05:00
David Steele
5242e8eec5 Generate suppressions when running Valgrind.
This makes it easier to suppress errors that are only occurring on Travis.
2018-02-14 09:18:27 -05:00
David Steele
c3f47bf240 Move config parsing out of Perl tests.
cfgParseTest() is provided in the C library for parsing configs in unit tests.
2018-02-05 12:32:30 -05:00
David Steele
be90028100 Rename db-* options to pg-* and backup-* options to repo-* to improve consistency.
* repo-* options are now indexed although only one is allowed.
* List deprecated option names in documentation and command-line help.
2018-02-03 18:27:38 -05:00
David Steele
a91a648019 Config parsing no longer implemented in Perl.
Options are passed to Perl as JSON from the C process.
2018-01-28 21:37:09 -05:00
David Steele
f0ef1dee05 Improve speed of C unit tests.
Preserve object files between tests and use a Makefile to avoid rebuilding object files.
2018-01-26 16:41:17 -05:00
David Steele
39cb971afb Add log module. 2018-01-16 13:52:20 -05:00
David Steele
69488ca8ff Add Storage object. 2017-12-22 22:36:01 -05:00
David Steele
5ea1263d8e Run valgrind on all C unit tests.
Also refactor code to make valgrind happy.
2017-12-22 18:36:36 -05:00
David Steele
74d6398ad2 The pgbackrest executable is now a C binary instead of Perl.
This allows certain time-critical commands (like async archive-push) to run more quickly.
2017-11-26 18:43:51 -05:00
David Steele
7cf955425e The C library is now required.
This eliminates conditional loading and eases development of new library features.
2017-11-26 17:45:00 -05:00
David Steele
8ccfaa9ad3 Designate a single distro (Ubuntu 16.04) for C coverage testing
Enable optimizations on the distros that don't do coverage testing.
2017-11-20 09:05:55 -05:00
David Steele
3b43047e0d Disable -Wclobber compiler warning.
...because it is mostly useless but keep the rest of of -Wextra.
2017-11-19 16:44:33 -05:00
David Steele
bcdfc7d0b5 Enable additional warnings for C builds. 2017-11-06 22:55:34 -05:00
David Steele
8d6a08a32b Library code for repository encryption support. 2017-11-03 13:57:58 -04:00
David Steele
ac542788da Convert configuration definitions from auto-generated functions to auto-generated data structures. 2017-11-02 08:14:13 -04:00
David Steele
9d78948a14 Improvements to page checksum module and tests:
* Page checksum module uses new C error handler.
* Convert page checksum tests into C unit tests.
2017-10-16 11:03:06 -04:00
David Steele
10dfbd90b5 Add C unit test infrastructure. 2017-10-12 12:55:48 -04:00
David Steele
1e0ed07455 Configuration rules are now pulled from the C library when present. 2017-08-25 16:47:47 -04:00
David Steele
7a9b1f6993 Container build improvements:
* More optimized container suite that greatly improves build time.
* Added static Debian packages for Devel::Cover to reduce build time.
* Add deprecated state for containers. Deprecated containers may only be used to build packages.
* Remove Debian 8 from CI because it does not provide additional coverage over Ubuntu 14.04 and Ubuntu 16.04.
2017-06-24 10:59:00 -04:00
David Steele
8b9965abcb Remove process-max option.
Parallelism is now tested in a more targeted manner and the high level option is no longer needed.
2017-06-21 19:21:16 -04:00
David Steele
de7fc37f88 Storage and IO layer refactor:
Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.)
Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 17:51:41 -04:00
David Steele
4049d59c9e Fixed issue with --dry-run requiring --vm-out to work properly. 2017-05-12 16:27:06 -04:00
David Steele
367e06f4be Coverage testing always enabled on Debian-based containers.
* Full coverage is verified when specified.
* Modules marked with partial coverage will error if they are actually fully covered.
* Simplified test representation is DefineTest.
* Added new representation for queries in DefineTest and added API functions.
* Update modules using DefineTest to use new API.
2017-04-10 12:31:30 -04:00
David Steele
31d2d790bd Added --retry option. 2017-02-21 22:10:02 -05:00
David Steele
01312f6260 Improved formatting of testResult() output. 2017-02-21 11:47:45 -05:00
David Steele
86fee3da18 Continuous integration using travis-ci. 2017-02-21 08:59:23 -05:00