1
0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2024-12-14 10:13:05 +02:00
Commit Graph

256 Commits

Author SHA1 Message Date
David Steele
cc6447356e Fix test binary name for gprof.
This got missed in 1f8931f7 when the test binary was renamed.

Also output call graph along with the flat report.  The flat report is generally most useful but it doesn't hurt to have both.
2018-12-05 09:15:45 -05:00
David Steele
56ce98b2f0 Explicitly compile with Posix 2001 standard.
This standard was being selectively applied in modules that needed it.

Instead, apply the standard to all compilation for consistency.
2018-11-25 10:06:31 -05:00
David Steele
315aa2c451 Conditional compilation of Perl logic in exit.c.
This file is the only one to contain Perl logic outside of the perl module.  Make the Perl logic conditional to improve reusability.
2018-11-25 08:39:41 -05:00
David Steele
78fe642eae Remove extraneous use/include statements.
Use conditional loading to make docs work in the absence of LibC.

Somehow this also required a use statement to be added. Perl, go figure.
2018-11-24 20:31:35 -05:00
David Steele
801e2a5a2c Rename PGBACKREST/BACKREST constants to PROJECT.
This brings consistency between the C and Perl constants and allows for easier code reuse.
2018-11-24 19:05:03 -05:00
David Steele
ac426bc456 New test containers with static test certificates.
Test certificates were generated dynamically but there are advantages to using static certificates.  For example, it possible to use the same certificate between container versions.  Mostly, it is easier to document the certificates if they are not buried deep in the container code.

The new test certificates are initially intended to be used with the C unit tests but they will eventually be used for integration tests as well.

Two new certificates have been defined. See test/certificate/README.md for details.

The old dynamic certificates will be retained until they are replaced.
2018-11-21 18:13:37 -05:00
David Steele
bc25db5667 Add interface objects for libxml2.
Add XmlDocument, XmlNode, and XmlNodeList objects as a thin interface layer on libxml2.

This interface is not intended to be comprehensive. Only a few libxml2 capabilities are exposed but more can be added as needed.
2018-11-20 20:40:11 -05:00
David Steele
f743d4e924 Add testRepoPath() to let C unit tests know where the code repository is located.
This allows a C unit test to access data in the code repository that might be useful for testing.

Add testRepoPathSet() to set the repository path.

In passing remove extra whitespace in the TEST_RESULT_VOID() macro.
2018-11-20 15:48:56 -05:00
David Steele
8c7e97a369 Clarify comment about main.c being excluded from unit testing.
Also remove !!! which by convention we use as a marker for code that needs attention before it can be committed to master.
2018-11-14 08:08:26 -05:00
David Steele
acb579c469 Tighten limits on code coverage context selection.
If the last } of a function was marked as uncovered then the context selection would overrun into the next function.

Start checking context on the current line to prevent this.  Make the same change for start context even though it doesn't seem to have an issue.
2018-11-13 10:37:58 -05:00
David Steele
7107cc68d2 Expand context shown in coverage and update colors.
Too few lines were shown for coverage context so show the entire function if it has any missing coverage.

Update colors to work with light and dark browser modes.
2018-11-12 18:11:16 -05:00
David Steele
22ecbc153a New, concise coverage report for C.
The report HTML generated by lcov is overly verbose and cumbersome to navigate. Since we maintain 100% coverage it's far more interesting to look at what is not covered than what is.

The new report presents all missing coverage on a single page and excludes code that is covered for brevity.
2018-11-11 17:32:42 -05:00
David Steele
3e695af961 New test containers.
* Add libxml2 library needed for S3 development.
* Minor version updates for PostgreSQL.
* Remove PostgreSQL 11 beta/rc repository.
2018-11-08 21:41:41 -05:00
David Steele
57d7809297 Improve efficiency of code generation.
Code generation saved files even when they had not changed, which often caused code generation cascades. So, don't save files unless they have changed.

Use rsync to determine which files have changed since the last test run. The manifest of changed files is saved and not removed until all code generation and builds have completed. If an error occurs the work will be redone on the next run.

The eventual goal is to do all the builds from the test/repo directory created by rsync but for now it is only used to track changes.
2018-11-03 19:52:46 -04:00
David Steele
1f8931f732 Improve single test run performance.
Improve on 7794ab50 by including the build flag files directly into the Makefile as dependencies (even though they are not includes).  This simplifies some of the rsync logic and allows make to do what it does best.

Also split build flag files into test, harness, and build to reduce rebuilds.  Test flags are used to build test.c, harness flags are used to build the rest of the files in the test harness, and build flags are used for the files that are not directly involved in testing.
2018-11-03 16:34:04 -04:00
David Steele
7794ab50dc Preserve contents of C unit test build directory between test.pl executions.
The contents were already preserved between tests in a single test.pl run but for a separate execution the entire project had to be built from scratch, which was getting slower as we added code.

Save the important build flags in a file so the new execution knows whether the build contents can be reused.
2018-11-02 11:56:13 -04:00
David Steele
9ae3d8c46a Install nodejs from deb.nodesource.com.
The standard npm packages on Ubuntu 18.04 suddenly required libssl1.0 which broke the pgbackrest package builds. Installing nodejs from deb.nodesource.com seems to work fine with standard libssl.

This package is required by ScalityS3 which is used for local S3 testing.
2018-10-15 23:13:08 +01:00
David Steele
e55d733041 Add -ftree-coalesce-vars option to unit test compilation.
This is a workaround for inefficient handling of many setjmps in gcc >= 4.9.  Setjmp is used in all error handling, but in the unit tests each test macro contains an error handling block so they add up pretty quickly for large unit tests.

Enabling -ftree-coalesce-vars in affected versions reduces build time and memory requirements by nearly an order of magnitude.  Even so, compiles are much slower than gcc <= 4.8.

We submitted a bug for this at: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=87316
Which was marked as a duplicate of: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63155
2018-09-17 11:38:10 -04:00
David Steele
4119ce208d Move test expect log out of the regular test directory.
Storing the expect log (created by common/harnessLog) in the regular test directory was not ideal.  It showed up in tests and made it difficult to clear the test directory between each run.

Move the expect log to a purpose-built directory one level up so it does not interfere with regular testing.
2018-09-16 15:58:46 -04:00
David Steele
f0ed89f21f Allow C or Perl coverage to run on more than one VM.
C or Perl coverage tests can now be run on any VM provided a recent enough version of Devel::Cover or lcov is available.

For now, leave u18 as the only VM to run coverage tests due to some issues with older versions of lcov.
2018-09-15 13:27:06 -04:00
David Steele
31cdd9d20b Remove compiler warnings that are not valid for u16. 2018-09-15 08:23:55 -04:00
David Steele
aeb1fa3dfb Don't perform valgrind when requested.
The --no-valgrind flag was not being honored.  It's not clear if this flag ever worked, but it does now.
2018-09-13 19:12:40 -04:00
David Steele
f7fc8422f7 Make Valgrind return an error even when a non-fatal issue is detected.
By default Valgrind does not exit with an error code when a non-fatal error is detected, e.g. unfreed memory.  Use the --error-exitcode option to enabled this behavior.

Update some minor issues discovered in the tests as a result.  Luckily, no issues were missed in the core code.
2018-09-07 16:50:01 -07:00
David Steele
8a8738308c Enable -Wvla. 2018-08-22 14:48:37 -04:00
David Steele
6643afe9a8 Add gzip compression/decompression filters for C. 2018-08-14 14:56:59 -04:00
David Steele
31167d8f98 Enable -Wduplicated-branches, and -Wduplicated-cond. 2018-08-05 08:35:18 -04:00
David Steele
14f21a8f6c Enable -Wpointer-arith. 2018-08-04 19:37:28 -04:00
David Steele
429a356e33 Enable -Wstrict-prototypes and update all void functions to conform. 2018-08-03 19:19:14 -04:00
David Steele
666537cbde Update code count for new file types and exclusions. 2018-07-28 08:13:30 -04:00
David Steele
1862630629 Use pre-built images from Docker Hub when the container definition has not changed.
Downloading an image is quite a bit faster than building a new image from scratch and saves minutes per test run in CI.
2018-07-21 17:02:42 -04:00
David Steele
8568622a6f Add --log-level-test option.
This allows setting the test log level independently from the general test harness setting, but current only works for the C tests. It is useful for seeing log output from functions on the console while a test is running.
2018-07-20 19:03:46 -04:00
David Steele
5dc8a2ec08 storageFileRead() accepts a buffer for output rather than creating one.
This is more efficient overall and allows the caller to specify how many bytes will be read on each call. Reads are appended if the buffer already contains data but the buffer size will never increase.

Allow Buffer object "used size" to be different than "allocated size". Add functions to manage used size and remaining size and update automatically when possible.
2018-07-17 19:01:54 -04:00
David Steele
22adb395db Add zero-length file to mock/all test. 2018-07-05 15:40:50 -04:00
David Steele
d55e609959 Use a prebuilt s3 server container for documentation and tests. 2018-06-12 13:43:15 -04:00
David Steele
835396751f Document generator improvements.
* Build containers from scratch for more accurate testing.
* Allow environment load to be skipped.
* Allow bash wrapping to be skipped.
* Allow forcing a command to run as a user without sudo.
2018-06-10 14:13:56 -04:00
David Steele
a385cb520b Update primary test environment (Vagrant and Docker) to Ubuntu 18.04. 2018-06-06 15:52:28 -04:00
David Steele
d309a85b51 PostgreSQL 11 Beta 1 support. 2018-06-05 08:59:17 -04:00
David Steele
e57840d739 Document generator improvements.
* Allow parameters to be passed when a container is created.
* Allow /etc/hosts update to be skipped (for containers without bash).
2018-05-25 13:42:09 -04:00
David Steele
40093f160c Improve efficiency of C library builds now that they are used only for testing. 2018-05-24 14:01:24 -04:00
David Steele
8eb9921c7c Auto-generate Makefile with dependencies. 2018-05-23 14:57:08 -04:00
David Steele
f7123c1bbf v2.03: Single Executable to Deploy
Bug Fixes:

* Fix potential buffer overrun in error message handling. (Reported by Lætitia.)
* Fix archive write lock being taken for the synchronous archive-get command. (Reported by Uspen.)

Improvements:

* Embed exported C functions and Perl modules directly into the pgBackRest executable.
* Use time_t instead of __time_t for better portability. (Suggested by Nick Floersch.)
* Print total runtime in milliseconds at command end.
2018-05-22 19:53:25 -04:00
David Steele
4c7408ea52 Embed exported C functions and Perl modules directly into the pgBackRest executable. 2018-05-22 12:53:08 -04:00
David Steele
52bc073234 Add stack trace macros to all functions.
Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 11:57:32 -04:00
David Steele
abb9651f4c Test harness improvements.
* Allow more than one test to provide coverage for the same module.
* Add option to disable valgrind.
* Add option to disabled coverage.
* Add option to disable debug build.
* Add option to disable compiler optimization.
* Add --dev-test mode.
2018-05-18 06:45:14 -04:00
David Steele
790f7c7312 Enforce full branch coverage in C code. 2018-05-05 11:50:11 -04:00
David Steele
88a633da17 Divide tests into three types (unit, integration, performance).
Many options that were set per test can instead be inferred from the types, i.e. container, c, expect, and individual.

Also finish renaming Perl unit tests with the -perl suffix.
2018-04-24 09:12:25 -04:00
David Steele
dce11ce3cf Generate code counts for all source files.
The source files are also classified by type and purpose.
2018-04-16 15:52:17 -04:00
David Steele
3e22977484 Use pip 9.03 in test VMs.
pip 10 drops support for Python 2.6 which is still used by the older test VMs.
2018-04-16 15:25:50 -04:00
David Steele
f0250dab4b Move async forking and more error handling to C.
The Perl process was exiting directly when called but that interfered with proper locking for the forked async process. Now Perl returns results to the C process which handles all errors, including signals.
2018-04-12 20:42:26 -04:00
David Steele
741acfd779 Move test definitions to test/define.yaml.
The location is better because it is no longer buried in the Perl test libs. Also, the data can be easily accessed from C.
2018-04-08 10:19:24 -04:00