1
0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2024-12-14 10:13:05 +02:00
Commit Graph

2621 Commits

Author SHA1 Message Date
David Steele
3ec630f5b2 Allow suppression of times in testing for reproducibility.
Timestamps, timings, etc. cause a lot of log churn when included in documentation.
2020-03-22 10:12:29 -04:00
David Steele
a180251011 Do variable replacements for cmd-extra in documentation. 2020-03-22 09:59:38 -04:00
David Steele
6a9fbccf76 Remove custom lcov build from contributing documentation.
Using the stock lcov no longer seems to be a problem, probably due to a better implementation in CoverageTest.pm.
2020-03-21 19:08:27 -04:00
David Steele
56fb399373 Build contributing documentation on Travis CI.
Building the contributing document has some special requirements because it runs Docker in Docker so the repo path must align on the host and all Docker containers.  Run `pgbackrest/doc/doc.pl` from within the home directory of the user that will do the doc build, e.g. `home/vagrant`.  If the repo is not located directly in the home directory, e.g. `/home/vagrant/pgbackrest`, then a symlink may be used, e.g. `ln -s /path/to/repo /home/vagrant/pgbackrest`.

Mount the repo in the Vagrantfile at /home/vagrant/pgbackrest but provide a link from the old location at /backrest to make the transition less painful.
2020-03-21 18:45:58 -04:00
David Steele
ee2e15bf55 Add missing package to contributing documentation. 2020-03-21 09:51:12 -04:00
David Steele
f405c82dcc Don't list directories as changed from the last run.
It's not very useful information and just clutters the list.
2020-03-20 15:00:20 -04:00
David Steele
cfab67a775 Enable coverage testing on Fedora 30.
Now that coverage testing works reliably with gcc9 it makes sense to enable it for CI.
2020-03-20 13:49:23 -04:00
David Steele
782c9f89f4 Remove old coverage data before starting new test.
The old coverage data has been recorded so it is no longer needed. In newer versions of gcc leaving this file around can lead to an error when writing profile data after forking off to a non-pgbackrest binary (which we do in some unit tests).
2020-03-20 13:43:08 -04:00
David Steele
787d3fd67b Improve custom coverage report.
* Show all uncovered branch parts even when there are more than two parts per branch. This is the way gcc9 reports coverage so it needs to work even if it doesn't make as much sense as the old way.

* Show covered branches in functions where coverage is missing.  Showing just the uncovered branches can be confusing because it's not always clear how the coverage relates to the code.  By showing all branch coverage (+ or -) this correspondence is made easier.
2020-03-20 12:54:29 -04:00
David Steele
8af8029006 Fix lcov report when test module missing coverage.
We don't report branch coverage on test modules (e.g. test/src/module/common/errorTest.c) but the code that excluded branch coverage from the test module would also exclude it from all core modules if the test module was included in the lcov report due to lack of function/line coverage.

Adjust the coverage code to only exclude branches during the extraction of test module coverage.
2020-03-20 09:16:38 -04:00
David Steele
f6e9bb0819 Remove obsolete -O2 option for Fedora 30 unit test builds.
For some reason gcc9 would not do -O0 builds in combination with one of the options that libperl required.  Now that libperl is gone this exception is no longer required.
2020-03-19 19:30:09 -04:00
David Steele
2241524c0b Remove obsolete deletes from Vagrantfile.
pgBackRest no longer writes files into the .vagrant path so there's no longer anything to delete.
2020-03-19 18:34:10 -04:00
David Steele
dcddf3a58b Limit backup file copy size to size reported at backup start.
If a file grows during the backup it will be reconstructed by WAL replay during recovery so there is no need to copy the additional data.

This also reduces the likelihood of seeing torn pages during the copy. Torn pages can still occur in the middle of the file, though, so they must be handled.
2020-03-19 13:16:05 -04:00
David Steele
e5bcc0c47e Clarifications of what limit == NULL or UINT64_MAX means. 2020-03-19 12:34:27 -04:00
Cynthia Shang
73315268fd Fix typo. 2020-03-19 12:11:20 -04:00
David Steele
d677b07081 Move coverage code to CoverageTest module.
This code needs some work, which will be easier if it is all in one place.
2020-03-19 12:07:51 -04:00
Cynthia Shang
f538da0571 Updates to contributing documentation. 2020-03-18 17:30:36 -04:00
David Steele
26c89b2c8c Improve testing of files that change size during the backup.
Files can change size during a backup so update and add tests to cover the various scenarios more thoroughly.
2020-03-18 13:40:16 -04:00
David Steele
4ec04e5163 Added redacted manifest to testBackupValidate().
The manifest is excellent for validation but including the entire manifest is too noisy and some values are architecture/algorithm dependent.

Output a redacted version that contains the most important information which can be improved on over time.
2020-03-18 10:10:10 -04:00
David Steele
b8cd1b6790 Add TEST_RESULT_STR_Z_KEYRPL() test macro.
This macro will automatically do key replacement before the comparison. This saves the indentation required for an embedded function call.

Possibly TEST_RESULT_Z_KEYRPL() would also be useful but it will be added when needed.
2020-03-18 10:05:08 -04:00
David Steele
f2548f45ce Allow storage reads to be limited by bytes.
The current use case is reading files from the PostgreSQL cluster during backup.

A file may grow during backup but we only need to copy the number of bytes that were reported during the manifest build.  The rest will be rebuilt from the WAL during recovery so copying more is just a waste of space.

Limiting the copy sizes in backup will be part of a future commit.
2020-03-17 18:16:17 -04:00
David Steele
307e741298 Test that shrunk file is backed up correctly.
It's possible, though rare, for a file to shrink during a backup.

There was no issue with the code but having a test is always a good idea.
2020-03-17 16:01:17 -04:00
David Steele
7f1817c8ad Fix typos. 2020-03-17 11:43:38 -04:00
David Steele
9a47b88da3 Add links to custom coverage report.
When multiple files were missing coverage it could be hard to locate the coverage report for a specific file.

Add links for uncovered files to make this easier.

Also move table titles out of the table so they are valid html.
2020-03-16 20:02:36 -04:00
David Steele
d6cea422a0 Make cfgLoadLogSetting() static.
This function was once required by the test harness.
2020-03-16 18:18:30 -04:00
David Steele
f7dac144a6 Reduce variables extern'd by the common/log module in debug builds.
These days it is better to include the module in define.yaml when we need to poke at the internal implementation.

This doesn't quite work for the log test harness, so for now some variables will need to remain extern'd in debug builds.
2020-03-16 18:16:27 -04:00
David Steele
3fbfcba811 Forbid access to /tmp/pgbackrest in the Vagrantfile.
This matches the error that will be thrown in the vm=none test on Travis CI if a unit test writes to /tmp/pgbackrest.
2020-03-16 17:27:01 -04:00
David Steele
46911c64c1 Make storage and logging dry-run aware.
Enhance dry-run support added in 2fa69af8 by forbidding writes in the storage layer and adding prefixes to log messages.

The former will protect against mistakes in dry-run implementations and the latter will make it clear when a command was executed in dry-run mode.

Update expire unit tests with the new log prefix.
2020-03-16 17:24:21 -04:00
Cynthia Shang
2fa69af8da Add --dry-run option to the expire command.
Use dry-run to see which backups/archive would be removed by the expire command without actually removing anything.
2020-03-16 13:56:52 -04:00
Cynthia Shang
4608ca473e Fix indentation. 2020-03-16 12:50:37 -04:00
David Steele
4328bc1ac6 Move raw coverage results to test/result/raw path.
These results were stored in the vagrant path along with a full copy of src.

Instead store the raw coverage data in test/result/raw and change source references to the files that already exist in [test-path]/repo.
2020-03-16 08:41:32 -04:00
David Steele
d702249507 Build binaries in the test path rather than the vagrant path.
It makes more sense to build in the test path since many developers won't have a vagrant path. Anyway, it's better not to modify the vagrant path since it belongs to vagrant.

Instead of installing the binary just mount it into the container from where it was built. This saves a bit of time and space.
2020-03-15 10:09:27 -04:00
David Steele
19d975346b Improve stability of command/check test module.
When pgbackrest was present this test behaved unexpectedly.

While the binary is not currently required for this test is might be in the future so fix the test to prevent a regression.
2020-03-15 09:59:22 -04:00
David Steele
da89d0ad61 Add VPATH to Makefile.
This allows builds in a separate directory without copying the source.
2020-03-15 09:53:42 -04:00
David Steele
959dce569b Update code classification and remove XS definition. 2020-03-14 18:30:24 -04:00
David Steele
213cc6e8be Move docker files to test/result. 2020-03-14 15:40:37 -04:00
David Steele
6827e248cd Move coverage results to test/result. 2020-03-14 15:29:42 -04:00
David Steele
75ff25f17f Move profile results to test/result. 2020-03-14 14:50:36 -04:00
David Steele
0f7fe55f72 Build packages on demand only and change build path.
Building packages is not a normal part of development so don't build packages by default. Instead build them in CI as needed.

Do the builds in test/result instead of .vagrant to be friendlier with hosts that are not running vagrant. Anyway, it's probably not a good idea to be creating files in the .vagrant path.
2020-03-14 14:35:09 -04:00
David Steele
5645c91ed5 Add comments to test/.gitignore. 2020-03-14 14:18:22 -04:00
David Steele
b4c94e42b9 Auto-detect presence of clobbered warning.
Not all compilers (e.g. clang) support this warning so only disable it (i.e. -Wno-clobbered) when supported.
2020-03-14 13:06:21 -04:00
David Steele
4cd060b7fe Generate src/build/aclocal.m4 automatically.
This file is required when macros from the autoconf archive are used in configure.ac
2020-03-14 12:48:08 -04:00
David Steele
9e80c5710e Use a checksum to build configure.ac more efficiently.
Building the configure.ac script can take multiple seconds depending on the state of the autoconf cache. Use a checksum to only rebuild when configure.ac has changed no matter how the timestamps have changed.
2020-03-14 12:39:29 -04:00
David Steele
748f9502eb Remove obsolete ignore. 2020-03-14 10:04:49 -04:00
David Steele
237a3da4d6 Configure and make improvements.
Configure:

* Use standard make variables, e.g. CFLAGS, rather than our own, e.g. CINCLUDE
* Add PG_CONFIG var for configuring custom pg_config location
* Don't error if xml_config or pg_config is missing (but error if libs/headers not found)
* Check for zlib.h header
* Check for lz4frame.h header when liblz4 is present

Make:

* Use gcc-style auto dependencies
* Put src list at the top since it is most frequently modified
* Add clean-all target to also remove auto-generated config files
2020-03-13 09:07:57 -04:00
David Steele
810b2a5265 Remove obsolete ignore. 2020-03-12 13:18:08 -04:00
Marc Cousin
4b24a74afb Fix detection of keepalive options on Linux.
This code stanza was not being included on Linux platforms because of a missing header file.

Also update the order of operations and make the timeout calculations more sensible.
2020-03-12 11:38:50 -04:00
David Steele
e9c1569d8e Fix missing cast.
This was a problem in testing when -Wconversion was enabled and TCP_KEEPIDLE was defined.
2020-03-12 10:48:40 -04:00
Marc Cousin
d6ef6c57c3 Add configure host detection to set standards flags correctly.
Linux and MacOS were tolerant of having all standards flags set but BSD was not.

Instead detect the host type and set standards flags as appropriate.
2020-03-12 10:37:19 -04:00
David Steele
fa1f63198d Add scripts required for host detection by src/configure.
Add to the src/build directory so they will be out of the way.

Also add instructions for updating the scripts to RELEASE.md.
2020-03-12 10:03:57 -04:00