1
0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2024-12-12 10:04:14 +02:00
Commit Graph

37 Commits

Author SHA1 Message Date
David Steele
ff98636e41 Update 32-bit CI host to Debian 10 from Ubuntu 18.04.
Ubuntu 18.04 will be EOL before the next release, so update to the oldest available Debian version.

Also fix one incorrect return value type, a test cast, and adjust some test timeouts.
2023-04-06 08:22:14 +04:00
David Steele
295b53e845 Update Github actions to new versions.
These updates remove various deprecation warnings.
2023-03-28 16:24:43 +06:00
David Steele
ce0ea2cfab Use uncrustify for code formatting.
The code is not completely reflowed yet so there are some cases that uncrustify will not catch. The formatting will be improved over time.

Some block of code require special formatting so have been surrounded with the {uncrustify-off}/{uncrustify-on} markers. These exceptions should be kept to a minimum.

Add --code-format (to reformat code) and --code-format-check (to check formatting) to test.pl.

Add a CI test that will check code formatting. Code must be correctly formatted before it can be merge to integration.

Add documentation to the coding standards for code formatting.
2023-01-30 12:22:41 +07:00
David Steele
a28f3d49c2 Add --no-back-trace option to test.pl.
Running valgrind and backtrace together has been causing tests to timeout in CI, mostly likely due to limited resources. This has not been a problem in normal development environments.

Since it is still important to run backtraces for debugging, split the u22 test that was doing all this work to run coverage and backtrace together and valgrind-only as a separate test. As a bonus these tests run faster separately and since they run in parallel the total execution time is faster.
2023-01-28 18:55:53 +07:00
David Steele
c826429123 Improve libbacktrace implementation.
The libbacktrace feature has not been working since the move to meson because libbacktrace detection was not added to the meson build. Add libbacktrace to meson and improve the feature so that it can be compiled into release builds.

The prior implementation fetched line numbers with each stack trace push. Not only was this slow but it missed any functions that were not being tracked on our stack.

Instead just examine the backtrace when an error happens and merge it with the info we have on our stack. If the backtrace is not available then the output remains as before.

Also remove --backtrace from test.pl since the library is now auto-detected.

Leave this library out of the production build for now to give it a little time to shake out in testing.
2023-01-11 11:19:26 +07:00
David Steele
f018912908 Split VR_EXTERN/FN_EXTERN macros from FV_EXTERN.
This should make it a little clearer what the variable (VR) macros are doing since the declaration/definition cannot both be set to extern (but functions can).

Splitting the variable macros out also allows them to be changed in the future with little churn, while changing the function macro creates a large amount of churn.
2023-01-02 15:24:51 +07:00
David Steele
4fb8a0ecdd Add meson unity build and tests.
This is immediately useful because it will detect any extern'd functions or variables that are not being used. It also detects functions or variables that are declared but not defined.

If a FV/VR_EXTERN macro is missing it will be detected either because of a mismatch in the declaration/definition or because a new defined symbol will appear in the nm test.

Eventually the unity build will be used to create a more optimized pgbackrest binary but that will need to wait.
2022-12-31 17:13:41 +07:00
David Steele
eb287b18c8 Add profiling, performance, and optimization to C test harness.
All unit and performance tests are now built by the C harness.

Remove all unit/performance test build code from Perl.

Remove code from C harness that is no longer used. This code was included so the C harness could be run separately, but that is no longer needed with this full integration.
2022-07-29 10:31:36 -04:00
David Steele
c99ea54f17 Integrate C test harness with Perl test harness.
The C test harness is used for unit tests from the Perl harness where possible. Currently, unit tests can be run in the C harness when --no-coverage is specified and --profile is not specified.

C harness tests work on meson 0.45.

The C harness runs with valgrind by default. Valgrind can be disabled with --no-valgrind.

Also rebuild containers to add meson and update the documentation so that meson builds will work (even though we don't do them yet).
2022-07-27 10:32:32 -04:00
David Steele
19d9941367 More C test harness improvements and CI.
Build pgbackrest binary and auto-generated code automatically.

Remove --module option and allow modules to run by parameter. This is less verbose and multiple modules can be run at a time.

Allow filtering of modules. Multiple tests can be passed as parameters and if the module ends in / it will be used as a prefix filter. For example, common/ will run all the common modules.

If a test errors the remaining tests will still run but the test process will eventually exit with an error.

CI tests are included but unit tests remain on the development branch.

With these changes all unit tests run except those that specify the define (e.g. common/assert-off) or containerReq (e.g. protocol/protocol) keywords.

Building the C test harness has been simplified:

meson -Dwerror=true -Dfatal-errors=true -Dbuildtype=debug test/build/none pgbackrest
ninja -C test/build/none test/src/test-pgbackrest

To run all modules:

test/build/none/test/src/test-pgbackrest test

Just the common/error module:

test/build/none/test/src/test-pgbackrest test common/error

All info modules:

test/build/none/test/src/test-pgbackrest test info/
2022-07-21 20:10:51 -04:00
David Steele
a16cf5eac7 Update CI to use Ubuntu 22.04 and Fedora 36.
Both have newer gcc and OpenSSL 3.

Fedora 36 runs horribly slow with valgrind enabled so run the valgrind tests on Ubuntu 22.04. Fedora 36 has a newer gcc so it is still worth testing on.
2022-06-06 16:32:20 -04:00
David Steele
f92ce674f7
Automatically create PostgreSQL version interfaces.
Maintaining the version interfaces was complicated by the fact that each interface needed to be in separate compilation unit to avoid type conflicts. This also meant that various build/test files needed to be updated to add the new interfaces.

Solve these problems by auto-generating all the interfaces into a single file. This is made possible by parsing defines and types out of the header files and creating macros to rename the types. At the end of the version interface everything is undef'd. Another benefit is that the auto-generated interfaces can be static and included directly into postgres/interface.c.

Since some code generation is now always required for tests, change --no-gen to --min-gen in test.pl.

It would also make sense to auto-generate the version defines in postgres/version.h, but that will be left for a future commit.
2022-06-06 13:52:56 -04:00
David Steele
b8fc20d5b8
Add experimental Meson build.
Meson is a new build system that offers simpler syntax and superior performance to autoconf/make. In addition, Windows is supported natively.

The Meson build appears complete, but currently is used only for auto-generation of code and the host build of pgbackrest. Some container upgrades will be required before Meson can be used for container builds.

Also patch the Debian package to force autoconf/make rather than Meson.
2022-06-03 14:13:56 -04:00
David Steele
be354c4890 Update CodeQL to version 2.
Version 1 will be deprecated on December 12, so upgrade now to avoid nasty surprises.
2022-05-31 16:52:26 -04:00
David Steele
cb891fa2d4 Run remaining Github Actions CI on Ubuntu 20.04.
The s390x/ppc64le tests are already running on 20.04.
2022-05-31 13:02:51 -04:00
David Steele
7d9b2e267c Move 32-bit CI testing from Debian 9 to Ubuntu 18.04.
Debian 9 will be EOL before our next release, so move 32-bit testing to Ubuntu 18.04, which is older than Debian 10.
2022-05-12 11:57:12 -04:00
David Steele
b4c1ca7b80 Split 32-bit CI tests.
This helps rebalance some of the tests that are running long, i.e. d9 and u20.

I would be better to move more PostgreSQL versions to d9, but the base VM does not contain more versions. New minor versions will be out later in the week so that seems a better time to be rebuilding containers.
2022-05-09 14:19:05 -04:00
David Steele
39dddbb6bc Add limited CI for ppc64le/s390x using emulation.
The emulation is so slow that running all the unit tests would be too expensive, but this at least shows that the build works and some of the more complex tests run. In particular, it is good to test on one big-endian architecture to be sure that checksums are correct.

Update checksums in the tests where they had gotten out of date since the last time we were testing on s390x. Also use a different test in command/archivePushTest to show the name of the file when a checksum does not match to aid in debugging.

The command/archive-push test was updated but not included because there is also a permissions issue, which looks to be the same as what we see on MacOS/FreedBSD. Hopefully we'll be able to fix all of those at the same time.
2022-05-09 12:48:19 -04:00
David Steele
8e849ae85d Add PRs to thread locking on Github actions.
It also makes sense to lock old PRs. They can be manually unlocked if they are needed for some reason.

Also add output logging to make it easier to determine if thread locking is completing.
2022-05-04 10:28:39 -04:00
David Steele
09b387fccd Move issue locking to Github actions.
The old plugin has been defunct for some time so there are currently a lot of unlocked issues.

Running this once per week seems sufficient for now. Worst case it can be run manually if it gets behind.
2022-05-04 09:24:35 -04:00
David Steele
c32e000ab9 Use Rocky Linux for documentation builds instead of CentOS.
Since CentOS 8 will be EOL at the end of the year it makes sense to do this now. The centos:8 image is still used in documentation.xml because changes there require manual testing, which will need to be done at a later date. The changes are not user-facing, however, and can be done at any time.

Also update CentOS references to RHEL since that is what we are emulating for testing purposes.
2021-10-28 15:15:49 -04:00
David Steele
360cff94e4 Update 32-bit test container to Debian 9.
Also rebalance PostgreSQL version integration tests.
2021-10-16 12:33:31 -04:00
David Steele
430efff98a Update documentation/links to main branch. 2021-10-13 12:01:53 -04:00
Cynthia Shang
e3d05e2b4a
Update contributing documentation and add pull request template. 2021-07-21 13:14:06 -04:00
David Steele
c5ae047e76 Partial migration of config code generation to C.
Parse enough of config.yaml to auto-generate config.auto.h and config.auto.c.

This commit implements most of the infrastructure needed to migrate the rest of the build code to C, but each set of auto-generated files will present its own challenges.

The build is now dependent on libyaml. At this point there is no need for a hard requirement, but that will come soon so it seems better to add the dependency now.
2021-07-18 19:02:01 -04:00
David Steele
6397d73535 Update test container OS versions.
Update Ubuntu 12.04 to 16.04. Version 16.04 is recently EOL but testing on an old version is beneficial.
Update Ubuntu 18.04 to 20.04.
Update Fedora 32 to 33. Version 34 would have been preferred but there were some build issues, i.e. the default shell did not work with configure, and after ksh was installed configure locked up.

Add --no-install-recommends to apt-get commands to save a bit of time and space.

Update test Dockerfile to run in multiple steps. This makes the container larger but also makes rebuilding after changes faster. The --squash option may be used to keep the container small.

Remove obsolete casts in protocol/parallel module. These casts were included in the original migration because Ubuntu 12.04 32-bit gcc required them, but Ubuntu 16.04 32-bit gcc complains. There is no production issue here since at this point in the code the file descriptors are guaranteed to be >= 0.
2021-07-18 17:29:20 -04:00
David Steele
3c8819e10f
Add CodeQL static code analysis.
Also fix some minor issues identified, specifically using gmtime_r()/localtime_r() vs gmtime()/localtime().
2021-07-09 14:16:10 -04:00
David Steele
87ba2ca253 Change CentOS 7 documentation test to CentOS 8.
The CentOS 7 documentation test relies on PostgreSQL 9.5 which has been removed from the yum.p.o repository package. Switch the test to CentOS 8 to fix the immediate issue, but a decision on the PostgreSQL 9.5 documentation will need to be made before the next release.
2021-05-11 16:54:42 -04:00
David Steele
0eb5b22ffa Add back performance tests lost during Github Actions switch.
These tests do not provide code coverage but testing them regularly is important so they work when needed.
2021-03-15 17:24:58 -04:00
David Steele
a7491182dc Set build-max to 2 for Github actions.
Github runners only provide 2 virtual CPUs so the default build-max of 4 is not ideal.
2021-01-04 09:18:17 -05:00
David Steele
715fa7a2f1 Fix indentation error preventing pull requests from being tested. 2020-12-28 08:51:22 -05:00
David Steele
d01669aa58 Move most tests to Github Actions.
Testing on Travis-CI has been getting slower (from ~18 minutes to 3-6 hours) and the travis-ci.org service will be terminated at the end of the year. Moving to travis-ci.com is an option but the quotas are too low for our purposes.

Instead use Github Actions, which does not currently have quotas, and runs our current tests with just a few tweaks.

This still leaves multi-architecture tests on Travis-CI but we may be able to run those and stay within the new quotas.

Also fix a minor bug in restoreTest.c exposed by Github Actions using a different name for the user and group.
2020-12-09 15:19:01 -05:00
David Steele
1fa3ae2fcd Prevent lock-bot from marking locked issues as "resolved".
This is certainly not true in all cases, e.g. an issue may be closed if it is added to the backlog.
2020-02-11 19:54:17 -07:00
David Steele
1be9e6854e Prevent lock-bot from adding comments to locked issues.
This will hopefully prevent users from getting notifications when an issue is locked.
2020-02-11 19:52:23 -07:00
David Steele
62f0c7fb37 Add lock bot configuration.
Lock closed issues after ninety days to prevent posting on old issues.
2019-07-22 13:09:51 -04:00
Cynthia Shang
f75ba7db94 Update backup host to repository host in the documentation.
Contributed by Cynthia Shang.
2018-02-19 13:17:58 -05:00
Cynthia Shang
bc8543494a Add template to improve initial information gathered for issue submissions.
Contributed by Cynthia Shang.
2017-11-15 11:59:27 -05:00