1
0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2025-07-07 00:35:37 +02:00
Commit Graph

132 Commits

Author SHA1 Message Date
3780057866 Update Fedora test image to Fedora 42. 2025-06-22 17:46:54 -04:00
8aa2d88bac Update unit tests to work with Alpine Linux and musl libc.
A few bugs have been reported over the years on Alpine with musl libc and generally it has been possible to figure them out without testing on that platform but a few newer ones cannot be reproduced elsewhere.

Also, testing with additional libc implementations helps portability so it makes sense to add support.

For the most part musl libc works as expected with a few caveats:

1) The FD_*() macros won't accept an int file descriptor without warning when -Wsign-conversion is enabled. I opened https://www.openwall.com/lists/musl/2025/05/23/4 to discuss this and I was referred to https://www.openwall.com/lists/musl/2024/07/16/1 which explains why this happens. It was not a very satisfying answer but clearly it is not going to be addressed so a meson probe was added to detect the issue and disable the warning.

2) Floating point numbers are rounded differently than in GNU/Mac libc when formatted with printf() and friends. This is fine for the core code but causes issues in the unit tests that expect log entries to match exactly. This was solved in ad7ba46b by adding our own rough and ready formatting routines.

3) Some error messages are different from GNU/Mac libc. This was solved with a new error macro that accepts multiple messages in b5fbb16c.

4) For some reason ninja on Alpine outputs "nothing to do" messages to stderr whereas they go to stdout on other distributions. Redirecting stderr to stdout is our standard fix for this issue so do that. A non-zero result code will let us know that something has gone wrong.

5) It appears that profiling is not supported on Alpine, which is pretty surprising. For now fix this by only unit testing the profiling code when coverage is enabled. This is not a great solution since we would rather test profiling on any system that supports it but for now this will do.
2025-06-05 17:33:18 -04:00
f494dd21ad Fix architecture labeling for multi-architecture container builds.
When the architecture was not specified it was being set to x86_64 rather than the host architecture.
2025-05-27 17:14:18 -04:00
7b1b1f5322 New CI container build for PostgreSQL 18 beta1.
Since the addition of libcurl4-openssl-dev requires a rebuild of the Debian containers go ahead and rebuild all containers to include new PostgreSQL minor release versions.
2025-05-09 17:45:05 -04:00
34b916f323 Use 'x86_64' instead of 'amd64' for container testing.
This aligns with what is reported by 'uname -m' and required by Docker naming conventions.

Also convert a 'aarch64' string to the VM_ARCH_AARCH64 constant.
2025-05-09 12:56:57 -04:00
565d2e0c47 PostgreSQL 18 experimental support.
Support is experimental since PostgreSQL 18 is still in development and has not released a beta, but it may be useful for testing.
2025-04-14 12:29:31 -05:00
57ffd2dfcb Fix multi-architecture unit testing.
The Github action we were using for multi-architecture testing stopped working. The project does not seem to be getting regular maintenance so it seems better to roll multi-architecture testing into our existing container builds.

Introduce multi-architecture builds and testing into our test framework. For now this only works for unit tests -- integration tests will still only run on x86_64. That could be updated in the future but since emulation is so slow it is not clear if it would be useful.

Also fix an invalid 32-bit checksum. The d11 test had not been running as 32-bit since d8ff89a so the checksum was not updated when it should have been in 48f511d.
2025-02-28 15:38:05 -06:00
c351263a1d Fix typos.
Found using `codespell -S *.eps,*.cache,*.xml -L inout,te,fo,bload,fase,collet,hilight,debians,keep-alives` and `typos --hidden --format brief`.
2024-11-22 15:25:43 -05:00
7f2dfc021c Update Fedora test image to Fedora 41. 2024-11-18 13:33:03 -05:00
047e3d0ed9 Finalize catalog number for PostgreSQL 17 release. 2024-10-02 19:49:15 +03:00
d8ff89ae75 Update EOL Debian 10 to Debian 11/Ubuntu 20.04 in CI.
Typically we use the oldest Debian/Ubuntu to run 32-bit unit and integration tests. However, 32-bit is no longer fully supported by Ubuntu (multiple packages we need are missing) and apt.postgresql.org no longer packages for any 32-bit version.

To address these changes, do 64-bit integration testing on the oldest Debian/Ubuntu (currently Ubuntu 20.04) and 32-bit unit/integration testing on the oldest Debian (currently 11) using the included version for integration testing.
2024-07-15 11:53:57 +07:00
2ec99ca4d9 Specify test images that get valgrind installed.
This means valgrind is no longer built from source, which caused image builds to run for a very long time.

Valgrind is only required in a few images for testing.
2024-06-12 16:08:17 +10:00
886bb281f6 Migrate CentOS 7 integration tests to Rocky 8.
CentOS 7 will be EOL on June 30 and since there is no CentOS 8 migrate instead to Rocky 8.
2024-06-11 15:20:04 +10:00
6f562fba60 Migrate coverage testing to C and remove dependency on lcov.
lcov does not seem to be very well maintained and is often not compatible with the version of gcc it ships with until a few months after a new distro is released. In any case, lcov is that not useful for us because it generates reports on all coverage while we are mainly interested in missing coverage during development.

Instead use the JSON output generated by gcov to generate our minimal coverage report and metrics for the documentation.

There are some slight differences in the metrics. The difference in the common module was due to a bug in the old code -- build/common was being added into common as well as being reported separately. The source of the two additional branches in the backup module is unknown but almost certainly down to how exclusions are processed with regular expressions. Since there is additional coverage rather than coverage missing this seems fine.

Since this was pretty much a rewrite it was also a good time to migrate to C.
2024-05-31 14:52:07 +10:00
899b892788 New CI container build for PostgreSQL 17 beta1.
Update the catalog version for beta 1 so pgbackrest will not work with any prior development versions.

Also improve the integration/all test so the catalog version does not need to be updated again during the beta period.
2024-05-24 12:24:11 +10:00
28ad1badd9 Update Fedora test image to Fedora 40. 2024-05-18 18:55:53 +10:00
e00e33b528 Dynamically find python in meson build.
This is better than requiring a python3 binary to be on the path because some installations might have, e.g. python3.9.

Also add the python3-distutils package to Debian builds to make this work.
2024-04-25 19:32:07 +10:00
4ea4e3f380 Update Fedora test image to Fedora 38.
This should have been done in 434938e3 but somehow it didn't happen.

Fedora 38 requires 2048 bit keys so update the VM builds to use them. Update the documentation to use 2048 bit keys. This is not technically required by this commit but it makes sense to do it now.

Also update the key location for the yum.p.o repository.

Lastly, shuffle test PostgreSQL versions since PostgreSQL 11 is not longer available in the yum.p.o repository.
2024-04-23 22:34:32 +10:00
fb22f04555 PostgreSQL 17 Support.
Add catalog version and WAL magic for PostgreSQL 17.
2024-04-18 10:56:24 +10:00
657c1a3e06 Finalize catalog number for PostgreSQL 16 release. 2023-09-14 18:41:36 -04:00
d24180e4da Fix conflicting group in test containers.
The tests expect the group name/id to match between the host system and the container. If there is a conflict rename the group with the required id to the expected name.

This could have unintended consequences but it seems reasonably safe since we control everything that runs in the container and there should never be any system processes running.
2023-08-29 12:28:02 -04:00
434938e32b Update Fedora test image to Fedora 38.
This adds testing for the latest tool versions, e.g. gcc.
2023-06-22 18:23:06 +02:00
5d671c63d8 New CI container build for PostgreSQL 16 beta1. 2023-05-25 20:08:12 +03:00
87087fac23 SFTP support for repository storage. 2023-05-13 19:16:16 +03:00
3fc3690dd7 PostgreSQL 16 Support.
Add catalog version and WAL magic for PostgreSQL 16.

The GUC to force parallel mode has be renamed so update that in the tests.
2023-04-27 10:30:50 +03:00
31cad5e09e Check for stray execute permissions in test.pl --code-format.
Sometimes these can get set while updating permissions from with a VM or container.

Also fix a few permissions that were not correct.
2023-02-01 12:57:04 +07:00
65be4c64a9 Finalize catalog number for PostgreSQL 15 release. 2022-10-16 09:58:35 +13:00
c99ea54f17 Integrate C test harness with Perl test harness.
The C test harness is used for unit tests from the Perl harness where possible. Currently, unit tests can be run in the C harness when --no-coverage is specified and --profile is not specified.

C harness tests work on meson 0.45.

The C harness runs with valgrind by default. Valgrind can be disabled with --no-valgrind.

Also rebuild containers to add meson and update the documentation so that meson builds will work (even though we don't do them yet).
2022-07-27 10:32:32 -04:00
a16cf5eac7 Update CI to use Ubuntu 22.04 and Fedora 36.
Both have newer gcc and OpenSSL 3.

Fedora 36 runs horribly slow with valgrind enabled so run the valgrind tests on Ubuntu 22.04. Fedora 36 has a newer gcc so it is still worth testing on.
2022-06-06 16:32:20 -04:00
c98baab6b5 New CI container builds for PostgreSQL 15 beta1 and minor releases.
Remove VM_OS_REPO since it is no longer required.

Rebalance PostgreSQL versions for more efficient test times.

Always print version of PostgreSQL when testing. This helps verify that new minor releases are being used.
2022-05-19 18:24:09 -04:00
20782c88bc PostgreSQL 15 support.
PostgreSQL 15 drops support for exclusive backup and renames the start/stop backup commands.

This is based on the pgdg-testing repo since beta1 has not been released yet, but it seems unlikely that breaking changes will be made at this point. beta1 should be tagged just before our next release so we'll retest before the release.
2022-05-04 11:55:59 -04:00
54b4187527 Show Docker output when building containers if --log-level=detail.
This helps with debugging and monitoring container builds.
2022-04-05 13:14:42 -04:00
2c96327e65 Remove extraneous double spaces in code and comments. 2022-03-15 17:55:48 -06:00
cf5b3a302f Fix language in rh7 test container for aarch64.
The /etc/profile.d/lang.sh script was causing issues but it does not exist on amd64, so it seems the easiest thing was to remove it.

Fix how 32-bit VMs are determined now that another 64-bit architecture has been added.

And remove some obsolete VM hashes.
2022-01-26 13:22:31 -06:00
c32e000ab9 Use Rocky Linux for documentation builds instead of CentOS.
Since CentOS 8 will be EOL at the end of the year it makes sense to do this now. The centos:8 image is still used in documentation.xml because changes there require manual testing, which will need to be done at a later date. The changes are not user-facing, however, and can be done at any time.

Also update CentOS references to RHEL since that is what we are emulating for testing purposes.
2021-10-28 15:15:49 -04:00
66bfd1327e Rename SSH connection control parameters in integration tests. 2021-10-13 19:48:41 -04:00
815377cc60 Finalize catalog number for PostgreSQL 14 release. 2021-09-30 13:27:14 -04:00
1afea449e2 Build newer valgrind and move CA cert install in test containers.
The newer version of valgrind helps with some arm64 issues that have been fixed since the architecture has become more popular. Also add the valgrind builds to the Vagrantfile and Dockerfile.

Move the CA cert install from the base container to the test container. This means the CA cert can be changed without rebuilding all the base containers.
2021-09-02 13:26:21 -04:00
ad5e063990 Update containers for new PostgreSQL releases.
Also add a package that is required for older Debian versions.
2021-08-19 17:25:57 -04:00
6397d73535 Update test container OS versions.
Update Ubuntu 12.04 to 16.04. Version 16.04 is recently EOL but testing on an old version is beneficial.
Update Ubuntu 18.04 to 20.04.
Update Fedora 32 to 33. Version 34 would have been preferred but there were some build issues, i.e. the default shell did not work with configure, and after ksh was installed configure locked up.

Add --no-install-recommends to apt-get commands to save a bit of time and space.

Update test Dockerfile to run in multiple steps. This makes the container larger but also makes rebuilding after changes faster. The --squash option may be used to keep the container small.

Remove obsolete casts in protocol/parallel module. These casts were included in the original migration because Ubuntu 12.04 32-bit gcc required them, but Ubuntu 16.04 32-bit gcc complains. There is no production issue here since at this point in the code the file descriptors are guaranteed to be >= 0.
2021-07-18 17:29:20 -04:00
2452c4d5a4 Add PostgreSQL 14 support.
There are no code changes from PostgreSQL 13 so simply add the new version.

Add CATALOG_VERSION_NO_MAX to allow the catalog version to "float" during the PostgreSQL beta/rc period so new pgBackRest versions are not required when the catalog version changes.

Update the integration tests to handle new PostgreSQL startup messages.
2021-05-24 17:17:03 -04:00
b270253a69 Add defines for many test*() getter functions.
A define was already added for TEST_PATH but it was not widely used. Replace all occurrences of testPath() with TEST_PATH in the tests.

Replace testUser() with TEST_USER, testGroup() with TEST_GROUP, testRepoPath() with HRN_PATH_REPO, testDataPath() with HRN_PATH, testProjectExe() with TEST_PROJECT_EXE, and testScale() with TEST_SCALE.

Replace {[path]}, {[user]}, {[group]}, etc. with defines and remove hrnReplaceKey(). This is better than having two ways to deal with replacements.

In some cases the original test*() getters were kept because they are used by the harness, which does not have access to the new defines. Move them to harnessTest.intern.h to indicate that the tests should no longer use them.
2021-05-22 09:30:54 -04:00
baddec1e9a Basic multi-architecture support for test containers.
The tests worked fine on multiple architectures, but would only run "bare metal", i.e. tests that required containers could not be run.

Enable basic multi-architecture support by allowing containers to be built using whatever architecture the host supports. Also allow cached containers to be defined for multiple architectures in container.yaml.

Add a Dockerfile which can be used as a container for other containers to provide a consistent development environment.

The primary goal is to allow development on Mac M1 but other architectures should find these improvements useful.
2021-05-03 16:31:27 -04:00
fe4ba455ed Move configuration definition to src/build/config/config.yaml.
Moving to YAML allows the configuration data to be read by C programs.

Also go back to using YAML::XS since it is the only implementation that has proper boolean support.
2021-03-08 16:01:05 -05:00
d2057c53bd Use YAML::Any module instead of YAML::XS in Perl.
YAML::XS requires libyaml so it not as portable as pure Perl versions of YAML.

Instead of using YAML:PP just use the general YAML::Any module which uses whatever is installed. We are not concerned about performance for YAML so whatever works is fine.
2021-01-24 15:06:38 -05:00
c2c702c09d Add co7 package to support llvm.
This is required for new package versions.

Also remove the obsolete 9.2 package and update the supported versions list.
2021-01-13 17:32:42 -05:00
ec9f23d31f Remove CentOS 6 from tests and documentation.
CentOS6 EOL'd and the mirrors were swiftly deleted, leading to failures in tests and documentation.

Remove CentOS 6 for now to get builds going again with the intention to replace it in the near future with CentOS 8.
2020-12-02 16:23:05 -05:00
b096a25b49 Update test containers for PostgreSQL 13.
Add older PostgreSQL versions to the u18 container that were not available before.

This also updates all minor versions for prior versions of PostgreSQL.
2020-09-24 11:19:51 -04:00
ec7b7c5a3e PostgreSQL 13 beta1 support.
There don't appear to be any behavioral changes since PostgreSQL 12 and all the tests pass.

Changes to the control/catalog/WAL versions in subsequent betas may break compatibility but pgBackRest will be updated with each release to keep pace.
2020-05-21 13:46:16 -04:00
faabf1227d Update Fedora container to Fedora 32.
This allows unit testing on gcc 10.

Also fix an incorrect enum in the config/config unit test that was caught by the new compiler.
2020-05-07 11:06:56 -04:00