Previously we were just checking for the existence of NextContinuationToken, which the S3 documentation indicates will not be present when the list is not truncated. However, recent versions of Scality send a blank NextContinuationToken when IsTruncated is false. Sending the blank continuation token back causes Scality to send another blank continuation token and an infinite loop occurs.
Instead use IsTruncated (which is required to be present) to determine whether NextContinuationToken should be present. Error if NextContinuationToken is then missing or empty, since an empty token caused an infinite loop with the Scality server (which arguably should have errored when passed an empty token).
The TEST_STORAGE_LIST() macro is more robust and hides the callback mechanism from the caller.
Add features to TEST_STORAGE_LIST() that hrnStorageInfoListCallback() had.
Update tests to use the abbreviated type output (e.g. path/) generated by TEST_STORAGE_LIST().
This provides reproducible builds and minimizes the file path in debug messages, just like an in-tree make build.
For test source, prefix with test/ in case there are any module name collisions.
Having the test harness in C will allow us to remove duplicated Perl code and test on systems where Perl support is not present.
Custom harnesses and shims are currently not implemented, which means only the following tests in the common module will run: error, stack-trace, type-convert, assert-on, mem-context, time, encode, type-object, type-string, type-list, type-buffer, type-variant, reg-exp, log.
The experimental test harness is being committed with partial functionality so it can be used in Windows development. The remaining features will follow as needed.
The meson builds are still experimental so for now the configure/make build process is preferred for release builds. This message should help prevent any automated build systems from picking up meson instead.
Some of the replacements that were being done already existed as constants, so use the constants instead.
Also fix a minor formatting error introduced when testAdd() was renamed to hrnAdd().
This module has dependencies on command/command so it does not make sense for it to be in the common module. Also move protocolFree() to main() since this is a very large dependency.
Adjust the tests so command/exit can be tested later. This is a bit messy but will get adjusted as we improve the test harness.
PG_WAL_SEGMENT_SIZE_DEFAULT is used to compare and check WAL size on pre-11 installations. However, there is a hard-coded assertion in walSegmentNext() which doesn't respect PG_WAL_SEGMENT_SIZE_DEFAULT.
Update the assertion to use PG_WAL_SEGMENT_SIZE_DEFAULT.
The storage/helper module is a very heavy dependency to introduce in the common module. Creating Posix storage objects is cheap so just do that instead.
If DEBUG is not defined then the ASSERT() macro expands to nothing. In this case the timeBegin variable is never used and a compilation error occurs.
This test should work without DEBUG defined so use CHECK() instead of ASSERT().
Change all instances of __attribute__((__noreturn__)) to a macro in meson.build / build.auto.h.in.
As compiler attributes written in the form of __attribute__ are not supported by MSVC, this is one of several commits to make the code-base more robust and allow using MSVC-style attributes later.
Both have newer gcc and OpenSSL 3.
Fedora 36 runs horribly slow with valgrind enabled so run the valgrind tests on Ubuntu 22.04. Fedora 36 has a newer gcc so it is still worth testing on.
There are two changes:
* Suppress deprecation warnings so we can build with -Werror and -Wfatal-errors. At some point we'll need to migrate to the new APIs but there does not seem to be a good reason to support two sets of code right now.
* Update the handling for unexpected EOF to handle EOF or error. The error code for EOF has changed and become harder to identify, but we probably don't care whether it is an error or EOF.
Maintaining the version interfaces was complicated by the fact that each interface needed to be in separate compilation unit to avoid type conflicts. This also meant that various build/test files needed to be updated to add the new interfaces.
Solve these problems by auto-generating all the interfaces into a single file. This is made possible by parsing defines and types out of the header files and creating macros to rename the types. At the end of the version interface everything is undef'd. Another benefit is that the auto-generated interfaces can be static and included directly into postgres/interface.c.
Since some code generation is now always required for tests, change --no-gen to --min-gen in test.pl.
It would also make sense to auto-generate the version defines in postgres/version.h, but that will be left for a future commit.
Meson is a new build system that offers simpler syntax and superior performance to autoconf/make. In addition, Windows is supported natively.
The Meson build appears complete, but currently is used only for auto-generation of code and the host build of pgbackrest. Some container upgrades will be required before Meson can be used for container builds.
Also patch the Debian package to force autoconf/make rather than Meson.
Stopping the cluster has started consistently running out of memory on PostgreSQL 9.1. This seems to have happened after pulling in new packages at some point so it might be build related.
Stopping the cluster is not critical for 9.1 so skip it.
These files were never intended to be compiled on their own so the .c extension was a bit misleading. In particular Meson does not like .c files that are not intended to be compiled independently.
Leave header files as is since they are already protected against being included more than once and are never expected to be compiled.
Most internal options were being skipped, but not in the case where an option was marked internal for a specific command.
The command-line help was not affected by this issue.
Some of the remote repo options were gated by repo-local, but the rest relied on repo-host-cmd.
Remove backup from the repo-host-cmd option since none of the dependent options are valid for backup.
31c7824a allowed these commands to run remotely but neglected to remove some internal flags, which prevented all the repo-* options from being visible in the documentation.
The manifest is saved on a regular basis during a backup so a failed backup can be resumed. For backups that the user has configured/invoked as not resumable, skip the incremental save of the manifest.
Waiting to write percent complete until the first file completed resulted in a period of time where the backup was running without status available to the user.
Remedy this by initializing percent complete to zero when the backup is ready to start copying files.
Most of these were probably never ported from Perl to C and others became obsolete over time.
Fix one error that was the wrong type.
Also fix/improve some comments.
Previously the behavior was to download the file from the repository when it was not exactly the same size in PGDATA. However, it may just be that the file was extended and the contents are the same up to the file size recorded in the manifest. This could also be very valuable for files that are always append only, like logs.
Change info.size to file->size in one place. Both are technically correct but file->size makes more sense.
Use the new fileName variable in a few existing places.
Also adjust some existing comments to make them clearer.
Remove VM_OS_REPO since it is no longer required.
Rebalance PostgreSQL versions for more efficient test times.
Always print version of PostgreSQL when testing. This helps verify that new minor releases are being used.
cfgOptionStr() may not have the correct value if the repo is remote.
Use storagePathP() instead since it can ask the remote for the correct value when required.
Each mem context can track child contexts, allocations, and a callback. Before this change memory was allocated for tracking all three even if they were not used for a particular context. This made mem contexts unsuitable for String and Variant objects since they are plentiful and need to be as small as possible.
This change allows mem contexts to be configured to track any combination of child contexts, allocations, and a callback. In addition, the mem context can be configured to track a single child context and/or allocation, which saves memory and is a common use case.
Another benefit is that Variants can own objects (e.g. KeyValue) that they encapsulate. All of this makes memory accounting simpler because mem contexts have names while allocations do not. No more memory is used than before since Variants and Strings still had to store the memory context they were originally allocated in so they could be easily freed.
Update the String and Variant objects to use this new functionality. The custom strFree() and varFree() functions are no longer required and can now be a wrapper around objFree().
Lastly, this will allow strMove() and varMove() to be implemented and used in cases where strDup() and varDup() are being used to move a String or Variant to a new context. Since this will be a bit noisy it is saved for a future commit.
For some reason /lib/systemd/system/sysinit.target.wants no longer exists in the rockylinux:8 container.
Create this directory explicitly in case it does not exist.
Because there is a lot of repetition in this file, changes can look very jumbled with existing data in a diff. Also, if can be hard to tell what is being modified if the diff does not show enough lines before and after.
This change adds labels to the end of the line to localize the diff and make it easier to see what has been changed. Also, remove some linefeeds and make separators more consistent.
The change to parse.auto.c will be committed separately so it can be ignored in history/blame.
