1
0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2024-12-12 10:04:14 +02:00
Commit Graph

4127 Commits

Author SHA1 Message Date
David Steele
a28f3d49c2 Add --no-back-trace option to test.pl.
Running valgrind and backtrace together has been causing tests to timeout in CI, mostly likely due to limited resources. This has not been a problem in normal development environments.

Since it is still important to run backtraces for debugging, split the u22 test that was doing all this work to run coverage and backtrace together and valgrind-only as a separate test. As a bonus these tests run faster separately and since they run in parallel the total execution time is faster.
2023-01-28 18:55:53 +07:00
David Steele
16c625353d Skip test modules in coverage report.
Coverage of the test code is useful during development but it does not seem worth adding it to the core code coverage report.
2023-01-28 16:22:04 +07:00
David Steele
d5ef4ce747 Fix permissions on source files. 2023-01-28 11:01:29 +07:00
David Steele
3aea997df5
Add warning about enabling "hierarchical namespace" on Azure storage.
If this feature is enabled expire will fail since directories need to be deleted separately.

Ideally we would add support for this feature but for now we'll just document the issue.
2023-01-25 10:59:13 +07:00
David Steele
ed818b3186 Add dark mode to the website.
The colors could use more tweaking but at least the website will no longer blind users running dark mode.
2023-01-25 10:35:03 +07:00
David Steele
1da2666a9e Add manifest test harness.
These macros make adding paths/files/etc to a manifest simpler and easier to read.
2023-01-21 14:03:27 +07:00
David Steele
912eec63bb
Block-level incremental backup.
The primary goal of the block incremental backup is to save space in the repository by only storing changed parts of a file rather than the entire file. This implementation is focused on restore performance more than saving space in the repository, though there may be substantial savings depending on the workload.

The repo-block option enables the feature (when repo-bundle is already enabled). The block size is determined based on the file size and age. Very old or very small files will not use block incremental.
2023-01-20 16:48:57 +07:00
David Steele
008a18555c Fix comment indentation and wrapping. 2023-01-20 15:44:01 +07:00
David Steele
bfc9a5c33e Fix incorrect FUNCTION_TEST_END() in hrnPgWalToBuffer(). 2023-01-14 18:01:48 +07:00
David Steele
ccee5c0fb1 Reduce log level of pgVersionFromStr() and pgVersionToStr(). 2023-01-14 17:12:15 +07:00
David Steele
596c62c54e Simply return in pgVersionToStr(). 2023-01-14 15:25:25 +07:00
David Steele
34e4835ff3
Refactor common/ini module to remove callbacks and duplicated code.
The callbacks in iniLoad() made the downstream code more complicated than it needed to be so use an iterator model instead.

Combine the two functions that were used to load the ini data to remove code duplication. In theory it would be nice to use iniValueNext() in the config/parse module rather than loading a KeyValue store but this would mean a big change to the parser, which does not seem worthwhile at this time.
2023-01-12 21:24:28 +07:00
David Steele
9ca492cecf Audit mem contexts returned from functions into the calling context.
It is possible for functions to accidentally leak child contexts into the calling context, which may use a lot of memory depending on the use case and where it happens.

Use the function return type to determine what should be returned and error when something else is returned. Add FUNCTION_AUDIT_*() macros to handle exceptions.

This checking is only performed during unit tests on the code being covered by the specific unit test.

Note that this does not work yet for memory allocations, i.e. memNew(). These are pretty rare so are not as much of an issue and they can be added in the future.
2023-01-12 17:36:57 +07:00
David Steele
de1dfb66ca Refactor logging functions to never allocate memory.
Allocating memory made these functions simpler but it meant that memory was leaking into the calling context when logging was enabled. It is not clear that this was an issue but it seems that trace level logging could result it a lot of memory usage depending on the use case.

This also makes it possible to audit allocations returned to the calling context, which will be done in a followup commit.

Also rename objToLog() to objNameToLog() since it seemed logical to name the new function objToLog().
2023-01-12 17:14:36 +07:00
David Steele
890b9d0093 Remove mostly useless mem context in storageRemoteInfoProtocolPut().
Maybe this once had a deeper purpose but now at least it is just used to avoid a few trivial allocations. If we really wanted to do that a flag would be better but it does not seem worth the trouble.
2023-01-12 15:04:49 +07:00
David Steele
6ec2241e15 Fix leak in protocolServer(). 2023-01-12 14:24:20 +07:00
David Steele
9d42435ffb Fix leaks in db module. 2023-01-12 14:22:09 +07:00
David Steele
7f7d515441 Fix leak in configOptionRemote(). 2023-01-12 14:19:10 +07:00
David Steele
123d5e77ae Minor optimizations on config/parse module. 2023-01-12 14:17:48 +07:00
David Steele
0becb6da31 Enhance libbacktrace to handle incomplete stack traces.
This fills in backtrace info at the bottom of the call stack when the stack trace is incomplete due to testing. This does not affect release builds, which is why it did not make the first cut, but it turns out to be useful for testing and barely changes the release code (when we do release this).

The recursion test in common/error was simplified because it would now return a very large trace.
2023-01-12 10:22:26 +07:00
David Steele
57fc4aaeb9 Update RHEL user guide to PostgreSQL 11/12.
PostgreSQL 10 has expired from the PGDG yum repository.
2023-01-12 09:06:01 +07:00
David Steele
74ca9b50bb Revert macro changes from 38d32045.
This seemed like it would be cleaner but an important detail was missed (logAny) and it does not seem simpler when factoring that back in.

Keep the removal of the extraneous semicolons and all the downstream changes required by the removal.
2023-01-11 19:43:10 +07:00
David Steele
9347313cbf Fix error detail output to stderr.
The error detail should be output when the error is an assert (this part was working) or the log level is at least debug. In cases where log-level-console was at least debug but log-level-stderr was not the detail was lost.

Improve the range checking to output error detail to stderr when log-level-console is at least debug.
2023-01-11 19:16:31 +07:00
David Steele
c826429123 Improve libbacktrace implementation.
The libbacktrace feature has not been working since the move to meson because libbacktrace detection was not added to the meson build. Add libbacktrace to meson and improve the feature so that it can be compiled into release builds.

The prior implementation fetched line numbers with each stack trace push. Not only was this slow but it missed any functions that were not being tracked on our stack.

Instead just examine the backtrace when an error happens and merge it with the info we have on our stack. If the backtrace is not available then the output remains as before.

Also remove --backtrace from test.pl since the library is now auto-detected.

Leave this library out of the production build for now to give it a little time to shake out in testing.
2023-01-11 11:19:26 +07:00
David Steele
0dd2997714 Fix leak in jsonReadVarRecurse(). 2023-01-10 11:45:48 +07:00
David Steele
675b4ae251 Fix leak in archivePushDropWarning(). 2023-01-10 11:17:08 +07:00
David Steele
bf9e331ac1 Fix incorrect return type in backupFile(). 2023-01-10 10:14:43 +07:00
David Steele
b722d2b6ca Fix formatting of archiveGetFile(). 2023-01-10 09:28:57 +07:00
David Steele
38d3204553 Remove duplication from FUNCTION_LOG_BEGIN_BASE() macro definitions.
It is a bit simpler to define STACK_TRACE_TEST_START()/STACK_TRACE_TEST_STOP() in a separate #ifdef so FUNCTION_LOG_BEGIN_BASE() does not need to be defined twice.

Also add missing semicolons exposed by this change.
2023-01-09 14:47:57 +07:00
David Steele
6633ccd18d Do not output stderr on unit/integration test errors.
Since stderr is being redirected to stdout this results in duplication of the error output.
2023-01-05 14:03:43 +07:00
David Steele
4429bc82f5 Add unit tests for the unit test build code.
When this code was migrated to C the unit tests were not included because there were more important priorities at the time.

This also requires some adjustments to coverage because of the new code location.
2023-01-05 12:59:06 +07:00
David Steele
a5499afabc Add CHECK_FMT() macros and tests for CHECK*() macros.
The CHECK_FMT() macro will be used in some upcoming tests but could also be useful in the core code.

The CHECK*() macros never had tests so add them.
2023-01-05 12:44:00 +07:00
David Steele
2e3513984e Remove extra linefeed. 2023-01-05 12:33:17 +07:00
David Steele
877bb2ac9e Update LICENSE.txt and PostgreSQL copyright for 2023. 2023-01-03 08:26:44 +07:00
David Steele
2b2a6edf35 Cleanup of common/ini module in preparation for a refactor. 2023-01-02 18:53:05 +07:00
David Steele
92d5fd17b4 Fix formatting of restoreFile() definition. 2023-01-02 17:32:09 +07:00
David Steele
d51a86c621 Add BUFFER_EXTERN() and cleanup hash zero constants.
BUFFER_EXTERN() provides a clean way to create buffer constants.

Convert HASH_TYPE_SHA256_ZERO_STR to HASH_TYPE_SHA256_ZERO_BUF to be consistent with HASH_TYPE_SHA1_ZERO_BUF.
2023-01-02 17:30:57 +07:00
David Steele
66f108ea8a Cleanup terminology in Buffer/Variant/String macros. 2023-01-02 17:26:53 +07:00
David Steele
f018912908 Split VR_EXTERN/FN_EXTERN macros from FV_EXTERN.
This should make it a little clearer what the variable (VR) macros are doing since the declaration/definition cannot both be set to extern (but functions can).

Splitting the variable macros out also allows them to be changed in the future with little churn, while changing the function macro creates a large amount of churn.
2023-01-02 15:24:51 +07:00
David Steele
9e29c01891 Remove MacOS from CirrusCI.
CirrusCI stopped supporting Intel but the arm builds are not working, even with the same steps that work on an arm Mac.

Remove to unstick the build pipeline until this can be resolved.
2023-01-02 15:11:37 +07:00
David Steele
8993267df9 Fix formatting. 2022-12-31 23:33:45 +07:00
David Steele
1793839e5a Clean up storage headers and move posix internal type. 2022-12-31 19:12:35 +07:00
David Steele
4fb8a0ecdd Add meson unity build and tests.
This is immediately useful because it will detect any extern'd functions or variables that are not being used. It also detects functions or variables that are declared but not defined.

If a FV/VR_EXTERN macro is missing it will be detected either because of a mismatch in the declaration/definition or because a new defined symbol will appear in the nm test.

Eventually the unity build will be used to create a more optimized pgbackrest binary but that will need to wait.
2022-12-31 17:13:41 +07:00
David Steele
8aa2b101bb Put logging functions unused by release builds into DEBUG blocks.
Also inline some functions that are needed in non-DEBUG builds.
2022-12-31 16:41:01 +07:00
David Steele
4a64c5d80c Use memcpy() instead of strncpy when source size is known.
In this case the destination will be large enough to hold the source so memcpy is more efficient.

Also, in highly optimized builds the compiler may warn for strncpy() when it can see that the source size won't include the terminator.
2022-12-31 15:49:32 +07:00
David Steele
2332ce8ffc Move storageHelperFree() to storageHelper test harness.
This function was only used for testing so move into a test harness.
2022-12-31 13:14:27 +07:00
David Steele
8b218158ae Move regExpMatchPtr()/regExpMatchStr() to build/common/regExp module.
Similar to b9be4fa5, these functions are not used by the core code so move them to the build module. The new implementation is a little less efficient but that is much less of a worry in the build/test code.

Also remove regExpMatchSize() since it was not longer needed.
2022-12-31 12:54:33 +07:00
David Steele
45ece13678 Use error*() getters where possible.
This makes it a little easier to change the underlying representation and ensures that the functions are used.
2022-12-31 11:57:06 +07:00
David Steele
fa9d831f9f Move xmlNodeAttribute() to build/common/xml module.
Similar to b9be4fa5, this function was not used by the core code so move it to the build module.
2022-12-31 11:09:50 +07:00
David Steele
c6264ad4cf Fix comment typo for strLower(). 2022-12-31 10:31:15 +07:00