1
0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2024-12-12 10:04:14 +02:00
Commit Graph

4114 Commits

Author SHA1 Message Date
David Steele
de1dfb66ca Refactor logging functions to never allocate memory.
Allocating memory made these functions simpler but it meant that memory was leaking into the calling context when logging was enabled. It is not clear that this was an issue but it seems that trace level logging could result it a lot of memory usage depending on the use case.

This also makes it possible to audit allocations returned to the calling context, which will be done in a followup commit.

Also rename objToLog() to objNameToLog() since it seemed logical to name the new function objToLog().
2023-01-12 17:14:36 +07:00
David Steele
890b9d0093 Remove mostly useless mem context in storageRemoteInfoProtocolPut().
Maybe this once had a deeper purpose but now at least it is just used to avoid a few trivial allocations. If we really wanted to do that a flag would be better but it does not seem worth the trouble.
2023-01-12 15:04:49 +07:00
David Steele
6ec2241e15 Fix leak in protocolServer(). 2023-01-12 14:24:20 +07:00
David Steele
9d42435ffb Fix leaks in db module. 2023-01-12 14:22:09 +07:00
David Steele
7f7d515441 Fix leak in configOptionRemote(). 2023-01-12 14:19:10 +07:00
David Steele
123d5e77ae Minor optimizations on config/parse module. 2023-01-12 14:17:48 +07:00
David Steele
0becb6da31 Enhance libbacktrace to handle incomplete stack traces.
This fills in backtrace info at the bottom of the call stack when the stack trace is incomplete due to testing. This does not affect release builds, which is why it did not make the first cut, but it turns out to be useful for testing and barely changes the release code (when we do release this).

The recursion test in common/error was simplified because it would now return a very large trace.
2023-01-12 10:22:26 +07:00
David Steele
57fc4aaeb9 Update RHEL user guide to PostgreSQL 11/12.
PostgreSQL 10 has expired from the PGDG yum repository.
2023-01-12 09:06:01 +07:00
David Steele
74ca9b50bb Revert macro changes from 38d32045.
This seemed like it would be cleaner but an important detail was missed (logAny) and it does not seem simpler when factoring that back in.

Keep the removal of the extraneous semicolons and all the downstream changes required by the removal.
2023-01-11 19:43:10 +07:00
David Steele
9347313cbf Fix error detail output to stderr.
The error detail should be output when the error is an assert (this part was working) or the log level is at least debug. In cases where log-level-console was at least debug but log-level-stderr was not the detail was lost.

Improve the range checking to output error detail to stderr when log-level-console is at least debug.
2023-01-11 19:16:31 +07:00
David Steele
c826429123 Improve libbacktrace implementation.
The libbacktrace feature has not been working since the move to meson because libbacktrace detection was not added to the meson build. Add libbacktrace to meson and improve the feature so that it can be compiled into release builds.

The prior implementation fetched line numbers with each stack trace push. Not only was this slow but it missed any functions that were not being tracked on our stack.

Instead just examine the backtrace when an error happens and merge it with the info we have on our stack. If the backtrace is not available then the output remains as before.

Also remove --backtrace from test.pl since the library is now auto-detected.

Leave this library out of the production build for now to give it a little time to shake out in testing.
2023-01-11 11:19:26 +07:00
David Steele
0dd2997714 Fix leak in jsonReadVarRecurse(). 2023-01-10 11:45:48 +07:00
David Steele
675b4ae251 Fix leak in archivePushDropWarning(). 2023-01-10 11:17:08 +07:00
David Steele
bf9e331ac1 Fix incorrect return type in backupFile(). 2023-01-10 10:14:43 +07:00
David Steele
b722d2b6ca Fix formatting of archiveGetFile(). 2023-01-10 09:28:57 +07:00
David Steele
38d3204553 Remove duplication from FUNCTION_LOG_BEGIN_BASE() macro definitions.
It is a bit simpler to define STACK_TRACE_TEST_START()/STACK_TRACE_TEST_STOP() in a separate #ifdef so FUNCTION_LOG_BEGIN_BASE() does not need to be defined twice.

Also add missing semicolons exposed by this change.
2023-01-09 14:47:57 +07:00
David Steele
6633ccd18d Do not output stderr on unit/integration test errors.
Since stderr is being redirected to stdout this results in duplication of the error output.
2023-01-05 14:03:43 +07:00
David Steele
4429bc82f5 Add unit tests for the unit test build code.
When this code was migrated to C the unit tests were not included because there were more important priorities at the time.

This also requires some adjustments to coverage because of the new code location.
2023-01-05 12:59:06 +07:00
David Steele
a5499afabc Add CHECK_FMT() macros and tests for CHECK*() macros.
The CHECK_FMT() macro will be used in some upcoming tests but could also be useful in the core code.

The CHECK*() macros never had tests so add them.
2023-01-05 12:44:00 +07:00
David Steele
2e3513984e Remove extra linefeed. 2023-01-05 12:33:17 +07:00
David Steele
877bb2ac9e Update LICENSE.txt and PostgreSQL copyright for 2023. 2023-01-03 08:26:44 +07:00
David Steele
2b2a6edf35 Cleanup of common/ini module in preparation for a refactor. 2023-01-02 18:53:05 +07:00
David Steele
92d5fd17b4 Fix formatting of restoreFile() definition. 2023-01-02 17:32:09 +07:00
David Steele
d51a86c621 Add BUFFER_EXTERN() and cleanup hash zero constants.
BUFFER_EXTERN() provides a clean way to create buffer constants.

Convert HASH_TYPE_SHA256_ZERO_STR to HASH_TYPE_SHA256_ZERO_BUF to be consistent with HASH_TYPE_SHA1_ZERO_BUF.
2023-01-02 17:30:57 +07:00
David Steele
66f108ea8a Cleanup terminology in Buffer/Variant/String macros. 2023-01-02 17:26:53 +07:00
David Steele
f018912908 Split VR_EXTERN/FN_EXTERN macros from FV_EXTERN.
This should make it a little clearer what the variable (VR) macros are doing since the declaration/definition cannot both be set to extern (but functions can).

Splitting the variable macros out also allows them to be changed in the future with little churn, while changing the function macro creates a large amount of churn.
2023-01-02 15:24:51 +07:00
David Steele
9e29c01891 Remove MacOS from CirrusCI.
CirrusCI stopped supporting Intel but the arm builds are not working, even with the same steps that work on an arm Mac.

Remove to unstick the build pipeline until this can be resolved.
2023-01-02 15:11:37 +07:00
David Steele
8993267df9 Fix formatting. 2022-12-31 23:33:45 +07:00
David Steele
1793839e5a Clean up storage headers and move posix internal type. 2022-12-31 19:12:35 +07:00
David Steele
4fb8a0ecdd Add meson unity build and tests.
This is immediately useful because it will detect any extern'd functions or variables that are not being used. It also detects functions or variables that are declared but not defined.

If a FV/VR_EXTERN macro is missing it will be detected either because of a mismatch in the declaration/definition or because a new defined symbol will appear in the nm test.

Eventually the unity build will be used to create a more optimized pgbackrest binary but that will need to wait.
2022-12-31 17:13:41 +07:00
David Steele
8aa2b101bb Put logging functions unused by release builds into DEBUG blocks.
Also inline some functions that are needed in non-DEBUG builds.
2022-12-31 16:41:01 +07:00
David Steele
4a64c5d80c Use memcpy() instead of strncpy when source size is known.
In this case the destination will be large enough to hold the source so memcpy is more efficient.

Also, in highly optimized builds the compiler may warn for strncpy() when it can see that the source size won't include the terminator.
2022-12-31 15:49:32 +07:00
David Steele
2332ce8ffc Move storageHelperFree() to storageHelper test harness.
This function was only used for testing so move into a test harness.
2022-12-31 13:14:27 +07:00
David Steele
8b218158ae Move regExpMatchPtr()/regExpMatchStr() to build/common/regExp module.
Similar to b9be4fa5, these functions are not used by the core code so move them to the build module. The new implementation is a little less efficient but that is much less of a worry in the build/test code.

Also remove regExpMatchSize() since it was not longer needed.
2022-12-31 12:54:33 +07:00
David Steele
45ece13678 Use error*() getters where possible.
This makes it a little easier to change the underlying representation and ensures that the functions are used.
2022-12-31 11:57:06 +07:00
David Steele
fa9d831f9f Move xmlNodeAttribute() to build/common/xml module.
Similar to b9be4fa5, this function was not used by the core code so move it to the build module.
2022-12-31 11:09:50 +07:00
David Steele
c6264ad4cf Fix comment typo for strLower(). 2022-12-31 10:31:15 +07:00
David Steele
163a004f30 Move strReplace() and strUpper() to build/common/string module.
Neither of these functions were used by the core code. strReplace() is only used in the tests but it doesn't hurt to put it in build since the build code is not distributed.
2022-12-31 10:26:11 +07:00
David Steele
d517d4a328 Add explicit keyword for covered modules included in other modules.
This was done by checking the extension but it is possible to include a module that does not have a vendor or auto extension. Instead make it explicit that the module is included in another module.

Also change the variable from "include" to "included" to make it clearer what it indicates.
2022-12-31 10:10:44 +07:00
David Steele
729d7f071d Put memContextSize() in a DEBUG block.
This function is only used in unit tests so no need to include it in the release build.
2022-12-30 18:45:11 +07:00
David Steele
cebbf0d012 Remove unused functions.
These functions were either added with the intention that they would be used or they became obsolete over time.
2022-12-30 16:26:48 +07:00
David Steele
6f0c652e2c Remove declaration for function that is no longer defined. 2022-12-30 14:59:16 +07:00
David Steele
416fadd60c Mark functions not used externally as static.
Also cleanup a stray linefeed.
2022-12-30 14:48:31 +07:00
Stefan Fercot
b9be4fa540
Restore errors when no backup matches the current version of PostgreSQL.
It is probably not a good idea to restore the latest backup when it was not made from the current PostgreSQL version. If there is no backup after a stanza-upgrade then replicas might be built with a prior version leading to failures.

Add an error in this case if the latest backup would be used, i.e. --set or --type=time/lsn is not specified.
2022-12-29 15:37:27 +07:00
David Steele
aa1e72dfe6 Fix improper formatting in config/parse module.
The prior formatting worked but was confusing to read.
2022-12-29 15:05:45 +07:00
David Steele
36ee30d118
Updates and clarifications to index page.
In particular the section about other backup solutions not supporting parallel processing was no longer accurate, so reword it.

Also update some other sections that used older nomenclature, had awkward wording, or needed clarification.
2022-12-28 19:15:44 +07:00
Chris Bandy
84a3ff8b7a
Clarify target-action behavior on various PostgreSQL versions.
The behavior of pause depends on the hot_standby parameter and the PostgreSQL version so mention both.

This behavior has been verified on PostgreSQL 9.6–15. PostgreSQL 12 is an inflection point because the behavior of an unset recovery_target_action with hot_standby=off changed in https://git.postgresql.org/gitweb/?p=postgresql.git;h=2dedf4d9a899b36d1a8ed29be5efbd1b31a8fe85.
2022-12-28 10:48:44 +07:00
David Steele
ae258f604e
Add replacement for linefeeds in monitoring example.
The copy command was converting \n to a linefeed, which the json conversion did not like. In a healthy repository there won't be any linefeeds but certain errors can contain them.

Fix by loading into a text field and then replacing the linefeed when converting to jsonb.
2022-12-27 20:28:38 +07:00
David Steele
44da314adb
Add compress-level range checking for each compress-type.
The prior range checking was done based on the valid values for gz. While this worked it was a subset of what is available for lz4 and zst.

Allow the range to be specified for each compress-type. Adding this functionality to the parse module would be a better solution but that is a bigger project than this fix deserves, at least for now.
2022-12-27 20:05:08 +07:00
Stefan Fercot
34110cfa01 Fix typo. 2022-12-27 11:31:59 +07:00