1
0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2024-12-12 10:04:14 +02:00
Commit Graph

4127 Commits

Author SHA1 Message Date
David Steele
fa6c68cb02 Mem allocation cleanup in command/verify module.
Move the temp mem context out of verifyJobCallback() into verifyBackup() and verifyArchive(). This makes it clearer that verifyJobCallback() allocates no memory and reduces mem usage when both verifyBackup() and verifyArchive() are called.

Update verifyErrorMsg() to return zero-terminated strings to save on allocations. The output of this function is used when formatting strings so this is also simpler. Do a similar thing in verifyRender().

Also fix a minor leak in verifyInfoFile().
2022-04-26 10:18:24 -04:00
David Steele
3f7c8bc923 Fix object allocations in incorrect mem context in execOpen().
Object variables were begin allocated in the calling context rather than the object context.

This is not a live bug because Exec objects are currently created and opened in a long-lived context.
2022-04-26 10:15:47 -04:00
Reid Thompson
4dbe76a5bb Fix comment formatting. 2022-04-26 07:33:57 -04:00
David Steele
41f9d69edc Combine functions in the command/stanza module into one function.
It is not clear why these were split out, but it probably had something to do with testing before storageList() could return NULL for an empty directory.

Also remove the tests that depended on a boolean return, which are no longer needed for coverage.
2022-04-25 15:38:49 -04:00
David Steele
55a828f999 Add temporary mem contexts and fix a leak in the command/restore module.
restoreRecoveryConf() and restoreRecoveryWriteConf() do enough work to deserve their own memory contexts.

restoreFilePgPath() was leaking a String every time it was called, which could be a lot.

Also fix a spacing issue.
2022-04-25 15:03:37 -04:00
David Steele
4e7414d48f Add temporary mem contexts in the command/help module.
These were not really leaks since memory was being freed by the calling function, but these functions do enough work to deserve their own memory contexts.
2022-04-25 14:49:08 -04:00
David Steele
fd295f002b Add temporary mem context to removeExpiredBackup().
This was not really a leak since memory was being freed by the calling function, but this function does enough work to deserve its own memory context.

Also fixed a doubled semicolon.
2022-04-25 14:19:10 -04:00
David Steele
9314be36b1 Improve result declaration in backupRegExp().
No need to set this to NULL when the initial value can be used instead.
2022-04-25 14:14:29 -04:00
David Steele
5b5dbe0e7e Fix memory leaks in archivePushDrop().
A string was leaked for every file processed.

Since the list can be quite large it also makes sense to reset the context occasionally.
2022-04-25 13:29:43 -04:00
David Steele
c3b08f71ce Fix formatting in common/string module. 2022-04-25 13:26:09 -04:00
David Steele
40ef64f2be Fix memory leaks in archivePushFile().
The errorList is only used when throwing an error and the joined list is not needed after the error is thrown, so put both in the temp mem context.
2022-04-25 12:59:46 -04:00
David Steele
774db65086 Remove useless context switches in archiveGetFile()/archivePushFile().
These context switches do nothing since the list is already in the prior context.
2022-04-25 12:55:51 -04:00
David Steele
582c3dab4c Add strLstAddSub*() and strLstAddSubZ*() functions.
These help with readability and remove a cause of leaks.
2022-04-25 12:32:33 -04:00
David Steele
ff45f463cf Use strLstAddZ() instead of strLstAdd() where possible.
Using STRDEF() to convert the zero-terminated string to a String has no performance advantage but generates more code.
2022-04-25 11:58:30 -04:00
David Steele
7900660d3a Add strLstNewFmt().
Simplifies adding a formatted string to a list and removes a common cause of leaks.
2022-04-25 11:47:43 -04:00
David Steele
3475514b61 Fix memory leak in archiveAsyncErrorClear().
A string was leaked on each call.
2022-04-25 10:57:36 -04:00
David Steele
23d645b5e7 Fix outdated comment. 2022-04-25 10:52:13 -04:00
David Steele
699f15dd2b Remove THIS_MEM_CONTEXT() macro.
objMemContext(this) performs the same task and is easier to read.

Most instances of this macro were removed by 6e7be3c0.
2022-04-25 09:24:00 -04:00
David Steele
6e7be3c052 Add MEM_CONTEXT_OBJ_*() macros.
These provide a standard way to switch to an object mem context.

Update the two different patterns that were used before to the new macros.
2022-04-25 09:12:25 -04:00
David Steele
45c3f4d53c
Improve JSON handling.
Previously read/writing JSON required parsing/render via a variant, which add many more memory allocations and loops.

Instead allow JSON to be read/written serially to improve performance and simplify the code. This also allows us to get rid of many String and Variant constant which are no longer required.

The goal is to be able to read/write very large (e.g. gigabyte manifest) JSON structures, which would not be practical with the current code.

Note that external JSON (GCS, S3, etc) is still handled using variants. Converting these will require more consideration about key ordering since it cannot be guaranteed as in our own formats.
2022-04-25 09:06:26 -04:00
David Steele
58f24568f5 Add ASSERT_PARAM() macro.
Allows adding a parameter to a function that is used only for assertions.
2022-04-25 08:25:48 -04:00
David Steele
1e2b545ba4 Require type for FUNCTION_TEST_RETURN*() macros.
This allows code to run after the return type has been generated in the case where it is an expression.

No new functionality here yet, but this will be used by a future commit that audits memory usage.
2022-04-24 19:19:46 -04:00
David Steele
91bbe81059 Add SIZE_OF_STRUCT_MEMBER() macro.
Gets the size of a struct member, which requires a bit of additional syntax.
2022-04-22 09:14:12 -04:00
David Steele
a2eee156b5 Fix instances where STRDEF() was used instead of STR().
In practice this didn't cause problems because the string buffer was still valid and strSize() was not being called.
2022-04-21 18:23:17 -04:00
David Steele
627921c72a Add STR_SIZE() macro.
This allows efficiently creating strings where the length is already known and simplifies the STR() and STRDEF() macros.
2022-04-21 08:35:31 -04:00
David Steele
1ce613a24d Use STRDEF() instead of STR() in command/help module.
STRDEF() is more efficient since this is a constant string.
2022-04-21 08:34:03 -04:00
David Steele
4daddebaca Add ASSERT_DECLARE() macro.
Declare variables that will be used by later assertions with the goal of making them easier to read and maintain.

This is particularly useful for variables that are used more than once and require a lot of syntax to extract.
2022-04-21 08:07:22 -04:00
David Steele
e18b70bf55 Allow *RETURN*() macros to accept struct initializers.
Struct initializers look like multiple parameters in a macro so use __VA_ARGS__ to reconstruct them.
2022-04-21 07:45:59 -04:00
David Steele
ea4d73f375 Fix ordering of backup-lsn-stop field in command/restore unit test.
All fields should be alphabetical. Currently the read code is tolerant of this, but that will not always be the case.

Fields are always written alphabetically so this is just a test issue introduced by d8d41321.
2022-04-20 19:56:26 -04:00
David Steele
cb7a5f1ef3 Add JSON error when value does not parse in Ini object.
If the JSON value fails to parse it is helpful to have the error message, at least for debugging.
2022-04-20 19:49:23 -04:00
David Steele
da6b4abc58 Handle missing archive start/stop in info/info backup unit test.
This is not a very realistic case since archive start/stop are always written, but it appears in many other unit tests so it should also be tested here.
2022-04-20 19:41:28 -04:00
David Steele
d897bf1ec2 Add size to info/manifest unit test.
This prevents the check from being order dependent.
2022-04-20 19:36:33 -04:00
David Steele
d6f466be2b Remove unnecessary mem contexts in the Manifest object.
The correct context is set by the various *Add() functions so these are not needed and cause leaks, though the leaks will only be noticeable in cases where there are a lot of page checksum errors.
2022-04-20 19:31:35 -04:00
David Steele
da9f261852 Add FUNCTION_TEST_NO_RETURN() macro.
This is required for the (currently) single place where a function with test FUNCTION_TEST*() macros does not return.

This allows return to be added to the FUNCTION_TEST_RETURN_VOID() macro, which means return no longer needs to be added when returning from a function early.
2022-04-20 14:09:49 -04:00
David Steele
c304fafd45
Refactor PgClient to return results in Pack format.
Packs support stronger typing than JSON and are more efficient. For the small result sets that we deal with efficiency is probably not very important, but this removes another place where we are using JSON instead of Pack.

Push checking for result struct (e.g. single row) down into PgClient since it has easy access to this information rather than needing to parse the result set to find out.

Refactor all code downstream that depends on PgClient results.
2022-04-20 08:36:53 -04:00
David Steele
e699402f99 Remove extra linefeed. 2022-04-19 18:07:11 -04:00
David Steele
b7fccaf994 Refactor remote storage protocol to use Packs instead of JSON.
Packs are more efficient and strongly typed so they make more sense for the protocol.
2022-04-18 14:08:53 -04:00
David Steele
cfd6c7ceb4 Use specific integer types in postgres/client and db unit tests.
This will work better once we are able to transmit the results with stronger typing.

Also remove int2 which was not being used.
2022-04-18 12:14:22 -04:00
David Steele
9751ddc4f8 Update postgres/client unit test to conform to current patterns.
This includes adding test titles and using constants for query and error values that repeat.
2022-04-18 11:53:31 -04:00
David Steele
bc5f6fac34 Update postgres/client unit test for changes in libpq.
There have been some behavioral changes in libpq which require changes to the test.

Also update the instructions since it is now a bit easier to run against a real cluster.
2022-04-18 10:47:44 -04:00
David Steele
214ee9eb0e Fix URL for apt.p.o archives.
A new archive repo was created in March of 2020: https://www.df7cb.de/blog/2020/apt-archive.postgresql.org.html
2022-04-17 09:41:22 -04:00
David Steele
d103dd6238 Return stats as a JSON string rather than a KeyValue object.
There is no need to process the stats so a KeyValue is overkill.

Also remove the performance tests that check the stat totals since this is covered in the unit tests.
2022-04-14 20:34:42 -04:00
David Steele
e1ce731f8a Add test for protocol greeting when a field is missing.
A missing field and a NULL field are not exactly the same so it seems best to test both.

Because of the way KeyValue objects work the error is the same, but that will not always be true.
2022-04-14 19:37:03 -04:00
David Steele
8a29d56f3c Update comment in pack module to cover a more common use case.
The KeyValue object is actively being removed so this is no longer the best example.

Instead use an example that should outlive the KeyValue object.
2022-04-14 19:18:00 -04:00
David Steele
aeecd07ad8 Fix reported error line number when ini key length is zero.
The line number was one less than it should have been, which could cause some confusion.

Since this only affected ini files with JSON values, which are always written programmatically, there is almost zero chance this has ever been a problem in the field.
2022-04-14 18:29:54 -04:00
David Steele
47f8e11889 Update FreeBSD and MacOS images for Cirrus CI.
This fixes fe1ac210. Apparently FreeBSD 12.2 went EOL and the image was immediately broken.

Also add FreeBSD 13.0 and update MacOS to Monterey.
2022-04-14 08:13:39 -04:00
David Steele
bcbac8a067 Add static keywords.
This is more efficient than pushing these variables onto the stack.
2022-04-11 17:54:18 -04:00
David Steele
fa40bcdc5c
Throw error when unable to read lock process.
Previously the process id was skipped if it did not exist. Instead, throw an error and handle the errors in downstream code.

This was probably ignored at some point to provide backward-compatibility, but that is no longer required, if it ever was.
2022-04-11 14:08:16 -04:00
David Steele
fe1ac210bb Disable FreeBSD builds on Cirrus CI.
This build has started breaking with the following error:

cd .. && perl ${CIRRUS_WORKING_DIR}/test/test.pl --no-gen --make-cmd=gmake --vm=none --vm-max=2 --no-coverage --no-valgrind --module=command --test=backup
2022-04-11 17:11:53.034 P00   INFO: test begin on amd64 - log level info
2022-04-11 17:11:53.107 P00   INFO: configure build
ld-elf.so.1: /usr/local/lib/perl5/5.32/mach/CORE/libperl.so.5.32: Undefined symbol "strerror_l@FBSD_1.6"

Disable the build to unstick the pipeline until this can be fixed.
2022-04-11 13:17:54 -04:00
David Steele
15021a0e97 Fix whitespace. 2022-04-09 18:29:57 -04:00