1
0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2024-12-14 10:13:05 +02:00
Commit Graph

3860 Commits

Author SHA1 Message Date
David Steele
03c71aa606 Add hint to check the log on archive-get/archive-push async error.
If this error is thrown rather than a specific error returned from the async process, it means the async process is unable to write the status files for some reason and the only way to get the error is out of the async log.

This hint includes the exact async log path and name to make finding errors easier.
2022-05-02 08:49:13 -04:00
David Steele
126fc99c77 Fix leaks in common/json, common/keyValue, and common/variantList.
This doesn't solve the problem of the variant code making far too many copies, but it at least plugs the leaks.

jsonReadVarRecurse() could leak KeyValue and VariantList.

kvDup() leaked object allocations into the calling context.

kvDefault() gets a more efficient return structure.

kvGetList() leaked a Variant into the calling context.

varLstNewStrLst() leaked object allocations into the calling context. Update varLstDup() to reflect changes made in varLstNewStrLst().
2022-04-28 14:10:53 -04:00
David Steele
96166539cf Fix leaked String in cfgParseSize(). 2022-04-28 13:33:03 -04:00
David Steele
4872a3f121 Improvements to test harness memory debugging.
Only set -DDEBUG_MEM for the modules currently being tested rather than globally.

Also run tests in a temp mem context. Running in the top context can confuse memory accounting when a new context is created in the top context.
2022-04-28 12:33:39 -04:00
David Steele
90f939b36f Fix leaks in common/io unit test.
These leaks make it harder to detect leaks in the core code, so fix them.
2022-04-28 12:31:59 -04:00
David Steele
8047e97e31 Fix leaked String and Variant in harnessPqScriptRun(). 2022-04-28 12:17:33 -04:00
David Steele
ceb303f9e2 Fix comment typo. 2022-04-28 11:46:55 -04:00
David Steele
c463993b4c Fix leaks in the storage/s3 module.
storageS3Helper() leaked a few Strings which ended up in a long-lived context.

storageS3AuthAuto() and storageS3AuthWebId() were cleaned up by their callers but since they are not called often a temp mem context seems better.

storageS3Request() leaked an HttpRequest.

storageS3Info() leaked an HttpResponse.

storageS3PathRemoveInternal() leaked a variety of objects. Fix by freeing some of them and adding a temp mem context.

storageS3Remove() leaked an HttpResponse object.

storageWriteS3Part() leaked an HttpResponse object.
2022-04-28 11:41:16 -04:00
David Steele
4750bc94dd Fix leaks in the storage/remote module.
storageRemoteFilterGroup() leaked a number of objects. Use a temp mem context to prevent that.

storageRemoteProtocolInfoListCallback() leaked a PackWrite.

storageWriteRemoteFreeResource() leaked a PackWrite.
2022-04-28 11:20:31 -04:00
David Steele
c123a6af9f Fix leaks in the storage/posix module.
storagePosixPathCreate() leaked a String.

storagePosixPathRemoveCallback() leaked a String.
2022-04-28 11:19:01 -04:00
David Steele
d89bc6f2d2 Fix leaks in the storage/gcs module.
storageGcsAuthToken() memory was being cleaned up by the calling context, but seems better to keep this tidy and add a temp mem context.

storageGcsRequest() leaked an HttpRequest.

storageGcsInfo() leaked a number of objects. Use a temp mem context to prevent that.

storageGcsPathRemoveCallback() leaked an HttpResponse.

storageGcsRemove() leaked an HttpReponse.

storageWriteGcsVerify() leaked a number of objects. Use a temp mem context to prevent that.

storageWriteGcsBlock) leaked an HttpReponse.
2022-04-28 10:53:11 -04:00
David Steele
083c93eaa3 Reuse Strings in iniLoad().
Reuse the section/key/value Strings by truncating them instead of creating a new one every time.

Also add an error for empty sections. This function is only used for loading info files (not config files), which should never contain an empty section.
2022-04-28 10:11:15 -04:00
David Steele
bc46d4e37b Add cvtZSubNTo*() functions.
These functions allow conversion from substrings without needing to create a String or a temporary buffer.

httpDateToTime() no longer requires a temp mem context. Also improve handling of month search to avoid an allocation.

httpUriDecode() no longer requires a temp mem context.

jsonReadStr() no longer requires a temp mem context.

pgLsnFromWalSegment() no longer requires a temp mem context.

pgVersionFromStr() no longer requires a temp mem context. Also do a bit of refactoring.

storageGcsCvtTime() no longer leaks six Strings per call.

storageS3CvtTime() no longer leaks six Strings per call.
2022-04-28 09:50:23 -04:00
David Steele
6e18235be8 Fix leak in varLstNewStrLst().
The duplicated list elements were leaked into the calling context.
2022-04-28 06:52:07 -04:00
David Steele
aeb7568a0f Remove unused header. 2022-04-28 06:48:02 -04:00
David Steele
5285b9aa7b Fix incorrect test param. 2022-04-27 20:18:18 -04:00
David Steele
eb65a5674d Use OBJ_NEW*() macros in SocketServer object.
This was missed in ccc255d3 when the TLS server was introduced, probably because work on that commit preceded when the macros were introduced in 475b57c8. It would have been easy to miss in a merge.
2022-04-27 11:40:29 -04:00
David Steele
6f2654a5eb Fix leaks in the storage/azure module.
storageAzureHelper() leaked a few Strings which ended up in a long-lived context.

storageAzureNew() failed to make a copy of the endpoint. This worked because storageAzureHelper() leaked the endpoint into the long-lived parent context.

storageAzureRequest() leaked an HttpRequest.

storageAzureInfo() leaked an HttpResponse.

storageAzurePathRemoveCallback() leaked an HttpResponse.

storageAzureRemove() leaked an HttpResponse.
2022-04-26 12:53:55 -04:00
David Steele
be120c746c Refactor and fix leak in storageRepoPathExpression().
storageRepoPathExpression() could leak a StringList. Also refactor to remove unneeded assignments and create the String one time.
2022-04-26 12:27:51 -04:00
David Steele
cca6df872a Refactor functions in postgres/interface module and fix leak.
pgLsnFromWalSegment() leaked two Strings.

Refactor pgLsnRangeToWalSegmentList() to create the StringList in the calling context rather than moving it later.
2022-04-26 12:09:44 -04:00
David Steele
a56fa0eb45 Fix leaks in protocol module.
These leaks were not a big deal individually and there are generally few protocol objects created, but the leaks ended up in mem contexts that persist for most of execution. It makes sense to keep long-lived contexts as tidy as possible.
2022-04-26 11:59:21 -04:00
David Steele
78e912a932 Fix leaks in info module.
*LoadFileCallback() all leaked between retries.

infoPgArchiveId() leaked a String.
2022-04-26 11:20:51 -04:00
David Steele
36b0a9fa58 Fix leaks in db module.
dbFreeResource() leaked ProtocolCommand.

dbTimeMSec() leaked PackRead.
2022-04-26 11:07:05 -04:00
David Steele
78b90e5ad8 Fix leak in configOptionRemote().
A Pack and String were leaked.
2022-04-26 11:03:06 -04:00
David Steele
9a6df39839 Fix spacing. 2022-04-26 11:00:20 -04:00
David Steele
8efb4e1e7f Fix leak in cfgCommandJobRetry().
A Variant was leaked for each retry record.

Also remove an extra linefeed.
2022-04-26 10:58:12 -04:00
David Steele
a8fed52ecb Fix leak in pckReadStrLst().
A String would be leaked on each loop iteration.
2022-04-26 10:53:13 -04:00
David Steele
d7e45f12a5 Refactor httpRequestNew().
Move httpRequestProcess() outside of the object mem context. In case it ever returns a value we don't want that to end up in the object context.
2022-04-26 10:49:04 -04:00
David Steele
cd1cf337ff Fix leak in httpHeaderPutRange().
The range string was leaked.
2022-04-26 10:45:30 -04:00
David Steele
07e2801997 Fix leak in ioFilterGroupResultAllSet().
The PackRead containing the filter results was leaked.
2022-04-26 10:39:28 -04:00
David Steele
3310decf8e Add missing FUNCTION_TEST*() macros. 2022-04-26 10:36:27 -04:00
David Steele
7eed9730aa Fix leak in cipherBlockNew().
The string used to look up the cipher type was leaked.
2022-04-26 10:34:10 -04:00
David Steele
6eed4125e6 Improved memory management in the common/exit module.
Refactor so that error detail is only logged in one place. This reduces calls to exitErrorDetail() and LOG_INTERNAL_FMT().

Fix minor leaks in exitErrorDetail() and exitSafe().
2022-04-26 10:29:48 -04:00
David Steele
fa6c68cb02 Mem allocation cleanup in command/verify module.
Move the temp mem context out of verifyJobCallback() into verifyBackup() and verifyArchive(). This makes it clearer that verifyJobCallback() allocates no memory and reduces mem usage when both verifyBackup() and verifyArchive() are called.

Update verifyErrorMsg() to return zero-terminated strings to save on allocations. The output of this function is used when formatting strings so this is also simpler. Do a similar thing in verifyRender().

Also fix a minor leak in verifyInfoFile().
2022-04-26 10:18:24 -04:00
David Steele
3f7c8bc923 Fix object allocations in incorrect mem context in execOpen().
Object variables were begin allocated in the calling context rather than the object context.

This is not a live bug because Exec objects are currently created and opened in a long-lived context.
2022-04-26 10:15:47 -04:00
Reid Thompson
4dbe76a5bb Fix comment formatting. 2022-04-26 07:33:57 -04:00
David Steele
41f9d69edc Combine functions in the command/stanza module into one function.
It is not clear why these were split out, but it probably had something to do with testing before storageList() could return NULL for an empty directory.

Also remove the tests that depended on a boolean return, which are no longer needed for coverage.
2022-04-25 15:38:49 -04:00
David Steele
55a828f999 Add temporary mem contexts and fix a leak in the command/restore module.
restoreRecoveryConf() and restoreRecoveryWriteConf() do enough work to deserve their own memory contexts.

restoreFilePgPath() was leaking a String every time it was called, which could be a lot.

Also fix a spacing issue.
2022-04-25 15:03:37 -04:00
David Steele
4e7414d48f Add temporary mem contexts in the command/help module.
These were not really leaks since memory was being freed by the calling function, but these functions do enough work to deserve their own memory contexts.
2022-04-25 14:49:08 -04:00
David Steele
fd295f002b Add temporary mem context to removeExpiredBackup().
This was not really a leak since memory was being freed by the calling function, but this function does enough work to deserve its own memory context.

Also fixed a doubled semicolon.
2022-04-25 14:19:10 -04:00
David Steele
9314be36b1 Improve result declaration in backupRegExp().
No need to set this to NULL when the initial value can be used instead.
2022-04-25 14:14:29 -04:00
David Steele
5b5dbe0e7e Fix memory leaks in archivePushDrop().
A string was leaked for every file processed.

Since the list can be quite large it also makes sense to reset the context occasionally.
2022-04-25 13:29:43 -04:00
David Steele
c3b08f71ce Fix formatting in common/string module. 2022-04-25 13:26:09 -04:00
David Steele
40ef64f2be Fix memory leaks in archivePushFile().
The errorList is only used when throwing an error and the joined list is not needed after the error is thrown, so put both in the temp mem context.
2022-04-25 12:59:46 -04:00
David Steele
774db65086 Remove useless context switches in archiveGetFile()/archivePushFile().
These context switches do nothing since the list is already in the prior context.
2022-04-25 12:55:51 -04:00
David Steele
582c3dab4c Add strLstAddSub*() and strLstAddSubZ*() functions.
These help with readability and remove a cause of leaks.
2022-04-25 12:32:33 -04:00
David Steele
ff45f463cf Use strLstAddZ() instead of strLstAdd() where possible.
Using STRDEF() to convert the zero-terminated string to a String has no performance advantage but generates more code.
2022-04-25 11:58:30 -04:00
David Steele
7900660d3a Add strLstNewFmt().
Simplifies adding a formatted string to a list and removes a common cause of leaks.
2022-04-25 11:47:43 -04:00
David Steele
3475514b61 Fix memory leak in archiveAsyncErrorClear().
A string was leaked on each call.
2022-04-25 10:57:36 -04:00
David Steele
23d645b5e7 Fix outdated comment. 2022-04-25 10:52:13 -04:00