db/pgbackrest
1
0
Fork 0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2025-07-15 01:04:37 +02:00
Code Issues Releases Activity

Summarize backup reference list for info command text output.

Browse Source 
The backup reference list can be very long so it seems better to summarize the list by default for text output and keep the full list when --set is specified.
This commit is contained in:
Stefan Fercot
2024-07-31 13:53:02 +02:00
committed by GitHub
parent c42d484c9d
commit b306f83493
4 changed files with 131 additions and 34 deletions
Download Patch File Download Diff File Expand all files Collapse all files

13
doc/xml/release/2024/2.54.xml
View File

@ -1,5 +1,18 @@
<release date="XXXX-XX-XX" version="2.54dev" title="UNDER DEVELOPMENT"> <release date="XXXX-XX-XX" version="2.54dev" title="UNDER DEVELOPMENT">
<release-core-list> <release-core-list>
<release-improvement-list>
<release-item>
<github-pull-request id="2399"/>
<release-item-contributor-list>
<release-item-contributor id="stefan.fercot"/>
<release-item-reviewer id="david.steele"/>
</release-item-contributor-list>
<p>Summarize backup reference list for <cmd>info</cmd> command text output.</p>
</release-item>
</release-improvement-list>
<release-development-list> <release-development-list>
<release-item> <release-item>
<github-pull-request id="2108"/> <github-pull-request id="2108"/>

4
src/build/help/help.xml
View File

@ -2523,7 +2523,7 @@
<p>The '<id>repo</id>' indicates in which repository this backup resides. The '<id>backup set size</id>' includes all the files from this backup and any referenced backups in the repository that are required to restore the database from this backup while '<id>backup size</id>' includes only the files in this backup (these will also be the same for full backups). Repository sizes reflect compressed file sizes if compression is enabled in <backrest/>.</p> <p>The '<id>repo</id>' indicates in which repository this backup resides. The '<id>backup set size</id>' includes all the files from this backup and any referenced backups in the repository that are required to restore the database from this backup while '<id>backup size</id>' includes only the files in this backup (these will also be the same for full backups). Repository sizes reflect compressed file sizes if compression is enabled in <backrest/>.</p>
<p>The '<id>backup reference list</id>' contains the additional backups that are required to restore this backup.</p> <p>The '<id>backup reference total</id>' summarizes the list of additional backups that are required to restore this backup. Use the <br-option>--set</br-option> option to display the complete reference list.</p>
<!-- Note, linking to the User Guide is limited since it can cause a cyclical reference --> <!-- Note, linking to the User Guide is limited since it can cause a cyclical reference -->
</text> </text>
@ -2548,7 +2548,7 @@
<summary>Backup set to detail.</summary> <summary>Backup set to detail.</summary>
<text> <text>
<p>Details include a list of databases (with OIDs) in the backup set (excluding template databases), tablespaces (with OIDs) with the destination where they will be restored by default, and symlinks with the destination where they will be restored when <setting>--link-all</setting> is specified.</p> <p>Details include a complete list of additional backups that are required to restore this backup, a list of databases (with OIDs) in the backup set (excluding template databases), tablespaces (with OIDs) with the destination where they will be restored by default, and symlinks with the destination where they will be restored when <setting>--link-all</setting> is specified.</p>
</text> </text>
<example>20150131-153358F_20150131-153401I</example> <example>20150131-153358F_20150131-153401I</example>

57
src/command/info/info.c
View File

@ -16,6 +16,7 @@ Info Command
#include "common/io/fdWrite.h" #include "common/io/fdWrite.h"
#include "common/log.h" #include "common/log.h"
#include "common/memContext.h" #include "common/memContext.h"
#include "common/regExp.h"
#include "common/type/json.h" #include "common/type/json.h"
#include "config/config.h" #include "config/config.h"
#include "info/info.h" #include "info/info.h"
@ -45,6 +46,9 @@ VARIANT_STRDEF_STATIC(BACKUP_KEY_LINK_VAR, "link");
VARIANT_STRDEF_STATIC(BACKUP_KEY_LSN_VAR, "lsn"); VARIANT_STRDEF_STATIC(BACKUP_KEY_LSN_VAR, "lsn");
VARIANT_STRDEF_STATIC(BACKUP_KEY_PRIOR_VAR, "prior"); VARIANT_STRDEF_STATIC(BACKUP_KEY_PRIOR_VAR, "prior");
VARIANT_STRDEF_STATIC(BACKUP_KEY_REFERENCE_VAR, "reference"); VARIANT_STRDEF_STATIC(BACKUP_KEY_REFERENCE_VAR, "reference");
VARIANT_STRDEF_STATIC(BACKUP_KEY_REFERENCE_FULL_VAR, "reference-full");
VARIANT_STRDEF_STATIC(BACKUP_KEY_REFERENCE_DIFF_VAR, "reference-diff");
VARIANT_STRDEF_STATIC(BACKUP_KEY_REFERENCE_INCR_VAR, "reference-incr");
VARIANT_STRDEF_STATIC(BACKUP_KEY_TABLESPACE_VAR, "tablespace"); VARIANT_STRDEF_STATIC(BACKUP_KEY_TABLESPACE_VAR, "tablespace");
VARIANT_STRDEF_STATIC(BACKUP_KEY_TIMESTAMP_VAR, "timestamp"); VARIANT_STRDEF_STATIC(BACKUP_KEY_TIMESTAMP_VAR, "timestamp");
VARIANT_STRDEF_STATIC(BACKUP_KEY_TYPE_VAR, "type"); VARIANT_STRDEF_STATIC(BACKUP_KEY_TYPE_VAR, "type");
@ -436,6 +440,38 @@ backupListAdd(
varKv(backupInfo), BACKUP_KEY_REFERENCE_VAR, varKv(backupInfo), BACKUP_KEY_REFERENCE_VAR,
(backupData->backupReference != NULL ? varNewVarLst(varLstNewStrLst(backupData->backupReference)) : NULL)); (backupData->backupReference != NULL ? varNewVarLst(varLstNewStrLst(backupData->backupReference)) : NULL));
// Display complete reference list only for json output or --set text. Otherwise keep track of full, diff and incremental to
// display a summary.
if (backupData->backupReference != NULL && backupLabel == NULL && !outputJson)
{
StringList *const fullList = strLstNew();
StringList *const diffList = strLstNew();
StringList *const incrList = strLstNew();
for (unsigned int backupIdx = 0; backupIdx < strLstSize(backupData->backupReference); backupIdx++)
{
const String *const backupReferenceLabel = strLstGet(backupData->backupReference, backupIdx);
if (regExpMatchOne(backupRegExpP(.full = true), backupReferenceLabel))
strLstAdd(fullList, backupReferenceLabel);
if (regExpMatchOne(backupRegExpP(.differential = true), backupReferenceLabel))
strLstAdd(diffList, backupReferenceLabel);
if (regExpMatchOne(backupRegExpP(.incremental = true), backupReferenceLabel))
strLstAdd(incrList, backupReferenceLabel);
}
// The reference list will always contain at least 1 full
kvPut(varKv(backupInfo), BACKUP_KEY_REFERENCE_FULL_VAR, varNewVarLst(varLstNewStrLst(fullList)));
kvPut(
varKv(backupInfo), BACKUP_KEY_REFERENCE_DIFF_VAR,
strLstSize(diffList) > 0 ? varNewVarLst(varLstNewStrLst(diffList)) : NULL);
kvPut(
varKv(backupInfo), BACKUP_KEY_REFERENCE_INCR_VAR,
strLstSize(incrList) > 0 ? varNewVarLst(varLstNewStrLst(incrList)) : NULL);
}
// archive section // archive section
KeyValue *const archiveInfo = kvPutKv(varKv(backupInfo), KEY_ARCHIVE_VAR); KeyValue *const archiveInfo = kvPutKv(varKv(backupInfo), KEY_ARCHIVE_VAR);
@ -954,8 +990,25 @@ formatTextBackup(const DbGroup *const dbGroup, String *const resultStr)
if (kvGet(backupInfo, BACKUP_KEY_REFERENCE_VAR) != NULL) if (kvGet(backupInfo, BACKUP_KEY_REFERENCE_VAR) != NULL)
{ {
const StringList *const referenceList = strLstNewVarLst(varVarLst(kvGet(backupInfo, BACKUP_KEY_REFERENCE_VAR))); if (kvGet(backupInfo, BACKUP_KEY_REFERENCE_FULL_VAR) != NULL)
strCatFmt(resultStr, " backup reference list: %s\n", strZ(strLstJoin(referenceList, ", "))); {
const String *const diffListSizeText =
kvGet(backupInfo, BACKUP_KEY_REFERENCE_DIFF_VAR) != NULL ?
strNewFmt(", %u diff", varLstSize(varVarLst(kvGet(backupInfo, BACKUP_KEY_REFERENCE_DIFF_VAR)))) : EMPTY_STR;
const String *const incrListSizeText =
kvGet(backupInfo, BACKUP_KEY_REFERENCE_INCR_VAR) != NULL ?
strNewFmt(", %u incr", varLstSize(varVarLst(kvGet(backupInfo, BACKUP_KEY_REFERENCE_INCR_VAR)))) : EMPTY_STR;
strCatFmt(
resultStr, " backup reference total: %u full%s%s\n",
varLstSize(varVarLst(kvGet(backupInfo, BACKUP_KEY_REFERENCE_FULL_VAR))), strZ(diffListSizeText),
strZ(incrListSizeText));
}
else
{
const StringList *const referenceList = strLstNewVarLst(varVarLst(kvGet(backupInfo, BACKUP_KEY_REFERENCE_VAR)));
strCatFmt(resultStr, " backup reference list: %s\n", strZ(strLstJoin(referenceList, ", ")));
}
} }
if (kvGet(backupInfo, BACKUP_KEY_DATABASE_REF_VAR) != NULL) if (kvGet(backupInfo, BACKUP_KEY_DATABASE_REF_VAR) != NULL)

91
test/src/module/command/infoTest.c
View File

@ -1547,14 +1547,14 @@ testRun(void)
" wal start/stop: 000000010000000000000003 / 000000020000000000000003\n" " wal start/stop: 000000010000000000000003 / 000000020000000000000003\n"
" database size: 19.2MB, database backup size: 8.2KB\n" " database size: 19.2MB, database backup size: 8.2KB\n"
" repo1: backup set size: 2.3MB, backup size: 346B\n" " repo1: backup set size: 2.3MB, backup size: 346B\n"
" backup reference list: 20181119-152138F\n" " backup reference total: 1 full\n"
"\n" "\n"
" incr backup: 20181119-152138F_20181119-152155I\n" " incr backup: 20181119-152138F_20181119-152155I\n"
" timestamp start/stop: 2018-11-19 15:21:55+00 / 2018-11-19 15:21:57+00\n" " timestamp start/stop: 2018-11-19 15:21:55+00 / 2018-11-19 15:21:57+00\n"
" wal start/stop: n/a\n" " wal start/stop: n/a\n"
" database size: 19.2MB, database backup size: 8.2KB\n" " database size: 19.2MB, database backup size: 8.2KB\n"
" repo1: backup set size: 2.3MB, backup size: 346B\n" " repo1: backup set size: 2.3MB, backup size: 346B\n"
" backup reference list: 20181119-152138F, 20181119-152138F_20181119-152152D\n" " backup reference total: 1 full, 1 diff\n"
"\n" "\n"
" db (current)\n" " db (current)\n"
" wal archive min/max (9.5): 000000010000000000000002/000000010000000000000005\n" " wal archive min/max (9.5): 000000010000000000000002/000000010000000000000005\n"
@ -1577,7 +1577,7 @@ testRun(void)
" wal start/stop: 000000010000000000000005 / 000000010000000000000005\n" " wal start/stop: 000000010000000000000005 / 000000010000000000000005\n"
" database size: 19.2MB, database backup size: 8.2KB\n" " database size: 19.2MB, database backup size: 8.2KB\n"
" repo1: backup size: 346B\n" " repo1: backup size: 346B\n"
" backup reference list: 20201116-155000F\n" " backup reference total: 1 full\n"
"\n" "\n"
"stanza: stanza2\n" "stanza: stanza2\n"
" status: mixed (backup/expire running - 55.55% complete)\n" " status: mixed (backup/expire running - 55.55% complete)\n"
@ -2565,14 +2565,14 @@ testRun(void)
" wal start/stop: 000000010000000000000003 / 000000020000000000000003\n" " wal start/stop: 000000010000000000000003 / 000000020000000000000003\n"
" database size: 19.2MB, database backup size: 8.2KB\n" " database size: 19.2MB, database backup size: 8.2KB\n"
" repo1: backup set size: 2.3MB, backup size: 346B\n" " repo1: backup set size: 2.3MB, backup size: 346B\n"
" backup reference list: 20181119-152138F\n" " backup reference total: 1 full\n"
"\n" "\n"
" incr backup: 20181119-152138F_20181119-152155I\n" " incr backup: 20181119-152138F_20181119-152155I\n"
" timestamp start/stop: 2018-11-19 15:21:55+00 / 2018-11-19 15:21:57+00\n" " timestamp start/stop: 2018-11-19 15:21:55+00 / 2018-11-19 15:21:57+00\n"
" wal start/stop: n/a\n" " wal start/stop: n/a\n"
" database size: 19.2MB, database backup size: 8.2KB\n" " database size: 19.2MB, database backup size: 8.2KB\n"
" repo1: backup set size: 2.3MB, backup size: 346B\n" " repo1: backup set size: 2.3MB, backup size: 346B\n"
" backup reference list: 20181119-152138F, 20181119-152138F_20181119-152152D\n" " backup reference total: 1 full, 1 diff\n"
"\n" "\n"
" db (current)\n" " db (current)\n"
" wal archive min/max (9.5): 000000010000000000000002/000000010000000000000005\n" " wal archive min/max (9.5): 000000010000000000000002/000000010000000000000005\n"
@ -2588,7 +2588,7 @@ testRun(void)
" wal start/stop: 000000010000000000000005 / 000000010000000000000005\n" " wal start/stop: 000000010000000000000005 / 000000010000000000000005\n"
" database size: 19.2MB, database backup size: 8.2KB\n" " database size: 19.2MB, database backup size: 8.2KB\n"
" repo1: backup size: 346B\n" " repo1: backup size: 346B\n"
" backup reference list: 20201116-155000F\n", " backup reference total: 1 full\n",
"text - multi-repo, valid backups only on repo1"); "text - multi-repo, valid backups only on repo1");
// Remove archives for prior backup so archiveMin prior DB == NULL but backupList > 0 (edge case) // Remove archives for prior backup so archiveMin prior DB == NULL but backupList > 0 (edge case)
@ -2598,6 +2598,53 @@ testRun(void)
HRN_STORAGE_PATH_REMOVE( HRN_STORAGE_PATH_REMOVE(
storageRepoIdxWrite(0), STORAGE_REPO_ARCHIVE "/9.4-1", .recurse = true, .comment = "remove archives on db prior"); storageRepoIdxWrite(0), STORAGE_REPO_ARCHIVE "/9.4-1", .recurse = true, .comment = "remove archives on db prior");
HRN_INFO_PUT(
storageTest, TEST_PATH "/repo/" STORAGE_PATH_BACKUP "/stanza1/" INFO_BACKUP_FILE,
"[backup:current]\n"
"20201116-155000F={"
"\"backrest-format\":5,\"backrest-version\":\"2.30\","
"\"backup-archive-start\":\"000000010000000000000002\",\"backup-archive-stop\":\"000000010000000000000003\","
"\"backup-info-repo-size\":3159000,\"backup-info-repo-size-delta\":3100,\"backup-info-size\":26897000,"
"\"backup-info-size-delta\":26897020,\"backup-timestamp-start\":1605541800,\"backup-timestamp-stop\":1605541802,"
"\"backup-type\":\"full\",\"db-id\":1,\"option-archive-check\":true,\"option-archive-copy\":true,"
"\"option-backup-standby\":false,\"option-checksum-page\":false,\"option-compress\":false,\"option-hardlink\":false,"
"\"option-online\":true}\n"
"20201116-155000F_20201119-152100I={"
"\"backrest-format\":5,\"backrest-version\":\"2.30\","
"\"backup-annotation\":{\"extra key\":\"this is an annotation\",\"source\":\"this is another annotation\"},"
"\"backup-archive-start\":\"000000010000000000000005\",\"backup-archive-stop\":\"000000010000000000000005\","
"\"backup-error\":false,\"backup-info-repo-size\":2369186,"
"\"backup-info-repo-size-delta\":346,\"backup-info-repo-size-map\":100,\"backup-info-repo-size-map-delta\":12"
",\"backup-info-size\":20162900,\"backup-info-size-delta\":8428,\"backup-lsn-start\":\"285/89000028\","
"\"backup-prior\":\"20201116-155000F\",\"backup-reference\":[\"20201116-155000F\"],"
"\"backup-timestamp-start\":1605799260,\"backup-timestamp-stop\":1605799263,\"backup-type\":\"incr\","
"\"db-id\":1,\"option-archive-check\":true,\"option-archive-copy\":false,\"option-backup-standby\":false,"
"\"option-checksum-page\":true,\"option-compress\":true,\"option-hardlink\":false,\"option-online\":true}\n"
"20201116-155000F_20201120-152100I={"
"\"backrest-format\":5,\"backrest-version\":\"2.30\","
"\"backup-annotation\":{\"extra key\":\"this is an annotation\",\"source\":\"this is another annotation\"},"
"\"backup-archive-start\":\"000000010000000000000006\",\"backup-archive-stop\":\"000000010000000000000006\","
"\"backup-error\":false,\"backup-info-repo-size\":2369186,"
"\"backup-info-repo-size-delta\":346,\"backup-info-repo-size-map\":100,\"backup-info-repo-size-map-delta\":12"
",\"backup-info-size\":20162900,\"backup-info-size-delta\":8428,\"backup-lsn-start\":\"285/89000028\","
"\"backup-prior\":\"20201116-155000F_20201119-152100I\","
"\"backup-reference\":[\"20201116-155000F\",\"20201116-155000F_20201119-152100I\"],"
"\"backup-timestamp-start\":1605799260,\"backup-timestamp-stop\":1605799263,\"backup-type\":\"incr\","
"\"db-id\":1,\"option-archive-check\":true,\"option-archive-copy\":false,\"option-backup-standby\":false,"
"\"option-checksum-page\":true,\"option-compress\":true,\"option-hardlink\":false,\"option-online\":true}\n"
"\n"
"[db]\n"
"db-catalog-version=201510051\n"
"db-control-version=942\n"
"db-id=1\n"
"db-system-id=6626363367545678089\n"
"db-version=\"9.5\"\n"
"\n"
"[db:history]\n"
"1={\"db-catalog-version\":201510051,\"db-control-version\":942,\"db-system-id\":6626363367545678089"
",\"db-version\":\"9.5\"}\n",
.comment = "put backup info to file - stanza1, repo1");
TEST_RESULT_STR_Z( TEST_RESULT_STR_Z(
infoRender(), infoRender(),
"stanza: stanza1\n" "stanza: stanza1\n"
@ -2608,29 +2655,6 @@ testRun(void)
" repo1: none\n" " repo1: none\n"
" repo2: aes-256-cbc\n" " repo2: aes-256-cbc\n"
"\n" "\n"
" db (prior)\n"
" wal archive min/max (9.4): none present\n"
"\n"
" full backup: 20181119-152138F\n"
" timestamp start/stop: 2018-11-19 15:21:38+00 / 2018-11-19 15:21:39+00\n"
" wal start/stop: 000000010000000000000002 / 000000010000000000000002\n"
" database size: 19.2MB, database backup size: 19.2MB\n"
" repo1: backup set size: 2.3MB, backup size: 2.3MB\n"
"\n"
" diff backup: 20181119-152138F_20181119-152152D\n"
" timestamp start/stop: 2018-11-19 15:21:52+00 / 2018-11-19 15:21:55+00\n"
" wal start/stop: 000000010000000000000003 / 000000020000000000000003\n"
" database size: 19.2MB, database backup size: 8.2KB\n"
" repo1: backup set size: 2.3MB, backup size: 346B\n"
" backup reference list: 20181119-152138F\n"
"\n"
" incr backup: 20181119-152138F_20181119-152155I\n"
" timestamp start/stop: 2018-11-19 15:21:55+00 / 2018-11-19 15:21:57+00\n"
" wal start/stop: n/a\n"
" database size: 19.2MB, database backup size: 8.2KB\n"
" repo1: backup set size: 2.3MB, backup size: 346B\n"
" backup reference list: 20181119-152138F, 20181119-152138F_20181119-152152D\n"
"\n"
" db (current)\n" " db (current)\n"
" wal archive min/max (9.5): 000000010000000000000002/000000010000000000000005\n" " wal archive min/max (9.5): 000000010000000000000002/000000010000000000000005\n"
"\n" "\n"
@ -2645,7 +2669,14 @@ testRun(void)
" wal start/stop: 000000010000000000000005 / 000000010000000000000005\n" " wal start/stop: 000000010000000000000005 / 000000010000000000000005\n"
" database size: 19.2MB, database backup size: 8.2KB\n" " database size: 19.2MB, database backup size: 8.2KB\n"
" repo1: backup size: 346B\n" " repo1: backup size: 346B\n"
" backup reference list: 20201116-155000F\n", " backup reference total: 1 full\n"
"\n"
" incr backup: 20201116-155000F_20201120-152100I\n"
" timestamp start/stop: 2020-11-19 15:21:00+00 / 2020-11-19 15:21:03+00\n"
" wal start/stop: 000000010000000000000006 / 000000010000000000000006\n"
" database size: 19.2MB, database backup size: 8.2KB\n"
" repo1: backup size: 346B\n"
" backup reference total: 1 full, 1 incr\n",
"text - multi-repo, prior backup: no archives but backups (code coverage)"); "text - multi-repo, prior backup: no archives but backups (code coverage)");
// ------------------------------------------------------------------------------------------------------------------------- // -------------------------------------------------------------------------------------------------------------------------