db/pgbackrest
1
0
Fork 0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2025-01-30 05:39:12 +02:00
Code Issues Releases Activity

Improve check command to verify that the backup manifest can be built.

Browse Source 
Contributed by Cynthia Shang.
This commit is contained in:
Cynthia Shang 2017-12-13 11:16:27 -05:00 committed by David Steele
parent 0bfda1a908
commit e5511d0f82
10 changed files with 210 additions and 80 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
doc/xml/release.xml
View File

@ -29,6 +29,14 @@
</release-feature-list>
<release-improvement-list>
<release-item>
<release-item-contributor-list>
<release-item-contributor id="shang.cynthia"/>
</release-item-contributor-list>
<p>Improve <cmd>check</cmd> command to verify that the backup manifest can be built.</p>
</release-item>
<release-item>
<p>The C library is now required. This eliminates conditional loading and eases development of new library features.</p>
</release-item>

46
doc/xml/user-guide.xml
View File

@ -2496,7 +2496,7 @@
</execute>
</execute-list>
<p>Test configuration using the <cmd>check</cmd> command. The warning on the <host>backup</host> host regarding the standby being down is expected and can be ignored.</p>
<p>Test configuration using the <cmd>check</cmd> command.</p>
<execute-list host="{[host-db-primary]}">
<title>Check configuration</title>
@ -2510,14 +2510,6 @@
</execute>
</execute-list>
<execute-list host="{[host-backup]}">
<title>Check configuration</title>
<execute user="backrest" output="y" filter="n" >
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} check</exe-cmd>
</execute>
</execute-list>
<p>Remove the old cluster.</p>
<execute-list host="{[host-db-primary]}">
@ -2532,16 +2524,6 @@
</execute>
</execute-list>
<p>Run a full backup on the new cluster and then restore the standby from the backup. The backup type will automatically be changed to <id>full</id> if <id>incr</id> or <id>diff</id> is requested.</p>
<execute-list host="{[host-backup]}">
<title>Run a full backup</title>
<execute user="backrest">
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} {[dash]}-type=full backup</exe-cmd>
</execute>
</execute-list>
<p>Install the new <postgres/> binaries on the standby and create the cluster.</p>
<execute-list host="{[host-db-standby]}">
@ -2566,6 +2548,26 @@
</execute>
</execute-list>
<p>Run the <cmd>check</cmd> on the backup host. The warning regarding the standby being down is expected since the standby cluster is down. Running this command demostrates that the backup server is aware of the standby and is confirgured properly for the primary server.</p>
<execute-list host="{[host-backup]}">
<title>Check configuration</title>
<execute user="backrest" output="y" filter="n" >
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} check</exe-cmd>
</execute>
</execute-list>
<p>Run a full backup on the new cluster and then restore the standby from the backup. The backup type will automatically be changed to <id>full</id> if <id>incr</id> or <id>diff</id> is requested.</p>
<execute-list host="{[host-backup]}">
<title>Run a full backup</title>
<execute user="backrest">
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} {[dash]}-type=full backup</exe-cmd>
</execute>
</execute-list>
<execute-list host="{[host-db-standby]}">
<title>Restore the {[postgres-cluster-demo]} standby cluster</title>
@ -2581,7 +2583,7 @@
</postgres-config>
<execute-list host="{[host-db-standby]}">
<title>Start <postgres/></title>
<title>Start <postgres/> and check the <backrest/> configuration</title>
<execute user="root">
<exe-cmd>{[db-cluster-start-upgrade]}</exe-cmd>
@ -2590,6 +2592,10 @@
<execute show="n">
<exe-cmd>{[db-cluster-wait]}</exe-cmd>
</execute>
<execute user="postgres" output="y" filter="n" >
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} check</exe-cmd>
</execute>
</execute-list>
<p>Backup from standby can be enabled now that the standby is restored.</p>

67
lib/pgBackRest/Check/Check.pm
View File

@ -55,17 +55,9 @@ sub process
# Assign function parameters, defaults, and log debug info
my $strOperation = logDebugParam(__PACKAGE__ . '->process');
# Initialize the database object. This will also check the configured replicas and throw an error if at least one is not
# able to be connected to and warnings for any that cannot be properly connected to.
my ($oDb) = dbObjectGet();
# Validate the database configuration
$oDb->configValidate();
# Get the timeout and error message to display - if it is 0 we are testing
# Initialize the local variables
my $iArchiveTimeout = cfgOption(CFGOPT_ARCHIVE_TIMEOUT);
# Initialize the result variables
my $iResult = 0;
my $strResultMessage = undef;
@ -73,6 +65,59 @@ sub process
my $strArchiveFile = undef;
my $strWalSegment = undef;
# Get the master database object to test to see if the manifest can be built
my ($oDb) = dbMasterGet();
# Get the databse version to pass to the manifest constructor and the system-id in the event of a failure
my ($strDbVersion, $iControlVersion, $iCatalogVersion, $ullDbSysId) = $oDb->info();
# Turn off console logging to control when to display the error
logLevelSet(undef, OFF);
# Loop through all defined databases and attempt to build a manifest
for (my $iRemoteIdx = 1; $iRemoteIdx <= cfgOptionIndexTotal(CFGOPT_DB_HOST); $iRemoteIdx++)
{
# Make sure a db is defined for this index
if (cfgOptionTest(cfgOptionIdFromIndex(CFGOPT_DB_PATH, $iRemoteIdx)) ||
cfgOptionTest(cfgOptionIdFromIndex(CFGOPT_DB_HOST, $iRemoteIdx)))
{
eval
{
# Passing file location dev/null so that the save will fail if it is ever attempted. Pass a miscellaneus value for
# encryption key since the file will not be saved.
my $oBackupManifest = new pgBackRest::Manifest("/dev/null/manifest.chk",
{bLoad => false, strDbVersion => $strDbVersion,
strCipherPass => 'x',
strCipherPassSub => 'x'});
$oBackupManifest->build(storageDb({iRemoteIdx => $iRemoteIdx}),
cfgOption(cfgOptionIdFromIndex(CFGOPT_DB_PATH, $iRemoteIdx)), undef, cfgOption(CFGOPT_ONLINE));
return true;
}
or do
{
# Capture error information
$strResultMessage = "Database: ${strDbVersion} ${ullDbSysId} " . exceptionMessage($EVAL_ERROR) .
(($iResult != 0) ? "\n[$iResult] : $strResultMessage" : "");
$iResult = exceptionCode($EVAL_ERROR);
};
}
}
# Reset the console logging
logLevelSet(undef, cfgOption(CFGOPT_LOG_LEVEL_CONSOLE));
# If the manifest builds are ok, then proceed with the other checks
if ($iResult == 0)
{
# Reinitialize the database object in order to check the configured replicas. This will throw an error if at least one is not
# able to be connected to and warnings for any that cannot be properly connected to.
($oDb) = dbObjectGet();
# Validate the database configuration
$oDb->configValidate();
# Turn off console logging to control when to display the error
logLevelSet(undef, OFF);
@ -129,6 +174,7 @@ sub process
# Reset the console logging
logLevelSet(undef, cfgOption(CFGOPT_LOG_LEVEL_CONSOLE));
}
# If the archiving was successful and backup.info check did not error in an unexpected way, then indicate success
# Else, log the error.
@ -147,10 +193,11 @@ sub process
}
else
{
# Throw the captured error
&log(ERROR, $strResultMessage, $iResult);
# If a WAL switch was attempted, then alert the user that the WAL that did not reach the archive
if (defined($strWalSegment))
if (defined($strWalSegment) && !defined($strArchiveFile))
{
&log(WARN,
"WAL segment ${strWalSegment} did not reach the archive:" . (defined($strArchiveId) ? $strArchiveId : '') . "\n" .

4
test/expect/real-all-001.log
View File

@ -49,6 +49,10 @@ check db - fail on backup info mismatch (db-master host)
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --log-level-console=detail --archive-timeout=5 --stanza=db check
------------------------------------------------------------------------------------------------------------------------------------
check db - confirm master manifest->build executed (db-master host)
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --log-level-console=detail --archive-timeout=5 --stanza=db check
------------------------------------------------------------------------------------------------------------------------------------
check db - verify success after backup (db-master host)
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --log-level-console=detail --archive-timeout=5 --stanza=db check
------------------------------------------------------------------------------------------------------------------------------------

4
test/expect/real-all-002.log
View File

@ -144,6 +144,10 @@ log-path=[TEST_PATH]/db-standby/log
protocol-timeout=60
spool-path=[TEST_PATH]/db-standby/spool
check db - confirm standby manifest->build executed (db-standby host)
> [CONTAINER-EXEC] db-standby [BACKREST-BIN] --config=[TEST_PATH]/db-standby/pgbackrest.conf --log-level-console=detail --archive-timeout=5 --stanza=db check
------------------------------------------------------------------------------------------------------------------------------------
check db - verify check command on standby (db-standby host)
> [CONTAINER-EXEC] db-standby [BACKREST-BIN] --config=[TEST_PATH]/db-standby/pgbackrest.conf --log-level-console=detail --stanza=db check
------------------------------------------------------------------------------------------------------------------------------------

4
test/expect/real-all-003.log
View File

@ -148,8 +148,8 @@ spool-path=[TEST_PATH]/db-standby/spool
archive-copy=y
start-fast=y
check db - verify check command on standby (db-standby host)
> [CONTAINER-EXEC] db-standby [BACKREST-BIN] --config=[TEST_PATH]/db-standby/pgbackrest.conf --log-level-console=detail --stanza=db check
check db - confirm standby manifest->build executed (db-standby host)
> [CONTAINER-EXEC] db-standby [BACKREST-BIN] --config=[TEST_PATH]/db-standby/pgbackrest.conf --log-level-console=detail --archive-timeout=5 --stanza=db check
------------------------------------------------------------------------------------------------------------------------------------
incr backup - update during backup (db-standby host)

4
test/expect/real-all-004.log
View File

@ -73,6 +73,10 @@ check db - fail on backup info mismatch (backup host)
> [CONTAINER-EXEC] backup [BACKREST-BIN] --config=[TEST_PATH]/backup/pgbackrest.conf --log-level-console=detail --archive-timeout=5 --stanza=db check
------------------------------------------------------------------------------------------------------------------------------------
check db - confirm master manifest->build executed (db-master host)
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --log-level-console=detail --archive-timeout=5 --stanza=db check
------------------------------------------------------------------------------------------------------------------------------------
check db - verify success after backup (db-master host)
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --log-level-console=detail --archive-timeout=5 --stanza=db check
------------------------------------------------------------------------------------------------------------------------------------

4
test/expect/real-all-005.log
View File

@ -202,6 +202,10 @@ repo-path=[TEST_PATH]/backup/repo
archive-copy=y
start-fast=y
check db - confirm standby manifest->build executed (db-standby host)
> [CONTAINER-EXEC] db-standby [BACKREST-BIN] --config=[TEST_PATH]/db-standby/pgbackrest.conf --log-level-console=detail --archive-timeout=5 --stanza=db check
------------------------------------------------------------------------------------------------------------------------------------
check db - verify check command on standby (db-standby host)
> [CONTAINER-EXEC] db-standby [BACKREST-BIN] --config=[TEST_PATH]/db-standby/pgbackrest.conf --log-level-console=detail --stanza=db check
------------------------------------------------------------------------------------------------------------------------------------

4
test/expect/real-all-006.log
View File

@ -73,6 +73,10 @@ check db - fail on backup info mismatch (backup host)
> [CONTAINER-EXEC] backup [BACKREST-BIN] --config=[TEST_PATH]/backup/pgbackrest.conf --log-level-console=detail --archive-timeout=5 --stanza=db check
------------------------------------------------------------------------------------------------------------------------------------
check db - confirm master manifest->build executed (db-master host)
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --log-level-console=detail --archive-timeout=5 --stanza=db check
------------------------------------------------------------------------------------------------------------------------------------
check db - verify success after backup (db-master host)
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --log-level-console=detail --archive-timeout=5 --stanza=db check
------------------------------------------------------------------------------------------------------------------------------------

51
test/lib/pgBackRestTest/Module/Real/RealAllTest.pm
View File

@ -262,6 +262,16 @@ sub run
# Restore the file to its original condition
$oHostBackup->infoRestore(storageRepo()->pathGet(STORAGE_REPO_BACKUP . qw{/} . FILE_BACKUP_INFO));
# Create a directory in pg_data location that is only readable by root to ensure manifest->build is called by check
my $strDir = $oHostDbMaster->dbBasePath() . '/rootreaddir';
executeTest('sudo mkdir ' . $strDir);
executeTest("sudo chown root:root ${strDir}");
executeTest("sudo chmod 400 ${strDir}");
$strComment = 'confirm master manifest->build executed';
$oHostDbMaster->check($strComment, {iTimeout => 5, iExpectedExitStatus => ERROR_FILE_OPEN});
executeTest("sudo rmdir ${strDir}");
# Providing a sufficient archive-timeout, verify that the check command runs successfully now with valid
# archive.info and backup.info files
$strComment = 'verify success after backup';
@ -483,8 +493,47 @@ sub run
$strFullBackup = $strStandbyBackup;
}
# Confirm the check command runs without error on a standby
# Create a directory in pg_data location that is only readable by root to ensure manifest->build is called by check
my $strDir = $oHostDbStandby->dbBasePath() . '/rootreaddir';
executeTest('sudo mkdir ' . $strDir);
executeTest("sudo chown root:root ${strDir}");
executeTest("sudo chmod 400 ${strDir}");
# Determine if there is an invalid db-host from the config file
my $rhConfig = iniParse(${storageTest()->get($oHostDbStandby->backrestConfig())}, {bRelaxed => true});
my $bBogusHost = false;
for (my $iRemoteIdx = 1; $iRemoteIdx <= cfgOptionIndexTotal(CFGOPT_DB_HOST); $iRemoteIdx++)
{
if (defined($rhConfig->{$self->stanza()}{$oHostDbStandby->optionIndexName(CFGOPT_DB_HOST, $iRemoteIdx)}) &&
($rhConfig->{$self->stanza()}{$oHostDbStandby->optionIndexName(CFGOPT_DB_HOST, $iRemoteIdx)} eq BOGUS))
{
$bBogusHost = true;
last;
}
}
my $strComment = 'confirm standby manifest->build executed';
# If there is an invalid host, the final error returned from check will be the inability to resolve the name which is
# a read error instead of an open error
if (!$bBogusHost)
{
$oHostDbStandby->check($strComment, {iTimeout => 5, iExpectedExitStatus => ERROR_FILE_OPEN});
}
else
{
$oHostDbStandby->check($strComment, {iTimeout => 5, iExpectedExitStatus => ERROR_FILE_READ});
}
# Remove the directory in pg_data location that is only readable by root
executeTest("sudo rmdir ${strDir}");
# Confirm the check command runs without error on a standby (when a bogus host is not configured)
if (!$bBogusHost)
{
$oHostDbStandby->check('verify check command on standby');
}
# Shutdown the stanby before creating tablespaces (this will error since paths are different)
$oHostDbStandby->clusterStop({bIgnoreLogError => true});