db/pgbackrest
1
0
Fork 0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2025-01-30 05:39:12 +02:00
Code Issues Releases Activity

Improved check command to run on a standby.

Browse Source 
Only basic checks are done because pg_switch_xlog() cannot be executed on a replica.

Contributed by Cynthia Shang.
This commit is contained in:
Cynthia Shang 2016-12-21 14:35:20 -05:00 committed by David Steele
parent 974a02dbe4
commit 56144c99c0
10 changed files with 74 additions and 33 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
doc/xml/reference.xml
View File

@ -719,7 +719,7 @@
<command id="check" name="Check">
<summary>Check the configuration.</summary>
<text>The <cmd>check</cmd> command validates that <backrest/> and the <pg-setting>archive_command</pg-setting> setting are configured correctly for archiving and backups. It detects misconfigurations, particularly in archiving, that result in incomplete backups because required WAL segments did not reach the archive. The command can be run on the database or the backup host.
<text>The <cmd>check</cmd> command validates that <backrest/> and the <pg-setting>archive_command</pg-setting> setting are configured correctly for archiving and backups. It detects misconfigurations, particularly in archiving, that result in incomplete backups because required WAL segments did not reach the archive. The command can be run on the database or the backup host. The command may also be run on the standby host, however, since <code>pg_switch_xlog()</code> cannot be performed on the standby, the command will only test the repository configuration.
Note that <code>pg_create_restore_point('pgBackRest Archive Check')</code> and <code>pg_switch_xlog()</code> are called to force <postgres/> to archive a WAL segment. Restore points are only supported in <postgres/> &gt;= 9.1 so for older versions the <cmd>check</cmd> command may fail if there has been no write activity since the last log rotation, therefore it is recommended that activity be generated by the user if there have been no writes since the last xlog switch before running the <cmd>check</cmd> command.</text>

8
doc/xml/release.xml
View File

@ -159,6 +159,14 @@
<p>Improved <cmd>stanza-create</cmd> command so that it can repair broken repositories in most cases and is robust enough to be made mandatory.</p>
</release-item>
<release-item>
<release-item-contributor-list>
<release-item-contributor id="shang.cynthia"/>
</release-item-contributor-list>
<p>Improved <cmd>check</cmd> command to run on a standby, though only basic checks are done because <code>pg_switch_xlog()</code> cannot be executed on a replica.</p>
</release-item>
</release-feature-list>
</release-core-list>
</release>

28
doc/xml/user-guide.xml
View File

@ -427,7 +427,7 @@
<execute output="y">
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} {[dash]}-log-level-console=info stanza-create</exe-cmd>
<exe-highlight>successfully created</exe-highlight>
<exe-highlight>completed successfully</exe-highlight>
</execute>
</execute-list>
</section>
@ -1617,12 +1617,13 @@
<p>Replication allows multiple copies of a <postgres/> cluster (called standbys) to be created from a single master. The standbys are useful for balancing reads and to provide redundancy in case the master host fails.</p>
<!-- SECTION => REPLICATION - HOT-STANDBY -->
<section id="hot-standby">
<title>Hot Standby</title>
<p>A hot standby performs replication using the WAL archive and allows read-only queries.</p>
<p>A new host named <host>db-standby</host> will be created to run the standby. Follow the instructions in <link section="/installation">Installation</link> to install <backrest/>, <link section="/quickstart/setup-demo-cluster">Setup Demo Cluster</link> to setup the demo cluster, and <link section="/quickstart/create-repository">Create the Repository</link> to create the <backrest/> repository on the <host>db-standby</host> host.</p>
<p>A new host named <host>db-standby</host> will be created to run the standby. Follow the instructions in <link section="/installation">Installation</link> to install <backrest/> and <link section="/quickstart/setup-demo-cluster">Setup Demo Cluster</link> to setup the demo cluster.</p>
<host-add name="{[host-db-standby]}" user="{[host-db-standby-user]}" image="{[host-db-standby-image]}" os="{[host-os]}" mount="{[host-db-standby-mount]}">
<!-- Install packages -->
@ -1659,17 +1660,6 @@
<exe-cmd>chown postgres:postgres /var/log/pgbackrest</exe-cmd>
</execute>
<!-- Create repository -->
<execute user="root">
<exe-cmd>mkdir {[backrest-repo-path]}</exe-cmd>
</execute>
<execute user="root">
<exe-cmd>chmod 750 {[backrest-repo-path]}</exe-cmd>
</execute>
<execute user="root">
<exe-cmd>chown postgres:postgres {[backrest-repo-path]}</exe-cmd>
</execute>
<!-- Start the cluster -->
<execute user="root" output="y" filter="n">
<exe-cmd>{[db-cluster-create]}</exe-cmd>
@ -1803,8 +1793,20 @@
<exe-highlight>{[test-table-data]}</exe-highlight>
</execute>
</execute-list>
<p>Check the standby configuration for access to the repository.</p>
<execute-list host="{[host-db-standby]}">
<title>Check the configuration</title>
<execute output="y">
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} {[dash]}-log-level-console=info check</exe-cmd>
<exe-highlight>all other checks passed</exe-highlight>
</execute>
</execute-list>
</section>
<!-- SECTION => REPLICATION - STREAMING -->
<section id="streaming" depend="hot-standby">
<title>Streaming Replication</title>

22
lib/pgBackRest/Archive.pm
View File

@ -1110,9 +1110,6 @@ sub check
# Validate the database configuration
$oDb->configValidate();
# Force archiving
my $strWalSegment = $oDb->xlogSwitch();
# Get the timeout and error message to display - if it is 0 we are testing
my $iArchiveTimeout = optionGet(OPTION_ARCHIVE_TIMEOUT);
@ -1125,6 +1122,7 @@ sub check
my $strArchiveId = undef;
my $strArchiveFile = undef;
my $strWalSegment = undef;
# Turn off console logging to control when to display the error
logLevelSet(undef, OFF);
@ -1167,9 +1165,11 @@ sub check
};
}
# If able to get the archive id then check the archived WAL file with the time specified
if ($iResult == 0)
# If able to get the archive id then force archiving and check the arrival of the archived WAL file with the time specified
if ($iResult == 0 && !$oDb->isStandby())
{
$strWalSegment = $oDb->xlogSwitch();
eval
{
$strArchiveFile = $self->walFileName($oFile, $strArchiveId, $strWalSegment, false, $iArchiveTimeout);
@ -1194,19 +1194,31 @@ sub check
# If the archiving was successful and backup.info check did not error in an unexpected way, then indicate success
# Else, log the error.
if ($iResult == 0)
{
if (!$oDb->isStandby())
{
&log(INFO,
"WAL segment ${strWalSegment} successfully stored in the archive at '" .
$oFile->pathGet(PATH_BACKUP_ARCHIVE, "$strArchiveId/${strArchiveFile}") . "'");
}
else
{
&log(INFO, "switch xlog cannot be performed on the standby, all other checks passed successfully");
}
}
else
{
&log(ERROR, $strResultMessage, $iResult);
# If a switch xlog was attempted, then alert the user to the WAL that did not reach the archive
if (defined($strWalSegment))
{
&log(WARN,
"WAL segment ${strWalSegment} did not reach the archive:" . (defined($strArchiveId) ? $strArchiveId : '') . "\n" .
"HINT: Check the archive_command to ensure that all options are correct (especialy --stanza).\n" .
"HINT: Check the PostreSQL server log for errors.");
}
}
# Return from function and log return values if any
return logDebugReturn

3
lib/pgBackRest/Config/ConfigHelpData.pm
View File

@ -983,7 +983,8 @@ my $oConfigHelpData =
"The check command validates that pgBackRest and the archive_command setting are configured correctly for " .
"archiving and backups. It detects misconfigurations, particularly in archiving, that result in incomplete " .
"backups because required WAL segments did not reach the archive. The command can be run on the database or " .
"the backup host.\n" .
"the backup host. The command may also be run on the standby host, however, since pg_switch_xlog() cannot " .
"be performed on the standby, the command will only test the repository configuration.\n" .
"\n" .
"Note that pg_create_restore_point('pgBackRest Archive Check') and pg_switch_xlog() are called to force " .
"PostgreSQL to archive a WAL segment. Restore points are only supported in PostgreSQL >= 9.1 so for older " .

4
test/expect/backup-full-005.log
View File

@ -115,6 +115,10 @@ log-level-stderr=off
log-path=[TEST_PATH]/db-standby/spool/log
repo-path=[TEST_PATH]/db-master/repo
check db - verify check command on standby (db-standby host)
> [CONTAINER-EXEC] db-standby [BACKREST-BIN] --config=[TEST_PATH]/db-standby/pgbackrest.conf --log-level-console=detail --stanza=db check
------------------------------------------------------------------------------------------------------------------------------------
incr backup - update during backup (db-master host)
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --stop-auto --no-archive-check --buffer-size=24576 --stanza=db backup --test --test-delay=1 --test-point=manifest-build=y
------------------------------------------------------------------------------------------------------------------------------------

4
test/expect/backup-full-006.log
View File

@ -115,6 +115,10 @@ repo-path=[TEST_PATH]/db-standby/repo
archive-copy=y
start-fast=y
check db - verify check command on standby (db-standby host)
> [CONTAINER-EXEC] db-standby [BACKREST-BIN] --config=[TEST_PATH]/db-standby/pgbackrest.conf --log-level-console=detail --stanza=db check
------------------------------------------------------------------------------------------------------------------------------------
incr backup - update during backup (db-standby host)
> [CONTAINER-EXEC] db-standby [BACKREST-BIN] --config=[TEST_PATH]/db-standby/pgbackrest.conf --stop-auto --no-archive-check --buffer-size=24576 --stanza=db backup --test --test-delay=1 --test-point=manifest-build=y
------------------------------------------------------------------------------------------------------------------------------------

4
test/expect/backup-full-011.log
View File

@ -163,6 +163,10 @@ repo-path=[TEST_PATH]/backup/repo
archive-copy=y
start-fast=y
check db - verify check command on standby (db-standby host)
> [CONTAINER-EXEC] db-standby [BACKREST-BIN] --config=[TEST_PATH]/db-standby/pgbackrest.conf --log-level-console=detail --stanza=db check
------------------------------------------------------------------------------------------------------------------------------------
incr backup - update during backup (backup host)
> [CONTAINER-EXEC] backup [BACKREST-BIN] --config=[TEST_PATH]/backup/pgbackrest.conf --stop-auto --no-archive-check --buffer-size=24576 --stanza=db backup --test --test-delay=1 --test-point=manifest-build=y
------------------------------------------------------------------------------------------------------------------------------------

3
test/lib/pgBackRestTest/Backup/BackupTest.pm
View File

@ -2349,6 +2349,9 @@ sub backupTestRun
{
$strFullBackup = $strStandbyBackup;
}
# Confirm the check command runs without error on a standby
$oHostDbStandby->check('verify check command on standby');
}
# Execute stop and make sure the backup fails

7
test/lib/pgBackRestTest/Backup/Common/HostBackupTest.pm
View File

@ -783,15 +783,18 @@ sub stanzaCreate
bLogOutput => $self->synthetic()});
# If the info file was created, then add it to the expect log
if ($self->synthetic() && fileExists($self->repoPath() . '/backup/' . $self->stanza() . '/backup.info'))
if (defined($self->{oLogTest}) && $self->synthetic())
{
if (fileExists($self->repoPath() . '/backup/' . $self->stanza() . '/backup.info'))
{
$self->{oLogTest}->supplementalAdd($self->repoPath() . '/backup/' . $self->stanza() . '/backup.info');
}
if ($self->synthetic() && fileExists($self->repoPath() . '/archive/' . $self->stanza() . '/archive.info'))
if (fileExists($self->repoPath() . '/archive/' . $self->stanza() . '/archive.info'))
{
$self->{oLogTest}->supplementalAdd($self->repoPath() . '/archive/' . $self->stanza() . '/archive.info');
}
}
# Return from function and log return values if any
return logDebugReturn($strOperation);