db/pgbackrest
1
0
Fork 0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2024-12-14 10:13:05 +02:00
Code Issues Releases Activity
0949b4d35f
pgbackrest/doc/xml/user-guide.xml

3703 lines
205 KiB
XML
Raw Normal View History

Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE doc SYSTEM "doc.dtd">
Update PostgreSQL versions used to build user guides. Also add version ranges to indicate that a user guide is accurate for a range of PostgreSQL versions even if it was built for a specific version.
2020-07-16 18:54:52 +02:00
<doc title="{[project]} User Guide" subtitle="{[user-guide-subtitle]} / {[postgres]} {[pg-version-min]}-{[pg-version-max]}" cmd-line-len="85">
Added SEO and sharing features to the website. Descriptions wordsmithed (for better or worse) by committer.
2016-05-16 23:01:48 +02:00
<description>The {[project]} User Guide demonstrates how to quickly and easily setup {[project]} for your {[postgres]} database. Step-by-step instructions lead the user through all the important features of the fastest, most reliable {[postgres]} backup and restore solution.</description>
Replace keywords with more flexible if statements. Keywords were extremely limited and prevented us from generating multi-version documentation and other improvements. Replace keywords with an if statement that can evaluate a Perl expression with variable replacement. Since keywords were used to generate cache keys, add a --key-var parameter to identify which variables should make up the key.
2018-12-01 19:40:01 +02:00
<!-- Variables used by the rest of the script -->
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
<variable-list>
Replace keywords with more flexible if statements. Keywords were extremely limited and prevented us from generating multi-version documentation and other improvements. Replace keywords with an if statement that can evaluate a Perl expression with variable replacement. Since keywords were used to generate cache keys, add a --key-var parameter to identify which variables should make up the key.
2018-12-01 19:40:01 +02:00
<!-- Defined OS types -->
<variable key="os-debian">debian</variable>
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<variable key="os-rhel">rhel</variable>
Added execution cache for document generation. Added an execution cache so that documentation can be generated without setting up the full container environment. This is useful for packaging, keeps the documentation consistent for a release, and speeds up generation when no changes are made in the execution list.
2016-06-02 15:32:56 +02:00
Add user guides for CentOS/RHEL 6/7. It would be better if the documentation could be generated on multiple operating systems all in one go, but the doc system currently does not allow vars to be changed once they are set. The solution is to run the docs for each required OS and stitch the documentation together. It's not pretty but it works and the automation in release.pl should at least make it easy to use.
2019-05-16 14:32:02 +02:00
<!-- OS titles -->
<variable key="os-debian-title">Debian &amp; Ubuntu</variable>
Use Rocky Linux for documentation builds instead of CentOS. Since CentOS 8 will be EOL at the end of the year it makes sense to do this now. The centos:8 image is still used in documentation.xml because changes there require manual testing, which will need to be done at a later date. The changes are not user-facing, however, and can be done at any time. Also update CentOS references to RHEL since that is what we are emulating for testing purposes.
2021-10-28 21:15:49 +02:00
<variable key="os-rhel-title">RHEL 7-8</variable>
Add user guides for CentOS/RHEL 6/7. It would be better if the documentation could be generated on multiple operating systems all in one go, but the doc system currently does not allow vars to be changed once they are set. The solution is to run the docs for each required OS and stitch the documentation together. It's not pretty but it works and the automation in release.pl should at least make it easy to use.
2019-05-16 14:32:02 +02:00
<!-- Base PostgreSQL versions -->
Update PostgreSQL versions used to build user guides. Also add version ranges to indicate that a user guide is accurate for a range of PostgreSQL versions even if it was built for a specific version.
2020-07-16 18:54:52 +02:00
<variable key="os-debian-pg-version">12</variable>
<variable key="os-debian-pg-version-min">12</variable>
<variable key="os-debian-pg-version-max">13</variable>
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<variable key="os-rhel-pg-version">10</variable>
<variable key="os-rhel-pg-version-min">9.6</variable>
<variable key="os-rhel-pg-version-max">11</variable>
Add user guides for CentOS/RHEL 6/7. It would be better if the documentation could be generated on multiple operating systems all in one go, but the doc system currently does not allow vars to be changed once they are set. The solution is to run the docs for each required OS and stitch the documentation together. It's not pretty but it works and the automation in release.pl should at least make it easy to use.
2019-05-16 14:32:02 +02:00
Fix typo.
2020-04-10 18:56:28 +02:00
<!-- User-defined package to use in documentation (use "apt" to install the current PGDG apt package) -->
Documentation may be built with user-specified packages. By default the documentation builds pgBackRest from source, but the documentation is also a good way to smoke-test packages. Allow a package file to be specified by passing --var=package=/path/to/package.ext. This works for Debian and CentOS 6 builds.
2018-12-04 20:17:55 +02:00
<variable key="package">none</variable>
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<!-- Defines the general OS type that will be used to generate commands. Also supported: rhel -->
Replace keywords with more flexible if statements. Keywords were extremely limited and prevented us from generating multi-version documentation and other improvements. Replace keywords with an if statement that can evaluate a Perl expression with variable replacement. Since keywords were used to generate cache keys, add a --key-var parameter to identify which variables should make up the key.
2018-12-01 19:40:01 +02:00
<variable key="os-type">debian</variable>
Allow documentation to be built without encryption. This is mainly useful for testing, where an encrypted repo is a real nuisance.
2019-09-25 22:38:13 +02:00
<!-- Will encryption be used in the documentation? It can be useful for testing to omit encryption. -->
<variable key="encrypt">y</variable>
Add optional stress test section to the user guide. This is intended to provide pre-release stress-testing. Include container memory limits to help check for memory leaks. Also add parallelism to make for faster builds.
2021-08-12 13:49:59 +02:00
<!-- Will stress-testing be performed? -->
<variable key="stress">n</variable>
<variable key="stress-scale-table">1</variable> <!-- Tables to create * 1000 -->
<variable key="stress-scale-data">1</variable> <!-- Value passed to pgbench -s -->
Create common if expressions for testing os-type. These expressions simplify os-type testing. This will be especially true as more OS types are added.
2018-12-05 00:33:56 +02:00
<!-- Common if expressions for testing os-type -->
<variable key="os-type-is-debian">'{[os-type]}' eq '{[os-debian]}'</variable>
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<variable key="os-type-is-rhel">'{[os-type]}' eq '{[os-rhel]}'</variable>
Create common if expressions for testing os-type. These expressions simplify os-type testing. This will be especially true as more OS types are added.
2018-12-05 00:33:56 +02:00
Documentation may be built with user-specified packages. By default the documentation builds pgBackRest from source, but the documentation is also a good way to smoke-test packages. Allow a package file to be specified by passing --var=package=/path/to/package.ext. This works for Debian and CentOS 6 builds.
2018-12-04 20:17:55 +02:00
<!-- Defines the container image that will be used to build the host -->
Partial migration of config code generation to C. Parse enough of config.yaml to auto-generate config.auto.h and config.auto.c. This commit implements most of the infrastructure needed to migrate the rest of the build code to C, but each set of auto-generated files will present its own challenges. The build is now dependent on libyaml. At this point there is no need for a hard requirement, but that will come soon so it seems better to add the dependency now.
2021-07-19 01:02:01 +02:00
<variable key="os-image" if="{[os-type-is-debian]}">ubuntu:18.04</variable>
Use Rocky Linux for documentation builds instead of CentOS. Since CentOS 8 will be EOL at the end of the year it makes sense to do this now. The centos:8 image is still used in documentation.xml because changes there require manual testing, which will need to be done at a later date. The changes are not user-facing, however, and can be done at any time. Also update CentOS references to RHEL since that is what we are emulating for testing purposes.
2021-10-28 21:15:49 +02:00
<variable key="os-image" if="{[os-type-is-rhel]}">rockylinux/rockylinux:8</variable>
Documentation may be built with user-specified packages. By default the documentation builds pgBackRest from source, but the documentation is also a good way to smoke-test packages. Allow a package file to be specified by passing --var=package=/path/to/package.ext. This works for Debian and CentOS 6 builds.
2018-12-04 20:17:55 +02:00
Add user guides for CentOS/RHEL 6/7. It would be better if the documentation could be generated on multiple operating systems all in one go, but the doc system currently does not allow vars to be changed once they are set. The solution is to run the docs for each required OS and stitch the documentation together. It's not pretty but it works and the automation in release.pl should at least make it easy to use.
2019-05-16 14:32:02 +02:00
<variable key="user-guide-subtitle" if="{[os-type-is-debian]}">{[os-debian-title]}</variable>
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<variable key="user-guide-subtitle" if="{[os-type-is-rhel]}">{[os-rhel-title]}</variable>
Replace keywords with more flexible if statements. Keywords were extremely limited and prevented us from generating multi-version documentation and other improvements. Replace keywords with an if statement that can evaluate a Perl expression with variable replacement. Since keywords were used to generate cache keys, add a --key-var parameter to identify which variables should make up the key.
2018-12-01 19:40:01 +02:00
Create common if expressions for testing os-type. These expressions simplify os-type testing. This will be especially true as more OS types are added.
2018-12-05 00:33:56 +02:00
<variable key="user-guide-os" if="{[os-type-is-debian]}">Debian/Ubuntu</variable>
Use Rocky Linux for documentation builds instead of CentOS. Since CentOS 8 will be EOL at the end of the year it makes sense to do this now. The centos:8 image is still used in documentation.xml because changes there require manual testing, which will need to be done at a later date. The changes are not user-facing, however, and can be done at any time. Also update CentOS references to RHEL since that is what we are emulating for testing purposes.
2021-10-28 21:15:49 +02:00
<variable key="user-guide-os" if="{[os-type-is-rhel]}">RHEL 7-8</variable>
Added execution cache for document generation. Added an execution cache so that documentation can be generated without setting up the full container environment. This is useful for packaging, keeps the documentation consistent for a release, and speeds up generation when no changes are made in the execution list.
2016-06-02 15:32:56 +02:00
Fix hard-coded repository path. This prevented packages from being passed to the documentation unless they were in the /backrest directory on the host. Also make the local path /pgbackrest instead of the deprecated /backrest. Reported by Heath Lord.
2019-01-30 13:39:22 +02:00
<variable key="pgbackrest-repo-path">/pgbackrest</variable>
Use minio as local S3 emulator in documentation. The documentation was relying on a ScalityS3 container built for testing which wasn't very transparent. Instead, use the stock minio container and configure it in the documentation. Also, install certificates and CA so that TLS verification can be enabled.
2019-05-27 13:37:20 +02:00
<!-- Path where CA certificates are installed -->
<variable key="ca-cert-path" if="{[os-type-is-debian]}">/usr/local/share/ca-certificates</variable>
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<variable key="ca-cert-path" if="{[os-type-is-rhel]}">/etc/pki/ca-trust/source/anchors</variable>
Use minio as local S3 emulator in documentation. The documentation was relying on a ScalityS3 container built for testing which wasn't very transparent. Instead, use the stock minio container and configure it in the documentation. Also, install certificates and CA so that TLS verification can be enabled.
2019-05-27 13:37:20 +02:00
<!-- Path where fake certificates are located -->
<variable key="fake-cert-path-relative">resource/fake-cert</variable>
Add host-repo-path variable internal replacement. This variable needs to be replaced right before being used without being added to the cache since the host repo path will vary from system to system. This is frankly a bit of a hack to get the documentation to build in the Debian packages for the upcoming release. We'll need to come up with something more flexible going forward.
2019-06-25 13:58:38 +02:00
<variable key="fake-cert-path">{[host-repo-path]}/doc/{[fake-cert-path-relative]}</variable>
Use minio as local S3 emulator in documentation. The documentation was relying on a ScalityS3 container built for testing which wasn't very transparent. Instead, use the stock minio container and configure it in the documentation. Also, install certificates and CA so that TLS verification can be enabled.
2019-05-27 13:37:20 +02:00
Add variables for build paths in the user guide. This eliminates repetition of the build path so it can be changed more easily. Also create the build path explicitly rather than suggest that the user do it.
2021-08-18 16:52:35 +02:00
<!-- Path where builds are performed -->
<variable key="build-path">/build</variable>
<variable key="build-br-path">{[build-path]}/pgbackrest-release-{[version]}</variable>
Update PostgreSQL versions used to build user guides. Also add version ranges to indicate that a user guide is accurate for a range of PostgreSQL versions even if it was built for a specific version.
2020-07-16 18:54:52 +02:00
<!-- PostreSQL versions to run documentation for and min/max versions represented -->
Add user guides for CentOS/RHEL 6/7. It would be better if the documentation could be generated on multiple operating systems all in one go, but the doc system currently does not allow vars to be changed once they are set. The solution is to run the docs for each required OS and stitch the documentation together. It's not pretty but it works and the automation in release.pl should at least make it easy to use.
2019-05-16 14:32:02 +02:00
<variable key="pg-version" if="{[os-type-is-debian]}">{[os-debian-pg-version]}</variable>
Update PostgreSQL versions used to build user guides. Also add version ranges to indicate that a user guide is accurate for a range of PostgreSQL versions even if it was built for a specific version.
2020-07-16 18:54:52 +02:00
<variable key="pg-version-min" if="{[os-type-is-debian]}">{[os-debian-pg-version-min]}</variable>
<variable key="pg-version-max" if="{[os-type-is-debian]}">{[os-debian-pg-version-max]}</variable>
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<variable key="pg-version" if="{[os-type-is-rhel]}">{[os-rhel-pg-version]}</variable>
<variable key="pg-version-min" if="{[os-type-is-rhel]}">{[os-rhel-pg-version-min]}</variable>
<variable key="pg-version-max" if="{[os-type-is-rhel]}">{[os-rhel-pg-version-max]}</variable>
Add RHEL/CentOS 8 documentation. Update RHEL/CentOS 7 to cover the versions that were previously covered by RHEL/CentOS 6. Since RHEL/CentOS 7/8 work the same update the documentation logic and labels to reflect this compatibility.
2020-12-04 17:59:57 +02:00
Documentation builds on PostgreSQL 9.4-10. More than likely 9.2-11 will work as well, but this has not been tested. However, 11 needs work on the group permissions introduced in that version.
2019-03-01 09:00:51 +02:00
<variable key="pg-version-nodot" eval="y">my $version = '{[pg-version]}'; $version =~ s/\.//g; return $version;</variable>
Use wal_level=replica in the documentation for PostgreSQL >= 9.6. The documentation was using wal_level=hot_standby which is a deprecated setting. Also remove the reference to wal_level=archive since it is no longer supported and is not recommended for older versions. Suggested by Patrick McLaughlin.
2019-06-05 13:27:24 +02:00
<!-- WAL level should be the minimum required for replication -->
<variable key="wal-level" if="{[pg-version]} &lt; 9.6">hot_standby</variable>
<variable key="wal-level" if="{[pg-version]} &gt;= 9.6">replica</variable>
Update PostgreSQL versions used to build user guides. Also add version ranges to indicate that a user guide is accurate for a range of PostgreSQL versions even if it was built for a specific version.
2020-07-16 18:54:52 +02:00
<variable key="pg-version-upgrade" if="{[os-type-is-debian]}">13</variable>
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<variable key="pg-version-upgrade" if="{[os-type-is-rhel]}">11</variable>
Documentation builds on PostgreSQL 9.4-10. More than likely 9.2-11 will work as well, but this has not been tested. However, 11 needs work on the group permissions introduced in that version.
2019-03-01 09:00:51 +02:00
<variable key="pg-version-upgrade-nodot" eval="y">my $version = '{[pg-version-upgrade]}'; $version =~ s/\.//g; return $version;</variable>
PostgreSQL version is now a variable to allow multi-version documentation.
2016-06-04 18:24:31 +02:00
Create common if expressions for testing os-type. These expressions simplify os-type testing. This will be especially true as more OS types are added.
2018-12-05 00:33:56 +02:00
<variable key="pg-bin-path" if="{[os-type-is-debian]}">/usr/lib/postgresql/{[pg-version]}/bin</variable>
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<variable key="pg-bin-path" if="{[os-type-is-rhel]}">/usr/pgsql-{[pg-version]}/bin</variable>
All clusters in the documentation are initialized with checksums.
2017-12-14 22:44:06 +02:00
Create common if expressions for testing os-type. These expressions simplify os-type testing. This will be especially true as more OS types are added.
2018-12-05 00:33:56 +02:00
<variable key="pg-bin-upgrade-path" if="{[os-type-is-debian]}">/usr/lib/postgresql/{[pg-version-upgrade]}/bin</variable>
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<variable key="pg-bin-upgrade-path" if="{[os-type-is-rhel]}">/usr/pgsql-{[pg-version-upgrade]}/bin</variable>
All clusters in the documentation are initialized with checksums.
2017-12-14 22:44:06 +02:00
Create common if expressions for testing os-type. These expressions simplify os-type testing. This will be especially true as more OS types are added.
2018-12-05 00:33:56 +02:00
<variable key="pg-home-path" if="{[os-type-is-debian]}">/var/lib/postgresql</variable>
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<variable key="pg-home-path" if="{[os-type-is-rhel]}">/var/lib/pgsql</variable>
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
Preliminary documentation for PostgreSQL 11 unprivileged user backup.
2018-06-15 01:05:35 +02:00
<variable key="pg-group">postgres</variable>
New simpler configuration and consistent project/exe/path naming. * The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required. * Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case. * The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally. * The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest. * Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory. * Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory. * Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 15:30:54 +02:00
<variable key="backrest-repo-path">/var/lib/pgbackrest</variable>
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
<variable key="backrest-repo-cipher-type">aes-256-cbc</variable>
<variable key="backrest-repo-cipher-pass">zWaf6XtpjIVZC5444yXB+cgFDFl7MxGlgkZSaoPvTGirhPygu4jOKOXf9LO4vjfO</variable>
pgBackRest is now pure C. Remove embedded Perl from the distributed binary. This includes code, configure, Makefile, and packages. The distributed binary is now pure C. Remove storagePathEnforceSet() from the C Storage object which allowed Perl to write outside of the storage base directory. Update mock/all and real/all integration tests to use storageLocal() where they were violating this rule. Remove "c" option that allowed the remote to tell if it was being called from C or Perl. Code to convert options to JSON for passing to Perl (perl/config.c) has been moved to LibC since it is still required for Perl integration tests. Update build and installation instructions in the user guide. Remove all Perl unit tests. Remove obsolete Perl code. In particular this included all the Perl protocol code which required modifications to the Perl storage, manifest, and db objects that are still required for integration testing but only run locally. Any remaining Perl code is required for testing, documentation, or code generation. Rename perlReq to binReq in define.yaml to indicate that the binary is required for a test. This had been the actual meaning for quite some time but the key was never renamed.
2019-12-14 00:55:41 +02:00
<variable key="br-bin">/usr/bin/pgbackrest</variable>
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<variable key="br-user">pgbackrest</variable>
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
<variable key="br-group">{[br-user]}</variable>
<variable key="br-home-path">/home/{[br-user]}</variable>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<variable key="postgres-cluster-demo">demo</variable>
Add support for additional pgBackRest configuration files. Configuration files are loaded from the directory specified by the --config-include-path option. Add --config-path option for overriding the default base path of the --config and --config-include-path option. Contributed by Cynthia Shang.
2018-04-19 17:09:39 +02:00
<variable key="backrest-config-path">/etc/{[project-exe]}</variable>
<variable key="backrest-config-include-path">{[backrest-config-path]}/conf.d</variable>
Documentation may be built with user-specified packages. By default the documentation builds pgBackRest from source, but the documentation is also a good way to smoke-test packages. Allow a package file to be specified by passing --var=package=/path/to/package.ext. This works for Debian and CentOS 6 builds.
2018-12-04 20:17:55 +02:00
<variable if="'{[package]}' eq 'none'" key="backrest-config-demo">{[backrest-config-path]}/{[project-exe]}.conf</variable>
<variable if="'{[package]}' ne 'none'" key="backrest-config-demo">/etc/{[project-exe]}.conf</variable>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Create common if expressions for testing os-type. These expressions simplify os-type testing. This will be especially true as more OS types are added.
2018-12-05 00:33:56 +02:00
<variable key="pg-path-default" if="{[os-type-is-debian]}">/var/lib/postgresql/[version]/[cluster]</variable>
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<variable key="pg-path-default" if="{[os-type-is-rhel]}">/var/lib/pgsql/[version]/data</variable>
Added execution cache for document generation. Added an execution cache so that documentation can be generated without setting up the full container environment. This is useful for packaging, keeps the documentation consistent for a release, and speeds up generation when no changes are made in the execution list.
2016-06-02 15:32:56 +02:00
Create common if expressions for testing os-type. These expressions simplify os-type testing. This will be especially true as more OS types are added.
2018-12-05 00:33:56 +02:00
<variable key="pg-path" if="{[os-type-is-debian]}">/var/lib/postgresql/{[pg-version]}/{[postgres-cluster-demo]}</variable>
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<variable key="pg-path" if="{[os-type-is-rhel]}">/var/lib/pgsql/{[pg-version]}/data</variable>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Create common if expressions for testing os-type. These expressions simplify os-type testing. This will be especially true as more OS types are added.
2018-12-05 00:33:56 +02:00
<variable key="pg-path-upgrade" if="{[os-type-is-debian]}">/var/lib/postgresql/{[pg-version-upgrade]}/{[postgres-cluster-demo]}</variable>
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<variable key="pg-path-upgrade" if="{[os-type-is-rhel]}">/var/lib/pgsql/{[pg-version-upgrade]}/data</variable>
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
Added documentation for asynchronous archiving. Also removed some obsolete sections. Contributed by Cynthia Shang.
2016-07-01 01:38:28 +02:00
<variable key="spool-path">/var/spool/pgbackrest</variable>
Create common if expressions for testing os-type. These expressions simplify os-type testing. This will be especially true as more OS types are added.
2018-12-05 00:33:56 +02:00
<variable key="postgres-config-demo" if="{[os-type-is-debian]}">/etc/postgresql/{[pg-version]}/{[postgres-cluster-demo]}/postgresql.conf</variable>
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<variable key="postgres-config-demo" if="{[os-type-is-rhel]}">{[pg-path]}/postgresql.conf</variable>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Create common if expressions for testing os-type. These expressions simplify os-type testing. This will be especially true as more OS types are added.
2018-12-05 00:33:56 +02:00
<variable key="postgres-config-demo-upgrade" if="{[os-type-is-debian]}">/etc/postgresql/{[pg-version-upgrade]}/{[postgres-cluster-demo]}/postgresql.conf</variable>
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<variable key="postgres-config-demo-upgrade" if="{[os-type-is-rhel]}">{[pg-path-upgrade]}/postgresql.conf</variable>
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
Create common if expressions for testing os-type. These expressions simplify os-type testing. This will be especially true as more OS types are added.
2018-12-05 00:33:56 +02:00
<variable key="postgres-hba-demo" if="{[os-type-is-debian]}">/etc/postgresql/{[pg-version]}/{[postgres-cluster-demo]}/pg_hba.conf</variable>
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<variable key="postgres-hba-demo" if="{[os-type-is-rhel]}">{[pg-path]}/pg_hba.conf</variable>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Create common if expressions for testing os-type. These expressions simplify os-type testing. This will be especially true as more OS types are added.
2018-12-05 00:33:56 +02:00
<variable key="postgres-hba-demo-upgrade" if="{[os-type-is-debian]}">/etc/postgresql/{[pg-version-upgrade]}/{[postgres-cluster-demo]}/pg_hba.conf</variable>
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<variable key="postgres-hba-demo-upgrade" if="{[os-type-is-rhel]}">{[pg-path-upgrade]}/pg_hba.conf</variable>
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
<variable key="postgres-pgpass">{[pg-home-path]}/.pgpass</variable>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Create common if expressions for testing os-type. These expressions simplify os-type testing. This will be especially true as more OS types are added.
2018-12-05 00:33:56 +02:00
<variable key="postgres-log-demo" if="{[os-type-is-debian]}">/var/log/postgresql/postgresql-{[pg-version]}-{[postgres-cluster-demo]}.log</variable>
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<variable key="postgres-log-demo" if="{[os-type-is-rhel]} &amp;&amp; {[pg-version]} &lt; 10">{[pg-path]}/pg_log/postgresql.log</variable>
<variable key="postgres-log-demo" if="{[os-type-is-rhel]} &amp;&amp; {[pg-version]} &gt;= 10">{[pg-path]}/log/postgresql.log</variable>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<variable key="postgres-log-pgstartup-demo" if="{[os-type-is-rhel]}">/var/lib/pgsql/{[pg-version]}/pgstartup.log</variable>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Update recovery.conf in user guide for PostgreSQL >= 12. The postgresql.auto.conf file was being used instead of recovery.conf, but there were still instances in the text that used recovery.conf. Update to postgresql.auto.conf for PostgreSQL >= 10 and change wording where needed.
2020-07-17 19:27:14 +02:00
<variable key="pg-recovery-file-demo" if="{[pg-version]} &lt; 12">recovery.conf</variable>
<variable key="pg-recovery-file-demo" if="{[pg-version]} &gt;= 12">postgresql.auto.conf</variable>
<variable key="pg-recovery-path-demo">{[pg-path]}/{[pg-recovery-file-demo]}</variable>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Pick pg_switch_wal()/pg_switch_xlog() based on PostgreSQL version. One step in making the documentation build for all versions that we support.
2018-12-07 19:19:28 +02:00
<!-- Select correct WAL switch function based on the version of PostgreSQL -->
<variable key="pg-switch-wal" if="{[pg-version]} &lt; 10">pg_switch_xlog</variable>
<variable key="pg-switch-wal" if="{[pg-version]} &gt;= 10">pg_switch_wal</variable>
Preliminary documentation for PostgreSQL 11 unprivileged user backup.
2018-06-15 01:05:35 +02:00
Azure support for repository storage. Azure and Azure-compatible object stores can now be used for repository storage. Currently only shared key authentication is supported but SAS will be added soon.
2020-07-02 22:24:34 +02:00
<!-- Azure Settings -->
Adjustments for new breaking change in Azurite. Azurite released another breaking change (see fbd018cd, 096829b3, c38d6926, and Azurite issue 1039) so make adjustments as needed to documentation and tests. Also remove some dead code that hid the repo-storage-host option and was made obsolete by all these changes.
2021-10-25 21:42:28 +02:00
<variable key="azure-image">mcr.microsoft.com/azure-storage/azurite</variable> <!-- Azurite docker image -->
Azure support for repository storage. Azure and Azure-compatible object stores can now be used for repository storage. Currently only shared key authentication is supported but SAS will be added soon.
2020-07-02 22:24:34 +02:00
<variable key="azure-all">n</variable> <!-- Build all the documentation with Azure? -->
<variable key="azure-local">y</variable>
<variable key="azure-account">pgbackrest</variable>
<variable key="azure-container">demo-container</variable>
<variable key="azure-repo">demo-repo</variable>
Add shared access signature (SAS) authorization for Azure. A shared access signature (SAS) provides granular, delegated access to resources in a storage account. This is often preferable to using a shared key which provides more access and is a greater security risk if compromised.
2020-07-09 20:46:48 +02:00
<variable key="azure-key-type">shared</variable>
Azure support for repository storage. Azure and Azure-compatible object stores can now be used for repository storage. Currently only shared key authentication is supported but SAS will be added soon.
2020-07-02 22:24:34 +02:00
<variable key="azure-key">YXpLZXk=</variable>
GCS support for repository storage. GCS and GCS-compatible object stores can now be used for repository storage.
2021-03-05 19:13:51 +02:00
<!-- GCS Settings -->
<variable key="gcs-all">n</variable> <!-- Build all the documentation with GCS? -->
<variable key="gcs-bucket">demo-bucket</variable>
<variable key="gcs-repo">demo-repo</variable>
<variable key="gcs-key-type">service</variable>
<variable key="gcs-key">/etc/pgbackrest/gcs-key.json</variable>
Use minio as local S3 emulator in documentation. The documentation was relying on a ScalityS3 container built for testing which wasn't very transparent. Instead, use the stock minio container and configure it in the documentation. Also, install certificates and CA so that TLS verification can be enabled.
2019-05-27 13:37:20 +02:00
<!-- S3 Settings -->
Allow S3/Azure Docker images to be specified in user guide. It is not uncommon for the S3/Azure emulators we use to introduce breaking changes without warning. If that happens the documentation can still be built by specifying a working version of the image. In general, it is better to let the version float so we know when things break. Azurite has yet another breaking change coming up (see 096829b3, c38d6926, and Azurite issue 1039) so set azure-image at the current version until the breaking change has been released.
2021-10-20 14:22:37 +02:00
<variable key="s3-image">minio/minio</variable> <!-- Minio docker image -->
Build all docs with S3 using --var=s3-all=y Force repo-type=s3 for all tests. This is not currently the default for any OS builds.
2019-05-29 14:38:45 +02:00
<variable key="s3-all">n</variable> <!-- Build all the documentation with S3? -->
Use minio as local S3 emulator in documentation. The documentation was relying on a ScalityS3 container built for testing which wasn't very transparent. Instead, use the stock minio container and configure it in the documentation. Also, install certificates and CA so that TLS verification can be enabled.
2019-05-27 13:37:20 +02:00
<variable key="s3-local">y</variable>
<variable key="s3-bucket">demo-bucket</variable>
<variable key="s3-repo">demo-repo</variable>
<variable key="s3-region">us-east-1</variable>
<variable key="s3-endpoint">s3.{[s3-region]}.amazonaws.com</variable>
<variable key="s3-key">accessKey1</variable>
<variable key="s3-key-secret">verySecretKey1</variable>
Multiple repository support. Up to four repositories may be configured. A potential benefit is the ability to have a local repository for fast restores and a remote repository for redundancy. Some commands, e.g. stanza-create/stanza-update, will automatically work with all configured repositories while others, e.g. stanza-delete, will require a repository to be specified using the repo option. See the command reference for details on which commands require the repository to be specified. Note that the repo option is not required when only repo1 is configured in order to maintain backward compatibility. However, the repo option is required when a single repo is configured as, e.g. repo2. This is to prevent command breakage if a new repository is added later. The archive-push command will always push WAL to the archive in all configured repositories but backups will need to be scheduled individually for each repository. In many cases this is desirable since backup types and retention will vary by repository. Likewise, restores must specify a repository. It is generally better to specify a repository for restores that has low latency/cost even if that means more recovery time. Only restore testing can determine which repository will be most efficient. For single repository configurations there should be no change in behavior.
2021-03-08 20:31:13 +02:00
<!-- Is any object store being used to build all the documentation? -->
<variable key="object-any-all">('{[azure-all]}' eq 'y' || '{[gcs-all]}' eq 'y' || '{[s3-all]}' eq 'y')</variable>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<!-- Hosts -->
Documentation may be built with user-specified packages. By default the documentation builds pgBackRest from source, but the documentation is also a good way to smoke-test packages. Allow a package file to be specified by passing --var=package=/path/to/package.ext. This works for Debian and CentOS 6 builds.
2018-12-04 20:17:55 +02:00
<variable key="host-image">pgbackrest/doc:{[os-type]}</variable>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
Fix Centos/RHEL 7 documentation builds. This was caused by a new container version that was released around December 5th. The new version explicitly denies user logons by leaving /var/run/nologin in place after boot. The solution is to enable the service that is responsible for removing this file on a successful boot.
2018-12-12 18:15:09 +02:00
<variable key="host-option">-v /sys/fs/cgroup:/sys/fs/cgroup:rw -v /tmp/$(mktemp -d):/run</variable>
Add Centos/RHEL 7 option to documentation build. Centos/RHEL 7 builds can be specified with --var=os-type=centos7. Also added to the documentation release and CI builds.
2018-12-05 14:12:28 +02:00
v2.04: Critical Bug Fix for Backup Resume IMPORTANT NOTE: This release fixes a critical bug in the backup resume feature. All resumed backups prior to this release should be considered inconsistent. A backup will be resumed after a prior backup fails, unless resume=n has been specified. A resumed backup can be identified by checking the backup log for the message "aborted backup of same type exists, will be cleaned to remove invalid files and resumed". If the message exists, do not use this backup or any backup in the same set for a restore and check the restore logs to see if a resumed backup was restored. If so, there may be inconsistent data in the cluster. Bug Fixes: * Fix critical bug in resume that resulted in inconsistent backups. A regression in v0.82 removed the timestamp comparison when deciding which files from the aborted backup to keep on resume. See note above for more details. (Reported by David Youatt, Yogesh Sharma, Stephen Frost.) * Fix error in selective restore when only one user database exists in the cluster. (Fixed by Cynthia Shang. Reported by Nj Baliyan.) * Fix non-compliant ISO-8601 timestamp format in S3 authorization headers. AWS and some gateways were tolerant of space rather than zero-padded hours while others were not. (Fixed by Andrew Schwartz.) Features: * PostgreSQL 11 Beta 2 support. Improvements: * Improve the HTTP client to set content-length to 0 when not specified by the server. S3 (and gateways) always set content-length or transfer-encoding but HTTP 1.1 does not require it and proxies (e.g. HAProxy) may not include either. (Suggested by Adam K. Sumner.) * Set search_path = 'pg_catalog' on PostgreSQL connections. (Suggested by Stephen Frost.)
2018-07-06 02:16:41 +02:00
<variable key="host-user" eval="y">use English; getpwuid($UID) eq 'root' ? 'vagrant' : getpwuid($UID) . ''</variable>
Add host-repo-path variable internal replacement. This variable needs to be replaced right before being used without being added to the cache since the host repo path will vary from system to system. This is frankly a bit of a hack to get the documentation to build in the Debian packages for the upcoming release. We'll need to come up with something more flexible going forward.
2019-06-25 13:58:38 +02:00
<variable key="host-mount">{[host-repo-path]}:{[pgbackrest-repo-path]}</variable>
Continuous integration using travis-ci.
2017-02-21 15:59:23 +02:00
<variable key="image-repo">pgbackrest/test</variable>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Azure support for repository storage. Azure and Azure-compatible object stores can now be used for repository storage. Currently only shared key authentication is supported but SAS will be added soon.
2020-07-02 22:24:34 +02:00
<variable key="host-azure-id">azure</variable>
<variable key="host-azure">azure-server</variable>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<variable key="host-s3-id">s3</variable>
<variable key="host-s3">s3-server</variable>
<variable key="host-pg1-id">pg1</variable>
<variable key="host-pg1">pg-primary</variable>
<variable key="host-pg1-user">{[host-user]}</variable>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<variable key="host-pg1-image">{[host-image]}</variable>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<variable key="host-pg1-mount">{[host-mount]}</variable>
Create a new section to describe building pgBackRest and build on a separate host.
2018-06-13 19:43:17 +02:00
<variable key="host-build-id">build</variable>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<variable key="host-build">build</variable>
Build pgBackRest as an unprivileged user. pgBackRest was being built by root in the documentation which is definitely not best practice. Instead build as the unprivileged default container user. Sudo privileges are still required to install. Suggested by Laurenz Albe.
2019-08-20 15:46:29 +02:00
<variable key="host-build-user">{[host-user]}</variable>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<variable key="host-build-image">{[host-image]}</variable>
<variable key="host-build-mount">{[host-mount]}</variable>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<variable key="host-pg2-id">pg2</variable>
<variable key="host-pg2">pg-standby</variable>
<variable key="host-pg2-user">{[host-pg1-user]}</variable>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<variable key="host-pg2-image">{[host-image]}</variable>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<variable key="host-pg2-mount">{[host-mount]}</variable>
<variable key="host-repo1-id">repo1</variable>
<variable key="host-repo1">repository</variable>
Update backup host to repository host in the documentation. Contributed by Cynthia Shang.
2018-02-19 20:17:58 +02:00
<variable key="host-repo1-user">{[host-user]}</variable>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<variable key="host-repo1-image">{[host-image]}</variable>
Update backup host to repository host in the documentation. Contributed by Cynthia Shang.
2018-02-19 20:17:58 +02:00
<variable key="host-repo1-mount">{[host-mount]}</variable>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
<!-- Commands for various operations -->
Rename ls command to repo-ls. This command only makes sense for the repository storage since other storage (e.g. pg and spool) must be located on a local Posix filesystem and can be listed using standard unix commands. Since the repo storage can be located lots of places having a common way to list it makes sense. Prefix with repo- to make the scope of this command clear. Update documentation to reflect this change.
2020-03-09 22:41:04 +02:00
<variable key="cmd-backup-last">pgbackrest repo-ls backup/demo --filter="(F|D|I)$" --sort=desc | head -1</variable>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
<!-- Data used to demonstrate backup/restore operations -->
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<variable key="test-table-data">Important Data</variable>
<!-- Database cluster commmands -->
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<variable key="pg-cluster-wait">sleep 2</variable>
Closed #163: Document Generation Issue Fixed an issue where document generation failed because some OSs are not tolerant of having multiple installed versions of PostgreSQL. A separate VM is now created for each version. Also added a sleep after database starts during document generation to ensure the database is running before the next command runs. Reported by John Harvey.
2016-01-23 01:01:21 +02:00
Create common if expressions for testing os-type. These expressions simplify os-type testing. This will be especially true as more OS types are added.
2018-12-05 00:33:56 +02:00
<variable key="pg-cluster-create" if="{[os-type-is-debian]}">pg_createcluster {[pg-version]} {[postgres-cluster-demo]}</variable>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Create common if expressions for testing os-type. These expressions simplify os-type testing. This will be especially true as more OS types are added.
2018-12-05 00:33:56 +02:00
<variable key="pg-cluster-create-upgrade" if="{[os-type-is-debian]}">pg_createcluster {[pg-version-upgrade]} {[postgres-cluster-demo]}</variable>
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
Create common if expressions for testing os-type. These expressions simplify os-type testing. This will be especially true as more OS types are added.
2018-12-05 00:33:56 +02:00
<variable key="pg-cluster-start" if="{[os-type-is-debian]}">pg_ctlcluster {[pg-version]} {[postgres-cluster-demo]} start</variable>
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<variable key="pg-cluster-start" if="{[os-type-is-rhel]}">systemctl start postgresql-{[pg-version]}.service</variable>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Create common if expressions for testing os-type. These expressions simplify os-type testing. This will be especially true as more OS types are added.
2018-12-05 00:33:56 +02:00
<variable key="pg-cluster-start-upgrade" if="{[os-type-is-debian]}">pg_ctlcluster {[pg-version-upgrade]} {[postgres-cluster-demo]} start</variable>
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<variable key="pg-cluster-start-upgrade" if="{[os-type-is-rhel]}">systemctl start postgresql-{[pg-version-upgrade]}.service</variable>
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
Create common if expressions for testing os-type. These expressions simplify os-type testing. This will be especially true as more OS types are added.
2018-12-05 00:33:56 +02:00
<variable key="pg-cluster-stop" if="{[os-type-is-debian]}">pg_ctlcluster {[pg-version]} {[postgres-cluster-demo]} stop</variable>
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<variable key="pg-cluster-stop" if="{[os-type-is-rhel]}">systemctl stop postgresql-{[pg-version]}.service</variable>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Create common if expressions for testing os-type. These expressions simplify os-type testing. This will be especially true as more OS types are added.
2018-12-05 00:33:56 +02:00
<variable key="pg-cluster-restart" if="{[os-type-is-debian]}">pg_ctlcluster {[pg-version]} {[postgres-cluster-demo]} restart</variable>
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<variable key="pg-cluster-restart" if="{[os-type-is-rhel]}">systemctl restart postgresql-{[pg-version]}.service</variable>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Create common if expressions for testing os-type. These expressions simplify os-type testing. This will be especially true as more OS types are added.
2018-12-05 00:33:56 +02:00
<variable key="pg-cluster-reload" if="{[os-type-is-debian]}">pg_ctlcluster {[pg-version]} {[postgres-cluster-demo]} reload</variable>
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<variable key="pg-cluster-reload" if="{[os-type-is-rhel]}">systemctl reload postgresql-{[pg-version]}.service</variable>
Add Centos/RHEL 7 option to documentation build. Centos/RHEL 7 builds can be specified with --var=os-type=centos7. Also added to the documentation release and CI builds.
2018-12-05 14:12:28 +02:00
<variable key="pg-cluster-check" if="{[os-type-is-debian]}">pg_lsclusters</variable>
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<variable key="pg-cluster-check" if="{[os-type-is-rhel]}">systemctl status postgresql-{[pg-version]}.service</variable>
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
Create common if expressions for testing os-type. These expressions simplify os-type testing. This will be especially true as more OS types are added.
2018-12-05 00:33:56 +02:00
<variable key="pg-cluster-check-upgrade" if="{[os-type-is-debian]}">pg_lsclusters</variable>
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<variable key="pg-cluster-check-upgrade" if="{[os-type-is-rhel]}">systemctl status postgresql-{[pg-version-upgrade]}.service</variable>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<!-- Common commands -->
<variable key="ssh-key-install">
Move most host setup to containers defined in the documentation. This includes PostgreSQL installation which had previously been included in the documentation. This way produces faster builds and there is no need for us to document PostgreSQL installation.
2018-07-23 13:59:24 +02:00
mkdir -p -m 700 /root/.ssh &amp;&amp; \
echo '-----BEGIN RSA PRIVATE KEY-----' > /root/.ssh/id_rsa &amp;&amp; \
echo 'MIICXwIBAAKBgQDR0yJsZW5d5LcqteiOtv8d+FFeFFHDPI0VTcTOdMn1iDiIP1ou' >> /root/.ssh/id_rsa &amp;&amp; \
echo 'X3Q2OyNjsBaDbsRJd+sp9IRq1LKX3zsBcgGZANwm0zduuNEPEU94ajS/uRoejIqY' >> /root/.ssh/id_rsa &amp;&amp; \
echo '/XkKOpnEF6ZbQ2S7TaE4sWeGLvba7kUFs0QTOO+N+nV2dMbdqZf6C8lazwIDAQAB' >> /root/.ssh/id_rsa &amp;&amp; \
echo 'AoGBAJXa6xzrnFVmwgK5BKzYuX/YF5TPgk2j80ch0ct50buQXH/Cb0/rUH5i4jWS' >> /root/.ssh/id_rsa &amp;&amp; \
echo 'T6Hy/DFUehnuzpvV6O9auTOhDs3BhEKFRuRLn1nBwTtZny5Hh+cw7azUCEHFCJlz' >> /root/.ssh/id_rsa &amp;&amp; \
echo 'makCrVbgawtno6oU/pFgQm1FcxD0f+Me5ruNcLHqUZsPQwkRAkEA+8pG+ckOlz6R' >> /root/.ssh/id_rsa &amp;&amp; \
echo 'AJLIHedmfcrEY9T7sfdo83bzMOz8H5soUUP4aOTLJYCla1LO7JdDnXMGo0KxaHBP' >> /root/.ssh/id_rsa &amp;&amp; \
echo 'l8j5zDmVewJBANVVPDJr1w37m0FBi37QgUOAijVfLXgyPMxYp2uc9ddjncif0063' >> /root/.ssh/id_rsa &amp;&amp; \
echo '0Wc0FQefoPszf3CDrHv/RHvhHq97jXDwTb0CQQDgH83NygoS1r57pCw9chzpG/R0' >> /root/.ssh/id_rsa &amp;&amp; \
echo 'aMEiSPhCvz757fj+qT3aGIal2AJ7/2c/gRZvwrWNETZ3XIZOUKqIkXzJLPjBAkEA' >> /root/.ssh/id_rsa &amp;&amp; \
echo 'wnP799W2Y8d4/+VX2pMBkF7lG7sSviHEq1sP2BZtPBRQKSQNvw3scM7XcGh/mxmY' >> /root/.ssh/id_rsa &amp;&amp; \
echo 'yx0qpqfKa8SKbNgI1+4iXQJBAOlg8MJLwkUtrG+p8wf69oCuZsnyv0K6UMDxm6/8' >> /root/.ssh/id_rsa &amp;&amp; \
echo 'cbvfmvODulYFaIahaqHWEZoRo5CLYZ7gN43WHPOrKxdDL78=' >> /root/.ssh/id_rsa &amp;&amp; \
echo '-----END RSA PRIVATE KEY-----' >> /root/.ssh/id_rsa &amp;&amp; \
echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAAgQDR0yJsZW5d5LcqteiOtv8d+FFeFFHDPI0VTcTOdMn1iDiIP1ouX3Q2OyNjsBaDbsRJd+sp9IRq1LKX3zsBcgGZANwm0zduuNEPEU94ajS/uRoejIqY/XkKOpnEF6ZbQ2S7TaE4sWeGLvba7kUFs0QTOO+N+nV2dMbdqZf6C8lazw== root@pgbackrest-doc' > /root/.ssh/authorized_keys &amp;&amp; \
echo 'Host *' > /root/.ssh/config &amp;&amp; \
echo ' StrictHostKeyChecking no' >> /root/.ssh/config &amp;&amp; \
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
chmod 600 /root/.ssh/*
</variable>
Use minio as local S3 emulator in documentation. The documentation was relying on a ScalityS3 container built for testing which wasn't very transparent. Instead, use the stock minio container and configure it in the documentation. Also, install certificates and CA so that TLS verification can be enabled.
2019-05-27 13:37:20 +02:00
<variable key="copy-ca-cert">COPY {[fake-cert-path-relative]}/ca.crt {[ca-cert-path]}/pgbackrest-ca.crt</variable>
Don't allow sudo to disable core dumps in documentation. Newer versions of sudo output this message to stdout when run in a container: sudo: setrlimit(RLIMIT_CORE): Operation not permitted See https://github.com/sudo-project/sudo/issues/42 for details. A simple workaround is to prevent sudo from disabling core dumps. This seems safe enough because if sudo is segfaulting then core files are the least of our worries.
2020-05-07 13:23:15 +02:00
<!-- Don't allow sudo to disable core dump (suppresses errors, see https://github.com/sudo-project/sudo/issues/42) -->
<variable key="sudo-disable-core-dump">RUN echo "Set disable_coredump false" >> /etc/sudo.conf</variable>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
</variable-list>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Move most host setup to containers defined in the documentation. This includes PostgreSQL installation which had previously been included in the documentation. This way produces faster builds and there is no need for us to document PostgreSQL installation.
2018-07-23 13:59:24 +02:00
<!-- Setup hosts used to build the documentation
============================================================================================================================ -->
Create common if expressions for testing os-type. These expressions simplify os-type testing. This will be especially true as more OS types are added.
2018-12-05 00:33:56 +02:00
<host-define if="{[os-type-is-debian]}" image="{[host-image]}" from="{[os-image]}">
Use minio as local S3 emulator in documentation. The documentation was relying on a ScalityS3 container built for testing which wasn't very transparent. Instead, use the stock minio container and configure it in the documentation. Also, install certificates and CA so that TLS verification can be enabled.
2019-05-27 13:37:20 +02:00
{[copy-ca-cert]}
Move most host setup to containers defined in the documentation. This includes PostgreSQL installation which had previously been included in the documentation. This way produces faster builds and there is no need for us to document PostgreSQL installation.
2018-07-23 13:59:24 +02:00
# Fix root tty
Add Zstandard compression support. Zstandard is a fast lossless compression algorithm targeting real-time compression scenarios at zlib-level and better compression ratios. It's backed by a very fast entropy stage, provided by Huff0 and FSE library. Zstandard version >= 1.0 is required, which is generally only available on newer distributions.
2020-05-04 21:25:27 +02:00
RUN sed -i 's/^mesg n/tty -s \&amp;\&amp; mesg n/g' /root/.profile
Move most host setup to containers defined in the documentation. This includes PostgreSQL installation which had previously been included in the documentation. This way produces faster builds and there is no need for us to document PostgreSQL installation.
2018-07-23 13:59:24 +02:00
Add Zstandard compression support. Zstandard is a fast lossless compression algorithm targeting real-time compression scenarios at zlib-level and better compression ratios. It's backed by a very fast entropy stage, provided by Huff0 and FSE library. Zstandard version >= 1.0 is required, which is generally only available on newer distributions.
2020-05-04 21:25:27 +02:00
# Install base packages (suppress dpkg interactive output)
RUN export DEBIAN_FRONTEND=noninteractive &amp;&amp; \
rm /etc/apt/apt.conf.d/70debconf &amp;&amp; \
apt-get update &amp;&amp; \
Update test container OS versions. Update Ubuntu 12.04 to 16.04. Version 16.04 is recently EOL but testing on an old version is beneficial. Update Ubuntu 18.04 to 20.04. Update Fedora 32 to 33. Version 34 would have been preferred but there were some build issues, i.e. the default shell did not work with configure, and after ksh was installed configure locked up. Add --no-install-recommends to apt-get commands to save a bit of time and space. Update test Dockerfile to run in multiple steps. This makes the container larger but also makes rebuilding after changes faster. The --squash option may be used to keep the container small. Remove obsolete casts in protocol/parallel module. These casts were included in the original migration because Ubuntu 12.04 32-bit gcc required them, but Ubuntu 16.04 32-bit gcc complains. There is no production issue here since at this point in the code the file descriptors are guaranteed to be >= 0.
2021-07-18 23:29:20 +02:00
apt-get install -y --no-install-recommends sudo ssh wget vim gnupg lsb-release iputils-ping ca-certificates \
tzdata locales 2>&amp;1
Use minio as local S3 emulator in documentation. The documentation was relying on a ScalityS3 container built for testing which wasn't very transparent. Instead, use the stock minio container and configure it in the documentation. Also, install certificates and CA so that TLS verification can be enabled.
2019-05-27 13:37:20 +02:00
Don't allow sudo to disable core dumps in documentation. Newer versions of sudo output this message to stdout when run in a container: sudo: setrlimit(RLIMIT_CORE): Operation not permitted See https://github.com/sudo-project/sudo/issues/42 for details. A simple workaround is to prevent sudo from disabling core dumps. This seems safe enough because if sudo is segfaulting then core files are the least of our worries.
2020-05-07 13:23:15 +02:00
{[sudo-disable-core-dump]}
Use minio as local S3 emulator in documentation. The documentation was relying on a ScalityS3 container built for testing which wasn't very transparent. Instead, use the stock minio container and configure it in the documentation. Also, install certificates and CA so that TLS verification can be enabled.
2019-05-27 13:37:20 +02:00
# Install CA certificate
RUN update-ca-certificates
Move most host setup to containers defined in the documentation. This includes PostgreSQL installation which had previously been included in the documentation. This way produces faster builds and there is no need for us to document PostgreSQL installation.
2018-07-23 13:59:24 +02:00
# Install PostgreSQL
Documentation may be built with user-specified packages. By default the documentation builds pgBackRest from source, but the documentation is also a good way to smoke-test packages. Allow a package file to be specified by passing --var=package=/path/to/package.ext. This works for Debian and CentOS 6 builds.
2018-12-04 20:17:55 +02:00
RUN RELEASE_CODENAME=`lsb_release -c | awk '{print $2}'` &amp;&amp; \
Update test containers for PostgreSQL 13. Add older PostgreSQL versions to the u18 container that were not available before. This also updates all minor versions for prior versions of PostgreSQL.
2020-09-24 17:19:51 +02:00
echo 'deb http://apt.postgresql.org/pub/repos/apt/ '${RELEASE_CODENAME?}'-pgdg main' | \
Move most host setup to containers defined in the documentation. This includes PostgreSQL installation which had previously been included in the documentation. This way produces faster builds and there is no need for us to document PostgreSQL installation.
2018-07-23 13:59:24 +02:00
tee -a /etc/apt/sources.list.d/pgdg.list &amp;&amp; \
wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add - &amp;&amp; \
apt-get update &amp;&amp; \
Update test container OS versions. Update Ubuntu 12.04 to 16.04. Version 16.04 is recently EOL but testing on an old version is beneficial. Update Ubuntu 18.04 to 20.04. Update Fedora 32 to 33. Version 34 would have been preferred but there were some build issues, i.e. the default shell did not work with configure, and after ksh was installed configure locked up. Add --no-install-recommends to apt-get commands to save a bit of time and space. Update test Dockerfile to run in multiple steps. This makes the container larger but also makes rebuilding after changes faster. The --squash option may be used to keep the container small. Remove obsolete casts in protocol/parallel module. These casts were included in the original migration because Ubuntu 12.04 32-bit gcc required them, but Ubuntu 16.04 32-bit gcc complains. There is no production issue here since at this point in the code the file descriptors are guaranteed to be >= 0.
2021-07-18 23:29:20 +02:00
apt-get install -y --no-install-recommends postgresql-common 2>&amp;1 &amp;&amp; \
Move most host setup to containers defined in the documentation. This includes PostgreSQL installation which had previously been included in the documentation. This way produces faster builds and there is no need for us to document PostgreSQL installation.
2018-07-23 13:59:24 +02:00
sed -i 's/^\#create\_main\_cluster.*$/create\_main\_cluster \= false/' \
/etc/postgresql-common/createcluster.conf &amp;&amp; \
Update test container OS versions. Update Ubuntu 12.04 to 16.04. Version 16.04 is recently EOL but testing on an old version is beneficial. Update Ubuntu 18.04 to 20.04. Update Fedora 32 to 33. Version 34 would have been preferred but there were some build issues, i.e. the default shell did not work with configure, and after ksh was installed configure locked up. Add --no-install-recommends to apt-get commands to save a bit of time and space. Update test Dockerfile to run in multiple steps. This makes the container larger but also makes rebuilding after changes faster. The --squash option may be used to keep the container small. Remove obsolete casts in protocol/parallel module. These casts were included in the original migration because Ubuntu 12.04 32-bit gcc required them, but Ubuntu 16.04 32-bit gcc complains. There is no production issue here since at this point in the code the file descriptors are guaranteed to be >= 0.
2021-07-18 23:29:20 +02:00
apt-get install -y --no-install-recommends postgresql-{[pg-version]} postgresql-{[pg-version-upgrade]} 2>&amp;1
Move most host setup to containers defined in the documentation. This includes PostgreSQL installation which had previously been included in the documentation. This way produces faster builds and there is no need for us to document PostgreSQL installation.
2018-07-23 13:59:24 +02:00
# Create an ssh key for root so all hosts can ssh to each other as root
RUN \ {[ssh-key-install]}
# Add doc user with sudo privileges
RUN adduser --disabled-password --gecos "" {[host-user]} &amp;&amp; \
echo '%{[host-user]} ALL=(ALL) NOPASSWD: ALL' >> /etc/sudoers
Set encoding to UTF8 for Debian documentation containers. This allows testing multi-byte encodings in PostgreSQL.
2020-01-22 03:37:43 +02:00
# Set UTF8 encoding
RUN sed -i -e 's/# en_US.UTF-8 UTF-8/en_US.UTF-8 UTF-8/' /etc/locale.gen &amp;&amp; \
dpkg-reconfigure --frontend=noninteractive locales &amp;&amp; \
update-locale LANG=en_US.UTF-8
ENV LANG en_US.UTF-8
Move most host setup to containers defined in the documentation. This includes PostgreSQL installation which had previously been included in the documentation. This way produces faster builds and there is no need for us to document PostgreSQL installation.
2018-07-23 13:59:24 +02:00
ENTRYPOINT service ssh restart &amp;&amp; bash
</host-define>
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<host-define if="{[os-type-is-rhel]}" image="{[host-image]}" from="{[os-image]}">
Add Centos/RHEL 7 option to documentation build. Centos/RHEL 7 builds can be specified with --var=os-type=centos7. Also added to the documentation release and CI builds.
2018-12-05 14:12:28 +02:00
ENV container docker
Use minio as local S3 emulator in documentation. The documentation was relying on a ScalityS3 container built for testing which wasn't very transparent. Instead, use the stock minio container and configure it in the documentation. Also, install certificates and CA so that TLS verification can be enabled.
2019-05-27 13:37:20 +02:00
{[copy-ca-cert]}
Add Centos/RHEL 7 option to documentation build. Centos/RHEL 7 builds can be specified with --var=os-type=centos7. Also added to the documentation release and CI builds.
2018-12-05 14:12:28 +02:00
RUN (cd /lib/systemd/system/sysinit.target.wants/; for i in *; do [ $i == \
systemd-tmpfiles-setup.service ] || rm -f $i; done); \
rm -f /lib/systemd/system/multi-user.target.wants/*;\
rm -f /etc/systemd/system/*.wants/*;\
rm -f /lib/systemd/system/local-fs.target.wants/*; \
rm -f /lib/systemd/system/sockets.target.wants/*udev*; \
rm -f /lib/systemd/system/sockets.target.wants/*initctl*; \
rm -f /lib/systemd/system/basic.target.wants/*;\
rm -f /lib/systemd/system/anaconda.target.wants/*;
VOLUME [ "/sys/fs/cgroup" ]
# Install packages
Add findutils package to RHEL 8 documentation container. This package was dropped from the most recent Rocky Linux 8 image.
2021-11-16 16:27:15 +02:00
RUN yum install -y openssh-server openssh-clients sudo wget vim findutils dnf-plugins-core 2>&amp;1
Partial migration of config code generation to C. Parse enough of config.yaml to auto-generate config.auto.h and config.auto.c. This commit implements most of the infrastructure needed to migrate the rest of the build code to C, but each set of auto-generated files will present its own challenges. The build is now dependent on libyaml. At this point there is no need for a hard requirement, but that will come soon so it seems better to add the dependency now.
2021-07-19 01:02:01 +02:00
# Enable PowerTools repository (only available on RHEL8)
RUN dnf config-manager --set-enabled powertools || true
Add Centos/RHEL 7 option to documentation build. Centos/RHEL 7 builds can be specified with --var=os-type=centos7. Also added to the documentation release and CI builds.
2018-12-05 14:12:28 +02:00
Use minio as local S3 emulator in documentation. The documentation was relying on a ScalityS3 container built for testing which wasn't very transparent. Instead, use the stock minio container and configure it in the documentation. Also, install certificates and CA so that TLS verification can be enabled.
2019-05-27 13:37:20 +02:00
# Install CA certificate
RUN update-ca-trust extract
Add Centos/RHEL 7 option to documentation build. Centos/RHEL 7 builds can be specified with --var=os-type=centos7. Also added to the documentation release and CI builds.
2018-12-05 14:12:28 +02:00
# Regenerate SSH keys
RUN rm -f /etc/ssh/ssh_host_rsa_key* &amp;&amp; \
ssh-keygen -t rsa -f /etc/ssh/ssh_host_rsa_key &amp;&amp; \
rm -f /etc/ssh/ssh_host_dsa_key* &amp;&amp; \
ssh-keygen -t dsa -f /etc/ssh/ssh_host_dsa_key
Add RHEL/CentOS 8 documentation. Update RHEL/CentOS 7 to cover the versions that were previously covered by RHEL/CentOS 6. Since RHEL/CentOS 7/8 work the same update the documentation logic and labels to reflect this compatibility.
2020-12-04 17:59:57 +02:00
# Install PGDG PostgreSQL repository
Begin v2.13 development. Also update CentOS packages so documentation builds.
2019-04-12 14:33:10 +02:00
RUN rpm --import http://yum.postgresql.org/RPM-GPG-KEY-PGDG-10 &amp;&amp; \
Add RHEL/CentOS 8 documentation. Update RHEL/CentOS 7 to cover the versions that were previously covered by RHEL/CentOS 6. Since RHEL/CentOS 7/8 work the same update the documentation logic and labels to reflect this compatibility.
2020-12-04 17:59:57 +02:00
rpm -ivh https://download.postgresql.org/pub/repos/yum/reporpms/EL-8-x86_64/pgdg-redhat-repo-latest.noarch.rpm
# Disable default PostgreSQL repository
RUN command -v dnf >/dev/null 2>&amp;1 &amp;&amp; dnf -qy module disable postgresql || true
# Install PostgreSQL
RUN yum install -y postgresql{[pg-version-nodot]}-server postgresql{[pg-version-upgrade-nodot]}-server
Add Centos/RHEL 7 option to documentation build. Centos/RHEL 7 builds can be specified with --var=os-type=centos7. Also added to the documentation release and CI builds.
2018-12-05 14:12:28 +02:00
# Create an ssh key for root so all hosts can ssh to each other as root
RUN \ {[ssh-key-install]}
# Add doc user with sudo privileges
RUN adduser -n {[host-user]} &amp;&amp; \
echo '{[host-user]} ALL=(ALL) NOPASSWD: ALL' > /etc/sudoers.d/{[host-user]}
Fix Centos/RHEL 7 documentation builds. This was caused by a new container version that was released around December 5th. The new version explicitly denies user logons by leaving /var/run/nologin in place after boot. The solution is to enable the service that is responsible for removing this file on a successful boot.
2018-12-12 18:15:09 +02:00
# Enable the user session service so logons are allowed
Add RHEL/CentOS 8 documentation. Update RHEL/CentOS 7 to cover the versions that were previously covered by RHEL/CentOS 6. Since RHEL/CentOS 7/8 work the same update the documentation logic and labels to reflect this compatibility.
2020-12-04 17:59:57 +02:00
RUN echo "[Install]" >> /usr/lib/systemd/system/systemd-user-sessions.service &amp;&amp; \
echo "[WantedBy=default.target]" >> /usr/lib/systemd/system/systemd-user-sessions.service &amp;&amp; \
systemctl enable systemd-user-sessions.service &amp;&amp; \
mkdir -p /etc/systemd/system/default.target.wants &amp;&amp; \
Fix Centos/RHEL 7 documentation builds. This was caused by a new container version that was released around December 5th. The new version explicitly denies user logons by leaving /var/run/nologin in place after boot. The solution is to enable the service that is responsible for removing this file on a successful boot.
2018-12-12 18:15:09 +02:00
ln -s /usr/lib/systemd/system/systemd-user-sessions.service \
/etc/systemd/system/default.target.wants/systemd-user-sessions.service
Add RHEL/CentOS 8 documentation. Update RHEL/CentOS 7 to cover the versions that were previously covered by RHEL/CentOS 6. Since RHEL/CentOS 7/8 work the same update the documentation logic and labels to reflect this compatibility.
2020-12-04 17:59:57 +02:00
# Set locale
RUN echo en_US.UTF-8 UTF-8 > /etc/locale.conf
# Add path to PostgreSQL
ENV PATH=/usr/pgsql-{[pg-version]}/bin:$PATH
Add Centos/RHEL 7 option to documentation build. Centos/RHEL 7 builds can be specified with --var=os-type=centos7. Also added to the documentation release and CI builds.
2018-12-05 14:12:28 +02:00
CMD ["/usr/sbin/init"]
</host-define>
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
<!-- ======================================================================================================================= -->
<block-define id="setup-ssh-intro">
Use command in authorized_hosts to improve SSH security. Unsecured, passwordless SSH can be a scary thing. If an attacker gains access to one system they can easily hop to other systems. Add documentation on how to use the command parameter in authorized_keys to limit ssh to running a single command, pgbackrest. There is more that could be done for security but this likely addresses most needs. Also change references to "trusted ssh" to "passwordless ssh" since this seems more correct. Suggested by Stephen Frost, Magnus Hagander.
2018-09-19 22:41:20 +02:00
<p><backrest/> requires passwordless SSH to enable communication between the hosts.</p>
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
</block-define>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
<block-define id="setup-ssh">
<execute-list host="{[setup-ssh-host]}">
<title>Create <host>{[setup-ssh-host]}</host> host key pair</title>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Preliminary documentation for PostgreSQL 11 unprivileged user backup.
2018-06-15 01:05:35 +02:00
<execute user="{[setup-ssh-user]}">
<exe-cmd>mkdir -m 750 -p {[setup-ssh-user-home-path]}/.ssh</exe-cmd>
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
</execute>
Preliminary documentation for PostgreSQL 11 unprivileged user backup.
2018-06-15 01:05:35 +02:00
<execute user="{[setup-ssh-user]}">
<exe-cmd>ssh-keygen -f {[setup-ssh-user-home-path]}/.ssh/id_rsa
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
-t rsa -b 4096 -N ""</exe-cmd>
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
</execute>
</execute-list>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Update backup host to repository host in the documentation. Contributed by Cynthia Shang.
2018-02-19 20:17:58 +02:00
<p>Exchange keys between <host>{[host-repo1]}</host> and <host>{[setup-ssh-host]}</host>.</p>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Update backup host to repository host in the documentation. Contributed by Cynthia Shang.
2018-02-19 20:17:58 +02:00
<execute-list host="{[host-repo1]}">
<title>Copy <host>{[setup-ssh-host]}</host> public key to <host>{[host-repo1]}</host></title>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Use command in authorized_hosts to improve SSH security. Unsecured, passwordless SSH can be a scary thing. If an attacker gains access to one system they can easily hop to other systems. Add documentation on how to use the command parameter in authorized_keys to limit ssh to running a single command, pgbackrest. There is more that could be done for security but this likely addresses most needs. Also change references to "trusted ssh" to "passwordless ssh" since this seems more correct. Suggested by Stephen Frost, Magnus Hagander.
2018-09-19 22:41:20 +02:00
<execute user="root" err-suppress="y" user-force="y">
<exe-cmd>
(echo -n 'no-agent-forwarding,no-X11-forwarding,no-port-forwarding,' &amp;&amp;
echo -n 'command="{[br-bin]} ${SSH_ORIGINAL_COMMAND#* }" ' &amp;&amp;
sudo ssh root@{[setup-ssh-host]} cat {[setup-ssh-user-home-path]}/.ssh/id_rsa.pub) |
sudo -u pgbackrest tee -a {[br-home-path]}/.ssh/authorized_keys
</exe-cmd>
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
</execute>
</execute-list>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
<execute-list host="{[setup-ssh-host]}">
Update backup host to repository host in the documentation. Contributed by Cynthia Shang.
2018-02-19 20:17:58 +02:00
<title>Copy <host>{[host-repo1]}</host> public key to <host>{[setup-ssh-host]}</host></title>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Use command in authorized_hosts to improve SSH security. Unsecured, passwordless SSH can be a scary thing. If an attacker gains access to one system they can easily hop to other systems. Add documentation on how to use the command parameter in authorized_keys to limit ssh to running a single command, pgbackrest. There is more that could be done for security but this likely addresses most needs. Also change references to "trusted ssh" to "passwordless ssh" since this seems more correct. Suggested by Stephen Frost, Magnus Hagander.
2018-09-19 22:41:20 +02:00
<execute user="root" err-suppress="y" user-force="y">
<exe-cmd>
(echo -n 'no-agent-forwarding,no-X11-forwarding,no-port-forwarding,' &amp;&amp;
echo -n 'command="{[br-bin]} ${SSH_ORIGINAL_COMMAND#* }" ' &amp;&amp;
sudo ssh root@{[host-repo1]} cat {[br-home-path]}/.ssh/id_rsa.pub) |
sudo -u {[setup-ssh-user]} tee -a {[setup-ssh-user-home-path]}/.ssh/authorized_keys
</exe-cmd>
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
</execute>
</execute-list>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Update backup host to repository host in the documentation. Contributed by Cynthia Shang.
2018-02-19 20:17:58 +02:00
<p>Test that connections can be made from <host>{[host-repo1]}</host> to <host>{[setup-ssh-host]}</host> and vice versa.</p>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Update backup host to repository host in the documentation. Contributed by Cynthia Shang.
2018-02-19 20:17:58 +02:00
<execute-list host="{[host-repo1]}">
<title>Test connection from <host>{[host-repo1]}</host> to <host>{[setup-ssh-host]}</host></title>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Preliminary documentation for PostgreSQL 11 unprivileged user backup.
2018-06-15 01:05:35 +02:00
<execute user="{[br-user]}" err-suppress="y">
<exe-cmd>ssh {[setup-ssh-user]}@{[setup-ssh-host]}</exe-cmd>
Use command in authorized_hosts to improve SSH security. Unsecured, passwordless SSH can be a scary thing. If an attacker gains access to one system they can easily hop to other systems. Add documentation on how to use the command parameter in authorized_keys to limit ssh to running a single command, pgbackrest. There is more that could be done for security but this likely addresses most needs. Also change references to "trusted ssh" to "passwordless ssh" since this seems more correct. Suggested by Stephen Frost, Magnus Hagander.
2018-09-19 22:41:20 +02:00
<exe-cmd-extra>-o StrictHostKeyChecking=no</exe-cmd-extra>
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
</execute>
</execute-list>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
<execute-list host="{[setup-ssh-host]}">
Update backup host to repository host in the documentation. Contributed by Cynthia Shang.
2018-02-19 20:17:58 +02:00
<title>Test connection from <host>{[setup-ssh-host]}</host> to <host>{[host-repo1]}</host></title>
S3 repository support.
2017-06-12 16:52:32 +02:00
Preliminary documentation for PostgreSQL 11 unprivileged user backup.
2018-06-15 01:05:35 +02:00
<execute user="{[setup-ssh-user]}" err-suppress="y">
Update backup host to repository host in the documentation. Contributed by Cynthia Shang.
2018-02-19 20:17:58 +02:00
<exe-cmd>ssh pgbackrest@{[host-repo1]}</exe-cmd>
Use command in authorized_hosts to improve SSH security. Unsecured, passwordless SSH can be a scary thing. If an attacker gains access to one system they can easily hop to other systems. Add documentation on how to use the command parameter in authorized_keys to limit ssh to running a single command, pgbackrest. There is more that could be done for security but this likely addresses most needs. Also change references to "trusted ssh" to "passwordless ssh" since this seems more correct. Suggested by Stephen Frost, Magnus Hagander.
2018-09-19 22:41:20 +02:00
<exe-cmd-extra>-o StrictHostKeyChecking=no</exe-cmd-extra>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
</execute>
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
</execute-list>
</block-define>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
<!-- ======================================================================================================================= -->
Documentation may be built with user-specified packages. By default the documentation builds pgBackRest from source, but the documentation is also a good way to smoke-test packages. Allow a package file to be specified by passing --var=package=/path/to/package.ext. This works for Debian and CentOS 6 builds.
2018-12-04 20:17:55 +02:00
<block-define if="'{[package]}' eq 'none'" id="br-install">
Create a new section to describe building pgBackRest and build on a separate host.
2018-06-13 19:43:17 +02:00
<p><backrest/> needs to be installed from a package or installed manually as shown here.</p>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Fix incorrect host names in user guide. pg1 was incorrectly used instead of {[host-pg1]} which meant the wrong host name was displayed. Also, the install block was installing packages to the build host no matter which host was specified.
2021-08-03 20:48:22 +02:00
<execute-list host="{[br-install-host]}">
pgBackRest is now pure C. Remove embedded Perl from the distributed binary. This includes code, configure, Makefile, and packages. The distributed binary is now pure C. Remove storagePathEnforceSet() from the C Storage object which allowed Perl to write outside of the storage base directory. Update mock/all and real/all integration tests to use storageLocal() where they were violating this rule. Remove "c" option that allowed the remote to tell if it was being called from C or Perl. Code to convert options to JSON for passing to Perl (perl/config.c) has been moved to LibC since it is still required for Perl integration tests. Update build and installation instructions in the user guide. Remove all Perl unit tests. Remove obsolete Perl code. In particular this included all the Perl protocol code which required modifications to the Perl storage, manifest, and db objects that are still required for integration testing but only run locally. Any remaining Perl code is required for testing, documentation, or code generation. Rename perlReq to binReq in define.yaml to indicate that the binary is required for a test. This had been the actual meaning for quite some time but the key was never renamed.
2019-12-14 00:55:41 +02:00
<title>Install dependencies</title>
Update installation documentation. * Recommended install location for pgBackRest modules is now /usr/share/perl5 since /usr/lib/perl5 has been removed from the search path in newer versions of Perl. * Added instructions for removing prior versions of pgBackRest.
2016-06-04 00:59:12 +02:00
pgBackRest is now pure C. Remove embedded Perl from the distributed binary. This includes code, configure, Makefile, and packages. The distributed binary is now pure C. Remove storagePathEnforceSet() from the C Storage object which allowed Perl to write outside of the storage base directory. Update mock/all and real/all integration tests to use storageLocal() where they were violating this rule. Remove "c" option that allowed the remote to tell if it was being called from C or Perl. Code to convert options to JSON for passing to Perl (perl/config.c) has been moved to LibC since it is still required for Perl integration tests. Update build and installation instructions in the user guide. Remove all Perl unit tests. Remove obsolete Perl code. In particular this included all the Perl protocol code which required modifications to the Perl storage, manifest, and db objects that are still required for integration testing but only run locally. Any remaining Perl code is required for testing, documentation, or code generation. Rename perlReq to binReq in define.yaml to indicate that the binary is required for a test. This had been the actual meaning for quite some time but the key was never renamed.
2019-12-14 00:55:41 +02:00
<execute if="{[os-type-is-debian]}" user="root" pre="y">
<exe-cmd>
apt-get install postgresql-client libxml2</exe-cmd>
<exe-cmd-extra>-y 2>&amp;1</exe-cmd-extra>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
</execute>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<execute if="{[os-type-is-rhel]}" user="root" pre="y">
pgBackRest is now pure C. Remove embedded Perl from the distributed binary. This includes code, configure, Makefile, and packages. The distributed binary is now pure C. Remove storagePathEnforceSet() from the C Storage object which allowed Perl to write outside of the storage base directory. Update mock/all and real/all integration tests to use storageLocal() where they were violating this rule. Remove "c" option that allowed the remote to tell if it was being called from C or Perl. Code to convert options to JSON for passing to Perl (perl/config.c) has been moved to LibC since it is still required for Perl integration tests. Update build and installation instructions in the user guide. Remove all Perl unit tests. Remove obsolete Perl code. In particular this included all the Perl protocol code which required modifications to the Perl storage, manifest, and db objects that are still required for integration testing but only run locally. Any remaining Perl code is required for testing, documentation, or code generation. Rename perlReq to binReq in define.yaml to indicate that the binary is required for a test. This had been the actual meaning for quite some time but the key was never renamed.
2019-12-14 00:55:41 +02:00
<exe-cmd>
yum install postgresql-libs
</exe-cmd>
<exe-cmd-extra>-y 2>&amp;1</exe-cmd-extra>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
</execute>
Embed exported C functions and Perl modules directly into the pgBackRest executable.
2018-05-22 18:53:08 +02:00
</execute-list>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute-list host="{[br-install-host]}">
pgBackRest is now pure C. Remove embedded Perl from the distributed binary. This includes code, configure, Makefile, and packages. The distributed binary is now pure C. Remove storagePathEnforceSet() from the C Storage object which allowed Perl to write outside of the storage base directory. Update mock/all and real/all integration tests to use storageLocal() where they were violating this rule. Remove "c" option that allowed the remote to tell if it was being called from C or Perl. Code to convert options to JSON for passing to Perl (perl/config.c) has been moved to LibC since it is still required for Perl integration tests. Update build and installation instructions in the user guide. Remove all Perl unit tests. Remove obsolete Perl code. In particular this included all the Perl protocol code which required modifications to the Perl storage, manifest, and db objects that are still required for integration testing but only run locally. Any remaining Perl code is required for testing, documentation, or code generation. Rename perlReq to binReq in define.yaml to indicate that the binary is required for a test. This had been the actual meaning for quite some time but the key was never renamed.
2019-12-14 00:55:41 +02:00
<title>Copy <backrest/> binary from build host</title>
Embed exported C functions and Perl modules directly into the pgBackRest executable.
2018-05-22 18:53:08 +02:00
pgBackRest is now pure C. Remove embedded Perl from the distributed binary. This includes code, configure, Makefile, and packages. The distributed binary is now pure C. Remove storagePathEnforceSet() from the C Storage object which allowed Perl to write outside of the storage base directory. Update mock/all and real/all integration tests to use storageLocal() where they were violating this rule. Remove "c" option that allowed the remote to tell if it was being called from C or Perl. Code to convert options to JSON for passing to Perl (perl/config.c) has been moved to LibC since it is still required for Perl integration tests. Update build and installation instructions in the user guide. Remove all Perl unit tests. Remove obsolete Perl code. In particular this included all the Perl protocol code which required modifications to the Perl storage, manifest, and db objects that are still required for integration testing but only run locally. Any remaining Perl code is required for testing, documentation, or code generation. Rename perlReq to binReq in define.yaml to indicate that the binary is required for a test. This had been the actual meaning for quite some time but the key was never renamed.
2019-12-14 00:55:41 +02:00
<execute user="root">
Add variables for build paths in the user guide. This eliminates repetition of the build path so it can be changed more easily. Also create the build path explicitly rather than suggest that the user do it.
2021-08-18 16:52:35 +02:00
<exe-cmd>scp {[host-build]}:{[build-br-path]}/src/pgbackrest /usr/bin</exe-cmd>
pgBackRest is now pure C. Remove embedded Perl from the distributed binary. This includes code, configure, Makefile, and packages. The distributed binary is now pure C. Remove storagePathEnforceSet() from the C Storage object which allowed Perl to write outside of the storage base directory. Update mock/all and real/all integration tests to use storageLocal() where they were violating this rule. Remove "c" option that allowed the remote to tell if it was being called from C or Perl. Code to convert options to JSON for passing to Perl (perl/config.c) has been moved to LibC since it is still required for Perl integration tests. Update build and installation instructions in the user guide. Remove all Perl unit tests. Remove obsolete Perl code. In particular this included all the Perl protocol code which required modifications to the Perl storage, manifest, and db objects that are still required for integration testing but only run locally. Any remaining Perl code is required for testing, documentation, or code generation. Rename perlReq to binReq in define.yaml to indicate that the binary is required for a test. This had been the actual meaning for quite some time but the key was never renamed.
2019-12-14 00:55:41 +02:00
<exe-cmd-extra>2>&amp;1</exe-cmd-extra>
Add support for additional pgBackRest configuration files. Configuration files are loaded from the directory specified by the --config-include-path option. Add --config-path option for overriding the default base path of the --config and --config-include-path option. Contributed by Cynthia Shang.
2018-04-19 17:09:39 +02:00
</execute>
Add Centos/RHEL 7 option to documentation build. Centos/RHEL 7 builds can be specified with --var=os-type=centos7. Also added to the documentation release and CI builds.
2018-12-05 14:12:28 +02:00
pgBackRest is now pure C. Remove embedded Perl from the distributed binary. This includes code, configure, Makefile, and packages. The distributed binary is now pure C. Remove storagePathEnforceSet() from the C Storage object which allowed Perl to write outside of the storage base directory. Update mock/all and real/all integration tests to use storageLocal() where they were violating this rule. Remove "c" option that allowed the remote to tell if it was being called from C or Perl. Code to convert options to JSON for passing to Perl (perl/config.c) has been moved to LibC since it is still required for Perl integration tests. Update build and installation instructions in the user guide. Remove all Perl unit tests. Remove obsolete Perl code. In particular this included all the Perl protocol code which required modifications to the Perl storage, manifest, and db objects that are still required for integration testing but only run locally. Any remaining Perl code is required for testing, documentation, or code generation. Rename perlReq to binReq in define.yaml to indicate that the binary is required for a test. This had been the actual meaning for quite some time but the key was never renamed.
2019-12-14 00:55:41 +02:00
<execute user="root">
<exe-cmd>chmod 755 /usr/bin/pgbackrest</exe-cmd>
Add Centos/RHEL 7 option to documentation build. Centos/RHEL 7 builds can be specified with --var=os-type=centos7. Also added to the documentation release and CI builds.
2018-12-05 14:12:28 +02:00
</execute>
Embed exported C functions and Perl modules directly into the pgBackRest executable.
2018-05-22 18:53:08 +02:00
</execute-list>
pgBackRest is now pure C. Remove embedded Perl from the distributed binary. This includes code, configure, Makefile, and packages. The distributed binary is now pure C. Remove storagePathEnforceSet() from the C Storage object which allowed Perl to write outside of the storage base directory. Update mock/all and real/all integration tests to use storageLocal() where they were violating this rule. Remove "c" option that allowed the remote to tell if it was being called from C or Perl. Code to convert options to JSON for passing to Perl (perl/config.c) has been moved to LibC since it is still required for Perl integration tests. Update build and installation instructions in the user guide. Remove all Perl unit tests. Remove obsolete Perl code. In particular this included all the Perl protocol code which required modifications to the Perl storage, manifest, and db objects that are still required for integration testing but only run locally. Any remaining Perl code is required for testing, documentation, or code generation. Rename perlReq to binReq in define.yaml to indicate that the binary is required for a test. This had been the actual meaning for quite some time but the key was never renamed.
2019-12-14 00:55:41 +02:00
<p><backrest/> requires log and configuration directories and a configuration file.</p>
Embed exported C functions and Perl modules directly into the pgBackRest executable.
2018-05-22 18:53:08 +02:00
<execute-list host="{[br-install-host]}">
<title>Create <backrest/> configuration file and directories</title>
Add support for additional pgBackRest configuration files. Configuration files are loaded from the directory specified by the --config-include-path option. Add --config-path option for overriding the default base path of the --config and --config-include-path option. Contributed by Cynthia Shang.
2018-04-19 17:09:39 +02:00
<execute user="root">
Embed exported C functions and Perl modules directly into the pgBackRest executable.
2018-05-22 18:53:08 +02:00
<exe-cmd>mkdir -p -m 770 /var/log/pgbackrest</exe-cmd>
Add support for additional pgBackRest configuration files. Configuration files are loaded from the directory specified by the --config-include-path option. Add --config-path option for overriding the default base path of the --config and --config-include-path option. Contributed by Cynthia Shang.
2018-04-19 17:09:39 +02:00
</execute>
Add creation of /etc/pgbackrest.conf to manual installation instructions.
2017-07-25 03:14:42 +02:00
<execute user="root">
Embed exported C functions and Perl modules directly into the pgBackRest executable.
2018-05-22 18:53:08 +02:00
<exe-cmd>chown {[br-install-user]}:{[br-install-group]} /var/log/pgbackrest</exe-cmd>
Add creation of /etc/pgbackrest.conf to manual installation instructions.
2017-07-25 03:14:42 +02:00
</execute>
<execute user="root">
Embed exported C functions and Perl modules directly into the pgBackRest executable.
2018-05-22 18:53:08 +02:00
<exe-cmd>mkdir -p {[backrest-config-path]}</exe-cmd>
Add creation of /etc/pgbackrest.conf to manual installation instructions.
2017-07-25 03:14:42 +02:00
</execute>
<execute user="root">
Embed exported C functions and Perl modules directly into the pgBackRest executable.
2018-05-22 18:53:08 +02:00
<exe-cmd>mkdir -p {[backrest-config-include-path]}</exe-cmd>
Add creation of /etc/pgbackrest.conf to manual installation instructions.
2017-07-25 03:14:42 +02:00
</execute>
Configuration rules are now pulled from the C library when present.
2017-08-25 22:47:47 +02:00
<execute user="root">
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<exe-cmd>touch {[backrest-config-demo]}</exe-cmd>
Configuration rules are now pulled from the C library when present.
2017-08-25 22:47:47 +02:00
</execute>
<execute user="root">
Embed exported C functions and Perl modules directly into the pgBackRest executable.
2018-05-22 18:53:08 +02:00
<exe-cmd>chmod 640 {[backrest-config-demo]}</exe-cmd>
The pgbackrest executable is now a C binary instead of Perl. This allows certain time-critical commands (like async archive-push) to run more quickly.
2017-11-27 01:43:51 +02:00
</execute>
<execute user="root">
Embed exported C functions and Perl modules directly into the pgBackRest executable.
2018-05-22 18:53:08 +02:00
<exe-cmd>chown {[br-install-user]}:{[br-install-group]} {[backrest-config-demo]}</exe-cmd>
The pgbackrest executable is now a C binary instead of Perl. This allows certain time-critical commands (like async archive-push) to run more quickly.
2017-11-27 01:43:51 +02:00
</execute>
</execute-list>
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
</block-define>
Documentation may be built with user-specified packages. By default the documentation builds pgBackRest from source, but the documentation is also a good way to smoke-test packages. Allow a package file to be specified by passing --var=package=/path/to/package.ext. This works for Debian and CentOS 6 builds.
2018-12-04 20:17:55 +02:00
<block-define if="'{[package]}' ne 'none'" id="br-install">
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
<execute-list host="{[br-install-host]}">
Documentation may be built with user-specified packages. By default the documentation builds pgBackRest from source, but the documentation is also a good way to smoke-test packages. Allow a package file to be specified by passing --var=package=/path/to/package.ext. This works for Debian and CentOS 6 builds.
2018-12-04 20:17:55 +02:00
<title>Install <backrest/> from package</title>
Option to build documentation from current apt.postgres.org packages. This is useful for testing any issue currently being seen in the field. We'll also need to add a "yum" option at some point.
2019-04-18 00:39:35 +02:00
<execute if="{[os-type-is-debian]} &amp;&amp; '{[package]}' ne 'apt'" user="root" err-suppress="y" show="n">
Fix hard-coded repository path. This prevented packages from being passed to the documentation unless they were in the /backrest directory on the host. Also make the local path /pgbackrest instead of the deprecated /backrest. Reported by Heath Lord.
2019-01-30 13:39:22 +02:00
<exe-cmd>dpkg -i {[pgbackrest-repo-path]}/{[package]}</exe-cmd>
Documentation may be built with user-specified packages. By default the documentation builds pgBackRest from source, but the documentation is also a good way to smoke-test packages. Allow a package file to be specified by passing --var=package=/path/to/package.ext. This works for Debian and CentOS 6 builds.
2018-12-04 20:17:55 +02:00
<exe-cmd-extra> 2>&amp;1</exe-cmd-extra>
</execute>
Option to build documentation from current apt.postgres.org packages. This is useful for testing any issue currently being seen in the field. We'll also need to add a "yum" option at some point.
2019-04-18 00:39:35 +02:00
<execute if="{[os-type-is-debian]} &amp;&amp; '{[package]}' ne 'apt'" user="root" show="n">
Documentation may be built with user-specified packages. By default the documentation builds pgBackRest from source, but the documentation is also a good way to smoke-test packages. Allow a package file to be specified by passing --var=package=/path/to/package.ext. This works for Debian and CentOS 6 builds.
2018-12-04 20:17:55 +02:00
<exe-cmd>apt-get -y install -f</exe-cmd>
<exe-cmd-extra>-y 2>&amp;1</exe-cmd-extra>
</execute>
Option to build documentation from current apt.postgres.org packages. This is useful for testing any issue currently being seen in the field. We'll also need to add a "yum" option at some point.
2019-04-18 00:39:35 +02:00
<execute if="{[os-type-is-debian]} &amp;&amp; '{[package]}' ne 'apt'" user="root" skip="y">
Documentation may be built with user-specified packages. By default the documentation builds pgBackRest from source, but the documentation is also a good way to smoke-test packages. Allow a package file to be specified by passing --var=package=/path/to/package.ext. This works for Debian and CentOS 6 builds.
2018-12-04 20:17:55 +02:00
<exe-cmd>apt-get install pgbackrest</exe-cmd>
</execute>
Option to build documentation from current apt.postgres.org packages. This is useful for testing any issue currently being seen in the field. We'll also need to add a "yum" option at some point.
2019-04-18 00:39:35 +02:00
<execute if="{[os-type-is-debian]} &amp;&amp; '{[package]}' eq 'apt'" user="root" show="n">
<exe-cmd>apt-get update</exe-cmd>
</execute>
<execute if="{[os-type-is-debian]} &amp;&amp; '{[package]}' eq 'apt'" user="root" show="y">
<exe-cmd>apt-get install pgbackrest</exe-cmd>
<exe-cmd-extra>-y 2>&amp;1</exe-cmd-extra>
</execute>
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<execute if="{[os-type-is-rhel]} &amp;&amp; '{[package]}' ne 'yum'" user="root" show="n">
Fix hard-coded repository path. This prevented packages from being passed to the documentation unless they were in the /backrest directory on the host. Also make the local path /pgbackrest instead of the deprecated /backrest. Reported by Heath Lord.
2019-01-30 13:39:22 +02:00
<exe-cmd>yum -y install {[pgbackrest-repo-path]}/{[package]}</exe-cmd>
Documentation may be built with user-specified packages. By default the documentation builds pgBackRest from source, but the documentation is also a good way to smoke-test packages. Allow a package file to be specified by passing --var=package=/path/to/package.ext. This works for Debian and CentOS 6 builds.
2018-12-04 20:17:55 +02:00
<exe-cmd-extra>-y 2>&amp;1</exe-cmd-extra>
</execute>
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<execute if="{[os-type-is-rhel]} &amp;&amp; '{[package]}' ne 'yum'" user="root" skip="y">
Documentation may be built with user-specified packages. By default the documentation builds pgBackRest from source, but the documentation is also a good way to smoke-test packages. Allow a package file to be specified by passing --var=package=/path/to/package.ext. This works for Debian and CentOS 6 builds.
2018-12-04 20:17:55 +02:00
<exe-cmd>yum install pgbackrest</exe-cmd>
</execute>
Community yum package can be installed with --var=package=yum. Like apt, the community yum package can now be installed instead of a user-specified package.
2019-06-27 20:39:11 +02:00
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<execute if="{[os-type-is-rhel]} &amp;&amp; '{[package]}' eq 'yum'" user="root">
Community yum package can be installed with --var=package=yum. Like apt, the community yum package can now be installed instead of a user-specified package.
2019-06-27 20:39:11 +02:00
<exe-cmd>yum install pgbackrest</exe-cmd>
<exe-cmd-extra>-y 2>&amp;1</exe-cmd-extra>
</execute>
Documentation may be built with user-specified packages. By default the documentation builds pgBackRest from source, but the documentation is also a good way to smoke-test packages. Allow a package file to be specified by passing --var=package=/path/to/package.ext. This works for Debian and CentOS 6 builds.
2018-12-04 20:17:55 +02:00
</execute-list>
<execute-list if="'{[br-install-user]}' ne 'postgres'" host="{[br-install-host]}">
<title>Update permissions on configuration file and directories</title>
<execute user="root">
<exe-cmd>chown {[br-install-user]}:{[br-install-group]} /var/log/pgbackrest</exe-cmd>
</execute>
<execute user="root">
<exe-cmd>chown {[br-install-user]}:{[br-install-group]} {[backrest-config-demo]}</exe-cmd>
</execute>
</execute-list>
</block-define>
<block-define id="br-install-repo">
<execute-list if="'{[package]}' eq 'none'" host="{[br-install-host]}">
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
<title>Create the <backrest/> repository</title>
<execute user="root">
Document build improvements. * Perform apt-get update to ensure packages are up to date before installing. * Add -p to the repository mkdir so it won't fail if the directory already exists, handy for testing packages.
2018-03-23 22:08:32 +02:00
<exe-cmd>mkdir -p {[backrest-repo-path]}</exe-cmd>
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
</execute>
<execute user="root">
<exe-cmd>chmod 750 {[backrest-repo-path]}</exe-cmd>
</execute>
<execute user="root">
<exe-cmd>chown {[br-install-user]}:{[br-install-group]} {[backrest-repo-path]}</exe-cmd>
</execute>
</execute-list>
Documentation may be built with user-specified packages. By default the documentation builds pgBackRest from source, but the documentation is also a good way to smoke-test packages. Allow a package file to be specified by passing --var=package=/path/to/package.ext. This works for Debian and CentOS 6 builds.
2018-12-04 20:17:55 +02:00
<execute-list if="'{[package]}' ne 'none' &amp;&amp; '{[br-install-user]}' ne 'postgres'" host="{[br-install-host]}">
<title>Update permissions on the <backrest/> repository</title>
<execute user="root">
<exe-cmd>chown {[br-install-user]}:{[br-install-group]} {[backrest-repo-path]}</exe-cmd>
</execute>
</execute-list>
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
</block-define>
Azure support for repository storage. Azure and Azure-compatible object stores can now be used for repository storage. Currently only shared key authentication is supported but SAS will be added soon.
2020-07-02 22:24:34 +02:00
<!-- ======================================================================================================================= -->
<block-define id="azure-setup">
<p><backrest/> supports locating repositories in <proper>Azure-compatible</proper> object stores. The container used to store the repository must be created in advance &amp;mdash; <backrest/> will not do it automatically. The repository can be located in the container root (<path>/</path>) but it's usually best to place it in a subpath so object store logs or other data can also be stored in the container without conflicts.</p>
<backrest-config host="{[azure-setup-host]}" file="{[backrest-config-demo]}" owner="{[azure-setup-config-owner]}">
<title>Configure <proper>Azure</proper></title>
Multiple repository support. Up to four repositories may be configured. A potential benefit is the ability to have a local repository for fast restores and a remote repository for redundancy. Some commands, e.g. stanza-create/stanza-update, will automatically work with all configured repositories while others, e.g. stanza-delete, will require a repository to be specified using the repo option. See the command reference for details on which commands require the repository to be specified. Note that the repo option is not required when only repo1 is configured in order to maintain backward compatibility. However, the repo option is required when a single repo is configured as, e.g. repo2. This is to prevent command breakage if a new repository is added later. The archive-push command will always push WAL to the archive in all configured repositories but backups will need to be scheduled individually for each repository. In many cases this is desirable since backup types and retention will vary by repository. Likewise, restores must specify a repository. It is generally better to specify a repository for restores that has low latency/cost even if that means more recovery time. Only restore testing can determine which repository will be most efficient. For single repository configurations there should be no change in behavior.
2021-03-08 20:31:13 +02:00
<backrest-config-option section="global" key="repo{[azure-setup-repo-id]}-type">azure</backrest-config-option>
<backrest-config-option section="global" key="repo{[azure-setup-repo-id]}-path">/{[azure-repo]}</backrest-config-option>
<backrest-config-option section="global" key="repo{[azure-setup-repo-id]}-azure-account">{[azure-account]}</backrest-config-option>
<backrest-config-option if="'{[azure-key-type]}' ne 'shared'" section="global" key="repo{[azure-setup-repo-id]}-azure-key-type">{[azure-key-type]}</backrest-config-option>
<backrest-config-option section="global" key="repo{[azure-setup-repo-id]}-azure-key">{[azure-key]}</backrest-config-option>
<backrest-config-option section="global" key="repo{[azure-setup-repo-id]}-azure-container">{[azure-container]}</backrest-config-option>
<backrest-config-option if="'{[azure-all]}' ne 'y'" section="global" key="repo{[azure-setup-repo-id]}-retention-full">4</backrest-config-option>
Azure support for repository storage. Azure and Azure-compatible object stores can now be used for repository storage. Currently only shared key authentication is supported but SAS will be added soon.
2020-07-02 22:24:34 +02:00
<backrest-config-option section="global" key="process-max">4</backrest-config-option>
</backrest-config>
<execute-list if="'{[azure-local]}' eq 'y'" host="{[azure-setup-host]}" show="n">
<title>Create the container</title>
<!-- Set host entries to redirect to local azure server -->
<execute user="root" user-force="y" show="n">
Add repo-azure-uri-style option. Azurite introduced a breaking change in 8f63964e to use automatically host-style URIs when the endpoint appears to be a multipart hostname. This option allows the user to configure which style URI will be used, but changing the endpoint might cause breakage if Azurite decides to use a different style. Future changes to Azurite may also cause breakage.
2021-09-27 15:01:53 +02:00
<exe-cmd>echo "{[host-azure-ip]} pgbackrest.blob.core.windows.net" | tee -a /etc/hosts</exe-cmd>
Azure support for repository storage. Azure and Azure-compatible object stores can now be used for repository storage. Currently only shared key authentication is supported but SAS will be added soon.
2020-07-02 22:24:34 +02:00
</execute>
<execute user="{[azure-setup-user]}" if="'{[azure-setup-create-container]}' eq 'y'" show='n'>
Multiple repository support. Up to four repositories may be configured. A potential benefit is the ability to have a local repository for fast restores and a remote repository for redundancy. Some commands, e.g. stanza-create/stanza-update, will automatically work with all configured repositories while others, e.g. stanza-delete, will require a repository to be specified using the repo option. See the command reference for details on which commands require the repository to be specified. Note that the repo option is not required when only repo1 is configured in order to maintain backward compatibility. However, the repo option is required when a single repo is configured as, e.g. repo2. This is to prevent command breakage if a new repository is added later. The archive-push command will always push WAL to the archive in all configured repositories but backups will need to be scheduled individually for each repository. In many cases this is desirable since backup types and retention will vary by repository. Likewise, restores must specify a repository. It is generally better to specify a repository for restores that has low latency/cost even if that means more recovery time. Only restore testing can determine which repository will be most efficient. For single repository configurations there should be no change in behavior.
2021-03-08 20:31:13 +02:00
<exe-cmd>{[project-exe]} --repo={[azure-setup-repo-id]} repo-create</exe-cmd>
Azure support for repository storage. Azure and Azure-compatible object stores can now be used for repository storage. Currently only shared key authentication is supported but SAS will be added soon.
2020-07-02 22:24:34 +02:00
</execute>
</execute-list>
Add shared access signature (SAS) authorization for Azure. A shared access signature (SAS) provides granular, delegated access to resources in a storage account. This is often preferable to using a shared key which provides more access and is a greater security risk if compromised.
2020-07-09 20:46:48 +02:00
Multiple repository support. Up to four repositories may be configured. A potential benefit is the ability to have a local repository for fast restores and a remote repository for redundancy. Some commands, e.g. stanza-create/stanza-update, will automatically work with all configured repositories while others, e.g. stanza-delete, will require a repository to be specified using the repo option. See the command reference for details on which commands require the repository to be specified. Note that the repo option is not required when only repo1 is configured in order to maintain backward compatibility. However, the repo option is required when a single repo is configured as, e.g. repo2. This is to prevent command breakage if a new repository is added later. The archive-push command will always push WAL to the archive in all configured repositories but backups will need to be scheduled individually for each repository. In many cases this is desirable since backup types and retention will vary by repository. Likewise, restores must specify a repository. It is generally better to specify a repository for restores that has low latency/cost even if that means more recovery time. Only restore testing can determine which repository will be most efficient. For single repository configurations there should be no change in behavior.
2021-03-08 20:31:13 +02:00
<p>Shared access signatures may be used by setting the <br-option>repo{[azure-setup-repo-id]}-azure-key-type</br-option> option to <id>sas</id> and the <br-option>repo{[azure-setup-repo-id]}-azure-key</br-option> option to the shared access signature token.</p>
Azure support for repository storage. Azure and Azure-compatible object stores can now be used for repository storage. Currently only shared key authentication is supported but SAS will be added soon.
2020-07-02 22:24:34 +02:00
</block-define>
GCS support for repository storage. GCS and GCS-compatible object stores can now be used for repository storage.
2021-03-05 19:13:51 +02:00
<!-- ======================================================================================================================= -->
<block-define id="gcs-setup">
<p><backrest/> supports locating repositories in <proper>GCS-compatible</proper> object stores. The bucket used to store the repository must be created in advance &amp;mdash; <backrest/> will not do it automatically. The repository can be located in the bucket root (<path>/</path>) but it's usually best to place it in a subpath so object store logs or other data can also be stored in the bucket without conflicts.</p>
<backrest-config host="{[gcs-setup-host]}" file="{[backrest-config-demo]}" owner="{[gcs-setup-config-owner]}">
<title>Configure <proper>GCS</proper></title>
Multiple repository support. Up to four repositories may be configured. A potential benefit is the ability to have a local repository for fast restores and a remote repository for redundancy. Some commands, e.g. stanza-create/stanza-update, will automatically work with all configured repositories while others, e.g. stanza-delete, will require a repository to be specified using the repo option. See the command reference for details on which commands require the repository to be specified. Note that the repo option is not required when only repo1 is configured in order to maintain backward compatibility. However, the repo option is required when a single repo is configured as, e.g. repo2. This is to prevent command breakage if a new repository is added later. The archive-push command will always push WAL to the archive in all configured repositories but backups will need to be scheduled individually for each repository. In many cases this is desirable since backup types and retention will vary by repository. Likewise, restores must specify a repository. It is generally better to specify a repository for restores that has low latency/cost even if that means more recovery time. Only restore testing can determine which repository will be most efficient. For single repository configurations there should be no change in behavior.
2021-03-08 20:31:13 +02:00
<backrest-config-option section="global" key="repo{[gcs-setup-repo-id]}-type">gcs</backrest-config-option>
<backrest-config-option section="global" key="repo{[gcs-setup-repo-id]}-path">/{[gcs-repo]}</backrest-config-option>
<backrest-config-option if="'{[gcs-key-type]}' ne 'service'" section="global" key="repo{[gcs-setup-repo-id]}-gcs-key-type">{[gcs-key-type]}</backrest-config-option>
<backrest-config-option section="global" key="repo{[gcs-setup-repo-id]}-gcs-key">{[gcs-key]}</backrest-config-option>
<backrest-config-option section="global" key="repo{[gcs-setup-repo-id]}-gcs-bucket">{[gcs-bucket]}</backrest-config-option>
GCS support for repository storage. GCS and GCS-compatible object stores can now be used for repository storage.
2021-03-05 19:13:51 +02:00
<backrest-config-option section="global" key="process-max">4</backrest-config-option>
</backrest-config>
Add automatic GCS authentication for GCE instances. When running on a GCE instance the authentication token can be pulled directly from the instance metadata. This is configured with repo-gcs-key-type=auto. In a separate commit (26fefa6), move the code that parses the token response into a separate function, storageGcsAuthToken(), since it is now needed by two key types. This drastically improves the readability of the main commit.
2021-05-17 20:55:50 +02:00
<p>When running in <proper>GCE</proper> set <br-option>repo{[gcs-setup-repo-id]}-gcs-key-type=auto</br-option> to automatically authenticate using the instance service account.</p>
GCS support for repository storage. GCS and GCS-compatible object stores can now be used for repository storage.
2021-03-05 19:13:51 +02:00
</block-define>
Build all docs with S3 using --var=s3-all=y Force repo-type=s3 for all tests. This is not currently the default for any OS builds.
2019-05-29 14:38:45 +02:00
<!-- ======================================================================================================================= -->
<block-define id="s3-setup">
<p><backrest/> supports locating repositories in <proper>S3-compatible</proper> object stores. The bucket used to store the repository must be created in advance &amp;mdash; <backrest/> will not do it automatically. The repository can be located in the bucket root (<path>/</path>) but it's usually best to place it in a subpath so object store logs or other data can also be stored in the bucket without conflicts.</p>
<backrest-config host="{[s3-setup-host]}" file="{[backrest-config-demo]}" owner="{[s3-setup-config-owner]}">
<title>Configure <proper>S3</proper></title>
Multiple repository support. Up to four repositories may be configured. A potential benefit is the ability to have a local repository for fast restores and a remote repository for redundancy. Some commands, e.g. stanza-create/stanza-update, will automatically work with all configured repositories while others, e.g. stanza-delete, will require a repository to be specified using the repo option. See the command reference for details on which commands require the repository to be specified. Note that the repo option is not required when only repo1 is configured in order to maintain backward compatibility. However, the repo option is required when a single repo is configured as, e.g. repo2. This is to prevent command breakage if a new repository is added later. The archive-push command will always push WAL to the archive in all configured repositories but backups will need to be scheduled individually for each repository. In many cases this is desirable since backup types and retention will vary by repository. Likewise, restores must specify a repository. It is generally better to specify a repository for restores that has low latency/cost even if that means more recovery time. Only restore testing can determine which repository will be most efficient. For single repository configurations there should be no change in behavior.
2021-03-08 20:31:13 +02:00
<backrest-config-option section="global" key="repo{[s3-setup-repo-id]}-type">s3</backrest-config-option>
<backrest-config-option section="global" key="repo{[s3-setup-repo-id]}-path">/{[s3-repo]}</backrest-config-option>
<backrest-config-option section="global" key="repo{[s3-setup-repo-id]}-s3-key">{[s3-key]}</backrest-config-option>
<backrest-config-option section="global" key="repo{[s3-setup-repo-id]}-s3-key-secret">{[s3-key-secret]}</backrest-config-option>
<backrest-config-option section="global" key="repo{[s3-setup-repo-id]}-s3-bucket">{[s3-bucket]}</backrest-config-option>
<backrest-config-option section="global" key="repo{[s3-setup-repo-id]}-s3-endpoint">{[s3-endpoint]}</backrest-config-option>
<backrest-config-option section="global" key="repo{[s3-setup-repo-id]}-s3-region">{[s3-region]}</backrest-config-option>
<backrest-config-option if="'{[s3-all]}' ne 'y'" section="global" key="repo{[s3-setup-repo-id]}-retention-full">4</backrest-config-option>
Build all docs with S3 using --var=s3-all=y Force repo-type=s3 for all tests. This is not currently the default for any OS builds.
2019-05-29 14:38:45 +02:00
<backrest-config-option section="global" key="process-max">4</backrest-config-option>
</backrest-config>
Add repo-create, repo-get, repo-put, and repo-rm commands. These commands are generally useful but more importantly they allow removing LibC by providing the Perl integration tests an alternate way to work with repository storage. All the commands are currently internal only and should not be used on production repositories.
2020-03-09 23:15:03 +02:00
<execute-list if="'{[s3-local]}' eq 'y'" host="{[s3-setup-host]}" show="n">
<title>Create the bucket</title>
<!-- Set host entries to redirect AWS to local s3 server -->
<execute user="root" user-force="y" show="n">
<exe-cmd>echo "{[host-s3-ip]} {[s3-bucket]}.{[s3-endpoint]} {[s3-endpoint]}" | tee -a /etc/hosts</exe-cmd>
</execute>
<execute user="{[s3-setup-user]}" if="'{[s3-setup-create-bucket]}' eq 'y'" show='n'>
Multiple repository support. Up to four repositories may be configured. A potential benefit is the ability to have a local repository for fast restores and a remote repository for redundancy. Some commands, e.g. stanza-create/stanza-update, will automatically work with all configured repositories while others, e.g. stanza-delete, will require a repository to be specified using the repo option. See the command reference for details on which commands require the repository to be specified. Note that the repo option is not required when only repo1 is configured in order to maintain backward compatibility. However, the repo option is required when a single repo is configured as, e.g. repo2. This is to prevent command breakage if a new repository is added later. The archive-push command will always push WAL to the archive in all configured repositories but backups will need to be scheduled individually for each repository. In many cases this is desirable since backup types and retention will vary by repository. Likewise, restores must specify a repository. It is generally better to specify a repository for restores that has low latency/cost even if that means more recovery time. Only restore testing can determine which repository will be most efficient. For single repository configurations there should be no change in behavior.
2021-03-08 20:31:13 +02:00
<exe-cmd>{[project-exe]} --repo={[s3-setup-repo-id]} repo-create</exe-cmd>
Add repo-create, repo-get, repo-put, and repo-rm commands. These commands are generally useful but more importantly they allow removing LibC by providing the Perl integration tests an alternate way to work with repository storage. All the commands are currently internal only and should not be used on production repositories.
2020-03-09 23:15:03 +02:00
</execute>
</execute-list>
Build all docs with S3 using --var=s3-all=y Force repo-type=s3 for all tests. This is not currently the default for any OS builds.
2019-05-29 14:38:45 +02:00
<admonition type="note">The region and endpoint will need to be configured to where the bucket is located. The values given here are for the <id>{[s3-region]}</id> region.</admonition>
</block-define>
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
<!-- SECTION => INTRODUCTION -->
<section id="introduction">
<title>Introduction</title>
Azure support for repository storage. Azure and Azure-compatible object stores can now be used for repository storage. Currently only shared key authentication is supported but SAS will be added soon.
2020-07-02 22:24:34 +02:00
<!-- Create Azure server first to allow it time to boot before being used -->
Allow S3/Azure Docker images to be specified in user guide. It is not uncommon for the S3/Azure emulators we use to introduce breaking changes without warning. If that happens the documentation can still be built by specifying a working version of the image. In general, it is better to let the version float so we know when things break. Azurite has yet another breaking change coming up (see 096829b3, c38d6926, and Azurite issue 1039) so set azure-image at the current version until the breaking change has been released.
2021-10-20 14:22:37 +02:00
<host-add if="'{[azure-local]}' eq 'y'" id="{[host-azure-id]}" name="{[host-azure]}" user="root" image="{[azure-image]}" os="{[os-type]}" option="-m 128m -v {[fake-cert-path]}/azure-server.crt:/root/public.crt:ro -v {[fake-cert-path]}/azure-server.key:/root/private.key:ro -e AZURITE_ACCOUNTS='{[azure-account]}:{[azure-key]}'" param="azurite-blob --blobPort 443 --blobHost 0.0.0.0 --cert=/root/public.crt --key=/root/private.key" update-hosts="n"/>
Azure support for repository storage. Azure and Azure-compatible object stores can now be used for repository storage. Currently only shared key authentication is supported but SAS will be added soon.
2020-07-02 22:24:34 +02:00
Use minio as local S3 emulator in documentation. The documentation was relying on a ScalityS3 container built for testing which wasn't very transparent. Instead, use the stock minio container and configure it in the documentation. Also, install certificates and CA so that TLS verification can be enabled.
2019-05-27 13:37:20 +02:00
<!-- Create S3 server first to allow it time to boot before being used -->
Allow S3/Azure Docker images to be specified in user guide. It is not uncommon for the S3/Azure emulators we use to introduce breaking changes without warning. If that happens the documentation can still be built by specifying a working version of the image. In general, it is better to let the version float so we know when things break. Azurite has yet another breaking change coming up (see 096829b3, c38d6926, and Azurite issue 1039) so set azure-image at the current version until the breaking change has been released.
2021-10-20 14:22:37 +02:00
<host-add if="'{[s3-local]}' eq 'y'" id="{[host-s3-id]}" name="{[host-s3]}" user="root" image="{[s3-image]}" os="{[os-type]}" option="-m 128m -v {[fake-cert-path]}/s3-server.crt:/root/.minio/certs/public.crt:ro -v {[fake-cert-path]}/s3-server.key:/root/.minio/certs/private.key:ro -e MINIO_REGION={[s3-region]} -e MINIO_DOMAIN={[s3-endpoint]} -e MINIO_BROWSER=off -e MINIO_ACCESS_KEY={[s3-key]} -e MINIO_SECRET_KEY={[s3-key-secret]}" param="server /data --address :443" update-hosts="n"/>
Use minio as local S3 emulator in documentation. The documentation was relying on a ScalityS3 container built for testing which wasn't very transparent. Instead, use the stock minio container and configure it in the documentation. Also, install certificates and CA so that TLS verification can be enabled.
2019-05-27 13:37:20 +02:00
Rearrange backup documentation in user guide. Remove the "Automatic Stop Option" section since it only applies to PostgreSQL <= 9.6, which will soon be EOL. Since we no longer build the user guide for PostgreSQL < 10 this section was no longer being tested. The stop-auto option is still documented in the reference. Move the "Fast Start Option" to "Quick Start - Perform Backup". This is a commonly-used option so it makes sense to mention it earlier. This also makes the backups run more quickly. In the worst case, backups in "Quick Start - Perform Backup" could take minutes to start Move the "Archive Timeout" section to "Quick Start - Perform Backup" since it is the last section in "Backup".
2021-08-11 18:59:25 +02:00
<p>This user guide is intended to be followed sequentially from beginning to end &amp;mdash; each section depends on the last. For example, the <link section="/restore">Restore</link> section relies on setup that is performed in the <link section="/quickstart">Quick Start</link> section. Once <backrest/> is up and running then skipping around is possible but following the user guide in order is recommended the first time through.</p>
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
Add missing paragraph tag in user guide.
2021-10-18 20:42:49 +02:00
<p>Although the examples are targeted at {[user-guide-os]} and <postgres/> {[pg-version-min]}-{[pg-version-max]}, it should be fairly easy to apply this guide to any Unix distribution and <postgres/> version. The only OS-specific commands are those to create, start, stop, and drop <postgres/> clusters. The <backrest/> commands will be the same on any Unix system though the location to install the executable may vary.</p>
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
Add missing paragraph tag in user guide.
2021-10-18 20:42:49 +02:00
<p>Configuration information and documentation for PostgreSQL can be found in the <postgres/> <link url='http://www.postgresql.org/docs/{[pg-version]}/static/index.html'>Manual</link>.</p>
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
<p>A somewhat novel approach is taken to documentation in this user guide. Each command is run on a virtual machine when the documentation is built from the XML source. This means you can have a high confidence that the commands work correctly in the order presented. Output is captured and displayed below the command when appropriate. If the output is not included it is because it was deemed not relevant or was considered a distraction from the narrative.</p>
<p>All commands are intended to be run as an unprivileged user that has sudo privileges for both the <user>root</user> and <user>postgres</user> users. It's also possible to run the commands directly as their respective users without modification and in that case the <cmd>sudo</cmd> commands can be stripped off.</p>
</section>
<!-- SECTION => CONCEPTS -->
<section id="concept">
<title>Concepts</title>
<p>The following concepts are defined as they are relevant to <backrest/>, <postgres/>, and this user guide.</p>
<!-- SECTION => CONCEPTS - BACKUP -->
<section id="backup">
<title>Backup</title>
<p>A backup is a consistent copy of a database cluster that can be restored to recover from a hardware failure, to perform Point-In-Time Recovery, or to bring up a new standby.</p>
Update backup host to repository host in the documentation. Contributed by Cynthia Shang.
2018-02-19 20:17:58 +02:00
<p><b>Full Backup</b>: <backrest/> copies the entire contents of the database cluster to the backup. The first backup of the database cluster is always a Full Backup. <backrest/> is always able to restore a full backup directly. The full backup does not depend on any files outside of the full backup for consistency.</p>
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
<p><b>Differential Backup</b>: <backrest/> copies only those database cluster files that have changed since the last full backup. <backrest/> restores a differential backup by copying all of the files in the chosen differential backup and the appropriate unchanged files from the previous full backup. The advantage of a differential backup is that it requires less disk space than a full backup, however, the differential backup and the full backup must both be valid to restore the differential backup.</p>
<p><b>Incremental Backup</b>: <backrest/> copies only those database cluster files that have changed since the last backup (which can be another incremental backup, a differential backup, or a full backup). As an incremental backup only includes those files changed since the prior backup, they are generally much smaller than full or differential backups. As with the differential backup, the incremental backup depends on other backups to be valid to restore the incremental backup. Since the incremental backup includes only those files since the last backup, all prior incremental backups back to the prior differential, the prior differential backup, and the prior full backup must all be valid to perform a restore of the incremental backup. If no differential backup exists then all prior incremental backups back to the prior full backup, which must exist, and the full backup itself must be valid to restore the incremental backup.</p>
</section>
<!-- SECTION => CONCEPTS - RESTORE -->
<section id="restore">
<title>Restore</title>
<p>A restore is the act of copying a backup to a system where it will be started as a live database cluster. A restore requires the backup files and one or more WAL segments in order to work correctly.</p>
</section>
<!-- SECTION => CONCEPTS - WAL -->
<section id="wal">
<title>Write Ahead Log (WAL)</title>
<p>WAL is the mechanism that <postgres/> uses to ensure that no committed changes are lost. Transactions are written sequentially to the WAL and a transaction is considered to be committed when those writes are flushed to disk. Afterwards, a background process writes the changes into the main database cluster files (also known as the heap). In the event of a crash, the WAL is replayed to make the database consistent.</p>
<p>WAL is conceptually infinite but in practice is broken up into individual 16MB files called segments. WAL segments follow the naming convention <id>0000000100000A1E000000FE</id> where the first 8 hexadecimal digits represent the timeline and the next 16 digits are the logical sequence number (LSN).</p>
</section>
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
<!-- SECTION => CONCEPTS - ENCRYPTION -->
<section id="encryption">
<title>Encryption</title>
<p>Encryption is the process of converting data into a format that is unrecognizable unless the appropriate password (also referred to as passphrase) is provided.</p>
<p><backrest/> will encrypt the repository based on a user-provided password, thereby preventing unauthorized access to data stored within the repository.</p>
</section>
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
</section>
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<!-- SECTION => UPGRADING -->
<section id="upgrading">
Update backup host to repository host in the documentation. Contributed by Cynthia Shang.
2018-02-19 20:17:58 +02:00
<title>Upgrading {[project]}</title>
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<section id="v1-v2">
Update backup host to repository host in the documentation. Contributed by Cynthia Shang.
2018-02-19 20:17:58 +02:00
<title>Upgrading {[project]} from v1 to v2</title>
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
Update how versions are displayed in the documentation.
2018-02-24 06:42:29 +02:00
<p>Upgrading from <proper>v1</proper> to <proper>v2</proper> is fairly straight-forward. The repository format has not changed and all non-deprecated options from <proper>v1</proper> are accepted, so for most installations it is simply a matter of installing the new version.</p>
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<p>However, there are a few caveats:</p>
<list>
<list-item>The deprecated <br-option>thread-max</br-option> option is no longer valid. Use <br-option>process-max</br-option> instead.</list-item>
Rename archive-queue-max option to archive-push-queue-max. This avoids confusion with the archive-get-queue-max option. The old option name will continue to be accepted.
2018-04-19 01:37:35 +02:00
<list-item>The deprecated <br-option>archive-max-mb</br-option> option is no longer valid. This has been replaced with the <br-option>archive-push-queue-max</br-option> option which has different semantics.</list-item>
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<list-item>The default for the <br-option>backup-user</br-option> option has changed from <id>backrest</id> to <id>pgbackrest</id>.</list-item>
Add support for additional pgBackRest configuration files. Configuration files are loaded from the directory specified by the --config-include-path option. Add --config-path option for overriding the default base path of the --config and --config-include-path option. Contributed by Cynthia Shang.
2018-04-19 17:09:39 +02:00
<list-item>In <proper>v2.02</proper> the default location of the <backrest/> configuration file has changed from <file>/etc/pgbackrest.conf</file> to <file>/etc/pgbackrest/pgbackrest.conf</file>. If <file>/etc/pgbackrest/pgbackrest.conf</file> does not exist, the <file>/etc/pgbackrest.conf</file> file will be loaded instead, if it exists.</list-item>
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
</list>
Update how versions are displayed in the documentation.
2018-02-24 06:42:29 +02:00
<p>Many option names have changed to improve consistency although the old names from <proper>v1</proper> are still accepted. In general, <id>db-*</id> options have been renamed to <id>pg-*</id> and <id>backup-*</id>/<id>retention-*</id> options have been renamed to <id>repo-*</id> when appropriate.</p>
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
Update how versions are displayed in the documentation.
2018-02-24 06:42:29 +02:00
<p><postgres/> and repository options must be indexed when using the new names introduced in <proper>v2</proper>, e.g. <br-option>pg1-host</br-option>, <br-option>pg1-path</br-option>, <br-option>repo1-path</br-option>, <br-option>repo1-type</br-option>, etc. Only one repository is allowed currently but more flexibility is planned for <proper>v2</proper>.</p>
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
</section>
</section>
Create a new section to describe building pgBackRest and build on a separate host.
2018-06-13 19:43:17 +02:00
<!-- ======================================================================================================================= -->
Documentation may be built with user-specified packages. By default the documentation builds pgBackRest from source, but the documentation is also a good way to smoke-test packages. Allow a package file to be specified by passing --var=package=/path/to/package.ext. This works for Debian and CentOS 6 builds.
2018-12-04 20:17:55 +02:00
<section if="'{[package]}' eq 'none'" id="build">
Create a new section to describe building pgBackRest and build on a separate host.
2018-06-13 19:43:17 +02:00
<title>Build</title>
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
Create common if expressions for testing os-type. These expressions simplify os-type testing. This will be especially true as more OS types are added.
2018-12-05 00:33:56 +02:00
<p if="{[os-type-is-debian]}">{[user-guide-os]} packages for <backrest/> are available at <link url="https://www.postgresql.org/download/linux/ubuntu/">apt.postgresql.org</link>. If they are not provided for your distribution/version it is easy to download the source and install manually.</p>
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<p if="{[os-type-is-rhel]}">{[user-guide-os]} packages for <backrest/> are available from <link url="{[crunchy-url-base]}">Crunchy Data</link> or <link url="http://yum.postgresql.org">yum.postgresql.org</link>, but it is also easy to download the source and install manually.</p>
Create a new section to describe building pgBackRest and build on a separate host.
2018-06-13 19:43:17 +02:00
Add optional stress test section to the user guide. This is intended to provide pre-release stress-testing. Include container memory limits to help check for memory leaks. Also add parallelism to make for faster builds.
2021-08-12 13:49:59 +02:00
<host-add id="{[host-build-id]}" name="{[host-build]}" user="{[host-build-user]}" image="{[host-build-image]}" os="{[os-type]}" mount="{[host-build-mount]}" option="-m 256m {[host-option]}"/>
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
Create a new section to describe building pgBackRest and build on a separate host.
2018-06-13 19:43:17 +02:00
<p>When building from source it is best to use a build host rather than building on production. Many of the tools required for the build should generally not be installed in production. <backrest/> consists of a single executable so it is easy to copy to a new host once it is built.</p>
Update docs with 32-bit support and caveats. 32-bit support was added in v1.26. Reported by Viorel Tabara.
2018-04-30 14:13:56 +02:00
Create a new section to describe building pgBackRest and build on a separate host.
2018-06-13 19:43:17 +02:00
<execute-list host="{[host-build]}">
Add variables for build paths in the user guide. This eliminates repetition of the build path so it can be changed more easily. Also create the build path explicitly rather than suggest that the user do it.
2021-08-18 16:52:35 +02:00
<title>Download version <id>{[version]}</id> of <backrest/> to <path>{[build-path]}</path> path</title>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
Create a new section to describe building pgBackRest and build on a separate host.
2018-06-13 19:43:17 +02:00
<!-- This is shown to the user but never actually run for the very good reason that the release is not available before the documentation is built -->
Fix sudo missed in "Build pgBackRest as an unprivileged user". 286a106a updated the documentation to build pgBackRest as an unprivileged user, but the wget command was missed. This command is not actually run, just displayed, because the release is not yet available when the documentation is built. Update the wget command to run as the local user.
2019-09-04 00:28:53 +02:00
<execute skip="y">
Add variables for build paths in the user guide. This eliminates repetition of the build path so it can be changed more easily. Also create the build path explicitly rather than suggest that the user do it.
2021-08-18 16:52:35 +02:00
<exe-cmd>mkdir -p {[build-path]}</exe-cmd>
</execute>
<execute skip="y">
<exe-cmd>
wget -q -O -
Create a new section to describe building pgBackRest and build on a separate host.
2018-06-13 19:43:17 +02:00
{[github-url-release]}/{[version]}.tar.gz |
Add variables for build paths in the user guide. This eliminates repetition of the build path so it can be changed more easily. Also create the build path explicitly rather than suggest that the user do it.
2021-08-18 16:52:35 +02:00
tar zx -C {[build-path]}
</exe-cmd>
Create a new section to describe building pgBackRest and build on a separate host.
2018-06-13 19:43:17 +02:00
</execute>
<!-- These commands simulate what the command above would do if it could be run -->
<execute user="root" show="n">
Add variables for build paths in the user guide. This eliminates repetition of the build path so it can be changed more easily. Also create the build path explicitly rather than suggest that the user do it.
2021-08-18 16:52:35 +02:00
<exe-cmd>mkdir -p {[build-br-path]}</exe-cmd>
Create a new section to describe building pgBackRest and build on a separate host.
2018-06-13 19:43:17 +02:00
</execute>
<execute user="root" show="n">
Add variables for build paths in the user guide. This eliminates repetition of the build path so it can be changed more easily. Also create the build path explicitly rather than suggest that the user do it.
2021-08-18 16:52:35 +02:00
<exe-cmd>cp -r {[pgbackrest-repo-path]}/src {[build-br-path]}</exe-cmd>
Build pgBackRest as an unprivileged user. pgBackRest was being built by root in the documentation which is definitely not best practice. Instead build as the unprivileged default container user. Sudo privileges are still required to install. Suggested by Laurenz Albe.
2019-08-20 15:46:29 +02:00
</execute>
<execute user="root" show="n">
Add variables for build paths in the user guide. This eliminates repetition of the build path so it can be changed more easily. Also create the build path explicitly rather than suggest that the user do it.
2021-08-18 16:52:35 +02:00
<exe-cmd>chown -R {[host-build-user]} {[build-br-path]}</exe-cmd>
Create a new section to describe building pgBackRest and build on a separate host.
2018-06-13 19:43:17 +02:00
</execute>
</execute-list>
<execute-list host="{[host-build]}">
<title>Install build dependencies</title>
Create common if expressions for testing os-type. These expressions simplify os-type testing. This will be especially true as more OS types are added.
2018-12-05 00:33:56 +02:00
<execute if="{[os-type-is-debian]}" user="root" show="n" pre="y">
v2.05: Environment Variable Options and Exclude Temporary/Unlogged Relations Bug Fixes: * Fix issue where relative links in $PGDATA could be stored in the backup with the wrong path. This issue did not affect absolute links and relative tablespace links were caught by other checks. (Reported by Cynthia Shang.) * Remove incompletely implemented online option from the check command. Offline operation runs counter to the purpose of this command, which is to check if archiving and backups are working correctly. (Reported by Jason O'Donnell.) * Fix issue where errors raised in C were not logged when called from Perl. pgBackRest properly terminated with the correct error code but lacked an error message to aid in debugging. (Reported by Douglas J Hunley.) * Fix issue when a boolean option (e.g. delta) was specified more than once. (Reported by Yogesh Sharma.) Features: * Allow any option to be set in an environment variable. This includes options that previously could only be specified on the command line, e.g. stanza, and secret options that could not be specified on the command-line, e.g. repo1-s3-key-secret. * Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. (Contributed by Cynthia Shang.) * Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using. (Reviewed by Cynthia Shang.) * Add log-subprocess option to allow file logging for local and remote subprocesses. * PostgreSQL 11 Beta 3 support. Improvements: * Allow zero-size files in backup manifest to reference a prior manifest regardless of timestamp delta. (Contributed by Cynthia Shang.) * Improve asynchronous archive-get/archive-push performance by directly checking status files. (Contributed by Stephen Frost.) * Improve error message when a command is missing the stanza option. (Suggested by Sarah Conway.)
2018-08-31 19:19:43 +02:00
<exe-cmd>apt-get update</exe-cmd>
</execute>
Create common if expressions for testing os-type. These expressions simplify os-type testing. This will be especially true as more OS types are added.
2018-12-05 00:33:56 +02:00
<execute if="{[os-type-is-debian]}" user="root" pre="y">
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 20:50:02 +02:00
<exe-cmd>
Add Zstandard compression support. Zstandard is a fast lossless compression algorithm targeting real-time compression scenarios at zlib-level and better compression ratios. It's backed by a very fast entropy stage, provided by Huff0 and FSE library. Zstandard version >= 1.0 is required, which is generally only available on newer distributions.
2020-05-04 21:25:27 +02:00
apt-get install make gcc libpq-dev libssl-dev libxml2-dev pkg-config
Partial migration of config code generation to C. Parse enough of config.yaml to auto-generate config.auto.h and config.auto.c. This commit implements most of the infrastructure needed to migrate the rest of the build code to C, but each set of auto-generated files will present its own challenges. The build is now dependent on libyaml. At this point there is no need for a hard requirement, but that will come soon so it seems better to add the dependency now.
2021-07-19 01:02:01 +02:00
liblz4-dev libzstd-dev libbz2-dev libz-dev libyaml-dev
Add Zstandard compression support. Zstandard is a fast lossless compression algorithm targeting real-time compression scenarios at zlib-level and better compression ratios. It's backed by a very fast entropy stage, provided by Huff0 and FSE library. Zstandard version >= 1.0 is required, which is generally only available on newer distributions.
2020-05-04 21:25:27 +02:00
</exe-cmd>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<exe-cmd-extra>-y 2>&amp;1</exe-cmd-extra>
</execute>
Add Centos/RHEL 7 option to documentation build. Centos/RHEL 7 builds can be specified with --var=os-type=centos7. Also added to the documentation release and CI builds.
2018-12-05 14:12:28 +02:00
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<execute if="{[os-type-is-rhel]}" user="root" pre="y">
Add Centos/RHEL 7 option to documentation build. Centos/RHEL 7 builds can be specified with --var=os-type=centos7. Also added to the documentation release and CI builds.
2018-12-05 14:12:28 +02:00
<exe-cmd>
Add RHEL/CentOS 8 documentation. Update RHEL/CentOS 7 to cover the versions that were previously covered by RHEL/CentOS 6. Since RHEL/CentOS 7/8 work the same update the documentation logic and labels to reflect this compatibility.
2020-12-04 17:59:57 +02:00
yum install make gcc postgresql{[pg-version-nodot]}-devel
Partial migration of config code generation to C. Parse enough of config.yaml to auto-generate config.auto.h and config.auto.c. This commit implements most of the infrastructure needed to migrate the rest of the build code to C, but each set of auto-generated files will present its own challenges. The build is now dependent on libyaml. At this point there is no need for a hard requirement, but that will come soon so it seems better to add the dependency now.
2021-07-19 01:02:01 +02:00
openssl-devel libxml2-devel lz4-devel libzstd-devel bzip2-devel libyaml-devel
Add Centos/RHEL 7 option to documentation build. Centos/RHEL 7 builds can be specified with --var=os-type=centos7. Also added to the documentation release and CI builds.
2018-12-05 14:12:28 +02:00
</exe-cmd>
<exe-cmd-extra>-y 2>&amp;1</exe-cmd-extra>
</execute>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
</execute-list>
Create a new section to describe building pgBackRest and build on a separate host.
2018-06-13 19:43:17 +02:00
<execute-list host="{[host-build]}">
Remove -s from make instructions. It's a good option but not required so let the user make the choice. Also combine configure and make into a single line.
2020-01-15 22:40:17 +02:00
<title>Configure and compile <backrest/></title>
Create a new section to describe building pgBackRest and build on a separate host.
2018-06-13 19:43:17 +02:00
Build pgBackRest as an unprivileged user. pgBackRest was being built by root in the documentation which is definitely not best practice. Instead build as the unprivileged default container user. Sudo privileges are still required to install. Suggested by Laurenz Albe.
2019-08-20 15:46:29 +02:00
<execute>
Add variables for build paths in the user guide. This eliminates repetition of the build path so it can be changed more easily. Also create the build path explicitly rather than suggest that the user do it.
2021-08-18 16:52:35 +02:00
<exe-cmd>cd {[build-br-path]}/src &amp;&amp; ./configure &amp;&amp; make</exe-cmd>
Add optional stress test section to the user guide. This is intended to provide pre-release stress-testing. Include container memory limits to help check for memory leaks. Also add parallelism to make for faster builds.
2021-08-12 13:49:59 +02:00
<exe-cmd-extra>-j 4</exe-cmd-extra>
Create a new section to describe building pgBackRest and build on a separate host.
2018-06-13 19:43:17 +02:00
</execute>
</execute-list>
</section>
<!-- SECTION => INSTALLATION -->
<section id="installation">
<title>Installation</title>
Fix incorrect host names in user guide. pg1 was incorrectly used instead of {[host-pg1]} which meant the wrong host name was displayed. Also, the install block was installing packages to the build host no matter which host was specified.
2021-08-03 20:48:22 +02:00
<p>A new host named <host>{[host-pg1]}</host> is created to contain the demo cluster and run <backrest/> examples.</p>
Create a new section to describe building pgBackRest and build on a separate host.
2018-06-13 19:43:17 +02:00
Add optional stress test section to the user guide. This is intended to provide pre-release stress-testing. Include container memory limits to help check for memory leaks. Also add parallelism to make for faster builds.
2021-08-12 13:49:59 +02:00
<host-add id="{[host-pg1-id]}" name="{[host-pg1]}" user="{[host-pg1-user]}" image="{[host-pg1-image]}" os="{[os-type]}" mount="{[host-pg1-mount]}" option="-m 512m {[host-option]}"/>
Create a new section to describe building pgBackRest and build on a separate host.
2018-06-13 19:43:17 +02:00
Comment out unprivileged user documentation. We haven't had the time to complete this documentation and it has suffered bit rot. This prevents us from building the docs on PostgreSQL >= 11 so just comment it all out until it can be updated.
2019-09-26 22:02:39 +02:00
<!-- <execute-list if="{[pg-version]} >= 11" host="{[host-pg1]}">
Preliminary documentation for PostgreSQL 11 unprivileged user backup.
2018-06-15 01:05:35 +02:00
<title>Create <user>{[br-user]}</user> user</title>
Create common if expressions for testing os-type. These expressions simplify os-type testing. This will be especially true as more OS types are added.
2018-12-05 00:33:56 +02:00
<execute if="{[os-type-is-debian]}" user="root">
Comment out unprivileged user documentation. We haven't had the time to complete this documentation and it has suffered bit rot. This prevents us from building the docs on PostgreSQL >= 11 so just comment it all out until it can be updated.
2019-09-26 22:02:39 +02:00
<exe-cmd>adduser {[dash]}-ingroup {[pg-group]} {[dash]}-disabled-password {[dash]}-gecos "" {[br-user]}</exe-cmd>
Preliminary documentation for PostgreSQL 11 unprivileged user backup.
2018-06-15 01:05:35 +02:00
</execute>
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<execute if="{[os-type-is-rhel]}" user="root">
Preliminary documentation for PostgreSQL 11 unprivileged user backup.
2018-06-15 01:05:35 +02:00
<exe-cmd>adduser -g{[pg-group]} -n {[br-user]}</exe-cmd>
</execute>
Comment out unprivileged user documentation. We haven't had the time to complete this documentation and it has suffered bit rot. This prevents us from building the docs on PostgreSQL >= 11 so just comment it all out until it can be updated.
2019-09-26 22:02:39 +02:00
</execute-list> -->
Preliminary documentation for PostgreSQL 11 unprivileged user backup.
2018-06-15 01:05:35 +02:00
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
<block id="br-install">
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<block-variable-replace key="br-install-host">{[host-pg1]}</block-variable-replace>
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
<block-variable-replace key="br-install-user">postgres</block-variable-replace>
<block-variable-replace key="br-install-group">postgres</block-variable-replace>
</block>
Configuration rules are now pulled from the C library when present.
2017-08-25 22:47:47 +02:00
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
<p><backrest/> should now be properly installed but it is best to check. If any dependencies were missed then you will get an error when running <backrest/> from the command line.</p>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
<title>Make sure the installation worked</title>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y" filter="n">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-cmd>{[project-exe]}</exe-cmd>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
</execute>
</execute-list>
</section>
<!-- SECTION => QUICKSTART -->
Improved support for --require option and section depends now default to the previous section.
2017-02-11 17:26:54 +02:00
<section id="quickstart">
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
<title>Quick Start</title>
<p>The Quick Start section will cover basic configuration of <backrest/> and <postgres/> and introduce the <cmd>backup</cmd>, <cmd>restore</cmd>, and <cmd>info</cmd> commands.</p>
<!-- SECTION => QUICKSTART - SETUP DEMO CLUSTER -->
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
<section id="setup-demo-cluster">
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
<title>Setup Demo Cluster</title>
User Guide clarifications.
2016-05-03 22:12:44 +02:00
<p>Creating the demo cluster is optional but is strongly recommended, especially for new users, since the example commands in the user guide reference the demo cluster; the examples assume the demo cluster is running on the default port (i.e. 5432). The cluster will not be started until a later section because there is still some configuration to do.</p>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Create the demo cluster</title>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
All clusters in the documentation are initialized with checksums.
2017-12-14 22:44:06 +02:00
<execute user="postgres">
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
<exe-cmd>
All clusters in the documentation are initialized with checksums.
2017-12-14 22:44:06 +02:00
{[pg-bin-path]}/initdb
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
-D {[pg-path]} -k -A peer</exe-cmd>
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
</execute>
All clusters in the documentation are initialized with checksums.
2017-12-14 22:44:06 +02:00
Create common if expressions for testing os-type. These expressions simplify os-type testing. This will be especially true as more OS types are added.
2018-12-05 00:33:56 +02:00
<execute if="{[os-type-is-debian]}" user="root" output="y" filter="n">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<exe-cmd>{[pg-cluster-create]}</exe-cmd>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
</execute>
Add optional stress test section to the user guide. This is intended to provide pre-release stress-testing. Include container memory limits to help check for memory leaks. Also add parallelism to make for faster builds.
2021-08-12 13:49:59 +02:00
<execute user="root" show="n" user-force="y">
<exe-cmd>echo 'shared_buffers = 16MB' >> {[postgres-config-demo]}</exe-cmd>
</execute>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
</execute-list>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<p>By default <postgres/> will only accept local connections. The examples in this guide will require connections from other servers so <pg-option>listen_addresses</pg-option> is configured to listen on all interfaces. This may not be appropriate for secure installations.</p>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<postgres-config host="{[host-pg1]}" file="{[postgres-config-demo]}">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Set <pg-option>listen_addresses</pg-option></title>
<postgres-config-option key="listen_addresses">'*'</postgres-config-option>
</postgres-config>
<p>For demonstration purposes the <pg-option>log_line_prefix</pg-option> setting will be minimally configured. This keeps the log output as brief as possible to better illustrate important information.</p>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<postgres-config host="{[host-pg1]}" file="{[postgres-config-demo]}">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Set <pg-option>log_line_prefix</pg-option></title>
<postgres-config-option key="log_line_prefix">''</postgres-config-option>
</postgres-config>
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<p if="{[os-type-is-rhel]}">By default {[user-guide-os]} includes the day of the week in the log filename. This makes automating the user guide a bit more complicated so the <pg-option>log_filename</pg-option> is set to a constant.</p>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<postgres-config host="{[host-pg1]}" if="{[os-type-is-rhel]}" file="{[postgres-config-demo]}">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Set <pg-option>log_filename</pg-option></title>
<postgres-config-option key="log_filename">'postgresql.log'</postgres-config-option>
</postgres-config>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
</section>
<!-- SECTION => QUICKSTART - CONFIGURE STANZA -->
Improved support for --require option and section depends now default to the previous section.
2017-02-11 17:26:54 +02:00
<section id="configure-stanza">
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
<title>Configure Cluster Stanza</title>
<option-description key="stanza"/>
User Guide clarifications.
2016-05-03 22:12:44 +02:00
<p>The name 'demo' describes the purpose of this cluster accurately so that will also make a good stanza name.</p>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<p><backrest/> needs to know where the base data directory for the <postgres/> cluster is located. The path can be requested from <postgres/> directly but in a recovery scenario the <postgres/> process will not be available. During backups the value supplied to <backrest/> will be compared against the path that <postgres/> is running on and they must be equal or the backup will return an error. Make sure that <br-option>pg-path</br-option> is exactly equal to <pg-option>data_directory</pg-option> in <file>postgresql.conf</file>.</p>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<p>By default {[user-guide-os]} stores clusters in <path>{[pg-path-default]}</path> so it is easy to determine the correct path for the data directory.</p>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
User Guide clarifications.
2016-05-03 22:12:44 +02:00
<p>When creating the <file>{[backrest-config-demo]}</file> file, the database owner (usually <id>postgres</id>) must be granted read privileges.</p>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<backrest-config host="{[host-pg1]}" file="{[backrest-config-demo]}">
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
<title>Configure the <postgres/> cluster data directory</title>
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<backrest-config-option section="demo" key="pg1-path">{[pg-path]}</backrest-config-option>
Added --log-timestamp option. Allow timestamps to be suppressed in logging. This is primarily used to avoid filters in the automated documentation.
2017-02-05 17:59:49 +02:00
Added the log-level-stderr option. Controls whether console log messages are sent to stderr or stdout. By default this is set to warn which represents a change in behavior from previous versions, even though it may be more intuitive. Setting log-level-stderr=off will preserve the old behavior. Suggested by Sascha Biberhofer.
2016-10-05 15:09:30 +02:00
<backrest-config-option section="global" key="log-level-stderr">off</backrest-config-option>
Added --log-timestamp option. Allow timestamps to be suppressed in logging. This is primarily used to avoid filters in the automated documentation.
2017-02-05 17:59:49 +02:00
<backrest-config-option section="global" key="log-timestamp">n</backrest-config-option>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
</backrest-config>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<p><backrest/> configuration files follow the Windows INI convention. Sections are denoted by text in brackets and key/value pairs are contained in each section. Lines beginning with <id>#</id> are ignored and can be used as comments.</p>
Add support for additional pgBackRest configuration files. Configuration files are loaded from the directory specified by the --config-include-path option. Add --config-path option for overriding the default base path of the --config and --config-include-path option. Contributed by Cynthia Shang.
2018-04-19 17:09:39 +02:00
<p>There are multiple ways the <backrest/> configuration files can be loaded:</p>
<list>
<list-item><br-option>config</br-option> and <br-option>config-include-path</br-option> are default: the default config file will be loaded, if it exists, and <file>*.conf</file> files in the default config include path will be appended, if they exist.</list-item>
<list-item><br-option>config</br-option> option is specified: only the specified config file will be loaded and is expected to exist.</list-item>
Fix typos in the documentation. Contributed by Josh Soref.
2019-08-26 18:26:00 +02:00
<list-item><br-option>config-include-path</br-option> is specified: <file>*.conf</file> files in the config include path will be loaded and the path is required to exist. The default config file will be be loaded if it exists. If it is desirable to load only the files in the specified config include path, then the <br-option>--no-config</br-option> option can also be passed.</list-item>
Add support for additional pgBackRest configuration files. Configuration files are loaded from the directory specified by the --config-include-path option. Add --config-path option for overriding the default base path of the --config and --config-include-path option. Contributed by Cynthia Shang.
2018-04-19 17:09:39 +02:00
<list-item><br-option>config</br-option> and <br-option>config-include-path</br-option> are specified: using the user-specified values, the config file will be loaded and <file>*.conf</file> files in the config include path will be appended. The files are expected to exist.</list-item>
Fix typos in the documentation. Contributed by Josh Soref.
2019-08-26 18:26:00 +02:00
<list-item><br-option>config-path</br-option> is specified: this setting will override the base path for the default location of the config file and/or the base path of the default config-include-path setting unless the config and/or config-include-path option is explicitly set.</list-item>
Add support for additional pgBackRest configuration files. Configuration files are loaded from the directory specified by the --config-include-path option. Add --config-path option for overriding the default base path of the --config and --config-include-path option. Contributed by Cynthia Shang.
2018-04-19 17:09:39 +02:00
</list>
<p>The files are concatenated as if they were one big file; order doesn't matter, but there is precedence based on sections. The precedence (highest to lowest) is:</p>
<list>
<list-item>[<i>stanza</i>:<i>command</i>]</list-item>
<list-item>[<i>stanza</i>]</list-item>
<list-item>[global:<i>command</i>]</list-item>
<list-item>[global]</list-item>
</list>
Add admonitions to documentation renderers. Admonitions call out places where the user should take special care. Support added for HTML, PDF, Markdown and help text renderers. XML files have been updated accordingly. Contributed by Cynthia Shang.
2018-12-30 16:40:20 +02:00
<admonition type="note"><br-option>--config</br-option>, <br-option>--config-include-path</br-option> and <br-option>--config-path</br-option> are command-line only options.</admonition>
Allow any option to be set in an environment variable. This includes options that previously could only be specified on the command line, e.g. stanza.
2018-08-15 16:52:53 +02:00
<p><backrest/> can also be configured using environment variables as described in the <link url="command.html">command reference</link>.</p>
<execute-list host="{[host-pg1]}">
<title>Configure <br-option>log-path</br-option> using the environment</title>
<execute user="postgres" output="y" filter="n">
<exe-cmd>bash -c '
export PGBACKREST_LOG_PATH=/path/set/by/env &amp;&amp;
{[project-exe]} --log-level-console=error help backup log-path'</exe-cmd>
<exe-highlight>current\: \/path\/set\/by\/env</exe-highlight>
</execute>
</execute-list>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
</section>
<!-- SECTION => QUICKSTART - CREATE REPOSITORY -->
Improved support for --require option and section depends now default to the previous section.
2017-02-11 17:26:54 +02:00
<section id="create-repository">
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
<title>Create the Repository</title>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<option-description key="repo-path"/>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<p>For this demonstration the repository will be stored on the same host as the <postgres/> server. This is the simplest configuration and is useful in cases where traditional backup software is employed to backup the database host.</p>
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
<block id="br-install-repo">
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<block-variable-replace key="br-install-host">{[host-pg1]}</block-variable-replace>
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
<block-variable-replace key="br-install-user">postgres</block-variable-replace>
<block-variable-replace key="br-install-group">postgres</block-variable-replace>
</block>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
<p>The repository path must be configured so <backrest/> knows where to find it.</p>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<backrest-config host="{[host-pg1]}" file="{[backrest-config-demo]}">
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
<title>Configure the <backrest/> repository path</title>
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<backrest-config-option section="global" key="repo1-path">{[backrest-repo-path]}</backrest-config-option>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
</backrest-config>
Multiple repository support. Up to four repositories may be configured. A potential benefit is the ability to have a local repository for fast restores and a remote repository for redundancy. Some commands, e.g. stanza-create/stanza-update, will automatically work with all configured repositories while others, e.g. stanza-delete, will require a repository to be specified using the repo option. See the command reference for details on which commands require the repository to be specified. Note that the repo option is not required when only repo1 is configured in order to maintain backward compatibility. However, the repo option is required when a single repo is configured as, e.g. repo2. This is to prevent command breakage if a new repository is added later. The archive-push command will always push WAL to the archive in all configured repositories but backups will need to be scheduled individually for each repository. In many cases this is desirable since backup types and retention will vary by repository. Likewise, restores must specify a repository. It is generally better to specify a repository for restores that has low latency/cost even if that means more recovery time. Only restore testing can determine which repository will be most efficient. For single repository configurations there should be no change in behavior.
2021-03-08 20:31:13 +02:00
<p if="!{[object-any-all]}">Multiple repositories may also be configured. See <link section="/multi-repo">Multiple Repositories</link> for details.</p>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
</section>
Azure support for repository storage. Azure and Azure-compatible object stores can now be used for repository storage. Currently only shared key authentication is supported but SAS will be added soon.
2020-07-02 22:24:34 +02:00
<!-- =================================================================================================================== -->
<section id="azure-support" if="'{[azure-all]}' eq 'y'">
<title>Azure-Compatible Object Store Support</title>
<block id="azure-setup">
Multiple repository support. Up to four repositories may be configured. A potential benefit is the ability to have a local repository for fast restores and a remote repository for redundancy. Some commands, e.g. stanza-create/stanza-update, will automatically work with all configured repositories while others, e.g. stanza-delete, will require a repository to be specified using the repo option. See the command reference for details on which commands require the repository to be specified. Note that the repo option is not required when only repo1 is configured in order to maintain backward compatibility. However, the repo option is required when a single repo is configured as, e.g. repo2. This is to prevent command breakage if a new repository is added later. The archive-push command will always push WAL to the archive in all configured repositories but backups will need to be scheduled individually for each repository. In many cases this is desirable since backup types and retention will vary by repository. Likewise, restores must specify a repository. It is generally better to specify a repository for restores that has low latency/cost even if that means more recovery time. Only restore testing can determine which repository will be most efficient. For single repository configurations there should be no change in behavior.
2021-03-08 20:31:13 +02:00
<block-variable-replace key="azure-setup-repo-id">1</block-variable-replace>
Azure support for repository storage. Azure and Azure-compatible object stores can now be used for repository storage. Currently only shared key authentication is supported but SAS will be added soon.
2020-07-02 22:24:34 +02:00
<block-variable-replace key="azure-setup-host">{[host-pg1]}</block-variable-replace>
<block-variable-replace key="azure-setup-user">postgres</block-variable-replace>
<block-variable-replace key="azure-setup-config-owner">postgres:postgres</block-variable-replace>
<block-variable-replace key="azure-setup-create-container">y</block-variable-replace>
</block>
</section>
GCS support for repository storage. GCS and GCS-compatible object stores can now be used for repository storage.
2021-03-05 19:13:51 +02:00
<!-- =================================================================================================================== -->
<section id="gcs-support" if="'{[gcs-all]}' eq 'y'">
<title>GCS-Compatible Object Store Support</title>
<block id="gcs-setup">
Multiple repository support. Up to four repositories may be configured. A potential benefit is the ability to have a local repository for fast restores and a remote repository for redundancy. Some commands, e.g. stanza-create/stanza-update, will automatically work with all configured repositories while others, e.g. stanza-delete, will require a repository to be specified using the repo option. See the command reference for details on which commands require the repository to be specified. Note that the repo option is not required when only repo1 is configured in order to maintain backward compatibility. However, the repo option is required when a single repo is configured as, e.g. repo2. This is to prevent command breakage if a new repository is added later. The archive-push command will always push WAL to the archive in all configured repositories but backups will need to be scheduled individually for each repository. In many cases this is desirable since backup types and retention will vary by repository. Likewise, restores must specify a repository. It is generally better to specify a repository for restores that has low latency/cost even if that means more recovery time. Only restore testing can determine which repository will be most efficient. For single repository configurations there should be no change in behavior.
2021-03-08 20:31:13 +02:00
<block-variable-replace key="gcs-setup-repo-id">1</block-variable-replace>
GCS support for repository storage. GCS and GCS-compatible object stores can now be used for repository storage.
2021-03-05 19:13:51 +02:00
<block-variable-replace key="gcs-setup-host">{[host-pg1]}</block-variable-replace>
<block-variable-replace key="gcs-setup-user">postgres</block-variable-replace>
<block-variable-replace key="gcs-setup-config-owner">postgres:postgres</block-variable-replace>
</block>
</section>
Move S3 initialization in user guide quickstart. The previous location was too late to allow --var=s3-all=y to work with --require=/repo-host, which depends on /quickstart/configure-archiving. Since the section is not included in production documentation, the position is not very important to flow so just move it to where it works.
2020-05-04 00:42:33 +02:00
<!-- =================================================================================================================== -->
<section id="s3-support" if="'{[s3-all]}' eq 'y'">
<title>S3-Compatible Object Store Support</title>
<block id="s3-setup">
Multiple repository support. Up to four repositories may be configured. A potential benefit is the ability to have a local repository for fast restores and a remote repository for redundancy. Some commands, e.g. stanza-create/stanza-update, will automatically work with all configured repositories while others, e.g. stanza-delete, will require a repository to be specified using the repo option. See the command reference for details on which commands require the repository to be specified. Note that the repo option is not required when only repo1 is configured in order to maintain backward compatibility. However, the repo option is required when a single repo is configured as, e.g. repo2. This is to prevent command breakage if a new repository is added later. The archive-push command will always push WAL to the archive in all configured repositories but backups will need to be scheduled individually for each repository. In many cases this is desirable since backup types and retention will vary by repository. Likewise, restores must specify a repository. It is generally better to specify a repository for restores that has low latency/cost even if that means more recovery time. Only restore testing can determine which repository will be most efficient. For single repository configurations there should be no change in behavior.
2021-03-08 20:31:13 +02:00
<block-variable-replace key="s3-setup-repo-id">1</block-variable-replace>
Move S3 initialization in user guide quickstart. The previous location was too late to allow --var=s3-all=y to work with --require=/repo-host, which depends on /quickstart/configure-archiving. Since the section is not included in production documentation, the position is not very important to flow so just move it to where it works.
2020-05-04 00:42:33 +02:00
<block-variable-replace key="s3-setup-host">{[host-pg1]}</block-variable-replace>
<block-variable-replace key="s3-setup-user">postgres</block-variable-replace>
<block-variable-replace key="s3-setup-config-owner">postgres:postgres</block-variable-replace>
<block-variable-replace key="s3-setup-create-bucket">y</block-variable-replace>
</block>
</section>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
<!-- SECTION => QUICKSTART - CONFIGURE ARCHIVING -->
Improved support for --require option and section depends now default to the previous section.
2017-02-11 17:26:54 +02:00
<section id="configure-archiving">
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
<title>Configure Archiving</title>
User Guide clarifications.
2016-05-03 22:12:44 +02:00
<p>Backing up a running <postgres/> cluster requires WAL archiving to be enabled. Note that <i>at least</i> one WAL segment will be created during the backup process even if no explicit writes are made to the cluster.</p>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<postgres-config host="{[host-pg1]}" file="{[postgres-config-demo]}">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Configure archive settings</title>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<postgres-config-option key="archive_command">'{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} archive-push %p'</postgres-config-option>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
<postgres-config-option key="archive_mode">on</postgres-config-option>
Use wal_level=replica in the documentation for PostgreSQL >= 9.6. The documentation was using wal_level=hot_standby which is a deprecated setting. Also remove the reference to wal_level=archive since it is no longer supported and is not recommended for older versions. Suggested by Patrick McLaughlin.
2019-06-05 13:27:24 +02:00
<postgres-config-option key="wal_level">{[wal-level]}</postgres-config-option>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<postgres-config-option key="max_wal_senders">3</postgres-config-option>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
</postgres-config>
Update reference to include links to user guide examples. The command-example and command-example-list elements were removed from the documentation rendering some time ago so these tags were dead code. The tags, however, contained some examples and information that were pertinent to the command, so where possible, the information was included in the description of the command and/or the user-guide and links to the relevant user guide sections were added. Note that some commands could not be updated with user guide references since doing so would cause a cyclical reference in the user guide. These commands have an internal comment to indicate this. In addition, some clarifications were added (e.g. expire --set option) where information was lacking.
2021-03-31 15:36:56 +02:00
<p><id>%p</id> is how <postgres/> specifies the location of the WAL segment to be archived. Setting <pg-option>wal_level</pg-option> to at least <pg-setting>{[wal-level]}</pg-setting> and increasing <pg-option>max_wal_senders</pg-option> is a good idea even if there are currently no replicas as this will allow them to be added later without restarting the primary cluster.</p>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
<p>The <postgres/> cluster must be restarted after making these changes and before performing a backup.</p>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
<title>Restart the {[postgres-cluster-demo]} cluster</title>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<execute user="root">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<exe-cmd>{[pg-cluster-restart]}</exe-cmd>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
</execute>
Closed #163: Document Generation Issue Fixed an issue where document generation failed because some OSs are not tolerant of having multiple installed versions of PostgreSQL. A separate VM is now created for each version. Also added a sleep after database starts during document generation to ensure the database is running before the next command runs. Reported by John Harvey.
2016-01-23 01:01:21 +02:00
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" show="n">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<exe-cmd>{[pg-cluster-wait]}</exe-cmd>
Closed #163: Document Generation Issue Fixed an issue where document generation failed because some OSs are not tolerant of having multiple installed versions of PostgreSQL. A separate VM is now created for each version. Also added a sleep after database starts during document generation to ensure the database is running before the next command runs. Reported by John Harvey.
2016-01-23 01:01:21 +02:00
</execute>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
</execute-list>
Added archive-timeout option documentation to the user guide. Contributed by Cynthia Shang.
2016-09-28 18:29:29 +02:00
Clarify usage of the archive-timeout option. Also describe how it is distinct from the PostgreSQL archive_timeout setting. Contributed by Cynthia Shang. Suggested by Keith Fiske.
2017-10-30 20:18:05 +02:00
<p>When archiving a WAL segment is expected to take more than 60 seconds (the default) to reach the <backrest/> repository, then the <backrest/> <br-option>archive-timeout</br-option> option should be increased. Note that this option is not the same as the <postgres/> <pg-option>archive_timeout</pg-option> option which is used to force a WAL segment switch; useful for databases where there are long periods of inactivity. For more information on the <postgres/> <pg-option>archive_timeout</pg-option> option, see <postgres/> <link url="https://www.postgresql.org/docs/current/static/runtime-config-wal.html">Write Ahead Log</link>.</p>
Add example of command section usage to archiving configuration. Suggested by Christophe Courtois.
2018-05-02 13:31:44 +02:00
<p>The <cmd>archive-push</cmd> command can be configured with its own options. For example, a lower compression level may be set to speed archiving without affecting the compression used for backups.</p>
<backrest-config host="{[host-pg1]}" file="{[backrest-config-demo]}">
<title>Config <cmd>archive-push</cmd> to use a lower compression level</title>
<backrest-config-option section="global:archive-push" key="compress-level">3</backrest-config-option>
</backrest-config>
<p>This configuration technique can be used for any command and can even target a specific stanza, e.g. <code>demo:archive-push</code>.</p>
Added check command. The check command validates that pgBackRest is configured correctly for archiving and backups. Contributed by Cynthia Shang.
2016-06-12 15:13:46 +02:00
</section>
Added Retention to QuickStart section.
2016-12-05 01:11:23 +02:00
<!-- SECTION => QUICKSTART - RETENTION -->
Improved support for --require option and section depends now default to the previous section.
2017-02-11 17:26:54 +02:00
<section id="retention">
Added Retention to QuickStart section.
2016-12-05 01:11:23 +02:00
<title>Configure Retention</title>
<p><backrest/> expires backups based on retention options.</p>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<backrest-config host="{[host-pg1]}" file="{[backrest-config-demo]}">
Added Retention to QuickStart section.
2016-12-05 01:11:23 +02:00
<title>Configure retention to 2 full backups</title>
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
<backrest-config-option section="global" key="repo1-retention-full">2</backrest-config-option>
Added Retention to QuickStart section.
2016-12-05 01:11:23 +02:00
</backrest-config>
<p>More information about retention can be found in the <link section="/retention">Retention</link> section.</p>
</section>
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
<!-- SECTION => QUICKSTART - CONFIGURE ENCRYPTION -->
Update backup host to repository host in the documentation. Contributed by Cynthia Shang.
2018-02-19 20:17:58 +02:00
<!-- Since S3 and repository host require configure-archiving, this section must come after. -->
Allow documentation to be built without encryption. This is mainly useful for testing, where an encrypted repo is a real nuisance.
2019-09-25 22:38:13 +02:00
<section if="'{[encrypt]}' eq 'y'" id="configure-encryption">
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
<title>Configure Repository Encryption</title>
Clarify that S3-compatible object stores are supported. The documentation mentioned Amazon S3 frequently but failed to mention that other S3-compatible object stores are also supported. Tone down the specific mentions of Amazon S3 and replace them with "S3-compatible object store" when appropriate. Suggested by Magnus Hagander.
2019-04-03 12:40:12 +02:00
<p>The repository will be configured with a cipher type and key to demonstrate encryption. Encryption is always performed client-side even if the repository type (e.g. <proper>S3</proper> or other object store) supports encryption.</p>
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
<p>It is important to use a long, random passphrase for the cipher key. A good way to generate one is to run: <code>openssl rand -base64 48</code>.</p>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<backrest-config host="{[host-pg1]}" file="{[backrest-config-demo]}">
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
<title>Configure <backrest/> repository encryption</title>
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<backrest-config-option section="global" key="repo1-cipher-type">{[backrest-repo-cipher-type]}</backrest-config-option>
<backrest-config-option section="global" key="repo1-cipher-pass">{[backrest-repo-cipher-pass]}</backrest-config-option>
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
</backrest-config>
<p>Once the repository has been configured and the stanza created and checked, the repository encryption settings cannot be changed.</p>
</section>
Added the stanza-create command to formalize creation of stanzas in the repository. Contributed by Cynthia Shang.
2016-10-14 13:21:47 +02:00
<!-- SECTION => QUICKSTART - STANZA CREATE -->
Improved support for --require option and section depends now default to the previous section.
2017-02-11 17:26:54 +02:00
<section id="create-stanza">
Added the stanza-create command to formalize creation of stanzas in the repository. Contributed by Cynthia Shang.
2016-10-14 13:21:47 +02:00
<title>Create the Stanza</title>
Allow stanza-* commands to be run remotely. The stanza-create, stanza-upgrade and stanza-delete were required to be run on the repository host. When there was only one repository allowed this was not a problem. However, with the introduction of multiple repository support, this becomes more of a burden to the user, therefore the stanza-create, stanza-upgrade and stanza-delete commands have been improved to allow for them to be run remotely.
2021-03-10 15:10:46 +02:00
<p>The <cmd>stanza-create</cmd> command must be run to initialize the stanza. It is recommended that the <cmd>check</cmd> command be run after <cmd>stanza-create</cmd> to ensure archiving and backups are properly configured.</p>
Added the stanza-create command to formalize creation of stanzas in the repository. Contributed by Cynthia Shang.
2016-10-14 13:21:47 +02:00
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Improved stanza-create command so that it can repair broken repositories in most cases and is robust enough to be made mandatory. Contributed by Cynthia Shang.
2016-12-20 23:52:20 +02:00
<title>Create the stanza and check the configuration</title>
Added the stanza-create command to formalize creation of stanzas in the repository. Contributed by Cynthia Shang.
2016-10-14 13:21:47 +02:00
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y">
Added the stanza-create command to formalize creation of stanzas in the repository. Contributed by Cynthia Shang.
2016-10-14 13:21:47 +02:00
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} {[dash]}-log-level-console=info stanza-create</exe-cmd>
Improved check command to run on a standby. Only basic checks are done because pg_switch_xlog() cannot be executed on a replica. Contributed by Cynthia Shang.
2016-12-21 21:35:20 +02:00
<exe-highlight>completed successfully</exe-highlight>
Added the stanza-create command to formalize creation of stanzas in the repository. Contributed by Cynthia Shang.
2016-10-14 13:21:47 +02:00
</execute>
</execute-list>
</section>
Added check command. The check command validates that pgBackRest is configured correctly for archiving and backups. Contributed by Cynthia Shang.
2016-06-12 15:13:46 +02:00
<!-- SECTION => QUICKSTART - CHECK CONFIGURATION -->
Improved support for --require option and section depends now default to the previous section.
2017-02-11 17:26:54 +02:00
<section id="check-configuration">
Added check command. The check command validates that pgBackRest is configured correctly for archiving and backups. Contributed by Cynthia Shang.
2016-06-12 15:13:46 +02:00
<title>Check the Configuration</title>
<cmd-description key="check"/>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Added check command. The check command validates that pgBackRest is configured correctly for archiving and backups. Contributed by Cynthia Shang.
2016-06-12 15:13:46 +02:00
<title>Check the configuration</title>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y">
Added check command. The check command validates that pgBackRest is configured correctly for archiving and backups. Contributed by Cynthia Shang.
2016-06-12 15:13:46 +02:00
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} {[dash]}-log-level-console=info check</exe-cmd>
The check command is implemented partly in C. Implement switch WAL and archive check in C but leave the rest in Perl for now. The main idea was to have some real integration tests for the new database code so the rest of the migration can wait. Reviewed by Cynthia Shang.
2019-08-02 02:35:01 +02:00
<exe-highlight> successfully archived to </exe-highlight>
Added check command. The check command validates that pgBackRest is configured correctly for archiving and backups. Contributed by Cynthia Shang.
2016-06-12 15:13:46 +02:00
</execute>
</execute-list>
<!-- Decided not to show the error in this part of the user guide but added as a debug statement for reference. -->
Replace keywords with more flexible if statements. Keywords were extremely limited and prevented us from generating multi-version documentation and other improvements. Replace keywords with an if statement that can evaluate a Perl expression with variable replacement. Since keywords were used to generate cache keys, add a --key-var parameter to identify which variables should make up the key.
2018-12-01 19:40:01 +02:00
<execute-list if="'{[debug]}' eq 'y'" host="{[host-pg1]}">
Added check command. The check command validates that pgBackRest is configured correctly for archiving and backups. Contributed by Cynthia Shang.
2016-06-12 15:13:46 +02:00
<title>Example of an invalid configuration</title>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y" err-expect="82">
Added check command. The check command validates that pgBackRest is configured correctly for archiving and backups. Contributed by Cynthia Shang.
2016-06-12 15:13:46 +02:00
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} --archive-timeout=.1 check</exe-cmd>
<exe-highlight>could not find WAL segment|did not reach the archive</exe-highlight>
</execute>
</execute-list>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
</section>
<!-- SECTION => QUICKSTART - PERFORM BACKUP -->
Improved support for --require option and section depends now default to the previous section.
2017-02-11 17:26:54 +02:00
<section id="perform-backup">
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
<title>Perform a Backup</title>
Rearrange backup documentation in user guide. Remove the "Automatic Stop Option" section since it only applies to PostgreSQL <= 9.6, which will soon be EOL. Since we no longer build the user guide for PostgreSQL < 10 this section was no longer being tested. The stop-auto option is still documented in the reference. Move the "Fast Start Option" to "Quick Start - Perform Backup". This is a commonly-used option so it makes sense to mention it earlier. This also makes the backups run more quickly. In the worst case, backups in "Quick Start - Perform Backup" could take minutes to start Move the "Archive Timeout" section to "Quick Start - Perform Backup" since it is the last section in "Backup".
2021-08-11 18:59:25 +02:00
<p>By default <backrest/> will wait for the next regularly scheduled checkpoint before starting a backup. Depending on the <pg-option>checkpoint_timeout</pg-option> and <pg-option>checkpoint_segments</pg-option> settings in <postgres/> it may be quite some time before a checkpoint completes and the backup can begin. Generally, it is best to set <br-setting>start-fast=y</br-setting> so that the backup starts immediately. This forces a checkpoint, but since backups are usually run once a day an additional checkpoint should not have a noticeable impact on performance. However, on very busy clusters it may be best to pass <br-setting>{[dash]}-start-fast</br-setting> on the command-line as needed.</p>
<backrest-config host="{[host-pg1]}" file="{[backrest-config-demo]}">
<title>Configure backup fast start</title>
<backrest-config-option section="global" key="start-fast">y</backrest-config-option>
</backrest-config>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
<p>To perform a backup of the <postgres/> cluster run <backrest/> with the <cmd>backup</cmd> command.</p>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
<title>Backup the {[postgres-cluster-demo]} cluster</title>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]}
Added execution cache for document generation. Added an execution cache so that documentation can be generated without setting up the full container environment. This is useful for packaging, keeps the documentation consistent for a release, and speeds up generation when no changes are made in the execution list.
2016-06-02 15:32:56 +02:00
--log-level-console=info backup</exe-cmd>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-highlight>no prior backup exists|full backup size</exe-highlight>
</execute>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" show="n" variable-key="backup-full-first">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-cmd>{[cmd-backup-last]}</exe-cmd>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
</execute>
</execute-list>
<p>By default <backrest/> will attempt to perform an incremental backup. However, an incremental backup must be based on a full backup and since no full backup existed <backrest/> ran a full backup instead.</p>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<p>The <br-option>type</br-option> option can be used to specify a full or differential backup.</p>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
<title>Differential backup of the {[postgres-cluster-demo]} cluster</title>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} {[dash]}-type=diff
Added execution cache for document generation. Added an execution cache so that documentation can be generated without setting up the full container environment. This is useful for packaging, keeps the documentation consistent for a release, and speeds up generation when no changes are made in the execution list.
2016-06-02 15:32:56 +02:00
--log-level-console=info backup</exe-cmd>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-highlight>diff backup size</exe-highlight>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
</execute>
</execute-list>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<p>This time there was no warning because a full backup already existed. While incremental backups can be based on a full <i>or</i> differential backup, differential backups must be based on a full backup. A full backup can be performed by running the <cmd>backup</cmd> command with <br-setting>{[dash]}-type=full</br-setting>.</p>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Rearrange backup documentation in user guide. Remove the "Automatic Stop Option" section since it only applies to PostgreSQL <= 9.6, which will soon be EOL. Since we no longer build the user guide for PostgreSQL < 10 this section was no longer being tested. The stop-auto option is still documented in the reference. Move the "Fast Start Option" to "Quick Start - Perform Backup". This is a commonly-used option so it makes sense to mention it earlier. This also makes the backups run more quickly. In the worst case, backups in "Quick Start - Perform Backup" could take minutes to start Move the "Archive Timeout" section to "Quick Start - Perform Backup" since it is the last section in "Backup".
2021-08-11 18:59:25 +02:00
<p>During an online backup <backrest/> waits for WAL segments that are required for backup consistency to be archived. This wait time is governed by the <backrest/> <br-option>archive-timeout</br-option> option which defaults to 60 seconds. If archiving an individual segment is known to take longer then this option should be increased.</p>
Enhanced text output of info command to include timestamps, sizes, and the reference list for all backups.
2016-05-06 16:25:46 +02:00
</section>
Added documentation for scheduling backups with cron. Contributed by Cynthia Shang.
2016-08-02 18:55:09 +02:00
<!-- SECTION => QUICKSTART - SCHEDULE BACKUP -->
<section id="schedule-backup">
<title>Schedule a Backup</title>
<p>Backups can be scheduled with utilities such as cron.</p>
<p>In the following example, two cron jobs are configured to run; full backups are scheduled for 6:30 AM every Sunday with differential backups scheduled for 6:30 AM Monday through Saturday. If this crontab is installed for the first time mid-week, then pgBackRest will run a full backup the first time the differential job is executed, followed the next day by a differential backup.</p>
<code-block title="crontab">
#m h dom mon dow command
30 06 * * 0 pgbackrest --type=full --stanza=demo backup
30 06 * * 1-6 pgbackrest --type=diff --stanza=demo backup
</code-block>
<p>Once backups are scheduled it's important to configure retention so backups are expired on a regular schedule, see <link section="/retention">Retention</link>.</p>
</section>
Enhanced text output of info command to include timestamps, sizes, and the reference list for all backups.
2016-05-06 16:25:46 +02:00
<!-- SECTION => QUICKSTART - BACKUP INFO -->
<section id="backup-info" depend="perform-backup">
<title>Backup Information</title>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
<p>Use the <cmd>info</cmd> command to get information about backups.</p>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
<title>Get info for the {[postgres-cluster-demo]} cluster</title>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" filter="n" output="y">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-cmd>{[project-exe]} info</exe-cmd>
Enhanced text output of info command to include timestamps, sizes, and the reference list for all backups.
2016-05-06 16:25:46 +02:00
<exe-highlight>(full|incr|diff) backup</exe-highlight>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
</execute>
</execute-list>
Move info command text to the reference and link to user guide. This means the same text will appear in both places, which should make it easier to find. Also update the link code to allow both page and section to be specified rather than only one or the other.
2020-09-25 17:26:27 +02:00
<cmd-description key="info"/>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
</section>
<!-- SECTION => QUICKSTART - PERFORM RESTORE -->
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<section id="perform-restore" depend="perform-backup">
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
<title>Restore a Backup</title>
<p>Backups can protect you from a number of disaster scenarios, the most common of which are hardware failure and data corruption. The easiest way to simulate data corruption is to remove an important <postgres/> cluster file.</p>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
<title>Stop the {[postgres-cluster-demo]} cluster and delete the <file>pg_control</file> file</title>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<execute user="root">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<exe-cmd>{[pg-cluster-stop]}</exe-cmd>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
</execute>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<exe-cmd>rm {[pg-path]}/global/pg_control</exe-cmd>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
</execute>
</execute-list>
<p>Starting the cluster without this important file will result in an error.</p>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
<title>Attempt to start the corrupted {[postgres-cluster-demo]} cluster</title>
Create common if expressions for testing os-type. These expressions simplify os-type testing. This will be especially true as more OS types are added.
2018-12-05 00:33:56 +02:00
<execute if="{[os-type-is-debian]}" user="root" output="y" err-expect="1">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<exe-cmd>{[pg-cluster-start]}</exe-cmd>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-highlight>could not find the database system</exe-highlight>
</execute>
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<execute if="{[os-type-is-rhel]}" user="root" err-expect="1">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<exe-cmd>{[pg-cluster-start]}</exe-cmd>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
</execute>
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<execute if="{[os-type-is-rhel]}" user="root" output="y" err-expect="3">
Add Centos/RHEL 7 option to documentation build. Centos/RHEL 7 builds can be specified with --var=os-type=centos7. Also added to the documentation release and CI builds.
2018-12-05 14:12:28 +02:00
<exe-cmd>{[pg-cluster-check]}</exe-cmd>
Add user guides for CentOS/RHEL 6/7. It would be better if the documentation could be generated on multiple operating systems all in one go, but the doc system currently does not allow vars to be changed once they are set. The solution is to run the docs for each required OS and stitch the documentation together. It's not pretty but it works and the automation in release.pl should at least make it easy to use.
2019-05-16 14:32:02 +02:00
<exe-highlight>Failed to start PostgreSQL</exe-highlight>
Add Centos/RHEL 7 option to documentation build. Centos/RHEL 7 builds can be specified with --var=os-type=centos7. Also added to the documentation release and CI builds.
2018-12-05 14:12:28 +02:00
</execute>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
</execute-list>
<p>To restore a backup of the <postgres/> cluster run <backrest/> with the <cmd>restore</cmd> command. The cluster needs to be stopped (in this case it is already stopped) and all files must be removed from the <postgres/> data directory.</p>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
<title>Remove old files from {[postgres-cluster-demo]} cluster</title>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<exe-cmd>find {[pg-path]} -mindepth 1 -delete</exe-cmd>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
</execute>
</execute-list>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
<title>Restore the {[postgres-cluster-demo]} cluster and start <postgres/></title>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} restore</exe-cmd>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
</execute>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<execute user="root">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<exe-cmd>{[pg-cluster-start]}</exe-cmd>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
</execute>
Closed #163: Document Generation Issue Fixed an issue where document generation failed because some OSs are not tolerant of having multiple installed versions of PostgreSQL. A separate VM is now created for each version. Also added a sleep after database starts during document generation to ensure the database is running before the next command runs. Reported by John Harvey.
2016-01-23 01:01:21 +02:00
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" show="n">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<exe-cmd>{[pg-cluster-wait]}</exe-cmd>
Closed #163: Document Generation Issue Fixed an issue where document generation failed because some OSs are not tolerant of having multiple installed versions of PostgreSQL. A separate VM is now created for each version. Also added a sleep after database starts during document generation to ensure the database is running before the next command runs. Reported by John Harvey.
2016-01-23 01:01:21 +02:00
</execute>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
</execute-list>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<p>This time the cluster started successfully since the restore replaced the missing <file>pg_control</file> file.</p>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
A number of doc improvements: 1) Started on a general markdown renderer 2) Internal links now work in PDF 3) Improvements to PDF styling 4) Some comment and formatting fixes 5) User guide edits.
2015-12-23 18:04:26 +02:00
<p>More information about the <cmd>restore</cmd> command can be found in the <link section="/restore">Restore</link> section.</p>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
</section>
</section>
Add monitoring examples using PostgreSQL and jq. Suggested by Stephen Frost, Brian Faherty.
2018-04-13 20:31:33 +02:00
<!-- *********************************************************************************************************************** -->
<section id="monitor" depend="/quickstart/perform-backup">
<title>Monitoring</title>
<p>Monitoring is an important part of any production system. There are many tools available and <backrest/> can be monitored on any of them with a little work.</p>
JSON output from the info command is no longer pretty-printed. Monitoring systems can more easily ingest the JSON without linefeeds. External tools such as jq can be used to pretty-print if desired.
2019-10-11 18:56:03 +02:00
<p><backrest/> can output information about the repository in JSON format which includes a list of all backups for each stanza and WAL archive info.</p>
Add monitoring examples using PostgreSQL and jq. Suggested by Stephen Frost, Brian Faherty.
2018-04-13 20:31:33 +02:00
<section id="postgresql">
<title>In <postgres/></title>
<p>The <postgres/> <id>COPY</id> command allows <backrest/> info to be loaded into a table. The following example wraps that logic in a function that can be used to perform real-time queries.</p>
<execute-list host="{[host-pg1]}">
<title>Load <backrest/> info function for <postgres/></title>
<execute user="postgres" show="n">
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<exe-cmd>mkdir -p {[pg-home-path]}/pgbackrest/doc/example</exe-cmd>
Add monitoring examples using PostgreSQL and jq. Suggested by Stephen Frost, Brian Faherty.
2018-04-13 20:31:33 +02:00
</execute>
<execute user="postgres" show="n">
Fix hard-coded repository path. This prevented packages from being passed to the documentation unless they were in the /backrest directory on the host. Also make the local path /pgbackrest instead of the deprecated /backrest. Reported by Heath Lord.
2019-01-30 13:39:22 +02:00
<exe-cmd>cp -r {[pgbackrest-repo-path]}/doc/example/*
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
{[pg-home-path]}/pgbackrest/doc/example</exe-cmd>
Add monitoring examples using PostgreSQL and jq. Suggested by Stephen Frost, Brian Faherty.
2018-04-13 20:31:33 +02:00
</execute>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y">
Add monitoring examples using PostgreSQL and jq. Suggested by Stephen Frost, Brian Faherty.
2018-04-13 20:31:33 +02:00
<exe-cmd>cat
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
{[pg-home-path]}/pgbackrest/doc/example/pgsql-pgbackrest-info.sql</exe-cmd>
Add monitoring examples using PostgreSQL and jq. Suggested by Stephen Frost, Brian Faherty.
2018-04-13 20:31:33 +02:00
</execute>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres">
Add monitoring examples using PostgreSQL and jq. Suggested by Stephen Frost, Brian Faherty.
2018-04-13 20:31:33 +02:00
<exe-cmd>psql -f
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
{[pg-home-path]}/pgbackrest/doc/example/pgsql-pgbackrest-info.sql</exe-cmd>
Add monitoring examples using PostgreSQL and jq. Suggested by Stephen Frost, Brian Faherty.
2018-04-13 20:31:33 +02:00
</execute>
</execute-list>
<p>Now the <code>monitor.pgbackrest_info()</code> function can be used to determine the last successful backup time and archived WAL for a stanza.</p>
<execute-list host="{[host-pg1]}">
<title>Query last successful backup time and archived WAL</title>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y">
Add monitoring examples using PostgreSQL and jq. Suggested by Stephen Frost, Brian Faherty.
2018-04-13 20:31:33 +02:00
<exe-cmd>cat
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
{[pg-home-path]}/pgbackrest/doc/example/pgsql-pgbackrest-query.sql</exe-cmd>
Add monitoring examples using PostgreSQL and jq. Suggested by Stephen Frost, Brian Faherty.
2018-04-13 20:31:33 +02:00
</execute>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y">
Add monitoring examples using PostgreSQL and jq. Suggested by Stephen Frost, Brian Faherty.
2018-04-13 20:31:33 +02:00
<exe-cmd>psql -f
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
{[pg-home-path]}/pgbackrest/doc/example/pgsql-pgbackrest-query.sql</exe-cmd>
Add monitoring examples using PostgreSQL and jq. Suggested by Stephen Frost, Brian Faherty.
2018-04-13 20:31:33 +02:00
</execute>
</execute-list>
</section>
Create common if expressions for testing os-type. These expressions simplify os-type testing. This will be especially true as more OS types are added.
2018-12-05 00:33:56 +02:00
<section if="{[os-type-is-debian]}" id="jq">
Add monitoring examples using PostgreSQL and jq. Suggested by Stephen Frost, Brian Faherty.
2018-04-13 20:31:33 +02:00
<title>Using <proper>jq</proper></title>
<p><proper>jq</proper> is a command-line utility that can easily extract data from JSON.</p>
<execute-list host="{[host-pg1]}">
<title>Install <proper>jq</proper> utility</title>
Pre-build containers for any execute elements marked pre. This allows the documentation to be built more quickly and offline during development when --pre is specified on the command line. Each host gets a pre-built container with all the execute elements marked pre. As long as the pre elements do not change the container will not need to be rebuilt. The feature should not be used for CI builds as it may hide errors in the documentation.
2018-11-29 21:45:15 +02:00
<execute user="root" pre="y">
Add monitoring examples using PostgreSQL and jq. Suggested by Stephen Frost, Brian Faherty.
2018-04-13 20:31:33 +02:00
<exe-cmd>apt-get install jq</exe-cmd>
<exe-cmd-extra>-y 2>&amp;1</exe-cmd-extra>
</execute>
</execute-list>
<p>Now <proper>jq</proper> can be used to query the last successful backup time for a stanza.</p>
<execute-list host="{[host-pg1]}">
<title>Query last successful backup time</title>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y">
Add monitoring examples using PostgreSQL and jq. Suggested by Stephen Frost, Brian Faherty.
2018-04-13 20:31:33 +02:00
<exe-cmd>
pgbackrest --output=json --stanza=demo info |
jq '.[0] | .backup[-1] | .timestamp.stop'
</exe-cmd>
</execute>
</execute-list>
<p>Or the last archived WAL.</p>
<execute-list host="{[host-pg1]}">
<title>Query last archived WAL</title>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y">
Add monitoring examples using PostgreSQL and jq. Suggested by Stephen Frost, Brian Faherty.
2018-04-13 20:31:33 +02:00
<exe-cmd>
pgbackrest --output=json --stanza=demo info |
jq '.[0] | .archive[-1] | .max'
</exe-cmd>
</execute>
</execute-list>
Add admonitions to documentation renderers. Admonitions call out places where the user should take special care. Support added for HTML, PDF, Markdown and help text renderers. XML files have been updated accordingly. Contributed by Cynthia Shang.
2018-12-30 16:40:20 +02:00
<admonition type="note">This syntax requires <proper>jq v1.5</proper>.</admonition>
Add a warning about jq rounding numbers to the documentation.
2019-10-11 19:14:47 +02:00
<admonition type="note"><proper>jq</proper> may round large numbers such as system identifiers. Test your queries carefully.</admonition>
Add monitoring examples using PostgreSQL and jq. Suggested by Stephen Frost, Brian Faherty.
2018-04-13 20:31:33 +02:00
</section>
</section>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<!-- SECTION => RETENTION -->
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
<section id="retention" depend="quickstart/perform-backup">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Retention</title>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
A number of doc improvements: 1) Started on a general markdown renderer 2) Internal links now work in PDF 3) Improvements to PDF styling 4) Some comment and formatting fixes 5) User guide edits.
2015-12-23 18:04:26 +02:00
<p>Generally it is best to retain as many backups as possible to provide a greater window for <link section="/pitr">Point-in-Time Recovery</link>, but practical concerns such as disk space must also be considered. Retention options remove older backups once they are no longer needed.</p>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Improve expire command documentation.
2021-02-05 18:48:07 +02:00
<cmd-description key="expire"/>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<!-- SECTION => RETENTION - FULL -->
<section id="full">
<title>Full Backup Retention</title>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Add time-based retention for full backups. The --repo-retention-full-type option allows retention of full backups based on a time period, specified in days. The new option will default to 'count' and therefore will not affect current installations. Setting repo-retention-full-type to 'time' will allow the user to use a time period, in days, to indicate full backup retention. Using this method, a full backup can be expired only if the time the backup completed is older than the number of days set with repo-retention-full (calculated from the moment the 'expire' command is run) and at least one full backup meets the retention period. If archive retention has not been configured, then the default settings will expire archives that are prior to the oldest retained full backup. For example, if there are three full backups ending in times that are 25 days old (F1), 20 days old (F2) and 10 days old (F3), then if the full retention period is 15 days, then only F1 will be expired; F2 will be retained because F1 is not at least 15 days old.
2020-05-08 21:25:03 +02:00
<p>The <br-option>repo1-retention-full-type</br-option> determines how the option <br-option>repo1-retention-full</br-option> is interpreted; either as the count of full backups to be retained or how many days to retain full backups. New backups must be completed before expiration will occur &amp;mdash; that means if <br-setting>repo1-retention-full-type=count</br-setting> and <br-setting>repo1-retention-full=2</br-setting> then there will be three full backups stored before the oldest one is expired, or if <br-setting>repo1-retention-full-type=time</br-setting> and <br-setting>repo1-retention-full=20</br-setting> then there must be one full backup that is at least 20 days old before expiration can occur.</p>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<backrest-config host="{[host-pg1]}" file="{[backrest-config-demo]}">
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
<title>Configure <br-option>repo1-retention-full</br-option></title>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
<backrest-config-option section="global" key="repo1-retention-full">2</backrest-config-option>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
</backrest-config>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
<p>Backup <br-setting>repo1-retention-full=2</br-setting> but currently there is only one full backup so the next full backup to run will not expire any full backups.</p>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Perform a full backup</title>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} --type=full
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
--log-level-console=detail backup</exe-cmd>
User guide highlighting fixes for #93.
2016-01-22 21:38:48 +02:00
<exe-highlight>archive retention on backup {[backup-full-first]}|remove archive</exe-highlight>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
</execute>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" show="n" variable-key="backup-full-second">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-cmd>{[cmd-backup-last]}</exe-cmd>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
</execute>
</execute-list>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<p>Archive <i>is</i> expired because WAL segments were generated before the oldest backup. These are not useful for recovery &amp;mdash; only WAL segments generated after a backup can be used to recover that backup.</p>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Perform a full backup</title>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} --type=full
Added execution cache for document generation. Added an execution cache so that documentation can be generated without setting up the full container environment. This is useful for packaging, keeps the documentation consistent for a release, and speeds up generation when no changes are made in the execution list.
2016-06-02 15:32:56 +02:00
--log-level-console=info backup</exe-cmd>
Expire continues if an error occurs processing a repository. Errors are logged to the log file rather than thrown. If, after processing all repos, one or more errors occurred, then a single error error will be thrown to indicate there were errors and the log file should be inspected. Also update log messages to be more consistent with new patterns.
2021-02-23 19:20:02 +02:00
<exe-highlight>expire full backup set {[backup-full-first]}|archive retention on backup {[backup-full-second]}|remove archive</exe-highlight>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
</execute>
</execute-list>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<p>The <id>{[backup-full-first]}</id> full backup is expired and archive retention is based on the <id>{[backup-full-second]}</id> which is now the oldest full backup.</p>
</section>
<!-- SECTION => RETENTION - DIFF -->
Improved support for --require option and section depends now default to the previous section.
2017-02-11 17:26:54 +02:00
<section id="diff">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Differential Backup Retention</title>
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
<p>Set <br-option>repo1-retention-diff</br-option> to the number of differential backups required. Differentials only rely on the prior full backup so it is possible to create a <quote>rolling</quote> set of differentials for the last day or more. This allows quick restores to recent points-in-time but reduces overall space consumption.</p>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<backrest-config host="{[host-pg1]}" file="{[backrest-config-demo]}">
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
<title>Configure <br-option>repo1-retention-diff</br-option></title>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
<backrest-config-option section="global" key="repo1-retention-diff">1</backrest-config-option>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
</backrest-config>
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
<p>Backup <br-setting>repo1-retention-diff=1</br-setting> so two differentials will need to be performed before one is expired. An incremental backup is added to demonstrate incremental expiration. Incremental backups cannot be expired independently &amp;mdash; they are always expired with their related full or differential backup.</p>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Perform differential and incremental backups</title>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} --type=diff backup</exe-cmd>
</execute>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" show="n" variable-key="backup-diff-second">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-cmd>{[cmd-backup-last]}</exe-cmd>
</execute>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} --type=incr backup</exe-cmd>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
</execute>
</execute-list>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<p>Now performing a differential backup will expire the previous differential and incremental backups leaving only one differential backup.</p>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Perform a differential backup</title>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} --type=diff
Added execution cache for document generation. Added an execution cache so that documentation can be generated without setting up the full container environment. This is useful for packaging, keeps the documentation consistent for a release, and speeds up generation when no changes are made in the execution list.
2016-06-02 15:32:56 +02:00
--log-level-console=info backup</exe-cmd>
Expire continues if an error occurs processing a repository. Errors are logged to the log file rather than thrown. If, after processing all repos, one or more errors occurred, then a single error error will be thrown to indicate there were errors and the log file should be inspected. Also update log messages to be more consistent with new patterns.
2021-02-23 19:20:02 +02:00
<exe-highlight>expire diff backup set {[backup-diff-second]}</exe-highlight>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
</execute>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
</execute-list>
</section>
Archive retention improvements: * Fixed an issue where retention-archive was not automatically being set when retention-archive-type=diff, resulting in a less aggressive than intended expiration of archive. * Additional warnings when archive retention settings may not have the intended effect or would allow indefinite retention. * Closed #235: "Retention policy question" by adding documentation for archive retention. Contributed by Cynthia Shang.
2016-09-07 14:07:37 +02:00
<!-- SECTION => RETENTION - ARCHIVE -->
Improved support for --require option and section depends now default to the previous section.
2017-02-11 17:26:54 +02:00
<section id="archive">
Archive retention improvements: * Fixed an issue where retention-archive was not automatically being set when retention-archive-type=diff, resulting in a less aggressive than intended expiration of archive. * Additional warnings when archive retention settings may not have the intended effect or would allow indefinite retention. * Closed #235: "Retention policy question" by adding documentation for archive retention. Contributed by Cynthia Shang.
2016-09-07 14:07:37 +02:00
<title>Archive Retention</title>
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
<p>Although <backrest/> automatically removes archived WAL segments when expiring backups (the default expires WAL for full backups based on the <br-option>repo1-retention-full</br-option> option), it may be useful to expire archive more aggressively to save disk space. Note that full backups are treated as differential backups for the purpose of differential archive retention.</p>
Archive retention improvements: * Fixed an issue where retention-archive was not automatically being set when retention-archive-type=diff, resulting in a less aggressive than intended expiration of archive. * Additional warnings when archive retention settings may not have the intended effect or would allow indefinite retention. * Closed #235: "Retention policy question" by adding documentation for archive retention. Contributed by Cynthia Shang.
2016-09-07 14:07:37 +02:00
Add --dry-run option to the expire command. Use dry-run to see which backups/archive would be removed by the expire command without actually removing anything.
2020-03-16 19:56:52 +02:00
<p>Expiring archive will never remove WAL segments that are required to make a backup consistent. However, since Point-in-Time-Recovery (PITR) only works on a continuous WAL stream, care should be taken when aggressively expiring archive outside of the normal backup expiration process. To determine what will be expired without actually expiring anything, the <br-option>dry-run</br-option> option can be provided on the command line with the <cmd>expire</cmd> command.</p>
Archive retention improvements: * Fixed an issue where retention-archive was not automatically being set when retention-archive-type=diff, resulting in a less aggressive than intended expiration of archive. * Additional warnings when archive retention settings may not have the intended effect or would allow indefinite retention. * Closed #235: "Retention policy question" by adding documentation for archive retention. Contributed by Cynthia Shang.
2016-09-07 14:07:37 +02:00
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<backrest-config host="{[host-pg1]}" file="{[backrest-config-demo]}">
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
<title>Configure <br-option>repo1-retention-diff</br-option></title>
Archive retention improvements: * Fixed an issue where retention-archive was not automatically being set when retention-archive-type=diff, resulting in a less aggressive than intended expiration of archive. * Additional warnings when archive retention settings may not have the intended effect or would allow indefinite retention. * Closed #235: "Retention policy question" by adding documentation for archive retention. Contributed by Cynthia Shang.
2016-09-07 14:07:37 +02:00
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
<backrest-config-option section="global" key="repo1-retention-diff">2</backrest-config-option>
Archive retention improvements: * Fixed an issue where retention-archive was not automatically being set when retention-archive-type=diff, resulting in a less aggressive than intended expiration of archive. * Additional warnings when archive retention settings may not have the intended effect or would allow indefinite retention. * Closed #235: "Retention policy question" by adding documentation for archive retention. Contributed by Cynthia Shang.
2016-09-07 14:07:37 +02:00
</backrest-config>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Archive retention improvements: * Fixed an issue where retention-archive was not automatically being set when retention-archive-type=diff, resulting in a less aggressive than intended expiration of archive. * Additional warnings when archive retention settings may not have the intended effect or would allow indefinite retention. * Closed #235: "Retention policy question" by adding documentation for archive retention. Contributed by Cynthia Shang.
2016-09-07 14:07:37 +02:00
<title>Perform differential backup</title>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" show="n" variable-key="backup-diff-first">
Archive retention improvements: * Fixed an issue where retention-archive was not automatically being set when retention-archive-type=diff, resulting in a less aggressive than intended expiration of archive. * Additional warnings when archive retention settings may not have the intended effect or would allow indefinite retention. * Closed #235: "Retention policy question" by adding documentation for archive retention. Contributed by Cynthia Shang.
2016-09-07 14:07:37 +02:00
<exe-cmd>{[cmd-backup-last]}</exe-cmd>
</execute>
PostgreSQL 10 support.
2017-09-01 18:29:34 +02:00
<!-- Push a few WAL segments to make the example below more interesting -->
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" show="n">
Archive retention improvements: * Fixed an issue where retention-archive was not automatically being set when retention-archive-type=diff, resulting in a less aggressive than intended expiration of archive. * Additional warnings when archive retention settings may not have the intended effect or would allow indefinite retention. * Closed #235: "Retention policy question" by adding documentation for archive retention. Contributed by Cynthia Shang.
2016-09-07 14:07:37 +02:00
<exe-cmd>psql -c "
Preliminary documentation for PostgreSQL 11 unprivileged user backup.
2018-06-15 01:05:35 +02:00
select pg_create_restore_point('generate WAL'); select {[pg-switch-wal]}();
select pg_create_restore_point('generate WAL'); select {[pg-switch-wal]}();"</exe-cmd>
Archive retention improvements: * Fixed an issue where retention-archive was not automatically being set when retention-archive-type=diff, resulting in a less aggressive than intended expiration of archive. * Additional warnings when archive retention settings may not have the intended effect or would allow indefinite retention. * Closed #235: "Retention policy question" by adding documentation for archive retention. Contributed by Cynthia Shang.
2016-09-07 14:07:37 +02:00
</execute>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y">
Archive retention improvements: * Fixed an issue where retention-archive was not automatically being set when retention-archive-type=diff, resulting in a less aggressive than intended expiration of archive. * Additional warnings when archive retention settings may not have the intended effect or would allow indefinite retention. * Closed #235: "Retention policy question" by adding documentation for archive retention. Contributed by Cynthia Shang.
2016-09-07 14:07:37 +02:00
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} --type=diff
--log-level-console=info backup</exe-cmd>
<exe-highlight>new backup label</exe-highlight>
</execute>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" show="n" variable-key="backup-diff-second">
Archive retention improvements: * Fixed an issue where retention-archive was not automatically being set when retention-archive-type=diff, resulting in a less aggressive than intended expiration of archive. * Additional warnings when archive retention settings may not have the intended effect or would allow indefinite retention. * Closed #235: "Retention policy question" by adding documentation for archive retention. Contributed by Cynthia Shang.
2016-09-07 14:07:37 +02:00
<exe-cmd>{[cmd-backup-last]}</exe-cmd>
</execute>
</execute-list>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Archive retention improvements: * Fixed an issue where retention-archive was not automatically being set when retention-archive-type=diff, resulting in a less aggressive than intended expiration of archive. * Additional warnings when archive retention settings may not have the intended effect or would allow indefinite retention. * Closed #235: "Retention policy question" by adding documentation for archive retention. Contributed by Cynthia Shang.
2016-09-07 14:07:37 +02:00
<title>Expire archive</title>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y">
Archive retention improvements: * Fixed an issue where retention-archive was not automatically being set when retention-archive-type=diff, resulting in a less aggressive than intended expiration of archive. * Additional warnings when archive retention settings may not have the intended effect or would allow indefinite retention. * Closed #235: "Retention policy question" by adding documentation for archive retention. Contributed by Cynthia Shang.
2016-09-07 14:07:37 +02:00
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} --log-level-console=detail
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
--repo1-retention-archive-type=diff --repo1-retention-archive=1 expire</exe-cmd>
Archive retention improvements: * Fixed an issue where retention-archive was not automatically being set when retention-archive-type=diff, resulting in a less aggressive than intended expiration of archive. * Additional warnings when archive retention settings may not have the intended effect or would allow indefinite retention. * Closed #235: "Retention policy question" by adding documentation for archive retention. Contributed by Cynthia Shang.
2016-09-07 14:07:37 +02:00
<exe-highlight>archive retention on backup {[backup-diff-first]}|remove archive</exe-highlight>
</execute>
</execute-list>
<p>The <id>{[backup-diff-first]}</id> differential backup has archived WAL segments that must be retained to make the older backups consistent even though they cannot be played any further forward with PITR. WAL segments generated after <id>{[backup-diff-first]}</id> but before <id>{[backup-diff-second]}</id> are removed. WAL segments generated after the new backup <id>{[backup-diff-second]}</id> remain and can be used for PITR.</p>
<p>Since full backups are considered differential backups for the purpose of differential archive retention, if a full backup is now performed with the same settings, only the archive for that full backup is retained for PITR.</p>
</section>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
</section>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<!-- SECTION => RESTORE -->
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
<section id="restore" depend="quickstart/perform-backup">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Restore</title>
Multiple repository support. Up to four repositories may be configured. A potential benefit is the ability to have a local repository for fast restores and a remote repository for redundancy. Some commands, e.g. stanza-create/stanza-update, will automatically work with all configured repositories while others, e.g. stanza-delete, will require a repository to be specified using the repo option. See the command reference for details on which commands require the repository to be specified. Note that the repo option is not required when only repo1 is configured in order to maintain backward compatibility. However, the repo option is required when a single repo is configured as, e.g. repo2. This is to prevent command breakage if a new repository is added later. The archive-push command will always push WAL to the archive in all configured repositories but backups will need to be scheduled individually for each repository. In many cases this is desirable since backup types and retention will vary by repository. Likewise, restores must specify a repository. It is generally better to specify a repository for restores that has low latency/cost even if that means more recovery time. Only restore testing can determine which repository will be most efficient. For single repository configurations there should be no change in behavior.
2021-03-08 20:31:13 +02:00
<cmd-description key="restore"/>
<p>The following sections introduce additional <cmd>restore</cmd> command features.</p>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
The restore command is implemented entirely in C. For the most part this is a direct migration of the Perl code into C. There is one important behavioral change with regard to how file permissions are handled. The Perl code tried to set ownership as it was in the manifest even when running as an unprivileged user. This usually just led to errors and frustration. The C code works like this: If a restore is run as a non-root user (the typical scenario) then all files restored will belong to the user/group executing pgBackRest. If existing files are not owned by the executing user/group then an error will result if the ownership cannot be updated to the executing user/group. In that case the file ownership will need to be updated by a privileged user before the restore can be retried. If a restore is run as the root user then pgBackRest will attempt to recreate the ownership recorded in the manifest when the backup was made. Only user/group names are stored in the manifest so the same names must exist on the restore host for this to work. If the user/group name cannot be found locally then the user/group of the PostgreSQL data directory will be used and finally root if the data directory user/group cannot be mapped to a name. Reviewed by Cynthia Shang.
2019-09-26 13:52:02 +02:00
<!-- ******************************************************************************************************************* -->
<section id="ownership">
<title>File Ownership</title>
<p>If a <cmd>restore</cmd> is run as a non-root user (the typical scenario) then all files restored will belong to the user/group executing <backrest/>. If existing files are not owned by the executing user/group then an error will result if the ownership cannot be updated to the executing user/group. In that case the file ownership will need to be updated by a privileged user before the restore can be retried.</p>
<p>If a <cmd>restore</cmd> is run as the <id>root</id> user then <backrest/> will attempt to recreate the ownership recorded in the manifest when the backup was made. Only user/group <b>names</b> are stored in the manifest so the same names must exist on the restore host for this to work. If the user/group name cannot be found locally then the user/group of the <postgres/> data directory will be used and finally <id>root</id> if the data directory user/group cannot be mapped to a name.</p>
</section>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<!-- SECTION => RESTORE - DELTA -->
<section id="option-delta">
<title>Delta Option</title>
Converted Perl threads to processes.
2016-09-06 15:35:02 +02:00
<p><link section="/quickstart/perform-restore">Restore a Backup</link> in <link section="/quickstart">Quick Start</link> required the database cluster directory to be cleaned before the <cmd>restore</cmd> could be performed. The <br-option>delta</br-option> option allows <backrest/> to automatically determine which files in the database cluster directory can be preserved and which ones need to be restored from the backup &amp;mdash; it also <i>removes</i> files not present in the backup manifest so it will dispose of divergent changes. This is accomplished by calculating a <link url="https://en.wikipedia.org/wiki/SHA-1">SHA-1</link> cryptographic hash for each file in the database cluster directory. If the <id>SHA-1</id> hash does not match the hash stored in the backup then that file will be restored. This operation is very efficient when combined with the <br-option>process-max</br-option> option. Since the <postgres/> server is shut down during the restore, a larger number of processes can be used than might be desirable during a backup when the <postgres/> server is running.</p>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Stop the {[postgres-cluster-demo]} cluster, perform delta restore</title>
<execute user="root">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<exe-cmd>{[pg-cluster-stop]}</exe-cmd>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
</execute>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y" filter="y">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} {[dash]}-delta
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
--log-level-console=detail restore</exe-cmd>
The restore command is implemented entirely in C. For the most part this is a direct migration of the Perl code into C. There is one important behavioral change with regard to how file permissions are handled. The Perl code tried to set ownership as it was in the manifest even when running as an unprivileged user. This usually just led to errors and frustration. The C code works like this: If a restore is run as a non-root user (the typical scenario) then all files restored will belong to the user/group executing pgBackRest. If existing files are not owned by the executing user/group then an error will result if the ownership cannot be updated to the executing user/group. In that case the file ownership will need to be updated by a privileged user before the restore can be retried. If a restore is run as the root user then pgBackRest will attempt to recreate the ownership recorded in the manifest when the backup was made. Only user/group names are stored in the manifest so the same names must exist on the restore host for this to work. If the user/group name cannot be found locally then the user/group of the PostgreSQL data directory will be used and finally root if the data directory user/group cannot be mapped to a name. Reviewed by Cynthia Shang.
2019-09-26 13:52:02 +02:00
<exe-highlight>demo\/PG_VERSION - exists and matches backup|remove invalid files|rename global\/pg_control</exe-highlight>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
</execute>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
</execute-list>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Restart <postgres/></title>
<execute user="root">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<exe-cmd>{[pg-cluster-start]}</exe-cmd>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
</execute>
Closed #163: Document Generation Issue Fixed an issue where document generation failed because some OSs are not tolerant of having multiple installed versions of PostgreSQL. A separate VM is now created for each version. Also added a sleep after database starts during document generation to ensure the database is running before the next command runs. Reported by John Harvey.
2016-01-23 01:01:21 +02:00
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" show="n">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<exe-cmd>{[pg-cluster-wait]}</exe-cmd>
Closed #163: Document Generation Issue Fixed an issue where document generation failed because some OSs are not tolerant of having multiple installed versions of PostgreSQL. A separate VM is now created for each version. Also added a sleep after database starts during document generation to ensure the database is running before the next command runs. Reported by John Harvey.
2016-01-23 01:01:21 +02:00
</execute>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
</execute-list>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
</section>
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
<!-- SECTION => RESTORE - DELTA -->
<section id="option-db-include">
<title>Restore Selected Databases</title>
<p>There may be cases where it is desirable to selectively restore specific databases from a cluster backup. This could be done for performance reasons or to move selected databases to a machine that does not have enough space to restore the entire cluster backup.</p>
Update selective restore documentation with caveats. Recovery may error unless --type=immediate is specified. This is because after consistency is reached PostgreSQL will flag zeroed pages as errors even for a full-page write. For PostgreSQL ≥ 13 the ignore_invalid_pages setting may be used to ignore invalid pages. In this case it is important to check the logs after recovery to ensure that no invalid pages were reported in the selected databases.
2021-03-11 17:19:50 +02:00
<p>To demonstrate this feature two databases are created: test1 and test2.</p>
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Update selective restore documentation with caveats. Recovery may error unless --type=immediate is specified. This is because after consistency is reached PostgreSQL will flag zeroed pages as errors even for a full-page write. For PostgreSQL ≥ 13 the ignore_invalid_pages setting may be used to ignore invalid pages. In this case it is important to check the logs after recovery to ensure that no invalid pages were reported in the selected databases.
2021-03-11 17:19:50 +02:00
<title>Create two test databases</title>
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y" filter="n">
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
<exe-cmd>
psql -c "create database test1;"
</exe-cmd>
</execute>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y" filter="n">
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
<exe-cmd>
psql -c "create database test2;"
</exe-cmd>
</execute>
</execute-list>
<p>Each test database will be seeded with tables and data to demonstrate that recovery works with selective restore.</p>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
<title>Create a test table in each database</title>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y" filter="n">
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
<exe-cmd>
psql -c "create table test1_table (id int);
insert into test1_table (id) values (1);" test1
</exe-cmd>
</execute>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y" filter="n">
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
<exe-cmd>
psql -c "create table test2_table (id int);
insert into test2_table (id) values (2);" test2
</exe-cmd>
</execute>
</execute-list>
Update selective restore documentation with caveats. Recovery may error unless --type=immediate is specified. This is because after consistency is reached PostgreSQL will flag zeroed pages as errors even for a full-page write. For PostgreSQL ≥ 13 the ignore_invalid_pages setting may be used to ignore invalid pages. In this case it is important to check the logs after recovery to ensure that no invalid pages were reported in the selected databases.
2021-03-11 17:19:50 +02:00
<p>A fresh backup is run so <backrest/> is aware of the new databases.</p>
<execute-list host="{[host-pg1]}">
<title>Perform a backup</title>
<execute user="postgres">
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} --type=incr backup</exe-cmd>
</execute>
</execute-list>
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
<p>One of the main reasons to use selective restore is to save space. The size of the test1 database is shown here so it can be compared with the disk utilization after a selective restore.</p>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
<title>Show space used by test1 database</title>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" show="n" variable-key="database-test1-oid">
Add logic to find the real oid of the test1 database during restore testing.
2018-04-13 18:00:22 +02:00
<exe-cmd>
psql -Atc "select oid from pg_database where datname = 'test1'"
</exe-cmd>
</execute>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y" filter="n">
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
<exe-cmd>
Add logic to find the real oid of the test1 database during restore testing.
2018-04-13 18:00:22 +02:00
du -sh {[pg-path]}/base/{[database-test1-oid]}
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
</exe-cmd>
</execute>
</execute-list>
Add info command set option for detailed text output. The additional details include databases that can be used for selective restore and a list of tablespaces and symlinks with their default destinations. This information is not included in the JSON output because it requires reading the manifest which is too IO intensive to do for all manifests. We plan to include this information for JSON in a future release.
2019-09-30 18:39:38 +02:00
<p>If the database to restore is not known, use the <cmd>info</cmd> command <br-option>set</br-option> option to discover databases that are part of the backup set.</p>
<execute-list host="{[host-pg1]}">
<title>Show database list for backup</title>
<execute user="postgres" show="n" variable-key="backup-last-incr">
<exe-cmd>{[cmd-backup-last]}</exe-cmd>
</execute>
<execute user="postgres">
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]}
{[dash]}-set={[backup-last-incr]} info</exe-cmd>
<exe-highlight>database list</exe-highlight>
</execute>
</execute-list>
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
<p>Stop the cluster and restore only the test2 database. Built-in databases (<id>template0</id>, <id>template1</id>, and <id>postgres</id>) are always restored.</p>
Update selective restore documentation with caveats. Recovery may error unless --type=immediate is specified. This is because after consistency is reached PostgreSQL will flag zeroed pages as errors even for a full-page write. For PostgreSQL ≥ 13 the ignore_invalid_pages setting may be used to ignore invalid pages. In this case it is important to check the logs after recovery to ensure that no invalid pages were reported in the selected databases.
2021-03-11 17:19:50 +02:00
<admonition type="warning">Recovery may error unless <br-option>--type=immediate</br-option> is specified. This is because after consistency is reached <postgres/> will flag zeroed pages as errors even for a full-page write. For <postgres/> &amp;ge; <proper>13</proper> the <pg-option>ignore_invalid_pages</pg-option> setting may be used to ignore invalid pages. In this case it is important to check the logs after recovery to ensure that no invalid pages were reported in the selected databases.</admonition>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
<title>Restore from last backup including only the test2 database</title>
<execute user="root">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<exe-cmd>{[pg-cluster-stop]}</exe-cmd>
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
</execute>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres">
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} {[dash]}-delta
Update selective restore documentation with caveats. Recovery may error unless --type=immediate is specified. This is because after consistency is reached PostgreSQL will flag zeroed pages as errors even for a full-page write. For PostgreSQL ≥ 13 the ignore_invalid_pages setting may be used to ignore invalid pages. In this case it is important to check the logs after recovery to ensure that no invalid pages were reported in the selected databases.
2021-03-11 17:19:50 +02:00
{[dash]}-db-include=test2 {[dash]}-type=immediate {[dash]}-target-action=promote restore</exe-cmd>
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
</execute>
<execute user="root">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<exe-cmd>{[pg-cluster-start]}</exe-cmd>
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
</execute>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" show="n">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<exe-cmd>{[pg-cluster-wait]}</exe-cmd>
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
</execute>
</execute-list>
<p>Once recovery is complete the test2 database will contain all previously created tables and data.</p>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
<title>Demonstrate that the test2 database was recovered</title>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y" filter="n">
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
<exe-cmd>
psql -c "select * from test2_table;" test2
</exe-cmd>
</execute>
</execute-list>
<p>The test1 database, despite successful recovery, is not accessible. This is because the entire database was restored as sparse, zeroed files. <postgres/> can successfully apply WAL on the zeroed files but the database as a whole will not be valid because key files contain no data. This is purposeful to prevent the database from being accidentally used when it might contain partial data that was applied during WAL replay.</p>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
<title>Attempting to connect to the test1 database will produce an error</title>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y" filter="n" err-expect="2">
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
<exe-cmd>
psql -c "select * from test1_table;" test1
</exe-cmd>
<exe-highlight>relation mapping file.*contains invalid data</exe-highlight>
</execute>
</execute-list>
<p>Since the test1 database is restored with sparse, zeroed files it will only require as much space as the amount of WAL that is written during recovery. While the amount of WAL generated during a backup and applied during recovery can be significant it will generally be a small fraction of the total database size, especially for large databases where this feature is most likely to be useful.</p>
<p>It is clear that the test1 database uses far less disk space during the selective restore than it would have if the entire database had been restored.</p>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
<title>Show space used by test1 database after recovery</title>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y" filter="n">
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
<exe-cmd>
Add logic to find the real oid of the test1 database during restore testing.
2018-04-13 18:00:22 +02:00
du -sh {[pg-path]}/base/{[database-test1-oid]}
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
</exe-cmd>
</execute>
</execute-list>
<p>At this point the only action that can be taken on the invalid test1 database is <id>drop database</id>. <backrest/> does not automatically drop the database since this cannot be done until recovery is complete and the cluster is accessible.</p>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
<title>Drop the test1 database</title>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y" filter="n">
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
<exe-cmd>
psql -c "drop database test1;"
</exe-cmd>
</execute>
</execute-list>
<p>Now that the invalid test1 database has been dropped only the test2 and built-in databases remain.</p>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
<title>List remaining databases</title>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y" filter="n">
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
<exe-cmd>
psql -c "select oid, datname from pg_database order by oid;"
</exe-cmd>
<exe-highlight>test2</exe-highlight>
</execute>
</execute-list>
</section>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
</section>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<!-- SECTION => PITR -->
Enhanced text output of info command to include timestamps, sizes, and the reference list for all backups.
2016-05-06 16:25:46 +02:00
<section id="pitr" depend="quickstart/perform-backup">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Point-in-Time Recovery</title>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Allow restore --type=lsn. The LSN recovery target was added in PostgreSQL 10 but did not get added to pgBackRest.
2021-07-15 19:02:41 +02:00
<p><link section="/quickstart/perform-restore">Restore a Backup</link> in <link section="/quickstart">Quick Start</link> performed default recovery, which is to play all the way to the end of the WAL stream. In the case of a hardware failure this is usually the best choice but for data corruption scenarios (whether machine or human in origin) Point-in-Time Recovery (PITR) is often more appropriate.</p>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Allow restore --type=lsn. The LSN recovery target was added in PostgreSQL 10 but did not get added to pgBackRest.
2021-07-15 19:02:41 +02:00
<p>Point-in-Time Recovery (PITR) allows the WAL to be played from the last backup to a specified lsn, time, transaction id, or recovery point. For common recovery scenarios time-based recovery is arguably the most useful. A typical recovery scenario is to restore a table that was accidentally dropped or data that was accidentally deleted. Recovering a dropped table is more dramatic so that's the example given here but deleted data would be recovered in exactly the same way.</p>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Backup the {[postgres-cluster-demo]} cluster and create a table with very important data</title>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} --type=diff backup</exe-cmd>
</execute>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-cmd>
psql -c "begin;
create table important_table (message text);
insert into important_table values ('{[test-table-data]}');
commit;
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
select * from important_table;"
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
</exe-cmd>
<exe-highlight>{[test-table-data]}</exe-highlight>
</execute>
</execute-list>
<p>It is important to represent the time as reckoned by <postgres/> and to include timezone offsets. This reduces the possibility of unintended timezone conversions and an unexpected recovery result.</p>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Get the time from <postgres/></title>
Add sleep before/after retrieving timestamp in the user guide. Adding a sleep before was necessary since only adding a sleep after did not always work. This helps to ensure the backup stop time for the previous backup does not equal time-recovery-timestamp. The sleep after allows enough time between the time retrieval and dropping important_table so PostgreSQL can consistently recover to before the table drop. Note that these issues were caused by picking a timestamp too close to the restore command or a database operation, not due to any problem in backup selection of the restore command.
2020-02-28 21:30:39 +02:00
<execute user="postgres" show="n">
<exe-cmd>sleep 1</exe-cmd>
</execute>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y" filter="n" variable-key="time-recovery-timestamp">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-cmd>
psql -Atc "select current_timestamp"
</exe-cmd>
</execute>
Add sleep before/after retrieving timestamp in the user guide. Adding a sleep before was necessary since only adding a sleep after did not always work. This helps to ensure the backup stop time for the previous backup does not equal time-recovery-timestamp. The sleep after allows enough time between the time retrieval and dropping important_table so PostgreSQL can consistently recover to before the table drop. Note that these issues were caused by picking a timestamp too close to the restore command or a database operation, not due to any problem in backup selection of the restore command.
2020-02-28 21:30:39 +02:00
<execute user="postgres" show="n">
<exe-cmd>sleep 1</exe-cmd>
</execute>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
</execute-list>
<p>Now that the time has been recorded the table is dropped. In practice finding the exact time that the table was dropped is a lot harder than in this example. It may not be possible to find the exact time, but some forensic work should be able to get you close.</p>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Drop the important table</title>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y" err-expect="1">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-cmd>psql -c "begin;
drop table important_table;
commit;
select * from important_table;"</exe-cmd>
<exe-highlight>does not exist</exe-highlight>
</execute>
</execute-list>
<p>Now the restore can be performed with time-based recovery to bring back the missing table.</p>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Update recovery.conf in user guide for PostgreSQL >= 12. The postgresql.auto.conf file was being used instead of recovery.conf, but there were still instances in the text that used recovery.conf. Update to postgresql.auto.conf for PostgreSQL >= 10 and change wording where needed.
2020-07-17 19:27:14 +02:00
<title>Stop <postgres/>, restore the {[postgres-cluster-demo]} cluster to <id>{[time-recovery-timestamp]}</id>, and display <file>{[pg-recovery-file-demo]}</file></title>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<execute user="root">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<exe-cmd>{[pg-cluster-stop]}</exe-cmd>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
</execute>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} {[dash]}-delta
Documentation builds on PostgreSQL 9.4-10. More than likely 9.2-11 will work as well, but this has not been tested. However, 11 needs work on the group permissions introduced in that version.
2019-03-01 09:00:51 +02:00
{[dash]}-type=time "{[dash]}-target={[time-recovery-timestamp]}"
--target-action=promote restore</exe-cmd>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
</execute>
<execute user="root" show="n">
<exe-cmd>rm {[postgres-log-demo]}</exe-cmd>
</execute>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y">
Update recovery.conf in user guide for PostgreSQL >= 12. The postgresql.auto.conf file was being used instead of recovery.conf, but there were still instances in the text that used recovery.conf. Update to postgresql.auto.conf for PostgreSQL >= 10 and change wording where needed.
2020-07-17 19:27:14 +02:00
<exe-cmd>cat {[pg-recovery-path-demo]}</exe-cmd>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-highlight>recovery_target_time</exe-highlight>
</execute>
</execute-list>
Update reference to include links to user guide examples. The command-example and command-example-list elements were removed from the documentation rendering some time ago so these tags were dead code. The tags, however, contained some examples and information that were pertinent to the command, so where possible, the information was included in the description of the command and/or the user-guide and links to the relevant user guide sections were added. Note that some commands could not be updated with user guide references since doing so would cause a cyclical reference in the user guide. These commands have an internal comment to indicate this. In addition, some clarifications were added (e.g. expire --set option) where information was lacking.
2021-03-31 15:36:56 +02:00
<p><backrest/> has automatically generated the recovery settings in <file>{[pg-recovery-file-demo]}</file> so <postgres/> can be started immediately. <id>%f</id> is how <postgres/> specifies the WAL segment it needs and <id>%p</id> is the location where it should be copied. Once <postgres/> has finished recovery the table will exist again and can be queried.</p>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Start <postgres/> and check that the important table exists</title>
<execute user="root">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<exe-cmd>{[pg-cluster-start]}</exe-cmd>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
</execute>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" show="n">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<exe-cmd>{[pg-cluster-wait]}</exe-cmd>
Closed #163: Document Generation Issue Fixed an issue where document generation failed because some OSs are not tolerant of having multiple installed versions of PostgreSQL. A separate VM is now created for each version. Also added a sleep after database starts during document generation to ensure the database is running before the next command runs. Reported by John Harvey.
2016-01-23 01:01:21 +02:00
</execute>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-cmd>psql -c "select * from important_table"</exe-cmd>
<exe-highlight>{[test-table-data]}</exe-highlight>
</execute>
</execute-list>
<p>The <postgres/> log also contains valuable information. It will indicate the time and transaction where the recovery stopped and also give the time of the last transaction to be applied.</p>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Examine the <postgres/> log output</title>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-cmd>cat {[postgres-log-demo]}</exe-cmd>
<exe-highlight>recovery stopping before|last completed transaction|starting point-in-time recovery</exe-highlight>
</execute>
</execute-list>
<p>This example was rigged to give the correct result. If a backup after the required time is chosen then <postgres/> will not be able to recover the lost table. <postgres/> can only play forward, not backward. To demonstrate this the important table must be dropped (again).</p>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Drop the important table (again)</title>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y" err-expect="1">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-cmd>psql -c "begin;
drop table important_table;
commit;
select * from important_table;"</exe-cmd>
<exe-highlight>does not exist</exe-highlight>
</execute>
</execute-list>
Auto-select backup set on restore when time target is specified. Auto-selection is performed only when --set is not specified. If a backup set for the given target time cannot not be found, the latest (default) backup set will be used. Currently a limited number of date formats are recognized and timezone names are not allowed, only timezone offsets.
2020-01-30 23:38:05 +02:00
<p>Now take a new backup and attempt recovery from the new backup by specifying the <br-option>{[dash]}-set</br-option> option. The <cmd>info</cmd> command can be used to find the new backup label.</p>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Auto-select backup set on restore when time target is specified. Auto-selection is performed only when --set is not specified. If a backup set for the given target time cannot not be found, the latest (default) backup set will be used. Currently a limited number of date formats are recognized and timezone names are not allowed, only timezone offsets.
2020-01-30 23:38:05 +02:00
<title>Perform a backup and get backup info</title>
<execute user="postgres">
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} {[dash]}-type=incr backup</exe-cmd>
</execute>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" show="n" variable-key="backup-last">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-cmd>{[cmd-backup-last]}</exe-cmd>
</execute>
Auto-select backup set on restore when time target is specified. Auto-selection is performed only when --set is not specified. If a backup set for the given target time cannot not be found, the latest (default) backup set will be used. Currently a limited number of date formats are recognized and timezone names are not allowed, only timezone offsets.
2020-01-30 23:38:05 +02:00
<execute user="postgres" filter="n" output="y">
<exe-cmd>{[project-exe]} info</exe-cmd>
<exe-highlight>{[backup-last]}</exe-highlight>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
</execute>
Auto-select backup set on restore when time target is specified. Auto-selection is performed only when --set is not specified. If a backup set for the given target time cannot not be found, the latest (default) backup set will be used. Currently a limited number of date formats are recognized and timezone names are not allowed, only timezone offsets.
2020-01-30 23:38:05 +02:00
</execute-list>
<execute-list host="{[host-pg1]}">
<title>Attempt recovery from the specified backup</title>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<execute user="root">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<exe-cmd>{[pg-cluster-stop]}</exe-cmd>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
</execute>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} {[dash]}-delta
Auto-select backup set on restore when time target is specified. Auto-selection is performed only when --set is not specified. If a backup set for the given target time cannot not be found, the latest (default) backup set will be used. Currently a limited number of date formats are recognized and timezone names are not allowed, only timezone offsets.
2020-01-30 23:38:05 +02:00
{[dash]}-set={[backup-last]}
{[dash]}-type=time "{[dash]}-target={[time-recovery-timestamp]}" {[dash]}-target-action=promote restore</exe-cmd>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
</execute>
<execute user="root" show="n">
<exe-cmd>rm {[postgres-log-demo]}</exe-cmd>
</execute>
<execute user="root">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<exe-cmd>{[pg-cluster-start]}</exe-cmd>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
</execute>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" show="n">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<exe-cmd>{[pg-cluster-wait]}</exe-cmd>
Closed #163: Document Generation Issue Fixed an issue where document generation failed because some OSs are not tolerant of having multiple installed versions of PostgreSQL. A separate VM is now created for each version. Also added a sleep after database starts during document generation to ensure the database is running before the next command runs. Reported by John Harvey.
2016-01-23 01:01:21 +02:00
</execute>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y" err-expect="1">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-cmd>psql -c "select * from important_table"</exe-cmd>
<exe-highlight>does not exist</exe-highlight>
</execute>
</execute-list>
<p>Looking at the log output it's not obvious that recovery failed to restore the table. The key is to look for the presence of the <quote>recovery stopping before...</quote> and <quote>last completed transaction...</quote> log messages. If they are not present then the recovery to the specified point-in-time was not successful.</p>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Examine the <postgres/> log output to discover the recovery was not successful</title>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-cmd>cat {[postgres-log-demo]}</exe-cmd>
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
<exe-highlight>starting point-in-time recovery|consistent recovery state reached</exe-highlight>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
</execute>
</execute-list>
Auto-select backup set on restore when time target is specified. Auto-selection is performed only when --set is not specified. If a backup set for the given target time cannot not be found, the latest (default) backup set will be used. Currently a limited number of date formats are recognized and timezone names are not allowed, only timezone offsets.
2020-01-30 23:38:05 +02:00
<p>The default behavior for time-based restore, if the <br-option>{[dash]}-set</br-option> option is not specified, is to attempt to discover an earlier backup to play forward from. If a backup set cannot be found, then restore will default to the latest backup which, as shown earlier, may not give the desired result.</p>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Auto-select backup set on restore when time target is specified. Auto-selection is performed only when --set is not specified. If a backup set for the given target time cannot not be found, the latest (default) backup set will be used. Currently a limited number of date formats are recognized and timezone names are not allowed, only timezone offsets.
2020-01-30 23:38:05 +02:00
<title>Stop <postgres/>, restore from auto-selected backup, and start <postgres/></title>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<execute user="root">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<exe-cmd>{[pg-cluster-stop]}</exe-cmd>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
</execute>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-cmd>
{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} {[dash]}-delta
{[dash]}-type=time "{[dash]}-target={[time-recovery-timestamp]}"
Auto-select backup set on restore when time target is specified. Auto-selection is performed only when --set is not specified. If a backup set for the given target time cannot not be found, the latest (default) backup set will be used. Currently a limited number of date formats are recognized and timezone names are not allowed, only timezone offsets.
2020-01-30 23:38:05 +02:00
{[dash]}-target-action=promote restore
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
</exe-cmd>
</execute>
<execute user="root" show="n">
<exe-cmd>rm {[postgres-log-demo]}</exe-cmd>
</execute>
<execute user="root">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<exe-cmd>{[pg-cluster-start]}</exe-cmd>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
</execute>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" show="n">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<exe-cmd>{[pg-cluster-wait]}</exe-cmd>
Closed #163: Document Generation Issue Fixed an issue where document generation failed because some OSs are not tolerant of having multiple installed versions of PostgreSQL. A separate VM is now created for each version. Also added a sleep after database starts during document generation to ensure the database is running before the next command runs. Reported by John Harvey.
2016-01-23 01:01:21 +02:00
</execute>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-cmd>psql -c "select * from important_table"</exe-cmd>
<exe-highlight>{[test-table-data]}</exe-highlight>
</execute>
</execute-list>
Fix documentation and comment typos. Identified using `ag -l | igor`.
2021-07-01 17:50:03 +02:00
<p>Now the log output will contain the expected <quote>recovery stopping before...</quote> and <quote>last completed transaction...</quote> messages showing that the recovery was successful.</p>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Examine the <postgres/> log output for log messages indicating success</title>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-cmd>cat {[postgres-log-demo]}</exe-cmd>
<exe-highlight>recovery stopping before|last completed transaction|starting point-in-time recovery</exe-highlight>
</execute>
</execute-list>
</section>
Azure support for repository storage. Azure and Azure-compatible object stores can now be used for repository storage. Currently only shared key authentication is supported but SAS will be added soon.
2020-07-02 22:24:34 +02:00
<!-- ======================================================================================================================= -->
Multiple repository support. Up to four repositories may be configured. A potential benefit is the ability to have a local repository for fast restores and a remote repository for redundancy. Some commands, e.g. stanza-create/stanza-update, will automatically work with all configured repositories while others, e.g. stanza-delete, will require a repository to be specified using the repo option. See the command reference for details on which commands require the repository to be specified. Note that the repo option is not required when only repo1 is configured in order to maintain backward compatibility. However, the repo option is required when a single repo is configured as, e.g. repo2. This is to prevent command breakage if a new repository is added later. The archive-push command will always push WAL to the archive in all configured repositories but backups will need to be scheduled individually for each repository. In many cases this is desirable since backup types and retention will vary by repository. Likewise, restores must specify a repository. It is generally better to specify a repository for restores that has low latency/cost even if that means more recovery time. Only restore testing can determine which repository will be most efficient. For single repository configurations there should be no change in behavior.
2021-03-08 20:31:13 +02:00
<section id="multi-repo" if="!{[object-any-all]}">
<title>Multiple Repositories</title>
<p>Multiple repositories may be configured as demonstrated in <link section="/s3-support">S3 Support</link>. A potential benefit is the ability to have a local repository for fast restores and a remote repository for redundancy.</p>
<p>Some commands, e.g. <cmd>stanza-create</cmd>/<cmd>stanza-update</cmd>, will automatically work with all configured repositories while others, e.g. <link section="/delete-stanza">stanza-delete</link>, will require a repository to be specified using the <br-option>repo</br-option> option. See the <link url="command.html">command reference</link> for details on which commands require the repository to be specified.</p>
<p>Note that the <br-option>repo</br-option> option is not required when only <br-option>repo1</br-option> is configured in order to maintain backward compatibility. However, the <br-option>repo</br-option> option <i>is</i> required when a single repo is configured as, e.g. <br-option>repo2</br-option>. This is to prevent command breakage if a new repository is added later.</p>
<p>The <cmd>archive-push</cmd> command will always push WAL to the archive in all configured repositories but backups will need to be scheduled individually for each repository. In many cases this is desirable since backup types and retention will vary by repository. Likewise, restores must specify a repository. It is generally better to specify a repository for restores that has low latency/cost even if that means more recovery time. Only restore testing can determine which repository will be most efficient.</p>
</section>
<!-- ======================================================================================================================= -->
<section id="azure-support" if="!{[object-any-all]}" depend="/quickstart/configure-archiving">
Azure support for repository storage. Azure and Azure-compatible object stores can now be used for repository storage. Currently only shared key authentication is supported but SAS will be added soon.
2020-07-02 22:24:34 +02:00
<title>Azure-Compatible Object Store Support</title>
<block id="azure-setup">
Multiple repository support. Up to four repositories may be configured. A potential benefit is the ability to have a local repository for fast restores and a remote repository for redundancy. Some commands, e.g. stanza-create/stanza-update, will automatically work with all configured repositories while others, e.g. stanza-delete, will require a repository to be specified using the repo option. See the command reference for details on which commands require the repository to be specified. Note that the repo option is not required when only repo1 is configured in order to maintain backward compatibility. However, the repo option is required when a single repo is configured as, e.g. repo2. This is to prevent command breakage if a new repository is added later. The archive-push command will always push WAL to the archive in all configured repositories but backups will need to be scheduled individually for each repository. In many cases this is desirable since backup types and retention will vary by repository. Likewise, restores must specify a repository. It is generally better to specify a repository for restores that has low latency/cost even if that means more recovery time. Only restore testing can determine which repository will be most efficient. For single repository configurations there should be no change in behavior.
2021-03-08 20:31:13 +02:00
<block-variable-replace key="azure-setup-repo-id">2</block-variable-replace>
Azure support for repository storage. Azure and Azure-compatible object stores can now be used for repository storage. Currently only shared key authentication is supported but SAS will be added soon.
2020-07-02 22:24:34 +02:00
<block-variable-replace key="azure-setup-host">{[host-pg1]}</block-variable-replace>
<block-variable-replace key="azure-setup-user">postgres</block-variable-replace>
<block-variable-replace key="azure-setup-config-owner">postgres:postgres</block-variable-replace>
<block-variable-replace key="azure-setup-create-container">y</block-variable-replace>
</block>
<p>Commands are run exactly as if the repository were stored on a local disk.</p>
<execute-list host="{[host-pg1]}">
<title>Create the stanza</title>
<execute user="postgres" output="y">
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} {[dash]}-log-level-console=info stanza-create</exe-cmd>
<exe-highlight>completed successfully</exe-highlight>
</execute>
</execute-list>
<p>File creation time in object stores is relatively slow so commands benefit by increasing <br-option>process-max</br-option> to parallelize file creation.</p>
<execute-list host="{[host-pg1]}">
<title>Backup the {[postgres-cluster-demo]} cluster</title>
<execute user="postgres" output="y">
Multiple repository support. Up to four repositories may be configured. A potential benefit is the ability to have a local repository for fast restores and a remote repository for redundancy. Some commands, e.g. stanza-create/stanza-update, will automatically work with all configured repositories while others, e.g. stanza-delete, will require a repository to be specified using the repo option. See the command reference for details on which commands require the repository to be specified. Note that the repo option is not required when only repo1 is configured in order to maintain backward compatibility. However, the repo option is required when a single repo is configured as, e.g. repo2. This is to prevent command breakage if a new repository is added later. The archive-push command will always push WAL to the archive in all configured repositories but backups will need to be scheduled individually for each repository. In many cases this is desirable since backup types and retention will vary by repository. Likewise, restores must specify a repository. It is generally better to specify a repository for restores that has low latency/cost even if that means more recovery time. Only restore testing can determine which repository will be most efficient. For single repository configurations there should be no change in behavior.
2021-03-08 20:31:13 +02:00
<exe-cmd>
{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} --repo=2
--log-level-console=info backup
</exe-cmd>
Azure support for repository storage. Azure and Azure-compatible object stores can now be used for repository storage. Currently only shared key authentication is supported but SAS will be added soon.
2020-07-02 22:24:34 +02:00
<exe-highlight>no prior backup exists|full backup size</exe-highlight>
</execute>
</execute-list>
</section>
S3 repository support.
2017-06-12 16:52:32 +02:00
<!-- SECTION => S3-SUPPORT -->
Multiple repository support. Up to four repositories may be configured. A potential benefit is the ability to have a local repository for fast restores and a remote repository for redundancy. Some commands, e.g. stanza-create/stanza-update, will automatically work with all configured repositories while others, e.g. stanza-delete, will require a repository to be specified using the repo option. See the command reference for details on which commands require the repository to be specified. Note that the repo option is not required when only repo1 is configured in order to maintain backward compatibility. However, the repo option is required when a single repo is configured as, e.g. repo2. This is to prevent command breakage if a new repository is added later. The archive-push command will always push WAL to the archive in all configured repositories but backups will need to be scheduled individually for each repository. In many cases this is desirable since backup types and retention will vary by repository. Likewise, restores must specify a repository. It is generally better to specify a repository for restores that has low latency/cost even if that means more recovery time. Only restore testing can determine which repository will be most efficient. For single repository configurations there should be no change in behavior.
2021-03-08 20:31:13 +02:00
<section id="s3-support" if="!{[object-any-all]}" depend="/azure-support">
Clarify that S3-compatible object stores are supported. The documentation mentioned Amazon S3 frequently but failed to mention that other S3-compatible object stores are also supported. Tone down the specific mentions of Amazon S3 and replace them with "S3-compatible object store" when appropriate. Suggested by Magnus Hagander.
2019-04-03 12:40:12 +02:00
<title>S3-Compatible Object Store Support</title>
S3 repository support.
2017-06-12 16:52:32 +02:00
Build all docs with S3 using --var=s3-all=y Force repo-type=s3 for all tests. This is not currently the default for any OS builds.
2019-05-29 14:38:45 +02:00
<block id="s3-setup">
Multiple repository support. Up to four repositories may be configured. A potential benefit is the ability to have a local repository for fast restores and a remote repository for redundancy. Some commands, e.g. stanza-create/stanza-update, will automatically work with all configured repositories while others, e.g. stanza-delete, will require a repository to be specified using the repo option. See the command reference for details on which commands require the repository to be specified. Note that the repo option is not required when only repo1 is configured in order to maintain backward compatibility. However, the repo option is required when a single repo is configured as, e.g. repo2. This is to prevent command breakage if a new repository is added later. The archive-push command will always push WAL to the archive in all configured repositories but backups will need to be scheduled individually for each repository. In many cases this is desirable since backup types and retention will vary by repository. Likewise, restores must specify a repository. It is generally better to specify a repository for restores that has low latency/cost even if that means more recovery time. Only restore testing can determine which repository will be most efficient. For single repository configurations there should be no change in behavior.
2021-03-08 20:31:13 +02:00
<block-variable-replace key="s3-setup-repo-id">3</block-variable-replace>
Build all docs with S3 using --var=s3-all=y Force repo-type=s3 for all tests. This is not currently the default for any OS builds.
2019-05-29 14:38:45 +02:00
<block-variable-replace key="s3-setup-host">{[host-pg1]}</block-variable-replace>
Add repo-create, repo-get, repo-put, and repo-rm commands. These commands are generally useful but more importantly they allow removing LibC by providing the Perl integration tests an alternate way to work with repository storage. All the commands are currently internal only and should not be used on production repositories.
2020-03-09 23:15:03 +02:00
<block-variable-replace key="s3-setup-user">postgres</block-variable-replace>
Build all docs with S3 using --var=s3-all=y Force repo-type=s3 for all tests. This is not currently the default for any OS builds.
2019-05-29 14:38:45 +02:00
<block-variable-replace key="s3-setup-config-owner">postgres:postgres</block-variable-replace>
<block-variable-replace key="s3-setup-create-bucket">y</block-variable-replace>
</block>
Clarify that region/endpoint must be configured correctly for the bucket. Suggested by Pritam Barhate.
2018-11-30 18:15:52 +02:00
Multiple repository support. Up to four repositories may be configured. A potential benefit is the ability to have a local repository for fast restores and a remote repository for redundancy. Some commands, e.g. stanza-create/stanza-update, will automatically work with all configured repositories while others, e.g. stanza-delete, will require a repository to be specified using the repo option. See the command reference for details on which commands require the repository to be specified. Note that the repo option is not required when only repo1 is configured in order to maintain backward compatibility. However, the repo option is required when a single repo is configured as, e.g. repo2. This is to prevent command breakage if a new repository is added later. The archive-push command will always push WAL to the archive in all configured repositories but backups will need to be scheduled individually for each repository. In many cases this is desirable since backup types and retention will vary by repository. Likewise, restores must specify a repository. It is generally better to specify a repository for restores that has low latency/cost even if that means more recovery time. Only restore testing can determine which repository will be most efficient. For single repository configurations there should be no change in behavior.
2021-03-08 20:31:13 +02:00
<p>A role should be created to run <backrest/> and the bucket permissions should be set as restrictively as possible. If the role is associated with an instance in <proper>AWS</proper> then <backrest/> will automatically retrieve temporary credentials when <br-option>repo3-s3-key-type=auto</br-option>, which means that keys do not need to be explicitly set in <file>{[backrest-config-demo]}</file>.</p>
Automatically retrieve temporary S3 credentials on AWS instances. Automatically retrieve the role and temporary credentials for S3 when the AWS instance is associated with an IAM role. Credentials are automatically updated when they are <= 5 minutes from expiring. Basic configuration is to set repo1-s3-key-type=auto. repo1-s3-role can be used to set a specific role, otherwise it will be retrieved automatically.
2020-08-25 16:38:49 +02:00
<p>This sample <proper>Amazon S3</proper> policy will restrict all reads and writes to the bucket and repository path.</p>
Add sample S3 policy to restrict bucket privileges. Suggested by Douglas J Hunley, Jason O'Donnell.
2018-06-23 19:09:56 +02:00
Clarify that S3-compatible object stores are supported. The documentation mentioned Amazon S3 frequently but failed to mention that other S3-compatible object stores are also supported. Tone down the specific mentions of Amazon S3 and replace them with "S3-compatible object store" when appropriate. Suggested by Magnus Hagander.
2019-04-03 12:40:12 +02:00
<code-block title="Sample Amazon S3 Policy">
Add sample S3 policy to restrict bucket privileges. Suggested by Douglas J Hunley, Jason O'Donnell.
2018-06-23 19:09:56 +02:00
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:ListBucket"
],
"Resource": [
Use minio as local S3 emulator in documentation. The documentation was relying on a ScalityS3 container built for testing which wasn't very transparent. Instead, use the stock minio container and configure it in the documentation. Also, install certificates and CA so that TLS verification can be enabled.
2019-05-27 13:37:20 +02:00
"arn:aws:s3:::{[s3-bucket]}"
Add sample S3 policy to restrict bucket privileges. Suggested by Douglas J Hunley, Jason O'Donnell.
2018-06-23 19:09:56 +02:00
],
"Condition": {
"StringEquals": {
"s3:prefix": [
"",
Use minio as local S3 emulator in documentation. The documentation was relying on a ScalityS3 container built for testing which wasn't very transparent. Instead, use the stock minio container and configure it in the documentation. Also, install certificates and CA so that TLS verification can be enabled.
2019-05-27 13:37:20 +02:00
"{[s3-repo]}"
Add sample S3 policy to restrict bucket privileges. Suggested by Douglas J Hunley, Jason O'Donnell.
2018-06-23 19:09:56 +02:00
],
"s3:delimiter": [
"/"
]
}
}
},
{
"Effect": "Allow",
"Action": [
"s3:ListBucket"
],
"Resource": [
Use minio as local S3 emulator in documentation. The documentation was relying on a ScalityS3 container built for testing which wasn't very transparent. Instead, use the stock minio container and configure it in the documentation. Also, install certificates and CA so that TLS verification can be enabled.
2019-05-27 13:37:20 +02:00
"arn:aws:s3:::{[s3-bucket]}"
Add sample S3 policy to restrict bucket privileges. Suggested by Douglas J Hunley, Jason O'Donnell.
2018-06-23 19:09:56 +02:00
],
"Condition": {
"StringLike": {
"s3:prefix": [
Use minio as local S3 emulator in documentation. The documentation was relying on a ScalityS3 container built for testing which wasn't very transparent. Instead, use the stock minio container and configure it in the documentation. Also, install certificates and CA so that TLS verification can be enabled.
2019-05-27 13:37:20 +02:00
"{[s3-repo]}/*"
Add sample S3 policy to restrict bucket privileges. Suggested by Douglas J Hunley, Jason O'Donnell.
2018-06-23 19:09:56 +02:00
]
}
}
},
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:DeleteObject"
],
"Resource": [
Use minio as local S3 emulator in documentation. The documentation was relying on a ScalityS3 container built for testing which wasn't very transparent. Instead, use the stock minio container and configure it in the documentation. Also, install certificates and CA so that TLS verification can be enabled.
2019-05-27 13:37:20 +02:00
"arn:aws:s3:::{[s3-bucket]}/{[s3-repo]}/*"
Add sample S3 policy to restrict bucket privileges. Suggested by Douglas J Hunley, Jason O'Donnell.
2018-06-23 19:09:56 +02:00
]
}
]
}
</code-block>
S3 repository support.
2017-06-12 16:52:32 +02:00
<p>Commands are run exactly as if the repository were stored on a local disk.</p>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
S3 repository support.
2017-06-12 16:52:32 +02:00
<title>Create the stanza</title>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y">
S3 repository support.
2017-06-12 16:52:32 +02:00
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} {[dash]}-log-level-console=info stanza-create</exe-cmd>
<exe-highlight>completed successfully</exe-highlight>
</execute>
</execute-list>
Clarify that S3-compatible object stores are supported. The documentation mentioned Amazon S3 frequently but failed to mention that other S3-compatible object stores are also supported. Tone down the specific mentions of Amazon S3 and replace them with "S3-compatible object store" when appropriate. Suggested by Magnus Hagander.
2019-04-03 12:40:12 +02:00
<p>File creation time in object stores is relatively slow so commands benefit by increasing <br-option>process-max</br-option> to parallelize file creation.</p>
S3 repository support.
2017-06-12 16:52:32 +02:00
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
S3 repository support.
2017-06-12 16:52:32 +02:00
<title>Backup the {[postgres-cluster-demo]} cluster</title>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y">
Multiple repository support. Up to four repositories may be configured. A potential benefit is the ability to have a local repository for fast restores and a remote repository for redundancy. Some commands, e.g. stanza-create/stanza-update, will automatically work with all configured repositories while others, e.g. stanza-delete, will require a repository to be specified using the repo option. See the command reference for details on which commands require the repository to be specified. Note that the repo option is not required when only repo1 is configured in order to maintain backward compatibility. However, the repo option is required when a single repo is configured as, e.g. repo2. This is to prevent command breakage if a new repository is added later. The archive-push command will always push WAL to the archive in all configured repositories but backups will need to be scheduled individually for each repository. In many cases this is desirable since backup types and retention will vary by repository. Likewise, restores must specify a repository. It is generally better to specify a repository for restores that has low latency/cost even if that means more recovery time. Only restore testing can determine which repository will be most efficient. For single repository configurations there should be no change in behavior.
2021-03-08 20:31:13 +02:00
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} --repo=3
S3 repository support.
2017-06-12 16:52:32 +02:00
--log-level-console=info backup</exe-cmd>
<exe-highlight>no prior backup exists|full backup size</exe-highlight>
</execute>
</execute-list>
</section>
Multiple repository support. Up to four repositories may be configured. A potential benefit is the ability to have a local repository for fast restores and a remote repository for redundancy. Some commands, e.g. stanza-create/stanza-update, will automatically work with all configured repositories while others, e.g. stanza-delete, will require a repository to be specified using the repo option. See the command reference for details on which commands require the repository to be specified. Note that the repo option is not required when only repo1 is configured in order to maintain backward compatibility. However, the repo option is required when a single repo is configured as, e.g. repo2. This is to prevent command breakage if a new repository is added later. The archive-push command will always push WAL to the archive in all configured repositories but backups will need to be scheduled individually for each repository. In many cases this is desirable since backup types and retention will vary by repository. Likewise, restores must specify a repository. It is generally better to specify a repository for restores that has low latency/cost even if that means more recovery time. Only restore testing can determine which repository will be most efficient. For single repository configurations there should be no change in behavior.
2021-03-08 20:31:13 +02:00
<!-- ======================================================================================================================= -->
<section id="gcs-support" if="!{[object-any-all]}" depend="/quickstart/configure-archiving">
<title>GCS-Compatible Object Store Support</title>
<block id="gcs-setup">
<block-variable-replace key="gcs-setup-repo-id">4</block-variable-replace>
<block-variable-replace key="gcs-setup-host">{[host-pg1]}</block-variable-replace>
<block-variable-replace key="gcs-setup-user">postgres</block-variable-replace>
<block-variable-replace key="gcs-setup-config-owner">postgres:postgres</block-variable-replace>
</block>
<p>Commands are run exactly as if the repository were stored on a local disk.</p>
<p>File creation time in object stores is relatively slow so commands benefit by increasing <br-option>process-max</br-option> to parallelize file creation.</p>
</section>
<!-- ======================================================================================================================= -->
<section id="delete-stanza" if="!{[object-any-all]}" depend="/quickstart">
Add stanza-delete command to cleanup unused stanzas. Contributed by Cynthia Shang.
2018-01-03 19:23:33 +02:00
<title>Delete a Stanza</title>
<cmd-description key="stanza-delete"/>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Add stanza-delete command to cleanup unused stanzas. Contributed by Cynthia Shang.
2018-01-03 19:23:33 +02:00
<title>Stop <postgres/> cluster to be removed</title>
<execute user="root">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<exe-cmd>{[pg-cluster-stop]}</exe-cmd>
Add stanza-delete command to cleanup unused stanzas. Contributed by Cynthia Shang.
2018-01-03 19:23:33 +02:00
</execute>
</execute-list>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Add stanza-delete command to cleanup unused stanzas. Contributed by Cynthia Shang.
2018-01-03 19:23:33 +02:00
<title>Stop <backrest/> for the stanza</title>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y">
Add stanza-delete command to cleanup unused stanzas. Contributed by Cynthia Shang.
2018-01-03 19:23:33 +02:00
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} {[dash]}-log-level-console=info stop</exe-cmd>
<exe-highlight>completed successfully</exe-highlight>
</execute>
</execute-list>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Multiple repository support. Up to four repositories may be configured. A potential benefit is the ability to have a local repository for fast restores and a remote repository for redundancy. Some commands, e.g. stanza-create/stanza-update, will automatically work with all configured repositories while others, e.g. stanza-delete, will require a repository to be specified using the repo option. See the command reference for details on which commands require the repository to be specified. Note that the repo option is not required when only repo1 is configured in order to maintain backward compatibility. However, the repo option is required when a single repo is configured as, e.g. repo2. This is to prevent command breakage if a new repository is added later. The archive-push command will always push WAL to the archive in all configured repositories but backups will need to be scheduled individually for each repository. In many cases this is desirable since backup types and retention will vary by repository. Likewise, restores must specify a repository. It is generally better to specify a repository for restores that has low latency/cost even if that means more recovery time. Only restore testing can determine which repository will be most efficient. For single repository configurations there should be no change in behavior.
2021-03-08 20:31:13 +02:00
<title>Delete the stanza from one repository</title>
Add stanza-delete command to cleanup unused stanzas. Contributed by Cynthia Shang.
2018-01-03 19:23:33 +02:00
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y">
Multiple repository support. Up to four repositories may be configured. A potential benefit is the ability to have a local repository for fast restores and a remote repository for redundancy. Some commands, e.g. stanza-create/stanza-update, will automatically work with all configured repositories while others, e.g. stanza-delete, will require a repository to be specified using the repo option. See the command reference for details on which commands require the repository to be specified. Note that the repo option is not required when only repo1 is configured in order to maintain backward compatibility. However, the repo option is required when a single repo is configured as, e.g. repo2. This is to prevent command breakage if a new repository is added later. The archive-push command will always push WAL to the archive in all configured repositories but backups will need to be scheduled individually for each repository. In many cases this is desirable since backup types and retention will vary by repository. Likewise, restores must specify a repository. It is generally better to specify a repository for restores that has low latency/cost even if that means more recovery time. Only restore testing can determine which repository will be most efficient. For single repository configurations there should be no change in behavior.
2021-03-08 20:31:13 +02:00
<exe-cmd>
{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} --repo=1
{[dash]}-log-level-console=info stanza-delete
</exe-cmd>
Add stanza-delete command to cleanup unused stanzas. Contributed by Cynthia Shang.
2018-01-03 19:23:33 +02:00
<exe-highlight>completed successfully</exe-highlight>
</execute>
<execute user="root" show="n">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<exe-cmd>{[pg-cluster-start]}</exe-cmd>
Add stanza-delete command to cleanup unused stanzas. Contributed by Cynthia Shang.
2018-01-03 19:23:33 +02:00
</execute>
</execute-list>
</section>
Update backup host to repository host in the documentation. Contributed by Cynthia Shang.
2018-02-19 20:17:58 +02:00
<!-- SECTION => REPOSITORY HOST -->
<section id="repo-host" depend="/quickstart/configure-archiving">
<title>Dedicated Repository Host</title>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Update backup host to repository host in the documentation. Contributed by Cynthia Shang.
2018-02-19 20:17:58 +02:00
<p>The configuration described in <link section="/quickstart">Quickstart</link> is suitable for simple installations but for enterprise configurations it is more typical to have a dedicated <host>repository</host> host where the backups and WAL archive files are stored. This separates the backups and WAL archive from the database server so <host>database</host> host failures have less impact. It is still a good idea to employ traditional backup software to backup the <host>repository</host> host.</p>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Add documentation clarifications regarding standby repositories. Clarify that placing the repository on a standby is not a robust configuration and recommend against it.
2019-09-16 19:56:25 +02:00
<p>On <postgres/> hosts, <br-option>pg1-path</br-option> is required to be the path of the local PostgreSQL cluster and no <br-option>pg1-host</br-option> should be configured. When configuring a repository host, the pgbackrest configuration file must have the <br-option>pg-host</br-option> option configured to connect to the primary and standby (if any) hosts. The repository host has the only pgbackrest configuration that should be aware of more than one <postgres/> host. Order does not matter, e.g. pg1-path/pg1-host, pg2-path/pg2-host can be primary or standby.</p>
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
<section id="install">
<title>Installation</title>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Update backup host to repository host in the documentation. Contributed by Cynthia Shang.
2018-02-19 20:17:58 +02:00
<p>A new host named <host>repository</host> is created to store the cluster backups.</p>
Configuration rules are now pulled from the C library when present.
2017-08-25 22:47:47 +02:00
Add hint and documentation for protocol version mismatches. Update the documentation to explicitly state that versions must match across hosts when running remotely. Add a hint to the protocol version mismatch error to help the user identify the problem.
2020-09-25 16:30:29 +02:00
<admonition type="note">The <backrest/> version installed on the <host>repository</host> host must exactly match the version installed on the <postgres/> host.</admonition>
Add optional stress test section to the user guide. This is intended to provide pre-release stress-testing. Include container memory limits to help check for memory leaks. Also add parallelism to make for faster builds.
2021-08-12 13:49:59 +02:00
<host-add id="{[host-repo1-id]}" name="{[host-repo1]}" user="{[host-repo1-user]}" image="{[host-repo1-image]}" os="{[os-type]}" mount="{[host-repo1-mount]}" option="-m 512m {[host-option]}"/>
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
<p>The <user>{[br-user]}</user> user is created to own the <backrest/> repository. Any user can own the repository but it is best not to use <user>postgres</user> (if it exists) to avoid confusion.</p>
Update backup host to repository host in the documentation. Contributed by Cynthia Shang.
2018-02-19 20:17:58 +02:00
<execute-list host="{[host-repo1]}">
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<title>Create <user>{[br-user]}</user> user</title>
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
Create common if expressions for testing os-type. These expressions simplify os-type testing. This will be especially true as more OS types are added.
2018-12-05 00:33:56 +02:00
<execute if="{[os-type-is-debian]}" user="root">
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
<exe-cmd>adduser --disabled-password --gecos "" {[br-user]}</exe-cmd>
Configuration rules are now pulled from the C library when present.
2017-08-25 22:47:47 +02:00
</execute>
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<execute if="{[os-type-is-rhel]}" user="root">
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
<exe-cmd>groupadd {[br-group]}</exe-cmd>
Configuration rules are now pulled from the C library when present.
2017-08-25 22:47:47 +02:00
</execute>
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<execute if="{[os-type-is-rhel]}" user="root">
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
<exe-cmd>adduser -g{[br-group]} -n {[br-user]}</exe-cmd>
Configuration rules are now pulled from the C library when present.
2017-08-25 22:47:47 +02:00
</execute>
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
</execute-list>
<block id="br-install">
Update backup host to repository host in the documentation. Contributed by Cynthia Shang.
2018-02-19 20:17:58 +02:00
<block-variable-replace key="br-install-host">{[host-repo1]}</block-variable-replace>
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
<block-variable-replace key="br-install-user">{[br-user]}</block-variable-replace>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<block-variable-replace key="br-install-group">{[br-group]}</block-variable-replace>
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
</block>
<block id="br-install-repo">
Update backup host to repository host in the documentation. Contributed by Cynthia Shang.
2018-02-19 20:17:58 +02:00
<block-variable-replace key="br-install-host">{[host-repo1]}</block-variable-replace>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<block-variable-replace key="br-install-user">{[br-user]}</block-variable-replace>
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
<block-variable-replace key="br-install-group">{[br-group]}</block-variable-replace>
</block>
</section>
<section id="setup-ssh">
Use command in authorized_hosts to improve SSH security. Unsecured, passwordless SSH can be a scary thing. If an attacker gains access to one system they can easily hop to other systems. Add documentation on how to use the command parameter in authorized_keys to limit ssh to running a single command, pgbackrest. There is more that could be done for security but this likely addresses most needs. Also change references to "trusted ssh" to "passwordless ssh" since this seems more correct. Suggested by Stephen Frost, Magnus Hagander.
2018-09-19 22:41:20 +02:00
<title>Setup Passwordless SSH</title>
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
<block id="setup-ssh-intro">
Add comment regarding vars being required in blocks. Documentation block syntax requires that at least one var be specified. This limitation should be removed but for now add a comment to describe why a bogus var is defined.
2018-11-14 15:13:01 +02:00
<!-- ??? Bogus variable is set because the syntax currently requires at least one -->
<block-variable-replace key="bogus"></block-variable-replace>
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
</block>
Update backup host to repository host in the documentation. Contributed by Cynthia Shang.
2018-02-19 20:17:58 +02:00
<execute-list host="{[host-repo1]}">
<title>Create <host>{[host-repo1]}</host> host key pair</title>
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
Preliminary documentation for PostgreSQL 11 unprivileged user backup.
2018-06-15 01:05:35 +02:00
<execute user="{[br-user]}">
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
<exe-cmd>mkdir -m 750 {[br-home-path]}/.ssh</exe-cmd>
Configuration rules are now pulled from the C library when present.
2017-08-25 22:47:47 +02:00
</execute>
Preliminary documentation for PostgreSQL 11 unprivileged user backup.
2018-06-15 01:05:35 +02:00
<execute user="{[br-user]}">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<exe-cmd>ssh-keygen -f {[br-home-path]}/.ssh/id_rsa
-t rsa -b 4096 -N ""</exe-cmd>
Configuration rules are now pulled from the C library when present.
2017-08-25 22:47:47 +02:00
</execute>
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
</execute-list>
<block id="setup-ssh">
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<block-variable-replace key="setup-ssh-host">{[host-pg1]}</block-variable-replace>
Preliminary documentation for PostgreSQL 11 unprivileged user backup.
2018-06-15 01:05:35 +02:00
<block-variable-replace key="setup-ssh-user">postgres</block-variable-replace>
<block-variable-replace key="setup-ssh-user-home-path">{[pg-home-path]}</block-variable-replace>
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
</block>
Preliminary documentation for PostgreSQL 11 unprivileged user backup.
2018-06-15 01:05:35 +02:00
Add admonitions to documentation renderers. Admonitions call out places where the user should take special care. Support added for HTML, PDF, Markdown and help text renderers. XML files have been updated accordingly. Contributed by Cynthia Shang.
2018-12-30 16:40:20 +02:00
<admonition type="note">ssh has been configured to only allow <backrest/> to be run via passwordless ssh. This enhances security in the event that one of the service accounts is hijacked.</admonition>
Use command in authorized_hosts to improve SSH security. Unsecured, passwordless SSH can be a scary thing. If an attacker gains access to one system they can easily hop to other systems. Add documentation on how to use the command parameter in authorized_keys to limit ssh to running a single command, pgbackrest. There is more that could be done for security but this likely addresses most needs. Also change references to "trusted ssh" to "passwordless ssh" since this seems more correct. Suggested by Stephen Frost, Magnus Hagander.
2018-09-19 22:41:20 +02:00
Replace keywords with more flexible if statements. Keywords were extremely limited and prevented us from generating multi-version documentation and other improvements. Replace keywords with an if statement that can evaluate a Perl expression with variable replacement. Since keywords were used to generate cache keys, add a --key-var parameter to identify which variables should make up the key.
2018-12-01 19:40:01 +02:00
<!-- <block if="{[pg-version]} >= 11" id="setup-ssh">
Preliminary documentation for PostgreSQL 11 unprivileged user backup.
2018-06-15 01:05:35 +02:00
<block-variable-replace key="setup-ssh-host">{[host-pg1]}</block-variable-replace>
<block-variable-replace key="setup-ssh-user">pgbackrest</block-variable-replace>
<block-variable-replace key="setup-ssh-user-home-path">{[br-home-path]}</block-variable-replace>
</block> -->
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
</section>
Update backup host to repository host in the documentation. Contributed by Cynthia Shang.
2018-02-19 20:17:58 +02:00
<!-- SECTION => REPOSITORY HOST - INSTALL/CONFIGURE -->
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
<section id="config">
<title>Configuration</title>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<backrest-config host="{[host-repo1]}" show="n" owner="{[br-user]}:{[br-group]}" file="{[backrest-config-demo]}">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Configure the <backrest/> repository path</title>
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<backrest-config-option section="global" key="repo1-path">{[backrest-repo-path]}</backrest-config-option>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
</backrest-config>
Fix old option name in documentation.
2019-09-27 19:33:14 +02:00
<p>The <host>repository</host> host must be configured with the <host>{[host-pg1]}</host> host/user and database path. The primary will be configured as <id>pg1</id> to allow a standby to be added later.</p>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<backrest-config host="{[host-repo1]}" owner="{[br-user]}:{[br-group]}" file="{[backrest-config-demo]}">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<title>Configure <br-option>pg1-host</br-option>/<br-option>pg1-host-user</br-option> and <br-option>pg1-path</br-option></title>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<backrest-config-option section="demo" key="pg1-path">{[pg-path]}</backrest-config-option>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<backrest-config-option section="demo" key="pg1-host">{[host-pg1]}</backrest-config-option>
Comment out unprivileged user documentation. We haven't had the time to complete this documentation and it has suffered bit rot. This prevents us from building the docs on PostgreSQL >= 11 so just comment it all out until it can be updated.
2019-09-26 22:02:39 +02:00
<!-- <backrest-config-option if="{[pg-version]} >= 11" section="demo" key="pg1-host-user">{[br-user]}</backrest-config-option> -->
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
New simpler configuration and consistent project/exe/path naming. * The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required. * Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case. * The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally. * The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest. * Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory. * Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory. * Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 15:30:54 +02:00
<backrest-config-option section="global" key="start-fast">y</backrest-config-option>
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
<backrest-config-option section="global" key="repo1-retention-full">2</backrest-config-option>
Added the log-level-stderr option. Controls whether console log messages are sent to stderr or stdout. By default this is set to warn which represents a change in behavior from previous versions, even though it may be more intuitive. Setting log-level-stderr=off will preserve the old behavior. Suggested by Sascha Biberhofer.
2016-10-05 15:09:30 +02:00
<backrest-config-option section="global" key="log-level-stderr">off</backrest-config-option>
Added --log-timestamp option. Allow timestamps to be suppressed in logging. This is primarily used to avoid filters in the automated documentation.
2017-02-05 17:59:49 +02:00
<backrest-config-option section="global" key="log-timestamp">n</backrest-config-option>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
</backrest-config>
Update backup host to repository host in the documentation. Contributed by Cynthia Shang.
2018-02-19 20:17:58 +02:00
<p>The database host must be configured with the repository host/user. The default for the <br-option>repo1-host-user</br-option> option is <id>pgbackrest</id>. If the <id>postgres</id> user does restores on the repository host it is best not to also allow the <id>postgres</id> user to perform backups. However, the <id>postgres</id> user can read the repository directly if it is in the same group as the <id>pgbackrest</id> user.</p>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<backrest-config host="{[host-pg1]}" file="{[backrest-config-demo]}" reset="y">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<title>Configure <br-option>repo1-host</br-option>/<br-option>repo1-host-user</br-option></title>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<backrest-config-option section="demo" key="pg1-path">{[pg-path]}</backrest-config-option>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Update backup host to repository host in the documentation. Contributed by Cynthia Shang.
2018-02-19 20:17:58 +02:00
<backrest-config-option section="global" key="repo1-host">{[host-repo1]}</backrest-config-option>
Added the log-level-stderr option. Controls whether console log messages are sent to stderr or stdout. By default this is set to warn which represents a change in behavior from previous versions, even though it may be more intuitive. Setting log-level-stderr=off will preserve the old behavior. Suggested by Sascha Biberhofer.
2016-10-05 15:09:30 +02:00
Added documentation for parallel archive-push.
2017-02-05 17:35:53 +02:00
<backrest-config-option section="global" key="log-level-file">detail</backrest-config-option>
Added the log-level-stderr option. Controls whether console log messages are sent to stderr or stdout. By default this is set to warn which represents a change in behavior from previous versions, even though it may be more intuitive. Setting log-level-stderr=off will preserve the old behavior. Suggested by Sascha Biberhofer.
2016-10-05 15:09:30 +02:00
<backrest-config-option section="global" key="log-level-stderr">off</backrest-config-option>
Added --log-timestamp option. Allow timestamps to be suppressed in logging. This is primarily used to avoid filters in the automated documentation.
2017-02-05 17:59:49 +02:00
<backrest-config-option section="global" key="log-timestamp">n</backrest-config-option>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
</backrest-config>
Update backup host to repository host in the documentation. Contributed by Cynthia Shang.
2018-02-19 20:17:58 +02:00
<p>Commands are run the same as on a single host configuration except that some commands such as <cmd>backup</cmd> and <cmd>expire</cmd> are run from the <host>repository</host> host instead of the <host>database</host> host.</p>
Added check command. The check command validates that pgBackRest is configured correctly for archiving and backups. Contributed by Cynthia Shang.
2016-06-12 15:13:46 +02:00
Comment out unprivileged user documentation. We haven't had the time to complete this documentation and it has suffered bit rot. This prevents us from building the docs on PostgreSQL >= 11 so just comment it all out until it can be updated.
2019-09-26 22:02:39 +02:00
<!-- <execute-list if="{[pg-version]} >= 11" host="{[host-pg1]}">
Preliminary documentation for PostgreSQL 11 unprivileged user backup.
2018-06-15 01:05:35 +02:00
<title>Set permissions required for backup</title>
<execute user="postgres">
<exe-cmd>
psql -c "
create user pgbackrest;
grant pg_read_all_settings to pgbackrest;
grant execute on function pg_start_backup(text, boolean, boolean) to pgbackrest;
grant execute on function pg_stop_backup(boolean, boolean) to pgbackrest;
grant execute on function pg_switch_wal() to pgbackrest;
grant execute on function pg_create_restore_point(text) to pgbackrest;"
</exe-cmd>
</execute>
Comment out unprivileged user documentation. We haven't had the time to complete this documentation and it has suffered bit rot. This prevents us from building the docs on PostgreSQL >= 11 so just comment it all out until it can be updated.
2019-09-26 22:02:39 +02:00
</execute-list> -->
Build all docs with S3 using --var=s3-all=y Force repo-type=s3 for all tests. This is not currently the default for any OS builds.
2019-05-29 14:38:45 +02:00
Azure support for repository storage. Azure and Azure-compatible object stores can now be used for repository storage. Currently only shared key authentication is supported but SAS will be added soon.
2020-07-02 22:24:34 +02:00
<p if="'{[azure-all]}' eq 'y'">Configure Azure-compatible object store if required.</p>
<block id="azure-setup" if="'{[azure-all]}' eq 'y'">
Multiple repository support. Up to four repositories may be configured. A potential benefit is the ability to have a local repository for fast restores and a remote repository for redundancy. Some commands, e.g. stanza-create/stanza-update, will automatically work with all configured repositories while others, e.g. stanza-delete, will require a repository to be specified using the repo option. See the command reference for details on which commands require the repository to be specified. Note that the repo option is not required when only repo1 is configured in order to maintain backward compatibility. However, the repo option is required when a single repo is configured as, e.g. repo2. This is to prevent command breakage if a new repository is added later. The archive-push command will always push WAL to the archive in all configured repositories but backups will need to be scheduled individually for each repository. In many cases this is desirable since backup types and retention will vary by repository. Likewise, restores must specify a repository. It is generally better to specify a repository for restores that has low latency/cost even if that means more recovery time. Only restore testing can determine which repository will be most efficient. For single repository configurations there should be no change in behavior.
2021-03-08 20:31:13 +02:00
<block-variable-replace key="azure-setup-repo-id">1</block-variable-replace>
Azure support for repository storage. Azure and Azure-compatible object stores can now be used for repository storage. Currently only shared key authentication is supported but SAS will be added soon.
2020-07-02 22:24:34 +02:00
<block-variable-replace key="azure-setup-host">{[host-repo1]}</block-variable-replace>
<block-variable-replace key="azure-setup-user">{[br-user]}</block-variable-replace>
<block-variable-replace key="azure-setup-config-owner">{[br-user]}:{[br-group]}</block-variable-replace>
<block-variable-replace key="azure-setup-create-container">n</block-variable-replace>
</block>
GCS support for repository storage. GCS and GCS-compatible object stores can now be used for repository storage.
2021-03-05 19:13:51 +02:00
<p if="'{[gcs-all]}' eq 'y'">Configure GCS-compatible object store if required.</p>
<block id="gcs-setup" if="'{[gcs-all]}' eq 'y'">
Multiple repository support. Up to four repositories may be configured. A potential benefit is the ability to have a local repository for fast restores and a remote repository for redundancy. Some commands, e.g. stanza-create/stanza-update, will automatically work with all configured repositories while others, e.g. stanza-delete, will require a repository to be specified using the repo option. See the command reference for details on which commands require the repository to be specified. Note that the repo option is not required when only repo1 is configured in order to maintain backward compatibility. However, the repo option is required when a single repo is configured as, e.g. repo2. This is to prevent command breakage if a new repository is added later. The archive-push command will always push WAL to the archive in all configured repositories but backups will need to be scheduled individually for each repository. In many cases this is desirable since backup types and retention will vary by repository. Likewise, restores must specify a repository. It is generally better to specify a repository for restores that has low latency/cost even if that means more recovery time. Only restore testing can determine which repository will be most efficient. For single repository configurations there should be no change in behavior.
2021-03-08 20:31:13 +02:00
<block-variable-replace key="gcs-setup-repo-id">1</block-variable-replace>
GCS support for repository storage. GCS and GCS-compatible object stores can now be used for repository storage.
2021-03-05 19:13:51 +02:00
<block-variable-replace key="gcs-setup-host">{[host-repo1]}</block-variable-replace>
<block-variable-replace key="gcs-setup-user">{[br-user]}</block-variable-replace>
<block-variable-replace key="gcs-setup-config-owner">{[br-user]}:{[br-group]}</block-variable-replace>
</block>
Build all docs with S3 using --var=s3-all=y Force repo-type=s3 for all tests. This is not currently the default for any OS builds.
2019-05-29 14:38:45 +02:00
<p if="'{[s3-all]}' eq 'y'">Configure S3-compatible object store if required.</p>
<block id="s3-setup" if="'{[s3-all]}' eq 'y'">
Multiple repository support. Up to four repositories may be configured. A potential benefit is the ability to have a local repository for fast restores and a remote repository for redundancy. Some commands, e.g. stanza-create/stanza-update, will automatically work with all configured repositories while others, e.g. stanza-delete, will require a repository to be specified using the repo option. See the command reference for details on which commands require the repository to be specified. Note that the repo option is not required when only repo1 is configured in order to maintain backward compatibility. However, the repo option is required when a single repo is configured as, e.g. repo2. This is to prevent command breakage if a new repository is added later. The archive-push command will always push WAL to the archive in all configured repositories but backups will need to be scheduled individually for each repository. In many cases this is desirable since backup types and retention will vary by repository. Likewise, restores must specify a repository. It is generally better to specify a repository for restores that has low latency/cost even if that means more recovery time. Only restore testing can determine which repository will be most efficient. For single repository configurations there should be no change in behavior.
2021-03-08 20:31:13 +02:00
<block-variable-replace key="s3-setup-repo-id">1</block-variable-replace>
Build all docs with S3 using --var=s3-all=y Force repo-type=s3 for all tests. This is not currently the default for any OS builds.
2019-05-29 14:38:45 +02:00
<block-variable-replace key="s3-setup-host">{[host-repo1]}</block-variable-replace>
Add repo-create, repo-get, repo-put, and repo-rm commands. These commands are generally useful but more importantly they allow removing LibC by providing the Perl integration tests an alternate way to work with repository storage. All the commands are currently internal only and should not be used on production repositories.
2020-03-09 23:15:03 +02:00
<block-variable-replace key="s3-setup-user">{[br-user]}</block-variable-replace>
Build all docs with S3 using --var=s3-all=y Force repo-type=s3 for all tests. This is not currently the default for any OS builds.
2019-05-29 14:38:45 +02:00
<block-variable-replace key="s3-setup-config-owner">{[br-user]}:{[br-group]}</block-variable-replace>
<block-variable-replace key="s3-setup-create-bucket">n</block-variable-replace>
</block>
Preliminary documentation for PostgreSQL 11 unprivileged user backup.
2018-06-15 01:05:35 +02:00
Multiple repository support. Up to four repositories may be configured. A potential benefit is the ability to have a local repository for fast restores and a remote repository for redundancy. Some commands, e.g. stanza-create/stanza-update, will automatically work with all configured repositories while others, e.g. stanza-delete, will require a repository to be specified using the repo option. See the command reference for details on which commands require the repository to be specified. Note that the repo option is not required when only repo1 is configured in order to maintain backward compatibility. However, the repo option is required when a single repo is configured as, e.g. repo2. This is to prevent command breakage if a new repository is added later. The archive-push command will always push WAL to the archive in all configured repositories but backups will need to be scheduled individually for each repository. In many cases this is desirable since backup types and retention will vary by repository. Likewise, restores must specify a repository. It is generally better to specify a repository for restores that has low latency/cost even if that means more recovery time. Only restore testing can determine which repository will be most efficient. For single repository configurations there should be no change in behavior.
2021-03-08 20:31:13 +02:00
<p if="!{[object-any-all]}">Create the stanza in the new repository.</p>
Improved stanza-create command so that it can repair broken repositories in most cases and is robust enough to be made mandatory. Contributed by Cynthia Shang.
2016-12-20 23:52:20 +02:00
Multiple repository support. Up to four repositories may be configured. A potential benefit is the ability to have a local repository for fast restores and a remote repository for redundancy. Some commands, e.g. stanza-create/stanza-update, will automatically work with all configured repositories while others, e.g. stanza-delete, will require a repository to be specified using the repo option. See the command reference for details on which commands require the repository to be specified. Note that the repo option is not required when only repo1 is configured in order to maintain backward compatibility. However, the repo option is required when a single repo is configured as, e.g. repo2. This is to prevent command breakage if a new repository is added later. The archive-push command will always push WAL to the archive in all configured repositories but backups will need to be scheduled individually for each repository. In many cases this is desirable since backup types and retention will vary by repository. Likewise, restores must specify a repository. It is generally better to specify a repository for restores that has low latency/cost even if that means more recovery time. Only restore testing can determine which repository will be most efficient. For single repository configurations there should be no change in behavior.
2021-03-08 20:31:13 +02:00
<execute-list host="{[host-repo1]}" if="!{[object-any-all]}">
Improved stanza-create command so that it can repair broken repositories in most cases and is robust enough to be made mandatory. Contributed by Cynthia Shang.
2016-12-20 23:52:20 +02:00
<title>Create the stanza</title>
Azure support for repository storage. Azure and Azure-compatible object stores can now be used for repository storage. Currently only shared key authentication is supported but SAS will be added soon.
2020-07-02 22:24:34 +02:00
<!-- Create the stanza -->
Preliminary documentation for PostgreSQL 11 unprivileged user backup.
2018-06-15 01:05:35 +02:00
<execute user="{[br-user]}" output="y" filter="n">
Improved stanza-create command so that it can repair broken repositories in most cases and is robust enough to be made mandatory. Contributed by Cynthia Shang.
2016-12-20 23:52:20 +02:00
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} stanza-create</exe-cmd>
</execute>
</execute-list>
Update backup host to repository host in the documentation. Contributed by Cynthia Shang.
2018-02-19 20:17:58 +02:00
<p>Check that the configuration is correct on both the <host>database</host> and <host>repository</host> hosts. More information about the <cmd>check</cmd> command can be found in <link section="/quickstart/check-configuration">Check the Configuration</link>.</p>
Added check command. The check command validates that pgBackRest is configured correctly for archiving and backups. Contributed by Cynthia Shang.
2016-06-12 15:13:46 +02:00
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Added check command. The check command validates that pgBackRest is configured correctly for archiving and backups. Contributed by Cynthia Shang.
2016-06-12 15:13:46 +02:00
<title>Check the configuration</title>
Preliminary documentation for PostgreSQL 11 unprivileged user backup.
2018-06-15 01:05:35 +02:00
<execute user="postgres" output="y" filter="n">
Added check command. The check command validates that pgBackRest is configured correctly for archiving and backups. Contributed by Cynthia Shang.
2016-06-12 15:13:46 +02:00
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} check</exe-cmd>
</execute>
</execute-list>
Update backup host to repository host in the documentation. Contributed by Cynthia Shang.
2018-02-19 20:17:58 +02:00
<execute-list host="{[host-repo1]}">
Added check command. The check command validates that pgBackRest is configured correctly for archiving and backups. Contributed by Cynthia Shang.
2016-06-12 15:13:46 +02:00
<title>Check the configuration</title>
Preliminary documentation for PostgreSQL 11 unprivileged user backup.
2018-06-15 01:05:35 +02:00
<execute user="{[br-user]}" output="y" filter="n">
Added check command. The check command validates that pgBackRest is configured correctly for archiving and backups. Contributed by Cynthia Shang.
2016-06-12 15:13:46 +02:00
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} check</exe-cmd>
</execute>
</execute-list>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
</section>
Update backup host to repository host in the documentation. Contributed by Cynthia Shang.
2018-02-19 20:17:58 +02:00
<!-- SECTION => REPOSITORY HOST - PERFORM BACKUP -->
Improved support for --require option and section depends now default to the previous section.
2017-02-11 17:26:54 +02:00
<section id="perform-backup">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Perform a Backup</title>
Update backup host to repository host in the documentation. Contributed by Cynthia Shang.
2018-02-19 20:17:58 +02:00
<p>To perform a backup of the <postgres/> cluster run <backrest/> with the <cmd>backup</cmd> command on the <host>repository</host> host.</p>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Update backup host to repository host in the documentation. Contributed by Cynthia Shang.
2018-02-19 20:17:58 +02:00
<execute-list host="{[host-repo1]}">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Backup the {[postgres-cluster-demo]} cluster</title>
Preliminary documentation for PostgreSQL 11 unprivileged user backup.
2018-06-15 01:05:35 +02:00
<execute user="{[br-user]}" output="y" filter="n">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} backup</exe-cmd>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
</execute>
</execute-list>
Update backup host to repository host in the documentation. Contributed by Cynthia Shang.
2018-02-19 20:17:58 +02:00
<p>Since a new repository was created on the <host>repository</host> host the warning about the incremental backup changing to a full backup was emitted.</p>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
</section>
Update backup host to repository host in the documentation. Contributed by Cynthia Shang.
2018-02-19 20:17:58 +02:00
<!-- SECTION => REPOSITORY HOST - PERFORM RESTORE -->
Improved support for --require option and section depends now default to the previous section.
2017-02-11 17:26:54 +02:00
<section id="perform-restore">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Restore a Backup</title>
<p>To perform a restore of the <postgres/> cluster run <backrest/> with the <cmd>restore</cmd> command on the <host>database</host> host.</p>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Stop the {[postgres-cluster-demo]} cluster, restore, and restart <postgres/></title>
<execute user="root">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<exe-cmd>{[pg-cluster-stop]}</exe-cmd>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
</execute>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} {[dash]}-delta restore</exe-cmd>
</execute>
<execute user="root">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<exe-cmd>{[pg-cluster-start]}</exe-cmd>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
</execute>
Closed #163: Document Generation Issue Fixed an issue where document generation failed because some OSs are not tolerant of having multiple installed versions of PostgreSQL. A separate VM is now created for each version. Also added a sleep after database starts during document generation to ensure the database is running before the next command runs. Reported by John Harvey.
2016-01-23 01:01:21 +02:00
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" show="n">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<exe-cmd>{[pg-cluster-wait]}</exe-cmd>
Closed #163: Document Generation Issue Fixed an issue where document generation failed because some OSs are not tolerant of having multiple installed versions of PostgreSQL. A separate VM is now created for each version. Also added a sleep after database starts during document generation to ensure the database is running before the next command runs. Reported by John Harvey.
2016-01-23 01:01:21 +02:00
</execute>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
</execute-list>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
</section>
</section>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Added documentation to the User Guide for the process-max option. Contributed by Cynthia Shang.
2016-12-23 23:50:02 +02:00
<!-- SECTION => PARALLEL BACKUP-RESTORE -->
Update backup host to repository host in the documentation. Contributed by Cynthia Shang.
2018-02-19 20:17:58 +02:00
<section id="parallel-backup-restore" depend="repo-host/config">
Added documentation to the User Guide for the process-max option. Contributed by Cynthia Shang.
2016-12-23 23:50:02 +02:00
<title>Parallel Backup / Restore</title>
<p><backrest/> offers parallel processing to improve performance of compression and transfer. The number of processes to be used for this feature is set using the <br-option>--process-max</br-option> option.</p>
Remove call to lscpu which can vary widely by build host.
2018-07-05 22:49:46 +02:00
<p>It is usually best not to use more than 25% of available CPUs for the <cmd>backup</cmd> command. Backups don't have to run that fast as long as they are performed regularly and the backup process should not impact database performance, if at all possible.</p>
Added documentation to the User Guide for the process-max option. Contributed by Cynthia Shang.
2016-12-23 23:50:02 +02:00
<p>The restore command can and should use all available CPUs because during a restore the <postgres/> cluster is shut down and there is generally no other important work being done on the host. If the host contains multiple clusters then that should be considered when setting restore parallelism.</p>
Update backup host to repository host in the documentation. Contributed by Cynthia Shang.
2018-02-19 20:17:58 +02:00
<execute-list host="{[host-repo1]}">
Added documentation to the User Guide for the process-max option. Contributed by Cynthia Shang.
2016-12-23 23:50:02 +02:00
<title>Perform a backup with single process</title>
Preliminary documentation for PostgreSQL 11 unprivileged user backup.
2018-06-15 01:05:35 +02:00
<execute user="{[br-user]}">
Added documentation to the User Guide for the process-max option. Contributed by Cynthia Shang.
2016-12-23 23:50:02 +02:00
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} {[dash]}-type=full backup</exe-cmd>
</execute>
</execute-list>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<backrest-config host="{[host-repo1]}" owner="{[br-user]}:{[br-group]}" file="{[backrest-config-demo]}">
Added documentation to the User Guide for the process-max option. Contributed by Cynthia Shang.
2016-12-23 23:50:02 +02:00
<title>Configure <backrest/> to use multiple <cmd>backup</cmd> processes</title>
<backrest-config-option section="global" key="process-max">3</backrest-config-option>
</backrest-config>
Update backup host to repository host in the documentation. Contributed by Cynthia Shang.
2018-02-19 20:17:58 +02:00
<execute-list host="{[host-repo1]}">
Added documentation to the User Guide for the process-max option. Contributed by Cynthia Shang.
2016-12-23 23:50:02 +02:00
<title>Perform a backup with multiple processes</title>
Preliminary documentation for PostgreSQL 11 unprivileged user backup.
2018-06-15 01:05:35 +02:00
<execute user="{[br-user]}">
Added documentation to the User Guide for the process-max option. Contributed by Cynthia Shang.
2016-12-23 23:50:02 +02:00
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} {[dash]}-type=full backup</exe-cmd>
</execute>
</execute-list>
Update backup host to repository host in the documentation. Contributed by Cynthia Shang.
2018-02-19 20:17:58 +02:00
<execute-list host="{[host-repo1]}">
Added documentation to the User Guide for the process-max option. Contributed by Cynthia Shang.
2016-12-23 23:50:02 +02:00
<title>Get backup info for the {[postgres-cluster-demo]} cluster</title>
Preliminary documentation for PostgreSQL 11 unprivileged user backup.
2018-06-15 01:05:35 +02:00
<execute filter="n" output="y" user="{[br-user]}">
Added documentation to the User Guide for the process-max option. Contributed by Cynthia Shang.
2016-12-23 23:50:02 +02:00
<exe-cmd>{[project-exe]} info</exe-cmd>
Added archive and backup WAL ranges to the info command.
2017-02-05 17:31:29 +02:00
<exe-highlight>timestamp start/stop</exe-highlight>
Added documentation to the User Guide for the process-max option. Contributed by Cynthia Shang.
2016-12-23 23:50:02 +02:00
</execute>
</execute-list>
<p>The performance of the last backup should be improved by using multiple processes. For very small backups the difference may not be very apparent, but as the size of the database increases so will time savings.</p>
</section>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<!-- SECTION => START/STOP -->
Update backup host to repository host in the documentation. Contributed by Cynthia Shang.
2018-02-19 20:17:58 +02:00
<section id="start-stop" depend="/repo-host/config">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Starting and Stopping</title>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Rename master to primary in documentation to align with PostgreSQL convention.
2017-09-07 19:59:40 +02:00
<p>Sometimes it is useful to prevent <backrest/> from running on a system. For example, when failing over from a primary to a standby it's best to prevent <backrest/> from running on the old primary in case <postgres/> gets restarted or can't be completely killed. This will also prevent <backrest/> from running on <id>cron</id>.</p>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Stop the <backrest/> services</title>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-cmd>{[project-exe]} stop</exe-cmd>
</execute>
</execute-list>
<p>New <backrest/> processes will no longer run.</p>
Update backup host to repository host in the documentation. Contributed by Cynthia Shang.
2018-02-19 20:17:58 +02:00
<execute-list host="{[host-repo1]}">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Attempt a backup</title>
The backup command is implemented entirely in C. For the most part this is a direct migration of the Perl code into C except as noted below. A backup can now be initiated from a linked directory. The link will not be stored in the manifest or recreated on restore. If a link or directory does not already exist in the restore location then a directory will be created. The logic for creating backup labels has been improved and it should no longer be possible to get a backup label earlier than the latest backup even with timezone changes or clock skew. This has never been an issue in the field that we know of, but we found it in testing. For online backups all times are fetched from the PostgreSQL primary host (before only copy start was). This doesn't affect backup integrity but it does prevent clock skew between hosts affecting backup duration reporting. Archive copy now works as expected when the archive and backup have different compression settings, i.e. when one is compressed and the other is not. This was a long-standing bug in the Perl code. Resume will now work even if hardlink settings have been changed. Reviewed by Cynthia Shang.
2019-12-14 00:14:26 +02:00
<execute user="{[br-user]}" err-expect="56" output="y">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} backup</exe-cmd>
Improve protocol error handling. In particular, "stop" errors are no longer reported as "unexpected".
2017-10-21 20:11:33 +02:00
<exe-highlight>\: stop file exists for all stanzas</exe-highlight>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
</execute>
</execute-list>
<p>Specify the <br-option>--force</br-option> option to terminate any <backrest/> process that are currently running. If <backrest/> is already stopped then stopping again will generate a warning.</p>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Stop the <backrest/> services again</title>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y" filter="n">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-cmd>{[project-exe]} stop</exe-cmd>
</execute>
</execute-list>
<p>Start <backrest/> processes again with the <cmd>start</cmd> command.</p>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Start the <backrest/> services</title>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-cmd>{[project-exe]} start</exe-cmd>
</execute>
</execute-list>
<p>It is also possible to stop <backrest/> for a single stanza.</p>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Stop <backrest/> services for the <id>demo</id> stanza</title>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} stop</exe-cmd>
</execute>
Added Retention to QuickStart section.
2016-12-05 01:11:23 +02:00
</execute-list>
<p>New <backrest/> processes for the specified stanza will no longer run.</p>
Update backup host to repository host in the documentation. Contributed by Cynthia Shang.
2018-02-19 20:17:58 +02:00
<execute-list host="{[host-repo1]}">
Added Retention to QuickStart section.
2016-12-05 01:11:23 +02:00
<title>Attempt a backup</title>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
The backup command is implemented entirely in C. For the most part this is a direct migration of the Perl code into C except as noted below. A backup can now be initiated from a linked directory. The link will not be stored in the manifest or recreated on restore. If a link or directory does not already exist in the restore location then a directory will be created. The logic for creating backup labels has been improved and it should no longer be possible to get a backup label earlier than the latest backup even with timezone changes or clock skew. This has never been an issue in the field that we know of, but we found it in testing. For online backups all times are fetched from the PostgreSQL primary host (before only copy start was). This doesn't affect backup integrity but it does prevent clock skew between hosts affecting backup duration reporting. Archive copy now works as expected when the archive and backup have different compression settings, i.e. when one is compressed and the other is not. This was a long-standing bug in the Perl code. Resume will now work even if hardlink settings have been changed. Reviewed by Cynthia Shang.
2019-12-14 00:14:26 +02:00
<execute user="{[br-user]}" err-expect="56" output="y">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} backup</exe-cmd>
Improve protocol error handling. In particular, "stop" errors are no longer reported as "unexpected".
2017-10-21 20:11:33 +02:00
<exe-highlight>\: stop file exists for stanza demo</exe-highlight>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
</execute>
</execute-list>
<p>The stanza must also be specified when starting the <backrest/> processes for a single stanza.</p>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Start the <backrest/> services for the <id>demo</id> stanza</title>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} start</exe-cmd>
</execute>
</execute-list>
</section>
<!-- SECTION => REPLICATION -->
Update backup host to repository host in the documentation. Contributed by Cynthia Shang.
2018-02-19 20:17:58 +02:00
<section id="replication" depend="/repo-host/perform-backup">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Replication</title>
Rename master to primary in documentation to align with PostgreSQL convention.
2017-09-07 19:59:40 +02:00
<p>Replication allows multiple copies of a <postgres/> cluster (called standbys) to be created from a single primary. The standbys are useful for balancing reads and to provide redundancy in case the primary host fails.</p>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
<!-- SECTION => REPLICATION - INSTALLATION -->
<section id="installation">
<title>Installation</title>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<p>A new host named <host>{[host-pg2]}</host> is created to run the standby.</p>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Reduce memory requirements of stress test section in the user guide. The standby memory was set to 1024mb in 86a651f9 to compensate for a memory leak in restore. The leak has been fixed (or at least mitigated) in e1e6e475 and 4fb6384f so the memory can be reduced to 512mb, the same as the primary.
2021-08-18 16:17:10 +02:00
<host-add id="{[host-pg2-id]}" name="{[host-pg2]}" user="{[host-pg2-user]}" image="{[host-pg2-image]}" os="{[os-type]}" mount="{[host-pg2-mount]}" option="-m 512m {[host-option]}"/>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Comment out unprivileged user documentation. We haven't had the time to complete this documentation and it has suffered bit rot. This prevents us from building the docs on PostgreSQL >= 11 so just comment it all out until it can be updated.
2019-09-26 22:02:39 +02:00
<!-- <execute-list if="{[pg-version]} >= 11" host="{[host-pg2]}">
Preliminary documentation for PostgreSQL 11 unprivileged user backup.
2018-06-15 01:05:35 +02:00
<title>Create <user>{[br-user]}</user> user</title>
Create common if expressions for testing os-type. These expressions simplify os-type testing. This will be especially true as more OS types are added.
2018-12-05 00:33:56 +02:00
<execute if="{[os-type-is-debian]}" user="root">
Comment out unprivileged user documentation. We haven't had the time to complete this documentation and it has suffered bit rot. This prevents us from building the docs on PostgreSQL >= 11 so just comment it all out until it can be updated.
2019-09-26 22:02:39 +02:00
<exe-cmd>adduser {[dash]}-ingroup {[pg-group]} {[dash]}-disabled-password {[dash]}-gecos "" {[br-user]}</exe-cmd>
Preliminary documentation for PostgreSQL 11 unprivileged user backup.
2018-06-15 01:05:35 +02:00
</execute>
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<execute if="{[os-type-is-rhel]}" user="root">
Preliminary documentation for PostgreSQL 11 unprivileged user backup.
2018-06-15 01:05:35 +02:00
<exe-cmd>adduser -g{[pg-group]} -n {[br-user]}</exe-cmd>
</execute>
Comment out unprivileged user documentation. We haven't had the time to complete this documentation and it has suffered bit rot. This prevents us from building the docs on PostgreSQL >= 11 so just comment it all out until it can be updated.
2019-09-26 22:02:39 +02:00
</execute-list> -->
Preliminary documentation for PostgreSQL 11 unprivileged user backup.
2018-06-15 01:05:35 +02:00
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
<block id="br-install">
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<block-variable-replace key="br-install-host">{[host-pg2]}</block-variable-replace>
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
<block-variable-replace key="br-install-user">postgres</block-variable-replace>
<block-variable-replace key="br-install-group">postgres</block-variable-replace>
</block>
</section>
<!-- SECTION => REPLICATION - SETUP-SSH -->
<section id="setup-ssh">
Use command in authorized_hosts to improve SSH security. Unsecured, passwordless SSH can be a scary thing. If an attacker gains access to one system they can easily hop to other systems. Add documentation on how to use the command parameter in authorized_keys to limit ssh to running a single command, pgbackrest. There is more that could be done for security but this likely addresses most needs. Also change references to "trusted ssh" to "passwordless ssh" since this seems more correct. Suggested by Stephen Frost, Magnus Hagander.
2018-09-19 22:41:20 +02:00
<title>Setup Passwordless SSH</title>
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
<block id="setup-ssh-intro">
Add comment regarding vars being required in blocks. Documentation block syntax requires that at least one var be specified. This limitation should be removed but for now add a comment to describe why a bogus var is defined.
2018-11-14 15:13:01 +02:00
<!-- ??? Bogus variable is set because the syntax currently requires at least one -->
<block-variable-replace key="bogus"></block-variable-replace>
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
</block>
<block id="setup-ssh">
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<block-variable-replace key="setup-ssh-host">{[host-pg2]}</block-variable-replace>
Preliminary documentation for PostgreSQL 11 unprivileged user backup.
2018-06-15 01:05:35 +02:00
<block-variable-replace key="setup-ssh-user">postgres</block-variable-replace>
<block-variable-replace key="setup-ssh-user-home-path">{[pg-home-path]}</block-variable-replace>
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
</block>
Preliminary documentation for PostgreSQL 11 unprivileged user backup.
2018-06-15 01:05:35 +02:00
Replace keywords with more flexible if statements. Keywords were extremely limited and prevented us from generating multi-version documentation and other improvements. Replace keywords with an if statement that can evaluate a Perl expression with variable replacement. Since keywords were used to generate cache keys, add a --key-var parameter to identify which variables should make up the key.
2018-12-01 19:40:01 +02:00
<!-- <block if="{[pg-version]} >= 11" id="setup-ssh">
Preliminary documentation for PostgreSQL 11 unprivileged user backup.
2018-06-15 01:05:35 +02:00
<block-variable-replace key="setup-ssh-host">{[host-pg2]}</block-variable-replace>
<block-variable-replace key="setup-ssh-user">{[br-user]}</block-variable-replace>
<block-variable-replace key="setup-ssh-user-home-path">{[br-home-path]}</block-variable-replace>
</block> -->
Updates to installation documentation. * Add trusted SSH configuration. * Add full installation where required and remove doc containers that included parts of the installation.
2017-09-09 06:50:00 +02:00
</section>
<!-- SECTION => REPLICATION - HOT-STANDBY -->
<section id="hot-standby">
<title>Hot Standby</title>
<p>A hot standby performs replication using the WAL archive and allows read-only queries.</p>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Add standby restore type. This restore type automatically adds standby_mode=on to recovery.conf. This could be accomplished previously by setting --recovery-option=standby_mode=on but PostgreSQL 12 requires standby mode to be enabled by a special file named standby.signal. The new restore type allows us to maintain a common interface between PostgreSQL versions.
2019-09-26 23:39:45 +02:00
<p><backrest/> configuration is very similar to <host>{[host-pg1]}</host> except that the <id>standby</id> recovery type will be used to keep the cluster in recovery mode when the end of the WAL stream has been reached.</p>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<backrest-config host="{[host-pg2]}" file="{[backrest-config-demo]}">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Configure <backrest/> on the standby</title>
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<backrest-config-option section="demo" key="pg1-path">{[pg-path]}</backrest-config-option>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Update backup host to repository host in the documentation. Contributed by Cynthia Shang.
2018-02-19 20:17:58 +02:00
<backrest-config-option section="global" key="repo1-host">{[host-repo1]}</backrest-config-option>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Add asynchronous, parallel archive-get. This feature maintains a queue of WAL segments to help reduce latency when PostgreSQL requests a WAL segment with restore_command.
2018-04-30 23:27:39 +02:00
<backrest-config-option section="global" key="log-level-file">detail</backrest-config-option>
Added --log-timestamp option. Allow timestamps to be suppressed in logging. This is primarily used to avoid filters in the automated documentation.
2017-02-05 17:59:49 +02:00
<backrest-config-option section="global" key="log-level-stderr">off</backrest-config-option>
<backrest-config-option section="global" key="log-timestamp">n</backrest-config-option>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
</backrest-config>
Create common if expressions for testing os-type. These expressions simplify os-type testing. This will be especially true as more OS types are added.
2018-12-05 00:33:56 +02:00
<p if="{[os-type-is-debian]}">The demo cluster must be created (even though it will be overwritten on restore) in order to create the <postgres/> configuration files.</p>
All clusters in the documentation are initialized with checksums.
2017-12-14 22:44:06 +02:00
Create common if expressions for testing os-type. These expressions simplify os-type testing. This will be especially true as more OS types are added.
2018-12-05 00:33:56 +02:00
<execute-list if="{[os-type-is-debian]}" host="{[host-pg2]}">
All clusters in the documentation are initialized with checksums.
2017-12-14 22:44:06 +02:00
<title>Create demo cluster</title>
<execute user="root">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<exe-cmd>{[pg-cluster-create]}</exe-cmd>
All clusters in the documentation are initialized with checksums.
2017-12-14 22:44:06 +02:00
</execute>
</execute-list>
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<p if="{[os-type-is-rhel]}">Create the path where <postgres/> will be restored.</p>
All clusters in the documentation are initialized with checksums.
2017-12-14 22:44:06 +02:00
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<execute-list if="{[os-type-is-rhel]}" host="{[host-pg2]}">
All clusters in the documentation are initialized with checksums.
2017-12-14 22:44:06 +02:00
<title>Create <postgres/> path</title>
<execute user="postgres">
<exe-cmd>
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
mkdir -p -m 700 {[pg-path]}
All clusters in the documentation are initialized with checksums.
2017-12-14 22:44:06 +02:00
</exe-cmd>
</execute>
</execute-list>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<p>Now the standby can be created with the <cmd>restore</cmd> command.</p>
Add archive-mode option to disable archiving on restore. When restoring a cluster that will be promoted but is not intended to be the new primary, it is important to disable archiving to avoid polluting the repository with useless WAL. This option makes disabling archiving a bit easier.
2020-08-25 21:05:41 +02:00
<admonition type="important">If the cluster is intended to be promoted without becoming the new primary (e.g. for reporting or testing), use <br-option>--archive-mode=off</br-option> or set <pg-option>archive_mode=off</pg-option> in <file>postgresql.conf</file> to disable archiving. If archiving is not disabled then the repository may be polluted with WAL that can make restores more difficult.</admonition>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg2]}">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Restore the {[postgres-cluster-demo]} standby cluster</title>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Create common if expressions for testing os-type. These expressions simplify os-type testing. This will be especially true as more OS types are added.
2018-12-05 00:33:56 +02:00
<execute user="postgres" if="{[os-type-is-debian]}">
Add standby restore type. This restore type automatically adds standby_mode=on to recovery.conf. This could be accomplished previously by setting --recovery-option=standby_mode=on but PostgreSQL 12 requires standby mode to be enabled by a special file named standby.signal. The new restore type allows us to maintain a common interface between PostgreSQL versions.
2019-09-26 23:39:45 +02:00
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} {[dash]}-delta --type=standby restore</exe-cmd>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
</execute>
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<execute user="postgres" if="{[os-type-is-rhel]}">
Add standby restore type. This restore type automatically adds standby_mode=on to recovery.conf. This could be accomplished previously by setting --recovery-option=standby_mode=on but PostgreSQL 12 requires standby mode to be enabled by a special file named standby.signal. The new restore type allows us to maintain a common interface between PostgreSQL versions.
2019-09-26 23:39:45 +02:00
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} --type=standby restore</exe-cmd>
All clusters in the documentation are initialized with checksums.
2017-12-14 22:44:06 +02:00
</execute>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y" filter="n">
Update recovery.conf in user guide for PostgreSQL >= 12. The postgresql.auto.conf file was being used instead of recovery.conf, but there were still instances in the text that used recovery.conf. Update to postgresql.auto.conf for PostgreSQL >= 10 and change wording where needed.
2020-07-17 19:27:14 +02:00
<exe-cmd>cat {[pg-recovery-path-demo]}</exe-cmd>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
</execute>
Add optional stress test section to the user guide. This is intended to provide pre-release stress-testing. Include container memory limits to help check for memory leaks. Also add parallelism to make for faster builds.
2021-08-12 13:49:59 +02:00
<execute user="root" show="n" user-force="y">
<exe-cmd>echo 'shared_buffers = 16MB' >> {[postgres-config-demo]}</exe-cmd>
</execute>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
</execute-list>
Add configuration to the standby so it works as a primary when promoted. This code was generated during testing and it seemed a good idea to keep it. It is only a partial solution since the primary also needs additional configuration to be able to fail back and forth.
2018-12-07 19:32:10 +02:00
<p>The <pg-setting>hot_standby</pg-setting> setting must be enabled before starting <postgres/> to allow read-only connections on <host>{[host-pg2]}</host>. Otherwise, connection attempts will be refused. The rest of the configuration is in case the standby is promoted to a primary.</p>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<postgres-config host="{[host-pg2]}" file="{[postgres-config-demo]}">
Add configuration to the standby so it works as a primary when promoted. This code was generated during testing and it seemed a good idea to keep it. It is only a partial solution since the primary also needs additional configuration to be able to fail back and forth.
2018-12-07 19:32:10 +02:00
<title>Configure <postgres/></title>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<postgres-config-option key="hot_standby">on</postgres-config-option>
Add configuration to the standby so it works as a primary when promoted. This code was generated during testing and it seemed a good idea to keep it. It is only a partial solution since the primary also needs additional configuration to be able to fail back and forth.
2018-12-07 19:32:10 +02:00
<postgres-config-option key="archive_command">'{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} archive-push %p'</postgres-config-option>
<postgres-config-option key="archive_mode">on</postgres-config-option>
Use wal_level=replica in the documentation for PostgreSQL >= 9.6. The documentation was using wal_level=hot_standby which is a deprecated setting. Also remove the reference to wal_level=archive since it is no longer supported and is not recommended for older versions. Suggested by Patrick McLaughlin.
2019-06-05 13:27:24 +02:00
<postgres-config-option key="wal_level">{[wal-level]}</postgres-config-option>
Add configuration to the standby so it works as a primary when promoted. This code was generated during testing and it seemed a good idea to keep it. It is only a partial solution since the primary also needs additional configuration to be able to fail back and forth.
2018-12-07 19:32:10 +02:00
<postgres-config-option key="max_wal_senders">3</postgres-config-option>
All clusters in the documentation are initialized with checksums.
2017-12-14 22:44:06 +02:00
<postgres-config-option key="log_filename">'postgresql.log'</postgres-config-option>
<postgres-config-option key="log_line_prefix">''</postgres-config-option>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
</postgres-config>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg2]}">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Start <postgres/></title>
<execute user="root" show="n">
<exe-cmd>rm {[postgres-log-demo]}</exe-cmd>
</execute>
<execute user="root">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<exe-cmd>{[pg-cluster-start]}</exe-cmd>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
</execute>
Closed #163: Document Generation Issue Fixed an issue where document generation failed because some OSs are not tolerant of having multiple installed versions of PostgreSQL. A separate VM is now created for each version. Also added a sleep after database starts during document generation to ensure the database is running before the next command runs. Reported by John Harvey.
2016-01-23 01:01:21 +02:00
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" show="n">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<exe-cmd>{[pg-cluster-wait]}</exe-cmd>
Closed #163: Document Generation Issue Fixed an issue where document generation failed because some OSs are not tolerant of having multiple installed versions of PostgreSQL. A separate VM is now created for each version. Also added a sleep after database starts during document generation to ensure the database is running before the next command runs. Reported by John Harvey.
2016-01-23 01:01:21 +02:00
</execute>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
</execute-list>
<p>The <postgres/> log gives valuable information about the recovery. Note especially that the cluster has entered standby mode and is ready to accept read-only connections.</p>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg2]}">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Examine the <postgres/> log output for log messages indicating success</title>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-cmd>cat {[postgres-log-demo]}</exe-cmd>
<exe-highlight>entering standby mode|database system is ready to accept read only connections</exe-highlight>
</execute>
</execute-list>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<p>An easy way to test that replication is properly configured is to create a table on <host>{[host-pg1]}</host>.</p>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Rename master to primary in documentation to align with PostgreSQL convention.
2017-09-07 19:59:40 +02:00
<title>Create a new table on the primary</title>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-cmd>
psql -c "
begin;
create table replicated_table (message text);
insert into replicated_table values ('{[test-table-data]}');
commit;
select * from replicated_table";
</exe-cmd>
<exe-highlight>{[test-table-data]}</exe-highlight>
</execute>
</execute-list>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<p>And then query the same table on <host>{[host-pg2]}</host>.</p>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg2]}">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Query new table on the standby</title>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y" err-expect="1">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-cmd>psql -c "select * from replicated_table;"</exe-cmd>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
<exe-highlight>does not exist</exe-highlight>
</execute>
</execute-list>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<p>So, what went wrong? Since <postgres/> is pulling WAL segments from the archive to perform replication, changes won't be seen on the standby until the WAL segment that contains those changes is pushed from <host>{[host-pg1]}</host>.</p>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Preliminary documentation for PostgreSQL 11 unprivileged user backup.
2018-06-15 01:05:35 +02:00
<p>This can be done manually by calling <code>{[pg-switch-wal]}()</code> which pushes the current WAL segment to the archive (a new WAL segment is created to contain further changes).</p>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Preliminary documentation for PostgreSQL 11 unprivileged user backup.
2018-06-15 01:05:35 +02:00
<title>Call <code>{[pg-switch-wal]}()</code></title>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y" filter="n">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-cmd>
Preliminary documentation for PostgreSQL 11 unprivileged user backup.
2018-06-15 01:05:35 +02:00
psql -c "select *, current_timestamp from {[pg-switch-wal]}()";
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
</exe-cmd>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
</execute>
</execute-list>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<p>Now after a short delay the table will appear on <host>{[host-pg2]}</host>.</p>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg2]}">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Now the new table exists on the standby (may require a few retries)</title>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y" retry="15" filter="n">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-cmd>psql -c "
select *, current_timestamp from replicated_table"</exe-cmd>
<exe-highlight>{[test-table-data]}</exe-highlight>
</execute>
</execute-list>
Improved check command to run on a standby. Only basic checks are done because pg_switch_xlog() cannot be executed on a replica. Contributed by Cynthia Shang.
2016-12-21 21:35:20 +02:00
<p>Check the standby configuration for access to the repository.</p>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg2]}">
Improved check command to run on a standby. Only basic checks are done because pg_switch_xlog() cannot be executed on a replica. Contributed by Cynthia Shang.
2016-12-21 21:35:20 +02:00
<title>Check the configuration</title>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y">
Improved check command to run on a standby. Only basic checks are done because pg_switch_xlog() cannot be executed on a replica. Contributed by Cynthia Shang.
2016-12-21 21:35:20 +02:00
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} {[dash]}-log-level-console=info check</exe-cmd>
The check command is implemented entirely in C. Note that building the manifest on each host has been temporarily removed. This feature will likely be brought back as a non-default option (after the manifest code has been fully migrated to C) since it can be fairly expensive.
2019-10-09 00:04:09 +02:00
<exe-highlight>because this is a standby</exe-highlight>
Improved check command to run on a standby. Only basic checks are done because pg_switch_xlog() cannot be executed on a replica. Contributed by Cynthia Shang.
2016-12-21 21:35:20 +02:00
</execute>
</execute-list>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
</section>
Improved check command to run on a standby. Only basic checks are done because pg_switch_xlog() cannot be executed on a replica. Contributed by Cynthia Shang.
2016-12-21 21:35:20 +02:00
<!-- SECTION => REPLICATION - STREAMING -->
Improved support for --require option and section depends now default to the previous section.
2017-02-11 17:26:54 +02:00
<section id="streaming">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Streaming Replication</title>
Rename master to primary in documentation to align with PostgreSQL convention.
2017-09-07 19:59:40 +02:00
<p>Instead of relying solely on the WAL archive, streaming replication makes a direct connection to the primary and applies changes as soon as they are made on the primary. This results in much less lag between the primary and standby.</p>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<p>Streaming replication requires a user with the replication privilege.</p>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Create replication user</title>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y" filter="n">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-cmd>
psql -c "
create user replicator password 'jw8s0F4' replication";
</exe-cmd>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
</execute>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
</execute-list>
Fix incorrect host name in user guide. The text indicates to populate the pg-primary IP address into the pg_hba.conf file to allow replication connections. It should indicate to populate the pg-standby IP address
2021-10-21 19:51:59 +02:00
<p>The <file>pg_hba.conf</file> file must be updated to allow the standby to connect as the replication user. Be sure to replace the IP address below with the actual IP address of your <host>{[host-pg2]}</host>. A reload will be required after modifying the <file>pg_hba.conf</file> file.</p>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Create <file>pg_hba.conf</file> entry for replication user</title>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-cmd>
sh -c 'echo
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
"host replication replicator {[host-pg2-ip]}/32 md5"
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
>> {[postgres-hba-demo]}'
</exe-cmd>
</execute>
<execute user="root">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<exe-cmd>{[pg-cluster-reload]}</exe-cmd>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
</execute>
</execute-list>
Rename master to primary in documentation to align with PostgreSQL convention.
2017-09-07 19:59:40 +02:00
<p>The standby needs to know how to contact the primary so the <pg-option>primary_conninfo</pg-option> setting will be configured in <backrest/>.</p>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<backrest-config host="{[host-pg2]}" file="{[backrest-config-demo]}">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Set <pg-option>primary_conninfo</pg-option></title>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<backrest-config-option section="demo" key="recovery-option">primary_conninfo=host={[host-pg1-ip]} port=5432 user=replicator</backrest-config-option>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
</backrest-config>
<p>It is possible to configure a password in the <pg-option>primary_conninfo</pg-option> setting but using a <file>.pgpass</file> file is more flexible and secure.</p>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg2]}">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Configure the replication password in the <file>.pgpass</file> file.</title>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-cmd>
sh -c 'echo
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
"{[host-pg1-ip]}:*:replication:replicator:jw8s0F4"
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
>> {[postgres-pgpass]}'
</exe-cmd>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
</execute>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-cmd>chmod 600 {[postgres-pgpass]}</exe-cmd>
</execute>
</execute-list>
<p>Now the standby can be created with the <cmd>restore</cmd> command.</p>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg2]}">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Stop <postgres/> and restore the {[postgres-cluster-demo]} standby cluster</title>
<execute user="root" err-suppress="y">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<exe-cmd>{[pg-cluster-stop]}</exe-cmd>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
</execute>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres">
Add standby restore type. This restore type automatically adds standby_mode=on to recovery.conf. This could be accomplished previously by setting --recovery-option=standby_mode=on but PostgreSQL 12 requires standby mode to be enabled by a special file named standby.signal. The new restore type allows us to maintain a common interface between PostgreSQL versions.
2019-09-26 23:39:45 +02:00
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} {[dash]}-delta --type=standby restore</exe-cmd>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
</execute>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y" filter="n">
Update recovery.conf in user guide for PostgreSQL >= 12. The postgresql.auto.conf file was being used instead of recovery.conf, but there were still instances in the text that used recovery.conf. Update to postgresql.auto.conf for PostgreSQL >= 10 and change wording where needed.
2020-07-17 19:27:14 +02:00
<exe-cmd>cat {[pg-recovery-path-demo]}</exe-cmd>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
</execute>
</execute-list>
Update recovery.conf in user guide for PostgreSQL >= 12. The postgresql.auto.conf file was being used instead of recovery.conf, but there were still instances in the text that used recovery.conf. Update to postgresql.auto.conf for PostgreSQL >= 10 and change wording where needed.
2020-07-17 19:27:14 +02:00
<admonition type="note">The <pg-setting>primary_conninfo</pg-setting> setting has been written into the <file>{[pg-recovery-file-demo]}</file> file because it was configured as a <br-option>recovery-option</br-option> in <file>{[project-exe]}.conf</file>. The <br-setting>{[dash]}-type=preserve</br-setting> option can be used with the <cmd>restore</cmd> to leave the existing <file>{[pg-recovery-file-demo]}</file> file in place if that behavior is preferred.</admonition>
Add standby restore type. This restore type automatically adds standby_mode=on to recovery.conf. This could be accomplished previously by setting --recovery-option=standby_mode=on but PostgreSQL 12 requires standby mode to be enabled by a special file named standby.signal. The new restore type allows us to maintain a common interface between PostgreSQL versions.
2019-09-26 23:39:45 +02:00
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<p if="{[os-type-is-rhel]}">By default {[user-guide-os]} stores the <file>postgresql.conf</file> file in the <postgres/> data directory. That means the change made to <file>postgresql.conf</file> was overwritten by the last restore and the <pg-option>hot_standby</pg-option> setting must be enabled again. Other solutions to this problem are to store the <file>postgresql.conf</file> file elsewhere or to enable the <pg-option>hot_standby</pg-option> setting on the <host>{[host-pg1]}</host> host where it will be ignored.</p>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<postgres-config host="{[host-pg2]}" if="{[os-type-is-rhel]}" file="{[postgres-config-demo]}">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Enable <pg-option>hot_standby</pg-option></title>
<postgres-config-option key="hot_standby">on</postgres-config-option>
</postgres-config>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg2]}">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Start <postgres/></title>
<execute user="root" show="n">
<exe-cmd>rm {[postgres-log-demo]}</exe-cmd>
</execute>
<execute user="root">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<exe-cmd>{[pg-cluster-start]}</exe-cmd>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
</execute>
Closed #163: Document Generation Issue Fixed an issue where document generation failed because some OSs are not tolerant of having multiple installed versions of PostgreSQL. A separate VM is now created for each version. Also added a sleep after database starts during document generation to ensure the database is running before the next command runs. Reported by John Harvey.
2016-01-23 01:01:21 +02:00
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" show="n">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<exe-cmd>{[pg-cluster-wait]}</exe-cmd>
Closed #163: Document Generation Issue Fixed an issue where document generation failed because some OSs are not tolerant of having multiple installed versions of PostgreSQL. A separate VM is now created for each version. Also added a sleep after database starts during document generation to ensure the database is running before the next command runs. Reported by John Harvey.
2016-01-23 01:01:21 +02:00
</execute>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
</execute-list>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<p>The <postgres/> log will confirm that streaming replication has started.</p>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg2]}">
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
<title>Examine the <postgres/> log output for log messages indicating success</title>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-cmd>cat {[postgres-log-demo]}</exe-cmd>
<exe-highlight>started streaming WAL from primary</exe-highlight>
</execute>
</execute-list>
Preliminary documentation for PostgreSQL 11 unprivileged user backup.
2018-06-15 01:05:35 +02:00
<p>Now when a table is created on <host>{[host-pg1]}</host> it will appear on <host>{[host-pg2]}</host> quickly and without the need to call <code>{[pg-switch-wal]}()</code>.</p>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Rename master to primary in documentation to align with PostgreSQL convention.
2017-09-07 19:59:40 +02:00
<title>Create a new table on the primary</title>
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-cmd>
psql -c "
begin;
create table stream_table (message text);
insert into stream_table values ('{[test-table-data]}');
commit;
select *, current_timestamp from stream_table";
</exe-cmd>
<exe-highlight>{[test-table-data]}</exe-highlight>
</execute>
</execute-list>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg2]}">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<title>Query table on the standby</title>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y" retry="2" filter="n">
Added documentation in the user guide for delta restores, expiration, dedicated backup hosts, starting and stopping pgBackRest, and replication.
2015-11-22 23:44:01 +02:00
<exe-cmd>psql -c "
select *, current_timestamp from stream_table"</exe-cmd>
<exe-highlight>{[test-table-data]}</exe-highlight>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
</execute>
</execute-list>
</section>
</section>
Add asynchronous, parallel archive-get. This feature maintains a queue of WAL segments to help reduce latency when PostgreSQL requests a WAL segment with restore_command.
2018-04-30 23:27:39 +02:00
<!-- SECTION => ASYNCHRONOUS ARCHIVING -->
<section id="async-archiving" depend="/replication">
<title>Asynchronous Archiving</title>
<p>Asynchronous archiving is enabled with the <br-option>archive-async</br-option> option. This option enables asynchronous operation for both the <cmd>archive-push</cmd> and <cmd>archive-get</cmd> commands.</p>
<p>A spool path is required. The commands will store transient data here but each command works quite a bit differently so spool path usage is described in detail in each section.</p>
<execute-list host="{[host-pg1]}">
<title>Create the spool directory</title>
<execute user="root">
Embed exported C functions and Perl modules directly into the pgBackRest executable.
2018-05-22 18:53:08 +02:00
<exe-cmd>mkdir -p -m 750 {[spool-path]}</exe-cmd>
Add asynchronous, parallel archive-get. This feature maintains a queue of WAL segments to help reduce latency when PostgreSQL requests a WAL segment with restore_command.
2018-04-30 23:27:39 +02:00
</execute>
<execute user="root">
<exe-cmd>chown postgres:postgres {[spool-path]}</exe-cmd>
</execute>
</execute-list>
<execute-list host="{[host-pg2]}">
<title>Create the spool directory</title>
<execute user="root">
Embed exported C functions and Perl modules directly into the pgBackRest executable.
2018-05-22 18:53:08 +02:00
<exe-cmd>mkdir -p -m 750 {[spool-path]}</exe-cmd>
Add asynchronous, parallel archive-get. This feature maintains a queue of WAL segments to help reduce latency when PostgreSQL requests a WAL segment with restore_command.
2018-04-30 23:27:39 +02:00
</execute>
<execute user="root">
<exe-cmd>chown postgres:postgres {[spool-path]}</exe-cmd>
</execute>
</execute-list>
<p>The spool path must be configured and asynchronous archiving enabled. Asynchronous archiving automatically confers some benefit by reducing the number of connections made to remote storage, but setting <br-option>process-max</br-option> can drastically improve performance by parallelizing operations. Be sure not to set <br-option>process-max</br-option> so high that it affects normal database operations.</p>
<backrest-config host="{[host-pg1]}" file="{[backrest-config-demo]}">
<title>Configure the spool path and asynchronous archiving</title>
<backrest-config-option section="global" key="spool-path">{[spool-path]}</backrest-config-option>
<backrest-config-option section="global" key="archive-async">y</backrest-config-option>
<backrest-config-option section="global:archive-push" key="process-max">2</backrest-config-option>
<backrest-config-option section="global:archive-get" key="process-max">2</backrest-config-option>
</backrest-config>
<backrest-config host="{[host-pg2]}" file="{[backrest-config-demo]}">
<title>Configure the spool path and asynchronous archiving</title>
<backrest-config-option section="global" key="spool-path">{[spool-path]}</backrest-config-option>
<backrest-config-option section="global" key="archive-async">y</backrest-config-option>
<backrest-config-option section="global:archive-push" key="process-max">2</backrest-config-option>
<backrest-config-option section="global:archive-get" key="process-max">2</backrest-config-option>
</backrest-config>
Add admonitions to documentation renderers. Admonitions call out places where the user should take special care. Support added for HTML, PDF, Markdown and help text renderers. XML files have been updated accordingly. Contributed by Cynthia Shang.
2018-12-30 16:40:20 +02:00
<admonition type="note"><br-option>process-max</br-option> is configured using command sections so that the option is not used by backup and restore. This also allows different values for <cmd>archive-push</cmd> and <cmd>archive-get</cmd>.</admonition>
Add asynchronous, parallel archive-get. This feature maintains a queue of WAL segments to help reduce latency when PostgreSQL requests a WAL segment with restore_command.
2018-04-30 23:27:39 +02:00
<p>For demonstration purposes streaming replication will be broken to force <postgres/> to get WAL using the <pg-option>restore_command</pg-option>.</p>
<execute-list host="{[host-pg1]}">
<title>Break streaming replication by changing the replication password</title>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y" filter="n">
Add asynchronous, parallel archive-get. This feature maintains a queue of WAL segments to help reduce latency when PostgreSQL requests a WAL segment with restore_command.
2018-04-30 23:27:39 +02:00
<exe-cmd>
psql -c "alter user replicator password 'bogus'"
</exe-cmd>
</execute>
</execute-list>
<execute-list host="{[host-pg2]}">
<title>Restart standby to break connection</title>
<execute user="root">
<exe-cmd>{[pg-cluster-restart]}</exe-cmd>
</execute>
</execute-list>
<section id="async-archive-push">
<title>Archive Push</title>
<p>The asynchronous <cmd>archive-push</cmd> command offloads WAL archiving to a separate process (or processes) to improve throughput. It works by <quote>looking ahead</quote> to see which WAL segments are ready to be archived beyond the request that <postgres/> is currently making via the <code>archive_command</code>. WAL segments are transferred to the archive directly from the <path>pg_xlog</path>/<path>pg_wal</path> directory and success is only returned by the <code>archive_command</code> when the WAL segment has been safely stored in the archive.</p>
<p>The spool path holds the current status of WAL archiving. Status files written into the spool directory are typically zero length and should consume a minimal amount of space (a few MB at most) and very little IO. All the information in this directory can be recreated so it is not necessary to preserve the spool directory if the cluster is moved to new hardware.</p>
Add admonitions to documentation renderers. Admonitions call out places where the user should take special care. Support added for HTML, PDF, Markdown and help text renderers. XML files have been updated accordingly. Contributed by Cynthia Shang.
2018-12-30 16:40:20 +02:00
<admonition type="important">In the original implementation of asynchronous archiving, WAL segments were copied to the spool directory before compression and transfer. The new implementation copies WAL directly from the <path>pg_xlog</path> directory. If asynchronous archiving was utilized in <proper>v1.12</proper> or prior, read the <proper>v1.13</proper> release notes carefully before upgrading.</admonition>
Add asynchronous, parallel archive-get. This feature maintains a queue of WAL segments to help reduce latency when PostgreSQL requests a WAL segment with restore_command.
2018-04-30 23:27:39 +02:00
<p>The <file>[stanza]-archive-push-async.log</file> file can be used to monitor the activity of the asynchronous process. A good way to test this is to quickly push a number of WAL segments.</p>
<execute-list host="{[host-pg1]}">
<title>Test parallel asynchronous archiving</title>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="n" show="n">
Add asynchronous, parallel archive-get. This feature maintains a queue of WAL segments to help reduce latency when PostgreSQL requests a WAL segment with restore_command.
2018-04-30 23:27:39 +02:00
<exe-cmd>rm -f /var/log/pgbackrest/demo-archive-push-async.log</exe-cmd>
</execute>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="n">
Add asynchronous, parallel archive-get. This feature maintains a queue of WAL segments to help reduce latency when PostgreSQL requests a WAL segment with restore_command.
2018-04-30 23:27:39 +02:00
<exe-cmd>
psql -c "
Preliminary documentation for PostgreSQL 11 unprivileged user backup.
2018-06-15 01:05:35 +02:00
select pg_create_restore_point('test async push'); select {[pg-switch-wal]}();
select pg_create_restore_point('test async push'); select {[pg-switch-wal]}();
select pg_create_restore_point('test async push'); select {[pg-switch-wal]}();
select pg_create_restore_point('test async push'); select {[pg-switch-wal]}();
select pg_create_restore_point('test async push'); select {[pg-switch-wal]}();"
Add asynchronous, parallel archive-get. This feature maintains a queue of WAL segments to help reduce latency when PostgreSQL requests a WAL segment with restore_command.
2018-04-30 23:27:39 +02:00
</exe-cmd>
</execute>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres">
Add asynchronous, parallel archive-get. This feature maintains a queue of WAL segments to help reduce latency when PostgreSQL requests a WAL segment with restore_command.
2018-04-30 23:27:39 +02:00
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} {[dash]}-log-level-console=info check</exe-cmd>
<exe-highlight>WAL segment</exe-highlight>
</execute>
</execute-list>
<p>Now the log file will contain parallel, asynchronous activity.</p>
<execute-list host="{[host-pg1]}">
<title>Check results in the log</title>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y">
Add asynchronous, parallel archive-get. This feature maintains a queue of WAL segments to help reduce latency when PostgreSQL requests a WAL segment with restore_command.
2018-04-30 23:27:39 +02:00
<exe-cmd>cat /var/log/pgbackrest/demo-archive-push-async.log</exe-cmd>
v2.12: C Implementation of Archive Push IMPORTANT NOTE: The new TLS/SSL implementation forbids dots in S3 bucket names per RFC-2818. This security fix is required for compliant hostname verification. Bug Fixes: * Fix issues when a path option is / terminated. (Reported by Marc Cousin.) * Fix issues when log-level-file=off is set for the archive-get command. (Reported by Brad Nicholson.) * Fix C code to recognize host:port option format like Perl does. (Reported by Kyle Nevins.) * Fix issues with remote/local command logging options. Improvements: * The archive-push command is implemented entirely in C. * Increase process-max limit to 999. (Suggested by Rakshitha-BR.) * Improve error message when an S3 bucket name contains dots. Documentation Improvements: * Clarify that S3-compatible object stores are supported. (Suggested by Magnus Hagander.)
2019-04-11 15:14:22 +02:00
<exe-highlight> WAL file\(s\) to archive|pushed WAL file \'0000000</exe-highlight>
Add asynchronous, parallel archive-get. This feature maintains a queue of WAL segments to help reduce latency when PostgreSQL requests a WAL segment with restore_command.
2018-04-30 23:27:39 +02:00
</execute>
</execute-list>
</section>
<section id="async-archive-get">
<title>Archive Get</title>
<p>The asynchronous <cmd>archive-get</cmd> command maintains a local queue of WAL to improve throughput. If a WAL segment is not found in the queue it is fetched from the repository along with enough consecutive WAL to fill the queue. The maximum size of the queue is defined by <br-option>archive-get-queue-max</br-option>. Whenever the queue is less than half full more WAL will be fetched to fill it.</p>
Clarify that S3-compatible object stores are supported. The documentation mentioned Amazon S3 frequently but failed to mention that other S3-compatible object stores are also supported. Tone down the specific mentions of Amazon S3 and replace them with "S3-compatible object store" when appropriate. Suggested by Magnus Hagander.
2019-04-03 12:40:12 +02:00
<p>Asynchronous operation is most useful in environments that generate a lot of WAL or have a high latency connection to the repository storage (i.e., <proper>S3</proper> or other object stores). In the case of a high latency connection it may be a good idea to increase <br-option>process-max</br-option>.</p>
Add asynchronous, parallel archive-get. This feature maintains a queue of WAL segments to help reduce latency when PostgreSQL requests a WAL segment with restore_command.
2018-04-30 23:27:39 +02:00
<p>The <file>[stanza]-archive-get-async.log</file> file can be used to monitor the activity of the asynchronous process.</p>
<execute-list host="{[host-pg2]}">
<title>Check results in the log</title>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" show="n">
Add asynchronous, parallel archive-get. This feature maintains a queue of WAL segments to help reduce latency when PostgreSQL requests a WAL segment with restore_command.
2018-04-30 23:27:39 +02:00
<exe-cmd>sleep 5</exe-cmd>
</execute>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y">
Add asynchronous, parallel archive-get. This feature maintains a queue of WAL segments to help reduce latency when PostgreSQL requests a WAL segment with restore_command.
2018-04-30 23:27:39 +02:00
<exe-cmd>cat /var/log/pgbackrest/demo-archive-get-async.log</exe-cmd>
Improve logging in archive-get command. Append "asynchronously" to messages when the async process fetched the file (not in the actual async process log, though). Add "repo1" to make it clear what archive we are talking about. This is not very useful by itself but soon we'll be able to add the archive id, which is very useful. Add constants for messages that are used multiple times to ensure they stay consistent.
2021-01-13 17:24:47 +02:00
<exe-highlight>found [0-F]{24} in the .* archive</exe-highlight>
Add asynchronous, parallel archive-get. This feature maintains a queue of WAL segments to help reduce latency when PostgreSQL requests a WAL segment with restore_command.
2018-04-30 23:27:39 +02:00
</execute>
</execute-list>
</section>
<execute-list host="{[host-pg1]}">
<title>Fix streaming replication by changing the replication password</title>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y" filter="n">
Add asynchronous, parallel archive-get. This feature maintains a queue of WAL segments to help reduce latency when PostgreSQL requests a WAL segment with restore_command.
2018-04-30 23:27:39 +02:00
<exe-cmd>
psql -c "alter user replicator password 'jw8s0F4'"
</exe-cmd>
</execute>
</execute-list>
</section>
Backup from a standby cluster. A connection to the primary cluster is still required to start/stop the backup and copy files that are not replicated, but the vast majority of files are copied from the standby in order to reduce load on the master.
2016-08-25 17:25:46 +02:00
<!-- SECTION => STANDBY-BACKUP -->
<section id="standby-backup" depend="/replication/streaming">
<title>Backup from a Standby</title>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<p><backrest/> can perform backups on a standby instead of the primary. Standby backups require the <host>{[host-pg2]}</host> host to be configured and the <br-option>backup-standby</br-option> option enabled. If more than one standby is configured then the first running standby found will be used for the backup.</p>
Backup from a standby cluster. A connection to the primary cluster is still required to start/stop the backup and copy files that are not replicated, but the vast majority of files are copied from the standby in order to reduce load on the master.
2016-08-25 17:25:46 +02:00
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<backrest-config host="{[host-repo1]}" owner="{[br-user]}:{[br-group]}" file="{[backrest-config-demo]}">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<title>Configure <br-option>pg2-host</br-option>/<br-option>pg2-host-user</br-option> and <br-option>pg2-path</br-option></title>
Backup from a standby cluster. A connection to the primary cluster is still required to start/stop the backup and copy files that are not replicated, but the vast majority of files are copied from the standby in order to reduce load on the master.
2016-08-25 17:25:46 +02:00
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<backrest-config-option section="demo" key="pg2-path">{[pg-path]}</backrest-config-option>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<backrest-config-option section="demo" key="pg2-host">{[host-pg2]}</backrest-config-option>
Comment out unprivileged user documentation. We haven't had the time to complete this documentation and it has suffered bit rot. This prevents us from building the docs on PostgreSQL >= 11 so just comment it all out until it can be updated.
2019-09-26 22:02:39 +02:00
<!-- <backrest-config-option if="{[pg-version]} >= 11" section="demo"
key="pg2-host-user">{[br-user]}</backrest-config-option> -->
Backup from a standby cluster. A connection to the primary cluster is still required to start/stop the backup and copy files that are not replicated, but the vast majority of files are copied from the standby in order to reduce load on the master.
2016-08-25 17:25:46 +02:00
<backrest-config-option section="global" key="backup-standby">y</backrest-config-option>
</backrest-config>
Rename master to primary in documentation to align with PostgreSQL convention.
2017-09-07 19:59:40 +02:00
<p>Both the primary and standby databases are required to perform the backup, though the vast majority of the files will be copied from the standby to reduce load on the primary. The database hosts can be configured in any order. <backrest/> will automatically determine which is the primary and which is the standby.</p>
Backup from a standby cluster. A connection to the primary cluster is still required to start/stop the backup and copy files that are not replicated, but the vast majority of files are copied from the standby in order to reduce load on the master.
2016-08-25 17:25:46 +02:00
Update backup host to repository host in the documentation. Contributed by Cynthia Shang.
2018-02-19 20:17:58 +02:00
<execute-list host="{[host-repo1]}">
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<title>Backup the {[postgres-cluster-demo]} cluster from <host>pg2</host></title>
Backup from a standby cluster. A connection to the primary cluster is still required to start/stop the backup and copy files that are not replicated, but the vast majority of files are copied from the standby in order to reduce load on the master.
2016-08-25 17:25:46 +02:00
Preliminary documentation for PostgreSQL 11 unprivileged user backup.
2018-06-15 01:05:35 +02:00
<execute user="{[br-user]}" output="y" filter="y">
Backup from a standby cluster. A connection to the primary cluster is still required to start/stop the backup and copy files that are not replicated, but the vast majority of files are copied from the standby in order to reduce load on the master.
2016-08-25 17:25:46 +02:00
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} --log-level-console=detail backup</exe-cmd>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<exe-highlight>backup file {[host-pg1]}|replay on the standby</exe-highlight>
Backup from a standby cluster. A connection to the primary cluster is still required to start/stop the backup and copy files that are not replicated, but the vast majority of files are copied from the standby in order to reduce load on the master.
2016-08-25 17:25:46 +02:00
</execute>
</execute-list>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<p>This incremental backup shows that most of the files are copied from the <host>{[host-pg2]}</host> host and only a few are copied from the <host>{[host-pg1]}</host> host.</p>
Backup from a standby cluster. A connection to the primary cluster is still required to start/stop the backup and copy files that are not replicated, but the vast majority of files are copied from the standby in order to reduce load on the master.
2016-08-25 17:25:46 +02:00
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<p><backrest/> creates a standby backup that is identical to a backup performed on the primary. It does this by starting/stopping the backup on the <host>{[host-pg1]}</host> host, copying only files that are replicated from the <host>{[host-pg2]}</host> host, then copying the remaining few files from the <host>{[host-pg1]}</host> host. This means that logs and statistics from the primary database will be included in the backup.</p>
Backup from a standby cluster. A connection to the primary cluster is still required to start/stop the backup and copy files that are not replicated, but the vast majority of files are copied from the standby in order to reduce load on the master.
2016-08-25 17:25:46 +02:00
</section>
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
Add instructions for dev builds to stress test in user guide.
2021-08-19 17:35:49 +02:00
<!-- ===========================================================================================================================
This optional section is used to perform stress testing. See the stress* variables for settings.
Once the stress test has been run, it is common to perform some additional tests with a custom/debug build.
Improve instructions for rebuilding pgbackrest during stress testing.
2021-10-07 17:01:48 +02:00
To initialize the build, run the following. This only needs to be done once per doc build or branch change, though running it more that once only costs additional time:
docker exec -it doc-build bash -c 'rm -rf /build/dev && mkdir -p /build/dev && cd /build/dev && /pgbackrest/src/configure -\-enable-test'
Add instructions for dev builds to stress test in user guide.
2021-08-19 17:35:49 +02:00
One each host where a new build is required, run the following. This will build pgbackrest, copy it to the host, and display the build time for sanity:
Improve instructions for rebuilding pgbackrest during stress testing.
2021-10-07 17:01:48 +02:00
docker exec -it <container> bash -c 'ssh build make -C /build/dev -j 8 && scp build:/build/dev/pgbackrest /usr/bin && ls -lah /usr/bin/pgbackrest'
Add instructions for dev builds to stress test in user guide.
2021-08-19 17:35:49 +02:00
============================================================================================================================ -->
Add optional stress test section to the user guide. This is intended to provide pre-release stress-testing. Include container memory limits to help check for memory leaks. Also add parallelism to make for faster builds.
2021-08-12 13:49:59 +02:00
<section id="stress" if="'{[stress]}' eq 'y'">
<title>Stress Testing</title>
<section id="configuration">
<title>Configuration</title>
<backrest-config host="{[host-repo1]}" owner="{[br-user]}:{[br-group]}" file="{[backrest-config-demo]}">
<title>Configure {[host-repo1]} for stress testing</title>
<backrest-config-option section="global" key="process-max">8</backrest-config-option>
<backrest-config-option section="global" key="compress-type">lz4</backrest-config-option>
<backrest-config-option section="global" key="compress-level">1</backrest-config-option>
<backrest-config-option section="global" key="repo1-retention-full">1</backrest-config-option>
<backrest-config-option section="global" key="log-timestamp">y</backrest-config-option>
</backrest-config>
<execute-list host="{[host-pg1]}">
<title>Create the {[host-pg1]} spool directory</title>
<execute user="root">
<exe-cmd>mkdir -p -m 750 {[spool-path]}</exe-cmd>
</execute>
<execute user="root">
<exe-cmd>chown postgres:postgres {[spool-path]}</exe-cmd>
</execute>
</execute-list>
<backrest-config host="{[host-pg1]}" file="{[backrest-config-demo]}">
<title>Configure {[host-pg1]} for stress testing</title>
<backrest-config-option section="global" key="process-max">8</backrest-config-option>
<backrest-config-option section="global" key="log-timestamp">y</backrest-config-option>
<backrest-config-option section="global" key="compress-type">lz4</backrest-config-option>
<backrest-config-option section="global" key="compress-level">1</backrest-config-option>
<backrest-config-option section="global" key="spool-path">{[spool-path]}</backrest-config-option>
<backrest-config-option section="global" key="archive-async">y</backrest-config-option>
<backrest-config-option section="global:archive-push" key="process-max">4</backrest-config-option>
<backrest-config-option section="global:archive-get" key="process-max">4</backrest-config-option>
</backrest-config>
<execute-list host="{[host-pg2]}">
<title>Create the {[host-pg2]} spool directory</title>
<execute user="root">
<exe-cmd>mkdir -p -m 750 {[spool-path]}</exe-cmd>
</execute>
<execute user="root">
<exe-cmd>chown postgres:postgres {[spool-path]}</exe-cmd>
</execute>
</execute-list>
<backrest-config host="{[host-pg2]}" file="{[backrest-config-demo]}">
<title>Configure {[host-pg2]} for stress testing</title>
<backrest-config-option section="global" key="process-max">8</backrest-config-option>
<backrest-config-option section="global" key="log-timestamp">y</backrest-config-option>
<backrest-config-option section="global" key="compress-type">lz4</backrest-config-option>
<backrest-config-option section="global" key="compress-level">1</backrest-config-option>
<backrest-config-option section="global" key="spool-path">{[spool-path]}</backrest-config-option>
<backrest-config-option section="global" key="archive-async">y</backrest-config-option>
<backrest-config-option section="global:archive-push" key="process-max">4</backrest-config-option>
<backrest-config-option section="global:archive-get" key="process-max">4</backrest-config-option>
</backrest-config>
</section>
<section id="data-load">
<title>Create Tables and Load Data</title>
<section id="streaming-break">
<title>Break Streaming Replication</title>
<p>Break streaming replication to force the standby to replicate from the archive during data load.</p>
<execute-list host="{[host-pg1]}">
<title>Break streaming replication by changing the replication password</title>
<execute user="postgres" output="y" filter="n">
<exe-cmd>
psql -c "alter user replicator password 'bogus'"
</exe-cmd>
</execute>
</execute-list>
<execute-list host="{[host-pg2]}">
<title>Restart standby to break connection</title>
<execute user="root">
<exe-cmd>{[pg-cluster-restart]}</exe-cmd>
</execute>
</execute-list>
</section>
<section id="table-create">
<title>Create Tables</title>
<execute-list host="{[host-pg1]}">
<title>Create tables</title>
<execute user="postgres" show="n">
<exe-cmd>psql -c "
create or replace function create_test_table(prefix int, scale int) returns void as \$\$
declare
index int;
begin
for index in 1 .. scale loop
execute 'create table test_' || prefix || '_' || index || ' (id int)';
end loop;
end \$\$ LANGUAGE plpgsql;"</exe-cmd>
</execute>
<execute user="postgres">
<exe-cmd>
bash -c 'for i in {1..{[stress-scale-table]}};
do psql -c "select create_test_table(${i?}, 1000)";
done'
</exe-cmd>
</execute>
</execute-list>
</section>
<section id="data-load">
<title>Load Data</title>
<execute-list host="{[host-pg1]}">
<title>Load data</title>
<execute user="postgres">
Add path to pgbench used for stress test in user guide. This allows the stress test to run on RHEL.
2021-11-13 00:15:45 +02:00
<exe-cmd>{[pg-bin-path]}/pgbench -n -i -s {[stress-scale-data]}</exe-cmd>
Add optional stress test section to the user guide. This is intended to provide pre-release stress-testing. Include container memory limits to help check for memory leaks. Also add parallelism to make for faster builds.
2021-08-12 13:49:59 +02:00
<exe-cmd-extra>2>&amp;1</exe-cmd-extra>
</execute>
</execute-list>
</section>
<section id="streaming-fix">
<title>Fix Streaming Replication</title>
<p>Fix streaming replication so backups will work. Note that streaming replication will not start again until all WAL in the archive has been exhausted.</p>
<execute-list host="{[host-pg1]}">
<title>Fix streaming replication by changing the replication password</title>
<execute user="postgres" output="y" filter="n">
<exe-cmd>
psql -c "alter user replicator password 'jw8s0F4'"
</exe-cmd>
</execute>
</execute-list>
</section>
</section>
<section id="test">
<title>Testing</title>
<section id="backup-full">
<title>Full Backup</title>
<execute-list host="{[host-repo1]}">
<title>Full backup</title>
<execute user="{[br-user]}" output="y" filter="n">
<exe-cmd>
pgbackrest --stanza=demo --type=full
--log-level-console=info --log-level-file=detail backup
</exe-cmd>
<exe-cmd-extra>2>&amp;1</exe-cmd-extra>
</execute>
</execute-list>
</section>
<section id="backup-diff-delta">
<title>Diff Backup with Delta</title>
<execute-list host="{[host-repo1]}">
<title>Diff backup</title>
<execute user="{[br-user]}" output="y" filter="n">
<exe-cmd>
pgbackrest --stanza=demo --type=diff --delta
--log-level-console=info --log-level-file=detail backup
</exe-cmd>
<exe-cmd-extra>2>&amp;1</exe-cmd-extra>
</execute>
</execute-list>
</section>
<section id="restore-delta">
<title>Restore with Delta</title>
<execute-list host="{[host-pg2]}">
<title>Stop <postgres/></title>
<execute user="root">
<exe-cmd>{[pg-cluster-stop]}</exe-cmd>
</execute>
</execute-list>
<execute-list host="{[host-pg2]}">
<title>Restore</title>
<execute user="postgres" output="y" filter="n">
<exe-cmd>
pgbackrest --stanza=demo --type=standby --delta
--log-level-console=info --log-level-file=detail restore
</exe-cmd>
<exe-cmd-extra>2>&amp;1</exe-cmd-extra>
</execute>
</execute-list>
</section>
<section id="restore">
<title>Restore</title>
<execute-list host="{[host-pg2]}">
<title>Remove data</title>
<execute user="postgres">
<exe-cmd>
rm -rf {[pg-path]}
</exe-cmd>
</execute>
</execute-list>
<execute-list host="{[host-pg2]}">
<title>Restore</title>
<execute user="postgres" output="y" filter="n">
<exe-cmd>
pgbackrest --stanza=demo --type=standby
--log-level-console=info --log-level-file=detail restore
</exe-cmd>
<exe-cmd-extra>2>&amp;1</exe-cmd-extra>
</execute>
</execute-list>
<execute-list host="{[host-pg2]}">
<title>Start <postgres/></title>
<execute user="root">
<exe-cmd>{[pg-cluster-start]}</exe-cmd>
</execute>
</execute-list>
<execute-list host="{[host-pg2]}">
<title>Check cluster</title>
<execute user="postgres" output="y" retry="30" filter="n">
<exe-cmd>psql -c "select count(*) from pg_class"</exe-cmd>
</execute>
</execute-list>
</section>
</section>
</section>
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
<!-- SECTION => STANZA UPGRADE -->
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
<section id="upgrade-stanza">
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
<title>Upgrading <postgres/></title>
<cmd-description key="stanza-upgrade"/>
All clusters in the documentation are initialized with checksums.
2017-12-14 22:44:06 +02:00
<p>The following instructions are not meant to be a comprehensive guide for upgrading <postgres/>, rather they outline the general process for upgrading a primary and standby with the intent of demonstrating the steps required to reconfigure <backrest/>. It is recommended that a backup be taken prior to upgrading.</p>
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Move most host setup to containers defined in the documentation. This includes PostgreSQL installation which had previously been included in the documentation. This way produces faster builds and there is no need for us to document PostgreSQL installation.
2018-07-23 13:59:24 +02:00
<title>Stop old cluster</title>
All clusters in the documentation are initialized with checksums.
2017-12-14 22:44:06 +02:00
<execute user="root">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<exe-cmd>{[pg-cluster-stop]}</exe-cmd>
All clusters in the documentation are initialized with checksums.
2017-12-14 22:44:06 +02:00
</execute>
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
</execute-list>
All clusters in the documentation are initialized with checksums.
2017-12-14 22:44:06 +02:00
<p>Stop the old cluster on the standby since it will be restored from the newly upgraded cluster.</p>
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg2]}">
Move most host setup to containers defined in the documentation. This includes PostgreSQL installation which had previously been included in the documentation. This way produces faster builds and there is no need for us to document PostgreSQL installation.
2018-07-23 13:59:24 +02:00
<title>Stop old cluster</title>
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
<execute user="root">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<exe-cmd>{[pg-cluster-stop]}</exe-cmd>
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
</execute>
</execute-list>
All clusters in the documentation are initialized with checksums.
2017-12-14 22:44:06 +02:00
<p>Create the new cluster and perform upgrade.</p>
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
All clusters in the documentation are initialized with checksums.
2017-12-14 22:44:06 +02:00
<title>Create new cluster and perform the upgrade</title>
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
All clusters in the documentation are initialized with checksums.
2017-12-14 22:44:06 +02:00
<execute user="postgres">
<exe-cmd>
{[pg-bin-upgrade-path]}/initdb
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
-D {[pg-path-upgrade]} -k -A peer
All clusters in the documentation are initialized with checksums.
2017-12-14 22:44:06 +02:00
</exe-cmd>
</execute>
Create common if expressions for testing os-type. These expressions simplify os-type testing. This will be especially true as more OS types are added.
2018-12-05 00:33:56 +02:00
<execute user="root" if="{[os-type-is-debian]}">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<exe-cmd>{[pg-cluster-create-upgrade]}</exe-cmd>
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
</execute>
Create common if expressions for testing os-type. These expressions simplify os-type testing. This will be especially true as more OS types are added.
2018-12-05 00:33:56 +02:00
<execute user="postgres" output="y" if="{[os-type-is-debian]}">
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
<exe-cmd>sh -c 'cd /var/lib/postgresql &amp;&amp;
/usr/lib/postgresql/{[pg-version-upgrade]}/bin/pg_upgrade
{[dash]}-old-bindir=/usr/lib/postgresql/{[pg-version]}/bin
{[dash]}-new-bindir=/usr/lib/postgresql/{[pg-version-upgrade]}/bin
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
{[dash]}-old-datadir={[pg-path]}
{[dash]}-new-datadir={[pg-path-upgrade]}
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
{[dash]}-old-options=" -c config_file={[postgres-config-demo]}"
{[dash]}-new-options=" -c config_file={[postgres-config-demo-upgrade]}"'
</exe-cmd>
<exe-highlight>Upgrade Complete</exe-highlight>
</execute>
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<execute user="postgres" output="y" if="{[os-type-is-rhel]}">
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
<exe-cmd>sh -c 'cd /var/lib/pgsql &amp;&amp;
/usr/pgsql-{[pg-version-upgrade]}/bin/pg_upgrade
{[dash]}-old-bindir=/usr/pgsql-{[pg-version]}/bin
{[dash]}-new-bindir=/usr/pgsql-{[pg-version-upgrade]}/bin
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
{[dash]}-old-datadir={[pg-path]}
{[dash]}-new-datadir={[pg-path-upgrade]}
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
{[dash]}-old-options=" -c config_file={[postgres-config-demo]}"
{[dash]}-new-options=" -c config_file={[postgres-config-demo-upgrade]}"'
</exe-cmd>
<exe-highlight>Upgrade Complete</exe-highlight>
</execute>
</execute-list>
<p>Configure the new cluster settings and port.</p>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<postgres-config host="{[host-pg1]}" file="{[postgres-config-demo-upgrade]}">
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
<title>Configure <postgres/></title>
<postgres-config-option key="archive_command">'{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} archive-push %p'</postgres-config-option>
<postgres-config-option key="archive_mode">on</postgres-config-option>
Use wal_level=replica in the documentation for PostgreSQL >= 9.6. The documentation was using wal_level=hot_standby which is a deprecated setting. Also remove the reference to wal_level=archive since it is no longer supported and is not recommended for older versions. Suggested by Patrick McLaughlin.
2019-06-05 13:27:24 +02:00
<postgres-config-option key="wal_level">{[wal-level]}</postgres-config-option>
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
<postgres-config-option key="max_wal_senders">3</postgres-config-option>
<postgres-config-option key="listen_addresses">'*'</postgres-config-option>
<postgres-config-option key="log_line_prefix">''</postgres-config-option>
<postgres-config-option key="port">5432</postgres-config-option>
</postgres-config>
<p>Update the <backrest/> configuration on all systems to point to the new cluster.</p>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<backrest-config host="{[host-pg1]}" file="{[backrest-config-demo]}">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<title>Upgrade the <br-option>pg1-path</br-option></title>
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<backrest-config-option section="demo" key="pg1-path">{[pg-path-upgrade]}</backrest-config-option>
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
</backrest-config>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<backrest-config host="{[host-pg2]}" file="{[backrest-config-demo]}">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<title>Upgrade the <br-option>pg-path</br-option></title>
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<backrest-config-option section="demo" key="pg1-path">{[pg-path-upgrade]}</backrest-config-option>
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
</backrest-config>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<backrest-config host="{[host-repo1]}" owner="{[br-user]}:{[br-group]}" file="{[backrest-config-demo]}">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<title>Upgrade <br-option>pg1-path</br-option> and <br-option>pg2-path</br-option>, disable backup from standby</title>
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<backrest-config-option section="demo" key="pg1-path">{[pg-path-upgrade]}</backrest-config-option>
<backrest-config-option section="demo" key="pg2-path">{[pg-path-upgrade]}</backrest-config-option>
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
<backrest-config-option section="global" key="backup-standby">n</backrest-config-option>
</backrest-config>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
<title>Copy hba configuration</title>
<execute user="root">
<exe-cmd>cp {[postgres-hba-demo]}
{[postgres-hba-demo-upgrade]}</exe-cmd>
</execute>
</execute-list>
Allow stanza-* commands to be run remotely. The stanza-create, stanza-upgrade and stanza-delete were required to be run on the repository host. When there was only one repository allowed this was not a problem. However, with the introduction of multiple repository support, this becomes more of a burden to the user, therefore the stanza-create, stanza-upgrade and stanza-delete commands have been improved to allow for them to be run remotely.
2021-03-10 15:10:46 +02:00
<p>Before starting the new cluster, the <cmd>stanza-upgrade</cmd> command must be run.</p>
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
Allow stanza-* commands to be run remotely. The stanza-create, stanza-upgrade and stanza-delete were required to be run on the repository host. When there was only one repository allowed this was not a problem. However, with the introduction of multiple repository support, this becomes more of a burden to the user, therefore the stanza-create, stanza-upgrade and stanza-delete commands have been improved to allow for them to be run remotely.
2021-03-10 15:10:46 +02:00
<execute-list host="{[host-pg1]}">
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
<title>Upgrade the stanza</title>
Allow stanza-* commands to be run remotely. The stanza-create, stanza-upgrade and stanza-delete were required to be run on the repository host. When there was only one repository allowed this was not a problem. However, with the introduction of multiple repository support, this becomes more of a burden to the user, therefore the stanza-create, stanza-upgrade and stanza-delete commands have been improved to allow for them to be run remotely.
2021-03-10 15:10:46 +02:00
<execute user="postgres" output="y">
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} {[dash]}-no-online
{[dash]}-log-level-console=info stanza-upgrade</exe-cmd>
<exe-highlight>completed successfully</exe-highlight>
</execute>
</execute-list>
<p>Start the new cluster and confirm it is successfully installed.</p>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
<title>Start new cluster</title>
<execute user="root" output="y">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<exe-cmd>{[pg-cluster-start-upgrade]}</exe-cmd>
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
</execute>
</execute-list>
Improve check command to verify that the backup manifest can be built. Contributed by Cynthia Shang.
2017-12-13 18:16:27 +02:00
<p>Test configuration using the <cmd>check</cmd> command.</p>
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
<title>Check configuration</title>
Do not show output of PostgreSQL upgrade status check in user guide. On some platforms the output may contain UTF-8 characters that the latex code is not prepared to handle. Showing the command is much more important than showing the output, so no big loss.
2021-10-13 19:20:11 +02:00
<execute user="postgres" output="n" filter="n">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<exe-cmd>{[pg-cluster-check-upgrade]}</exe-cmd>
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
</execute>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" output="y" filter="n">
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} check</exe-cmd>
</execute>
</execute-list>
Improvements and additional tests for stanza-upgrade. Contributed by Cynthia Shang.
2017-04-07 01:36:37 +02:00
<p>Remove the old cluster.</p>
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg1]}">
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
<title>Remove old cluster</title>
Create common if expressions for testing os-type. These expressions simplify os-type testing. This will be especially true as more OS types are added.
2018-12-05 00:33:56 +02:00
<execute if="{[os-type-is-debian]}" user="root">
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
<exe-cmd>pg_dropcluster {[pg-version]} {[postgres-cluster-demo]}</exe-cmd>
</execute>
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<execute if="{[os-type-is-rhel]}" user="root">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<exe-cmd>rm -rf {[pg-path]}</exe-cmd>
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
</execute>
</execute-list>
Improvements and additional tests for stanza-upgrade. Contributed by Cynthia Shang.
2017-04-07 01:36:37 +02:00
<p>Install the new <postgres/> binaries on the standby and create the cluster.</p>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg2]}">
All clusters in the documentation are initialized with checksums.
2017-12-14 22:44:06 +02:00
<title>Remove old cluster and create the new cluster</title>
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
Create common if expressions for testing os-type. These expressions simplify os-type testing. This will be especially true as more OS types are added.
2018-12-05 00:33:56 +02:00
<execute if="{[os-type-is-debian]}" user="root">
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
<exe-cmd>pg_dropcluster {[pg-version]} {[postgres-cluster-demo]}</exe-cmd>
</execute>
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<execute if="{[os-type-is-rhel]}" user="root">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<exe-cmd>rm -rf {[pg-path]}</exe-cmd>
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
</execute>
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<execute user="postgres" if="{[os-type-is-rhel]}">
Improvements and additional tests for stanza-upgrade. Contributed by Cynthia Shang.
2017-04-07 01:36:37 +02:00
<exe-cmd>
All clusters in the documentation are initialized with checksums.
2017-12-14 22:44:06 +02:00
mkdir -p -m 700 {[pg-bin-upgrade-path]}
</exe-cmd>
Improvements and additional tests for stanza-upgrade. Contributed by Cynthia Shang.
2017-04-07 01:36:37 +02:00
</execute>
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
Create common if expressions for testing os-type. These expressions simplify os-type testing. This will be especially true as more OS types are added.
2018-12-05 00:33:56 +02:00
<execute user="root" if="{[os-type-is-debian]}">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<exe-cmd>{[pg-cluster-create-upgrade]}</exe-cmd>
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
</execute>
</execute-list>
Update backup host to repository host in the documentation. Contributed by Cynthia Shang.
2018-02-19 20:17:58 +02:00
<p>Run the <cmd>check</cmd> on the repository host. The warning regarding the standby being down is expected since the standby cluster is down. Running this command demonstrates that the repository server is aware of the standby and is configured properly for the primary server.</p>
Improve check command to verify that the backup manifest can be built. Contributed by Cynthia Shang.
2017-12-13 18:16:27 +02:00
Update backup host to repository host in the documentation. Contributed by Cynthia Shang.
2018-02-19 20:17:58 +02:00
<execute-list host="{[host-repo1]}">
Improve check command to verify that the backup manifest can be built. Contributed by Cynthia Shang.
2017-12-13 18:16:27 +02:00
<title>Check configuration</title>
Preliminary documentation for PostgreSQL 11 unprivileged user backup.
2018-06-15 01:05:35 +02:00
<execute user="{[br-user]}" output="y" filter="n" >
Improve check command to verify that the backup manifest can be built. Contributed by Cynthia Shang.
2017-12-13 18:16:27 +02:00
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} check</exe-cmd>
</execute>
</execute-list>
<p>Run a full backup on the new cluster and then restore the standby from the backup. The backup type will automatically be changed to <id>full</id> if <id>incr</id> or <id>diff</id> is requested.</p>
Update backup host to repository host in the documentation. Contributed by Cynthia Shang.
2018-02-19 20:17:58 +02:00
<execute-list host="{[host-repo1]}">
Improve check command to verify that the backup manifest can be built. Contributed by Cynthia Shang.
2017-12-13 18:16:27 +02:00
<title>Run a full backup</title>
Preliminary documentation for PostgreSQL 11 unprivileged user backup.
2018-06-15 01:05:35 +02:00
<execute user="{[br-user]}">
Improve check command to verify that the backup manifest can be built. Contributed by Cynthia Shang.
2017-12-13 18:16:27 +02:00
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} {[dash]}-type=full backup</exe-cmd>
</execute>
</execute-list>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg2]}">
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
<title>Restore the {[postgres-cluster-demo]} standby cluster</title>
Create common if expressions for testing os-type. These expressions simplify os-type testing. This will be especially true as more OS types are added.
2018-12-05 00:33:56 +02:00
<execute user="postgres" if="{[os-type-is-debian]}">
Add standby restore type. This restore type automatically adds standby_mode=on to recovery.conf. This could be accomplished previously by setting --recovery-option=standby_mode=on but PostgreSQL 12 requires standby mode to be enabled by a special file named standby.signal. The new restore type allows us to maintain a common interface between PostgreSQL versions.
2019-09-26 23:39:45 +02:00
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} {[dash]}-delta --type=standby restore</exe-cmd>
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
</execute>
All clusters in the documentation are initialized with checksums.
2017-12-14 22:44:06 +02:00
Consolidate RHEL/CentOS user guide into a single document. The user guide was split primarily to provide documentation for the stop-auto option in PostgreSQL <= 9.5. Now that 9.5 is EOL there does not seem to be a good reason to generate an extra user guide. The stop-auto function is still documented in the reference. Leave the stop-auto documentation in the user guide in case we want to manually generate documentation for older versions. Also rename centos to rhel for most identifiers since that is the core platform we are building for, similar to how we label 'debian' builds even though we generally use Ubuntu. With CentOS set to become an upstream for RHEL later this year, we'll likely need to pick a new test distribution, perhaps Rocky Linux if that gets off the ground.
2021-05-24 22:03:15 +02:00
<execute user="postgres" if="{[os-type-is-rhel]}">
Add standby restore type. This restore type automatically adds standby_mode=on to recovery.conf. This could be accomplished previously by setting --recovery-option=standby_mode=on but PostgreSQL 12 requires standby mode to be enabled by a special file named standby.signal. The new restore type allows us to maintain a common interface between PostgreSQL versions.
2019-09-26 23:39:45 +02:00
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} --type=standby restore</exe-cmd>
All clusters in the documentation are initialized with checksums.
2017-12-14 22:44:06 +02:00
</execute>
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
</execute-list>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<postgres-config host="{[host-pg2]}" file="{[postgres-config-demo-upgrade]}">
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
<title>Configure <postgres/></title>
<postgres-config-option key="hot_standby">on</postgres-config-option>
</postgres-config>
Rename pg-primary and pg-standby variables to pg1 and pg2. It would be better if the hostnames were also pg1 and pg2 to illustrate that primaries and standbys can change hosts, but at this time the configuration ends up being confusing since pg1, pg2, etc. are also used in the option naming. So, for now leave the names as pg-primary and pg-standby to avoid confusion.
2018-02-22 01:15:40 +02:00
<execute-list host="{[host-pg2]}">
Improve check command to verify that the backup manifest can be built. Contributed by Cynthia Shang.
2017-12-13 18:16:27 +02:00
<title>Start <postgres/> and check the <backrest/> configuration</title>
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
<execute user="root">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<exe-cmd>{[pg-cluster-start-upgrade]}</exe-cmd>
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
</execute>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<execute user="postgres" show="n">
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
<exe-cmd>{[pg-cluster-wait]}</exe-cmd>
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
</execute>
Improve check command to verify that the backup manifest can be built. Contributed by Cynthia Shang.
2017-12-13 18:16:27 +02:00
<execute user="postgres" output="y" filter="n" >
<exe-cmd>{[project-exe]} {[dash]}-stanza={[postgres-cluster-demo]} check</exe-cmd>
</execute>
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
</execute-list>
<p>Backup from standby can be enabled now that the standby is restored.</p>
Document generator improvements. * Build containers from scratch for more accurate testing. * Allow environment load to be skipped. * Allow bash wrapping to be skipped. * Allow forcing a command to run as a user without sudo.
2018-06-10 20:13:56 +02:00
<backrest-config host="{[host-repo1]}" owner="{[br-user]}:{[br-group]}" file="{[backrest-config-demo]}">
Added stanza-upgrade command. The stanza-upgrade command provides a mechanism for upgrading a stanza after upgrading to a new major version of PostgreSQL. Contributed by Cynthia Shang.
2017-04-03 16:42:55 +02:00
<title>Reenable backup from standby</title>
<backrest-config-option section="global" key="backup-standby">y</backrest-config-option>
</backrest-config>
</section>
Implemented issue #148: Website with basic user guide.
2015-10-28 11:10:36 +02:00
</doc>
Reference in New Issue Copy Permalink