1
0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2025-07-13 01:00:23 +02:00
Commit Graph

41 Commits

Author SHA1 Message Date
b620e6b282 Improvements and additional tests for stanza-upgrade.
Contributed by Cynthia Shang.
2017-04-06 19:36:37 -04:00
c7b61a2002 Refactor backupLabel() and add unit tests. 2017-04-05 10:16:16 -04:00
b2f43b5601 Allow functions to accept optional parameters as a hash.
Refactor File->list() and fileList() to accept optional parameters.
2017-04-04 21:17:19 -04:00
6b99d770b5 Reduced the likelihood of torn pages causing a false positive in page checksums by filtering on start backup LSN. 2017-01-30 13:59:00 -05:00
43ada9fb33 Improved, multi-process implementation of asynchronous archiving. 2017-01-27 11:02:27 -05:00
7d1347cdec Split the archive-get and archive-push commands into separate modules. 2017-01-10 19:59:32 -05:00
b58045f575 Moved the Archive modules to the Archive directory. 2017-01-10 19:51:20 -05:00
974a02dbe4 Improved stanza-create command so that it can repair broken repositories in most cases and is robust enough to be made mandatory.
Contributed by Cynthia Shang.
2016-12-20 16:52:20 -05:00
aecc0f737e Fixed a few directory syncs that were missed for the --repo-sync option. 2016-12-19 14:40:22 -05:00
36a5349b1c Added the --checksum-page option.
This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-12 18:54:07 -05:00
f818901c0e Added the --repo-sync option to allow directory syncs in the repository to be disabled.
This is for file systems that do not support them, e.g. NTFS
2016-12-10 09:06:45 -05:00
2e01e6b9e1 Improved consistency and flexibility of the protocol layer by using JSON for all messages. 2016-12-03 17:34:51 -05:00
b7f1738200 Code cleanup in preparation for improved stanza-create command.
Contributed by Cynthia Shang.
2016-12-03 10:28:08 -05:00
005fc08795 Added the --repo-link option.
Allow internal symlinks to be suppressed when the repository is located on a filesystem that does not support symlinks. This does not affect any pgBackRest functionality, but the convenience link latest will not be created and neither will internal tablespace symlinks, which will affect the ability to bring up clusters in-place manually using filesystem snapshots.
2016-12-03 10:23:26 -05:00
6ae4d7de81 Fixed a bug where internal symlinks were not being created for tablespaces in the repository.
This issue was only apparent when trying to bring up clusters in-place with filesystem snapshots and did not affect normal backup and restore.
2016-12-03 09:56:05 -05:00
a850335015 Simplified the result hash of File->manifest(), Db->tablespaceMapGet(), and Db->databaseMapGet(). 2016-11-30 14:36:39 -05:00
09df07efb7 For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process.
The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 14:23:05 -05:00
e951c4f9c2 Abstracted code to determine which database cluster is the master and which are standbys.
Contributed by Cynthia Shang.
2016-11-30 14:15:11 -05:00
f43e5bc52d Removed extraneous use lib directives from Perl modules.
Suggested by Devrim Gündüz.
2016-11-04 13:56:26 +02:00
07898d33e3 Fixes for archive checks added in 5d033d0:
* Don't execute pg_is_in_recovery() on PostreSQL < 9.0.
* Don't check archive_mode or archive_command on the standby.
2016-10-03 07:28:44 -04:00
f54145c0cc Fixed timeout issues.
* Fixed an issue where local processes were not disconnecting when complete and could later timeout. (Reported by Todd Vernick.)
* Fixed an issue where the protocol layer could timeout while waiting for WAL segments to arrive in the archive. (Reported by Todd Vernick.)
2016-09-14 16:37:07 -05:00
0d5a1eaf9e The archive-start and archive-stop settings are now filled in backup.manifest. 2016-09-07 07:20:07 -04:00
dd8d781217 Safely check eval blocks and convert $@ to $EVAL_ERROR. 2016-09-06 09:44:50 -04:00
a701309453 Converted Perl threads to processes. 2016-09-06 09:35:02 -04:00
bcdb5cdac8 Fixed a issue where tablespaces were copied from the master during standby backup. 2016-09-04 09:19:44 -04:00
5ada189a92 Backup from a standby cluster.
A connection to the primary cluster is still required to start/stop the backup and copy files that are not replicated, but the vast majority of files are copied from the standby in order to reduce load on the master.
2016-08-25 11:25:46 -04:00
d0b6f78b20 More flexible configuration for databases
Master and standby can both be configured on the backup server and pgBackRest will automatically determine which is the master. This means no configuration changes for backup are required after failing over from a master to standby when a separate backup server is used.
2016-08-24 12:39:27 -04:00
f1412baccf Exclude directories during backup that are cleaned, recreated, or zeroed by PostgreSQL at startup.
These include (depending on the version where they were introduced): pgsql_tmp, pg_dynshmem, pg_notify, pg_replslot, pg_serial, pg_snapshots, pg_stat_tmp, pg_subtrans. The postgresql.auto.conf.tmp file is now excluded in addition to files that were already excluded: backup_label.old, postmaster.opts, postmaster.pid, recovery.conf, recovery.done.
2016-08-16 09:35:16 -04:00
17b79d6279 Database version refactoring.
* Refactor db version constants into a separate module.
* Update synthetic backup tests to PostgreSQL 9.4.
2016-08-11 22:35:24 -04:00
bff262ac47 Removed all OP_* function constants that were used only for debugging, not in the protocol, and replaced with __PACKAGE__. 2016-08-11 17:32:28 -04:00
f31ec20fcb Reduce calls to protocolGet() in backup/restore. 2016-07-26 16:21:14 -04:00
23a3911830 Stop using pg_xlogfile_name().
The pg_xlogfile_name() function is no longer used to construct WAL filenames from LSNs. While this function is convenient it is not available on a standby. Instead, the archive is searched for the LSN in order to find the timeline. If due to some misadventure the LSN appears on multiple timelines then an error will be thrown, whereas before this condition would have passed unnoticed.
2016-06-24 08:06:20 -04:00
41b27c0456 Protocol timeout option and keep-alive fixes.
* Fixed an issue where keep-alives could be starved out by lots of small files during multi-threaded operation and were completely absent during single-threaded operation when resuming from a previous incomplete backup.

Reported by Janice Parkinson.

* Added the protocol-timeout option. Previously protocol-timeout was set as db-timeout + 30 seconds.
* Failure to shutdown remotes at the end of the backup no longer throws an exception. A warning is still generated that recommends a higher protocol-timeout.
2016-06-22 18:01:18 -04:00
7e45ed8366 Added check command.
The check command validates that pgBackRest is configured correctly for archiving and backups.

Contributed by Cynthia Shang.
2016-06-12 09:13:46 -04:00
0451d3afdd Support for non-exclusive backups in PostgreSQL 9.6. 2016-05-16 17:59:26 -04:00
9b5a27f657 Add Manifest->addFile().
Some files need to be added to the manifest after the initial build.  This is currently done in only one place but usage will expand in the future so the functionality has been encapsulated in addFile().
2016-05-14 10:39:56 -04:00
77b01e980f Make the initial capital for hints consistent.
Some hints started with a capital letter and others did not. Make them all lower case for consistency.
2016-05-14 10:36:35 -04:00
512d006346 Refactor database version identification for archive and backup commands.
Added database version constants and changed version identification code to use hash tables instead of if-else.  Propagated the db version constants to the rest of the code and in passing fixed some path/filename constants.

Added new regression tests to check that specific files are never copied.
2016-05-14 10:33:12 -04:00
0c320e7df7 Allow selective restore of databases from a cluster backup.
This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 09:21:39 -04:00
9457e15347 New manifest format.
* All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location.

* Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function.

* Added detail log level which will output more information than info without being as verbose as debug.
2016-04-14 22:50:02 -04:00
18fd25233b New simpler configuration and consistent project/exe/path naming.
* The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required.

* Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case.

* The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally.

* The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest.

* Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory.

* Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory.

* Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 09:30:54 -04:00