1
0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2024-12-14 10:13:05 +02:00
Commit Graph

41 Commits

Author SHA1 Message Date
David Steele
005fc08795 Added the --repo-link option.
Allow internal symlinks to be suppressed when the repository is located on a filesystem that does not support symlinks. This does not affect any pgBackRest functionality, but the convenience link latest will not be created and neither will internal tablespace symlinks, which will affect the ability to bring up clusters in-place manually using filesystem snapshots.
2016-12-03 10:23:26 -05:00
Cynthia Shang
22cd29771f Update control and wal test files to 9.4 with matching system identifiers.
Contributed by Cynthia Shang.
2016-12-01 09:33:14 -05:00
David Steele
a850335015 Simplified the result hash of File->manifest(), Db->tablespaceMapGet(), and Db->databaseMapGet(). 2016-11-30 14:36:39 -05:00
David Steele
09df07efb7 For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process.
The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 14:23:05 -05:00
Cynthia Shang
e951c4f9c2 Abstracted code to determine which database cluster is the master and which are standbys.
Contributed by Cynthia Shang.
2016-11-30 14:15:11 -05:00
David Steele
dd621081b9 Fixed an issue where tablespace paths with the same prefix would cause an invalid link error.
Reported by Nikhilchandra Kulkarni.
2016-11-07 16:37:16 +02:00
David Steele
21ca412702 Fixed an issue where a backup could error if no changes were made to a database between backups and only pg_control changed. 2016-11-05 19:22:30 +02:00
David Steele
2a18674111 Added the log-level-stderr option.
Controls whether console log messages are sent to stderr or stdout. By default this is set to warn which represents a change in behavior from previous versions, even though it may be more intuitive. Setting log-level-stderr=off will preserve the old behavior.

Suggested by Sascha Biberhofer.
2016-10-05 09:09:30 -04:00
David Steele
1491862716 Regression tests can now be run as any properly-configured user, not just vagrant. 2016-10-01 13:39:44 -04:00
David Steele
25f40f9b15 Log improvements:
* pgBackRest version number included in command start INFO log output.
* Process ID logged for local process start/stop INFO log output.
* Fixed missing expect output for help module.
2016-09-17 11:53:28 -04:00
David Steele
f54145c0cc Fixed timeout issues.
* Fixed an issue where local processes were not disconnecting when complete and could later timeout. (Reported by Todd Vernick.)
* Fixed an issue where the protocol layer could timeout while waiting for WAL segments to arrive in the archive. (Reported by Todd Vernick.)
2016-09-14 16:37:07 -05:00
David Steele
e08b56a4e3 Show Process ID in log output instead of filtering it out with the timestamp. 2016-09-14 16:30:57 -05:00
Cynthia Shang
d2add293c7 Archive retention improvements:
* Fixed an issue where retention-archive was not automatically being set when retention-archive-type=diff, resulting in a less aggressive than intended expiration of archive.
* Additional warnings when archive retention settings may not have the intended effect or would allow indefinite retention.
* Closed #235: "Retention policy question" by adding documentation for archive retention.

Contributed by Cynthia Shang.
2016-09-07 08:07:37 -04:00
David Steele
dd8d781217 Safely check eval blocks and convert $@ to $EVAL_ERROR. 2016-09-06 09:44:50 -04:00
David Steele
a701309453 Converted Perl threads to processes. 2016-09-06 09:35:02 -04:00
David Steele
f986e6ae33 File module improvements.
* Remove obsolete thread index variable from File() module.
* Changed temporary file names to consistently use .pgbackrest.tmp.
2016-09-04 09:22:53 -04:00
David Steele
bcdb5cdac8 Fixed a issue where tablespaces were copied from the master during standby backup. 2016-09-04 09:19:44 -04:00
David Steele
2feaaf225e Exclude contents of $PGDATA/pg_replslot directory. 2016-09-04 09:13:13 -04:00
David Steele
5ada189a92 Backup from a standby cluster.
A connection to the primary cluster is still required to start/stop the backup and copy files that are not replicated, but the vast majority of files are copied from the standby in order to reduce load on the master.
2016-08-25 11:25:46 -04:00
David Steele
d0b6f78b20 More flexible configuration for databases
Master and standby can both be configured on the backup server and pgBackRest will automatically determine which is the master. This means no configuration changes for backup are required after failing over from a master to standby when a separate backup server is used.
2016-08-24 12:39:27 -04:00
David Steele
cd6278e5af Revert some backup exclusions until they have been tested more thoroughly. 2016-08-24 12:27:48 -04:00
David Steele
f1412baccf Exclude directories during backup that are cleaned, recreated, or zeroed by PostgreSQL at startup.
These include (depending on the version where they were introduced): pgsql_tmp, pg_dynshmem, pg_notify, pg_replslot, pg_serial, pg_snapshots, pg_stat_tmp, pg_subtrans. The postgresql.auto.conf.tmp file is now excluded in addition to files that were already excluded: backup_label.old, postmaster.opts, postmaster.pid, recovery.conf, recovery.done.
2016-08-16 09:35:16 -04:00
David Steele
1e0f15f425 Improve error message for links that reference links in manifest build. 2016-08-15 17:23:37 -04:00
David Steele
f9fa1270b2 Fixed #236: Recursive user tablespace symlink.
A tablespace link that referenced another link would not produce an error, but instead skip the tablespace entirely.
2016-08-15 17:11:45 -04:00
David Steele
17b79d6279 Database version refactoring.
* Refactor db version constants into a separate module.
* Update synthetic backup tests to PostgreSQL 9.4.
2016-08-11 22:35:24 -04:00
David Steele
bff262ac47 Removed all OP_* function constants that were used only for debugging, not in the protocol, and replaced with __PACKAGE__. 2016-08-11 17:32:28 -04:00
David Steele
34afe5e85b Fixed issue with tablespace link checking.
* Tablespace paths that had $PGDATA as a substring would be identified as a subdirectories of $PGDATA even when they were not.
* Also hardened relative path checking a bit.
2016-08-09 09:05:27 -04:00
David Steele
cc2a8777d5 User/group permissions improvements.
Improved handling of users/groups captured during backup that do not exist on the restore host. Also explicitly handle the case where user/group is not mapped to a name.
2016-06-26 21:01:20 -04:00
David Steele
bd25223fd6 Rename test paths for clarity.
This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved.  Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 10:54:31 -04:00
David Steele
c8f806a293 Major refactor of the test suite.
* Make the code more modular and object-oriented.
* Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 08:12:58 -04:00
David Steele
012405a33b Closed #207: Expire fails with unhandled exception.
* Fixed an issue where the expire command would refuse to run when explicitly called from the command line if the db-host option was set.  This was not an issue when expire was run after a backup, which is the usual case.
* Option handling is now far more strict. Previously it was possible for a command to use an option that was not explicitly assigned to it. This was especially true for the backup-host and db-host options which are used to determine locality.

Reported by Chris Barber.
2016-06-18 09:55:00 -04:00
David Steele
77b01e980f Make the initial capital for hints consistent.
Some hints started with a capital letter and others did not. Make them all lower case for consistency.
2016-05-14 10:36:35 -04:00
David Steele
512d006346 Refactor database version identification for archive and backup commands.
Added database version constants and changed version identification code to use hash tables instead of if-else.  Propagated the db version constants to the rest of the code and in passing fixed some path/filename constants.

Added new regression tests to check that specific files are never copied.
2016-05-14 10:33:12 -04:00
David Steele
0c320e7df7 Allow selective restore of databases from a cluster backup.
This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 09:21:39 -04:00
Cynthia Shang
60b901948a Enhanced text output of info command to include timestamps, sizes, and the reference list for all backups. 2016-05-06 10:25:46 -04:00
David Steele
7b64843221 Added trim() to process id read from lock file.
The prior way worked fine but added extraneous linefeeds to the log output.
2016-05-06 09:37:01 -04:00
David Steele
9457e15347 New manifest format.
* All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location.

* Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function.

* Added detail log level which will output more information than info without being as verbose as debug.
2016-04-14 22:50:02 -04:00
David Steele
18fd25233b New simpler configuration and consistent project/exe/path naming.
* The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required.

* Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case.

* The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally.

* The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest.

* Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory.

* Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory.

* Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 09:30:54 -04:00
David Steele
885797e4b5 Migrated many functions from File.pm to FileCommon.pm.
This makes make the migrated file functions available to parts of the code that don't have access to a File object.  They still exist as wrappers in the File object to support remote calls.
2016-04-13 19:09:35 -04:00
David Steele
723abadb75 Change spacing in expect logs to accommodate DETAIL level. 2016-04-13 18:55:34 -04:00
David Steele
8adbcccd02 Closed #184: Change test/log directory to test/expect.
This should make it clearer that these are expect logs for regression testing.
2016-04-06 10:00:11 -04:00