db/pgbackrest
1
0
Fork 0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2024-12-14 10:13:05 +02:00
Code Issues Releases Activity
d245f8eb42
pgbackrest/test/expect/mock-all-001.log

6099 lines
554 KiB
Plaintext
Raw Normal View History

Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
run 001 - rmt 0, s3 0, enc 0
============================
Log testing can now be enabled for certain deterministic tests. This works by comparing the generated logs against a previous copy. Currently only enabled for the backup/synthetic tests.
2015-05-05 19:08:48 +02:00
Move help/version integration tests to mock/all. Help and version are covered by unit tests, so we really just to need to make sure there is output when called from the command line.
2018-04-08 18:16:32 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] version
------------------------------------------------------------------------------------------------------------------------------------
[BACKREST-NAME-VERSION]
> [CONTAINER-EXEC] db-master [BACKREST-BIN] help version
------------------------------------------------------------------------------------------------------------------------------------
[BACKREST-NAME-VERSION] - 'version' command help
Get version.
Displays installed pgBackRest version.
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
info all stanzas - no stanzas exist (db-master host)
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --log-level-console=warn info
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Simplify info command text message when no stanza are present. Replace the repository path with just "the repository". The path is not important in this context and it is clearer to state where the stanzas are missing from.
2019-01-16 19:23:10 +02:00
No stanzas exist in the repository.
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
info all stanzas - no stanzas exist (db-master host)
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --log-level-console=warn --output=json info
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
------------------------------------------------------------------------------------------------------------------------------------
[]
Improved stanza-create command so that it can repair broken repositories in most cases and is robust enough to be made mandatory. Contributed by Cynthia Shang.
2016-12-20 23:52:20 +02:00
stanza-create db - create required data for stanza (db-master host)
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --stanza=db --log-level-console=detail --no-online stanza-create
------------------------------------------------------------------------------------------------------------------------------------
Add log-subprocess option to allow file logging for local and remote subprocesses.
2018-08-23 02:05:49 +02:00
P00 INFO: stanza-create command begin [BACKREST-VERSION]: --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --db-timeout=45 --lock-path=[TEST_PATH]/db-master/lock --log-level-console=detail --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --no-online --pg1-path=[TEST_PATH]/db-master/db/base --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --stanza=db
Improved stanza-create command so that it can repair broken repositories in most cases and is robust enough to be made mandatory. Contributed by Cynthia Shang.
2016-12-20 23:52:20 +02:00
P00 INFO: stanza-create command end: completed successfully
+ supplemental file: [TEST_PATH]/db-master/repo/backup/db/backup.info
---------------------------------------------------------------------
[backrest]
backrest-checksum="[CHECKSUM]"
backrest-format=5
backrest-version="[VERSION-1]"
[db]
db-catalog-version=201409291
db-control-version=942
db-id=1
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
db-system-id=1000000000000000094
Improved stanza-create command so that it can repair broken repositories in most cases and is robust enough to be made mandatory. Contributed by Cynthia Shang.
2016-12-20 23:52:20 +02:00
db-version="9.4"
[db:history]
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
1={"db-catalog-version":201409291,"db-control-version":942,"db-system-id":1000000000000000094,"db-version":"9.4"}
Improved stanza-create command so that it can repair broken repositories in most cases and is robust enough to be made mandatory. Contributed by Cynthia Shang.
2016-12-20 23:52:20 +02:00
+ supplemental file: [TEST_PATH]/db-master/repo/archive/db/archive.info
-----------------------------------------------------------------------
[backrest]
backrest-checksum="[CHECKSUM]"
backrest-format=5
backrest-version="[VERSION-1]"
[db]
db-id=1
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
db-system-id=1000000000000000094
Improved stanza-create command so that it can repair broken repositories in most cases and is robust enough to be made mandatory. Contributed by Cynthia Shang.
2016-12-20 23:52:20 +02:00
db-version="9.4"
[db:history]
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
1={"db-id":1000000000000000094,"db-version":"9.4"}
Improved stanza-create command so that it can repair broken repositories in most cases and is robust enough to be made mandatory. Contributed by Cynthia Shang.
2016-12-20 23:52:20 +02:00
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
full backup - error on identical link destinations (db-master host)
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --no-online --log-level-console=detail --type=full --stanza=db backup
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
P00 INFO: backup command begin [BACKREST-VERSION]: --no-compress --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --db-timeout=45 --exclude=postgresql.auto.conf --exclude=pg_log/ --exclude=pg_log2 --lock-path=[TEST_PATH]/db-master/lock --log-level-console=detail --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --no-online --pg1-path=[TEST_PATH]/db-master/db/base --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --stanza=db --start-fast --type=full
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
P00 WARN: option repo1-retention-full is not set, the repository may run out of space
HINT: to retain full backups indefinitely (without warning), set option 'repo1-retention-full' to the maximum.
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
P00 INFO: exclude pg_log/logfile from backup using 'pg_log/' exclusion
P00 INFO: exclude pg_log2 from backup using 'pg_log2' exclusion
P00 INFO: exclude pg_log2/logfile from backup using 'pg_log2' exclusion
P00 INFO: exclude postgresql.auto.conf from backup using 'postgresql.auto.conf' exclusion
Fix failure in manifest build when two or more files in PGDATA are linked to the same directory. Reported by Vitaliy Kukharik.
2018-05-02 18:19:54 +02:00
P00 ERROR: [070]: link [TEST_PATH]/db-master/db/base/pg_hba.conf (../pg_config) references a subdirectory of or the same directory as link [TEST_PATH]/db-master/db/base/pg_config_bad (../../db/pg_config)
Renumber return codes between 25 and 125 to avoid PostgreSQL interpreting some as fatal signal exceptions. Suggested by Yogesh Sharma.
2017-04-12 18:41:34 +02:00
P00 INFO: backup command end: aborted with exception [070]
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
full backup - error on link to a link (db-master host)
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --no-online --log-level-console=detail --type=full --stanza=db backup
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
P00 INFO: backup command begin [BACKREST-VERSION]: --no-compress --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --db-timeout=45 --exclude=postgresql.auto.conf --exclude=pg_log/ --exclude=pg_log2 --lock-path=[TEST_PATH]/db-master/lock --log-level-console=detail --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --no-online --pg1-path=[TEST_PATH]/db-master/db/base --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --stanza=db --start-fast --type=full
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
P00 WARN: option repo1-retention-full is not set, the repository may run out of space
HINT: to retain full backups indefinitely (without warning), set option 'repo1-retention-full' to the maximum.
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
P00 INFO: exclude pg_log/logfile from backup using 'pg_log/' exclusion
P00 INFO: exclude pg_log2 from backup using 'pg_log2' exclusion
P00 INFO: exclude pg_log2/logfile from backup using 'pg_log2' exclusion
P00 INFO: exclude postgresql.auto.conf from backup using 'postgresql.auto.conf' exclusion
Renumber return codes between 25 and 125 to avoid PostgreSQL interpreting some as fatal signal exceptions. Suggested by Yogesh Sharma.
2017-04-12 18:41:34 +02:00
P00 ERROR: [070]: link '[TEST_PATH]/db-master/db/base/postgresql.conf.bad' -> '../pg_config/postgresql.conf.link' cannot reference another link
P00 INFO: backup command end: aborted with exception [070]
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
full backup - create pg_stat link, pg_clog dir (db-master host)
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --no-online --manifest-save-threshold=3 --buffer-size=16384 --checksum-page --process-max=1 --repo1-type=cifs --type=full --stanza=db backup
Log testing can now be enabled for certain deterministic tests. This works by comparing the generated logs against a previous copy. Currently only enabled for the backup/synthetic tests.
2015-05-05 19:08:48 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
P00 INFO: backup command begin [BACKREST-VERSION]: --buffer-size=16384 --checksum-page --no-compress --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --db-timeout=45 --exclude=postgresql.auto.conf --exclude=pg_log/ --exclude=pg_log2 --lock-path=[TEST_PATH]/db-master/lock --log-level-console=debug --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --manifest-save-threshold=3 --no-online --pg1-path=[TEST_PATH]/db-master/db/base --process-max=1 --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --repo1-type=cifs --stanza=db --start-fast --type=full
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: common/lock::lockAcquire: (lockPath: {"[TEST_PATH]/db-master/lock"}, stanza: {"db"}, lockType: 1, lockTimeout: 0, failOnNoLock: true)
P00 DEBUG: common/lock::lockAcquire: => true
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
P00 WARN: option repo1-retention-full is not set, the repository may run out of space
HINT: to retain full backups indefinitely (without warning), set option 'repo1-retention-full' to the maximum.
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: config/load::cfgLoad: => void
P00 DEBUG: perl/exec::perlExec: (void)
Allow functions with sensitive options to be logged at debug level with redactions. Previously, functions with sensitive options had to be logged at trace level to avoid exposing them. Trace level logging may still expose secrets so use with caution.
2017-10-24 18:35:36 +02:00
P00 DEBUG: Storage::Posix::Driver->new(): bFileSync = <true>, bPathSync = <true>
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->new(): bAllowTemp = <true>, hRule = [hash], lBufferMax = 16384, oDriver = [object], strCipherPassUser = [undef], strCipherType = [undef], strDefaultFileMode = <0640>, strDefaultPathMode = <0750>, strPathBase = [TEST_PATH]/db-master/repo, strTempExtension = pgbackrest.tmp
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp =
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
Config parsing no longer implemented in Perl. Options are passed to Perl as JSON from the C process.
2018-01-29 04:37:09 +02:00
P00 DEBUG: Storage::Posix::Driver->new(): bFileSync = <true>, bPathSync = <true>
P00 DEBUG: Storage::Local->new(): bAllowTemp = <true>, hRule = [undef], lBufferMax = 16384, oDriver = [object], strCipherPassUser = [undef], strCipherType = [undef], strDefaultFileMode = <0640>, strDefaultPathMode = <0750>, strPathBase = /, strTempExtension = pgbackrest.tmp
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = true, bIgnoreExists = true, strMode = 0770, strPathExp = [TEST_PATH]/db-master/log
Move async forking and more error handling to C. The Perl process was exiting directly when called but that interfered with proper locking for the forked async process. Now Perl returns results to the C process which handles all errors, including signals.
2018-04-13 02:42:26 +02:00
P00 DEBUG: Common::Lock::lockStopTest(): bStanzaStopRequired = <false>
P00 DEBUG: Common::Lock::lockStopTest=>: bStopExists = false
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Backup::Info->new(): bIgnoreMissing = <false>, bLoad = <true>, bRequired = <true>, bValidate = <true>, oStorage = <[object]>, strBackupClusterPath = [TEST_PATH]/db-master/repo/backup/db, strCipherPassSub = [undef]
P00 DEBUG: Storage::Local->encrypted(): bIgnoreMissing = true, strFileName = [TEST_PATH]/db-master/repo/backup/db/backup.info
P00 DEBUG: Storage::Local->encrypted=>: bEncrypted = false
P00 DEBUG: Storage::Local->encryptionValid(): bEncrypted = false
P00 DEBUG: Storage::Local->encryptionValid=>: bValid = true
P00 DEBUG: Storage::Local->openRead(): bIgnoreMissing = true, rhyFilter = [undef], strCipherPass = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/backup.info
P00 DEBUG: Storage::Base->get(): strCipherPass = [undef], xFile = [object]
Move backup modules into Backup directory.
2017-05-15 22:01:00 +02:00
P00 DEBUG: Backup::Info->reconstruct(): bRequired = <true>, bSave = <true>, iCatalogVersion = [undef], iControlVersion = [undef], strDbVersion = [undef], ullDbSysId = [undef]
P00 DEBUG: Backup::Common::backupRegExpGet(): bAnchor = <true>, bDifferential = true, bFull = true, bIncremental = true
P00 DEBUG: Backup::Common::backupRegExpGet=>: strRegExp = ^[0-9]{8}\-[0-9]{6}F(\_[0-9]{8}\-[0-9]{6}(D|I)){0,1}$
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->list(): bIgnoreMissing = <false>, strExpression = ^[0-9]{8}\-[0-9]{6}F(\_[0-9]{8}\-[0-9]{6}(D|I)){0,1}$, strPathExp = [TEST_PATH]/db-master/repo/backup/db, strSortOrder = <forward>
P00 DEBUG: Storage::Local->list=>: stryFileList = ()
Up to seven standbys can be configured for backup from standby.` Contributed by Cynthia Shang.
2017-09-01 01:15:44 +02:00
P00 DEBUG: Db::dbObjectGet(): bMasterOnly = <false>
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 DEBUG: Db->new(): iRemoteIdx = 1
Abstracted code to determine which database cluster is the master and which are standbys. Contributed by Cynthia Shang.
2016-11-30 21:15:11 +02:00
P00 DEBUG: Db::dbObjectGet=>: iDbMasterIdx = 1, iDbStandbyIdx = [undef], oDbMaster = [object], oDbStandby = [undef]
Allow functions with sensitive options to be logged at debug level with redactions. Previously, functions with sensitive options had to be logged at trace level to avoid exposing them. Trace level logging may still expose secrets so use with caution.
2017-10-24 18:35:36 +02:00
P00 DEBUG: Storage::Posix::Driver->new(): bFileSync = <true>, bPathSync = <true>
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->new(): bAllowTemp = <true>, hRule = [undef], lBufferMax = 16384, oDriver = [object], strCipherPassUser = [undef], strCipherType = [undef], strDefaultFileMode = <0640>, strDefaultPathMode = <0750>, strPathBase = [TEST_PATH]/db-master/db/base, strTempExtension = pgbackrest.tmp
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 DEBUG: Db->info(): strDbPath = <[TEST_PATH]/db-master/db/base>
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
P00 DEBUG: Db->info=>: iDbCatalogVersion = 201409291, iDbControlVersion = 942, strDbVersion = 9.4, ullDbSysId = 1000000000000000094
P00 DEBUG: Backup::Info->check(): bRequired = <true>, iCatalogVersion = 201409291, iControlVersion = 942, strDbVersion = 9.4, ullDbSysId = 1000000000000000094
Move backup modules into Backup directory.
2017-05-15 22:01:00 +02:00
P00 DEBUG: Backup::Info->check=>: iDbHistoryId = 1
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Backup::Common::backupRegExpGet(): bAnchor = <true>, bDifferential = true, bFull = true, bIncremental = true
P00 DEBUG: Backup::Common::backupRegExpGet=>: strRegExp = ^[0-9]{8}\-[0-9]{6}F(\_[0-9]{8}\-[0-9]{6}(D|I)){0,1}$
P00 DEBUG: Storage::Local->list(): bIgnoreMissing = <false>, strExpression = ^[0-9]{8}\-[0-9]{6}F(\_[0-9]{8}\-[0-9]{6}(D|I)){0,1}$, strPathExp = <REPO:BACKUP>, strSortOrder = reverse
P00 DEBUG: Storage::Local->list=>: stryFileList = ()
P00 DEBUG: Storage::Local->list(): bIgnoreMissing = <false>, strExpression = ^[BACKUP-FULL-1], strPathExp = <REPO:BACKUP>, strSortOrder = <forward>
P00 DEBUG: Storage::Local->list=>: stryFileList = ()
P00 DEBUG: Storage::Local->list(): bIgnoreMissing = true, strExpression = ^[BACKUP-FULL-1], strPathExp = <REPO:BACKUP>/backup.history/[YEAR-1], strSortOrder = <forward>
P00 DEBUG: Storage::Local->list=>: stryFileList = ()
Require PostgreSQL catalog version when instantiating a Manifest object (and not loading it from disk). Contributed by Cynthia Shang.
2018-07-16 23:25:15 +02:00
P00 DEBUG: Manifest->new(): bLoad = false, iDbCatalogVersion = 201409291, oStorage = <[object]>, strCipherPass = [undef], strCipherPassSub = [undef], strDbVersion = 9.4, strFileName = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-1]/backup.manifest
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/db/base/postmaster.pid
P00 DEBUG: Storage::Local->exists=>: bExists = false
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: Manifest->build(): bDelta = false, bOnline = false, bTablespace = [undef], hDatabaseMap = [undef], hTablespaceMap = [undef], iLevel = <0>, oLastManifest = [undef], oStorageDbMaster = [object], rhExclude = [hash], strFilter = [undef], strLevel = [undef], strParentPath = [undef], strPath = [TEST_PATH]/db-master/db/base, strTimelineCurrent = [undef], strTimelineLast = [undef]
Ignore all files in a linked tablespace directory except the subdirectory for the current version of PostgreSQL. Previously an error would be generated if other files were present and not owned by the PostgreSQL user. This hasn't been a big deal in practice but it could cause issues. Also add tests to make sure the same logic applies with links to files, i.e. all other files in the directory should be ignored. This was actually working correctly, but there were no tests for it before.
2018-08-31 22:06:40 +02:00
P00 DEBUG: Storage::Local->manifest(): strFilter = [undef], strPathExp = [TEST_PATH]/db-master/db/base/pg_tblspc
P00 DEBUG: Storage::Local->manifest(): strFilter = [undef], strPathExp = [TEST_PATH]/db-master/db/base
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: Manifest->build(): bDelta = false, bOnline = false, bTablespace = false, hDatabaseMap = [undef], hTablespaceMap = [hash], iLevel = 1, oLastManifest = [undef], oStorageDbMaster = [object], rhExclude = [hash], strFilter = [undef], strLevel = pg_data/pg_hba.conf, strParentPath = [TEST_PATH]/db-master/db/base, strPath = ../pg_config/pg_hba.conf, strTimelineCurrent = [undef], strTimelineLast = [undef]
Ignore all files in a linked tablespace directory except the subdirectory for the current version of PostgreSQL. Previously an error would be generated if other files were present and not owned by the PostgreSQL user. This hasn't been a big deal in practice but it could cause issues. Also add tests to make sure the same logic applies with links to files, i.e. all other files in the directory should be ignored. This was actually working correctly, but there were no tests for it before.
2018-08-31 22:06:40 +02:00
P00 DEBUG: Storage::Local->manifest(): strFilter = [undef], strPathExp = [TEST_PATH]/db-master/db/pg_config/pg_hba.conf
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
P00 INFO: exclude pg_log/logfile from backup using 'pg_log/' exclusion
P00 INFO: exclude pg_log2 from backup using 'pg_log2' exclusion
P00 INFO: exclude pg_log2/logfile from backup using 'pg_log2' exclusion
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: Manifest->build(): bDelta = false, bOnline = false, bTablespace = false, hDatabaseMap = [undef], hTablespaceMap = [hash], iLevel = 1, oLastManifest = [undef], oStorageDbMaster = [object], rhExclude = [hash], strFilter = [undef], strLevel = pg_data/pg_stat, strParentPath = [TEST_PATH]/db-master/db/base, strPath = ../pg_stat, strTimelineCurrent = [undef], strTimelineLast = [undef]
Ignore all files in a linked tablespace directory except the subdirectory for the current version of PostgreSQL. Previously an error would be generated if other files were present and not owned by the PostgreSQL user. This hasn't been a big deal in practice but it could cause issues. Also add tests to make sure the same logic applies with links to files, i.e. all other files in the directory should be ignored. This was actually working correctly, but there were no tests for it before.
2018-08-31 22:06:40 +02:00
P00 DEBUG: Storage::Local->manifest(): strFilter = [undef], strPathExp = [TEST_PATH]/db-master/db/pg_stat
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
P00 INFO: exclude postgresql.auto.conf from backup using 'postgresql.auto.conf' exclusion
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: Manifest->build(): bDelta = false, bOnline = false, bTablespace = false, hDatabaseMap = [undef], hTablespaceMap = [hash], iLevel = 1, oLastManifest = [undef], oStorageDbMaster = [object], rhExclude = [hash], strFilter = [undef], strLevel = pg_data/postgresql.conf, strParentPath = [TEST_PATH]/db-master/db/base, strPath = ../pg_config/postgresql.conf, strTimelineCurrent = [undef], strTimelineLast = [undef]
Ignore all files in a linked tablespace directory except the subdirectory for the current version of PostgreSQL. Previously an error would be generated if other files were present and not owned by the PostgreSQL user. This hasn't been a big deal in practice but it could cause issues. Also add tests to make sure the same logic applies with links to files, i.e. all other files in the directory should be ignored. This was actually working correctly, but there were no tests for it before.
2018-08-31 22:06:40 +02:00
P00 DEBUG: Storage::Local->manifest(): strFilter = [undef], strPathExp = [TEST_PATH]/db-master/db/pg_config/postgresql.conf
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: Manifest->checkDeltaFile(): lTimeBegin = [MODIFICATION-TIME-1], oPriorManifest = [undef], stryFileList = (pg_data/PG_VERSION, pg_data/base/1/12000, pg_data/base/1/PG_VERSION, pg_data/base/16384/17000, pg_data/base/16384/PG_VERSION, pg_data/base/32768/33000, pg_data/base/32768/33000.32767, pg_data/base/32768/33001, pg_data/base/32768/44000_init, pg_data/base/32768/PG_VERSION, pg_data/global/pg_control, pg_data/pg_hba.conf, pg_data/pg_stat/global.stat, pg_data/postgresql.conf, pg_data/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?%)
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Backup::Backup->process: create backup path [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-1]
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = <false>, strMode = <0750>, strPathExp = <REPO:BACKUP>/[BACKUP-FULL-1]
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-1]/backup.manifest
P00 DEBUG: Storage::Local->exists=>: bExists = false
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->openWrite(): bAtomic = <false>, bPathCreate = <false>, lTimestamp = [undef], rhyFilter = [undef], strCipherPass = [undef], strGroup = [undef], strMode = <0640>, strUser = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-1]/backup.manifest.copy
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Backup::Backup->processManifest(): bCompress = false, bHardLink = false, oBackupManifest = [object], strBackupLabel = [BACKUP-FULL-1], strDbCopyPath = [TEST_PATH]/db-master/db/base, strDbMasterPath = [TEST_PATH]/db-master/db/base, strDbVersion = 9.4, strLsnStart = [undef], strType = full
P00 DEBUG: Protocol::Local::Process->new(): bConfessError = <true>, iSelectTimeout = <30>, strBackRestBin = <[BACKREST-BIN]>, strHostType = db
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->hostAdd(): iHostConfigIdx = 1, iProcessMax = 1
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = true, strMode = <0750>, strPathExp = <REPO:BACKUP>/[BACKUP-FULL-1]/pg_data
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = true, strMode = <0750>, strPathExp = <REPO:BACKUP>/[BACKUP-FULL-1]/pg_data/base
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = true, strMode = <0750>, strPathExp = <REPO:BACKUP>/[BACKUP-FULL-1]/pg_data/base/1
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = true, strMode = <0750>, strPathExp = <REPO:BACKUP>/[BACKUP-FULL-1]/pg_data/base/16384
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = true, strMode = <0750>, strPathExp = <REPO:BACKUP>/[BACKUP-FULL-1]/pg_data/base/32768
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = true, strMode = <0750>, strPathExp = <REPO:BACKUP>/[BACKUP-FULL-1]/pg_data/global
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = true, strMode = <0750>, strPathExp = <REPO:BACKUP>/[BACKUP-FULL-1]/pg_data/pg_clog
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = true, strMode = <0750>, strPathExp = <REPO:BACKUP>/[BACKUP-FULL-1]/pg_data/pg_dynshmem
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = true, strMode = <0750>, strPathExp = <REPO:BACKUP>/[BACKUP-FULL-1]/pg_data/pg_log
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = true, strMode = <0750>, strPathExp = <REPO:BACKUP>/[BACKUP-FULL-1]/pg_data/pg_notify
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = true, strMode = <0750>, strPathExp = <REPO:BACKUP>/[BACKUP-FULL-1]/pg_data/pg_replslot
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = true, strMode = <0750>, strPathExp = <REPO:BACKUP>/[BACKUP-FULL-1]/pg_data/pg_serial
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = true, strMode = <0750>, strPathExp = <REPO:BACKUP>/[BACKUP-FULL-1]/pg_data/pg_snapshots
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = true, strMode = <0750>, strPathExp = <REPO:BACKUP>/[BACKUP-FULL-1]/pg_data/pg_stat
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = true, strMode = <0750>, strPathExp = <REPO:BACKUP>/[BACKUP-FULL-1]/pg_data/pg_stat_tmp
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = true, strMode = <0750>, strPathExp = <REPO:BACKUP>/[BACKUP-FULL-1]/pg_data/pg_subtrans
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = true, strMode = <0750>, strPathExp = <REPO:BACKUP>/[BACKUP-FULL-1]/pg_data/pg_tblspc
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/base/32768/33001, pg_data/base/32768/33001, 65536, [undef], 1, [BACKUP-FULL-1], 0, 3, [MODIFICATION-TIME-2], 1, {iWalId => 65535, iWalOffset => 65535}, 0, 0), rParamSecure = [undef], strKey = pg_data/base/32768/33001, strOp = backupFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/base/32768/44000_init, pg_data/base/32768/44000_init, 32768, [undef], 1, [BACKUP-FULL-1], 0, 3, [MODIFICATION-TIME-2], 1, {iWalId => 65535, iWalOffset => 65535}, 0, 0), rParamSecure = [undef], strKey = pg_data/base/32768/44000_init, strOp = backupFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/base/32768/33000.32767, pg_data/base/32768/33000.32767, 32768, [undef], 1, [BACKUP-FULL-1], 0, 3, [MODIFICATION-TIME-2], 1, {iWalId => 65535, iWalOffset => 65535}, 0, 0), rParamSecure = [undef], strKey = pg_data/base/32768/33000.32767, strOp = backupFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/base/32768/33000, pg_data/base/32768/33000, 32768, [undef], 1, [BACKUP-FULL-1], 0, 3, [MODIFICATION-TIME-2], 1, {iWalId => 65535, iWalOffset => 65535}, 0, 0), rParamSecure = [undef], strKey = pg_data/base/32768/33000, strOp = backupFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/base/16384/17000, pg_data/base/16384/17000, 16384, [undef], 1, [BACKUP-FULL-1], 0, 3, [MODIFICATION-TIME-2], 1, {iWalId => 65535, iWalOffset => 65535}, 0, 0), rParamSecure = [undef], strKey = pg_data/base/16384/17000, strOp = backupFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/global/pg_control, pg_data/global/pg_control, 8192, [undef], 0, [BACKUP-FULL-1], 0, 3, [MODIFICATION-TIME-3], 0, [undef], 0, 0), rParamSecure = [undef], strKey = pg_data/global/pg_control, strOp = backupFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/base/1/12000, pg_data/base/1/12000, 8192, [undef], 1, [BACKUP-FULL-1], 0, 3, [MODIFICATION-TIME-2], 1, {iWalId => 65535, iWalOffset => 65535}, 0, 0), rParamSecure = [undef], strKey = pg_data/base/1/12000, strOp = backupFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/postgresql.conf, pg_data/postgresql.conf, 21, [undef], 0, [BACKUP-FULL-1], 0, 3, [MODIFICATION-TIME-3], 1, [undef], 0, 0), rParamSecure = [undef], strKey = pg_data/postgresql.conf, strOp = backupFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/pg_hba.conf, pg_data/pg_hba.conf, 9, [undef], 0, [BACKUP-FULL-1], 0, 3, [MODIFICATION-TIME-3], 1, [undef], 0, 0), rParamSecure = [undef], strKey = pg_data/pg_hba.conf, strOp = backupFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/pg_stat/global.stat, pg_data/pg_stat/global.stat, 5, [undef], 0, [BACKUP-FULL-1], 0, 3, [MODIFICATION-TIME-3], 1, [undef], 0, 0), rParamSecure = [undef], strKey = pg_data/pg_stat/global.stat, strOp = backupFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/base/32768/PG_VERSION, pg_data/base/32768/PG_VERSION, 3, [undef], 0, [BACKUP-FULL-1], 0, 3, [MODIFICATION-TIME-2], 1, [undef], 0, 0), rParamSecure = [undef], strKey = pg_data/base/32768/PG_VERSION, strOp = backupFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/base/16384/PG_VERSION, pg_data/base/16384/PG_VERSION, 3, [undef], 0, [BACKUP-FULL-1], 0, 3, [MODIFICATION-TIME-2], 1, [undef], 0, 0), rParamSecure = [undef], strKey = pg_data/base/16384/PG_VERSION, strOp = backupFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/base/1/PG_VERSION, pg_data/base/1/PG_VERSION, 3, [undef], 0, [BACKUP-FULL-1], 0, 3, [MODIFICATION-TIME-2], 1, [undef], 0, 0), rParamSecure = [undef], strKey = pg_data/base/1/PG_VERSION, strOp = backupFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/PG_VERSION, pg_data/PG_VERSION, 3, [undef], 0, [BACKUP-FULL-1], 0, 3, [MODIFICATION-TIME-2], 1, [undef], 0, 0), rParamSecure = [undef], strKey = pg_data/PG_VERSION, strOp = backupFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?%, pg_data/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?%, 0, [undef], 0, [BACKUP-FULL-1], 0, 3, [MODIFICATION-TIME-2], 1, [undef], 0, 0), rParamSecure = [undef], strKey = pg_data/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?%, strOp = backupFile, strQueue = pg_data
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->hostConnect: start local process: iHostConfigIdx = 1, iHostIdx = 0, iHostProcessIdx = 0, iProcessId = 1, strHostType = db
Add log-subprocess option to allow file logging for local and remote subprocesses.
2018-08-23 02:05:49 +02:00
P00 DEBUG: Protocol::Local::Master->new(): iProcessIdx = 1, strCommand = [BACKREST-BIN] --buffer-size=16384 --command=backup --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --db-timeout=45 --host-id=1 --lock-path=[TEST_PATH]/db-master/lock --log-level-file=trace --log-path=[TEST_PATH]/db-master/log --log-subprocess --pg1-path=[TEST_PATH]/db-master/db/base --process=1 --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --repo1-type=cifs --stanza=db --type=db local
P00 DEBUG: Protocol::Command::Master->new(): iBufferMax = 16384, iCompressLevel = 3, iCompressLevelNetwork = 3, iProtocolTimeout = 60, strCommand = [BACKREST-BIN] --buffer-size=16384 --command=backup --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --db-timeout=45 --host-id=1 --lock-path=[TEST_PATH]/db-master/lock --log-level-file=trace --log-path=[TEST_PATH]/db-master/log --log-subprocess --pg1-path=[TEST_PATH]/db-master/db/base --process=1 --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --repo1-type=cifs --stanza=db --type=db local, strId = local-1 process, strName = local
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->hostConnect=>: bResult = true
P00 DEBUG: Protocol::Local::Process->init: init local process: iDirection = 1, iHostIdx = 0, iProcessId = 1, iQueueIdx = 0, iQueueLastIdx = 0
P00 DEBUG: Protocol::Local::Process->init=>: bResult = true
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/base/32768/33001, strQueueIdx = 0
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1, 65536, 65536, 6bf316f11d28c28914ea9be92c00de9bea6d9a6b, {bAlign => 1, bValid => 0, iyPageError => (0, (3, 5), 7)}), strKey = pg_data/base/32768/33001
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/base/32768/44000_init, strQueueIdx = 0
P01 INFO: backup file [TEST_PATH]/db-master/db/base/base/32768/33001 (64KB, 33%) checksum 6bf316f11d28c28914ea9be92c00de9bea6d9a6b
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
P00 WARN: invalid page checksums found in file [TEST_PATH]/db-master/db/base/base/32768/33001 at pages 0, 3-5, 7
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-1]/backup.manifest
P00 DEBUG: Storage::Local->exists=>: bExists = false
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->openWrite(): bAtomic = <false>, bPathCreate = <false>, lTimestamp = [undef], rhyFilter = [undef], strCipherPass = [undef], strGroup = [undef], strMode = <0640>, strUser = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-1]/backup.manifest.copy
Move backup modules into Backup directory.
2017-05-15 22:01:00 +02:00
P00 DEBUG: Backup::File::backupManifestUpdate: save manifest: lManifestSaveCurrent = 65536, lManifestSaveSize = 3
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1, 32768, 32768, 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f, {bAlign => 1, bValid => 1}), strKey = pg_data/base/32768/44000_init
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/base/32768/33000.32767, strQueueIdx = 0
P01 INFO: backup file [TEST_PATH]/db-master/db/base/base/32768/44000_init (32KB, 49%) checksum 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-1]/backup.manifest
P00 DEBUG: Storage::Local->exists=>: bExists = false
P00 DEBUG: Storage::Local->openWrite(): bAtomic = <false>, bPathCreate = <false>, lTimestamp = [undef], rhyFilter = [undef], strCipherPass = [undef], strGroup = [undef], strMode = <0640>, strUser = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-1]/backup.manifest.copy
P00 DEBUG: Backup::File::backupManifestUpdate: save manifest: lManifestSaveCurrent = 32768, lManifestSaveSize = 3
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1, 32768, 32768, 21e2c7c1a326682c07053b7d6a5a40dbd49c2ec5, {bAlign => 1, bValid => 1}), strKey = pg_data/base/32768/33000.32767
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/base/32768/33000, strQueueIdx = 0
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
P01 INFO: backup file [TEST_PATH]/db-master/db/base/base/32768/33000.32767 (32KB, 66%) checksum 21e2c7c1a326682c07053b7d6a5a40dbd49c2ec5
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-1]/backup.manifest
P00 DEBUG: Storage::Local->exists=>: bExists = false
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->openWrite(): bAtomic = <false>, bPathCreate = <false>, lTimestamp = [undef], rhyFilter = [undef], strCipherPass = [undef], strGroup = [undef], strMode = <0640>, strUser = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-1]/backup.manifest.copy
Move backup modules into Backup directory.
2017-05-15 22:01:00 +02:00
P00 DEBUG: Backup::File::backupManifestUpdate: save manifest: lManifestSaveCurrent = 32768, lManifestSaveSize = 3
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1, 32768, 32768, 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f, {bAlign => 1, bValid => 1}), strKey = pg_data/base/32768/33000
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/base/16384/17000, strQueueIdx = 0
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
P01 INFO: backup file [TEST_PATH]/db-master/db/base/base/32768/33000 (32KB, 83%) checksum 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-1]/backup.manifest
P00 DEBUG: Storage::Local->exists=>: bExists = false
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->openWrite(): bAtomic = <false>, bPathCreate = <false>, lTimestamp = [undef], rhyFilter = [undef], strCipherPass = [undef], strGroup = [undef], strMode = <0640>, strUser = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-1]/backup.manifest.copy
Move backup modules into Backup directory.
2017-05-15 22:01:00 +02:00
P00 DEBUG: Backup::File::backupManifestUpdate: save manifest: lManifestSaveCurrent = 32768, lManifestSaveSize = 3
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1, 16384, 16384, e0101dd8ffb910c9c202ca35b5f828bcb9697bed, {bAlign => 1, bValid => 0, iyPageError => (1)}), strKey = pg_data/base/16384/17000
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/global/pg_control, strQueueIdx = 0
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
P01 INFO: backup file [TEST_PATH]/db-master/db/base/base/16384/17000 (16KB, 91%) checksum e0101dd8ffb910c9c202ca35b5f828bcb9697bed
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
P00 WARN: invalid page checksum found in file [TEST_PATH]/db-master/db/base/base/16384/17000 at page 1
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-1]/backup.manifest
P00 DEBUG: Storage::Local->exists=>: bExists = false
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->openWrite(): bAtomic = <false>, bPathCreate = <false>, lTimestamp = [undef], rhyFilter = [undef], strCipherPass = [undef], strGroup = [undef], strMode = <0640>, strUser = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-1]/backup.manifest.copy
Move backup modules into Backup directory.
2017-05-15 22:01:00 +02:00
P00 DEBUG: Backup::File::backupManifestUpdate: save manifest: lManifestSaveCurrent = 16384, lManifestSaveSize = 3
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 11:24:42 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1, 8192, 8192, 4c77c900f7af0d9ab13fa9982051a42e0b637f6c, [undef]), strKey = pg_data/global/pg_control
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/base/1/12000, strQueueIdx = 0
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 11:24:42 +02:00
P01 INFO: backup file [TEST_PATH]/db-master/db/base/global/pg_control (8KB, 95%) checksum 4c77c900f7af0d9ab13fa9982051a42e0b637f6c
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-1]/backup.manifest
P00 DEBUG: Storage::Local->exists=>: bExists = false
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->openWrite(): bAtomic = <false>, bPathCreate = <false>, lTimestamp = [undef], rhyFilter = [undef], strCipherPass = [undef], strGroup = [undef], strMode = <0640>, strUser = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-1]/backup.manifest.copy
Move backup modules into Backup directory.
2017-05-15 22:01:00 +02:00
P00 DEBUG: Backup::File::backupManifestUpdate: save manifest: lManifestSaveCurrent = 8192, lManifestSaveSize = 3
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1, 8192, 8192, 22c98d248ff548311eda88559e4a8405ed77c003, {bAlign => 1, bValid => 1}), strKey = pg_data/base/1/12000
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/postgresql.conf, strQueueIdx = 0
Add more realistic data files to synthetic backup and restore tests.
2016-12-10 16:13:55 +02:00
P01 INFO: backup file [TEST_PATH]/db-master/db/base/base/1/12000 (8KB, 99%) checksum 22c98d248ff548311eda88559e4a8405ed77c003
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-1]/backup.manifest
P00 DEBUG: Storage::Local->exists=>: bExists = false
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->openWrite(): bAtomic = <false>, bPathCreate = <false>, lTimestamp = [undef], rhyFilter = [undef], strCipherPass = [undef], strGroup = [undef], strMode = <0640>, strUser = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-1]/backup.manifest.copy
Move backup modules into Backup directory.
2017-05-15 22:01:00 +02:00
P00 DEBUG: Backup::File::backupManifestUpdate: save manifest: lManifestSaveCurrent = 8192, lManifestSaveSize = 3
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1, 21, 21, 6721d92c9fcdf4248acff1f9a1377127d9064807, [undef]), strKey = pg_data/postgresql.conf
Fix failure in manifest build when two or more files in PGDATA are linked to the same directory. Reported by Vitaliy Kukharik.
2018-05-02 18:19:54 +02:00
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/pg_hba.conf, strQueueIdx = 0
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 INFO: backup file [TEST_PATH]/db-master/db/base/postgresql.conf (21B, 99%) checksum 6721d92c9fcdf4248acff1f9a1377127d9064807
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-1]/backup.manifest
P00 DEBUG: Storage::Local->exists=>: bExists = false
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->openWrite(): bAtomic = <false>, bPathCreate = <false>, lTimestamp = [undef], rhyFilter = [undef], strCipherPass = [undef], strGroup = [undef], strMode = <0640>, strUser = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-1]/backup.manifest.copy
Move backup modules into Backup directory.
2017-05-15 22:01:00 +02:00
P00 DEBUG: Backup::File::backupManifestUpdate: save manifest: lManifestSaveCurrent = 21, lManifestSaveSize = 3
Fix failure in manifest build when two or more files in PGDATA are linked to the same directory. Reported by Vitaliy Kukharik.
2018-05-02 18:19:54 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1, 9, 9, dd4cea0cae348309f9de28ad4ded8ee2cc2e6d5b, [undef]), strKey = pg_data/pg_hba.conf
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/pg_stat/global.stat, strQueueIdx = 0
P01 INFO: backup file [TEST_PATH]/db-master/db/base/pg_hba.conf (9B, 99%) checksum dd4cea0cae348309f9de28ad4ded8ee2cc2e6d5b
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-1]/backup.manifest
P00 DEBUG: Storage::Local->exists=>: bExists = false
P00 DEBUG: Storage::Local->openWrite(): bAtomic = <false>, bPathCreate = <false>, lTimestamp = [undef], rhyFilter = [undef], strCipherPass = [undef], strGroup = [undef], strMode = <0640>, strUser = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-1]/backup.manifest.copy
P00 DEBUG: Backup::File::backupManifestUpdate: save manifest: lManifestSaveCurrent = 9, lManifestSaveSize = 3
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1, 5, 5, e350d5ce0153f3e22d5db21cf2a4eff00f3ee877, [undef]), strKey = pg_data/pg_stat/global.stat
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/base/32768/PG_VERSION, strQueueIdx = 0
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 INFO: backup file [TEST_PATH]/db-master/db/base/pg_stat/global.stat (5B, 99%) checksum e350d5ce0153f3e22d5db21cf2a4eff00f3ee877
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-1]/backup.manifest
P00 DEBUG: Storage::Local->exists=>: bExists = false
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->openWrite(): bAtomic = <false>, bPathCreate = <false>, lTimestamp = [undef], rhyFilter = [undef], strCipherPass = [undef], strGroup = [undef], strMode = <0640>, strUser = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-1]/backup.manifest.copy
Move backup modules into Backup directory.
2017-05-15 22:01:00 +02:00
P00 DEBUG: Backup::File::backupManifestUpdate: save manifest: lManifestSaveCurrent = 5, lManifestSaveSize = 3
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1, 3, 3, 184473f470864e067ee3a22e64b47b0a1c356f29, [undef]), strKey = pg_data/base/32768/PG_VERSION
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/base/16384/PG_VERSION, strQueueIdx = 0
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 INFO: backup file [TEST_PATH]/db-master/db/base/base/32768/PG_VERSION (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-1]/backup.manifest
P00 DEBUG: Storage::Local->exists=>: bExists = false
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->openWrite(): bAtomic = <false>, bPathCreate = <false>, lTimestamp = [undef], rhyFilter = [undef], strCipherPass = [undef], strGroup = [undef], strMode = <0640>, strUser = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-1]/backup.manifest.copy
Move backup modules into Backup directory.
2017-05-15 22:01:00 +02:00
P00 DEBUG: Backup::File::backupManifestUpdate: save manifest: lManifestSaveCurrent = 3, lManifestSaveSize = 3
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1, 3, 3, 184473f470864e067ee3a22e64b47b0a1c356f29, [undef]), strKey = pg_data/base/16384/PG_VERSION
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/base/1/PG_VERSION, strQueueIdx = 0
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 INFO: backup file [TEST_PATH]/db-master/db/base/base/16384/PG_VERSION (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-1]/backup.manifest
P00 DEBUG: Storage::Local->exists=>: bExists = false
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->openWrite(): bAtomic = <false>, bPathCreate = <false>, lTimestamp = [undef], rhyFilter = [undef], strCipherPass = [undef], strGroup = [undef], strMode = <0640>, strUser = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-1]/backup.manifest.copy
Move backup modules into Backup directory.
2017-05-15 22:01:00 +02:00
P00 DEBUG: Backup::File::backupManifestUpdate: save manifest: lManifestSaveCurrent = 3, lManifestSaveSize = 3
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1, 3, 3, 184473f470864e067ee3a22e64b47b0a1c356f29, [undef]), strKey = pg_data/base/1/PG_VERSION
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/PG_VERSION, strQueueIdx = 0
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 INFO: backup file [TEST_PATH]/db-master/db/base/base/1/PG_VERSION (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-1]/backup.manifest
P00 DEBUG: Storage::Local->exists=>: bExists = false
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->openWrite(): bAtomic = <false>, bPathCreate = <false>, lTimestamp = [undef], rhyFilter = [undef], strCipherPass = [undef], strGroup = [undef], strMode = <0640>, strUser = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-1]/backup.manifest.copy
Move backup modules into Backup directory.
2017-05-15 22:01:00 +02:00
P00 DEBUG: Backup::File::backupManifestUpdate: save manifest: lManifestSaveCurrent = 3, lManifestSaveSize = 3
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1, 3, 3, 184473f470864e067ee3a22e64b47b0a1c356f29, [undef]), strKey = pg_data/PG_VERSION
Improve support for special characters in filenames. % characters caused issues in backup/restore due to filenames being appended directly into a format string. Reserved XML characters (<>&') caused issues in the S3 driver due to improper escaping. Add a file with all common special characters to regression testing.
2018-09-10 16:54:34 +02:00
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?%, strQueueIdx = 0
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 INFO: backup file [TEST_PATH]/db-master/db/base/PG_VERSION (3B, 100%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-1]/backup.manifest
P00 DEBUG: Storage::Local->exists=>: bExists = false
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->openWrite(): bAtomic = <false>, bPathCreate = <false>, lTimestamp = [undef], rhyFilter = [undef], strCipherPass = [undef], strGroup = [undef], strMode = <0640>, strUser = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-1]/backup.manifest.copy
Move backup modules into Backup directory.
2017-05-15 22:01:00 +02:00
P00 DEBUG: Backup::File::backupManifestUpdate: save manifest: lManifestSaveCurrent = 3, lManifestSaveSize = 3
Improve support for special characters in filenames. % characters caused issues in backup/restore due to filenames being appended directly into a format string. Reserved XML characters (<>&') caused issues in the S3 driver due to improper escaping. Add a file with all common special characters to regression testing.
2018-09-10 16:54:34 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1, 0, 0, da39a3ee5e6b4b0d3255bfef95601890afd80709, [undef]), strKey = pg_data/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?%
P00 DEBUG: Protocol::Local::Process->process: no jobs found, stop local: iHostConfigIdx = [undef], iHostIdx = 0, iProcessId = 1, strHostType = [undef]
P00 DEBUG: Protocol::Command::Master->close=>: iExitStatus = 0
P01 INFO: backup file [TEST_PATH]/db-master/db/base/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?% (0B, 100%)
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->process: all jobs complete
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
P00 DEBUG: Backup::Backup->processManifest=>: lSizeTotal = 196655
P00 INFO: full backup size = 192KB
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Helper::protocolDestroy(): bComplete = true, iRemoteIdx = [undef], strRemoteType = [undef]
P00 DEBUG: Protocol::Helper::protocolDestroy=>: iExitStatus = 0
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->openWrite(): bAtomic = <false>, bPathCreate = <false>, lTimestamp = [undef], rhyFilter = [undef], strCipherPass = [undef], strGroup = [undef], strMode = <0640>, strUser = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-1]/backup.manifest
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-1]
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->openWrite(): bAtomic = <false>, bPathCreate = <false>, lTimestamp = [undef], rhyFilter = [undef], strCipherPass = [undef], strGroup = [undef], strMode = <0640>, strUser = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-1]/backup.manifest.copy
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-1]
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: new backup label = [BACKUP-FULL-1]
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->openRead(): bIgnoreMissing = <false>, rhyFilter = [undef], strCipherPass = [undef], xFileExp = <REPO:BACKUP>/[BACKUP-FULL-1]/backup.manifest
Make backup.history sync more efficient. Only the backup.history/[year] directory was being synced, so check if the backup.history is newly created and sync it as well.
2018-04-04 02:54:18 +02:00
P00 DEBUG: Storage::Local->openWrite(): bAtomic = true, bPathCreate = true, lTimestamp = [undef], rhyFilter = ({strClass => pgBackRest::Storage::Filter::Gzip}), strCipherPass = [undef], strGroup = [undef], strMode = <0640>, strUser = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/backup.history/[YEAR-1]/[BACKUP-FULL-1].manifest.gz
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Base->copy(): xDestinationFile = [object], xSourceFile = [object]
Add bIgnoreMissing parameter to Local->manifest().
2017-07-25 18:44:38 +02:00
P00 DEBUG: Storage::Local->remove(): bIgnoreMissing = <true>, bRecurse = <false>, xstryPathFileExp = <REPO:BACKUP>/latest
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->remove=>: bRemoved = false
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->openWrite(): bAtomic = <false>, bPathCreate = <false>, lTimestamp = [undef], rhyFilter = [undef], strCipherPass = [undef], strGroup = [undef], strMode = <0640>, strUser = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/backup.info
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/repo/backup/db
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->openWrite(): bAtomic = <false>, bPathCreate = <false>, lTimestamp = [undef], rhyFilter = [undef], strCipherPass = [undef], strGroup = [undef], strMode = <0640>, strUser = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/backup.info.copy
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/repo/backup/db
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: perl/exec::perlExec: => 0
Added a predictable log entry to signal that a command has completed successfully. For example a backup ends successfully with: INFO: backup command end: completed successfully.
2016-12-04 21:44:53 +02:00
P00 INFO: backup command end: completed successfully
Move async forking and more error handling to C. The Perl process was exiting directly when called but that interfered with proper locking for the forked async process. Now Perl returns results to the C process which handles all errors, including signals.
2018-04-13 02:42:26 +02:00
P00 INFO: expire command begin
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: perl/exec::perlExec: (void)
Move async forking and more error handling to C. The Perl process was exiting directly when called but that interfered with proper locking for the forked async process. Now Perl returns results to the C process which handles all errors, including signals.
2018-04-13 02:42:26 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp =
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = true, bIgnoreExists = true, strMode = 0770, strPathExp = [TEST_PATH]/db-master/log
P00 DEBUG: Common::Lock::lockStopTest(): bStanzaStopRequired = <false>
P00 DEBUG: Common::Lock::lockStopTest=>: bStopExists = false
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Backup::Info->new(): bIgnoreMissing = <false>, bLoad = <true>, bRequired = <true>, bValidate = <true>, oStorage = <[object]>, strBackupClusterPath = [TEST_PATH]/db-master/repo/backup/db, strCipherPassSub = [undef]
P00 DEBUG: Storage::Local->encrypted(): bIgnoreMissing = true, strFileName = [TEST_PATH]/db-master/repo/backup/db/backup.info
P00 DEBUG: Storage::Local->encrypted=>: bEncrypted = false
P00 DEBUG: Storage::Local->encryptionValid(): bEncrypted = false
P00 DEBUG: Storage::Local->encryptionValid=>: bValid = true
P00 DEBUG: Storage::Local->openRead(): bIgnoreMissing = true, rhyFilter = [undef], strCipherPass = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/backup.info
P00 DEBUG: Storage::Base->get(): strCipherPass = [undef], xFile = [object]
Move backup modules into Backup directory.
2017-05-15 22:01:00 +02:00
P00 DEBUG: Backup::Info->reconstruct(): bRequired = <true>, bSave = <true>, iCatalogVersion = [undef], iControlVersion = [undef], strDbVersion = [undef], ullDbSysId = [undef]
P00 DEBUG: Backup::Common::backupRegExpGet(): bAnchor = <true>, bDifferential = true, bFull = true, bIncremental = true
P00 DEBUG: Backup::Common::backupRegExpGet=>: strRegExp = ^[0-9]{8}\-[0-9]{6}F(\_[0-9]{8}\-[0-9]{6}(D|I)){0,1}$
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->list(): bIgnoreMissing = <false>, strExpression = ^[0-9]{8}\-[0-9]{6}F(\_[0-9]{8}\-[0-9]{6}(D|I)){0,1}$, strPathExp = [TEST_PATH]/db-master/repo/backup/db, strSortOrder = <forward>
P00 DEBUG: Storage::Local->list=>: stryFileList = ([BACKUP-FULL-1])
Move backup modules into Backup directory.
2017-05-15 22:01:00 +02:00
P00 DEBUG: Backup::Info->current(): strBackup = [BACKUP-FULL-1]
P00 DEBUG: Backup::Info->current=>: bTest = true
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-1]
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-1]/backup.manifest
P00 DEBUG: Storage::Local->exists=>: bExists = true
Move backup modules into Backup directory.
2017-05-15 22:01:00 +02:00
P00 DEBUG: Backup::Common::backupRegExpGet(): bAnchor = <true>, bDifferential = true, bFull = true, bIncremental = true
P00 DEBUG: Backup::Common::backupRegExpGet=>: strRegExp = ^[0-9]{8}\-[0-9]{6}F(\_[0-9]{8}\-[0-9]{6}(D|I)){0,1}$
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->list(): bIgnoreMissing = <false>, strExpression = ^[0-9]{8}\-[0-9]{6}F(\_[0-9]{8}\-[0-9]{6}(D|I)){0,1}$, strPathExp = <REPO:BACKUP>, strSortOrder = reverse
P00 DEBUG: Storage::Local->list=>: stryFileList = ([BACKUP-FULL-1])
Move backup modules into Backup directory.
2017-05-15 22:01:00 +02:00
P00 DEBUG: Backup::Info->current(): strBackup = [BACKUP-FULL-1]
P00 DEBUG: Backup::Info->current=>: bTest = true
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
P00 INFO: option 'repo1-retention-archive' is not set - archive logs will not be expired
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: perl/exec::perlExec: => 0
P00 DEBUG: common/exit::exitSafe: (result: 0, error: false, signalType: 0)
P00 DEBUG: common/lock::lockRelease: (failOnNoLock: false)
P00 DEBUG: common/lock::lockRelease: => true
Move async forking and more error handling to C. The Perl process was exiting directly when called but that interfered with proper locking for the forked async process. Now Perl returns results to the C process which handles all errors, including signals.
2018-04-13 02:42:26 +02:00
P00 DEBUG: Main::mainCleanup(): iExitCode = 0
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Helper::protocolDestroy(): bComplete = true, iRemoteIdx = [undef], strRemoteType = [undef]
P00 DEBUG: Protocol::Helper::protocolDestroy=>: iExitStatus = 0
Added a predictable log entry to signal that a command has completed successfully. For example a backup ends successfully with: INFO: backup command end: completed successfully.
2016-12-04 21:44:53 +02:00
P00 INFO: expire command end: completed successfully
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: common/exit::exitSafe: => 0
P00 DEBUG: main::main: => 0
Log testing can now be enabled for certain deterministic tests. This works by comparing the generated logs against a previous copy. Currently only enabled for the backup/synthetic tests.
2015-05-05 19:08:48 +02:00
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
+ supplemental file: [TEST_PATH]/db-master/pgbackrest.conf
----------------------------------------------------------
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[db]
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
pg1-path=[TEST_PATH]/db-master/db/base
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
New simpler configuration and consistent project/exe/path naming. * The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required. * Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case. * The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally. * The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest. * Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory. * Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory. * Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 15:30:54 +02:00
[global]
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
compress=n
Fix custom settings for compress-level option being ignored. Reported by Jens Wilke.
2017-10-23 00:05:46 +02:00
compress-level=3
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
db-timeout=45
lock-path=[TEST_PATH]/db-master/lock
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
log-level-console=debug
log-level-file=trace
Added the log-level-stderr option. Controls whether console log messages are sent to stderr or stdout. By default this is set to warn which represents a change in behavior from previous versions, even though it may be more intuitive. Setting log-level-stderr=off will preserve the old behavior. Suggested by Sascha Biberhofer.
2016-10-05 15:09:30 +02:00
log-level-stderr=off
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
log-path=[TEST_PATH]/db-master/log
Add log-subprocess option to allow file logging for local and remote subprocesses.
2018-08-23 02:05:49 +02:00
log-subprocess=y
Set log-timestamp=n for integration tests. This means less filtering of logs needs to be done and new timestamps can be added without adding new filters.
2018-05-11 17:24:38 +02:00
log-timestamp=n
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
protocol-timeout=60
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
repo1-path=[TEST_PATH]/db-master/repo
Consolidate archive-push tests into new archive test.
2017-06-21 14:38:53 +02:00
spool-path=[TEST_PATH]/db-master/spool
New simpler configuration and consistent project/exe/path naming. * The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required. * Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case. * The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally. * The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest. * Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory. * Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory. * Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 15:30:54 +02:00
[global:backup]
archive-copy=y
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
exclude=postgresql.auto.conf
exclude=pg_log/
exclude=pg_log2
New simpler configuration and consistent project/exe/path naming. * The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required. * Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case. * The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally. * The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest. * Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory. * Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory. * Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 15:30:54 +02:00
start-fast=y
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
+ supplemental file: [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-1]/backup.manifest
-----------------------------------------------------------------------------------------
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[backrest]
backrest-checksum="[CHECKSUM]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
backrest-format=5
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
backrest-version="[VERSION-1]"
[backup]
backup-label="[BACKUP-FULL-1]"
backup-timestamp-copy-start=[TIMESTAMP]
backup-timestamp-start=[TIMESTAMP]
backup-timestamp-stop=[TIMESTAMP]
backup-type="full"
[backup:db]
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-catalog-version=201409291
db-control-version=942
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
db-id=1
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
db-system-id=1000000000000000094
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-version="9.4"
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[backup:option]
option-archive-check=true
option-archive-copy=true
Backup from a standby cluster. A connection to the primary cluster is still required to start/stop the backup and copy files that are not replicated, but the vast majority of files are copied from the standby in order to reduce load on the master.
2016-08-25 17:25:46 +02:00
option-backup-standby=false
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
option-checksum-page=true
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
option-compress=false
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
option-delta=false
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
option-hardlink=false
Closed #167: Rename `--no-start-stop` option to `--no-online`.
2016-02-20 02:32:02 +02:00
option-online=false
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[backup:target]
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
pg_data={"path":"[TEST_PATH]/db-master/db/base","type":"path"}
Fix failure in manifest build when two or more files in PGDATA are linked to the same directory. Reported by Vitaliy Kukharik.
2018-05-02 18:19:54 +02:00
pg_data/pg_hba.conf={"file":"pg_hba.conf","path":"../pg_config","type":"link"}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_data/pg_stat={"path":"../pg_stat","type":"link"}
pg_data/postgresql.conf={"file":"postgresql.conf","path":"../pg_config","type":"link"}
[target:file]
Fixed a issue where tablespaces were copied from the master during standby backup.
2016-09-04 15:19:44 +02:00
pg_data/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","master":true,"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
pg_data/base/1/12000={"checksum":"22c98d248ff548311eda88559e4a8405ed77c003","checksum-page":true,"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
pg_data/base/1/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","mode":"0660","size":[SIZE],"timestamp":[TIMESTAMP-1],"user":false}
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
pg_data/base/16384/17000={"checksum":"e0101dd8ffb910c9c202ca35b5f828bcb9697bed","checksum-page":false,"checksum-page-error":[1],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
pg_data/base/16384/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","group":false,"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Reduced the likelihood of torn pages causing a false positive in page checksums by filtering on start backup LSN.
2017-01-30 20:59:00 +02:00
pg_data/base/32768/33000={"checksum":"4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f","checksum-page":true,"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Fixed an issue where newly initialized (but unused) pages would cause page checksum warnings. Reported by Stephen Frost.
2017-03-13 21:50:13 +02:00
pg_data/base/32768/33000.32767={"checksum":"21e2c7c1a326682c07053b7d6a5a40dbd49c2ec5","checksum-page":true,"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
pg_data/base/32768/33001={"checksum":"6bf316f11d28c28914ea9be92c00de9bea6d9a6b","checksum-page":false,"checksum-page-error":[0,[3,5],7],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
pg_data/base/32768/44000_init={"checksum":"4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f","checksum-page":true,"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
pg_data/base/32768/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 11:24:42 +02:00
pg_data/global/pg_control={"checksum":"4c77c900f7af0d9ab13fa9982051a42e0b637f6c","master":true,"size":[SIZE],"timestamp":[TIMESTAMP-2]}
Fix failure in manifest build when two or more files in PGDATA are linked to the same directory. Reported by Vitaliy Kukharik.
2018-05-02 18:19:54 +02:00
pg_data/pg_hba.conf={"checksum":"dd4cea0cae348309f9de28ad4ded8ee2cc2e6d5b","master":true,"size":[SIZE],"timestamp":[TIMESTAMP-2]}
Fixed a issue where tablespaces were copied from the master during standby backup.
2016-09-04 15:19:44 +02:00
pg_data/pg_stat/global.stat={"checksum":"e350d5ce0153f3e22d5db21cf2a4eff00f3ee877","master":true,"size":[SIZE],"timestamp":[TIMESTAMP-2]}
pg_data/postgresql.conf={"checksum":"6721d92c9fcdf4248acff1f9a1377127d9064807","master":true,"size":[SIZE],"timestamp":[TIMESTAMP-2]}
Improve support for special characters in filenames. % characters caused issues in backup/restore due to filenames being appended directly into a format string. Reserved XML characters (<>&') caused issues in the S3 driver due to improper escaping. Add a file with all common special characters to regression testing.
2018-09-10 16:54:34 +02:00
pg_data/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?%={"master":true,"size":[SIZE],"timestamp":[TIMESTAMP-1]}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:file:default]
Continuous integration using travis-ci.
2017-02-21 15:59:23 +02:00
group="[GROUP-1]"
Fixed a issue where tablespaces were copied from the master during standby backup.
2016-09-04 15:19:44 +02:00
master=false
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
mode="0600"
Regression tests can now be run as any properly-configured user, not just vagrant.
2016-10-01 19:39:44 +02:00
user="[USER-1]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:link]
Fix failure in manifest build when two or more files in PGDATA are linked to the same directory. Reported by Vitaliy Kukharik.
2018-05-02 18:19:54 +02:00
pg_data/pg_hba.conf={"destination":"../pg_config/pg_hba.conf"}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_data/pg_stat={"destination":"../pg_stat"}
pg_data/postgresql.conf={"destination":"../pg_config/postgresql.conf"}
[target:link:default]
Continuous integration using travis-ci.
2017-02-21 15:59:23 +02:00
group="[GROUP-1]"
Regression tests can now be run as any properly-configured user, not just vagrant.
2016-10-01 19:39:44 +02:00
user="[USER-1]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:path]
pg_data={}
pg_data/base={}
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
pg_data/base/1={}
pg_data/base/16384={}
pg_data/base/32768={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_data/global={}
pg_data/pg_clog={}
Exclude directories during backup that are cleaned, recreated, or zeroed by PostgreSQL at startup. These include (depending on the version where they were introduced): pgsql_tmp, pg_dynshmem, pg_notify, pg_replslot, pg_serial, pg_snapshots, pg_stat_tmp, pg_subtrans. The postgresql.auto.conf.tmp file is now excluded in addition to files that were already excluded: backup_label.old, postmaster.opts, postmaster.pid, recovery.conf, recovery.done.
2016-08-16 15:35:16 +02:00
pg_data/pg_dynshmem={}
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
pg_data/pg_log={}
Exclude directories during backup that are cleaned, recreated, or zeroed by PostgreSQL at startup. These include (depending on the version where they were introduced): pgsql_tmp, pg_dynshmem, pg_notify, pg_replslot, pg_serial, pg_snapshots, pg_stat_tmp, pg_subtrans. The postgresql.auto.conf.tmp file is now excluded in addition to files that were already excluded: backup_label.old, postmaster.opts, postmaster.pid, recovery.conf, recovery.done.
2016-08-16 15:35:16 +02:00
pg_data/pg_notify={}
pg_data/pg_replslot={}
pg_data/pg_serial={}
pg_data/pg_snapshots={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_data/pg_stat={}
Exclude directories during backup that are cleaned, recreated, or zeroed by PostgreSQL at startup. These include (depending on the version where they were introduced): pgsql_tmp, pg_dynshmem, pg_notify, pg_replslot, pg_serial, pg_snapshots, pg_stat_tmp, pg_subtrans. The postgresql.auto.conf.tmp file is now excluded in addition to files that were already excluded: backup_label.old, postmaster.opts, postmaster.pid, recovery.conf, recovery.done.
2016-08-16 15:35:16 +02:00
pg_data/pg_stat_tmp={}
pg_data/pg_subtrans={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_data/pg_tblspc={}
[target:path:default]
Continuous integration using travis-ci.
2017-02-21 15:59:23 +02:00
group="[GROUP-1]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
mode="0700"
Regression tests can now be run as any properly-configured user, not just vagrant.
2016-10-01 19:39:44 +02:00
user="[USER-1]"
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
+ supplemental file: [TEST_PATH]/db-master/repo/backup/db/backup.info
---------------------------------------------------------------------
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[backrest]
backrest-checksum="[CHECKSUM]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
backrest-format=5
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
backrest-version="[VERSION-1]"
[backup:current]
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
[BACKUP-FULL-1]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"full","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":true,"option-compress":false,"option-hardlink":false,"option-online":false}
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[db]
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-catalog-version=201409291
db-control-version=942
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
db-id=1
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
db-system-id=1000000000000000094
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-version="9.4"
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[db:history]
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
1={"db-catalog-version":201409291,"db-control-version":942,"db-system-id":1000000000000000094,"db-version":"9.4"}
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
Fixed an issue that prevented errors from being output to the console before the logging system was initialized. That is, while parsing options. Error codes were still being returned accurately so this would not have made a process look like it succeeded when it did not.
2016-12-04 01:01:17 +02:00
full backup - invalid cmd line (db-master host)
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --no-online --type=full --stanza=bogus backup
------------------------------------------------------------------------------------------------------------------------------------
STDERR:
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
ERROR: [037]: backup command requires option: pg1-path
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
HINT: does this stanza exist?
Fixed an issue that prevented errors from being output to the console before the logging system was initialized. That is, while parsing options. Error codes were still being returned accurately so this would not have made a process look like it succeeded when it did not.
2016-12-04 01:01:17 +02:00
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
stop all stanzas (db-master host)
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --force stop
Implemented issue #129: Stop/Start Commands. Implemented issue #136: Add stderr back into protocol layer. There were also miscellaneous bug fixes and improvements to buffering in IO.pm.
2015-10-08 17:43:56 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Add log-subprocess option to allow file logging for local and remote subprocesses.
2018-08-23 02:05:49 +02:00
P00 INFO: stop command begin [BACKREST-VERSION]: --config=[TEST_PATH]/db-master/pgbackrest.conf --force --lock-path=[TEST_PATH]/db-master/lock --log-level-console=debug --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --repo1-path=[TEST_PATH]/db-master/repo
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: config/load::cfgLoad: => void
P00 DEBUG: perl/exec::perlExec: (void)
Config parsing no longer implemented in Perl. Options are passed to Perl as JSON from the C process.
2018-01-29 04:37:09 +02:00
P00 DEBUG: Storage::Posix::Driver->new(): bFileSync = <true>, bPathSync = <true>
P00 DEBUG: Storage::Local->new(): bAllowTemp = <true>, hRule = [undef], lBufferMax = [undef], oDriver = [object], strCipherPassUser = [undef], strCipherType = [undef], strDefaultFileMode = <0640>, strDefaultPathMode = <0750>, strPathBase = /, strTempExtension = pgbackrest.tmp
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = true, bIgnoreExists = true, strMode = 770, strPathExp = [TEST_PATH]/db-master/lock
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: sent term signal to process [PROCESS-ID]
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: perl/exec::perlExec: => 0
P00 DEBUG: common/exit::exitSafe: (result: 0, error: false, signalType: 0)
P00 DEBUG: common/lock::lockRelease: (failOnNoLock: false)
P00 DEBUG: common/lock::lockRelease: => false
Move async forking and more error handling to C. The Perl process was exiting directly when called but that interfered with proper locking for the forked async process. Now Perl returns results to the C process which handles all errors, including signals.
2018-04-13 02:42:26 +02:00
P00 DEBUG: Main::mainCleanup(): iExitCode = 0
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Helper::protocolDestroy(): bComplete = true, iRemoteIdx = [undef], strRemoteType = [undef]
P00 DEBUG: Protocol::Helper::protocolDestroy=>: iExitStatus = 0
Added a predictable log entry to signal that a command has completed successfully. For example a backup ends successfully with: INFO: backup command end: completed successfully.
2016-12-04 21:44:53 +02:00
P00 INFO: stop command end: completed successfully
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: common/exit::exitSafe: => 0
P00 DEBUG: main::main: => 0
Implemented issue #129: Stop/Start Commands. Implemented issue #136: Add stderr back into protocol layer. There were also miscellaneous bug fixes and improvements to buffering in IO.pm.
2015-10-08 17:43:56 +02:00
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
full backup - abort backup - local (db-master host)
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --no-online --type=full --stanza=db backup --test --test-delay=5 --test-point=backup-start=y
Implemented issue #129: Stop/Start Commands. Implemented issue #136: Add stderr back into protocol layer. There were also miscellaneous bug fixes and improvements to buffering in IO.pm.
2015-10-08 17:43:56 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
P00 INFO: backup command begin [BACKREST-VERSION]: --no-compress --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --db-timeout=45 --exclude=postgresql.auto.conf --exclude=pg_log/ --exclude=pg_log2 --lock-path=[TEST_PATH]/db-master/lock --log-level-console=debug --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --no-online --pg1-path=[TEST_PATH]/db-master/db/base --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --stanza=db --start-fast --test --test-delay=5 --test-point=backup-start=y --type=full
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: common/lock::lockAcquire: (lockPath: {"[TEST_PATH]/db-master/lock"}, stanza: {"db"}, lockType: 1, lockTimeout: 0, failOnNoLock: true)
P00 DEBUG: common/lock::lockAcquire: => true
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
P00 WARN: option repo1-retention-full is not set, the repository may run out of space
HINT: to retain full backups indefinitely (without warning), set option 'repo1-retention-full' to the maximum.
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: config/load::cfgLoad: => void
P00 DEBUG: perl/exec::perlExec: (void)
Allow functions with sensitive options to be logged at debug level with redactions. Previously, functions with sensitive options had to be logged at trace level to avoid exposing them. Trace level logging may still expose secrets so use with caution.
2017-10-24 18:35:36 +02:00
P00 DEBUG: Storage::Posix::Driver->new(): bFileSync = <true>, bPathSync = <true>
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->new(): bAllowTemp = <true>, hRule = [hash], lBufferMax = 4194304, oDriver = [object], strCipherPassUser = [undef], strCipherType = [undef], strDefaultFileMode = <0640>, strDefaultPathMode = <0750>, strPathBase = [TEST_PATH]/db-master/repo, strTempExtension = pgbackrest.tmp
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp =
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
Config parsing no longer implemented in Perl. Options are passed to Perl as JSON from the C process.
2018-01-29 04:37:09 +02:00
P00 DEBUG: Storage::Posix::Driver->new(): bFileSync = <true>, bPathSync = <true>
P00 DEBUG: Storage::Local->new(): bAllowTemp = <true>, hRule = [undef], lBufferMax = 4194304, oDriver = [object], strCipherPassUser = [undef], strCipherType = [undef], strDefaultFileMode = <0640>, strDefaultPathMode = <0750>, strPathBase = /, strTempExtension = pgbackrest.tmp
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = true, bIgnoreExists = true, strMode = 0770, strPathExp = [TEST_PATH]/db-master/log
Move async forking and more error handling to C. The Perl process was exiting directly when called but that interfered with proper locking for the forked async process. Now Perl returns results to the C process which handles all errors, including signals.
2018-04-13 02:42:26 +02:00
P00 DEBUG: Common::Lock::lockStopTest(): bStanzaStopRequired = <false>
P00 DEBUG: Common::Lock::lockStopTest=>: bStopExists = false
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Backup::Info->new(): bIgnoreMissing = <false>, bLoad = <true>, bRequired = <true>, bValidate = <true>, oStorage = <[object]>, strBackupClusterPath = [TEST_PATH]/db-master/repo/backup/db, strCipherPassSub = [undef]
P00 DEBUG: Storage::Local->encrypted(): bIgnoreMissing = true, strFileName = [TEST_PATH]/db-master/repo/backup/db/backup.info
P00 DEBUG: Storage::Local->encrypted=>: bEncrypted = false
P00 DEBUG: Storage::Local->encryptionValid(): bEncrypted = false
P00 DEBUG: Storage::Local->encryptionValid=>: bValid = true
P00 DEBUG: Storage::Local->openRead(): bIgnoreMissing = true, rhyFilter = [undef], strCipherPass = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/backup.info
P00 DEBUG: Storage::Base->get(): strCipherPass = [undef], xFile = [object]
Move backup modules into Backup directory.
2017-05-15 22:01:00 +02:00
P00 DEBUG: Backup::Info->reconstruct(): bRequired = <true>, bSave = <true>, iCatalogVersion = [undef], iControlVersion = [undef], strDbVersion = [undef], ullDbSysId = [undef]
P00 DEBUG: Backup::Common::backupRegExpGet(): bAnchor = <true>, bDifferential = true, bFull = true, bIncremental = true
P00 DEBUG: Backup::Common::backupRegExpGet=>: strRegExp = ^[0-9]{8}\-[0-9]{6}F(\_[0-9]{8}\-[0-9]{6}(D|I)){0,1}$
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->list(): bIgnoreMissing = <false>, strExpression = ^[0-9]{8}\-[0-9]{6}F(\_[0-9]{8}\-[0-9]{6}(D|I)){0,1}$, strPathExp = [TEST_PATH]/db-master/repo/backup/db, strSortOrder = <forward>
P00 DEBUG: Storage::Local->list=>: stryFileList = ([BACKUP-FULL-1])
Move backup modules into Backup directory.
2017-05-15 22:01:00 +02:00
P00 DEBUG: Backup::Info->current(): strBackup = [BACKUP-FULL-1]
P00 DEBUG: Backup::Info->current=>: bTest = true
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-1]
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-1]/backup.manifest
P00 DEBUG: Storage::Local->exists=>: bExists = true
Up to seven standbys can be configured for backup from standby.` Contributed by Cynthia Shang.
2017-09-01 01:15:44 +02:00
P00 DEBUG: Db::dbObjectGet(): bMasterOnly = <false>
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 DEBUG: Db->new(): iRemoteIdx = 1
Abstracted code to determine which database cluster is the master and which are standbys. Contributed by Cynthia Shang.
2016-11-30 21:15:11 +02:00
P00 DEBUG: Db::dbObjectGet=>: iDbMasterIdx = 1, iDbStandbyIdx = [undef], oDbMaster = [object], oDbStandby = [undef]
Allow functions with sensitive options to be logged at debug level with redactions. Previously, functions with sensitive options had to be logged at trace level to avoid exposing them. Trace level logging may still expose secrets so use with caution.
2017-10-24 18:35:36 +02:00
P00 DEBUG: Storage::Posix::Driver->new(): bFileSync = <true>, bPathSync = <true>
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->new(): bAllowTemp = <true>, hRule = [undef], lBufferMax = 4194304, oDriver = [object], strCipherPassUser = [undef], strCipherType = [undef], strDefaultFileMode = <0640>, strDefaultPathMode = <0750>, strPathBase = [TEST_PATH]/db-master/db/base, strTempExtension = pgbackrest.tmp
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 DEBUG: Db->info(): strDbPath = <[TEST_PATH]/db-master/db/base>
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
P00 DEBUG: Db->info=>: iDbCatalogVersion = 201409291, iDbControlVersion = 942, strDbVersion = 9.4, ullDbSysId = 1000000000000000094
P00 DEBUG: Backup::Info->check(): bRequired = <true>, iCatalogVersion = 201409291, iControlVersion = 942, strDbVersion = 9.4, ullDbSysId = 1000000000000000094
Move backup modules into Backup directory.
2017-05-15 22:01:00 +02:00
P00 DEBUG: Backup::Info->check=>: iDbHistoryId = 1
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Backup::Common::backupRegExpGet(): bAnchor = <true>, bDifferential = true, bFull = true, bIncremental = true
P00 DEBUG: Backup::Common::backupRegExpGet=>: strRegExp = ^[0-9]{8}\-[0-9]{6}F(\_[0-9]{8}\-[0-9]{6}(D|I)){0,1}$
P00 DEBUG: Storage::Local->list(): bIgnoreMissing = <false>, strExpression = ^[0-9]{8}\-[0-9]{6}F(\_[0-9]{8}\-[0-9]{6}(D|I)){0,1}$, strPathExp = <REPO:BACKUP>, strSortOrder = reverse
P00 DEBUG: Storage::Local->list=>: stryFileList = ([BACKUP-FULL-1])
P00 DEBUG: Storage::Local->exists(): strFileExp = <REPO:BACKUP>/[BACKUP-FULL-1]/backup.manifest.copy
P00 DEBUG: Storage::Local->exists=>: bExists = true
P00 DEBUG: Storage::Local->exists(): strFileExp = <REPO:BACKUP>/[BACKUP-FULL-1]/backup.manifest
P00 DEBUG: Storage::Local->exists=>: bExists = true
P00 DEBUG: Storage::Local->list(): bIgnoreMissing = <false>, strExpression = ^[BACKUP-FULL-2], strPathExp = <REPO:BACKUP>, strSortOrder = <forward>
P00 DEBUG: Storage::Local->list=>: stryFileList = ()
P00 DEBUG: Storage::Local->list(): bIgnoreMissing = true, strExpression = ^[BACKUP-FULL-2], strPathExp = <REPO:BACKUP>/backup.history/[YEAR-1], strSortOrder = <forward>
P00 DEBUG: Storage::Local->list=>: stryFileList = ()
Require PostgreSQL catalog version when instantiating a Manifest object (and not loading it from disk). Contributed by Cynthia Shang.
2018-07-16 23:25:15 +02:00
P00 DEBUG: Manifest->new(): bLoad = false, iDbCatalogVersion = 201409291, oStorage = <[object]>, strCipherPass = [undef], strCipherPassSub = [undef], strDbVersion = 9.4, strFileName = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]/backup.manifest
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/db/base/postmaster.pid
P00 DEBUG: Storage::Local->exists=>: bExists = false
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: Manifest->build(): bDelta = false, bOnline = false, bTablespace = [undef], hDatabaseMap = [undef], hTablespaceMap = [undef], iLevel = <0>, oLastManifest = [undef], oStorageDbMaster = [object], rhExclude = [hash], strFilter = [undef], strLevel = [undef], strParentPath = [undef], strPath = [TEST_PATH]/db-master/db/base, strTimelineCurrent = [undef], strTimelineLast = [undef]
Ignore all files in a linked tablespace directory except the subdirectory for the current version of PostgreSQL. Previously an error would be generated if other files were present and not owned by the PostgreSQL user. This hasn't been a big deal in practice but it could cause issues. Also add tests to make sure the same logic applies with links to files, i.e. all other files in the directory should be ignored. This was actually working correctly, but there were no tests for it before.
2018-08-31 22:06:40 +02:00
P00 DEBUG: Storage::Local->manifest(): strFilter = [undef], strPathExp = [TEST_PATH]/db-master/db/base/pg_tblspc
P00 DEBUG: Storage::Local->manifest(): strFilter = [undef], strPathExp = [TEST_PATH]/db-master/db/base
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: Manifest->build(): bDelta = false, bOnline = false, bTablespace = false, hDatabaseMap = [undef], hTablespaceMap = [hash], iLevel = 1, oLastManifest = [undef], oStorageDbMaster = [object], rhExclude = [hash], strFilter = [undef], strLevel = pg_data/pg_hba.conf, strParentPath = [TEST_PATH]/db-master/db/base, strPath = ../pg_config/pg_hba.conf, strTimelineCurrent = [undef], strTimelineLast = [undef]
Ignore all files in a linked tablespace directory except the subdirectory for the current version of PostgreSQL. Previously an error would be generated if other files were present and not owned by the PostgreSQL user. This hasn't been a big deal in practice but it could cause issues. Also add tests to make sure the same logic applies with links to files, i.e. all other files in the directory should be ignored. This was actually working correctly, but there were no tests for it before.
2018-08-31 22:06:40 +02:00
P00 DEBUG: Storage::Local->manifest(): strFilter = [undef], strPathExp = [TEST_PATH]/db-master/db/pg_config/pg_hba.conf
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
P00 INFO: exclude pg_log/logfile from backup using 'pg_log/' exclusion
P00 INFO: exclude pg_log2 from backup using 'pg_log2' exclusion
P00 INFO: exclude pg_log2/logfile from backup using 'pg_log2' exclusion
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: Manifest->build(): bDelta = false, bOnline = false, bTablespace = false, hDatabaseMap = [undef], hTablespaceMap = [hash], iLevel = 1, oLastManifest = [undef], oStorageDbMaster = [object], rhExclude = [hash], strFilter = [undef], strLevel = pg_data/pg_stat, strParentPath = [TEST_PATH]/db-master/db/base, strPath = ../pg_stat, strTimelineCurrent = [undef], strTimelineLast = [undef]
Ignore all files in a linked tablespace directory except the subdirectory for the current version of PostgreSQL. Previously an error would be generated if other files were present and not owned by the PostgreSQL user. This hasn't been a big deal in practice but it could cause issues. Also add tests to make sure the same logic applies with links to files, i.e. all other files in the directory should be ignored. This was actually working correctly, but there were no tests for it before.
2018-08-31 22:06:40 +02:00
P00 DEBUG: Storage::Local->manifest(): strFilter = [undef], strPathExp = [TEST_PATH]/db-master/db/pg_stat
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
P00 INFO: exclude postgresql.auto.conf from backup using 'postgresql.auto.conf' exclusion
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: Manifest->build(): bDelta = false, bOnline = false, bTablespace = false, hDatabaseMap = [undef], hTablespaceMap = [hash], iLevel = 1, oLastManifest = [undef], oStorageDbMaster = [object], rhExclude = [hash], strFilter = [undef], strLevel = pg_data/postgresql.conf, strParentPath = [TEST_PATH]/db-master/db/base, strPath = ../pg_config/postgresql.conf, strTimelineCurrent = [undef], strTimelineLast = [undef]
Ignore all files in a linked tablespace directory except the subdirectory for the current version of PostgreSQL. Previously an error would be generated if other files were present and not owned by the PostgreSQL user. This hasn't been a big deal in practice but it could cause issues. Also add tests to make sure the same logic applies with links to files, i.e. all other files in the directory should be ignored. This was actually working correctly, but there were no tests for it before.
2018-08-31 22:06:40 +02:00
P00 DEBUG: Storage::Local->manifest(): strFilter = [undef], strPathExp = [TEST_PATH]/db-master/db/pg_config/postgresql.conf
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: Manifest->checkDeltaFile(): lTimeBegin = [MODIFICATION-TIME-4], oPriorManifest = [undef], stryFileList = (pg_data/PG_VERSION, pg_data/base/1/12000, pg_data/base/1/PG_VERSION, pg_data/base/16384/17000, pg_data/base/16384/PG_VERSION, pg_data/base/32768/33000, pg_data/base/32768/33000.32767, pg_data/base/32768/33001, pg_data/base/32768/44000_init, pg_data/base/32768/PG_VERSION, pg_data/global/pg_control, pg_data/pg_hba.conf, pg_data/pg_stat/global.stat, pg_data/postgresql.conf, pg_data/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?%)
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Backup::Backup->process: create backup path [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = <false>, strMode = <0750>, strPathExp = <REPO:BACKUP>/[BACKUP-FULL-2]
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]/backup.manifest
P00 DEBUG: Storage::Local->exists=>: bExists = false
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->openWrite(): bAtomic = <false>, bPathCreate = <false>, lTimestamp = [undef], rhyFilter = [undef], strCipherPass = [undef], strGroup = [undef], strMode = <0640>, strUser = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]/backup.manifest.copy
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Backup::Backup->processManifest(): bCompress = false, bHardLink = false, oBackupManifest = [object], strBackupLabel = [BACKUP-FULL-2], strDbCopyPath = [TEST_PATH]/db-master/db/base, strDbMasterPath = [TEST_PATH]/db-master/db/base, strDbVersion = 9.4, strLsnStart = [undef], strType = full
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 TEST: PgBaCkReStTeSt-BACKUP-START-PgBaCkReStTeSt
Move async forking and more error handling to C. The Perl process was exiting directly when called but that interfered with proper locking for the forked async process. Now Perl returns results to the C process which handles all errors, including signals.
2018-04-13 02:42:26 +02:00
P00 DEBUG: Main::mainCleanup(): iExitCode = 63
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Helper::protocolDestroy(): bComplete = false, iRemoteIdx = [undef], strRemoteType = [undef]
P00 DEBUG: Protocol::Helper::protocolDestroy=>: iExitStatus = 0
Added a predictable log entry to signal that a command has completed successfully. For example a backup ends successfully with: INFO: backup command end: completed successfully.
2016-12-04 21:44:53 +02:00
P00 INFO: backup command end: terminated on signal [SIGTERM]
Implemented issue #129: Stop/Start Commands. Implemented issue #136: Add stderr back into protocol layer. There were also miscellaneous bug fixes and improvements to buffering in IO.pm.
2015-10-08 17:43:56 +02:00
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
full backup - global stop (db-master host)
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --no-online --type=full --stanza=db backup
Implemented issue #129: Stop/Start Commands. Implemented issue #136: Add stderr back into protocol layer. There were also miscellaneous bug fixes and improvements to buffering in IO.pm.
2015-10-08 17:43:56 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
P00 INFO: backup command begin [BACKREST-VERSION]: --no-compress --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --db-timeout=45 --exclude=postgresql.auto.conf --exclude=pg_log/ --exclude=pg_log2 --lock-path=[TEST_PATH]/db-master/lock --log-level-console=debug --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --no-online --pg1-path=[TEST_PATH]/db-master/db/base --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --stanza=db --start-fast --type=full
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: common/lock::lockAcquire: (lockPath: {"[TEST_PATH]/db-master/lock"}, stanza: {"db"}, lockType: 1, lockTimeout: 0, failOnNoLock: true)
P00 DEBUG: common/lock::lockAcquire: => true
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
P00 WARN: option repo1-retention-full is not set, the repository may run out of space
HINT: to retain full backups indefinitely (without warning), set option 'repo1-retention-full' to the maximum.
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: config/load::cfgLoad: => void
P00 DEBUG: perl/exec::perlExec: (void)
Allow functions with sensitive options to be logged at debug level with redactions. Previously, functions with sensitive options had to be logged at trace level to avoid exposing them. Trace level logging may still expose secrets so use with caution.
2017-10-24 18:35:36 +02:00
P00 DEBUG: Storage::Posix::Driver->new(): bFileSync = <true>, bPathSync = <true>
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->new(): bAllowTemp = <true>, hRule = [hash], lBufferMax = 4194304, oDriver = [object], strCipherPassUser = [undef], strCipherType = [undef], strDefaultFileMode = <0640>, strDefaultPathMode = <0750>, strPathBase = [TEST_PATH]/db-master/repo, strTempExtension = pgbackrest.tmp
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp =
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
Move async forking and more error handling to C. The Perl process was exiting directly when called but that interfered with proper locking for the forked async process. Now Perl returns results to the C process which handles all errors, including signals.
2018-04-13 02:42:26 +02:00
P00 DEBUG: Storage::Posix::Driver->new(): bFileSync = <true>, bPathSync = <true>
P00 DEBUG: Storage::Local->new(): bAllowTemp = <true>, hRule = [undef], lBufferMax = 4194304, oDriver = [object], strCipherPassUser = [undef], strCipherType = [undef], strDefaultFileMode = <0640>, strDefaultPathMode = <0750>, strPathBase = /, strTempExtension = pgbackrest.tmp
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = true, bIgnoreExists = true, strMode = 0770, strPathExp = [TEST_PATH]/db-master/log
Add stanza-delete command to cleanup unused stanzas. Contributed by Cynthia Shang.
2018-01-03 19:23:33 +02:00
P00 DEBUG: Common::Lock::lockStopTest(): bStanzaStopRequired = <false>
Renumber return codes between 25 and 125 to avoid PostgreSQL interpreting some as fatal signal exceptions. Suggested by Yogesh Sharma.
2017-04-12 18:41:34 +02:00
P00 ERROR: [062]: stop file exists for all stanzas
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: common/exit::exitSafe: (result: 0, error: true, signalType: 0)
P00 DEBUG: common/lock::lockRelease: (failOnNoLock: false)
P00 DEBUG: common/lock::lockRelease: => true
Move async forking and more error handling to C. The Perl process was exiting directly when called but that interfered with proper locking for the forked async process. Now Perl returns results to the C process which handles all errors, including signals.
2018-04-13 02:42:26 +02:00
P00 DEBUG: Main::mainCleanup(): iExitCode = 62
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Helper::protocolDestroy(): bComplete = false, iRemoteIdx = [undef], strRemoteType = [undef]
P00 DEBUG: Protocol::Helper::protocolDestroy=>: iExitStatus = 0
Renumber return codes between 25 and 125 to avoid PostgreSQL interpreting some as fatal signal exceptions. Suggested by Yogesh Sharma.
2017-04-12 18:41:34 +02:00
P00 INFO: backup command end: aborted with exception [062]
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: common/exit::exitSafe: => 62
P00 DEBUG: main::main: => 62
Implemented issue #129: Stop/Start Commands. Implemented issue #136: Add stderr back into protocol layer. There were also miscellaneous bug fixes and improvements to buffering in IO.pm.
2015-10-08 17:43:56 +02:00
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
stop db stanza (db-master host)
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --stanza=db stop
Implemented issue #129: Stop/Start Commands. Implemented issue #136: Add stderr back into protocol layer. There were also miscellaneous bug fixes and improvements to buffering in IO.pm.
2015-10-08 17:43:56 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Add log-subprocess option to allow file logging for local and remote subprocesses.
2018-08-23 02:05:49 +02:00
P00 INFO: stop command begin [BACKREST-VERSION]: --config=[TEST_PATH]/db-master/pgbackrest.conf --lock-path=[TEST_PATH]/db-master/lock --log-level-console=debug --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --repo1-path=[TEST_PATH]/db-master/repo --stanza=db
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: config/load::cfgLoad: => void
P00 DEBUG: perl/exec::perlExec: (void)
Config parsing no longer implemented in Perl. Options are passed to Perl as JSON from the C process.
2018-01-29 04:37:09 +02:00
P00 DEBUG: Storage::Posix::Driver->new(): bFileSync = <true>, bPathSync = <true>
P00 DEBUG: Storage::Local->new(): bAllowTemp = <true>, hRule = [undef], lBufferMax = [undef], oDriver = [object], strCipherPassUser = [undef], strCipherType = [undef], strDefaultFileMode = <0640>, strDefaultPathMode = <0750>, strPathBase = /, strTempExtension = pgbackrest.tmp
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = true, bIgnoreExists = true, strMode = 770, strPathExp = [TEST_PATH]/db-master/lock
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: perl/exec::perlExec: => 0
P00 DEBUG: common/exit::exitSafe: (result: 0, error: false, signalType: 0)
P00 DEBUG: common/lock::lockRelease: (failOnNoLock: false)
P00 DEBUG: common/lock::lockRelease: => false
Move async forking and more error handling to C. The Perl process was exiting directly when called but that interfered with proper locking for the forked async process. Now Perl returns results to the C process which handles all errors, including signals.
2018-04-13 02:42:26 +02:00
P00 DEBUG: Main::mainCleanup(): iExitCode = 0
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Helper::protocolDestroy(): bComplete = true, iRemoteIdx = [undef], strRemoteType = [undef]
P00 DEBUG: Protocol::Helper::protocolDestroy=>: iExitStatus = 0
Added a predictable log entry to signal that a command has completed successfully. For example a backup ends successfully with: INFO: backup command end: completed successfully.
2016-12-04 21:44:53 +02:00
P00 INFO: stop command end: completed successfully
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: common/exit::exitSafe: => 0
P00 DEBUG: main::main: => 0
Implemented issue #129: Stop/Start Commands. Implemented issue #136: Add stderr back into protocol layer. There were also miscellaneous bug fixes and improvements to buffering in IO.pm.
2015-10-08 17:43:56 +02:00
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
stop db stanza (db-master host)
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --stanza=db stop
Implemented issue #129: Stop/Start Commands. Implemented issue #136: Add stderr back into protocol layer. There were also miscellaneous bug fixes and improvements to buffering in IO.pm.
2015-10-08 17:43:56 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Add log-subprocess option to allow file logging for local and remote subprocesses.
2018-08-23 02:05:49 +02:00
P00 INFO: stop command begin [BACKREST-VERSION]: --config=[TEST_PATH]/db-master/pgbackrest.conf --lock-path=[TEST_PATH]/db-master/lock --log-level-console=debug --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --repo1-path=[TEST_PATH]/db-master/repo --stanza=db
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: config/load::cfgLoad: => void
P00 DEBUG: perl/exec::perlExec: (void)
Config parsing no longer implemented in Perl. Options are passed to Perl as JSON from the C process.
2018-01-29 04:37:09 +02:00
P00 DEBUG: Storage::Posix::Driver->new(): bFileSync = <true>, bPathSync = <true>
P00 DEBUG: Storage::Local->new(): bAllowTemp = <true>, hRule = [undef], lBufferMax = [undef], oDriver = [object], strCipherPassUser = [undef], strCipherType = [undef], strDefaultFileMode = <0640>, strDefaultPathMode = <0750>, strPathBase = /, strTempExtension = pgbackrest.tmp
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = true, bIgnoreExists = true, strMode = 770, strPathExp = [TEST_PATH]/db-master/lock
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 WARN: stop file already exists for stanza db
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: perl/exec::perlExec: => 0
P00 DEBUG: common/exit::exitSafe: (result: 0, error: false, signalType: 0)
P00 DEBUG: common/lock::lockRelease: (failOnNoLock: false)
P00 DEBUG: common/lock::lockRelease: => false
Move async forking and more error handling to C. The Perl process was exiting directly when called but that interfered with proper locking for the forked async process. Now Perl returns results to the C process which handles all errors, including signals.
2018-04-13 02:42:26 +02:00
P00 DEBUG: Main::mainCleanup(): iExitCode = 0
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Helper::protocolDestroy(): bComplete = true, iRemoteIdx = [undef], strRemoteType = [undef]
P00 DEBUG: Protocol::Helper::protocolDestroy=>: iExitStatus = 0
Added a predictable log entry to signal that a command has completed successfully. For example a backup ends successfully with: INFO: backup command end: completed successfully.
2016-12-04 21:44:53 +02:00
P00 INFO: stop command end: completed successfully
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: common/exit::exitSafe: => 0
P00 DEBUG: main::main: => 0
Implemented issue #129: Stop/Start Commands. Implemented issue #136: Add stderr back into protocol layer. There were also miscellaneous bug fixes and improvements to buffering in IO.pm.
2015-10-08 17:43:56 +02:00
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
full backup - stanza stop (db-master host)
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --no-online --type=full --stanza=db backup
Implemented issue #129: Stop/Start Commands. Implemented issue #136: Add stderr back into protocol layer. There were also miscellaneous bug fixes and improvements to buffering in IO.pm.
2015-10-08 17:43:56 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
P00 INFO: backup command begin [BACKREST-VERSION]: --no-compress --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --db-timeout=45 --exclude=postgresql.auto.conf --exclude=pg_log/ --exclude=pg_log2 --lock-path=[TEST_PATH]/db-master/lock --log-level-console=debug --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --no-online --pg1-path=[TEST_PATH]/db-master/db/base --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --stanza=db --start-fast --type=full
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: common/lock::lockAcquire: (lockPath: {"[TEST_PATH]/db-master/lock"}, stanza: {"db"}, lockType: 1, lockTimeout: 0, failOnNoLock: true)
P00 DEBUG: common/lock::lockAcquire: => true
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
P00 WARN: option repo1-retention-full is not set, the repository may run out of space
HINT: to retain full backups indefinitely (without warning), set option 'repo1-retention-full' to the maximum.
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: config/load::cfgLoad: => void
P00 DEBUG: perl/exec::perlExec: (void)
Allow functions with sensitive options to be logged at debug level with redactions. Previously, functions with sensitive options had to be logged at trace level to avoid exposing them. Trace level logging may still expose secrets so use with caution.
2017-10-24 18:35:36 +02:00
P00 DEBUG: Storage::Posix::Driver->new(): bFileSync = <true>, bPathSync = <true>
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->new(): bAllowTemp = <true>, hRule = [hash], lBufferMax = 4194304, oDriver = [object], strCipherPassUser = [undef], strCipherType = [undef], strDefaultFileMode = <0640>, strDefaultPathMode = <0750>, strPathBase = [TEST_PATH]/db-master/repo, strTempExtension = pgbackrest.tmp
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp =
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
Move async forking and more error handling to C. The Perl process was exiting directly when called but that interfered with proper locking for the forked async process. Now Perl returns results to the C process which handles all errors, including signals.
2018-04-13 02:42:26 +02:00
P00 DEBUG: Storage::Posix::Driver->new(): bFileSync = <true>, bPathSync = <true>
P00 DEBUG: Storage::Local->new(): bAllowTemp = <true>, hRule = [undef], lBufferMax = 4194304, oDriver = [object], strCipherPassUser = [undef], strCipherType = [undef], strDefaultFileMode = <0640>, strDefaultPathMode = <0750>, strPathBase = /, strTempExtension = pgbackrest.tmp
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = true, bIgnoreExists = true, strMode = 0770, strPathExp = [TEST_PATH]/db-master/log
Add stanza-delete command to cleanup unused stanzas. Contributed by Cynthia Shang.
2018-01-03 19:23:33 +02:00
P00 DEBUG: Common::Lock::lockStopTest(): bStanzaStopRequired = <false>
Renumber return codes between 25 and 125 to avoid PostgreSQL interpreting some as fatal signal exceptions. Suggested by Yogesh Sharma.
2017-04-12 18:41:34 +02:00
P00 ERROR: [062]: stop file exists for stanza db
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: common/exit::exitSafe: (result: 0, error: true, signalType: 0)
P00 DEBUG: common/lock::lockRelease: (failOnNoLock: false)
P00 DEBUG: common/lock::lockRelease: => true
Move async forking and more error handling to C. The Perl process was exiting directly when called but that interfered with proper locking for the forked async process. Now Perl returns results to the C process which handles all errors, including signals.
2018-04-13 02:42:26 +02:00
P00 DEBUG: Main::mainCleanup(): iExitCode = 62
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Helper::protocolDestroy(): bComplete = false, iRemoteIdx = [undef], strRemoteType = [undef]
P00 DEBUG: Protocol::Helper::protocolDestroy=>: iExitStatus = 0
Renumber return codes between 25 and 125 to avoid PostgreSQL interpreting some as fatal signal exceptions. Suggested by Yogesh Sharma.
2017-04-12 18:41:34 +02:00
P00 INFO: backup command end: aborted with exception [062]
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: common/exit::exitSafe: => 62
P00 DEBUG: main::main: => 62
Implemented issue #129: Stop/Start Commands. Implemented issue #136: Add stderr back into protocol layer. There were also miscellaneous bug fixes and improvements to buffering in IO.pm.
2015-10-08 17:43:56 +02:00
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
start db stanza (db-master host)
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --stanza=db start
Implemented issue #129: Stop/Start Commands. Implemented issue #136: Add stderr back into protocol layer. There were also miscellaneous bug fixes and improvements to buffering in IO.pm.
2015-10-08 17:43:56 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Add log-subprocess option to allow file logging for local and remote subprocesses.
2018-08-23 02:05:49 +02:00
P00 INFO: start command begin [BACKREST-VERSION]: --config=[TEST_PATH]/db-master/pgbackrest.conf --lock-path=[TEST_PATH]/db-master/lock --log-level-console=debug --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --repo1-path=[TEST_PATH]/db-master/repo --stanza=db
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: config/load::cfgLoad: => void
P00 DEBUG: perl/exec::perlExec: (void)
P00 DEBUG: perl/exec::perlExec: => 0
P00 DEBUG: common/exit::exitSafe: (result: 0, error: false, signalType: 0)
P00 DEBUG: common/lock::lockRelease: (failOnNoLock: false)
P00 DEBUG: common/lock::lockRelease: => false
Move async forking and more error handling to C. The Perl process was exiting directly when called but that interfered with proper locking for the forked async process. Now Perl returns results to the C process which handles all errors, including signals.
2018-04-13 02:42:26 +02:00
P00 DEBUG: Main::mainCleanup(): iExitCode = 0
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Helper::protocolDestroy(): bComplete = true, iRemoteIdx = [undef], strRemoteType = [undef]
P00 DEBUG: Protocol::Helper::protocolDestroy=>: iExitStatus = 0
Added a predictable log entry to signal that a command has completed successfully. For example a backup ends successfully with: INFO: backup command end: completed successfully.
2016-12-04 21:44:53 +02:00
P00 INFO: start command end: completed successfully
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: common/exit::exitSafe: => 0
P00 DEBUG: main::main: => 0
Implemented issue #129: Stop/Start Commands. Implemented issue #136: Add stderr back into protocol layer. There were also miscellaneous bug fixes and improvements to buffering in IO.pm.
2015-10-08 17:43:56 +02:00
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
start all stanzas (db-master host)
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf start
Implemented issue #129: Stop/Start Commands. Implemented issue #136: Add stderr back into protocol layer. There were also miscellaneous bug fixes and improvements to buffering in IO.pm.
2015-10-08 17:43:56 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Add log-subprocess option to allow file logging for local and remote subprocesses.
2018-08-23 02:05:49 +02:00
P00 INFO: start command begin [BACKREST-VERSION]: --config=[TEST_PATH]/db-master/pgbackrest.conf --lock-path=[TEST_PATH]/db-master/lock --log-level-console=debug --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --repo1-path=[TEST_PATH]/db-master/repo
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: config/load::cfgLoad: => void
P00 DEBUG: perl/exec::perlExec: (void)
P00 DEBUG: perl/exec::perlExec: => 0
P00 DEBUG: common/exit::exitSafe: (result: 0, error: false, signalType: 0)
P00 DEBUG: common/lock::lockRelease: (failOnNoLock: false)
P00 DEBUG: common/lock::lockRelease: => false
Move async forking and more error handling to C. The Perl process was exiting directly when called but that interfered with proper locking for the forked async process. Now Perl returns results to the C process which handles all errors, including signals.
2018-04-13 02:42:26 +02:00
P00 DEBUG: Main::mainCleanup(): iExitCode = 0
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Helper::protocolDestroy(): bComplete = true, iRemoteIdx = [undef], strRemoteType = [undef]
P00 DEBUG: Protocol::Helper::protocolDestroy=>: iExitStatus = 0
Added a predictable log entry to signal that a command has completed successfully. For example a backup ends successfully with: INFO: backup command end: completed successfully.
2016-12-04 21:44:53 +02:00
P00 INFO: start command end: completed successfully
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: common/exit::exitSafe: => 0
P00 DEBUG: main::main: => 0
Implemented issue #129: Stop/Start Commands. Implemented issue #136: Add stderr back into protocol layer. There were also miscellaneous bug fixes and improvements to buffering in IO.pm.
2015-10-08 17:43:56 +02:00
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
start all stanzas (db-master host)
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf start
Implemented issue #129: Stop/Start Commands. Implemented issue #136: Add stderr back into protocol layer. There were also miscellaneous bug fixes and improvements to buffering in IO.pm.
2015-10-08 17:43:56 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Add log-subprocess option to allow file logging for local and remote subprocesses.
2018-08-23 02:05:49 +02:00
P00 INFO: start command begin [BACKREST-VERSION]: --config=[TEST_PATH]/db-master/pgbackrest.conf --lock-path=[TEST_PATH]/db-master/lock --log-level-console=debug --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --repo1-path=[TEST_PATH]/db-master/repo
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: config/load::cfgLoad: => void
P00 DEBUG: perl/exec::perlExec: (void)
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 WARN: stop file does not exist
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: perl/exec::perlExec: => 0
P00 DEBUG: common/exit::exitSafe: (result: 0, error: false, signalType: 0)
P00 DEBUG: common/lock::lockRelease: (failOnNoLock: false)
P00 DEBUG: common/lock::lockRelease: => false
Move async forking and more error handling to C. The Perl process was exiting directly when called but that interfered with proper locking for the forked async process. Now Perl returns results to the C process which handles all errors, including signals.
2018-04-13 02:42:26 +02:00
P00 DEBUG: Main::mainCleanup(): iExitCode = 0
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Helper::protocolDestroy(): bComplete = true, iRemoteIdx = [undef], strRemoteType = [undef]
P00 DEBUG: Protocol::Helper::protocolDestroy=>: iExitStatus = 0
Added a predictable log entry to signal that a command has completed successfully. For example a backup ends successfully with: INFO: backup command end: completed successfully.
2016-12-04 21:44:53 +02:00
P00 INFO: start command end: completed successfully
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: common/exit::exitSafe: => 0
P00 DEBUG: main::main: => 0
Implemented issue #129: Stop/Start Commands. Implemented issue #136: Add stderr back into protocol layer. There were also miscellaneous bug fixes and improvements to buffering in IO.pm.
2015-10-08 17:43:56 +02:00
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
full backup - resume (db-master host)
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --no-online --force --checksum-page --delta --type=full --stanza=db backup --test --test-delay=0.2 --test-point=backup-resume=y
Log testing can now be enabled for certain deterministic tests. This works by comparing the generated logs against a previous copy. Currently only enabled for the backup/synthetic tests.
2015-05-05 19:08:48 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P00 INFO: backup command begin [BACKREST-VERSION]: --checksum-page --no-compress --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --db-timeout=45 --delta --exclude=postgresql.auto.conf --exclude=pg_log/ --exclude=pg_log2 --force --lock-path=[TEST_PATH]/db-master/lock --log-level-console=debug --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --no-online --pg1-path=[TEST_PATH]/db-master/db/base --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --stanza=db --start-fast --test --test-delay=0.2 --test-point=backup-resume=y --type=full
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: common/lock::lockAcquire: (lockPath: {"[TEST_PATH]/db-master/lock"}, stanza: {"db"}, lockType: 1, lockTimeout: 0, failOnNoLock: true)
P00 DEBUG: common/lock::lockAcquire: => true
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
P00 WARN: option repo1-retention-full is not set, the repository may run out of space
HINT: to retain full backups indefinitely (without warning), set option 'repo1-retention-full' to the maximum.
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: config/load::cfgLoad: => void
P00 DEBUG: perl/exec::perlExec: (void)
Allow functions with sensitive options to be logged at debug level with redactions. Previously, functions with sensitive options had to be logged at trace level to avoid exposing them. Trace level logging may still expose secrets so use with caution.
2017-10-24 18:35:36 +02:00
P00 DEBUG: Storage::Posix::Driver->new(): bFileSync = <true>, bPathSync = <true>
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->new(): bAllowTemp = <true>, hRule = [hash], lBufferMax = 4194304, oDriver = [object], strCipherPassUser = [undef], strCipherType = [undef], strDefaultFileMode = <0640>, strDefaultPathMode = <0750>, strPathBase = [TEST_PATH]/db-master/repo, strTempExtension = pgbackrest.tmp
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp =
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
Config parsing no longer implemented in Perl. Options are passed to Perl as JSON from the C process.
2018-01-29 04:37:09 +02:00
P00 DEBUG: Storage::Posix::Driver->new(): bFileSync = <true>, bPathSync = <true>
P00 DEBUG: Storage::Local->new(): bAllowTemp = <true>, hRule = [undef], lBufferMax = 4194304, oDriver = [object], strCipherPassUser = [undef], strCipherType = [undef], strDefaultFileMode = <0640>, strDefaultPathMode = <0750>, strPathBase = /, strTempExtension = pgbackrest.tmp
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = true, bIgnoreExists = true, strMode = 0770, strPathExp = [TEST_PATH]/db-master/log
Move async forking and more error handling to C. The Perl process was exiting directly when called but that interfered with proper locking for the forked async process. Now Perl returns results to the C process which handles all errors, including signals.
2018-04-13 02:42:26 +02:00
P00 DEBUG: Common::Lock::lockStopTest(): bStanzaStopRequired = <false>
P00 DEBUG: Common::Lock::lockStopTest=>: bStopExists = false
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Backup::Info->new(): bIgnoreMissing = <false>, bLoad = <true>, bRequired = <true>, bValidate = <true>, oStorage = <[object]>, strBackupClusterPath = [TEST_PATH]/db-master/repo/backup/db, strCipherPassSub = [undef]
P00 DEBUG: Storage::Local->encrypted(): bIgnoreMissing = true, strFileName = [TEST_PATH]/db-master/repo/backup/db/backup.info
P00 DEBUG: Storage::Local->encrypted=>: bEncrypted = false
P00 DEBUG: Storage::Local->encryptionValid(): bEncrypted = false
P00 DEBUG: Storage::Local->encryptionValid=>: bValid = true
P00 DEBUG: Storage::Local->openRead(): bIgnoreMissing = true, rhyFilter = [undef], strCipherPass = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/backup.info
P00 DEBUG: Storage::Base->get(): strCipherPass = [undef], xFile = [object]
Move backup modules into Backup directory.
2017-05-15 22:01:00 +02:00
P00 DEBUG: Backup::Info->reconstruct(): bRequired = <true>, bSave = <true>, iCatalogVersion = [undef], iControlVersion = [undef], strDbVersion = [undef], ullDbSysId = [undef]
P00 DEBUG: Backup::Common::backupRegExpGet(): bAnchor = <true>, bDifferential = true, bFull = true, bIncremental = true
P00 DEBUG: Backup::Common::backupRegExpGet=>: strRegExp = ^[0-9]{8}\-[0-9]{6}F(\_[0-9]{8}\-[0-9]{6}(D|I)){0,1}$
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->list(): bIgnoreMissing = <false>, strExpression = ^[0-9]{8}\-[0-9]{6}F(\_[0-9]{8}\-[0-9]{6}(D|I)){0,1}$, strPathExp = [TEST_PATH]/db-master/repo/backup/db, strSortOrder = <forward>
P00 DEBUG: Storage::Local->list=>: stryFileList = ([BACKUP-FULL-2])
P00 DEBUG: Backup::Info->current(): strBackup = [BACKUP-FULL-2]
P00 DEBUG: Backup::Info->current=>: bTest = false
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]/backup.manifest
P00 DEBUG: Storage::Local->exists=>: bExists = false
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-1]
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 WARN: backup [BACKUP-FULL-1] missing in repository removed from backup.info
Move backup modules into Backup directory.
2017-05-15 22:01:00 +02:00
P00 DEBUG: Backup::Info->delete(): strBackupLabel = [BACKUP-FULL-1]
Up to seven standbys can be configured for backup from standby.` Contributed by Cynthia Shang.
2017-09-01 01:15:44 +02:00
P00 DEBUG: Db::dbObjectGet(): bMasterOnly = <false>
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 DEBUG: Db->new(): iRemoteIdx = 1
Abstracted code to determine which database cluster is the master and which are standbys. Contributed by Cynthia Shang.
2016-11-30 21:15:11 +02:00
P00 DEBUG: Db::dbObjectGet=>: iDbMasterIdx = 1, iDbStandbyIdx = [undef], oDbMaster = [object], oDbStandby = [undef]
Allow functions with sensitive options to be logged at debug level with redactions. Previously, functions with sensitive options had to be logged at trace level to avoid exposing them. Trace level logging may still expose secrets so use with caution.
2017-10-24 18:35:36 +02:00
P00 DEBUG: Storage::Posix::Driver->new(): bFileSync = <true>, bPathSync = <true>
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->new(): bAllowTemp = <true>, hRule = [undef], lBufferMax = 4194304, oDriver = [object], strCipherPassUser = [undef], strCipherType = [undef], strDefaultFileMode = <0640>, strDefaultPathMode = <0750>, strPathBase = [TEST_PATH]/db-master/db/base, strTempExtension = pgbackrest.tmp
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 DEBUG: Db->info(): strDbPath = <[TEST_PATH]/db-master/db/base>
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
P00 DEBUG: Db->info=>: iDbCatalogVersion = 201409291, iDbControlVersion = 942, strDbVersion = 9.4, ullDbSysId = 1000000000000000094
P00 DEBUG: Backup::Info->check(): bRequired = <true>, iCatalogVersion = 201409291, iControlVersion = 942, strDbVersion = 9.4, ullDbSysId = 1000000000000000094
Move backup modules into Backup directory.
2017-05-15 22:01:00 +02:00
P00 DEBUG: Backup::Info->check=>: iDbHistoryId = 1
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Backup::Common::backupRegExpGet(): bAnchor = <true>, bDifferential = true, bFull = true, bIncremental = true
P00 DEBUG: Backup::Common::backupRegExpGet=>: strRegExp = ^[0-9]{8}\-[0-9]{6}F(\_[0-9]{8}\-[0-9]{6}(D|I)){0,1}$
P00 DEBUG: Storage::Local->list(): bIgnoreMissing = <false>, strExpression = ^[0-9]{8}\-[0-9]{6}F(\_[0-9]{8}\-[0-9]{6}(D|I)){0,1}$, strPathExp = <REPO:BACKUP>, strSortOrder = reverse
P00 DEBUG: Storage::Local->list=>: stryFileList = ([BACKUP-FULL-2])
P00 DEBUG: Storage::Local->exists(): strFileExp = <REPO:BACKUP>/[BACKUP-FULL-2]/backup.manifest.copy
P00 DEBUG: Storage::Local->exists=>: bExists = true
P00 DEBUG: Storage::Local->exists(): strFileExp = <REPO:BACKUP>/[BACKUP-FULL-2]/backup.manifest
P00 DEBUG: Storage::Local->exists=>: bExists = false
Require PostgreSQL catalog version when instantiating a Manifest object (and not loading it from disk). Contributed by Cynthia Shang.
2018-07-16 23:25:15 +02:00
P00 DEBUG: Manifest->new(): bLoad = <true>, iDbCatalogVersion = [undef], oStorage = <[object]>, strCipherPass = [undef], strCipherPassSub = [undef], strDbVersion = [undef], strFileName = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]/backup.manifest
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->encrypted(): bIgnoreMissing = true, strFileName = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]/backup.manifest
P00 DEBUG: Storage::Local->encrypted=>: bEncrypted = false
P00 DEBUG: Storage::Local->encryptionValid(): bEncrypted = false
P00 DEBUG: Storage::Local->encryptionValid=>: bValid = true
P00 DEBUG: Storage::Local->openRead(): bIgnoreMissing = true, rhyFilter = [undef], strCipherPass = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]/backup.manifest
P00 DEBUG: Storage::Base->get(): strCipherPass = [undef], xFile = [undef]
P00 DEBUG: Storage::Local->encrypted(): bIgnoreMissing = true, strFileName = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]/backup.manifest.copy
P00 DEBUG: Storage::Local->encrypted=>: bEncrypted = false
P00 DEBUG: Storage::Local->encryptionValid(): bEncrypted = false
P00 DEBUG: Storage::Local->encryptionValid=>: bValid = true
P00 DEBUG: Storage::Local->openRead(): bIgnoreMissing = true, rhyFilter = [undef], strCipherPass = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]/backup.manifest.copy
P00 DEBUG: Storage::Base->get(): strCipherPass = [undef], xFile = [object]
Require PostgreSQL catalog version when instantiating a Manifest object (and not loading it from disk). Contributed by Cynthia Shang.
2018-07-16 23:25:15 +02:00
P00 DEBUG: Manifest->new(): bLoad = false, iDbCatalogVersion = 201409291, oStorage = <[object]>, strCipherPass = [undef], strCipherPassSub = [undef], strDbVersion = 9.4, strFileName = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]/backup.manifest
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/db/base/postmaster.pid
P00 DEBUG: Storage::Local->exists=>: bExists = true
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 WARN: --no-online passed and postmaster.pid exists but --force was passed so backup will continue though it looks like the postmaster is running and the backup will probably not be consistent
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: Manifest->build(): bDelta = true, bOnline = false, bTablespace = [undef], hDatabaseMap = [undef], hTablespaceMap = [undef], iLevel = <0>, oLastManifest = [undef], oStorageDbMaster = [object], rhExclude = [hash], strFilter = [undef], strLevel = [undef], strParentPath = [undef], strPath = [TEST_PATH]/db-master/db/base, strTimelineCurrent = [undef], strTimelineLast = [undef]
Ignore all files in a linked tablespace directory except the subdirectory for the current version of PostgreSQL. Previously an error would be generated if other files were present and not owned by the PostgreSQL user. This hasn't been a big deal in practice but it could cause issues. Also add tests to make sure the same logic applies with links to files, i.e. all other files in the directory should be ignored. This was actually working correctly, but there were no tests for it before.
2018-08-31 22:06:40 +02:00
P00 DEBUG: Storage::Local->manifest(): strFilter = [undef], strPathExp = [TEST_PATH]/db-master/db/base/pg_tblspc
P00 DEBUG: Storage::Local->manifest(): strFilter = [undef], strPathExp = [TEST_PATH]/db-master/db/base
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: Manifest->build(): bDelta = true, bOnline = false, bTablespace = false, hDatabaseMap = [undef], hTablespaceMap = [hash], iLevel = 1, oLastManifest = [undef], oStorageDbMaster = [object], rhExclude = [hash], strFilter = [undef], strLevel = pg_data/pg_hba.conf, strParentPath = [TEST_PATH]/db-master/db/base, strPath = ../pg_config/pg_hba.conf, strTimelineCurrent = [undef], strTimelineLast = [undef]
Ignore all files in a linked tablespace directory except the subdirectory for the current version of PostgreSQL. Previously an error would be generated if other files were present and not owned by the PostgreSQL user. This hasn't been a big deal in practice but it could cause issues. Also add tests to make sure the same logic applies with links to files, i.e. all other files in the directory should be ignored. This was actually working correctly, but there were no tests for it before.
2018-08-31 22:06:40 +02:00
P00 DEBUG: Storage::Local->manifest(): strFilter = [undef], strPathExp = [TEST_PATH]/db-master/db/pg_config/pg_hba.conf
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
P00 INFO: exclude pg_log/logfile from backup using 'pg_log/' exclusion
P00 INFO: exclude pg_log2 from backup using 'pg_log2' exclusion
P00 INFO: exclude pg_log2/logfile from backup using 'pg_log2' exclusion
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: Manifest->build(): bDelta = true, bOnline = false, bTablespace = false, hDatabaseMap = [undef], hTablespaceMap = [hash], iLevel = 1, oLastManifest = [undef], oStorageDbMaster = [object], rhExclude = [hash], strFilter = [undef], strLevel = pg_data/pg_stat, strParentPath = [TEST_PATH]/db-master/db/base, strPath = ../pg_stat, strTimelineCurrent = [undef], strTimelineLast = [undef]
Ignore all files in a linked tablespace directory except the subdirectory for the current version of PostgreSQL. Previously an error would be generated if other files were present and not owned by the PostgreSQL user. This hasn't been a big deal in practice but it could cause issues. Also add tests to make sure the same logic applies with links to files, i.e. all other files in the directory should be ignored. This was actually working correctly, but there were no tests for it before.
2018-08-31 22:06:40 +02:00
P00 DEBUG: Storage::Local->manifest(): strFilter = [undef], strPathExp = [TEST_PATH]/db-master/db/pg_stat
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
P00 INFO: exclude postgresql.auto.conf from backup using 'postgresql.auto.conf' exclusion
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: Manifest->build(): bDelta = true, bOnline = false, bTablespace = false, hDatabaseMap = [undef], hTablespaceMap = [hash], iLevel = 1, oLastManifest = [undef], oStorageDbMaster = [object], rhExclude = [hash], strFilter = [undef], strLevel = pg_data/postgresql.conf, strParentPath = [TEST_PATH]/db-master/db/base, strPath = ../pg_config/postgresql.conf, strTimelineCurrent = [undef], strTimelineLast = [undef]
Ignore all files in a linked tablespace directory except the subdirectory for the current version of PostgreSQL. Previously an error would be generated if other files were present and not owned by the PostgreSQL user. This hasn't been a big deal in practice but it could cause issues. Also add tests to make sure the same logic applies with links to files, i.e. all other files in the directory should be ignored. This was actually working correctly, but there were no tests for it before.
2018-08-31 22:06:40 +02:00
P00 DEBUG: Storage::Local->manifest(): strFilter = [undef], strPathExp = [TEST_PATH]/db-master/db/pg_config/postgresql.conf
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 WARN: aborted backup [BACKUP-FULL-2] of same type exists, will be cleaned to remove invalid files and resumed
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 TEST: PgBaCkReStTeSt-BACKUP-RESUME-PgBaCkReStTeSt
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: Backup::Backup->resumeClean(): bDelta = true, bOnline = false, oAbortedManifest = [object], oManifest = [object], oStorageRepo = [object], strBackupLabel = [BACKUP-FULL-2], strTimelineCurrent = [undef], strTimelineLast = [undef]
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DETAIL: clean resumed backup path: [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]
Ignore all files in a linked tablespace directory except the subdirectory for the current version of PostgreSQL. Previously an error would be generated if other files were present and not owned by the PostgreSQL user. This hasn't been a big deal in practice but it could cause issues. Also add tests to make sure the same logic applies with links to files, i.e. all other files in the directory should be ignored. This was actually working correctly, but there were no tests for it before.
2018-08-31 22:06:40 +02:00
P00 DEBUG: Storage::Local->manifest(): strFilter = [undef], strPathExp = <REPO:BACKUP>/[BACKUP-FULL-2]
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Backup::Backup->resumeClean: remove file file.tmp
P00 DEBUG: Backup::Backup->resumeClean: remove file pg_data/PG_VERSION
Improve support for special characters in filenames. % characters caused issues in backup/restore due to filenames being appended directly into a format string. Reserved XML characters (<>&') caused issues in the S3 driver due to improper escaping. Add a file with all common special characters to regression testing.
2018-09-10 16:54:34 +02:00
P00 DEBUG: Backup::Backup->resumeClean: remove file pg_data/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?%
P00 DEBUG: Storage::Local->remove(): bIgnoreMissing = <true>, bRecurse = <false>, xstryPathFileExp = (<REPO:BACKUP>/[BACKUP-FULL-2]/file.tmp, <REPO:BACKUP>/[BACKUP-FULL-2]/pg_data/PG_VERSION, <REPO:BACKUP>/[BACKUP-FULL-2]/pg_data/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?%)
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->remove=>: bRemoved = true
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]/backup.manifest
P00 DEBUG: Storage::Local->exists=>: bExists = false
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->openWrite(): bAtomic = <false>, bPathCreate = <false>, lTimestamp = [undef], rhyFilter = [undef], strCipherPass = [undef], strGroup = [undef], strMode = <0640>, strUser = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]/backup.manifest.copy
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Backup::Backup->processManifest(): bCompress = false, bHardLink = false, oBackupManifest = [object], strBackupLabel = [BACKUP-FULL-2], strDbCopyPath = [TEST_PATH]/db-master/db/base, strDbMasterPath = [TEST_PATH]/db-master/db/base, strDbVersion = 9.4, strLsnStart = [undef], strType = full
P00 DEBUG: Protocol::Local::Process->new(): bConfessError = <true>, iSelectTimeout = <30>, strBackRestBin = <[BACKREST-BIN]>, strHostType = db
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->hostAdd(): iHostConfigIdx = 1, iProcessMax = 1
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = true, strMode = <0750>, strPathExp = <REPO:BACKUP>/[BACKUP-FULL-2]/pg_data
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = true, strMode = <0750>, strPathExp = <REPO:BACKUP>/[BACKUP-FULL-2]/pg_data/base
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = true, strMode = <0750>, strPathExp = <REPO:BACKUP>/[BACKUP-FULL-2]/pg_data/base/1
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = true, strMode = <0750>, strPathExp = <REPO:BACKUP>/[BACKUP-FULL-2]/pg_data/base/16384
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = true, strMode = <0750>, strPathExp = <REPO:BACKUP>/[BACKUP-FULL-2]/pg_data/base/32768
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = true, strMode = <0750>, strPathExp = <REPO:BACKUP>/[BACKUP-FULL-2]/pg_data/global
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = true, strMode = <0750>, strPathExp = <REPO:BACKUP>/[BACKUP-FULL-2]/pg_data/pg_clog
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = true, strMode = <0750>, strPathExp = <REPO:BACKUP>/[BACKUP-FULL-2]/pg_data/pg_dynshmem
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = true, strMode = <0750>, strPathExp = <REPO:BACKUP>/[BACKUP-FULL-2]/pg_data/pg_log
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = true, strMode = <0750>, strPathExp = <REPO:BACKUP>/[BACKUP-FULL-2]/pg_data/pg_notify
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = true, strMode = <0750>, strPathExp = <REPO:BACKUP>/[BACKUP-FULL-2]/pg_data/pg_replslot
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = true, strMode = <0750>, strPathExp = <REPO:BACKUP>/[BACKUP-FULL-2]/pg_data/pg_serial
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = true, strMode = <0750>, strPathExp = <REPO:BACKUP>/[BACKUP-FULL-2]/pg_data/pg_snapshots
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = true, strMode = <0750>, strPathExp = <REPO:BACKUP>/[BACKUP-FULL-2]/pg_data/pg_stat
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = true, strMode = <0750>, strPathExp = <REPO:BACKUP>/[BACKUP-FULL-2]/pg_data/pg_stat_tmp
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = true, strMode = <0750>, strPathExp = <REPO:BACKUP>/[BACKUP-FULL-2]/pg_data/pg_subtrans
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = true, strMode = <0750>, strPathExp = <REPO:BACKUP>/[BACKUP-FULL-2]/pg_data/pg_tblspc
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/base/32768/33001, pg_data/base/32768/33001, 65536, 6bf316f11d28c28914ea9be92c00de9bea6d9a6b, 1, [BACKUP-FULL-2], 0, 3, [MODIFICATION-TIME-2], 1, {iWalId => 65535, iWalOffset => 65535}, 1, 0), rParamSecure = [undef], strKey = pg_data/base/32768/33001, strOp = backupFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/base/32768/44000_init, pg_data/base/32768/44000_init, 32768, 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f, 1, [BACKUP-FULL-2], 0, 3, [MODIFICATION-TIME-2], 1, {iWalId => 65535, iWalOffset => 65535}, 1, 0), rParamSecure = [undef], strKey = pg_data/base/32768/44000_init, strOp = backupFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/base/32768/33000.32767, pg_data/base/32768/33000.32767, 32768, 21e2c7c1a326682c07053b7d6a5a40dbd49c2ec5, 1, [BACKUP-FULL-2], 0, 3, [MODIFICATION-TIME-2], 1, {iWalId => 65535, iWalOffset => 65535}, 1, 0), rParamSecure = [undef], strKey = pg_data/base/32768/33000.32767, strOp = backupFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/base/32768/33000, pg_data/base/32768/33000, 32768, 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f, 1, [BACKUP-FULL-2], 0, 3, [MODIFICATION-TIME-2], 1, {iWalId => 65535, iWalOffset => 65535}, 1, 0), rParamSecure = [undef], strKey = pg_data/base/32768/33000, strOp = backupFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/base/16384/17000, pg_data/base/16384/17000, 16384, e0101dd8ffb910c9c202ca35b5f828bcb9697bed, 1, [BACKUP-FULL-2], 0, 3, [MODIFICATION-TIME-2], 1, {iWalId => 65535, iWalOffset => 65535}, 1, 0), rParamSecure = [undef], strKey = pg_data/base/16384/17000, strOp = backupFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/global/pg_control, pg_data/global/pg_control, 8192, 4c77c900f7af0d9ab13fa9982051a42e0b637f6c, 0, [BACKUP-FULL-2], 0, 3, [MODIFICATION-TIME-3], 0, [undef], 1, 0), rParamSecure = [undef], strKey = pg_data/global/pg_control, strOp = backupFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/base/1/12000, pg_data/base/1/12000, 8192, 22c98d248ff548311eda88559e4a8405ed77c003, 1, [BACKUP-FULL-2], 0, 3, [MODIFICATION-TIME-2], 1, {iWalId => 65535, iWalOffset => 65535}, 1, 0), rParamSecure = [undef], strKey = pg_data/base/1/12000, strOp = backupFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/postgresql.conf, pg_data/postgresql.conf, 21, 6721d92c9fcdf4248acff1f9a1377127d9064807, 0, [BACKUP-FULL-2], 0, 3, [MODIFICATION-TIME-3], 1, [undef], 1, 0), rParamSecure = [undef], strKey = pg_data/postgresql.conf, strOp = backupFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/pg_hba.conf, pg_data/pg_hba.conf, 9, dd4cea0cae348309f9de28ad4ded8ee2cc2e6d5b, 0, [BACKUP-FULL-2], 0, 3, [MODIFICATION-TIME-3], 1, [undef], 1, 0), rParamSecure = [undef], strKey = pg_data/pg_hba.conf, strOp = backupFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/changecontent.txt, pg_data/changecontent.txt, 7, [undef], 0, [BACKUP-FULL-2], 0, 3, [MODIFICATION-TIME-2], 1, [undef], 1, 0), rParamSecure = [undef], strKey = pg_data/changecontent.txt, strOp = backupFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/pg_stat/global.stat, pg_data/pg_stat/global.stat, 5, e350d5ce0153f3e22d5db21cf2a4eff00f3ee877, 0, [BACKUP-FULL-2], 0, 3, [MODIFICATION-TIME-3], 1, [undef], 1, 0), rParamSecure = [undef], strKey = pg_data/pg_stat/global.stat, strOp = backupFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/changetime.txt, pg_data/changetime.txt, 4, [undef], 0, [BACKUP-FULL-2], 0, 3, [MODIFICATION-TIME-2], 1, [undef], 1, 0), rParamSecure = [undef], strKey = pg_data/changetime.txt, strOp = backupFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/base/32768/PG_VERSION, pg_data/base/32768/PG_VERSION, 3, 184473f470864e067ee3a22e64b47b0a1c356f29, 0, [BACKUP-FULL-2], 0, 3, [MODIFICATION-TIME-2], 1, [undef], 1, 0), rParamSecure = [undef], strKey = pg_data/base/32768/PG_VERSION, strOp = backupFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/base/16384/PG_VERSION, pg_data/base/16384/PG_VERSION, 3, 184473f470864e067ee3a22e64b47b0a1c356f29, 0, [BACKUP-FULL-2], 0, 3, [MODIFICATION-TIME-2], 1, [undef], 1, 0), rParamSecure = [undef], strKey = pg_data/base/16384/PG_VERSION, strOp = backupFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/base/1/PG_VERSION, pg_data/base/1/PG_VERSION, 3, 184473f470864e067ee3a22e64b47b0a1c356f29, 0, [BACKUP-FULL-2], 0, 3, [MODIFICATION-TIME-2], 1, [undef], 1, 0), rParamSecure = [undef], strKey = pg_data/base/1/PG_VERSION, strOp = backupFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/PG_VERSION, pg_data/PG_VERSION, 3, [undef], 0, [BACKUP-FULL-2], 0, 3, [MODIFICATION-TIME-2], 1, [undef], 1, 0), rParamSecure = [undef], strKey = pg_data/PG_VERSION, strOp = backupFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/zero_from_start, pg_data/zero_from_start, 0, [undef], 0, [BACKUP-FULL-2], 0, 3, [MODIFICATION-TIME-2], 1, [undef], 1, 0), rParamSecure = [undef], strKey = pg_data/zero_from_start, strOp = backupFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?%, pg_data/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?%, 0, [undef], 0, [BACKUP-FULL-2], 0, 3, [MODIFICATION-TIME-2], 1, [undef], 1, 0), rParamSecure = [undef], strKey = pg_data/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?%, strOp = backupFile, strQueue = pg_data
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->hostConnect: start local process: iHostConfigIdx = 1, iHostIdx = 0, iHostProcessIdx = 0, iProcessId = 1, strHostType = db
Add log-subprocess option to allow file logging for local and remote subprocesses.
2018-08-23 02:05:49 +02:00
P00 DEBUG: Protocol::Local::Master->new(): iProcessIdx = 1, strCommand = [BACKREST-BIN] --command=backup --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --db-timeout=45 --host-id=1 --lock-path=[TEST_PATH]/db-master/lock --log-level-file=trace --log-path=[TEST_PATH]/db-master/log --log-subprocess --pg1-path=[TEST_PATH]/db-master/db/base --process=1 --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --stanza=db --type=db local
P00 DEBUG: Protocol::Command::Master->new(): iBufferMax = 4194304, iCompressLevel = 3, iCompressLevelNetwork = 3, iProtocolTimeout = 60, strCommand = [BACKREST-BIN] --command=backup --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --db-timeout=45 --host-id=1 --lock-path=[TEST_PATH]/db-master/lock --log-level-file=trace --log-path=[TEST_PATH]/db-master/log --log-subprocess --pg1-path=[TEST_PATH]/db-master/db/base --process=1 --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --stanza=db --type=db local, strId = local-1 process, strName = local
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->hostConnect=>: bResult = true
P00 DEBUG: Protocol::Local::Process->init: init local process: iDirection = 1, iHostIdx = 0, iProcessId = 1, iQueueIdx = 0, iQueueLastIdx = 0
P00 DEBUG: Protocol::Local::Process->init=>: bResult = true
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/base/32768/33001, strQueueIdx = 0
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (0, 65536, 65536, 6bf316f11d28c28914ea9be92c00de9bea6d9a6b, [undef]), strKey = pg_data/base/32768/33001
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/base/32768/44000_init, strQueueIdx = 0
P01 DETAIL: checksum resumed file [TEST_PATH]/db-master/db/base/base/32768/33001 (64KB, 33%) checksum 6bf316f11d28c28914ea9be92c00de9bea6d9a6b
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (0, 32768, 32768, 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f, [undef]), strKey = pg_data/base/32768/44000_init
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/base/32768/33000.32767, strQueueIdx = 0
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
P01 DETAIL: checksum resumed file [TEST_PATH]/db-master/db/base/base/32768/44000_init (32KB, 49%) checksum 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (0, 32768, 32768, 21e2c7c1a326682c07053b7d6a5a40dbd49c2ec5, [undef]), strKey = pg_data/base/32768/33000.32767
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/base/32768/33000, strQueueIdx = 0
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
P01 DETAIL: checksum resumed file [TEST_PATH]/db-master/db/base/base/32768/33000.32767 (32KB, 66%) checksum 21e2c7c1a326682c07053b7d6a5a40dbd49c2ec5
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (0, 32768, 32768, 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f, [undef]), strKey = pg_data/base/32768/33000
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/base/16384/17000, strQueueIdx = 0
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
P01 DETAIL: checksum resumed file [TEST_PATH]/db-master/db/base/base/32768/33000 (32KB, 83%) checksum 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (0, 16384, 16384, e0101dd8ffb910c9c202ca35b5f828bcb9697bed, [undef]), strKey = pg_data/base/16384/17000
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/global/pg_control, strQueueIdx = 0
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
P01 DETAIL: checksum resumed file [TEST_PATH]/db-master/db/base/base/16384/17000 (16KB, 91%) checksum e0101dd8ffb910c9c202ca35b5f828bcb9697bed
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 11:24:42 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (0, 8192, 8192, 4c77c900f7af0d9ab13fa9982051a42e0b637f6c, [undef]), strKey = pg_data/global/pg_control
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/base/1/12000, strQueueIdx = 0
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 11:24:42 +02:00
P01 DETAIL: checksum resumed file [TEST_PATH]/db-master/db/base/global/pg_control (8KB, 95%) checksum 4c77c900f7af0d9ab13fa9982051a42e0b637f6c
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (0, 8192, 8192, 22c98d248ff548311eda88559e4a8405ed77c003, [undef]), strKey = pg_data/base/1/12000
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/postgresql.conf, strQueueIdx = 0
Add more realistic data files to synthetic backup and restore tests.
2016-12-10 16:13:55 +02:00
P01 DETAIL: checksum resumed file [TEST_PATH]/db-master/db/base/base/1/12000 (8KB, 99%) checksum 22c98d248ff548311eda88559e4a8405ed77c003
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (0, 21, 21, 6721d92c9fcdf4248acff1f9a1377127d9064807, [undef]), strKey = pg_data/postgresql.conf
Fix failure in manifest build when two or more files in PGDATA are linked to the same directory. Reported by Vitaliy Kukharik.
2018-05-02 18:19:54 +02:00
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/pg_hba.conf, strQueueIdx = 0
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 DETAIL: checksum resumed file [TEST_PATH]/db-master/db/base/postgresql.conf (21B, 99%) checksum 6721d92c9fcdf4248acff1f9a1377127d9064807
Fix failure in manifest build when two or more files in PGDATA are linked to the same directory. Reported by Vitaliy Kukharik.
2018-05-02 18:19:54 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (0, 9, 9, dd4cea0cae348309f9de28ad4ded8ee2cc2e6d5b, [undef]), strKey = pg_data/pg_hba.conf
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/changecontent.txt, strQueueIdx = 0
Fix failure in manifest build when two or more files in PGDATA are linked to the same directory. Reported by Vitaliy Kukharik.
2018-05-02 18:19:54 +02:00
P01 DETAIL: checksum resumed file [TEST_PATH]/db-master/db/base/pg_hba.conf (9B, 99%) checksum dd4cea0cae348309f9de28ad4ded8ee2cc2e6d5b
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1, 7, 7, 238a131a3e8eb98d1fc5b27d882ca40b7618fd2a, [undef]), strKey = pg_data/changecontent.txt
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/pg_stat/global.stat, strQueueIdx = 0
P01 INFO: backup file [TEST_PATH]/db-master/db/base/changecontent.txt (7B, 99%) checksum 238a131a3e8eb98d1fc5b27d882ca40b7618fd2a
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (0, 5, 5, e350d5ce0153f3e22d5db21cf2a4eff00f3ee877, [undef]), strKey = pg_data/pg_stat/global.stat
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/changetime.txt, strQueueIdx = 0
Add more realistic data files to synthetic backup and restore tests.
2016-12-10 16:13:55 +02:00
P01 DETAIL: checksum resumed file [TEST_PATH]/db-master/db/base/pg_stat/global.stat (5B, 99%) checksum e350d5ce0153f3e22d5db21cf2a4eff00f3ee877
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1, 4, 4, 88087292ed82e26f3eb824d0bffc05ccf7a30f8d, [undef]), strKey = pg_data/changetime.txt
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/base/32768/PG_VERSION, strQueueIdx = 0
P01 INFO: backup file [TEST_PATH]/db-master/db/base/changetime.txt (4B, 99%) checksum 88087292ed82e26f3eb824d0bffc05ccf7a30f8d
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (0, 3, 3, 184473f470864e067ee3a22e64b47b0a1c356f29, [undef]), strKey = pg_data/base/32768/PG_VERSION
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/base/16384/PG_VERSION, strQueueIdx = 0
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 DETAIL: checksum resumed file [TEST_PATH]/db-master/db/base/base/32768/PG_VERSION (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (0, 3, 3, 184473f470864e067ee3a22e64b47b0a1c356f29, [undef]), strKey = pg_data/base/16384/PG_VERSION
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/base/1/PG_VERSION, strQueueIdx = 0
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 DETAIL: checksum resumed file [TEST_PATH]/db-master/db/base/base/16384/PG_VERSION (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (0, 3, 3, 184473f470864e067ee3a22e64b47b0a1c356f29, [undef]), strKey = pg_data/base/1/PG_VERSION
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/PG_VERSION, strQueueIdx = 0
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 DETAIL: checksum resumed file [TEST_PATH]/db-master/db/base/base/1/PG_VERSION (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1, 3, 3, 184473f470864e067ee3a22e64b47b0a1c356f29, [undef]), strKey = pg_data/PG_VERSION
Allow zero-size files in backup manifest to reference a prior manifest regardless of timestamp delta. Contributed by Cynthia Shang.
2018-08-24 22:50:33 +02:00
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/zero_from_start, strQueueIdx = 0
P01 INFO: backup file [TEST_PATH]/db-master/db/base/PG_VERSION (3B, 100%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1, 0, 0, da39a3ee5e6b4b0d3255bfef95601890afd80709, [undef]), strKey = pg_data/zero_from_start
Improve support for special characters in filenames. % characters caused issues in backup/restore due to filenames being appended directly into a format string. Reserved XML characters (<>&') caused issues in the S3 driver due to improper escaping. Add a file with all common special characters to regression testing.
2018-09-10 16:54:34 +02:00
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?%, strQueueIdx = 0
P01 INFO: backup file [TEST_PATH]/db-master/db/base/zero_from_start (0B, 100%)
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1, 0, 0, da39a3ee5e6b4b0d3255bfef95601890afd80709, [undef]), strKey = pg_data/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?%
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->process: no jobs found, stop local: iHostConfigIdx = [undef], iHostIdx = 0, iProcessId = 1, strHostType = [undef]
P00 DEBUG: Protocol::Command::Master->close=>: iExitStatus = 0
Improve support for special characters in filenames. % characters caused issues in backup/restore due to filenames being appended directly into a format string. Reserved XML characters (<>&') caused issues in the S3 driver due to improper escaping. Add a file with all common special characters to regression testing.
2018-09-10 16:54:34 +02:00
P01 INFO: backup file [TEST_PATH]/db-master/db/base/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?% (0B, 100%)
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->process: all jobs complete
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P00 DEBUG: Backup::Backup->processManifest=>: lSizeTotal = 196666
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
P00 INFO: full backup size = 192KB
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Helper::protocolDestroy(): bComplete = true, iRemoteIdx = [undef], strRemoteType = [undef]
P00 DEBUG: Protocol::Helper::protocolDestroy=>: iExitStatus = 0
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Storage::Local->pathSync(): strPathExp = <REPO:BACKUP>/[BACKUP-FULL-2]
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]/pg_data
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]/pg_data/base
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]/pg_data/base/1
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]/pg_data/base/16384
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]/pg_data/base/32768
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]/pg_data/global
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]/pg_data/pg_clog
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]/pg_data/pg_dynshmem
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]/pg_data/pg_log
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]/pg_data/pg_notify
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]/pg_data/pg_replslot
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]/pg_data/pg_serial
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]/pg_data/pg_snapshots
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]/pg_data/pg_stat
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]/pg_data/pg_stat_tmp
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]/pg_data/pg_subtrans
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]/pg_data/pg_tblspc
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->openWrite(): bAtomic = <false>, bPathCreate = <false>, lTimestamp = [undef], rhyFilter = [undef], strCipherPass = [undef], strGroup = [undef], strMode = <0640>, strUser = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]/backup.manifest
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->openWrite(): bAtomic = <false>, bPathCreate = <false>, lTimestamp = [undef], rhyFilter = [undef], strCipherPass = [undef], strGroup = [undef], strMode = <0640>, strUser = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]/backup.manifest.copy
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: new backup label = [BACKUP-FULL-2]
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->openRead(): bIgnoreMissing = <false>, rhyFilter = [undef], strCipherPass = [undef], xFileExp = <REPO:BACKUP>/[BACKUP-FULL-2]/backup.manifest
Make backup.history sync more efficient. Only the backup.history/[year] directory was being synced, so check if the backup.history is newly created and sync it as well.
2018-04-04 02:54:18 +02:00
P00 DEBUG: Storage::Local->openWrite(): bAtomic = true, bPathCreate = true, lTimestamp = [undef], rhyFilter = ({strClass => pgBackRest::Storage::Filter::Gzip}), strCipherPass = [undef], strGroup = [undef], strMode = <0640>, strUser = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/backup.history/[YEAR-1]/[BACKUP-FULL-2].manifest.gz
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Base->copy(): xDestinationFile = [object], xSourceFile = [object]
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Storage::Local->pathSync(): strPathExp = <REPO:BACKUP>/backup.history
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/backup.history/[YEAR-1]
Add bIgnoreMissing parameter to Local->manifest().
2017-07-25 18:44:38 +02:00
P00 DEBUG: Storage::Local->remove(): bIgnoreMissing = <true>, bRecurse = <false>, xstryPathFileExp = <REPO:BACKUP>/latest
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->remove=>: bRemoved = false
P00 DEBUG: Storage::Local->linkCreate(): bHard = <false>, bIgnoreExists = <false>, bPathCreate = <true>, bRelative = true, strDestinationLinkExp = <REPO:BACKUP>/latest, strSourcePathFileExp = <REPO:BACKUP>/[BACKUP-FULL-2]
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->openWrite(): bAtomic = <false>, bPathCreate = <false>, lTimestamp = [undef], rhyFilter = [undef], strCipherPass = [undef], strGroup = [undef], strMode = <0640>, strUser = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/backup.info
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/repo/backup/db
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->openWrite(): bAtomic = <false>, bPathCreate = <false>, lTimestamp = [undef], rhyFilter = [undef], strCipherPass = [undef], strGroup = [undef], strMode = <0640>, strUser = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/backup.info.copy
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/repo/backup/db
P00 DEBUG: Storage::Local->pathSync(): strPathExp = <REPO:BACKUP>
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: perl/exec::perlExec: => 0
Added a predictable log entry to signal that a command has completed successfully. For example a backup ends successfully with: INFO: backup command end: completed successfully.
2016-12-04 21:44:53 +02:00
P00 INFO: backup command end: completed successfully
Move async forking and more error handling to C. The Perl process was exiting directly when called but that interfered with proper locking for the forked async process. Now Perl returns results to the C process which handles all errors, including signals.
2018-04-13 02:42:26 +02:00
P00 INFO: expire command begin
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: perl/exec::perlExec: (void)
Move async forking and more error handling to C. The Perl process was exiting directly when called but that interfered with proper locking for the forked async process. Now Perl returns results to the C process which handles all errors, including signals.
2018-04-13 02:42:26 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp =
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = true, bIgnoreExists = true, strMode = 0770, strPathExp = [TEST_PATH]/db-master/log
P00 DEBUG: Common::Lock::lockStopTest(): bStanzaStopRequired = <false>
P00 DEBUG: Common::Lock::lockStopTest=>: bStopExists = false
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Backup::Info->new(): bIgnoreMissing = <false>, bLoad = <true>, bRequired = <true>, bValidate = <true>, oStorage = <[object]>, strBackupClusterPath = [TEST_PATH]/db-master/repo/backup/db, strCipherPassSub = [undef]
P00 DEBUG: Storage::Local->encrypted(): bIgnoreMissing = true, strFileName = [TEST_PATH]/db-master/repo/backup/db/backup.info
P00 DEBUG: Storage::Local->encrypted=>: bEncrypted = false
P00 DEBUG: Storage::Local->encryptionValid(): bEncrypted = false
P00 DEBUG: Storage::Local->encryptionValid=>: bValid = true
P00 DEBUG: Storage::Local->openRead(): bIgnoreMissing = true, rhyFilter = [undef], strCipherPass = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/backup.info
P00 DEBUG: Storage::Base->get(): strCipherPass = [undef], xFile = [object]
Move backup modules into Backup directory.
2017-05-15 22:01:00 +02:00
P00 DEBUG: Backup::Info->reconstruct(): bRequired = <true>, bSave = <true>, iCatalogVersion = [undef], iControlVersion = [undef], strDbVersion = [undef], ullDbSysId = [undef]
P00 DEBUG: Backup::Common::backupRegExpGet(): bAnchor = <true>, bDifferential = true, bFull = true, bIncremental = true
P00 DEBUG: Backup::Common::backupRegExpGet=>: strRegExp = ^[0-9]{8}\-[0-9]{6}F(\_[0-9]{8}\-[0-9]{6}(D|I)){0,1}$
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->list(): bIgnoreMissing = <false>, strExpression = ^[0-9]{8}\-[0-9]{6}F(\_[0-9]{8}\-[0-9]{6}(D|I)){0,1}$, strPathExp = [TEST_PATH]/db-master/repo/backup/db, strSortOrder = <forward>
P00 DEBUG: Storage::Local->list=>: stryFileList = ([BACKUP-FULL-2])
Move backup modules into Backup directory.
2017-05-15 22:01:00 +02:00
P00 DEBUG: Backup::Info->current(): strBackup = [BACKUP-FULL-2]
P00 DEBUG: Backup::Info->current=>: bTest = true
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]/backup.manifest
P00 DEBUG: Storage::Local->exists=>: bExists = true
Move backup modules into Backup directory.
2017-05-15 22:01:00 +02:00
P00 DEBUG: Backup::Common::backupRegExpGet(): bAnchor = <true>, bDifferential = true, bFull = true, bIncremental = true
P00 DEBUG: Backup::Common::backupRegExpGet=>: strRegExp = ^[0-9]{8}\-[0-9]{6}F(\_[0-9]{8}\-[0-9]{6}(D|I)){0,1}$
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->list(): bIgnoreMissing = <false>, strExpression = ^[0-9]{8}\-[0-9]{6}F(\_[0-9]{8}\-[0-9]{6}(D|I)){0,1}$, strPathExp = <REPO:BACKUP>, strSortOrder = reverse
P00 DEBUG: Storage::Local->list=>: stryFileList = ([BACKUP-FULL-2])
Move backup modules into Backup directory.
2017-05-15 22:01:00 +02:00
P00 DEBUG: Backup::Info->current(): strBackup = [BACKUP-FULL-2]
P00 DEBUG: Backup::Info->current=>: bTest = true
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
P00 INFO: option 'repo1-retention-archive' is not set - archive logs will not be expired
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: perl/exec::perlExec: => 0
P00 DEBUG: common/exit::exitSafe: (result: 0, error: false, signalType: 0)
P00 DEBUG: common/lock::lockRelease: (failOnNoLock: false)
P00 DEBUG: common/lock::lockRelease: => true
Move async forking and more error handling to C. The Perl process was exiting directly when called but that interfered with proper locking for the forked async process. Now Perl returns results to the C process which handles all errors, including signals.
2018-04-13 02:42:26 +02:00
P00 DEBUG: Main::mainCleanup(): iExitCode = 0
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Helper::protocolDestroy(): bComplete = true, iRemoteIdx = [undef], strRemoteType = [undef]
P00 DEBUG: Protocol::Helper::protocolDestroy=>: iExitStatus = 0
Added a predictable log entry to signal that a command has completed successfully. For example a backup ends successfully with: INFO: backup command end: completed successfully.
2016-12-04 21:44:53 +02:00
P00 INFO: expire command end: completed successfully
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: common/exit::exitSafe: => 0
P00 DEBUG: main::main: => 0
Log testing can now be enabled for certain deterministic tests. This works by comparing the generated logs against a previous copy. Currently only enabled for the backup/synthetic tests.
2015-05-05 19:08:48 +02:00
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
+ supplemental file: [TEST_PATH]/db-master/pgbackrest.conf
----------------------------------------------------------
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[db]
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
pg1-path=[TEST_PATH]/db-master/db/base
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
New simpler configuration and consistent project/exe/path naming. * The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required. * Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case. * The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally. * The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest. * Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory. * Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory. * Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 15:30:54 +02:00
[global]
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
compress=n
Fix custom settings for compress-level option being ignored. Reported by Jens Wilke.
2017-10-23 00:05:46 +02:00
compress-level=3
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
db-timeout=45
lock-path=[TEST_PATH]/db-master/lock
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
log-level-console=debug
log-level-file=trace
Added the log-level-stderr option. Controls whether console log messages are sent to stderr or stdout. By default this is set to warn which represents a change in behavior from previous versions, even though it may be more intuitive. Setting log-level-stderr=off will preserve the old behavior. Suggested by Sascha Biberhofer.
2016-10-05 15:09:30 +02:00
log-level-stderr=off
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
log-path=[TEST_PATH]/db-master/log
Add log-subprocess option to allow file logging for local and remote subprocesses.
2018-08-23 02:05:49 +02:00
log-subprocess=y
Set log-timestamp=n for integration tests. This means less filtering of logs needs to be done and new timestamps can be added without adding new filters.
2018-05-11 17:24:38 +02:00
log-timestamp=n
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
protocol-timeout=60
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
repo1-path=[TEST_PATH]/db-master/repo
Consolidate archive-push tests into new archive test.
2017-06-21 14:38:53 +02:00
spool-path=[TEST_PATH]/db-master/spool
New simpler configuration and consistent project/exe/path naming. * The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required. * Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case. * The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally. * The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest. * Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory. * Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory. * Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 15:30:54 +02:00
[global:backup]
archive-copy=y
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
exclude=postgresql.auto.conf
exclude=pg_log/
exclude=pg_log2
New simpler configuration and consistent project/exe/path naming. * The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required. * Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case. * The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally. * The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest. * Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory. * Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory. * Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 15:30:54 +02:00
start-fast=y
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
+ supplemental file: [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]/backup.manifest
-----------------------------------------------------------------------------------------
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[backrest]
backrest-checksum="[CHECKSUM]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
backrest-format=5
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
backrest-version="[VERSION-1]"
[backup]
backup-label="[BACKUP-FULL-2]"
backup-timestamp-copy-start=[TIMESTAMP]
backup-timestamp-start=[TIMESTAMP]
backup-timestamp-stop=[TIMESTAMP]
backup-type="full"
[backup:db]
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-catalog-version=201409291
db-control-version=942
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
db-id=1
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
db-system-id=1000000000000000094
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-version="9.4"
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[backup:option]
option-archive-check=true
option-archive-copy=true
Backup from a standby cluster. A connection to the primary cluster is still required to start/stop the backup and copy files that are not replicated, but the vast majority of files are copied from the standby in order to reduce load on the master.
2016-08-25 17:25:46 +02:00
option-backup-standby=false
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
option-checksum-page=true
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
option-compress=false
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
option-delta=true
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
option-hardlink=false
Closed #167: Rename `--no-start-stop` option to `--no-online`.
2016-02-20 02:32:02 +02:00
option-online=false
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[backup:target]
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
pg_data={"path":"[TEST_PATH]/db-master/db/base","type":"path"}
Fix failure in manifest build when two or more files in PGDATA are linked to the same directory. Reported by Vitaliy Kukharik.
2018-05-02 18:19:54 +02:00
pg_data/pg_hba.conf={"file":"pg_hba.conf","path":"../pg_config","type":"link"}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_data/pg_stat={"path":"../pg_stat","type":"link"}
pg_data/postgresql.conf={"file":"postgresql.conf","path":"../pg_config","type":"link"}
[target:file]
Fixed a issue where tablespaces were copied from the master during standby backup.
2016-09-04 15:19:44 +02:00
pg_data/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","master":true,"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
pg_data/base/1/12000={"checksum":"22c98d248ff548311eda88559e4a8405ed77c003","checksum-page":true,"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
pg_data/base/1/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","mode":"0660","size":[SIZE],"timestamp":[TIMESTAMP-1],"user":false}
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
pg_data/base/16384/17000={"checksum":"e0101dd8ffb910c9c202ca35b5f828bcb9697bed","checksum-page":false,"checksum-page-error":[1],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
pg_data/base/16384/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","group":false,"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Reduced the likelihood of torn pages causing a false positive in page checksums by filtering on start backup LSN.
2017-01-30 20:59:00 +02:00
pg_data/base/32768/33000={"checksum":"4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f","checksum-page":true,"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Fixed an issue where newly initialized (but unused) pages would cause page checksum warnings. Reported by Stephen Frost.
2017-03-13 21:50:13 +02:00
pg_data/base/32768/33000.32767={"checksum":"21e2c7c1a326682c07053b7d6a5a40dbd49c2ec5","checksum-page":true,"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
pg_data/base/32768/33001={"checksum":"6bf316f11d28c28914ea9be92c00de9bea6d9a6b","checksum-page":false,"checksum-page-error":[0,[3,5],7],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
pg_data/base/32768/44000_init={"checksum":"4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f","checksum-page":true,"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
pg_data/base/32768/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
pg_data/changecontent.txt={"checksum":"238a131a3e8eb98d1fc5b27d882ca40b7618fd2a","master":true,"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/changetime.txt={"checksum":"88087292ed82e26f3eb824d0bffc05ccf7a30f8d","master":true,"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 11:24:42 +02:00
pg_data/global/pg_control={"checksum":"4c77c900f7af0d9ab13fa9982051a42e0b637f6c","master":true,"size":[SIZE],"timestamp":[TIMESTAMP-2]}
Fix failure in manifest build when two or more files in PGDATA are linked to the same directory. Reported by Vitaliy Kukharik.
2018-05-02 18:19:54 +02:00
pg_data/pg_hba.conf={"checksum":"dd4cea0cae348309f9de28ad4ded8ee2cc2e6d5b","master":true,"size":[SIZE],"timestamp":[TIMESTAMP-2]}
Fixed a issue where tablespaces were copied from the master during standby backup.
2016-09-04 15:19:44 +02:00
pg_data/pg_stat/global.stat={"checksum":"e350d5ce0153f3e22d5db21cf2a4eff00f3ee877","master":true,"size":[SIZE],"timestamp":[TIMESTAMP-2]}
pg_data/postgresql.conf={"checksum":"6721d92c9fcdf4248acff1f9a1377127d9064807","master":true,"size":[SIZE],"timestamp":[TIMESTAMP-2]}
Improve support for special characters in filenames. % characters caused issues in backup/restore due to filenames being appended directly into a format string. Reserved XML characters (<>&') caused issues in the S3 driver due to improper escaping. Add a file with all common special characters to regression testing.
2018-09-10 16:54:34 +02:00
pg_data/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?%={"master":true,"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Allow zero-size files in backup manifest to reference a prior manifest regardless of timestamp delta. Contributed by Cynthia Shang.
2018-08-24 22:50:33 +02:00
pg_data/zero_from_start={"master":true,"size":[SIZE],"timestamp":[TIMESTAMP-1]}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:file:default]
Continuous integration using travis-ci.
2017-02-21 15:59:23 +02:00
group="[GROUP-1]"
Fixed a issue where tablespaces were copied from the master during standby backup.
2016-09-04 15:19:44 +02:00
master=false
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
mode="0600"
Regression tests can now be run as any properly-configured user, not just vagrant.
2016-10-01 19:39:44 +02:00
user="[USER-1]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:link]
Fix failure in manifest build when two or more files in PGDATA are linked to the same directory. Reported by Vitaliy Kukharik.
2018-05-02 18:19:54 +02:00
pg_data/pg_hba.conf={"destination":"../pg_config/pg_hba.conf"}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_data/pg_stat={"destination":"../pg_stat"}
pg_data/postgresql.conf={"destination":"../pg_config/postgresql.conf"}
[target:link:default]
Continuous integration using travis-ci.
2017-02-21 15:59:23 +02:00
group="[GROUP-1]"
Regression tests can now be run as any properly-configured user, not just vagrant.
2016-10-01 19:39:44 +02:00
user="[USER-1]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:path]
Fixed an issue with invalid backup groups being set correctly on restore. If the backup cannot map a group to a name it stores the group in the manifest as false then uses either the owner of $PGDATA to set the group during restore or failing that the group of the current user. This logic was not working correctly because the selected group was overwriting the user on restore leaving the group undefined and the user incorrectly set to the group. (Reported by Jeff McCormick.)
2017-11-20 19:46:35 +02:00
pg_data={"group":false,"mode":"0777","user":false}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_data/base={}
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
pg_data/base/1={}
pg_data/base/16384={}
pg_data/base/32768={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_data/global={}
pg_data/pg_clog={}
Exclude directories during backup that are cleaned, recreated, or zeroed by PostgreSQL at startup. These include (depending on the version where they were introduced): pgsql_tmp, pg_dynshmem, pg_notify, pg_replslot, pg_serial, pg_snapshots, pg_stat_tmp, pg_subtrans. The postgresql.auto.conf.tmp file is now excluded in addition to files that were already excluded: backup_label.old, postmaster.opts, postmaster.pid, recovery.conf, recovery.done.
2016-08-16 15:35:16 +02:00
pg_data/pg_dynshmem={}
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
pg_data/pg_log={}
Exclude directories during backup that are cleaned, recreated, or zeroed by PostgreSQL at startup. These include (depending on the version where they were introduced): pgsql_tmp, pg_dynshmem, pg_notify, pg_replslot, pg_serial, pg_snapshots, pg_stat_tmp, pg_subtrans. The postgresql.auto.conf.tmp file is now excluded in addition to files that were already excluded: backup_label.old, postmaster.opts, postmaster.pid, recovery.conf, recovery.done.
2016-08-16 15:35:16 +02:00
pg_data/pg_notify={}
pg_data/pg_replslot={}
pg_data/pg_serial={}
pg_data/pg_snapshots={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_data/pg_stat={}
Exclude directories during backup that are cleaned, recreated, or zeroed by PostgreSQL at startup. These include (depending on the version where they were introduced): pgsql_tmp, pg_dynshmem, pg_notify, pg_replslot, pg_serial, pg_snapshots, pg_stat_tmp, pg_subtrans. The postgresql.auto.conf.tmp file is now excluded in addition to files that were already excluded: backup_label.old, postmaster.opts, postmaster.pid, recovery.conf, recovery.done.
2016-08-16 15:35:16 +02:00
pg_data/pg_stat_tmp={}
pg_data/pg_subtrans={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_data/pg_tblspc={}
[target:path:default]
Continuous integration using travis-ci.
2017-02-21 15:59:23 +02:00
group="[GROUP-1]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
mode="0700"
Regression tests can now be run as any properly-configured user, not just vagrant.
2016-10-01 19:39:44 +02:00
user="[USER-1]"
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
+ supplemental file: [TEST_PATH]/db-master/repo/backup/db/backup.info
---------------------------------------------------------------------
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[backrest]
backrest-checksum="[CHECKSUM]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
backrest-format=5
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
backrest-version="[VERSION-1]"
[backup:current]
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
[BACKUP-FULL-2]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"full","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":true,"option-compress":false,"option-hardlink":false,"option-online":false}
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[db]
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-catalog-version=201409291
db-control-version=942
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
db-id=1
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
db-system-id=1000000000000000094
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-version="9.4"
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[db:history]
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
1={"db-catalog-version":201409291,"db-control-version":942,"db-system-id":1000000000000000094,"db-version":"9.4"}
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
full backup - invalid repo (db-master host)
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --no-online --repo1-path=/bogus_path --log-level-console=detail --type=full --stanza=db backup
Closed #113: Better error message when repo does not exist.
2016-02-06 23:49:12 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
P00 INFO: backup command begin [BACKREST-VERSION]: --no-compress --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --db-timeout=45 --exclude=postgresql.auto.conf --exclude=pg_log/ --exclude=pg_log2 --lock-path=[TEST_PATH]/db-master/lock --log-level-console=detail --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --no-online --pg1-path=[TEST_PATH]/db-master/db/base --protocol-timeout=60 --repo1-path=/bogus_path --stanza=db --start-fast --type=full
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
P00 WARN: option repo1-retention-full is not set, the repository may run out of space
HINT: to retain full backups indefinitely (without warning), set option 'repo1-retention-full' to the maximum.
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
P00 ERROR: [073]: repo1-path '/bogus_path' does not exist
Renumber return codes between 25 and 125 to avoid PostgreSQL interpreting some as fatal signal exceptions. Suggested by Yogesh Sharma.
2017-04-12 18:41:34 +02:00
P00 INFO: backup command end: aborted with exception [073]
Closed #113: Better error message when repo does not exist.
2016-02-06 23:49:12 +02:00
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
restore delta, backup '[BACKUP-FULL-2]' - add and delete files (db-master host)
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --delta --set=[BACKUP-FULL-2] --link-all --stanza=db restore
Log testing can now be enabled for certain deterministic tests. This works by comparing the generated logs against a previous copy. Currently only enabled for the backup/synthetic tests.
2015-05-05 19:08:48 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Add log-subprocess option to allow file logging for local and remote subprocesses.
2018-08-23 02:05:49 +02:00
P00 INFO: restore command begin [BACKREST-VERSION]: --no-compress --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --delta --link-all --lock-path=[TEST_PATH]/db-master/lock --log-level-console=debug --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --pg1-path=[TEST_PATH]/db-master/db/base --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --set=[BACKUP-FULL-2] --stanza=db
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: config/load::cfgLoad: => void
P00 DEBUG: perl/exec::perlExec: (void)
Allow functions with sensitive options to be logged at debug level with redactions. Previously, functions with sensitive options had to be logged at trace level to avoid exposing them. Trace level logging may still expose secrets so use with caution.
2017-10-24 18:35:36 +02:00
P00 DEBUG: Storage::Posix::Driver->new(): bFileSync = <true>, bPathSync = <true>
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->new(): bAllowTemp = <true>, hRule = [hash], lBufferMax = 4194304, oDriver = [object], strCipherPassUser = [undef], strCipherType = [undef], strDefaultFileMode = <0640>, strDefaultPathMode = <0750>, strPathBase = [TEST_PATH]/db-master/repo, strTempExtension = pgbackrest.tmp
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp =
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
Config parsing no longer implemented in Perl. Options are passed to Perl as JSON from the C process.
2018-01-29 04:37:09 +02:00
P00 DEBUG: Storage::Posix::Driver->new(): bFileSync = <true>, bPathSync = <true>
P00 DEBUG: Storage::Local->new(): bAllowTemp = <true>, hRule = [undef], lBufferMax = 4194304, oDriver = [object], strCipherPassUser = [undef], strCipherType = [undef], strDefaultFileMode = <0640>, strDefaultPathMode = <0750>, strPathBase = /, strTempExtension = pgbackrest.tmp
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = true, bIgnoreExists = true, strMode = 0770, strPathExp = [TEST_PATH]/db-master/log
Allow functions with sensitive options to be logged at debug level with redactions. Previously, functions with sensitive options had to be logged at trace level to avoid exposing them. Trace level logging may still expose secrets so use with caution.
2017-10-24 18:35:36 +02:00
P00 DEBUG: Storage::Posix::Driver->new(): bFileSync = <true>, bPathSync = <true>
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->new(): bAllowTemp = <true>, hRule = [undef], lBufferMax = 4194304, oDriver = [object], strCipherPassUser = [undef], strCipherType = [undef], strDefaultFileMode = <0640>, strDefaultPathMode = <0750>, strPathBase = [TEST_PATH]/db-master/db/base, strTempExtension = pgbackrest.tmp
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/base
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/db/base/postmaster.pid
P00 DEBUG: Storage::Local->exists=>: bExists = false
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/db/base/PG_VERSION
P00 DEBUG: Storage::Local->exists=>: bExists = true
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->openRead(): bIgnoreMissing = <false>, rhyFilter = [undef], strCipherPass = [undef], xFileExp = <REPO:BACKUP>/backup.info
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Base->copy(): xDestinationFile = [TEST_PATH]/db-master/db/base/backup.info, xSourceFile = [object]
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->openWrite(): bAtomic = <false>, bPathCreate = <false>, lTimestamp = [undef], rhyFilter = [undef], strCipherPass = [undef], strGroup = [undef], strMode = <0640>, strUser = [undef], xFileExp = [TEST_PATH]/db-master/db/base/backup.info
P00 DEBUG: Backup::Info->new(): bIgnoreMissing = <false>, bLoad = <true>, bRequired = <true>, bValidate = false, oStorage = [object], strBackupClusterPath = [TEST_PATH]/db-master/db/base, strCipherPassSub = [undef]
P00 DEBUG: Storage::Local->encrypted(): bIgnoreMissing = true, strFileName = [TEST_PATH]/db-master/db/base/backup.info
P00 DEBUG: Storage::Local->encrypted=>: bEncrypted = false
P00 DEBUG: Storage::Local->encryptionValid(): bEncrypted = false
P00 DEBUG: Storage::Local->encryptionValid=>: bValid = true
P00 DEBUG: Storage::Local->openRead(): bIgnoreMissing = true, rhyFilter = [undef], strCipherPass = [undef], xFileExp = [TEST_PATH]/db-master/db/base/backup.info
P00 DEBUG: Storage::Base->get(): strCipherPass = [undef], xFile = [object]
Add bIgnoreMissing parameter to Local->manifest().
2017-07-25 18:44:38 +02:00
P00 DEBUG: Storage::Local->remove(): bIgnoreMissing = <true>, bRecurse = <false>, xstryPathFileExp = [TEST_PATH]/db-master/db/base/backup.info
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->remove=>: bRemoved = true
Move backup modules into Backup directory.
2017-05-15 22:01:00 +02:00
P00 DEBUG: Backup::Info->current(): strBackup = [BACKUP-FULL-2]
P00 DEBUG: Backup::Info->current=>: bTest = true
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: restore backup set [BACKUP-FULL-2]
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Restore->manifestLoad(): strCipherPass = [undef]
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->exists(): strFileExp = <REPO:BACKUP>/[BACKUP-FULL-2]/backup.manifest
P00 DEBUG: Storage::Local->exists=>: bExists = true
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->openRead(): bIgnoreMissing = <false>, rhyFilter = [undef], strCipherPass = [undef], xFileExp = <REPO:BACKUP>/[BACKUP-FULL-2]/backup.manifest
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Base->copy(): xDestinationFile = [TEST_PATH]/db-master/db/base/backup.manifest, xSourceFile = [object]
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->openWrite(): bAtomic = <false>, bPathCreate = <false>, lTimestamp = [undef], rhyFilter = [undef], strCipherPass = [undef], strGroup = [undef], strMode = <0640>, strUser = [undef], xFileExp = [TEST_PATH]/db-master/db/base/backup.manifest
Require PostgreSQL catalog version when instantiating a Manifest object (and not loading it from disk). Contributed by Cynthia Shang.
2018-07-16 23:25:15 +02:00
P00 DEBUG: Manifest->new(): bLoad = <true>, iDbCatalogVersion = [undef], oStorage = [object], strCipherPass = [undef], strCipherPassSub = [undef], strDbVersion = [undef], strFileName = [TEST_PATH]/db-master/db/base/backup.manifest
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->encrypted(): bIgnoreMissing = true, strFileName = [TEST_PATH]/db-master/db/base/backup.manifest
P00 DEBUG: Storage::Local->encrypted=>: bEncrypted = false
P00 DEBUG: Storage::Local->encryptionValid(): bEncrypted = false
P00 DEBUG: Storage::Local->encryptionValid=>: bValid = true
P00 DEBUG: Storage::Local->openRead(): bIgnoreMissing = true, rhyFilter = [undef], strCipherPass = [undef], xFileExp = [TEST_PATH]/db-master/db/base/backup.manifest
P00 DEBUG: Storage::Base->get(): strCipherPass = [undef], xFile = [object]
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 DEBUG: Restore->manifestOwnershipCheck(): oManifest = [object]
Fixed an issue with invalid backup groups being set correctly on restore. If the backup cannot map a group to a name it stores the group in the manifest as false then uses either the owner of $PGDATA to set the group during restore or failing that the group of the current user. This logic was not working correctly because the selected group was overwriting the user on restore leaving the group undefined and the user incorrectly set to the group. (Reported by Jeff McCormick.)
2017-11-20 19:46:35 +02:00
P00 WARN: backup group for pg_data/base/16384/PG_VERSION was not mapped to a name, set to [GROUP-2]
P00 WARN: backup group for pg_data was not mapped to a name, set to [GROUP-2]
P00 WARN: backup user for pg_data/base/1/PG_VERSION was not mapped to a name, set to [USER-2]
P00 WARN: backup user for pg_data was not mapped to a name, set to [USER-2]
Add bIgnoreMissing parameter to Local->manifest().
2017-07-25 18:44:38 +02:00
P00 DEBUG: Storage::Local->remove(): bIgnoreMissing = <true>, bRecurse = <false>, xstryPathFileExp = [TEST_PATH]/db-master/db/base/global/pg_control
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->remove=>: bRemoved = true
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 DEBUG: Restore->clean(): oManifest = [object]
P00 DETAIL: check [TEST_PATH]/db-master/db/base exists
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/base
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
Ignore all files in a linked tablespace directory except the subdirectory for the current version of PostgreSQL. Previously an error would be generated if other files were present and not owned by the PostgreSQL user. This hasn't been a big deal in practice but it could cause issues. Also add tests to make sure the same logic applies with links to files, i.e. all other files in the directory should be ignored. This was actually working correctly, but there were no tests for it before.
2018-08-31 22:06:40 +02:00
P00 DEBUG: Storage::Local->manifest(): strFilter = [undef], strPathExp = [TEST_PATH]/db-master/db/base
Fix failure in manifest build when two or more files in PGDATA are linked to the same directory. Reported by Vitaliy Kukharik.
2018-05-02 18:19:54 +02:00
P00 DETAIL: check [TEST_PATH]/db-master/db/pg_config exists
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/pg_config
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/db/pg_config/pg_hba.conf
P00 DEBUG: Storage::Local->exists=>: bExists = true
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 DETAIL: check [TEST_PATH]/db-master/db/pg_stat exists
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/pg_stat
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
Ignore all files in a linked tablespace directory except the subdirectory for the current version of PostgreSQL. Previously an error would be generated if other files were present and not owned by the PostgreSQL user. This hasn't been a big deal in practice but it could cause issues. Also add tests to make sure the same logic applies with links to files, i.e. all other files in the directory should be ignored. This was actually working correctly, but there were no tests for it before.
2018-08-31 22:06:40 +02:00
P00 DEBUG: Storage::Local->manifest(): strFilter = [undef], strPathExp = [TEST_PATH]/db-master/db/pg_stat
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 DETAIL: check [TEST_PATH]/db-master/db/pg_config exists
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/pg_config
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/db/pg_config/postgresql.conf
P00 DEBUG: Storage::Local->exists=>: bExists = true
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: remove invalid files/paths/links from [TEST_PATH]/db-master/db/pg_config
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/pg_config
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
Ignore all files in a linked tablespace directory except the subdirectory for the current version of PostgreSQL. Previously an error would be generated if other files were present and not owned by the PostgreSQL user. This hasn't been a big deal in practice but it could cause issues. Also add tests to make sure the same logic applies with links to files, i.e. all other files in the directory should be ignored. This was actually working correctly, but there were no tests for it before.
2018-08-31 22:06:40 +02:00
P00 DEBUG: Storage::Local->manifest(): strFilter = [undef], strPathExp = [TEST_PATH]/db-master/db/pg_config
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: remove invalid files/paths/links from [TEST_PATH]/db-master/db/pg_stat
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/pg_stat
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
Ignore all files in a linked tablespace directory except the subdirectory for the current version of PostgreSQL. Previously an error would be generated if other files were present and not owned by the PostgreSQL user. This hasn't been a big deal in practice but it could cause issues. Also add tests to make sure the same logic applies with links to files, i.e. all other files in the directory should be ignored. This was actually working correctly, but there were no tests for it before.
2018-08-31 22:06:40 +02:00
P00 DEBUG: Storage::Local->manifest(): strFilter = [undef], strPathExp = [TEST_PATH]/db-master/db/pg_stat
Fix failure in manifest build when two or more files in PGDATA are linked to the same directory. Reported by Vitaliy Kukharik.
2018-05-02 18:19:54 +02:00
P00 INFO: remove invalid files/paths/links from [TEST_PATH]/db-master/db/pg_config
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/pg_config
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
Ignore all files in a linked tablespace directory except the subdirectory for the current version of PostgreSQL. Previously an error would be generated if other files were present and not owned by the PostgreSQL user. This hasn't been a big deal in practice but it could cause issues. Also add tests to make sure the same logic applies with links to files, i.e. all other files in the directory should be ignored. This was actually working correctly, but there were no tests for it before.
2018-08-31 22:06:40 +02:00
P00 DEBUG: Storage::Local->manifest(): strFilter = [undef], strPathExp = [TEST_PATH]/db-master/db/pg_config
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: remove invalid files/paths/links from [TEST_PATH]/db-master/db/base
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/base
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
Ignore all files in a linked tablespace directory except the subdirectory for the current version of PostgreSQL. Previously an error would be generated if other files were present and not owned by the PostgreSQL user. This hasn't been a big deal in practice but it could cause issues. Also add tests to make sure the same logic applies with links to files, i.e. all other files in the directory should be ignored. This was actually working correctly, but there were no tests for it before.
2018-08-31 22:06:40 +02:00
P00 DEBUG: Storage::Local->manifest(): strFilter = [undef], strPathExp = [TEST_PATH]/db-master/db/base
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 DETAIL: remove file [TEST_PATH]/db-master/db/base/recovery.done
Add bIgnoreMissing parameter to Local->manifest().
2017-07-25 18:44:38 +02:00
P00 DEBUG: Storage::Local->remove(): bIgnoreMissing = <true>, bRecurse = <false>, xstryPathFileExp = [TEST_PATH]/db-master/db/base/recovery.done
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->remove=>: bRemoved = true
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 DETAIL: preserve file [TEST_PATH]/db-master/db/base/recovery.conf
P00 DETAIL: remove file [TEST_PATH]/db-master/db/base/postmaster.opts
Add bIgnoreMissing parameter to Local->manifest().
2017-07-25 18:44:38 +02:00
P00 DEBUG: Storage::Local->remove(): bIgnoreMissing = <true>, bRecurse = <false>, xstryPathFileExp = [TEST_PATH]/db-master/db/base/postmaster.opts
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->remove=>: bRemoved = true
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 DETAIL: remove file [TEST_PATH]/db-master/db/base/postgresql.auto.conf.tmp
Add bIgnoreMissing parameter to Local->manifest().
2017-07-25 18:44:38 +02:00
P00 DEBUG: Storage::Local->remove(): bIgnoreMissing = <true>, bRecurse = <false>, xstryPathFileExp = [TEST_PATH]/db-master/db/base/postgresql.auto.conf.tmp
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->remove=>: bRemoved = true
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
P00 DETAIL: remove link [TEST_PATH]/db-master/db/base/postgresql.auto.conf
P00 DEBUG: Storage::Local->remove(): bIgnoreMissing = <true>, bRecurse = <false>, xstryPathFileExp = [TEST_PATH]/db-master/db/base/postgresql.auto.conf
P00 DEBUG: Storage::Local->remove=>: bRemoved = true
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 DETAIL: remove file [TEST_PATH]/db-master/db/base/pg_subtrans/anything.tmp
Add bIgnoreMissing parameter to Local->manifest().
2017-07-25 18:44:38 +02:00
P00 DEBUG: Storage::Local->remove(): bIgnoreMissing = <true>, bRecurse = <false>, xstryPathFileExp = [TEST_PATH]/db-master/db/base/pg_subtrans/anything.tmp
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->remove=>: bRemoved = true
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 DETAIL: remove file [TEST_PATH]/db-master/db/base/pg_stat_tmp/anything.tmp
Add bIgnoreMissing parameter to Local->manifest().
2017-07-25 18:44:38 +02:00
P00 DEBUG: Storage::Local->remove(): bIgnoreMissing = <true>, bRecurse = <false>, xstryPathFileExp = [TEST_PATH]/db-master/db/base/pg_stat_tmp/anything.tmp
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->remove=>: bRemoved = true
Additional backup exclusions. * Exclude contents of pg_snapshots, pg_serial, pg_notify, and pg_dynshmem from backup since they are rebuilt on startup. * Exclude pg_internal.init files from backup since they are rebuilt on startup.
2017-09-04 14:26:57 +02:00
P00 DETAIL: remove file [TEST_PATH]/db-master/db/base/pg_snapshots/anything.tmp
P00 DEBUG: Storage::Local->remove(): bIgnoreMissing = <true>, bRecurse = <false>, xstryPathFileExp = [TEST_PATH]/db-master/db/base/pg_snapshots/anything.tmp
P00 DEBUG: Storage::Local->remove=>: bRemoved = true
P00 DETAIL: remove file [TEST_PATH]/db-master/db/base/pg_serial/anything.tmp
P00 DEBUG: Storage::Local->remove(): bIgnoreMissing = <true>, bRecurse = <false>, xstryPathFileExp = [TEST_PATH]/db-master/db/base/pg_serial/anything.tmp
P00 DEBUG: Storage::Local->remove=>: bRemoved = true
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 DETAIL: remove file [TEST_PATH]/db-master/db/base/pg_replslot/anything.tmp
Add bIgnoreMissing parameter to Local->manifest().
2017-07-25 18:44:38 +02:00
P00 DEBUG: Storage::Local->remove(): bIgnoreMissing = <true>, bRecurse = <false>, xstryPathFileExp = [TEST_PATH]/db-master/db/base/pg_replslot/anything.tmp
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->remove=>: bRemoved = true
Additional backup exclusions. * Exclude contents of pg_snapshots, pg_serial, pg_notify, and pg_dynshmem from backup since they are rebuilt on startup. * Exclude pg_internal.init files from backup since they are rebuilt on startup.
2017-09-04 14:26:57 +02:00
P00 DETAIL: remove file [TEST_PATH]/db-master/db/base/pg_notify/anything.tmp
P00 DEBUG: Storage::Local->remove(): bIgnoreMissing = <true>, bRecurse = <false>, xstryPathFileExp = [TEST_PATH]/db-master/db/base/pg_notify/anything.tmp
P00 DEBUG: Storage::Local->remove=>: bRemoved = true
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
P00 DETAIL: remove file [TEST_PATH]/db-master/db/base/pg_log2/logfile
P00 DEBUG: Storage::Local->remove(): bIgnoreMissing = <true>, bRecurse = <false>, xstryPathFileExp = [TEST_PATH]/db-master/db/base/pg_log2/logfile
P00 DEBUG: Storage::Local->remove=>: bRemoved = true
P00 DETAIL: remove path [TEST_PATH]/db-master/db/base/pg_log2
P00 DETAIL: remove file [TEST_PATH]/db-master/db/base/pg_log/logfile
P00 DEBUG: Storage::Local->remove(): bIgnoreMissing = <true>, bRecurse = <false>, xstryPathFileExp = [TEST_PATH]/db-master/db/base/pg_log/logfile
P00 DEBUG: Storage::Local->remove=>: bRemoved = true
Additional backup exclusions. * Exclude contents of pg_snapshots, pg_serial, pg_notify, and pg_dynshmem from backup since they are rebuilt on startup. * Exclude pg_internal.init files from backup since they are rebuilt on startup.
2017-09-04 14:26:57 +02:00
P00 DETAIL: remove file [TEST_PATH]/db-master/db/base/pg_dynshmem/anything.tmp
P00 DEBUG: Storage::Local->remove(): bIgnoreMissing = <true>, bRecurse = <false>, xstryPathFileExp = [TEST_PATH]/db-master/db/base/pg_dynshmem/anything.tmp
P00 DEBUG: Storage::Local->remove=>: bRemoved = true
P00 DETAIL: remove file [TEST_PATH]/db-master/db/base/global/pg_internal.init
P00 DEBUG: Storage::Local->remove(): bIgnoreMissing = <true>, bRecurse = <false>, xstryPathFileExp = [TEST_PATH]/db-master/db/base/global/pg_internal.init
P00 DEBUG: Storage::Local->remove=>: bRemoved = true
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 DETAIL: remove file [TEST_PATH]/db-master/db/base/deleteme/deleteme.txt
Add bIgnoreMissing parameter to Local->manifest().
2017-07-25 18:44:38 +02:00
P00 DEBUG: Storage::Local->remove(): bIgnoreMissing = <true>, bRecurse = <false>, xstryPathFileExp = [TEST_PATH]/db-master/db/base/deleteme/deleteme.txt
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->remove=>: bRemoved = true
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 DETAIL: remove path [TEST_PATH]/db-master/db/base/deleteme
P00 DETAIL: remove file [TEST_PATH]/db-master/db/base/base/pgsql_tmp/pgsql_tmp.1
Add bIgnoreMissing parameter to Local->manifest().
2017-07-25 18:44:38 +02:00
P00 DEBUG: Storage::Local->remove(): bIgnoreMissing = <true>, bRecurse = <false>, xstryPathFileExp = [TEST_PATH]/db-master/db/base/base/pgsql_tmp/pgsql_tmp.1
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->remove=>: bRemoved = true
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 DETAIL: remove path [TEST_PATH]/db-master/db/base/base/pgsql_tmp
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
P00 DETAIL: remove file [TEST_PATH]/db-master/db/base/base/32768/t333_44000
P00 DEBUG: Storage::Local->remove(): bIgnoreMissing = <true>, bRecurse = <false>, xstryPathFileExp = [TEST_PATH]/db-master/db/base/base/32768/t333_44000
P00 DEBUG: Storage::Local->remove=>: bRemoved = true
P00 DETAIL: remove file [TEST_PATH]/db-master/db/base/base/32768/44000
P00 DEBUG: Storage::Local->remove(): bIgnoreMissing = <true>, bRecurse = <false>, xstryPathFileExp = [TEST_PATH]/db-master/db/base/base/32768/44000
P00 DEBUG: Storage::Local->remove=>: bRemoved = true
Fixed an issue with invalid backup groups being set correctly on restore. If the backup cannot map a group to a name it stores the group in the manifest as false then uses either the owner of $PGDATA to set the group during restore or failing that the group of the current user. This logic was not working correctly because the selected group was overwriting the user on restore leaving the group undefined and the user incorrectly set to the group. (Reported by Jeff McCormick.)
2017-11-20 19:46:35 +02:00
P00 DETAIL: set ownership [USER-1]:[GROUP-2] on [TEST_PATH]/db-master/db/base/base/16384/PG_VERSION
P00 DEBUG: Storage::Local->owner(): strGroup = [GROUP-2], strPathFileExp = [TEST_PATH]/db-master/db/base/base/16384/PG_VERSION, strUser = [USER-1]
P00 DETAIL: set ownership [USER-2]:[GROUP-1] on [TEST_PATH]/db-master/db/base/base/1/PG_VERSION
P00 DEBUG: Storage::Local->owner(): strGroup = [GROUP-1], strPathFileExp = [TEST_PATH]/db-master/db/base/base/1/PG_VERSION, strUser = [USER-2]
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 DETAIL: set mode 0660 on [TEST_PATH]/db-master/db/base/base/1/PG_VERSION
P00 DETAIL: set mode 0700 on [TEST_PATH]/db-master/db/base/base
P00 DETAIL: remove file [TEST_PATH]/db-master/db/base/backup_label.old
Add bIgnoreMissing parameter to Local->manifest().
2017-07-25 18:44:38 +02:00
P00 DEBUG: Storage::Local->remove(): bIgnoreMissing = <true>, bRecurse = <false>, xstryPathFileExp = [TEST_PATH]/db-master/db/base/backup_label.old
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->remove=>: bRemoved = true
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
P00 INFO: cleanup removed 18 files, 1 link, 3 paths
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 DEBUG: Restore->build(): oManifest = [object]
Fix failure in manifest build when two or more files in PGDATA are linked to the same directory. Reported by Vitaliy Kukharik.
2018-05-02 18:19:54 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/pg_config
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/pg_stat
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/pg_config
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 DEBUG: build level 1 paths/links
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/base/base
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/base/global
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/base/pg_clog
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/db/base/pg_clog
P00 DEBUG: Storage::Local->exists=>: bExists = false
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = <false>, strMode = 0700, strPathExp = [TEST_PATH]/db-master/db/base/pg_clog
P00 DEBUG: Storage::Local->owner(): strGroup = [GROUP-1], strPathFileExp = [TEST_PATH]/db-master/db/base/pg_clog, strUser = [USER-1]
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/base/pg_dynshmem
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/base/pg_log
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/base/pg_notify
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/base/pg_replslot
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/base/pg_serial
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/base/pg_snapshots
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/base/pg_stat_tmp
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/base/pg_subtrans
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/base/pg_tblspc
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
Fix failure in manifest build when two or more files in PGDATA are linked to the same directory. Reported by Vitaliy Kukharik.
2018-05-02 18:19:54 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/base/pg_hba.conf
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/db/base/pg_hba.conf
P00 DEBUG: Storage::Local->exists=>: bExists = true
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/base/pg_stat
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/db/base/pg_stat
P00 DEBUG: Storage::Local->exists=>: bExists = true
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/base/postgresql.conf
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/db/base/postgresql.conf
P00 DEBUG: Storage::Local->exists=>: bExists = true
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 DEBUG: build level 2 paths/links
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/base/base/1
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/base/base/16384
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/base/base/32768
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 DEBUG: build level 3 paths/links
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Protocol::Local::Process->new(): bConfessError = <true>, iSelectTimeout = <30>, strBackRestBin = <[BACKREST-BIN]>, strHostType = backup
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->hostAdd(): iHostConfigIdx = 1, iProcessMax = 1
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/base/32768/33001, 65536, [MODIFICATION-TIME-2], 6bf316f11d28c28914ea9be92c00de9bea6d9a6b, 0, 0, pg_data/base/32768/33001, [undef], 0600, [USER-1], [GROUP-1], [MODIFICATION-TIME-5], 1, [BACKUP-FULL-2], 0), rParamSecure = [undef], strKey = pg_data/base/32768/33001, strOp = restoreFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/base/32768/44000_init, 32768, [MODIFICATION-TIME-2], 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f, 0, 0, pg_data/base/32768/44000_init, [undef], 0600, [USER-1], [GROUP-1], [MODIFICATION-TIME-5], 1, [BACKUP-FULL-2], 0), rParamSecure = [undef], strKey = pg_data/base/32768/44000_init, strOp = restoreFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/base/32768/33000.32767, 32768, [MODIFICATION-TIME-2], 21e2c7c1a326682c07053b7d6a5a40dbd49c2ec5, 0, 0, pg_data/base/32768/33000.32767, [undef], 0600, [USER-1], [GROUP-1], [MODIFICATION-TIME-5], 1, [BACKUP-FULL-2], 0), rParamSecure = [undef], strKey = pg_data/base/32768/33000.32767, strOp = restoreFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/base/32768/33000, 32768, [MODIFICATION-TIME-2], 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f, 0, 0, pg_data/base/32768/33000, [undef], 0600, [USER-1], [GROUP-1], [MODIFICATION-TIME-5], 1, [BACKUP-FULL-2], 0), rParamSecure = [undef], strKey = pg_data/base/32768/33000, strOp = restoreFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/base/16384/17000, 16384, [MODIFICATION-TIME-2], e0101dd8ffb910c9c202ca35b5f828bcb9697bed, 0, 0, pg_data/base/16384/17000, [undef], 0600, [USER-1], [GROUP-1], [MODIFICATION-TIME-5], 1, [BACKUP-FULL-2], 0), rParamSecure = [undef], strKey = pg_data/base/16384/17000, strOp = restoreFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/global/pg_control.pgbackrest.tmp, 8192, [MODIFICATION-TIME-3], 4c77c900f7af0d9ab13fa9982051a42e0b637f6c, 0, 0, pg_data/global/pg_control, [undef], 0600, [USER-1], [GROUP-1], [MODIFICATION-TIME-5], 1, [BACKUP-FULL-2], 0), rParamSecure = [undef], strKey = pg_data/global/pg_control, strOp = restoreFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/base/1/12000, 8192, [MODIFICATION-TIME-2], 22c98d248ff548311eda88559e4a8405ed77c003, 0, 0, pg_data/base/1/12000, [undef], 0600, [USER-1], [GROUP-1], [MODIFICATION-TIME-5], 1, [BACKUP-FULL-2], 0), rParamSecure = [undef], strKey = pg_data/base/1/12000, strOp = restoreFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/postgresql.conf, 21, [MODIFICATION-TIME-3], 6721d92c9fcdf4248acff1f9a1377127d9064807, 0, 0, pg_data/postgresql.conf, [undef], 0600, [USER-1], [GROUP-1], [MODIFICATION-TIME-5], 1, [BACKUP-FULL-2], 0), rParamSecure = [undef], strKey = pg_data/postgresql.conf, strOp = restoreFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/pg_hba.conf, 9, [MODIFICATION-TIME-3], dd4cea0cae348309f9de28ad4ded8ee2cc2e6d5b, 0, 0, pg_data/pg_hba.conf, [undef], 0600, [USER-1], [GROUP-1], [MODIFICATION-TIME-5], 1, [BACKUP-FULL-2], 0), rParamSecure = [undef], strKey = pg_data/pg_hba.conf, strOp = restoreFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/changecontent.txt, 7, [MODIFICATION-TIME-2], 238a131a3e8eb98d1fc5b27d882ca40b7618fd2a, 0, 0, pg_data/changecontent.txt, [undef], 0600, [USER-1], [GROUP-1], [MODIFICATION-TIME-5], 1, [BACKUP-FULL-2], 0), rParamSecure = [undef], strKey = pg_data/changecontent.txt, strOp = restoreFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/pg_stat/global.stat, 5, [MODIFICATION-TIME-3], e350d5ce0153f3e22d5db21cf2a4eff00f3ee877, 0, 0, pg_data/pg_stat/global.stat, [undef], 0600, [USER-1], [GROUP-1], [MODIFICATION-TIME-5], 1, [BACKUP-FULL-2], 0), rParamSecure = [undef], strKey = pg_data/pg_stat/global.stat, strOp = restoreFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/changetime.txt, 4, [MODIFICATION-TIME-2], 88087292ed82e26f3eb824d0bffc05ccf7a30f8d, 0, 0, pg_data/changetime.txt, [undef], 0600, [USER-1], [GROUP-1], [MODIFICATION-TIME-5], 1, [BACKUP-FULL-2], 0), rParamSecure = [undef], strKey = pg_data/changetime.txt, strOp = restoreFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/base/32768/PG_VERSION, 3, [MODIFICATION-TIME-2], 184473f470864e067ee3a22e64b47b0a1c356f29, 0, 0, pg_data/base/32768/PG_VERSION, [undef], 0600, [USER-1], [GROUP-1], [MODIFICATION-TIME-5], 1, [BACKUP-FULL-2], 0), rParamSecure = [undef], strKey = pg_data/base/32768/PG_VERSION, strOp = restoreFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/base/16384/PG_VERSION, 3, [MODIFICATION-TIME-2], 184473f470864e067ee3a22e64b47b0a1c356f29, 0, 0, pg_data/base/16384/PG_VERSION, [undef], 0600, [USER-1], root, [MODIFICATION-TIME-5], 1, [BACKUP-FULL-2], 0), rParamSecure = [undef], strKey = pg_data/base/16384/PG_VERSION, strOp = restoreFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/base/1/PG_VERSION, 3, [MODIFICATION-TIME-2], 184473f470864e067ee3a22e64b47b0a1c356f29, 0, 0, pg_data/base/1/PG_VERSION, [undef], 0660, root, [USER-1], [MODIFICATION-TIME-5], 1, [BACKUP-FULL-2], 0), rParamSecure = [undef], strKey = pg_data/base/1/PG_VERSION, strOp = restoreFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/PG_VERSION, 3, [MODIFICATION-TIME-2], 184473f470864e067ee3a22e64b47b0a1c356f29, 0, 0, pg_data/PG_VERSION, [undef], 0600, [USER-1], [GROUP-1], [MODIFICATION-TIME-5], 1, [BACKUP-FULL-2], 0), rParamSecure = [undef], strKey = pg_data/PG_VERSION, strOp = restoreFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/zero_from_start, 0, [MODIFICATION-TIME-2], [undef], 0, 0, pg_data/zero_from_start, [undef], 0600, [USER-1], [GROUP-1], [MODIFICATION-TIME-5], 1, [BACKUP-FULL-2], 0), rParamSecure = [undef], strKey = pg_data/zero_from_start, strOp = restoreFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?%, 0, [MODIFICATION-TIME-2], [undef], 0, 0, pg_data/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?%, [undef], 0600, [USER-1], [GROUP-1], [MODIFICATION-TIME-5], 1, [BACKUP-FULL-2], 0), rParamSecure = [undef], strKey = pg_data/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?%, strOp = restoreFile, strQueue = pg_data
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->hostConnect: start local process: iHostConfigIdx = 1, iHostIdx = 0, iHostProcessIdx = 0, iProcessId = 1, strHostType = backup
Add log-subprocess option to allow file logging for local and remote subprocesses.
2018-08-23 02:05:49 +02:00
P00 DEBUG: Protocol::Local::Master->new(): iProcessIdx = 1, strCommand = [BACKREST-BIN] --command=restore --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --host-id=1 --lock-path=[TEST_PATH]/db-master/lock --log-level-file=trace --log-path=[TEST_PATH]/db-master/log --log-subprocess --pg1-path=[TEST_PATH]/db-master/db/base --process=1 --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --stanza=db --type=backup local
P00 DEBUG: Protocol::Command::Master->new(): iBufferMax = 4194304, iCompressLevel = 3, iCompressLevelNetwork = 3, iProtocolTimeout = 60, strCommand = [BACKREST-BIN] --command=restore --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --host-id=1 --lock-path=[TEST_PATH]/db-master/lock --log-level-file=trace --log-path=[TEST_PATH]/db-master/log --log-subprocess --pg1-path=[TEST_PATH]/db-master/db/base --process=1 --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --stanza=db --type=backup local, strId = local-1 process, strName = local
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->hostConnect=>: bResult = true
P00 DEBUG: Protocol::Local::Process->init: init local process: iDirection = 1, iHostIdx = 0, iProcessId = 1, iQueueIdx = 0, iQueueLastIdx = 0
P00 DEBUG: Protocol::Local::Process->init=>: bResult = true
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/base/32768/33001, strQueueIdx = 0
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (0), strKey = pg_data/base/32768/33001
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/base/32768/44000_init, strQueueIdx = 0
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: RestoreFile::restoreLog(): bCopy = false, bForce = false, bZero = false, iLocalId = 1, lModificationTime = [MODIFICATION-TIME-2], lSize = 65536, lSizeCurrent = 0, lSizeTotal = 196666, strChecksum = 6bf316f11d28c28914ea9be92c00de9bea6d9a6b, strDbFile = [TEST_PATH]/db-master/db/base/base/32768/33001
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/base/32768/33001 - exists and matches backup (64KB, 33%) checksum 6bf316f11d28c28914ea9be92c00de9bea6d9a6b
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (0), strKey = pg_data/base/32768/44000_init
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/base/32768/33000.32767, strQueueIdx = 0
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: RestoreFile::restoreLog(): bCopy = false, bForce = false, bZero = false, iLocalId = 1, lModificationTime = [MODIFICATION-TIME-2], lSize = 32768, lSizeCurrent = 65536, lSizeTotal = 196666, strChecksum = 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f, strDbFile = [TEST_PATH]/db-master/db/base/base/32768/44000_init
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/base/32768/44000_init - exists and matches backup (32KB, 49%) checksum 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (0), strKey = pg_data/base/32768/33000.32767
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/base/32768/33000, strQueueIdx = 0
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: RestoreFile::restoreLog(): bCopy = false, bForce = false, bZero = false, iLocalId = 1, lModificationTime = [MODIFICATION-TIME-2], lSize = 32768, lSizeCurrent = 98304, lSizeTotal = 196666, strChecksum = 21e2c7c1a326682c07053b7d6a5a40dbd49c2ec5, strDbFile = [TEST_PATH]/db-master/db/base/base/32768/33000.32767
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/base/32768/33000.32767 - exists and matches backup (32KB, 66%) checksum 21e2c7c1a326682c07053b7d6a5a40dbd49c2ec5
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (0), strKey = pg_data/base/32768/33000
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/base/16384/17000, strQueueIdx = 0
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: RestoreFile::restoreLog(): bCopy = false, bForce = false, bZero = false, iLocalId = 1, lModificationTime = [MODIFICATION-TIME-2], lSize = 32768, lSizeCurrent = 131072, lSizeTotal = 196666, strChecksum = 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f, strDbFile = [TEST_PATH]/db-master/db/base/base/32768/33000
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/base/32768/33000 - exists and matches backup (32KB, 83%) checksum 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1), strKey = pg_data/base/16384/17000
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/global/pg_control, strQueueIdx = 0
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: RestoreFile::restoreLog(): bCopy = true, bForce = false, bZero = false, iLocalId = 1, lModificationTime = [MODIFICATION-TIME-2], lSize = 16384, lSizeCurrent = 163840, lSizeTotal = 196666, strChecksum = e0101dd8ffb910c9c202ca35b5f828bcb9697bed, strDbFile = [TEST_PATH]/db-master/db/base/base/16384/17000
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base/base/16384/17000 (16KB, 91%) checksum e0101dd8ffb910c9c202ca35b5f828bcb9697bed
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1), strKey = pg_data/global/pg_control
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/base/1/12000, strQueueIdx = 0
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: RestoreFile::restoreLog(): bCopy = true, bForce = false, bZero = false, iLocalId = 1, lModificationTime = [MODIFICATION-TIME-3], lSize = 8192, lSizeCurrent = 180224, lSizeTotal = 196666, strChecksum = 4c77c900f7af0d9ab13fa9982051a42e0b637f6c, strDbFile = [TEST_PATH]/db-master/db/base/global/pg_control.pgbackrest.tmp
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 11:24:42 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base/global/pg_control.pgbackrest.tmp (8KB, 95%) checksum 4c77c900f7af0d9ab13fa9982051a42e0b637f6c
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (0), strKey = pg_data/base/1/12000
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/postgresql.conf, strQueueIdx = 0
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: RestoreFile::restoreLog(): bCopy = false, bForce = false, bZero = false, iLocalId = 1, lModificationTime = [MODIFICATION-TIME-2], lSize = 8192, lSizeCurrent = 188416, lSizeTotal = 196666, strChecksum = 22c98d248ff548311eda88559e4a8405ed77c003, strDbFile = [TEST_PATH]/db-master/db/base/base/1/12000
Add more realistic data files to synthetic backup and restore tests.
2016-12-10 16:13:55 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/base/1/12000 - exists and matches backup (8KB, 99%) checksum 22c98d248ff548311eda88559e4a8405ed77c003
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (0), strKey = pg_data/postgresql.conf
Fix failure in manifest build when two or more files in PGDATA are linked to the same directory. Reported by Vitaliy Kukharik.
2018-05-02 18:19:54 +02:00
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/pg_hba.conf, strQueueIdx = 0
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: RestoreFile::restoreLog(): bCopy = false, bForce = false, bZero = false, iLocalId = 1, lModificationTime = [MODIFICATION-TIME-3], lSize = 21, lSizeCurrent = 196608, lSizeTotal = 196666, strChecksum = 6721d92c9fcdf4248acff1f9a1377127d9064807, strDbFile = [TEST_PATH]/db-master/db/base/postgresql.conf
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/postgresql.conf - exists and matches backup (21B, 99%) checksum 6721d92c9fcdf4248acff1f9a1377127d9064807
Fix failure in manifest build when two or more files in PGDATA are linked to the same directory. Reported by Vitaliy Kukharik.
2018-05-02 18:19:54 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (0), strKey = pg_data/pg_hba.conf
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/changecontent.txt, strQueueIdx = 0
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: RestoreFile::restoreLog(): bCopy = false, bForce = false, bZero = false, iLocalId = 1, lModificationTime = [MODIFICATION-TIME-3], lSize = 9, lSizeCurrent = 196629, lSizeTotal = 196666, strChecksum = dd4cea0cae348309f9de28ad4ded8ee2cc2e6d5b, strDbFile = [TEST_PATH]/db-master/db/base/pg_hba.conf
Fix failure in manifest build when two or more files in PGDATA are linked to the same directory. Reported by Vitaliy Kukharik.
2018-05-02 18:19:54 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/pg_hba.conf - exists and matches backup (9B, 99%) checksum dd4cea0cae348309f9de28ad4ded8ee2cc2e6d5b
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (0), strKey = pg_data/changecontent.txt
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/pg_stat/global.stat, strQueueIdx = 0
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: RestoreFile::restoreLog(): bCopy = false, bForce = false, bZero = false, iLocalId = 1, lModificationTime = [MODIFICATION-TIME-2], lSize = 7, lSizeCurrent = 196638, lSizeTotal = 196666, strChecksum = 238a131a3e8eb98d1fc5b27d882ca40b7618fd2a, strDbFile = [TEST_PATH]/db-master/db/base/changecontent.txt
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/changecontent.txt - exists and matches backup (7B, 99%) checksum 238a131a3e8eb98d1fc5b27d882ca40b7618fd2a
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (0), strKey = pg_data/pg_stat/global.stat
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/changetime.txt, strQueueIdx = 0
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: RestoreFile::restoreLog(): bCopy = false, bForce = false, bZero = false, iLocalId = 1, lModificationTime = [MODIFICATION-TIME-3], lSize = 5, lSizeCurrent = 196645, lSizeTotal = 196666, strChecksum = e350d5ce0153f3e22d5db21cf2a4eff00f3ee877, strDbFile = [TEST_PATH]/db-master/db/base/pg_stat/global.stat
Add more realistic data files to synthetic backup and restore tests.
2016-12-10 16:13:55 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/pg_stat/global.stat - exists and matches backup (5B, 99%) checksum e350d5ce0153f3e22d5db21cf2a4eff00f3ee877
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (0), strKey = pg_data/changetime.txt
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/base/32768/PG_VERSION, strQueueIdx = 0
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: RestoreFile::restoreLog(): bCopy = false, bForce = false, bZero = false, iLocalId = 1, lModificationTime = [MODIFICATION-TIME-2], lSize = 4, lSizeCurrent = 196650, lSizeTotal = 196666, strChecksum = 88087292ed82e26f3eb824d0bffc05ccf7a30f8d, strDbFile = [TEST_PATH]/db-master/db/base/changetime.txt
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/changetime.txt - exists and matches backup (4B, 99%) checksum 88087292ed82e26f3eb824d0bffc05ccf7a30f8d
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (0), strKey = pg_data/base/32768/PG_VERSION
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/base/16384/PG_VERSION, strQueueIdx = 0
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: RestoreFile::restoreLog(): bCopy = false, bForce = false, bZero = false, iLocalId = 1, lModificationTime = [MODIFICATION-TIME-2], lSize = 3, lSizeCurrent = 196654, lSizeTotal = 196666, strChecksum = 184473f470864e067ee3a22e64b47b0a1c356f29, strDbFile = [TEST_PATH]/db-master/db/base/base/32768/PG_VERSION
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/base/32768/PG_VERSION - exists and matches backup (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (0), strKey = pg_data/base/16384/PG_VERSION
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/base/1/PG_VERSION, strQueueIdx = 0
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: RestoreFile::restoreLog(): bCopy = false, bForce = false, bZero = false, iLocalId = 1, lModificationTime = [MODIFICATION-TIME-2], lSize = 3, lSizeCurrent = 196657, lSizeTotal = 196666, strChecksum = 184473f470864e067ee3a22e64b47b0a1c356f29, strDbFile = [TEST_PATH]/db-master/db/base/base/16384/PG_VERSION
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/base/16384/PG_VERSION - exists and matches backup (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (0), strKey = pg_data/base/1/PG_VERSION
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/PG_VERSION, strQueueIdx = 0
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: RestoreFile::restoreLog(): bCopy = false, bForce = false, bZero = false, iLocalId = 1, lModificationTime = [MODIFICATION-TIME-2], lSize = 3, lSizeCurrent = 196660, lSizeTotal = 196666, strChecksum = 184473f470864e067ee3a22e64b47b0a1c356f29, strDbFile = [TEST_PATH]/db-master/db/base/base/1/PG_VERSION
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/base/1/PG_VERSION - exists and matches backup (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (0), strKey = pg_data/PG_VERSION
Allow zero-size files in backup manifest to reference a prior manifest regardless of timestamp delta. Contributed by Cynthia Shang.
2018-08-24 22:50:33 +02:00
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/zero_from_start, strQueueIdx = 0
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: RestoreFile::restoreLog(): bCopy = false, bForce = false, bZero = false, iLocalId = 1, lModificationTime = [MODIFICATION-TIME-2], lSize = 3, lSizeCurrent = 196663, lSizeTotal = 196666, strChecksum = 184473f470864e067ee3a22e64b47b0a1c356f29, strDbFile = [TEST_PATH]/db-master/db/base/PG_VERSION
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/PG_VERSION - exists and matches backup (3B, 100%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
Allow zero-size files in backup manifest to reference a prior manifest regardless of timestamp delta. Contributed by Cynthia Shang.
2018-08-24 22:50:33 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (0), strKey = pg_data/zero_from_start
Improve support for special characters in filenames. % characters caused issues in backup/restore due to filenames being appended directly into a format string. Reserved XML characters (<>&') caused issues in the S3 driver due to improper escaping. Add a file with all common special characters to regression testing.
2018-09-10 16:54:34 +02:00
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?%, strQueueIdx = 0
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: RestoreFile::restoreLog(): bCopy = false, bForce = false, bZero = false, iLocalId = 1, lModificationTime = [MODIFICATION-TIME-2], lSize = 0, lSizeCurrent = 196666, lSizeTotal = 196666, strChecksum = [undef], strDbFile = [TEST_PATH]/db-master/db/base/zero_from_start
Allow zero-size files in backup manifest to reference a prior manifest regardless of timestamp delta. Contributed by Cynthia Shang.
2018-08-24 22:50:33 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/zero_from_start - exists and is zero size (0B, 100%)
Improve support for special characters in filenames. % characters caused issues in backup/restore due to filenames being appended directly into a format string. Reserved XML characters (<>&') caused issues in the S3 driver due to improper escaping. Add a file with all common special characters to regression testing.
2018-09-10 16:54:34 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (0), strKey = pg_data/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?%
P00 DEBUG: Protocol::Local::Process->process: no jobs found, stop local: iHostConfigIdx = [undef], iHostIdx = 0, iProcessId = 1, strHostType = [undef]
P00 DEBUG: Protocol::Command::Master->close=>: iExitStatus = 0
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: RestoreFile::restoreLog(): bCopy = false, bForce = false, bZero = false, iLocalId = 1, lModificationTime = [MODIFICATION-TIME-2], lSize = 0, lSizeCurrent = 196666, lSizeTotal = 196666, strChecksum = [undef], strDbFile = [TEST_PATH]/db-master/db/base/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?%
Improve support for special characters in filenames. % characters caused issues in backup/restore due to filenames being appended directly into a format string. Reserved XML characters (<>&') caused issues in the S3 driver due to improper escaping. Add a file with all common special characters to regression testing.
2018-09-10 16:54:34 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?% - exists and is zero size (0B, 100%)
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->process: all jobs complete
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/db/base/recovery.conf
P00 DEBUG: Storage::Local->exists=>: bExists = true
Add bIgnoreMissing parameter to Local->manifest().
2017-07-25 18:44:38 +02:00
P00 DEBUG: Storage::Local->remove(): bIgnoreMissing = <true>, bRecurse = <false>, xstryPathFileExp = [TEST_PATH]/db-master/db/base/recovery.conf
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->remove=>: bRemoved = true
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P00 INFO: write [TEST_PATH]/db-master/db/base/recovery.conf
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/db/base/base
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/db/base/base/1
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/db/base/base/16384
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/db/base/base/32768
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/db/base/global
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/db/base/pg_clog
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/db/base/pg_dynshmem
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/db/base/pg_log
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/db/base/pg_notify
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/db/base/pg_replslot
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/db/base/pg_serial
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/db/base/pg_snapshots
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/db/base/pg_stat_tmp
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/db/base/pg_subtrans
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/db/base/pg_tblspc
Fix failure in manifest build when two or more files in PGDATA are linked to the same directory. Reported by Vitaliy Kukharik.
2018-05-02 18:19:54 +02:00
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/db/pg_config
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/db/pg_stat
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/db/pg_config
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 INFO: restore global/pg_control (performed last to ensure aborted restores cannot be started)
P00 DEBUG: Storage::Local->move(): bPathCreate = <false>, strDestinationPathExp = [TEST_PATH]/db-master/db/base/global/pg_control, strSourcePathExp = [TEST_PATH]/db-master/db/base/global/pg_control.pgbackrest.tmp
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/db/base
Add bIgnoreMissing parameter to Local->manifest().
2017-07-25 18:44:38 +02:00
P00 DEBUG: Storage::Local->remove(): bIgnoreMissing = <true>, bRecurse = <false>, xstryPathFileExp = [TEST_PATH]/db-master/db/base/backup.manifest
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->remove=>: bRemoved = true
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/db/base
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: perl/exec::perlExec: => 0
P00 DEBUG: common/exit::exitSafe: (result: 0, error: false, signalType: 0)
P00 DEBUG: common/lock::lockRelease: (failOnNoLock: false)
P00 DEBUG: common/lock::lockRelease: => false
Move async forking and more error handling to C. The Perl process was exiting directly when called but that interfered with proper locking for the forked async process. Now Perl returns results to the C process which handles all errors, including signals.
2018-04-13 02:42:26 +02:00
P00 DEBUG: Main::mainCleanup(): iExitCode = 0
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Helper::protocolDestroy(): bComplete = true, iRemoteIdx = [undef], strRemoteType = [undef]
P00 DEBUG: Protocol::Helper::protocolDestroy=>: iExitStatus = 0
Added a predictable log entry to signal that a command has completed successfully. For example a backup ends successfully with: INFO: backup command end: completed successfully.
2016-12-04 21:44:53 +02:00
P00 INFO: restore command end: completed successfully
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: common/exit::exitSafe: => 0
P00 DEBUG: main::main: => 0
Log testing can now be enabled for certain deterministic tests. This works by comparing the generated logs against a previous copy. Currently only enabled for the backup/synthetic tests.
2015-05-05 19:08:48 +02:00
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
+ supplemental file: [TEST_PATH]/db-master/db/base/recovery.conf
----------------------------------------------------------------
restore_command = '[BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --stanza=db archive-get %f "%p"'
Added vagrant configuration for Ubuntu 12.04. Changes to regression tests to work better with VMs.
2015-07-07 03:00:48 +02:00
Fixed an issue with invalid backup groups being set correctly on restore. If the backup cannot map a group to a name it stores the group in the manifest as false then uses either the owner of $PGDATA to set the group during restore or failing that the group of the current user. This logic was not working correctly because the selected group was overwriting the user on restore leaving the group undefined and the user incorrectly set to the group. (Reported by Jeff McCormick.)
2017-11-20 19:46:35 +02:00
restore delta, backup '[BACKUP-FULL-2]' - fix permissions (db-master host)
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --delta --set=[BACKUP-FULL-2] --link-all --log-level-console=detail --stanza=db restore
------------------------------------------------------------------------------------------------------------------------------------
Add log-subprocess option to allow file logging for local and remote subprocesses.
2018-08-23 02:05:49 +02:00
P00 INFO: restore command begin [BACKREST-VERSION]: --no-compress --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --delta --link-all --lock-path=[TEST_PATH]/db-master/lock --log-level-console=detail --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --pg1-path=[TEST_PATH]/db-master/db/base --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --set=[BACKUP-FULL-2] --stanza=db
Fixed an issue with invalid backup groups being set correctly on restore. If the backup cannot map a group to a name it stores the group in the manifest as false then uses either the owner of $PGDATA to set the group during restore or failing that the group of the current user. This logic was not working correctly because the selected group was overwriting the user on restore leaving the group undefined and the user incorrectly set to the group. (Reported by Jeff McCormick.)
2017-11-20 19:46:35 +02:00
P00 INFO: restore backup set [BACKUP-FULL-2]
P00 WARN: backup group for pg_data/base/16384/PG_VERSION was not mapped to a name, set to [GROUP-1]
P00 WARN: backup user for pg_data/base/1/PG_VERSION was not mapped to a name, set to [USER-1]
P00 DETAIL: check [TEST_PATH]/db-master/db/base exists
Fix failure in manifest build when two or more files in PGDATA are linked to the same directory. Reported by Vitaliy Kukharik.
2018-05-02 18:19:54 +02:00
P00 DETAIL: check [TEST_PATH]/db-master/db/pg_config exists
Fixed an issue with invalid backup groups being set correctly on restore. If the backup cannot map a group to a name it stores the group in the manifest as false then uses either the owner of $PGDATA to set the group during restore or failing that the group of the current user. This logic was not working correctly because the selected group was overwriting the user on restore leaving the group undefined and the user incorrectly set to the group. (Reported by Jeff McCormick.)
2017-11-20 19:46:35 +02:00
P00 DETAIL: check [TEST_PATH]/db-master/db/pg_stat exists
P00 DETAIL: check [TEST_PATH]/db-master/db/pg_config exists
P00 INFO: remove invalid files/paths/links from [TEST_PATH]/db-master/db/pg_config
P00 INFO: remove invalid files/paths/links from [TEST_PATH]/db-master/db/pg_stat
Fix failure in manifest build when two or more files in PGDATA are linked to the same directory. Reported by Vitaliy Kukharik.
2018-05-02 18:19:54 +02:00
P00 INFO: remove invalid files/paths/links from [TEST_PATH]/db-master/db/pg_config
Fixed an issue with invalid backup groups being set correctly on restore. If the backup cannot map a group to a name it stores the group in the manifest as false then uses either the owner of $PGDATA to set the group during restore or failing that the group of the current user. This logic was not working correctly because the selected group was overwriting the user on restore leaving the group undefined and the user incorrectly set to the group. (Reported by Jeff McCormick.)
2017-11-20 19:46:35 +02:00
P00 INFO: remove invalid files/paths/links from [TEST_PATH]/db-master/db/base
P00 DETAIL: preserve file [TEST_PATH]/db-master/db/base/recovery.conf
P00 DETAIL: set ownership [USER-1]:[GROUP-1] on [TEST_PATH]/db-master/db/base/base/16384/PG_VERSION
P00 DETAIL: set ownership [USER-1]:[GROUP-1] on [TEST_PATH]/db-master/db/base/base/1/PG_VERSION
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/base/32768/33001 - exists and matches backup (64KB, 33%) checksum 6bf316f11d28c28914ea9be92c00de9bea6d9a6b
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/base/32768/44000_init - exists and matches backup (32KB, 49%) checksum 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/base/32768/33000.32767 - exists and matches backup (32KB, 66%) checksum 21e2c7c1a326682c07053b7d6a5a40dbd49c2ec5
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/base/32768/33000 - exists and matches backup (32KB, 83%) checksum 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/base/16384/17000 - exists and matches backup (16KB, 91%) checksum e0101dd8ffb910c9c202ca35b5f828bcb9697bed
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 11:24:42 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base/global/pg_control.pgbackrest.tmp (8KB, 95%) checksum 4c77c900f7af0d9ab13fa9982051a42e0b637f6c
Fixed an issue with invalid backup groups being set correctly on restore. If the backup cannot map a group to a name it stores the group in the manifest as false then uses either the owner of $PGDATA to set the group during restore or failing that the group of the current user. This logic was not working correctly because the selected group was overwriting the user on restore leaving the group undefined and the user incorrectly set to the group. (Reported by Jeff McCormick.)
2017-11-20 19:46:35 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/base/1/12000 - exists and matches backup (8KB, 99%) checksum 22c98d248ff548311eda88559e4a8405ed77c003
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/postgresql.conf - exists and matches backup (21B, 99%) checksum 6721d92c9fcdf4248acff1f9a1377127d9064807
Fix failure in manifest build when two or more files in PGDATA are linked to the same directory. Reported by Vitaliy Kukharik.
2018-05-02 18:19:54 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/pg_hba.conf - exists and matches backup (9B, 99%) checksum dd4cea0cae348309f9de28ad4ded8ee2cc2e6d5b
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/changecontent.txt - exists and matches backup (7B, 99%) checksum 238a131a3e8eb98d1fc5b27d882ca40b7618fd2a
Fixed an issue with invalid backup groups being set correctly on restore. If the backup cannot map a group to a name it stores the group in the manifest as false then uses either the owner of $PGDATA to set the group during restore or failing that the group of the current user. This logic was not working correctly because the selected group was overwriting the user on restore leaving the group undefined and the user incorrectly set to the group. (Reported by Jeff McCormick.)
2017-11-20 19:46:35 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/pg_stat/global.stat - exists and matches backup (5B, 99%) checksum e350d5ce0153f3e22d5db21cf2a4eff00f3ee877
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/changetime.txt - exists and matches backup (4B, 99%) checksum 88087292ed82e26f3eb824d0bffc05ccf7a30f8d
Fixed an issue with invalid backup groups being set correctly on restore. If the backup cannot map a group to a name it stores the group in the manifest as false then uses either the owner of $PGDATA to set the group during restore or failing that the group of the current user. This logic was not working correctly because the selected group was overwriting the user on restore leaving the group undefined and the user incorrectly set to the group. (Reported by Jeff McCormick.)
2017-11-20 19:46:35 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/base/32768/PG_VERSION - exists and matches backup (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/base/16384/PG_VERSION - exists and matches backup (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/base/1/PG_VERSION - exists and matches backup (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/PG_VERSION - exists and matches backup (3B, 100%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
Allow zero-size files in backup manifest to reference a prior manifest regardless of timestamp delta. Contributed by Cynthia Shang.
2018-08-24 22:50:33 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/zero_from_start - exists and is zero size (0B, 100%)
Improve support for special characters in filenames. % characters caused issues in backup/restore due to filenames being appended directly into a format string. Reserved XML characters (<>&') caused issues in the S3 driver due to improper escaping. Add a file with all common special characters to regression testing.
2018-09-10 16:54:34 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?% - exists and is zero size (0B, 100%)
Fixed an issue with invalid backup groups being set correctly on restore. If the backup cannot map a group to a name it stores the group in the manifest as false then uses either the owner of $PGDATA to set the group during restore or failing that the group of the current user. This logic was not working correctly because the selected group was overwriting the user on restore leaving the group undefined and the user incorrectly set to the group. (Reported by Jeff McCormick.)
2017-11-20 19:46:35 +02:00
P00 INFO: write [TEST_PATH]/db-master/db/base/recovery.conf
P00 INFO: restore global/pg_control (performed last to ensure aborted restores cannot be started)
P00 INFO: restore command end: completed successfully
+ supplemental file: [TEST_PATH]/db-master/db/base/recovery.conf
----------------------------------------------------------------
restore_command = '[BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --log-level-console=detail --stanza=db archive-get %f "%p"'
User/group permissions improvements. Improved handling of users/groups captured during backup that do not exist on the restore host. Also explicitly handle the case where user/group is not mapped to a name.
2016-06-27 03:01:20 +02:00
restore delta, backup '[BACKUP-FULL-2]' - fix broken symlink (db-master host)
Disable gzip filter when --compress-level-network=0. The filter was used with compress level set to 0 which added overhead without any benefit.
2017-11-21 16:31:15 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --delta --set=[BACKUP-FULL-2] --link-all --log-level-console=detail --stanza=db restore
User/group permissions improvements. Improved handling of users/groups captured during backup that do not exist on the restore host. Also explicitly handle the case where user/group is not mapped to a name.
2016-06-27 03:01:20 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Add log-subprocess option to allow file logging for local and remote subprocesses.
2018-08-23 02:05:49 +02:00
P00 INFO: restore command begin [BACKREST-VERSION]: --no-compress --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --delta --link-all --lock-path=[TEST_PATH]/db-master/lock --log-level-console=detail --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --pg1-path=[TEST_PATH]/db-master/db/base --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --set=[BACKUP-FULL-2] --stanza=db
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: restore backup set [BACKUP-FULL-2]
Continuous integration using travis-ci.
2017-02-21 15:59:23 +02:00
P00 WARN: backup group for pg_data/base/16384/PG_VERSION was not mapped to a name, set to [GROUP-1]
Regression tests can now be run as any properly-configured user, not just vagrant.
2016-10-01 19:39:44 +02:00
P00 WARN: backup user for pg_data/base/1/PG_VERSION was not mapped to a name, set to [USER-1]
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 DETAIL: check [TEST_PATH]/db-master/db/base exists
Fix failure in manifest build when two or more files in PGDATA are linked to the same directory. Reported by Vitaliy Kukharik.
2018-05-02 18:19:54 +02:00
P00 DETAIL: check [TEST_PATH]/db-master/db/pg_config exists
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 DETAIL: check [TEST_PATH]/db-master/db/pg_stat exists
P00 DETAIL: check [TEST_PATH]/db-master/db/pg_config exists
P00 INFO: remove invalid files/paths/links from [TEST_PATH]/db-master/db/pg_config
P00 INFO: remove invalid files/paths/links from [TEST_PATH]/db-master/db/pg_stat
Fix failure in manifest build when two or more files in PGDATA are linked to the same directory. Reported by Vitaliy Kukharik.
2018-05-02 18:19:54 +02:00
P00 INFO: remove invalid files/paths/links from [TEST_PATH]/db-master/db/pg_config
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: remove invalid files/paths/links from [TEST_PATH]/db-master/db/base
P00 DETAIL: preserve file [TEST_PATH]/db-master/db/base/recovery.conf
P00 DETAIL: remove link [TEST_PATH]/db-master/db/base/pg_stat - destination changed
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/base/32768/33001 - exists and matches backup (64KB, 33%) checksum 6bf316f11d28c28914ea9be92c00de9bea6d9a6b
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/base/32768/44000_init - exists and matches backup (32KB, 49%) checksum 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/base/32768/33000.32767 - exists and matches backup (32KB, 66%) checksum 21e2c7c1a326682c07053b7d6a5a40dbd49c2ec5
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/base/32768/33000 - exists and matches backup (32KB, 83%) checksum 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/base/16384/17000 - exists and matches backup (16KB, 91%) checksum e0101dd8ffb910c9c202ca35b5f828bcb9697bed
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 11:24:42 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base/global/pg_control.pgbackrest.tmp (8KB, 95%) checksum 4c77c900f7af0d9ab13fa9982051a42e0b637f6c
Add more realistic data files to synthetic backup and restore tests.
2016-12-10 16:13:55 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/base/1/12000 - exists and matches backup (8KB, 99%) checksum 22c98d248ff548311eda88559e4a8405ed77c003
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/postgresql.conf - exists and matches backup (21B, 99%) checksum 6721d92c9fcdf4248acff1f9a1377127d9064807
Fix failure in manifest build when two or more files in PGDATA are linked to the same directory. Reported by Vitaliy Kukharik.
2018-05-02 18:19:54 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/pg_hba.conf - exists and matches backup (9B, 99%) checksum dd4cea0cae348309f9de28ad4ded8ee2cc2e6d5b
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/changecontent.txt - exists and matches backup (7B, 99%) checksum 238a131a3e8eb98d1fc5b27d882ca40b7618fd2a
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/pg_stat/global.stat - exists and matches backup (5B, 99%) checksum e350d5ce0153f3e22d5db21cf2a4eff00f3ee877
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/changetime.txt - exists and matches backup (4B, 99%) checksum 88087292ed82e26f3eb824d0bffc05ccf7a30f8d
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/base/32768/PG_VERSION - exists and matches backup (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/base/16384/PG_VERSION - exists and matches backup (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/base/1/PG_VERSION - exists and matches backup (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/PG_VERSION - exists and matches backup (3B, 100%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
Allow zero-size files in backup manifest to reference a prior manifest regardless of timestamp delta. Contributed by Cynthia Shang.
2018-08-24 22:50:33 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/zero_from_start - exists and is zero size (0B, 100%)
Improve support for special characters in filenames. % characters caused issues in backup/restore due to filenames being appended directly into a format string. Reserved XML characters (<>&') caused issues in the S3 driver due to improper escaping. Add a file with all common special characters to regression testing.
2018-09-10 16:54:34 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?% - exists and is zero size (0B, 100%)
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P00 INFO: write [TEST_PATH]/db-master/db/base/recovery.conf
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 INFO: restore global/pg_control (performed last to ensure aborted restores cannot be started)
Added a predictable log entry to signal that a command has completed successfully. For example a backup ends successfully with: INFO: backup command end: completed successfully.
2016-12-04 21:44:53 +02:00
P00 INFO: restore command end: completed successfully
User/group permissions improvements. Improved handling of users/groups captured during backup that do not exist on the restore host. Also explicitly handle the case where user/group is not mapped to a name.
2016-06-27 03:01:20 +02:00
+ supplemental file: [TEST_PATH]/db-master/db/base/recovery.conf
----------------------------------------------------------------
restore_command = '[BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --log-level-console=detail --stanza=db archive-get %f "%p"'
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
restore delta, backup '[BACKUP-FULL-2]' - restore all links by mapping (db-master host)
Fix failure in manifest build when two or more files in PGDATA are linked to the same directory. Reported by Vitaliy Kukharik.
2018-05-02 18:19:54 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --delta --set=[BACKUP-FULL-2] --log-level-console=detail --link-map=pg_stat=../pg_stat --link-map=postgresql.conf=../pg_config/postgresql.conf --link-map=pg_hba.conf=../pg_config/pg_hba.conf --stanza=db restore
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Add log-subprocess option to allow file logging for local and remote subprocesses.
2018-08-23 02:05:49 +02:00
P00 INFO: restore command begin [BACKREST-VERSION]: --no-compress --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --delta --link-map=pg_stat=../pg_stat --link-map=postgresql.conf=../pg_config/postgresql.conf --link-map=pg_hba.conf=../pg_config/pg_hba.conf --lock-path=[TEST_PATH]/db-master/lock --log-level-console=detail --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --pg1-path=[TEST_PATH]/db-master/db/base --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --set=[BACKUP-FULL-2] --stanza=db
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: restore backup set [BACKUP-FULL-2]
Continuous integration using travis-ci.
2017-02-21 15:59:23 +02:00
P00 WARN: backup group for pg_data/base/16384/PG_VERSION was not mapped to a name, set to [GROUP-1]
Regression tests can now be run as any properly-configured user, not just vagrant.
2016-10-01 19:39:44 +02:00
P00 WARN: backup user for pg_data/base/1/PG_VERSION was not mapped to a name, set to [USER-1]
Fix failure in manifest build when two or more files in PGDATA are linked to the same directory. Reported by Vitaliy Kukharik.
2018-05-02 18:19:54 +02:00
P00 INFO: remap link pg_data/pg_hba.conf destination to ../pg_config/pg_hba.conf
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: remap link pg_data/pg_stat destination to ../pg_stat
P00 INFO: remap link pg_data/postgresql.conf destination to ../pg_config/postgresql.conf
P00 DETAIL: check [TEST_PATH]/db-master/db/base exists
Fix failure in manifest build when two or more files in PGDATA are linked to the same directory. Reported by Vitaliy Kukharik.
2018-05-02 18:19:54 +02:00
P00 DETAIL: check [TEST_PATH]/db-master/db/pg_config exists
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 DETAIL: check [TEST_PATH]/db-master/db/pg_stat exists
P00 DETAIL: check [TEST_PATH]/db-master/db/pg_config exists
P00 INFO: remove invalid files/paths/links from [TEST_PATH]/db-master/db/pg_config
P00 INFO: remove invalid files/paths/links from [TEST_PATH]/db-master/db/pg_stat
Fix failure in manifest build when two or more files in PGDATA are linked to the same directory. Reported by Vitaliy Kukharik.
2018-05-02 18:19:54 +02:00
P00 INFO: remove invalid files/paths/links from [TEST_PATH]/db-master/db/pg_config
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: remove invalid files/paths/links from [TEST_PATH]/db-master/db/base
P00 DETAIL: preserve file [TEST_PATH]/db-master/db/base/recovery.conf
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/base/32768/33001 - exists and matches backup (64KB, 33%) checksum 6bf316f11d28c28914ea9be92c00de9bea6d9a6b
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/base/32768/44000_init - exists and matches backup (32KB, 49%) checksum 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/base/32768/33000.32767 - exists and matches backup (32KB, 66%) checksum 21e2c7c1a326682c07053b7d6a5a40dbd49c2ec5
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/base/32768/33000 - exists and matches backup (32KB, 83%) checksum 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/base/16384/17000 - exists and matches backup (16KB, 91%) checksum e0101dd8ffb910c9c202ca35b5f828bcb9697bed
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 11:24:42 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base/global/pg_control.pgbackrest.tmp (8KB, 95%) checksum 4c77c900f7af0d9ab13fa9982051a42e0b637f6c
Add more realistic data files to synthetic backup and restore tests.
2016-12-10 16:13:55 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/base/1/12000 - exists and matches backup (8KB, 99%) checksum 22c98d248ff548311eda88559e4a8405ed77c003
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/postgresql.conf - exists and matches backup (21B, 99%) checksum 6721d92c9fcdf4248acff1f9a1377127d9064807
Fix failure in manifest build when two or more files in PGDATA are linked to the same directory. Reported by Vitaliy Kukharik.
2018-05-02 18:19:54 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/pg_hba.conf - exists and matches backup (9B, 99%) checksum dd4cea0cae348309f9de28ad4ded8ee2cc2e6d5b
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/changecontent.txt - exists and matches backup (7B, 99%) checksum 238a131a3e8eb98d1fc5b27d882ca40b7618fd2a
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/pg_stat/global.stat - exists and matches backup (5B, 99%) checksum e350d5ce0153f3e22d5db21cf2a4eff00f3ee877
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/changetime.txt - exists and matches backup (4B, 99%) checksum 88087292ed82e26f3eb824d0bffc05ccf7a30f8d
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/base/32768/PG_VERSION - exists and matches backup (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/base/16384/PG_VERSION - exists and matches backup (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/base/1/PG_VERSION - exists and matches backup (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/PG_VERSION - exists and matches backup (3B, 100%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
Allow zero-size files in backup manifest to reference a prior manifest regardless of timestamp delta. Contributed by Cynthia Shang.
2018-08-24 22:50:33 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/zero_from_start - exists and is zero size (0B, 100%)
Improve support for special characters in filenames. % characters caused issues in backup/restore due to filenames being appended directly into a format string. Reserved XML characters (<>&') caused issues in the S3 driver due to improper escaping. Add a file with all common special characters to regression testing.
2018-09-10 16:54:34 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?% - exists and is zero size (0B, 100%)
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P00 INFO: write [TEST_PATH]/db-master/db/base/recovery.conf
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 INFO: restore global/pg_control (performed last to ensure aborted restores cannot be started)
Added a predictable log entry to signal that a command has completed successfully. For example a backup ends successfully with: INFO: backup command end: completed successfully.
2016-12-04 21:44:53 +02:00
P00 INFO: restore command end: completed successfully
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
+ supplemental file: [TEST_PATH]/db-master/db/base/recovery.conf
----------------------------------------------------------------
restore_command = '[BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --log-level-console=detail --stanza=db archive-get %f "%p"'
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
Renumber return codes between 25 and 125 to avoid PostgreSQL interpreting some as fatal signal exceptions. Suggested by Yogesh Sharma.
2017-04-12 18:41:34 +02:00
restore delta, backup '[BACKUP-FULL-2]', expect exit 70 - restore all links by mapping (db-master host)
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --delta --set=[BACKUP-FULL-2] --log-level-console=warn --link-map=pg_stat=../pg_stat --link-map=postgresql.conf=../pg_stat/postgresql.conf --stanza=db restore
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Continuous integration using travis-ci.
2017-02-21 15:59:23 +02:00
P00 WARN: backup group for pg_data/base/16384/PG_VERSION was not mapped to a name, set to [GROUP-1]
Regression tests can now be run as any properly-configured user, not just vagrant.
2016-10-01 19:39:44 +02:00
P00 WARN: backup user for pg_data/base/1/PG_VERSION was not mapped to a name, set to [USER-1]
Fix failure in manifest build when two or more files in PGDATA are linked to the same directory. Reported by Vitaliy Kukharik.
2018-05-02 18:19:54 +02:00
P00 WARN: file link pg_hba.conf will be restored as a file at the same location
Renumber return codes between 25 and 125 to avoid PostgreSQL interpreting some as fatal signal exceptions. Suggested by Yogesh Sharma.
2017-04-12 18:41:34 +02:00
P00 ERROR: [070]: link [TEST_PATH]/db-master/db/base/postgresql.conf (../pg_stat) references a subdirectory of or the same directory as link [TEST_PATH]/db-master/db/base/pg_stat (../pg_stat)
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
Fix failure in manifest build when two or more files in PGDATA are linked to the same directory. Reported by Vitaliy Kukharik.
2018-05-02 18:19:54 +02:00
restore, backup '[BACKUP-FULL-2]', expect exit 40 - error on existing linked file (db-master host)
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --set=[BACKUP-FULL-2] --log-level-console=warn --link-all --stanza=db restore
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Continuous integration using travis-ci.
2017-02-21 15:59:23 +02:00
P00 WARN: backup group for pg_data/base/16384/PG_VERSION was not mapped to a name, set to [GROUP-1]
Regression tests can now be run as any properly-configured user, not just vagrant.
2016-10-01 19:39:44 +02:00
P00 WARN: backup user for pg_data/base/1/PG_VERSION was not mapped to a name, set to [USER-1]
Fix failure in manifest build when two or more files in PGDATA are linked to the same directory. Reported by Vitaliy Kukharik.
2018-05-02 18:19:54 +02:00
P00 ERROR: [040]: cannot restore file '[TEST_PATH]/db-master/db/pg_config/pg_hba.conf' that already exists - try using --delta if this is what you intended
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
Fix failure in manifest build when two or more files in PGDATA are linked to the same directory. Reported by Vitaliy Kukharik.
2018-05-02 18:19:54 +02:00
restore, backup '[BACKUP-FULL-2]', expect exit 40 - error on existing linked path (db-master host)
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --set=[BACKUP-FULL-2] --log-level-console=warn --link-all --stanza=db restore
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Continuous integration using travis-ci.
2017-02-21 15:59:23 +02:00
P00 WARN: backup group for pg_data/base/16384/PG_VERSION was not mapped to a name, set to [GROUP-1]
Regression tests can now be run as any properly-configured user, not just vagrant.
2016-10-01 19:39:44 +02:00
P00 WARN: backup user for pg_data/base/1/PG_VERSION was not mapped to a name, set to [USER-1]
Fix failure in manifest build when two or more files in PGDATA are linked to the same directory. Reported by Vitaliy Kukharik.
2018-05-02 18:19:54 +02:00
P00 ERROR: [040]: cannot restore to path '[TEST_PATH]/db-master/db/pg_stat' that contains files - try using --delta if this is what you intended
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
Renumber return codes between 25 and 125 to avoid PostgreSQL interpreting some as fatal signal exceptions. Suggested by Yogesh Sharma.
2017-04-12 18:41:34 +02:00
restore, backup '[BACKUP-FULL-2]', expect exit 38 - error on postmaster.pid exists (db-master host)
Improved error message when the restore command detects the presence of postmaster.pid. Suggested by Yogesh Sharma.
2017-04-12 15:53:07 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --set=[BACKUP-FULL-2] --log-level-console=warn --stanza=db restore
------------------------------------------------------------------------------------------------------------------------------------
Renumber return codes between 25 and 125 to avoid PostgreSQL interpreting some as fatal signal exceptions. Suggested by Yogesh Sharma.
2017-04-12 18:41:34 +02:00
P00 ERROR: [038]: unable to restore while PostgreSQL is running
Improved error message when the restore command detects the presence of postmaster.pid. Suggested by Yogesh Sharma.
2017-04-12 15:53:07 +02:00
HINT: presence of 'postmaster.pid' in '[TEST_PATH]/db-master/db/base' indicates PostgreSQL is running.
HINT: remove 'postmaster.pid' only if PostgreSQL is not running.
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
restore delta, backup '[BACKUP-FULL-2]' - restore all links --link-all and mapping (db-master host)
Fix failure in manifest build when two or more files in PGDATA are linked to the same directory. Reported by Vitaliy Kukharik.
2018-05-02 18:19:54 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --delta --set=[BACKUP-FULL-2] --log-level-console=detail --link-map=pg_stat=../pg_stat --link-all --stanza=db restore
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Add log-subprocess option to allow file logging for local and remote subprocesses.
2018-08-23 02:05:49 +02:00
P00 INFO: restore command begin [BACKREST-VERSION]: --no-compress --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --delta --link-all --link-map=pg_stat=../pg_stat --lock-path=[TEST_PATH]/db-master/lock --log-level-console=detail --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --pg1-path=[TEST_PATH]/db-master/db/base --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --set=[BACKUP-FULL-2] --stanza=db
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: restore backup set [BACKUP-FULL-2]
Continuous integration using travis-ci.
2017-02-21 15:59:23 +02:00
P00 WARN: backup group for pg_data/base/16384/PG_VERSION was not mapped to a name, set to [GROUP-1]
Regression tests can now be run as any properly-configured user, not just vagrant.
2016-10-01 19:39:44 +02:00
P00 WARN: backup user for pg_data/base/1/PG_VERSION was not mapped to a name, set to [USER-1]
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: remap link pg_data/pg_stat destination to ../pg_stat
P00 DETAIL: check [TEST_PATH]/db-master/db/base exists
Fix failure in manifest build when two or more files in PGDATA are linked to the same directory. Reported by Vitaliy Kukharik.
2018-05-02 18:19:54 +02:00
P00 DETAIL: check [TEST_PATH]/db-master/db/pg_config exists
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 DETAIL: check [TEST_PATH]/db-master/db/pg_stat exists
P00 DETAIL: check [TEST_PATH]/db-master/db/pg_config exists
P00 INFO: remove invalid files/paths/links from [TEST_PATH]/db-master/db/pg_config
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base/base/32768/33001 (64KB, 33%) checksum 6bf316f11d28c28914ea9be92c00de9bea6d9a6b
P01 INFO: restore file [TEST_PATH]/db-master/db/base/base/32768/44000_init (32KB, 49%) checksum 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f
P01 INFO: restore file [TEST_PATH]/db-master/db/base/base/32768/33000.32767 (32KB, 66%) checksum 21e2c7c1a326682c07053b7d6a5a40dbd49c2ec5
P01 INFO: restore file [TEST_PATH]/db-master/db/base/base/32768/33000 (32KB, 83%) checksum 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f
P01 INFO: restore file [TEST_PATH]/db-master/db/base/base/16384/17000 (16KB, 91%) checksum e0101dd8ffb910c9c202ca35b5f828bcb9697bed
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 11:24:42 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base/global/pg_control.pgbackrest.tmp (8KB, 95%) checksum 4c77c900f7af0d9ab13fa9982051a42e0b637f6c
Add more realistic data files to synthetic backup and restore tests.
2016-12-10 16:13:55 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base/base/1/12000 (8KB, 99%) checksum 22c98d248ff548311eda88559e4a8405ed77c003
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/postgresql.conf - exists and matches backup (21B, 99%) checksum 6721d92c9fcdf4248acff1f9a1377127d9064807
Fix failure in manifest build when two or more files in PGDATA are linked to the same directory. Reported by Vitaliy Kukharik.
2018-05-02 18:19:54 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base/pg_hba.conf (9B, 99%) checksum dd4cea0cae348309f9de28ad4ded8ee2cc2e6d5b
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base/changecontent.txt (7B, 99%) checksum 238a131a3e8eb98d1fc5b27d882ca40b7618fd2a
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base/pg_stat/global.stat (5B, 99%) checksum e350d5ce0153f3e22d5db21cf2a4eff00f3ee877
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base/changetime.txt (4B, 99%) checksum 88087292ed82e26f3eb824d0bffc05ccf7a30f8d
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base/base/32768/PG_VERSION (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P01 INFO: restore file [TEST_PATH]/db-master/db/base/base/16384/PG_VERSION (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P01 INFO: restore file [TEST_PATH]/db-master/db/base/base/1/PG_VERSION (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P01 INFO: restore file [TEST_PATH]/db-master/db/base/PG_VERSION (3B, 100%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
Allow zero-size files in backup manifest to reference a prior manifest regardless of timestamp delta. Contributed by Cynthia Shang.
2018-08-24 22:50:33 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base/zero_from_start (0B, 100%)
Improve support for special characters in filenames. % characters caused issues in backup/restore due to filenames being appended directly into a format string. Reserved XML characters (<>&') caused issues in the S3 driver due to improper escaping. Add a file with all common special characters to regression testing.
2018-09-10 16:54:34 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?% (0B, 100%)
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P00 INFO: write [TEST_PATH]/db-master/db/base/recovery.conf
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 INFO: restore global/pg_control (performed last to ensure aborted restores cannot be started)
Added a predictable log entry to signal that a command has completed successfully. For example a backup ends successfully with: INFO: backup command end: completed successfully.
2016-12-04 21:44:53 +02:00
P00 INFO: restore command end: completed successfully
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
+ supplemental file: [TEST_PATH]/db-master/db/base/recovery.conf
----------------------------------------------------------------
restore_command = '[BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --log-level-console=detail --stanza=db archive-get %f "%p"'
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
Renumber return codes between 25 and 125 to avoid PostgreSQL interpreting some as fatal signal exceptions. Suggested by Yogesh Sharma.
2017-04-12 18:41:34 +02:00
restore delta, force, backup '[BACKUP-FULL-2]', expect exit 40 - fail on missing PG_VERSION (db-master host)
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --delta --force --set=[BACKUP-FULL-2] --log-level-console=detail --stanza=db restore
Closed #127: More sanity checking for --delta restores Added checks for `--delta` and `--force` restore options to ensure that the destination is a valid $PGDATA directory. pgBackRest will check for the presence of `PG_VERSION` or `backup.manifest` (left over from an aborted restore). If neither is found then `--delta` and `--force` will be disabled but the restore will proceed unless there are files in the $PGDATA directory (or any tablespace directories) in which case the operation will be aborted.
2016-02-06 06:03:29 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Add log-subprocess option to allow file logging for local and remote subprocesses.
2018-08-23 02:05:49 +02:00
P00 INFO: restore command begin [BACKREST-VERSION]: --no-compress --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --delta --force --lock-path=[TEST_PATH]/db-master/lock --log-level-console=detail --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --pg1-path=[TEST_PATH]/db-master/db/base --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --set=[BACKUP-FULL-2] --stanza=db
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 WARN: --delta or --force specified but unable to find 'PG_VERSION' or 'backup.manifest' in '[TEST_PATH]/db-master/db/base' to confirm that this is a valid $PGDATA directory. --delta and --force have been disabled and if any files exist in the destination directories the restore will be aborted.
P00 INFO: restore backup set [BACKUP-FULL-2]
Continuous integration using travis-ci.
2017-02-21 15:59:23 +02:00
P00 WARN: backup group for pg_data/base/16384/PG_VERSION was not mapped to a name, set to [GROUP-1]
Regression tests can now be run as any properly-configured user, not just vagrant.
2016-10-01 19:39:44 +02:00
P00 WARN: backup user for pg_data/base/1/PG_VERSION was not mapped to a name, set to [USER-1]
Fix failure in manifest build when two or more files in PGDATA are linked to the same directory. Reported by Vitaliy Kukharik.
2018-05-02 18:19:54 +02:00
P00 WARN: file link pg_hba.conf will be restored as a file at the same location
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 WARN: contents of directory link pg_stat will be restored in a directory at the same location
P00 WARN: file link postgresql.conf will be restored as a file at the same location
P00 DETAIL: check [TEST_PATH]/db-master/db/base exists
Renumber return codes between 25 and 125 to avoid PostgreSQL interpreting some as fatal signal exceptions. Suggested by Yogesh Sharma.
2017-04-12 18:41:34 +02:00
P00 ERROR: [040]: cannot restore to path '[TEST_PATH]/db-master/db/base' that contains files - try using --delta if this is what you intended
P00 INFO: restore command end: aborted with exception [040]
Closed #127: More sanity checking for --delta restores Added checks for `--delta` and `--force` restore options to ensure that the destination is a valid $PGDATA directory. pgBackRest will check for the presence of `PG_VERSION` or `backup.manifest` (left over from an aborted restore). If neither is found then `--delta` and `--force` will be disabled but the restore will proceed unless there are files in the $PGDATA directory (or any tablespace directories) in which case the operation will be aborted.
2016-02-06 06:03:29 +02:00
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
restore delta, force, backup '[BACKUP-FULL-2]' - restore succeeds with backup.manifest file (db-master host)
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --delta --force --set=[BACKUP-FULL-2] --log-level-console=detail --stanza=db restore
Closed #127: More sanity checking for --delta restores Added checks for `--delta` and `--force` restore options to ensure that the destination is a valid $PGDATA directory. pgBackRest will check for the presence of `PG_VERSION` or `backup.manifest` (left over from an aborted restore). If neither is found then `--delta` and `--force` will be disabled but the restore will proceed unless there are files in the $PGDATA directory (or any tablespace directories) in which case the operation will be aborted.
2016-02-06 06:03:29 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Add log-subprocess option to allow file logging for local and remote subprocesses.
2018-08-23 02:05:49 +02:00
P00 INFO: restore command begin [BACKREST-VERSION]: --no-compress --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --delta --force --lock-path=[TEST_PATH]/db-master/lock --log-level-console=detail --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --pg1-path=[TEST_PATH]/db-master/db/base --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --set=[BACKUP-FULL-2] --stanza=db
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: restore backup set [BACKUP-FULL-2]
Continuous integration using travis-ci.
2017-02-21 15:59:23 +02:00
P00 WARN: backup group for pg_data/base/16384/PG_VERSION was not mapped to a name, set to [GROUP-1]
P00 WARN: group bogus in manifest cannot be used for restore, set to [USER-1]
Regression tests can now be run as any properly-configured user, not just vagrant.
2016-10-01 19:39:44 +02:00
P00 WARN: backup user for pg_data/base/1/PG_VERSION was not mapped to a name, set to [USER-1]
P00 WARN: user bogus in manifest cannot be used for restore, set to [USER-1]
Fix failure in manifest build when two or more files in PGDATA are linked to the same directory. Reported by Vitaliy Kukharik.
2018-05-02 18:19:54 +02:00
P00 WARN: file link pg_hba.conf will be restored as a file at the same location
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 WARN: contents of directory link pg_stat will be restored in a directory at the same location
P00 WARN: file link postgresql.conf will be restored as a file at the same location
P00 DETAIL: check [TEST_PATH]/db-master/db/base exists
P00 INFO: remove invalid files/paths/links from [TEST_PATH]/db-master/db/base
P00 DETAIL: preserve file [TEST_PATH]/db-master/db/base/recovery.conf
P00 DETAIL: remove link [TEST_PATH]/db-master/db/base/postgresql.conf
P00 DETAIL: remove link [TEST_PATH]/db-master/db/base/pg_stat
Fix failure in manifest build when two or more files in PGDATA are linked to the same directory. Reported by Vitaliy Kukharik.
2018-05-02 18:19:54 +02:00
P00 DETAIL: remove link [TEST_PATH]/db-master/db/base/pg_hba.conf
P00 INFO: cleanup removed 3 links
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/base/32768/33001 - exists and matches size 65536 and modification time [MODIFICATION-TIME-2] (64KB, 33%) checksum 6bf316f11d28c28914ea9be92c00de9bea6d9a6b
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/base/32768/44000_init - exists and matches size 32768 and modification time [MODIFICATION-TIME-2] (32KB, 49%) checksum 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/base/32768/33000.32767 - exists and matches size 32768 and modification time [MODIFICATION-TIME-2] (32KB, 66%) checksum 21e2c7c1a326682c07053b7d6a5a40dbd49c2ec5
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/base/32768/33000 - exists and matches size 32768 and modification time [MODIFICATION-TIME-2] (32KB, 83%) checksum 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/base/16384/17000 - exists and matches size 16384 and modification time [MODIFICATION-TIME-2] (16KB, 91%) checksum e0101dd8ffb910c9c202ca35b5f828bcb9697bed
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 11:24:42 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base/global/pg_control.pgbackrest.tmp (8KB, 95%) checksum 4c77c900f7af0d9ab13fa9982051a42e0b637f6c
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/base/1/12000 - exists and matches size 8192 and modification time [MODIFICATION-TIME-2] (8KB, 99%) checksum 22c98d248ff548311eda88559e4a8405ed77c003
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base/postgresql.conf (21B, 99%) checksum 6721d92c9fcdf4248acff1f9a1377127d9064807
Fix failure in manifest build when two or more files in PGDATA are linked to the same directory. Reported by Vitaliy Kukharik.
2018-05-02 18:19:54 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base/pg_hba.conf (9B, 99%) checksum dd4cea0cae348309f9de28ad4ded8ee2cc2e6d5b
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/changecontent.txt - exists and matches size 7 and modification time [MODIFICATION-TIME-2] (7B, 99%) checksum 238a131a3e8eb98d1fc5b27d882ca40b7618fd2a
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base/pg_stat/global.stat (5B, 99%) checksum e350d5ce0153f3e22d5db21cf2a4eff00f3ee877
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/changetime.txt - exists and matches size 4 and modification time [MODIFICATION-TIME-2] (4B, 99%) checksum 88087292ed82e26f3eb824d0bffc05ccf7a30f8d
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/base/32768/PG_VERSION - exists and matches size 3 and modification time [MODIFICATION-TIME-2] (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/base/16384/PG_VERSION - exists and matches size 3 and modification time [MODIFICATION-TIME-2] (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/base/1/PG_VERSION - exists and matches size 3 and modification time [MODIFICATION-TIME-2] (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base/PG_VERSION (3B, 100%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/zero_from_start - exists and matches size 0 and modification time [MODIFICATION-TIME-2] (0B, 100%)
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?% - exists and matches size 0 and modification time [MODIFICATION-TIME-2] (0B, 100%)
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P00 INFO: write [TEST_PATH]/db-master/db/base/recovery.conf
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 INFO: restore global/pg_control (performed last to ensure aborted restores cannot be started)
Added a predictable log entry to signal that a command has completed successfully. For example a backup ends successfully with: INFO: backup command end: completed successfully.
2016-12-04 21:44:53 +02:00
P00 INFO: restore command end: completed successfully
Closed #127: More sanity checking for --delta restores Added checks for `--delta` and `--force` restore options to ensure that the destination is a valid $PGDATA directory. pgBackRest will check for the presence of `PG_VERSION` or `backup.manifest` (left over from an aborted restore). If neither is found then `--delta` and `--force` will be disabled but the restore will proceed unless there are files in the $PGDATA directory (or any tablespace directories) in which case the operation will be aborted.
2016-02-06 06:03:29 +02:00
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
+ supplemental file: [TEST_PATH]/db-master/db/base/recovery.conf
----------------------------------------------------------------
restore_command = '[BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --log-level-console=detail --stanza=db archive-get %f "%p"'
Closed #127: More sanity checking for --delta restores Added checks for `--delta` and `--force` restore options to ensure that the destination is a valid $PGDATA directory. pgBackRest will check for the presence of `PG_VERSION` or `backup.manifest` (left over from an aborted restore). If neither is found then `--delta` and `--force` will be disabled but the restore will proceed unless there are files in the $PGDATA directory (or any tablespace directories) in which case the operation will be aborted.
2016-02-06 06:03:29 +02:00
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
incr backup - invalid database version (db-master host)
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --no-online --log-level-console=detail --stanza=db backup
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
P00 INFO: backup command begin [BACKREST-VERSION]: --no-compress --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --db-timeout=45 --exclude=postgresql.auto.conf --exclude=pg_log/ --exclude=pg_log2 --lock-path=[TEST_PATH]/db-master/lock --log-level-console=detail --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --no-online --pg1-path=[TEST_PATH]/db-master/db/base --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --stanza=db --start-fast
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
P00 WARN: option repo1-retention-full is not set, the repository may run out of space
HINT: to retain full backups indefinitely (without warning), set option 'repo1-retention-full' to the maximum.
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
P00 ERROR: [051]: database version = 9.4, system-id 1000000000000000094 does not match backup version = 8.0, system-id = 1000000000000000094
Fixed alignment issues with multiline logging.
2017-02-06 02:58:52 +02:00
HINT: is this the correct stanza?
Renumber return codes between 25 and 125 to avoid PostgreSQL interpreting some as fatal signal exceptions. Suggested by Yogesh Sharma.
2017-04-12 18:41:34 +02:00
P00 INFO: backup command end: aborted with exception [051]
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
incr backup - invalid system id (db-master host)
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --no-online --log-level-console=detail --stanza=db backup
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
P00 INFO: backup command begin [BACKREST-VERSION]: --no-compress --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --db-timeout=45 --exclude=postgresql.auto.conf --exclude=pg_log/ --exclude=pg_log2 --lock-path=[TEST_PATH]/db-master/lock --log-level-console=detail --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --no-online --pg1-path=[TEST_PATH]/db-master/db/base --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --stanza=db --start-fast
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
P00 WARN: option repo1-retention-full is not set, the repository may run out of space
HINT: to retain full backups indefinitely (without warning), set option 'repo1-retention-full' to the maximum.
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
P00 ERROR: [051]: database version = 9.4, system-id 1000000000000000094 does not match backup version = 9.4, system-id = 6999999999999999999
Fixed alignment issues with multiline logging.
2017-02-06 02:58:52 +02:00
HINT: is this the correct stanza?
Renumber return codes between 25 and 125 to avoid PostgreSQL interpreting some as fatal signal exceptions. Suggested by Yogesh Sharma.
2017-04-12 18:41:34 +02:00
P00 INFO: backup command end: aborted with exception [051]
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
incr backup - invalid control version (db-master host)
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --no-online --log-level-console=detail --stanza=db backup
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
P00 INFO: backup command begin [BACKREST-VERSION]: --no-compress --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --db-timeout=45 --exclude=postgresql.auto.conf --exclude=pg_log/ --exclude=pg_log2 --lock-path=[TEST_PATH]/db-master/lock --log-level-console=detail --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --no-online --pg1-path=[TEST_PATH]/db-master/db/base --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --stanza=db --start-fast
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
P00 WARN: option repo1-retention-full is not set, the repository may run out of space
HINT: to retain full backups indefinitely (without warning), set option 'repo1-retention-full' to the maximum.
Renumber return codes between 25 and 125 to avoid PostgreSQL interpreting some as fatal signal exceptions. Suggested by Yogesh Sharma.
2017-04-12 18:41:34 +02:00
P00 ERROR: [051]: database control-version = 942, catalog-version 201409291 does not match backup control-version = 842, catalog-version = 201409291
Fixed alignment issues with multiline logging.
2017-02-06 02:58:52 +02:00
HINT: this may be a symptom of database or repository corruption!
Renumber return codes between 25 and 125 to avoid PostgreSQL interpreting some as fatal signal exceptions. Suggested by Yogesh Sharma.
2017-04-12 18:41:34 +02:00
P00 INFO: backup command end: aborted with exception [051]
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
incr backup - invalid catalog version (db-master host)
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --no-online --log-level-console=detail --stanza=db backup
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
P00 INFO: backup command begin [BACKREST-VERSION]: --no-compress --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --db-timeout=45 --exclude=postgresql.auto.conf --exclude=pg_log/ --exclude=pg_log2 --lock-path=[TEST_PATH]/db-master/lock --log-level-console=detail --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --no-online --pg1-path=[TEST_PATH]/db-master/db/base --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --stanza=db --start-fast
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
P00 WARN: option repo1-retention-full is not set, the repository may run out of space
HINT: to retain full backups indefinitely (without warning), set option 'repo1-retention-full' to the maximum.
Renumber return codes between 25 and 125 to avoid PostgreSQL interpreting some as fatal signal exceptions. Suggested by Yogesh Sharma.
2017-04-12 18:41:34 +02:00
P00 ERROR: [051]: database control-version = 942, catalog-version 201409291 does not match backup control-version = 942, catalog-version = 197208141
Fixed alignment issues with multiline logging.
2017-02-06 02:58:52 +02:00
HINT: this may be a symptom of database or repository corruption!
Renumber return codes between 25 and 125 to avoid PostgreSQL interpreting some as fatal signal exceptions. Suggested by Yogesh Sharma.
2017-04-12 18:41:34 +02:00
P00 INFO: backup command end: aborted with exception [051]
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
incr backup - invalid path in pg_tblspc (db-master host)
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --no-online --log-level-console=detail --stanza=db backup
Closed #19: The following tablespace checks have been added: paths or files in pg_tblspc, relative links in pg_tblspc, tablespaces in $PGDATA. All three will generate errors.
2016-01-15 14:07:00 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
P00 INFO: backup command begin [BACKREST-VERSION]: --no-compress --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --db-timeout=45 --exclude=postgresql.auto.conf --exclude=pg_log/ --exclude=pg_log2 --lock-path=[TEST_PATH]/db-master/lock --log-level-console=detail --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --no-online --pg1-path=[TEST_PATH]/db-master/db/base --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --stanza=db --start-fast
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
P00 WARN: option repo1-retention-full is not set, the repository may run out of space
HINT: to retain full backups indefinitely (without warning), set option 'repo1-retention-full' to the maximum.
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: last backup label = [BACKUP-FULL-2], version = [VERSION-1]
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
P00 WARN: incr backup cannot alter 'checksum-page' option to 'false', reset to 'true' from [BACKUP-FULL-2]
Renumber return codes between 25 and 125 to avoid PostgreSQL interpreting some as fatal signal exceptions. Suggested by Yogesh Sharma.
2017-04-12 18:41:34 +02:00
P00 ERROR: [069]: pg_tblspc/path is not a symlink - pg_tblspc should contain only symlinks
P00 INFO: backup command end: aborted with exception [069]
Closed #19: The following tablespace checks have been added: paths or files in pg_tblspc, relative links in pg_tblspc, tablespaces in $PGDATA. All three will generate errors.
2016-01-15 14:07:00 +02:00
Fixed issue with tablespace link checking. * Tablespace paths that had $PGDATA as a substring would be identified as a subdirectories of $PGDATA even when they were not. * Also hardened relative path checking a bit.
2016-08-09 15:05:27 +02:00
incr backup - invalid relative tablespace is ../ (db-master host)
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --no-online --log-level-console=detail --stanza=db backup
------------------------------------------------------------------------------------------------------------------------------------
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
P00 INFO: backup command begin [BACKREST-VERSION]: --no-compress --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --db-timeout=45 --exclude=postgresql.auto.conf --exclude=pg_log/ --exclude=pg_log2 --lock-path=[TEST_PATH]/db-master/lock --log-level-console=detail --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --no-online --pg1-path=[TEST_PATH]/db-master/db/base --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --stanza=db --start-fast
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
P00 WARN: option repo1-retention-full is not set, the repository may run out of space
HINT: to retain full backups indefinitely (without warning), set option 'repo1-retention-full' to the maximum.
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: last backup label = [BACKUP-FULL-2], version = [VERSION-1]
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
P00 WARN: incr backup cannot alter 'checksum-page' option to 'false', reset to 'true' from [BACKUP-FULL-2]
Renumber return codes between 25 and 125 to avoid PostgreSQL interpreting some as fatal signal exceptions. Suggested by Yogesh Sharma.
2017-04-12 18:41:34 +02:00
P00 ERROR: [071]: tablespace symlink ../ destination must not be in $PGDATA
P00 INFO: backup command end: aborted with exception [071]
Fixed issue with tablespace link checking. * Tablespace paths that had $PGDATA as a substring would be identified as a subdirectories of $PGDATA even when they were not. * Also hardened relative path checking a bit.
2016-08-09 15:05:27 +02:00
incr backup - invalid relative tablespace is .. (db-master host)
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --no-online --log-level-console=detail --stanza=db backup
------------------------------------------------------------------------------------------------------------------------------------
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
P00 INFO: backup command begin [BACKREST-VERSION]: --no-compress --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --db-timeout=45 --exclude=postgresql.auto.conf --exclude=pg_log/ --exclude=pg_log2 --lock-path=[TEST_PATH]/db-master/lock --log-level-console=detail --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --no-online --pg1-path=[TEST_PATH]/db-master/db/base --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --stanza=db --start-fast
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
P00 WARN: option repo1-retention-full is not set, the repository may run out of space
HINT: to retain full backups indefinitely (without warning), set option 'repo1-retention-full' to the maximum.
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: last backup label = [BACKUP-FULL-2], version = [VERSION-1]
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
P00 WARN: incr backup cannot alter 'checksum-page' option to 'false', reset to 'true' from [BACKUP-FULL-2]
Renumber return codes between 25 and 125 to avoid PostgreSQL interpreting some as fatal signal exceptions. Suggested by Yogesh Sharma.
2017-04-12 18:41:34 +02:00
P00 ERROR: [071]: tablespace symlink .. destination must not be in $PGDATA
P00 INFO: backup command end: aborted with exception [071]
Fixed issue with tablespace link checking. * Tablespace paths that had $PGDATA as a substring would be identified as a subdirectories of $PGDATA even when they were not. * Also hardened relative path checking a bit.
2016-08-09 15:05:27 +02:00
incr backup - invalid relative tablespace is ../../$PGDATA (db-master host)
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --no-online --log-level-console=detail --stanza=db backup
------------------------------------------------------------------------------------------------------------------------------------
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
P00 INFO: backup command begin [BACKREST-VERSION]: --no-compress --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --db-timeout=45 --exclude=postgresql.auto.conf --exclude=pg_log/ --exclude=pg_log2 --lock-path=[TEST_PATH]/db-master/lock --log-level-console=detail --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --no-online --pg1-path=[TEST_PATH]/db-master/db/base --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --stanza=db --start-fast
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
P00 WARN: option repo1-retention-full is not set, the repository may run out of space
HINT: to retain full backups indefinitely (without warning), set option 'repo1-retention-full' to the maximum.
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: last backup label = [BACKUP-FULL-2], version = [VERSION-1]
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
P00 WARN: incr backup cannot alter 'checksum-page' option to 'false', reset to 'true' from [BACKUP-FULL-2]
Renumber return codes between 25 and 125 to avoid PostgreSQL interpreting some as fatal signal exceptions. Suggested by Yogesh Sharma.
2017-04-12 18:41:34 +02:00
P00 ERROR: [071]: tablespace symlink ../../base/ destination must not be in $PGDATA
P00 INFO: backup command end: aborted with exception [071]
Fixed issue with tablespace link checking. * Tablespace paths that had $PGDATA as a substring would be identified as a subdirectories of $PGDATA even when they were not. * Also hardened relative path checking a bit.
2016-08-09 15:05:27 +02:00
incr backup - invalid relative tablespace is ../../$PGDATA (db-master host)
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --no-online --log-level-console=detail --stanza=db backup
------------------------------------------------------------------------------------------------------------------------------------
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
P00 INFO: backup command begin [BACKREST-VERSION]: --no-compress --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --db-timeout=45 --exclude=postgresql.auto.conf --exclude=pg_log/ --exclude=pg_log2 --lock-path=[TEST_PATH]/db-master/lock --log-level-console=detail --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --no-online --pg1-path=[TEST_PATH]/db-master/db/base --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --stanza=db --start-fast
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
P00 WARN: option repo1-retention-full is not set, the repository may run out of space
HINT: to retain full backups indefinitely (without warning), set option 'repo1-retention-full' to the maximum.
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: last backup label = [BACKUP-FULL-2], version = [VERSION-1]
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
P00 WARN: incr backup cannot alter 'checksum-page' option to 'false', reset to 'true' from [BACKUP-FULL-2]
Renumber return codes between 25 and 125 to avoid PostgreSQL interpreting some as fatal signal exceptions. Suggested by Yogesh Sharma.
2017-04-12 18:41:34 +02:00
P00 ERROR: [071]: tablespace symlink ../../base destination must not be in $PGDATA
P00 INFO: backup command end: aborted with exception [071]
Fixed issue with tablespace link checking. * Tablespace paths that had $PGDATA as a substring would be identified as a subdirectories of $PGDATA even when they were not. * Also hardened relative path checking a bit.
2016-08-09 15:05:27 +02:00
Fixed #236: Recursive user tablespace symlink. A tablespace link that referenced another link would not produce an error, but instead skip the tablespace entirely.
2016-08-15 23:11:45 +02:00
incr backup - tablespace link references a link (db-master host)
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --no-online --log-level-console=detail --stanza=db backup
------------------------------------------------------------------------------------------------------------------------------------
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
P00 INFO: backup command begin [BACKREST-VERSION]: --no-compress --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --db-timeout=45 --exclude=postgresql.auto.conf --exclude=pg_log/ --exclude=pg_log2 --lock-path=[TEST_PATH]/db-master/lock --log-level-console=detail --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --no-online --pg1-path=[TEST_PATH]/db-master/db/base --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --stanza=db --start-fast
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
P00 WARN: option repo1-retention-full is not set, the repository may run out of space
HINT: to retain full backups indefinitely (without warning), set option 'repo1-retention-full' to the maximum.
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: last backup label = [BACKUP-FULL-2], version = [VERSION-1]
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
P00 WARN: incr backup cannot alter 'checksum-page' option to 'false', reset to 'true' from [BACKUP-FULL-2]
Renumber return codes between 25 and 125 to avoid PostgreSQL interpreting some as fatal signal exceptions. Suggested by Yogesh Sharma.
2017-04-12 18:41:34 +02:00
P00 ERROR: [070]: link '[TEST_PATH]/db-master/db/base/pg_tblspc/99999' -> '[TEST_PATH]/db-master/db/intermediate_link' cannot reference another link
P00 INFO: backup command end: aborted with exception [070]
Fixed #236: Recursive user tablespace symlink. A tablespace link that referenced another link would not produce an error, but instead skip the tablespace entirely.
2016-08-15 23:11:45 +02:00
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
incr backup - invalid relative tablespace in $PGDATA (db-master host)
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --no-online --log-level-console=detail --stanza=db backup
Closed #19: The following tablespace checks have been added: paths or files in pg_tblspc, relative links in pg_tblspc, tablespaces in $PGDATA. All three will generate errors.
2016-01-15 14:07:00 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
P00 INFO: backup command begin [BACKREST-VERSION]: --no-compress --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --db-timeout=45 --exclude=postgresql.auto.conf --exclude=pg_log/ --exclude=pg_log2 --lock-path=[TEST_PATH]/db-master/lock --log-level-console=detail --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --no-online --pg1-path=[TEST_PATH]/db-master/db/base --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --stanza=db --start-fast
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
P00 WARN: option repo1-retention-full is not set, the repository may run out of space
HINT: to retain full backups indefinitely (without warning), set option 'repo1-retention-full' to the maximum.
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: last backup label = [BACKUP-FULL-2], version = [VERSION-1]
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
P00 WARN: incr backup cannot alter 'checksum-page' option to 'false', reset to 'true' from [BACKUP-FULL-2]
Renumber return codes between 25 and 125 to avoid PostgreSQL interpreting some as fatal signal exceptions. Suggested by Yogesh Sharma.
2017-04-12 18:41:34 +02:00
P00 ERROR: [071]: tablespace symlink ../invalid_tblspc destination must not be in $PGDATA
P00 INFO: backup command end: aborted with exception [071]
Closed #19: The following tablespace checks have been added: paths or files in pg_tblspc, relative links in pg_tblspc, tablespaces in $PGDATA. All three will generate errors.
2016-01-15 14:07:00 +02:00
Fixed a regression introduced in v1.13 that could cause backups to fail. This happened if files were removed (e.g. tables dropped) while the manifest was being built. Reported by Navid Golpayegani.
2017-02-14 02:59:14 +02:00
incr backup - $PGDATA is a substring of valid tblspc excluding / (file missing err expected) (db-master host)
Fixed issue with tablespace link checking. * Tablespace paths that had $PGDATA as a substring would be identified as a subdirectories of $PGDATA even when they were not. * Also hardened relative path checking a bit.
2016-08-09 15:05:27 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --no-online --log-level-console=detail --stanza=db backup
------------------------------------------------------------------------------------------------------------------------------------
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
P00 INFO: backup command begin [BACKREST-VERSION]: --no-compress --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --db-timeout=45 --exclude=postgresql.auto.conf --exclude=pg_log/ --exclude=pg_log2 --lock-path=[TEST_PATH]/db-master/lock --log-level-console=detail --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --no-online --pg1-path=[TEST_PATH]/db-master/db/base --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --stanza=db --start-fast
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
P00 WARN: option repo1-retention-full is not set, the repository may run out of space
HINT: to retain full backups indefinitely (without warning), set option 'repo1-retention-full' to the maximum.
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: last backup label = [BACKUP-FULL-2], version = [VERSION-1]
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
P00 WARN: incr backup cannot alter 'checksum-page' option to 'false', reset to 'true' from [BACKUP-FULL-2]
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 ERROR: [055]: unable to stat '[TEST_PATH]/db-master/db/base_tbs': No such file or directory
Renumber return codes between 25 and 125 to avoid PostgreSQL interpreting some as fatal signal exceptions. Suggested by Yogesh Sharma.
2017-04-12 18:41:34 +02:00
P00 INFO: backup command end: aborted with exception [055]
Fixed issue with tablespace link checking. * Tablespace paths that had $PGDATA as a substring would be identified as a subdirectories of $PGDATA even when they were not. * Also hardened relative path checking a bit.
2016-08-09 15:05:27 +02:00
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
incr backup - invalid tablespace in $PGDATA (db-master host)
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --no-online --log-level-console=detail --stanza=db backup
Closed #19: The following tablespace checks have been added: paths or files in pg_tblspc, relative links in pg_tblspc, tablespaces in $PGDATA. All three will generate errors.
2016-01-15 14:07:00 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
P00 INFO: backup command begin [BACKREST-VERSION]: --no-compress --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --db-timeout=45 --exclude=postgresql.auto.conf --exclude=pg_log/ --exclude=pg_log2 --lock-path=[TEST_PATH]/db-master/lock --log-level-console=detail --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --no-online --pg1-path=[TEST_PATH]/db-master/db/base --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --stanza=db --start-fast
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
P00 WARN: option repo1-retention-full is not set, the repository may run out of space
HINT: to retain full backups indefinitely (without warning), set option 'repo1-retention-full' to the maximum.
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: last backup label = [BACKUP-FULL-2], version = [VERSION-1]
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
P00 WARN: incr backup cannot alter 'checksum-page' option to 'false', reset to 'true' from [BACKUP-FULL-2]
Renumber return codes between 25 and 125 to avoid PostgreSQL interpreting some as fatal signal exceptions. Suggested by Yogesh Sharma.
2017-04-12 18:41:34 +02:00
P00 ERROR: [071]: tablespace symlink [TEST_PATH]/db-master/db/base/invalid_tblspc destination must not be in $PGDATA
P00 INFO: backup command end: aborted with exception [071]
Closed #19: The following tablespace checks have been added: paths or files in pg_tblspc, relative links in pg_tblspc, tablespaces in $PGDATA. All three will generate errors.
2016-01-15 14:07:00 +02:00
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
incr backup - add tablespace 1 (db-master host)
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --no-online --test --stanza=db backup
Log testing can now be enabled for certain deterministic tests. This works by comparing the generated logs against a previous copy. Currently only enabled for the backup/synthetic tests.
2015-05-05 19:08:48 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
P00 INFO: backup command begin [BACKREST-VERSION]: --no-compress --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --db-timeout=45 --exclude=postgresql.auto.conf --exclude=pg_log/ --exclude=pg_log2 --lock-path=[TEST_PATH]/db-master/lock --log-level-console=debug --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --no-online --pg1-path=[TEST_PATH]/db-master/db/base --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --stanza=db --start-fast --test
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: common/lock::lockAcquire: (lockPath: {"[TEST_PATH]/db-master/lock"}, stanza: {"db"}, lockType: 1, lockTimeout: 0, failOnNoLock: true)
P00 DEBUG: common/lock::lockAcquire: => true
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
P00 WARN: option repo1-retention-full is not set, the repository may run out of space
HINT: to retain full backups indefinitely (without warning), set option 'repo1-retention-full' to the maximum.
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: config/load::cfgLoad: => void
P00 DEBUG: perl/exec::perlExec: (void)
Allow functions with sensitive options to be logged at debug level with redactions. Previously, functions with sensitive options had to be logged at trace level to avoid exposing them. Trace level logging may still expose secrets so use with caution.
2017-10-24 18:35:36 +02:00
P00 DEBUG: Storage::Posix::Driver->new(): bFileSync = <true>, bPathSync = <true>
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->new(): bAllowTemp = <true>, hRule = [hash], lBufferMax = 4194304, oDriver = [object], strCipherPassUser = [undef], strCipherType = [undef], strDefaultFileMode = <0640>, strDefaultPathMode = <0750>, strPathBase = [TEST_PATH]/db-master/repo, strTempExtension = pgbackrest.tmp
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp =
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
Config parsing no longer implemented in Perl. Options are passed to Perl as JSON from the C process.
2018-01-29 04:37:09 +02:00
P00 DEBUG: Storage::Posix::Driver->new(): bFileSync = <true>, bPathSync = <true>
P00 DEBUG: Storage::Local->new(): bAllowTemp = <true>, hRule = [undef], lBufferMax = 4194304, oDriver = [object], strCipherPassUser = [undef], strCipherType = [undef], strDefaultFileMode = <0640>, strDefaultPathMode = <0750>, strPathBase = /, strTempExtension = pgbackrest.tmp
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = true, bIgnoreExists = true, strMode = 0770, strPathExp = [TEST_PATH]/db-master/log
Move async forking and more error handling to C. The Perl process was exiting directly when called but that interfered with proper locking for the forked async process. Now Perl returns results to the C process which handles all errors, including signals.
2018-04-13 02:42:26 +02:00
P00 DEBUG: Common::Lock::lockStopTest(): bStanzaStopRequired = <false>
P00 DEBUG: Common::Lock::lockStopTest=>: bStopExists = false
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Backup::Info->new(): bIgnoreMissing = <false>, bLoad = <true>, bRequired = <true>, bValidate = <true>, oStorage = <[object]>, strBackupClusterPath = [TEST_PATH]/db-master/repo/backup/db, strCipherPassSub = [undef]
P00 DEBUG: Storage::Local->encrypted(): bIgnoreMissing = true, strFileName = [TEST_PATH]/db-master/repo/backup/db/backup.info
P00 DEBUG: Storage::Local->encrypted=>: bEncrypted = false
P00 DEBUG: Storage::Local->encryptionValid(): bEncrypted = false
P00 DEBUG: Storage::Local->encryptionValid=>: bValid = true
P00 DEBUG: Storage::Local->openRead(): bIgnoreMissing = true, rhyFilter = [undef], strCipherPass = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/backup.info
P00 DEBUG: Storage::Base->get(): strCipherPass = [undef], xFile = [object]
Move backup modules into Backup directory.
2017-05-15 22:01:00 +02:00
P00 DEBUG: Backup::Info->reconstruct(): bRequired = <true>, bSave = <true>, iCatalogVersion = [undef], iControlVersion = [undef], strDbVersion = [undef], ullDbSysId = [undef]
P00 DEBUG: Backup::Common::backupRegExpGet(): bAnchor = <true>, bDifferential = true, bFull = true, bIncremental = true
P00 DEBUG: Backup::Common::backupRegExpGet=>: strRegExp = ^[0-9]{8}\-[0-9]{6}F(\_[0-9]{8}\-[0-9]{6}(D|I)){0,1}$
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->list(): bIgnoreMissing = <false>, strExpression = ^[0-9]{8}\-[0-9]{6}F(\_[0-9]{8}\-[0-9]{6}(D|I)){0,1}$, strPathExp = [TEST_PATH]/db-master/repo/backup/db, strSortOrder = <forward>
P00 DEBUG: Storage::Local->list=>: stryFileList = ([BACKUP-FULL-2])
Move backup modules into Backup directory.
2017-05-15 22:01:00 +02:00
P00 DEBUG: Backup::Info->current(): strBackup = [BACKUP-FULL-2]
P00 DEBUG: Backup::Info->current=>: bTest = true
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]/backup.manifest
P00 DEBUG: Storage::Local->exists=>: bExists = true
Up to seven standbys can be configured for backup from standby.` Contributed by Cynthia Shang.
2017-09-01 01:15:44 +02:00
P00 DEBUG: Db::dbObjectGet(): bMasterOnly = <false>
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 DEBUG: Db->new(): iRemoteIdx = 1
Abstracted code to determine which database cluster is the master and which are standbys. Contributed by Cynthia Shang.
2016-11-30 21:15:11 +02:00
P00 DEBUG: Db::dbObjectGet=>: iDbMasterIdx = 1, iDbStandbyIdx = [undef], oDbMaster = [object], oDbStandby = [undef]
Allow functions with sensitive options to be logged at debug level with redactions. Previously, functions with sensitive options had to be logged at trace level to avoid exposing them. Trace level logging may still expose secrets so use with caution.
2017-10-24 18:35:36 +02:00
P00 DEBUG: Storage::Posix::Driver->new(): bFileSync = <true>, bPathSync = <true>
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->new(): bAllowTemp = <true>, hRule = [undef], lBufferMax = 4194304, oDriver = [object], strCipherPassUser = [undef], strCipherType = [undef], strDefaultFileMode = <0640>, strDefaultPathMode = <0750>, strPathBase = [TEST_PATH]/db-master/db/base, strTempExtension = pgbackrest.tmp
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 DEBUG: Db->info(): strDbPath = <[TEST_PATH]/db-master/db/base>
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
P00 DEBUG: Db->info=>: iDbCatalogVersion = 201409291, iDbControlVersion = 942, strDbVersion = 9.4, ullDbSysId = 1000000000000000094
P00 DEBUG: Backup::Info->check(): bRequired = <true>, iCatalogVersion = 201409291, iControlVersion = 942, strDbVersion = 9.4, ullDbSysId = 1000000000000000094
Move backup modules into Backup directory.
2017-05-15 22:01:00 +02:00
P00 DEBUG: Backup::Info->check=>: iDbHistoryId = 1
P00 DEBUG: Backup::Info->last(): strType = incr
P00 DEBUG: Backup::Common::backupRegExpGet(): bAnchor = <true>, bDifferential = true, bFull = true, bIncremental = true
P00 DEBUG: Backup::Common::backupRegExpGet=>: strRegExp = ^[0-9]{8}\-[0-9]{6}F(\_[0-9]{8}\-[0-9]{6}(D|I)){0,1}$
P00 DEBUG: Backup::Info->list(): strFilter = ^[0-9]{8}\-[0-9]{6}F(\_[0-9]{8}\-[0-9]{6}(D|I)){0,1}$, strOrder = reverse
P00 DEBUG: Backup::Info->list=>: stryBackup = ([BACKUP-FULL-2])
P00 DEBUG: Backup::Info->last=>: strBackup = [BACKUP-FULL-2]
P00 DEBUG: Backup::Info->dbHistoryList=>: hDbHash = [hash]
P00 DEBUG: Backup::Info->confirmDb=>: bConfirmDb = true
Require PostgreSQL catalog version when instantiating a Manifest object (and not loading it from disk). Contributed by Cynthia Shang.
2018-07-16 23:25:15 +02:00
P00 DEBUG: Manifest->new(): bLoad = <true>, iDbCatalogVersion = [undef], oStorage = <[object]>, strCipherPass = [undef], strCipherPassSub = [undef], strDbVersion = [undef], strFileName = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]/backup.manifest
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->encrypted(): bIgnoreMissing = true, strFileName = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]/backup.manifest
P00 DEBUG: Storage::Local->encrypted=>: bEncrypted = false
P00 DEBUG: Storage::Local->encryptionValid(): bEncrypted = false
P00 DEBUG: Storage::Local->encryptionValid=>: bValid = true
P00 DEBUG: Storage::Local->openRead(): bIgnoreMissing = true, rhyFilter = [undef], strCipherPass = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]/backup.manifest
P00 DEBUG: Storage::Base->get(): strCipherPass = [undef], xFile = [object]
Improvements and additional tests for stanza-upgrade. Contributed by Cynthia Shang.
2017-04-07 01:36:37 +02:00
P00 INFO: last backup label = [BACKUP-FULL-2], version = [VERSION-1]
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Backup::Common::backupRegExpGet(): bAnchor = <true>, bDifferential = true, bFull = true, bIncremental = true
P00 DEBUG: Backup::Common::backupRegExpGet=>: strRegExp = ^[0-9]{8}\-[0-9]{6}F(\_[0-9]{8}\-[0-9]{6}(D|I)){0,1}$
P00 DEBUG: Storage::Local->list(): bIgnoreMissing = <false>, strExpression = ^[0-9]{8}\-[0-9]{6}F(\_[0-9]{8}\-[0-9]{6}(D|I)){0,1}$, strPathExp = <REPO:BACKUP>, strSortOrder = reverse
P00 DEBUG: Storage::Local->list=>: stryFileList = ([BACKUP-FULL-2])
P00 DEBUG: Storage::Local->exists(): strFileExp = <REPO:BACKUP>/[BACKUP-FULL-2]/backup.manifest.copy
P00 DEBUG: Storage::Local->exists=>: bExists = true
P00 DEBUG: Storage::Local->exists(): strFileExp = <REPO:BACKUP>/[BACKUP-FULL-2]/backup.manifest
P00 DEBUG: Storage::Local->exists=>: bExists = true
P00 DEBUG: Storage::Local->list(): bIgnoreMissing = <false>, strExpression = [BACKUP-EXPR](D|I)$, strPathExp = <REPO:BACKUP>, strSortOrder = <forward>
P00 DEBUG: Storage::Local->list=>: stryFileList = ()
P00 DEBUG: Storage::Local->list(): bIgnoreMissing = true, strExpression = [BACKUP-EXPR](D|I)\.manifest\.gz$, strPathExp = <REPO:BACKUP>/backup.history/[YEAR-1], strSortOrder = <forward>
P00 DEBUG: Storage::Local->list=>: stryFileList = ()
Require PostgreSQL catalog version when instantiating a Manifest object (and not loading it from disk). Contributed by Cynthia Shang.
2018-07-16 23:25:15 +02:00
P00 DEBUG: Manifest->new(): bLoad = false, iDbCatalogVersion = 201409291, oStorage = <[object]>, strCipherPass = [undef], strCipherPassSub = [undef], strDbVersion = 9.4, strFileName = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-1]/backup.manifest
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/db/base/postmaster.pid
P00 DEBUG: Storage::Local->exists=>: bExists = false
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
P00 WARN: incr backup cannot alter 'checksum-page' option to 'false', reset to 'true' from [BACKUP-FULL-2]
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: Manifest->build(): bDelta = false, bOnline = false, bTablespace = [undef], hDatabaseMap = [undef], hTablespaceMap = [undef], iLevel = <0>, oLastManifest = [object], oStorageDbMaster = [object], rhExclude = [hash], strFilter = [undef], strLevel = [undef], strParentPath = [undef], strPath = [TEST_PATH]/db-master/db/base, strTimelineCurrent = [undef], strTimelineLast = [undef]
Ignore all files in a linked tablespace directory except the subdirectory for the current version of PostgreSQL. Previously an error would be generated if other files were present and not owned by the PostgreSQL user. This hasn't been a big deal in practice but it could cause issues. Also add tests to make sure the same logic applies with links to files, i.e. all other files in the directory should be ignored. This was actually working correctly, but there were no tests for it before.
2018-08-31 22:06:40 +02:00
P00 DEBUG: Storage::Local->manifest(): strFilter = [undef], strPathExp = [TEST_PATH]/db-master/db/base/pg_tblspc
Simplified the result hash of File->manifest(), Db->tablespaceMapGet(), and Db->databaseMapGet().
2016-11-30 21:36:39 +02:00
P00 DEBUG: Manifest->build: found tablespace 1 in offline mode
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: Manifest->checkDelta(): bOnlineSame = true, strLastBackupSource = last, strTimelineCurrent = [undef], strTimelineLast = [undef]
Ignore all files in a linked tablespace directory except the subdirectory for the current version of PostgreSQL. Previously an error would be generated if other files were present and not owned by the PostgreSQL user. This hasn't been a big deal in practice but it could cause issues. Also add tests to make sure the same logic applies with links to files, i.e. all other files in the directory should be ignored. This was actually working correctly, but there were no tests for it before.
2018-08-31 22:06:40 +02:00
P00 DEBUG: Storage::Local->manifest(): strFilter = [undef], strPathExp = [TEST_PATH]/db-master/db/base
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: Manifest->build(): bDelta = false, bOnline = false, bTablespace = true, hDatabaseMap = [undef], hTablespaceMap = [hash], iLevel = 1, oLastManifest = [undef], oStorageDbMaster = [object], rhExclude = [hash], strFilter = [TS_PATH-1], strLevel = pg_tblspc/1, strParentPath = [TEST_PATH]/db-master/db/base/pg_tblspc, strPath = [TEST_PATH]/db-master/db/tablespace/ts1, strTimelineCurrent = [undef], strTimelineLast = [undef]
Ignore all files in a linked tablespace directory except the subdirectory for the current version of PostgreSQL. Previously an error would be generated if other files were present and not owned by the PostgreSQL user. This hasn't been a big deal in practice but it could cause issues. Also add tests to make sure the same logic applies with links to files, i.e. all other files in the directory should be ignored. This was actually working correctly, but there were no tests for it before.
2018-08-31 22:06:40 +02:00
P00 DEBUG: Storage::Local->manifest(): strFilter = [TS_PATH-1], strPathExp = [TEST_PATH]/db-master/db/tablespace/ts1
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: Manifest->checkDeltaFile(): lTimeBegin = [MODIFICATION-TIME-6], oPriorManifest = [object], stryFileList = (pg_data/PG_VERSION, pg_data/badchecksum.txt, pg_data/base/1/12000, pg_data/base/1/PG_VERSION, pg_data/base/16384/17000, pg_data/base/16384/PG_VERSION, pg_data/base/32768/33000, pg_data/base/32768/33000.32767, pg_data/base/32768/33001, pg_data/base/32768/44000_init, pg_data/base/32768/PG_VERSION, pg_data/changecontent.txt, pg_data/changesize.txt, pg_data/changetime.txt, pg_data/global/pg_control, pg_data/pg_stat/global.stat, pg_data/postgresql.conf, pg_data/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?%, pg_data/zero_from_start, pg_data/zerosize.txt, pg_tblspc/1/[TS_PATH-1]/16384/tablespace1.txt)
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Backup::Backup->process: create backup path [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-1]
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = <false>, strMode = <0750>, strPathExp = <REPO:BACKUP>/[BACKUP-INCR-1]
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-1]/backup.manifest
P00 DEBUG: Storage::Local->exists=>: bExists = false
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->openWrite(): bAtomic = <false>, bPathCreate = <false>, lTimestamp = [undef], rhyFilter = [undef], strCipherPass = [undef], strGroup = [undef], strMode = <0640>, strUser = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-1]/backup.manifest.copy
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Backup::Backup->processManifest(): bCompress = false, bHardLink = false, oBackupManifest = [object], strBackupLabel = [BACKUP-INCR-1], strDbCopyPath = [TEST_PATH]/db-master/db/base, strDbMasterPath = [TEST_PATH]/db-master/db/base, strDbVersion = 9.4, strLsnStart = [undef], strType = incr
P00 DEBUG: Protocol::Local::Process->new(): bConfessError = <true>, iSelectTimeout = <30>, strBackRestBin = <[BACKREST-BIN]>, strHostType = db
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->hostAdd(): iHostConfigIdx = 1, iProcessMax = 1
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/badchecksum.txt, pg_data/badchecksum.txt, 11, [undef], 0, [BACKUP-INCR-1], 0, 3, [MODIFICATION-TIME-2], 1, [undef], 0, 0), rParamSecure = [undef], strKey = pg_data/badchecksum.txt, strOp = backupFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/pg_tblspc/1/[TS_PATH-1]/16384/tablespace1.txt, pg_tblspc/1/[TS_PATH-1]/16384/tablespace1.txt, 7, [undef], 1, [BACKUP-INCR-1], 0, 3, [MODIFICATION-TIME-3], 1, {iWalId => 65535, iWalOffset => 65535}, 0, 0), rParamSecure = [undef], strKey = pg_tblspc/1/[TS_PATH-1]/16384/tablespace1.txt, strOp = backupFile, strQueue = pg_tblspc/1
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/changesize.txt, pg_data/changesize.txt, 4, [undef], 0, [BACKUP-INCR-1], 0, 3, [MODIFICATION-TIME-2], 1, [undef], 0, 0), rParamSecure = [undef], strKey = pg_data/changesize.txt, strOp = backupFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/zerosize.txt, pg_data/zerosize.txt, 0, [undef], 0, [BACKUP-INCR-1], 0, 3, [MODIFICATION-TIME-3], 1, [undef], 0, 0), rParamSecure = [undef], strKey = pg_data/zerosize.txt, strOp = backupFile, strQueue = pg_data
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->hostConnect: start local process: iHostConfigIdx = 1, iHostIdx = 0, iHostProcessIdx = 0, iProcessId = 1, strHostType = db
Add log-subprocess option to allow file logging for local and remote subprocesses.
2018-08-23 02:05:49 +02:00
P00 DEBUG: Protocol::Local::Master->new(): iProcessIdx = 1, strCommand = [BACKREST-BIN] --command=backup --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --db-timeout=45 --host-id=1 --lock-path=[TEST_PATH]/db-master/lock --log-level-file=trace --log-path=[TEST_PATH]/db-master/log --log-subprocess --pg1-path=[TEST_PATH]/db-master/db/base --process=1 --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --stanza=db --type=db local
P00 DEBUG: Protocol::Command::Master->new(): iBufferMax = 4194304, iCompressLevel = 3, iCompressLevelNetwork = 3, iProtocolTimeout = 60, strCommand = [BACKREST-BIN] --command=backup --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --db-timeout=45 --host-id=1 --lock-path=[TEST_PATH]/db-master/lock --log-level-file=trace --log-path=[TEST_PATH]/db-master/log --log-subprocess --pg1-path=[TEST_PATH]/db-master/db/base --process=1 --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --stanza=db --type=db local, strId = local-1 process, strName = local
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->hostConnect=>: bResult = true
P00 DEBUG: Protocol::Local::Process->init: init local process: iDirection = 1, iHostIdx = 0, iProcessId = 1, iQueueIdx = 0, iQueueLastIdx = 1
P00 DEBUG: Protocol::Local::Process->init=>: bResult = true
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/badchecksum.txt, strQueueIdx = 0
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1, 11, 11, f927212cd08d11a42a666b2f04235398e9ceeb51, [undef]), strKey = pg_data/badchecksum.txt
Fix critical bug in resume that resulted in inconsistent backups. A regression in v0.82 removed the timestamp comparison when deciding which files from the aborted backup to keep on resume. All resumed backups should be considered inconsistent. A resumed backup can be identified by checking the log for the message "aborted backup of same type exists, will be cleaned to remove invalid files and resumed". Reported by David Youatt, Yogesh Sharma, Stephen Frost.
2018-07-03 20:01:57 +02:00
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/changesize.txt, strQueueIdx = 0
P01 INFO: backup file [TEST_PATH]/db-master/db/base/badchecksum.txt (11B, 50%) checksum f927212cd08d11a42a666b2f04235398e9ceeb51
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1, 4, 4, 88087292ed82e26f3eb824d0bffc05ccf7a30f8d, [undef]), strKey = pg_data/changesize.txt
Add zero-length file to mock/all test.
2018-07-05 21:40:50 +02:00
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/zerosize.txt, strQueueIdx = 0
Fix critical bug in resume that resulted in inconsistent backups. A regression in v0.82 removed the timestamp comparison when deciding which files from the aborted backup to keep on resume. All resumed backups should be considered inconsistent. A resumed backup can be identified by checking the log for the message "aborted backup of same type exists, will be cleaned to remove invalid files and resumed". Reported by David Youatt, Yogesh Sharma, Stephen Frost.
2018-07-03 20:01:57 +02:00
P01 INFO: backup file [TEST_PATH]/db-master/db/base/changesize.txt (4B, 68%) checksum 88087292ed82e26f3eb824d0bffc05ccf7a30f8d
Add zero-length file to mock/all test.
2018-07-05 21:40:50 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1, 0, 0, da39a3ee5e6b4b0d3255bfef95601890afd80709, [undef]), strKey = pg_data/zerosize.txt
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_tblspc/1/[TS_PATH-1]/16384/tablespace1.txt, strQueueIdx = 1
P01 INFO: backup file [TEST_PATH]/db-master/db/base/zerosize.txt (0B, 68%)
Fix critical bug in resume that resulted in inconsistent backups. A regression in v0.82 removed the timestamp comparison when deciding which files from the aborted backup to keep on resume. All resumed backups should be considered inconsistent. A resumed backup can be identified by checking the log for the message "aborted backup of same type exists, will be cleaned to remove invalid files and resumed". Reported by David Youatt, Yogesh Sharma, Stephen Frost.
2018-07-03 20:01:57 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1, 7, 7, 14c44cef6287269b08d41de489fd492bb9fc795d, {bAlign => 0, bValid => 0}), strKey = pg_tblspc/1/[TS_PATH-1]/16384/tablespace1.txt
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->process: no jobs found, stop local: iHostConfigIdx = [undef], iHostIdx = 0, iProcessId = 1, strHostType = [undef]
P00 DEBUG: Protocol::Command::Master->close=>: iExitStatus = 0
Fix critical bug in resume that resulted in inconsistent backups. A regression in v0.82 removed the timestamp comparison when deciding which files from the aborted backup to keep on resume. All resumed backups should be considered inconsistent. A resumed backup can be identified by checking the log for the message "aborted backup of same type exists, will be cleaned to remove invalid files and resumed". Reported by David Youatt, Yogesh Sharma, Stephen Frost.
2018-07-03 20:01:57 +02:00
P01 INFO: backup file [TEST_PATH]/db-master/db/base/pg_tblspc/1/[TS_PATH-1]/16384/tablespace1.txt (7B, 100%) checksum 14c44cef6287269b08d41de489fd492bb9fc795d
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
P00 WARN: page misalignment in file [TEST_PATH]/db-master/db/base/pg_tblspc/1/[TS_PATH-1]/16384/tablespace1.txt: file size 7 is not divisible by page size 8192
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->process: all jobs complete
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P00 DEBUG: Backup::Backup->processManifest: reference pg_data/PG_VERSION to [BACKUP-FULL-2]
P00 DEBUG: Backup::Backup->processManifest: reference pg_data/base/1/12000 to [BACKUP-FULL-2]
P00 DEBUG: Backup::Backup->processManifest: reference pg_data/base/1/PG_VERSION to [BACKUP-FULL-2]
P00 DEBUG: Backup::Backup->processManifest: reference pg_data/base/16384/17000 to [BACKUP-FULL-2]
P00 DEBUG: Backup::Backup->processManifest: reference pg_data/base/16384/PG_VERSION to [BACKUP-FULL-2]
P00 DEBUG: Backup::Backup->processManifest: reference pg_data/base/32768/33000 to [BACKUP-FULL-2]
P00 DEBUG: Backup::Backup->processManifest: reference pg_data/base/32768/33000.32767 to [BACKUP-FULL-2]
P00 DEBUG: Backup::Backup->processManifest: reference pg_data/base/32768/33001 to [BACKUP-FULL-2]
P00 DEBUG: Backup::Backup->processManifest: reference pg_data/base/32768/44000_init to [BACKUP-FULL-2]
P00 DEBUG: Backup::Backup->processManifest: reference pg_data/base/32768/PG_VERSION to [BACKUP-FULL-2]
P00 DEBUG: Backup::Backup->processManifest: reference pg_data/changecontent.txt to [BACKUP-FULL-2]
P00 DEBUG: Backup::Backup->processManifest: reference pg_data/changetime.txt to [BACKUP-FULL-2]
P00 DEBUG: Backup::Backup->processManifest: reference pg_data/global/pg_control to [BACKUP-FULL-2]
P00 DEBUG: Backup::Backup->processManifest: reference pg_data/pg_stat/global.stat to [BACKUP-FULL-2]
P00 DEBUG: Backup::Backup->processManifest: reference pg_data/postgresql.conf to [BACKUP-FULL-2]
P00 DEBUG: Backup::Backup->processManifest: reference pg_data/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?% to [BACKUP-FULL-2]
P00 DEBUG: Backup::Backup->processManifest: reference pg_data/zero_from_start to [BACKUP-FULL-2]
Fix critical bug in resume that resulted in inconsistent backups. A regression in v0.82 removed the timestamp comparison when deciding which files from the aborted backup to keep on resume. All resumed backups should be considered inconsistent. A resumed backup can be identified by checking the log for the message "aborted backup of same type exists, will be cleaned to remove invalid files and resumed". Reported by David Youatt, Yogesh Sharma, Stephen Frost.
2018-07-03 20:01:57 +02:00
P00 DEBUG: Backup::Backup->processManifest=>: lSizeTotal = 22
P00 INFO: incr backup size = 22B
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Helper::protocolDestroy(): bComplete = true, iRemoteIdx = [undef], strRemoteType = [undef]
P00 DEBUG: Protocol::Helper::protocolDestroy=>: iExitStatus = 0
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Storage::Local->pathSync(): strPathExp = <REPO:BACKUP>/[BACKUP-INCR-1]
Make backup directory sync more efficient. Scanning the entire backup directory can be very expensive if there are a lot of small tables. The backup manifest contains the backup directory list so use it to perform syncs instead of scanning the backup directory.
2018-04-04 03:30:15 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-1]/pg_data
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-1]/pg_data
Make backup directory sync more efficient. Scanning the entire backup directory can be very expensive if there are a lot of small tables. The backup manifest contains the backup directory list so use it to perform syncs instead of scanning the backup directory.
2018-04-04 03:30:15 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-1]/pg_data/base
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-1]/pg_data/base/1
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-1]/pg_data/base/16384
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-1]/pg_data/base/32768
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-1]/pg_data/global
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-1]/pg_data/pg_clog
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-1]/pg_data/pg_dynshmem
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-1]/pg_data/pg_log
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
Make backup directory sync more efficient. Scanning the entire backup directory can be very expensive if there are a lot of small tables. The backup manifest contains the backup directory list so use it to perform syncs instead of scanning the backup directory.
2018-04-04 03:30:15 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-1]/pg_data/pg_notify
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-1]/pg_data/pg_replslot
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-1]/pg_data/pg_serial
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-1]/pg_data/pg_snapshots
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-1]/pg_data/pg_stat
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-1]/pg_data/pg_stat_tmp
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-1]/pg_data/pg_subtrans
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-1]/pg_data/pg_tblspc
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-1]/pg_tblspc
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-1]/pg_tblspc
Make backup directory sync more efficient. Scanning the entire backup directory can be very expensive if there are a lot of small tables. The backup manifest contains the backup directory list so use it to perform syncs instead of scanning the backup directory.
2018-04-04 03:30:15 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-1]/pg_tblspc/1
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-1]/pg_tblspc/1
Make backup directory sync more efficient. Scanning the entire backup directory can be very expensive if there are a lot of small tables. The backup manifest contains the backup directory list so use it to perform syncs instead of scanning the backup directory.
2018-04-04 03:30:15 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-1]/pg_tblspc/1/[TS_PATH-1]
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-1]/pg_tblspc/1/[TS_PATH-1]
Make backup directory sync more efficient. Scanning the entire backup directory can be very expensive if there are a lot of small tables. The backup manifest contains the backup directory list so use it to perform syncs instead of scanning the backup directory.
2018-04-04 03:30:15 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-1]/pg_tblspc/1/[TS_PATH-1]/16384
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-1]/pg_tblspc/1/[TS_PATH-1]/16384
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->openWrite(): bAtomic = <false>, bPathCreate = <false>, lTimestamp = [undef], rhyFilter = [undef], strCipherPass = [undef], strGroup = [undef], strMode = <0640>, strUser = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-1]/backup.manifest
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-1]
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->openWrite(): bAtomic = <false>, bPathCreate = <false>, lTimestamp = [undef], rhyFilter = [undef], strCipherPass = [undef], strGroup = [undef], strMode = <0640>, strUser = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-1]/backup.manifest.copy
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-1]
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: new backup label = [BACKUP-INCR-1]
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->openRead(): bIgnoreMissing = <false>, rhyFilter = [undef], strCipherPass = [undef], xFileExp = <REPO:BACKUP>/[BACKUP-INCR-1]/backup.manifest
Make backup.history sync more efficient. Only the backup.history/[year] directory was being synced, so check if the backup.history is newly created and sync it as well.
2018-04-04 02:54:18 +02:00
P00 DEBUG: Storage::Local->openWrite(): bAtomic = true, bPathCreate = true, lTimestamp = [undef], rhyFilter = ({strClass => pgBackRest::Storage::Filter::Gzip}), strCipherPass = [undef], strGroup = [undef], strMode = <0640>, strUser = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/backup.history/[YEAR-1]/[BACKUP-INCR-1].manifest.gz
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Base->copy(): xDestinationFile = [object], xSourceFile = [object]
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Storage::Local->pathSync(): strPathExp = <REPO:BACKUP>/backup.history
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/backup.history/[YEAR-1]
Add bIgnoreMissing parameter to Local->manifest().
2017-07-25 18:44:38 +02:00
P00 DEBUG: Storage::Local->remove(): bIgnoreMissing = <true>, bRecurse = <false>, xstryPathFileExp = <REPO:BACKUP>/latest
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->remove=>: bRemoved = true
P00 DEBUG: Storage::Local->linkCreate(): bHard = <false>, bIgnoreExists = <false>, bPathCreate = <true>, bRelative = true, strDestinationLinkExp = <REPO:BACKUP>/latest, strSourcePathFileExp = <REPO:BACKUP>/[BACKUP-INCR-1]
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->openWrite(): bAtomic = <false>, bPathCreate = <false>, lTimestamp = [undef], rhyFilter = [undef], strCipherPass = [undef], strGroup = [undef], strMode = <0640>, strUser = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/backup.info
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/repo/backup/db
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->openWrite(): bAtomic = <false>, bPathCreate = <false>, lTimestamp = [undef], rhyFilter = [undef], strCipherPass = [undef], strGroup = [undef], strMode = <0640>, strUser = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/backup.info.copy
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/repo/backup/db
P00 DEBUG: Storage::Local->pathSync(): strPathExp = <REPO:BACKUP>
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: perl/exec::perlExec: => 0
Added a predictable log entry to signal that a command has completed successfully. For example a backup ends successfully with: INFO: backup command end: completed successfully.
2016-12-04 21:44:53 +02:00
P00 INFO: backup command end: completed successfully
Move async forking and more error handling to C. The Perl process was exiting directly when called but that interfered with proper locking for the forked async process. Now Perl returns results to the C process which handles all errors, including signals.
2018-04-13 02:42:26 +02:00
P00 INFO: expire command begin
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: perl/exec::perlExec: (void)
Move async forking and more error handling to C. The Perl process was exiting directly when called but that interfered with proper locking for the forked async process. Now Perl returns results to the C process which handles all errors, including signals.
2018-04-13 02:42:26 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp =
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = true, bIgnoreExists = true, strMode = 0770, strPathExp = [TEST_PATH]/db-master/log
P00 DEBUG: Common::Lock::lockStopTest(): bStanzaStopRequired = <false>
P00 DEBUG: Common::Lock::lockStopTest=>: bStopExists = false
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Backup::Info->new(): bIgnoreMissing = <false>, bLoad = <true>, bRequired = <true>, bValidate = <true>, oStorage = <[object]>, strBackupClusterPath = [TEST_PATH]/db-master/repo/backup/db, strCipherPassSub = [undef]
P00 DEBUG: Storage::Local->encrypted(): bIgnoreMissing = true, strFileName = [TEST_PATH]/db-master/repo/backup/db/backup.info
P00 DEBUG: Storage::Local->encrypted=>: bEncrypted = false
P00 DEBUG: Storage::Local->encryptionValid(): bEncrypted = false
P00 DEBUG: Storage::Local->encryptionValid=>: bValid = true
P00 DEBUG: Storage::Local->openRead(): bIgnoreMissing = true, rhyFilter = [undef], strCipherPass = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/backup.info
P00 DEBUG: Storage::Base->get(): strCipherPass = [undef], xFile = [object]
Move backup modules into Backup directory.
2017-05-15 22:01:00 +02:00
P00 DEBUG: Backup::Info->reconstruct(): bRequired = <true>, bSave = <true>, iCatalogVersion = [undef], iControlVersion = [undef], strDbVersion = [undef], ullDbSysId = [undef]
P00 DEBUG: Backup::Common::backupRegExpGet(): bAnchor = <true>, bDifferential = true, bFull = true, bIncremental = true
P00 DEBUG: Backup::Common::backupRegExpGet=>: strRegExp = ^[0-9]{8}\-[0-9]{6}F(\_[0-9]{8}\-[0-9]{6}(D|I)){0,1}$
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->list(): bIgnoreMissing = <false>, strExpression = ^[0-9]{8}\-[0-9]{6}F(\_[0-9]{8}\-[0-9]{6}(D|I)){0,1}$, strPathExp = [TEST_PATH]/db-master/repo/backup/db, strSortOrder = <forward>
P00 DEBUG: Storage::Local->list=>: stryFileList = ([BACKUP-FULL-2], [BACKUP-INCR-1])
Move backup modules into Backup directory.
2017-05-15 22:01:00 +02:00
P00 DEBUG: Backup::Info->current(): strBackup = [BACKUP-FULL-2]
P00 DEBUG: Backup::Info->current=>: bTest = true
P00 DEBUG: Backup::Info->current(): strBackup = [BACKUP-INCR-1]
P00 DEBUG: Backup::Info->current=>: bTest = true
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]/backup.manifest
P00 DEBUG: Storage::Local->exists=>: bExists = true
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-1]
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-1]/backup.manifest
P00 DEBUG: Storage::Local->exists=>: bExists = true
Move backup modules into Backup directory.
2017-05-15 22:01:00 +02:00
P00 DEBUG: Backup::Common::backupRegExpGet(): bAnchor = <true>, bDifferential = true, bFull = true, bIncremental = true
P00 DEBUG: Backup::Common::backupRegExpGet=>: strRegExp = ^[0-9]{8}\-[0-9]{6}F(\_[0-9]{8}\-[0-9]{6}(D|I)){0,1}$
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->list(): bIgnoreMissing = <false>, strExpression = ^[0-9]{8}\-[0-9]{6}F(\_[0-9]{8}\-[0-9]{6}(D|I)){0,1}$, strPathExp = <REPO:BACKUP>, strSortOrder = reverse
P00 DEBUG: Storage::Local->list=>: stryFileList = ([BACKUP-INCR-1], [BACKUP-FULL-2])
Move backup modules into Backup directory.
2017-05-15 22:01:00 +02:00
P00 DEBUG: Backup::Info->current(): strBackup = [BACKUP-INCR-1]
P00 DEBUG: Backup::Info->current=>: bTest = true
P00 DEBUG: Backup::Info->current(): strBackup = [BACKUP-FULL-2]
P00 DEBUG: Backup::Info->current=>: bTest = true
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
P00 INFO: option 'repo1-retention-archive' is not set - archive logs will not be expired
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: perl/exec::perlExec: => 0
P00 DEBUG: common/exit::exitSafe: (result: 0, error: false, signalType: 0)
P00 DEBUG: common/lock::lockRelease: (failOnNoLock: false)
P00 DEBUG: common/lock::lockRelease: => true
Move async forking and more error handling to C. The Perl process was exiting directly when called but that interfered with proper locking for the forked async process. Now Perl returns results to the C process which handles all errors, including signals.
2018-04-13 02:42:26 +02:00
P00 DEBUG: Main::mainCleanup(): iExitCode = 0
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Helper::protocolDestroy(): bComplete = true, iRemoteIdx = [undef], strRemoteType = [undef]
P00 DEBUG: Protocol::Helper::protocolDestroy=>: iExitStatus = 0
Added a predictable log entry to signal that a command has completed successfully. For example a backup ends successfully with: INFO: backup command end: completed successfully.
2016-12-04 21:44:53 +02:00
P00 INFO: expire command end: completed successfully
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: common/exit::exitSafe: => 0
P00 DEBUG: main::main: => 0
Log testing can now be enabled for certain deterministic tests. This works by comparing the generated logs against a previous copy. Currently only enabled for the backup/synthetic tests.
2015-05-05 19:08:48 +02:00
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
+ supplemental file: [TEST_PATH]/db-master/pgbackrest.conf
----------------------------------------------------------
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[db]
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
pg1-path=[TEST_PATH]/db-master/db/base
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
New simpler configuration and consistent project/exe/path naming. * The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required. * Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case. * The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally. * The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest. * Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory. * Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory. * Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 15:30:54 +02:00
[global]
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
compress=n
Fix custom settings for compress-level option being ignored. Reported by Jens Wilke.
2017-10-23 00:05:46 +02:00
compress-level=3
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
db-timeout=45
lock-path=[TEST_PATH]/db-master/lock
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
log-level-console=debug
log-level-file=trace
Added the log-level-stderr option. Controls whether console log messages are sent to stderr or stdout. By default this is set to warn which represents a change in behavior from previous versions, even though it may be more intuitive. Setting log-level-stderr=off will preserve the old behavior. Suggested by Sascha Biberhofer.
2016-10-05 15:09:30 +02:00
log-level-stderr=off
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
log-path=[TEST_PATH]/db-master/log
Add log-subprocess option to allow file logging for local and remote subprocesses.
2018-08-23 02:05:49 +02:00
log-subprocess=y
Set log-timestamp=n for integration tests. This means less filtering of logs needs to be done and new timestamps can be added without adding new filters.
2018-05-11 17:24:38 +02:00
log-timestamp=n
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
protocol-timeout=60
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
repo1-path=[TEST_PATH]/db-master/repo
Consolidate archive-push tests into new archive test.
2017-06-21 14:38:53 +02:00
spool-path=[TEST_PATH]/db-master/spool
New simpler configuration and consistent project/exe/path naming. * The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required. * Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case. * The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally. * The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest. * Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory. * Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory. * Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 15:30:54 +02:00
[global:backup]
archive-copy=y
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
exclude=postgresql.auto.conf
exclude=pg_log/
exclude=pg_log2
New simpler configuration and consistent project/exe/path naming. * The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required. * Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case. * The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally. * The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest. * Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory. * Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory. * Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 15:30:54 +02:00
start-fast=y
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
+ supplemental file: [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-1]/backup.manifest
-----------------------------------------------------------------------------------------
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[backrest]
backrest-checksum="[CHECKSUM]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
backrest-format=5
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
backrest-version="[VERSION-1]"
[backup]
backup-label="[BACKUP-INCR-1]"
backup-prior="[BACKUP-FULL-2]"
backup-timestamp-copy-start=[TIMESTAMP]
backup-timestamp-start=[TIMESTAMP]
backup-timestamp-stop=[TIMESTAMP]
backup-type="incr"
[backup:db]
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-catalog-version=201409291
db-control-version=942
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
db-id=1
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
db-system-id=1000000000000000094
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-version="9.4"
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[backup:option]
option-archive-check=true
option-archive-copy=true
Backup from a standby cluster. A connection to the primary cluster is still required to start/stop the backup and copy files that are not replicated, but the vast majority of files are copied from the standby in order to reduce load on the master.
2016-08-25 17:25:46 +02:00
option-backup-standby=false
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
option-checksum-page=true
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
option-compress=false
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
option-delta=false
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
option-hardlink=false
Closed #167: Rename `--no-start-stop` option to `--no-online`.
2016-02-20 02:32:02 +02:00
option-online=false
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[backup:target]
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
pg_data={"path":"[TEST_PATH]/db-master/db/base","type":"path"}
pg_tblspc/1={"path":"[TEST_PATH]/db-master/db/tablespace/ts1","tablespace-id":"1","tablespace-name":"ts1","type":"link"}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:file]
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
pg_data/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/badchecksum.txt={"checksum":"f927212cd08d11a42a666b2f04235398e9ceeb51","size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/base/1/12000={"checksum":"22c98d248ff548311eda88559e4a8405ed77c003","checksum-page":true,"master":false,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/base/1/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","master":false,"mode":"0660","reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/base/16384/17000={"checksum":"e0101dd8ffb910c9c202ca35b5f828bcb9697bed","checksum-page":false,"checksum-page-error":[1],"master":false,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/base/16384/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","master":false,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/base/32768/33000={"checksum":"4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f","checksum-page":true,"master":false,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/base/32768/33000.32767={"checksum":"21e2c7c1a326682c07053b7d6a5a40dbd49c2ec5","checksum-page":true,"master":false,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/base/32768/33001={"checksum":"6bf316f11d28c28914ea9be92c00de9bea6d9a6b","checksum-page":false,"checksum-page-error":[0,[3,5],7],"master":false,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/base/32768/44000_init={"checksum":"4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f","checksum-page":true,"master":false,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/base/32768/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","master":false,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/changecontent.txt={"checksum":"238a131a3e8eb98d1fc5b27d882ca40b7618fd2a","reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/changesize.txt={"checksum":"88087292ed82e26f3eb824d0bffc05ccf7a30f8d","size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/changetime.txt={"checksum":"88087292ed82e26f3eb824d0bffc05ccf7a30f8d","reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 11:24:42 +02:00
pg_data/global/pg_control={"checksum":"4c77c900f7af0d9ab13fa9982051a42e0b637f6c","reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-2]}
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
pg_data/pg_stat/global.stat={"checksum":"e350d5ce0153f3e22d5db21cf2a4eff00f3ee877","reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-2]}
pg_data/postgresql.conf={"checksum":"6721d92c9fcdf4248acff1f9a1377127d9064807","reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-2]}
pg_data/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?%={"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/zero_from_start={"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/zerosize.txt={"size":[SIZE],"timestamp":[TIMESTAMP-2]}
pg_tblspc/1/[TS_PATH-1]/16384/tablespace1.txt={"checksum":"14c44cef6287269b08d41de489fd492bb9fc795d","checksum-page":false,"master":false,"size":[SIZE],"timestamp":[TIMESTAMP-2]}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:file:default]
Continuous integration using travis-ci.
2017-02-21 15:59:23 +02:00
group="[GROUP-1]"
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
master=true
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
mode="0600"
Regression tests can now be run as any properly-configured user, not just vagrant.
2016-10-01 19:39:44 +02:00
user="[USER-1]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:link]
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
pg_data/pg_tblspc/1={"destination":"[TEST_PATH]/db-master/db/tablespace/ts1"}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:link:default]
Continuous integration using travis-ci.
2017-02-21 15:59:23 +02:00
group="[GROUP-1]"
Regression tests can now be run as any properly-configured user, not just vagrant.
2016-10-01 19:39:44 +02:00
user="[USER-1]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:path]
Fixed an issue with invalid backup groups being set correctly on restore. If the backup cannot map a group to a name it stores the group in the manifest as false then uses either the owner of $PGDATA to set the group during restore or failing that the group of the current user. This logic was not working correctly because the selected group was overwriting the user on restore leaving the group undefined and the user incorrectly set to the group. (Reported by Jeff McCormick.)
2017-11-20 19:46:35 +02:00
pg_data={"mode":"0777"}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_data/base={}
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
pg_data/base/1={}
pg_data/base/16384={}
pg_data/base/32768={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_data/global={}
pg_data/pg_clog={}
Exclude directories during backup that are cleaned, recreated, or zeroed by PostgreSQL at startup. These include (depending on the version where they were introduced): pgsql_tmp, pg_dynshmem, pg_notify, pg_replslot, pg_serial, pg_snapshots, pg_stat_tmp, pg_subtrans. The postgresql.auto.conf.tmp file is now excluded in addition to files that were already excluded: backup_label.old, postmaster.opts, postmaster.pid, recovery.conf, recovery.done.
2016-08-16 15:35:16 +02:00
pg_data/pg_dynshmem={}
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
pg_data/pg_log={}
Exclude directories during backup that are cleaned, recreated, or zeroed by PostgreSQL at startup. These include (depending on the version where they were introduced): pgsql_tmp, pg_dynshmem, pg_notify, pg_replslot, pg_serial, pg_snapshots, pg_stat_tmp, pg_subtrans. The postgresql.auto.conf.tmp file is now excluded in addition to files that were already excluded: backup_label.old, postmaster.opts, postmaster.pid, recovery.conf, recovery.done.
2016-08-16 15:35:16 +02:00
pg_data/pg_notify={}
pg_data/pg_replslot={}
pg_data/pg_serial={}
pg_data/pg_snapshots={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_data/pg_stat={}
Exclude directories during backup that are cleaned, recreated, or zeroed by PostgreSQL at startup. These include (depending on the version where they were introduced): pgsql_tmp, pg_dynshmem, pg_notify, pg_replslot, pg_serial, pg_snapshots, pg_stat_tmp, pg_subtrans. The postgresql.auto.conf.tmp file is now excluded in addition to files that were already excluded: backup_label.old, postmaster.opts, postmaster.pid, recovery.conf, recovery.done.
2016-08-16 15:35:16 +02:00
pg_data/pg_stat_tmp={}
pg_data/pg_subtrans={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_data/pg_tblspc={}
Fixed an issue with invalid backup groups being set correctly on restore. If the backup cannot map a group to a name it stores the group in the manifest as false then uses either the owner of $PGDATA to set the group during restore or failing that the group of the current user. This logic was not working correctly because the selected group was overwriting the user on restore leaving the group undefined and the user incorrectly set to the group. (Reported by Jeff McCormick.)
2017-11-20 19:46:35 +02:00
pg_tblspc={"mode":"0777"}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_tblspc/1={}
pg_tblspc/1/[TS_PATH-1]={}
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
pg_tblspc/1/[TS_PATH-1]/16384={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:path:default]
Continuous integration using travis-ci.
2017-02-21 15:59:23 +02:00
group="[GROUP-1]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
mode="0700"
Regression tests can now be run as any properly-configured user, not just vagrant.
2016-10-01 19:39:44 +02:00
user="[USER-1]"
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
+ supplemental file: [TEST_PATH]/db-master/repo/backup/db/backup.info
---------------------------------------------------------------------
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[backrest]
backrest-checksum="[CHECKSUM]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
backrest-format=5
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
backrest-version="[VERSION-1]"
[backup:current]
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
[BACKUP-FULL-2]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"full","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":true,"option-compress":false,"option-hardlink":false,"option-online":false}
[BACKUP-INCR-1]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-prior":"[BACKUP-FULL-2]","backup-reference":["[BACKUP-FULL-2]"],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"incr","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":true,"option-compress":false,"option-hardlink":false,"option-online":false}
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[db]
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-catalog-version=201409291
db-control-version=942
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
db-id=1
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
db-system-id=1000000000000000094
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-version="9.4"
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[db:history]
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
1={"db-catalog-version":201409291,"db-control-version":942,"db-system-id":1000000000000000094,"db-version":"9.4"}
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
incr backup - resume and add tablespace 2 (db-master host)
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --no-online --process-max=1 --delta --stanza=db backup --test --test-delay=0.2 --test-point=backup-resume=y
Log testing can now be enabled for certain deterministic tests. This works by comparing the generated logs against a previous copy. Currently only enabled for the backup/synthetic tests.
2015-05-05 19:08:48 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P00 INFO: backup command begin [BACKREST-VERSION]: --no-compress --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --db-timeout=45 --delta --exclude=postgresql.auto.conf --exclude=pg_log/ --exclude=pg_log2 --lock-path=[TEST_PATH]/db-master/lock --log-level-console=debug --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --no-online --pg1-path=[TEST_PATH]/db-master/db/base --process-max=1 --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --stanza=db --start-fast --test --test-delay=0.2 --test-point=backup-resume=y
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: common/lock::lockAcquire: (lockPath: {"[TEST_PATH]/db-master/lock"}, stanza: {"db"}, lockType: 1, lockTimeout: 0, failOnNoLock: true)
P00 DEBUG: common/lock::lockAcquire: => true
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
P00 WARN: option repo1-retention-full is not set, the repository may run out of space
HINT: to retain full backups indefinitely (without warning), set option 'repo1-retention-full' to the maximum.
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: config/load::cfgLoad: => void
P00 DEBUG: perl/exec::perlExec: (void)
Allow functions with sensitive options to be logged at debug level with redactions. Previously, functions with sensitive options had to be logged at trace level to avoid exposing them. Trace level logging may still expose secrets so use with caution.
2017-10-24 18:35:36 +02:00
P00 DEBUG: Storage::Posix::Driver->new(): bFileSync = <true>, bPathSync = <true>
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->new(): bAllowTemp = <true>, hRule = [hash], lBufferMax = 4194304, oDriver = [object], strCipherPassUser = [undef], strCipherType = [undef], strDefaultFileMode = <0640>, strDefaultPathMode = <0750>, strPathBase = [TEST_PATH]/db-master/repo, strTempExtension = pgbackrest.tmp
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp =
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
Config parsing no longer implemented in Perl. Options are passed to Perl as JSON from the C process.
2018-01-29 04:37:09 +02:00
P00 DEBUG: Storage::Posix::Driver->new(): bFileSync = <true>, bPathSync = <true>
P00 DEBUG: Storage::Local->new(): bAllowTemp = <true>, hRule = [undef], lBufferMax = 4194304, oDriver = [object], strCipherPassUser = [undef], strCipherType = [undef], strDefaultFileMode = <0640>, strDefaultPathMode = <0750>, strPathBase = /, strTempExtension = pgbackrest.tmp
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = true, bIgnoreExists = true, strMode = 0770, strPathExp = [TEST_PATH]/db-master/log
Move async forking and more error handling to C. The Perl process was exiting directly when called but that interfered with proper locking for the forked async process. Now Perl returns results to the C process which handles all errors, including signals.
2018-04-13 02:42:26 +02:00
P00 DEBUG: Common::Lock::lockStopTest(): bStanzaStopRequired = <false>
P00 DEBUG: Common::Lock::lockStopTest=>: bStopExists = false
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Backup::Info->new(): bIgnoreMissing = <false>, bLoad = <true>, bRequired = <true>, bValidate = <true>, oStorage = <[object]>, strBackupClusterPath = [TEST_PATH]/db-master/repo/backup/db, strCipherPassSub = [undef]
P00 DEBUG: Storage::Local->encrypted(): bIgnoreMissing = true, strFileName = [TEST_PATH]/db-master/repo/backup/db/backup.info
P00 DEBUG: Storage::Local->encrypted=>: bEncrypted = false
P00 DEBUG: Storage::Local->encryptionValid(): bEncrypted = false
P00 DEBUG: Storage::Local->encryptionValid=>: bValid = true
P00 DEBUG: Storage::Local->openRead(): bIgnoreMissing = true, rhyFilter = [undef], strCipherPass = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/backup.info
P00 DEBUG: Storage::Base->get(): strCipherPass = [undef], xFile = [object]
Move backup modules into Backup directory.
2017-05-15 22:01:00 +02:00
P00 DEBUG: Backup::Info->reconstruct(): bRequired = <true>, bSave = <true>, iCatalogVersion = [undef], iControlVersion = [undef], strDbVersion = [undef], ullDbSysId = [undef]
P00 DEBUG: Backup::Common::backupRegExpGet(): bAnchor = <true>, bDifferential = true, bFull = true, bIncremental = true
P00 DEBUG: Backup::Common::backupRegExpGet=>: strRegExp = ^[0-9]{8}\-[0-9]{6}F(\_[0-9]{8}\-[0-9]{6}(D|I)){0,1}$
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->list(): bIgnoreMissing = <false>, strExpression = ^[0-9]{8}\-[0-9]{6}F(\_[0-9]{8}\-[0-9]{6}(D|I)){0,1}$, strPathExp = [TEST_PATH]/db-master/repo/backup/db, strSortOrder = <forward>
P00 DEBUG: Storage::Local->list=>: stryFileList = ([BACKUP-FULL-2], [BACKUP-INCR-2])
Move backup modules into Backup directory.
2017-05-15 22:01:00 +02:00
P00 DEBUG: Backup::Info->current(): strBackup = [BACKUP-FULL-2]
P00 DEBUG: Backup::Info->current=>: bTest = true
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Backup::Info->current(): strBackup = [BACKUP-INCR-2]
P00 DEBUG: Backup::Info->current=>: bTest = false
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]/backup.manifest
P00 DEBUG: Storage::Local->exists=>: bExists = false
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]/backup.manifest
P00 DEBUG: Storage::Local->exists=>: bExists = true
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-1]
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 WARN: backup [BACKUP-INCR-1] missing in repository removed from backup.info
Move backup modules into Backup directory.
2017-05-15 22:01:00 +02:00
P00 DEBUG: Backup::Info->delete(): strBackupLabel = [BACKUP-INCR-1]
Up to seven standbys can be configured for backup from standby.` Contributed by Cynthia Shang.
2017-09-01 01:15:44 +02:00
P00 DEBUG: Db::dbObjectGet(): bMasterOnly = <false>
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 DEBUG: Db->new(): iRemoteIdx = 1
Abstracted code to determine which database cluster is the master and which are standbys. Contributed by Cynthia Shang.
2016-11-30 21:15:11 +02:00
P00 DEBUG: Db::dbObjectGet=>: iDbMasterIdx = 1, iDbStandbyIdx = [undef], oDbMaster = [object], oDbStandby = [undef]
Allow functions with sensitive options to be logged at debug level with redactions. Previously, functions with sensitive options had to be logged at trace level to avoid exposing them. Trace level logging may still expose secrets so use with caution.
2017-10-24 18:35:36 +02:00
P00 DEBUG: Storage::Posix::Driver->new(): bFileSync = <true>, bPathSync = <true>
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->new(): bAllowTemp = <true>, hRule = [undef], lBufferMax = 4194304, oDriver = [object], strCipherPassUser = [undef], strCipherType = [undef], strDefaultFileMode = <0640>, strDefaultPathMode = <0750>, strPathBase = [TEST_PATH]/db-master/db/base, strTempExtension = pgbackrest.tmp
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 DEBUG: Db->info(): strDbPath = <[TEST_PATH]/db-master/db/base>
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
P00 DEBUG: Db->info=>: iDbCatalogVersion = 201409291, iDbControlVersion = 942, strDbVersion = 9.4, ullDbSysId = 1000000000000000094
P00 DEBUG: Backup::Info->check(): bRequired = <true>, iCatalogVersion = 201409291, iControlVersion = 942, strDbVersion = 9.4, ullDbSysId = 1000000000000000094
Move backup modules into Backup directory.
2017-05-15 22:01:00 +02:00
P00 DEBUG: Backup::Info->check=>: iDbHistoryId = 1
P00 DEBUG: Backup::Info->last(): strType = incr
P00 DEBUG: Backup::Common::backupRegExpGet(): bAnchor = <true>, bDifferential = true, bFull = true, bIncremental = true
P00 DEBUG: Backup::Common::backupRegExpGet=>: strRegExp = ^[0-9]{8}\-[0-9]{6}F(\_[0-9]{8}\-[0-9]{6}(D|I)){0,1}$
P00 DEBUG: Backup::Info->list(): strFilter = ^[0-9]{8}\-[0-9]{6}F(\_[0-9]{8}\-[0-9]{6}(D|I)){0,1}$, strOrder = reverse
P00 DEBUG: Backup::Info->list=>: stryBackup = ([BACKUP-FULL-2])
P00 DEBUG: Backup::Info->last=>: strBackup = [BACKUP-FULL-2]
P00 DEBUG: Backup::Info->dbHistoryList=>: hDbHash = [hash]
P00 DEBUG: Backup::Info->confirmDb=>: bConfirmDb = true
Require PostgreSQL catalog version when instantiating a Manifest object (and not loading it from disk). Contributed by Cynthia Shang.
2018-07-16 23:25:15 +02:00
P00 DEBUG: Manifest->new(): bLoad = <true>, iDbCatalogVersion = [undef], oStorage = <[object]>, strCipherPass = [undef], strCipherPassSub = [undef], strDbVersion = [undef], strFileName = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]/backup.manifest
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->encrypted(): bIgnoreMissing = true, strFileName = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]/backup.manifest
P00 DEBUG: Storage::Local->encrypted=>: bEncrypted = false
P00 DEBUG: Storage::Local->encryptionValid(): bEncrypted = false
P00 DEBUG: Storage::Local->encryptionValid=>: bValid = true
P00 DEBUG: Storage::Local->openRead(): bIgnoreMissing = true, rhyFilter = [undef], strCipherPass = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]/backup.manifest
P00 DEBUG: Storage::Base->get(): strCipherPass = [undef], xFile = [object]
Improvements and additional tests for stanza-upgrade. Contributed by Cynthia Shang.
2017-04-07 01:36:37 +02:00
P00 INFO: last backup label = [BACKUP-FULL-2], version = [VERSION-1]
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Backup::Common::backupRegExpGet(): bAnchor = <true>, bDifferential = true, bFull = true, bIncremental = true
P00 DEBUG: Backup::Common::backupRegExpGet=>: strRegExp = ^[0-9]{8}\-[0-9]{6}F(\_[0-9]{8}\-[0-9]{6}(D|I)){0,1}$
P00 DEBUG: Storage::Local->list(): bIgnoreMissing = <false>, strExpression = ^[0-9]{8}\-[0-9]{6}F(\_[0-9]{8}\-[0-9]{6}(D|I)){0,1}$, strPathExp = <REPO:BACKUP>, strSortOrder = reverse
P00 DEBUG: Storage::Local->list=>: stryFileList = ([BACKUP-INCR-2], [BACKUP-FULL-2])
P00 DEBUG: Storage::Local->exists(): strFileExp = <REPO:BACKUP>/[BACKUP-INCR-2]/backup.manifest.copy
P00 DEBUG: Storage::Local->exists=>: bExists = true
P00 DEBUG: Storage::Local->exists(): strFileExp = <REPO:BACKUP>/[BACKUP-INCR-2]/backup.manifest
P00 DEBUG: Storage::Local->exists=>: bExists = false
Require PostgreSQL catalog version when instantiating a Manifest object (and not loading it from disk). Contributed by Cynthia Shang.
2018-07-16 23:25:15 +02:00
P00 DEBUG: Manifest->new(): bLoad = <true>, iDbCatalogVersion = [undef], oStorage = <[object]>, strCipherPass = [undef], strCipherPassSub = [undef], strDbVersion = [undef], strFileName = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]/backup.manifest
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->encrypted(): bIgnoreMissing = true, strFileName = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]/backup.manifest
P00 DEBUG: Storage::Local->encrypted=>: bEncrypted = false
P00 DEBUG: Storage::Local->encryptionValid(): bEncrypted = false
P00 DEBUG: Storage::Local->encryptionValid=>: bValid = true
P00 DEBUG: Storage::Local->openRead(): bIgnoreMissing = true, rhyFilter = [undef], strCipherPass = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]/backup.manifest
P00 DEBUG: Storage::Base->get(): strCipherPass = [undef], xFile = [undef]
P00 DEBUG: Storage::Local->encrypted(): bIgnoreMissing = true, strFileName = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]/backup.manifest.copy
P00 DEBUG: Storage::Local->encrypted=>: bEncrypted = false
P00 DEBUG: Storage::Local->encryptionValid(): bEncrypted = false
P00 DEBUG: Storage::Local->encryptionValid=>: bValid = true
P00 DEBUG: Storage::Local->openRead(): bIgnoreMissing = true, rhyFilter = [undef], strCipherPass = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]/backup.manifest.copy
P00 DEBUG: Storage::Base->get(): strCipherPass = [undef], xFile = [object]
Require PostgreSQL catalog version when instantiating a Manifest object (and not loading it from disk). Contributed by Cynthia Shang.
2018-07-16 23:25:15 +02:00
P00 DEBUG: Manifest->new(): bLoad = false, iDbCatalogVersion = 201409291, oStorage = <[object]>, strCipherPass = [undef], strCipherPassSub = [undef], strDbVersion = 9.4, strFileName = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]/backup.manifest
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/db/base/postmaster.pid
P00 DEBUG: Storage::Local->exists=>: bExists = false
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
P00 WARN: incr backup cannot alter 'checksum-page' option to 'false', reset to 'true' from [BACKUP-FULL-2]
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: Manifest->build(): bDelta = true, bOnline = false, bTablespace = [undef], hDatabaseMap = [undef], hTablespaceMap = [undef], iLevel = <0>, oLastManifest = [object], oStorageDbMaster = [object], rhExclude = [hash], strFilter = [undef], strLevel = [undef], strParentPath = [undef], strPath = [TEST_PATH]/db-master/db/base, strTimelineCurrent = [undef], strTimelineLast = [undef]
Ignore all files in a linked tablespace directory except the subdirectory for the current version of PostgreSQL. Previously an error would be generated if other files were present and not owned by the PostgreSQL user. This hasn't been a big deal in practice but it could cause issues. Also add tests to make sure the same logic applies with links to files, i.e. all other files in the directory should be ignored. This was actually working correctly, but there were no tests for it before.
2018-08-31 22:06:40 +02:00
P00 DEBUG: Storage::Local->manifest(): strFilter = [undef], strPathExp = [TEST_PATH]/db-master/db/base/pg_tblspc
Simplified the result hash of File->manifest(), Db->tablespaceMapGet(), and Db->databaseMapGet().
2016-11-30 21:36:39 +02:00
P00 DEBUG: Manifest->build: found tablespace 1 in offline mode
P00 DEBUG: Manifest->build: found tablespace 11 in offline mode
P00 DEBUG: Manifest->build: found tablespace 2 in offline mode
Ignore all files in a linked tablespace directory except the subdirectory for the current version of PostgreSQL. Previously an error would be generated if other files were present and not owned by the PostgreSQL user. This hasn't been a big deal in practice but it could cause issues. Also add tests to make sure the same logic applies with links to files, i.e. all other files in the directory should be ignored. This was actually working correctly, but there were no tests for it before.
2018-08-31 22:06:40 +02:00
P00 DEBUG: Storage::Local->manifest(): strFilter = [undef], strPathExp = [TEST_PATH]/db-master/db/base
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: Manifest->build(): bDelta = true, bOnline = false, bTablespace = true, hDatabaseMap = [undef], hTablespaceMap = [hash], iLevel = 1, oLastManifest = [undef], oStorageDbMaster = [object], rhExclude = [hash], strFilter = [TS_PATH-1], strLevel = pg_tblspc/1, strParentPath = [TEST_PATH]/db-master/db/base/pg_tblspc, strPath = [TEST_PATH]/db-master/db/tablespace/ts1, strTimelineCurrent = [undef], strTimelineLast = [undef]
Ignore all files in a linked tablespace directory except the subdirectory for the current version of PostgreSQL. Previously an error would be generated if other files were present and not owned by the PostgreSQL user. This hasn't been a big deal in practice but it could cause issues. Also add tests to make sure the same logic applies with links to files, i.e. all other files in the directory should be ignored. This was actually working correctly, but there were no tests for it before.
2018-08-31 22:06:40 +02:00
P00 DEBUG: Storage::Local->manifest(): strFilter = [TS_PATH-1], strPathExp = [TEST_PATH]/db-master/db/tablespace/ts1
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: Manifest->build(): bDelta = true, bOnline = false, bTablespace = true, hDatabaseMap = [undef], hTablespaceMap = [hash], iLevel = 1, oLastManifest = [undef], oStorageDbMaster = [object], rhExclude = [hash], strFilter = [TS_PATH-1], strLevel = pg_tblspc/11, strParentPath = [TEST_PATH]/db-master/db/base/pg_tblspc, strPath = [TEST_PATH]/db-master/db/tablespace/ts11, strTimelineCurrent = [undef], strTimelineLast = [undef]
Ignore all files in a linked tablespace directory except the subdirectory for the current version of PostgreSQL. Previously an error would be generated if other files were present and not owned by the PostgreSQL user. This hasn't been a big deal in practice but it could cause issues. Also add tests to make sure the same logic applies with links to files, i.e. all other files in the directory should be ignored. This was actually working correctly, but there were no tests for it before.
2018-08-31 22:06:40 +02:00
P00 DEBUG: Storage::Local->manifest(): strFilter = [TS_PATH-1], strPathExp = [TEST_PATH]/db-master/db/tablespace/ts11
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: Manifest->build(): bDelta = true, bOnline = false, bTablespace = true, hDatabaseMap = [undef], hTablespaceMap = [hash], iLevel = 1, oLastManifest = [undef], oStorageDbMaster = [object], rhExclude = [hash], strFilter = [TS_PATH-1], strLevel = pg_tblspc/2, strParentPath = [TEST_PATH]/db-master/db/base/pg_tblspc, strPath = [TEST_PATH]/db-master/db/tablespace/ts2, strTimelineCurrent = [undef], strTimelineLast = [undef]
Ignore all files in a linked tablespace directory except the subdirectory for the current version of PostgreSQL. Previously an error would be generated if other files were present and not owned by the PostgreSQL user. This hasn't been a big deal in practice but it could cause issues. Also add tests to make sure the same logic applies with links to files, i.e. all other files in the directory should be ignored. This was actually working correctly, but there were no tests for it before.
2018-08-31 22:06:40 +02:00
P00 DEBUG: Storage::Local->manifest(): strFilter = [TS_PATH-1], strPathExp = [TEST_PATH]/db-master/db/tablespace/ts2
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 WARN: aborted backup [BACKUP-INCR-2] of same type exists, will be cleaned to remove invalid files and resumed
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 TEST: PgBaCkReStTeSt-BACKUP-RESUME-PgBaCkReStTeSt
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: Backup::Backup->resumeClean(): bDelta = true, bOnline = false, oAbortedManifest = [object], oManifest = [object], oStorageRepo = [object], strBackupLabel = [BACKUP-INCR-2], strTimelineCurrent = [undef], strTimelineLast = [undef]
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DETAIL: clean resumed backup path: [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]
Ignore all files in a linked tablespace directory except the subdirectory for the current version of PostgreSQL. Previously an error would be generated if other files were present and not owned by the PostgreSQL user. This hasn't been a big deal in practice but it could cause issues. Also add tests to make sure the same logic applies with links to files, i.e. all other files in the directory should be ignored. This was actually working correctly, but there were no tests for it before.
2018-08-31 22:06:40 +02:00
P00 DEBUG: Storage::Local->manifest(): strFilter = [undef], strPathExp = <REPO:BACKUP>/[BACKUP-INCR-2]
Fix critical bug in resume that resulted in inconsistent backups. A regression in v0.82 removed the timestamp comparison when deciding which files from the aborted backup to keep on resume. All resumed backups should be considered inconsistent. A resumed backup can be identified by checking the log for the message "aborted backup of same type exists, will be cleaned to remove invalid files and resumed". Reported by David Youatt, Yogesh Sharma, Stephen Frost.
2018-07-03 20:01:57 +02:00
P00 DEBUG: Backup::Backup->resumeClean: remove file pg_data/changesize.txt
Add zero-length file to mock/all test.
2018-07-05 21:40:50 +02:00
P00 DEBUG: Backup::Backup->resumeClean: remove file pg_data/zerosize.txt
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P00 DEBUG: Storage::Local->remove(): bIgnoreMissing = <true>, bRecurse = <false>, xstryPathFileExp = (<REPO:BACKUP>/[BACKUP-INCR-2]/pg_data/changesize.txt, <REPO:BACKUP>/[BACKUP-INCR-2]/pg_data/zerosize.txt)
Fix critical bug in resume that resulted in inconsistent backups. A regression in v0.82 removed the timestamp comparison when deciding which files from the aborted backup to keep on resume. All resumed backups should be considered inconsistent. A resumed backup can be identified by checking the log for the message "aborted backup of same type exists, will be cleaned to remove invalid files and resumed". Reported by David Youatt, Yogesh Sharma, Stephen Frost.
2018-07-03 20:01:57 +02:00
P00 DEBUG: Storage::Local->remove=>: bRemoved = true
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]/backup.manifest
P00 DEBUG: Storage::Local->exists=>: bExists = false
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->openWrite(): bAtomic = <false>, bPathCreate = <false>, lTimestamp = [undef], rhyFilter = [undef], strCipherPass = [undef], strGroup = [undef], strMode = <0640>, strUser = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]/backup.manifest.copy
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Backup::Backup->processManifest(): bCompress = false, bHardLink = false, oBackupManifest = [object], strBackupLabel = [BACKUP-INCR-2], strDbCopyPath = [TEST_PATH]/db-master/db/base, strDbMasterPath = [TEST_PATH]/db-master/db/base, strDbVersion = 9.4, strLsnStart = [undef], strType = incr
P00 DEBUG: Protocol::Local::Process->new(): bConfessError = <true>, iSelectTimeout = <30>, strBackRestBin = <[BACKREST-BIN]>, strHostType = db
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->hostAdd(): iHostConfigIdx = 1, iProcessMax = 1
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/base/32768/33001, pg_data/base/32768/33001, 65536, 6bf316f11d28c28914ea9be92c00de9bea6d9a6b, 1, [BACKUP-INCR-2], 0, 3, [MODIFICATION-TIME-2], 1, {iWalId => 65535, iWalOffset => 65535}, 1, 1), rParamSecure = [undef], strKey = pg_data/base/32768/33001, strOp = backupFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/base/32768/44000_init, pg_data/base/32768/44000_init, 32768, 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f, 1, [BACKUP-INCR-2], 0, 3, [MODIFICATION-TIME-2], 1, {iWalId => 65535, iWalOffset => 65535}, 1, 1), rParamSecure = [undef], strKey = pg_data/base/32768/44000_init, strOp = backupFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/base/32768/33000.32767, pg_data/base/32768/33000.32767, 32768, 21e2c7c1a326682c07053b7d6a5a40dbd49c2ec5, 1, [BACKUP-INCR-2], 0, 3, [MODIFICATION-TIME-2], 1, {iWalId => 65535, iWalOffset => 65535}, 1, 1), rParamSecure = [undef], strKey = pg_data/base/32768/33000.32767, strOp = backupFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/base/32768/33000, pg_data/base/32768/33000, 32768, 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f, 1, [BACKUP-INCR-2], 0, 3, [MODIFICATION-TIME-2], 1, {iWalId => 65535, iWalOffset => 65535}, 1, 1), rParamSecure = [undef], strKey = pg_data/base/32768/33000, strOp = backupFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/base/16384/17000, pg_data/base/16384/17000, 16384, e0101dd8ffb910c9c202ca35b5f828bcb9697bed, 1, [BACKUP-INCR-2], 0, 3, [MODIFICATION-TIME-2], 1, {iWalId => 65535, iWalOffset => 65535}, 1, 1), rParamSecure = [undef], strKey = pg_data/base/16384/17000, strOp = backupFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/global/pg_control, pg_data/global/pg_control, 8192, 4c77c900f7af0d9ab13fa9982051a42e0b637f6c, 0, [BACKUP-INCR-2], 0, 3, [MODIFICATION-TIME-3], 0, [undef], 1, 1), rParamSecure = [undef], strKey = pg_data/global/pg_control, strOp = backupFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/base/1/12000, pg_data/base/1/12000, 8192, 22c98d248ff548311eda88559e4a8405ed77c003, 1, [BACKUP-INCR-2], 0, 3, [MODIFICATION-TIME-2], 1, {iWalId => 65535, iWalOffset => 65535}, 1, 1), rParamSecure = [undef], strKey = pg_data/base/1/12000, strOp = backupFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/postgresql.conf, pg_data/postgresql.conf, 21, 6721d92c9fcdf4248acff1f9a1377127d9064807, 0, [BACKUP-INCR-2], 0, 3, [MODIFICATION-TIME-3], 1, [undef], 1, 1), rParamSecure = [undef], strKey = pg_data/postgresql.conf, strOp = backupFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/badchecksum.txt, pg_data/badchecksum.txt, 11, f927212cd08d11a42a666b2f04235398e9ceeb51, 0, [BACKUP-INCR-2], 0, 3, [MODIFICATION-TIME-2], 1, [undef], 1, 0), rParamSecure = [undef], strKey = pg_data/badchecksum.txt, strOp = backupFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/changesize.txt, pg_data/changesize.txt, 9, [undef], 0, [BACKUP-INCR-2], 0, 3, [MODIFICATION-TIME-2], 1, [undef], 1, 0), rParamSecure = [undef], strKey = pg_data/changesize.txt, strOp = backupFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/pg_tblspc/2/[TS_PATH-1]/32768/tablespace2.txt, pg_tblspc/2/[TS_PATH-1]/32768/tablespace2.txt, 7, [undef], 1, [BACKUP-INCR-2], 0, 3, [MODIFICATION-TIME-2], 1, {iWalId => 65535, iWalOffset => 65535}, 1, 0), rParamSecure = [undef], strKey = pg_tblspc/2/[TS_PATH-1]/32768/tablespace2.txt, strOp = backupFile, strQueue = pg_tblspc/2
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/pg_tblspc/1/[TS_PATH-1]/16384/tablespace1.txt, pg_tblspc/1/[TS_PATH-1]/16384/tablespace1.txt, 7, 14c44cef6287269b08d41de489fd492bb9fc795d, 1, [BACKUP-INCR-2], 0, 3, [MODIFICATION-TIME-2], 1, {iWalId => 65535, iWalOffset => 65535}, 1, 0), rParamSecure = [undef], strKey = pg_tblspc/1/[TS_PATH-1]/16384/tablespace1.txt, strOp = backupFile, strQueue = pg_tblspc/1
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/changecontent.txt, pg_data/changecontent.txt, 7, 238a131a3e8eb98d1fc5b27d882ca40b7618fd2a, 0, [BACKUP-INCR-2], 0, 3, [MODIFICATION-TIME-2], 1, [undef], 1, 1), rParamSecure = [undef], strKey = pg_data/changecontent.txt, strOp = backupFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/pg_stat/global.stat, pg_data/pg_stat/global.stat, 5, e350d5ce0153f3e22d5db21cf2a4eff00f3ee877, 0, [BACKUP-INCR-2], 0, 3, [MODIFICATION-TIME-3], 1, [undef], 1, 1), rParamSecure = [undef], strKey = pg_data/pg_stat/global.stat, strOp = backupFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/changetime.txt, pg_data/changetime.txt, 4, 88087292ed82e26f3eb824d0bffc05ccf7a30f8d, 0, [BACKUP-INCR-2], 0, 3, [MODIFICATION-TIME-3], 1, [undef], 1, 1), rParamSecure = [undef], strKey = pg_data/changetime.txt, strOp = backupFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/base/32768/PG_VERSION, pg_data/base/32768/PG_VERSION, 3, 184473f470864e067ee3a22e64b47b0a1c356f29, 0, [BACKUP-INCR-2], 0, 3, [MODIFICATION-TIME-2], 1, [undef], 1, 1), rParamSecure = [undef], strKey = pg_data/base/32768/PG_VERSION, strOp = backupFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/base/16384/PG_VERSION, pg_data/base/16384/PG_VERSION, 3, 184473f470864e067ee3a22e64b47b0a1c356f29, 0, [BACKUP-INCR-2], 0, 3, [MODIFICATION-TIME-2], 1, [undef], 1, 1), rParamSecure = [undef], strKey = pg_data/base/16384/PG_VERSION, strOp = backupFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/base/1/PG_VERSION, pg_data/base/1/PG_VERSION, 3, 184473f470864e067ee3a22e64b47b0a1c356f29, 0, [BACKUP-INCR-2], 0, 3, [MODIFICATION-TIME-2], 1, [undef], 1, 1), rParamSecure = [undef], strKey = pg_data/base/1/PG_VERSION, strOp = backupFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/PG_VERSION, pg_data/PG_VERSION, 3, 184473f470864e067ee3a22e64b47b0a1c356f29, 0, [BACKUP-INCR-2], 0, 3, [MODIFICATION-TIME-2], 1, [undef], 1, 1), rParamSecure = [undef], strKey = pg_data/PG_VERSION, strOp = backupFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base/zerosize.txt, pg_data/zerosize.txt, 0, [undef], 0, [BACKUP-INCR-2], 0, 3, [MODIFICATION-TIME-3], 1, [undef], 1, 0), rParamSecure = [undef], strKey = pg_data/zerosize.txt, strOp = backupFile, strQueue = pg_data
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->hostConnect: start local process: iHostConfigIdx = 1, iHostIdx = 0, iHostProcessIdx = 0, iProcessId = 1, strHostType = db
Add log-subprocess option to allow file logging for local and remote subprocesses.
2018-08-23 02:05:49 +02:00
P00 DEBUG: Protocol::Local::Master->new(): iProcessIdx = 1, strCommand = [BACKREST-BIN] --command=backup --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --db-timeout=45 --host-id=1 --lock-path=[TEST_PATH]/db-master/lock --log-level-file=trace --log-path=[TEST_PATH]/db-master/log --log-subprocess --pg1-path=[TEST_PATH]/db-master/db/base --process=1 --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --stanza=db --type=db local
P00 DEBUG: Protocol::Command::Master->new(): iBufferMax = 4194304, iCompressLevel = 3, iCompressLevelNetwork = 3, iProtocolTimeout = 60, strCommand = [BACKREST-BIN] --command=backup --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --db-timeout=45 --host-id=1 --lock-path=[TEST_PATH]/db-master/lock --log-level-file=trace --log-path=[TEST_PATH]/db-master/log --log-subprocess --pg1-path=[TEST_PATH]/db-master/db/base --process=1 --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --stanza=db --type=db local, strId = local-1 process, strName = local
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->hostConnect=>: bResult = true
P00 DEBUG: Protocol::Local::Process->init: init local process: iDirection = 1, iHostIdx = 0, iProcessId = 1, iQueueIdx = 0, iQueueLastIdx = 2
P00 DEBUG: Protocol::Local::Process->init=>: bResult = true
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/base/32768/33001, strQueueIdx = 0
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (4, 65536, [undef], 6bf316f11d28c28914ea9be92c00de9bea6d9a6b, [undef]), strKey = pg_data/base/32768/33001
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/base/32768/44000_init, strQueueIdx = 0
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base/base/32768/33001 (64KB, 33%) checksum 6bf316f11d28c28914ea9be92c00de9bea6d9a6b
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (4, 32768, [undef], 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f, [undef]), strKey = pg_data/base/32768/44000_init
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/base/32768/33000.32767, strQueueIdx = 0
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base/base/32768/44000_init (32KB, 49%) checksum 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (4, 32768, [undef], 21e2c7c1a326682c07053b7d6a5a40dbd49c2ec5, [undef]), strKey = pg_data/base/32768/33000.32767
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/base/32768/33000, strQueueIdx = 0
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base/base/32768/33000.32767 (32KB, 66%) checksum 21e2c7c1a326682c07053b7d6a5a40dbd49c2ec5
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (4, 32768, [undef], 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f, [undef]), strKey = pg_data/base/32768/33000
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/base/16384/17000, strQueueIdx = 0
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base/base/32768/33000 (32KB, 83%) checksum 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (4, 16384, [undef], e0101dd8ffb910c9c202ca35b5f828bcb9697bed, [undef]), strKey = pg_data/base/16384/17000
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/global/pg_control, strQueueIdx = 0
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base/base/16384/17000 (16KB, 91%) checksum e0101dd8ffb910c9c202ca35b5f828bcb9697bed
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 11:24:42 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (4, 8192, [undef], 4c77c900f7af0d9ab13fa9982051a42e0b637f6c, [undef]), strKey = pg_data/global/pg_control
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/base/1/12000, strQueueIdx = 0
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 11:24:42 +02:00
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base/global/pg_control (8KB, 95%) checksum 4c77c900f7af0d9ab13fa9982051a42e0b637f6c
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (4, 8192, [undef], 22c98d248ff548311eda88559e4a8405ed77c003, [undef]), strKey = pg_data/base/1/12000
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/postgresql.conf, strQueueIdx = 0
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base/base/1/12000 (8KB, 99%) checksum 22c98d248ff548311eda88559e4a8405ed77c003
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (4, 21, [undef], 6721d92c9fcdf4248acff1f9a1377127d9064807, [undef]), strKey = pg_data/postgresql.conf
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/badchecksum.txt, strQueueIdx = 0
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base/postgresql.conf (21B, 99%) checksum 6721d92c9fcdf4248acff1f9a1377127d9064807
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (2, 11, 11, f927212cd08d11a42a666b2f04235398e9ceeb51, [undef]), strKey = pg_data/badchecksum.txt
Fix critical bug in resume that resulted in inconsistent backups. A regression in v0.82 removed the timestamp comparison when deciding which files from the aborted backup to keep on resume. All resumed backups should be considered inconsistent. A resumed backup can be identified by checking the log for the message "aborted backup of same type exists, will be cleaned to remove invalid files and resumed". Reported by David Youatt, Yogesh Sharma, Stephen Frost.
2018-07-03 20:01:57 +02:00
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/changesize.txt, strQueueIdx = 0
P00 WARN: resumed backup file pg_data/badchecksum.txt does not have expected checksum f927212cd08d11a42a666b2f04235398e9ceeb51. The file will be recopied and backup will continue but this may be an issue unless the resumed backup path in the repository is known to be corrupted.
NOTE: this does not indicate a problem with the PostgreSQL page checksums.
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P01 INFO: backup file [TEST_PATH]/db-master/db/base/badchecksum.txt (11B, 99%) checksum f927212cd08d11a42a666b2f04235398e9ceeb51
Fix critical bug in resume that resulted in inconsistent backups. A regression in v0.82 removed the timestamp comparison when deciding which files from the aborted backup to keep on resume. All resumed backups should be considered inconsistent. A resumed backup can be identified by checking the log for the message "aborted backup of same type exists, will be cleaned to remove invalid files and resumed". Reported by David Youatt, Yogesh Sharma, Stephen Frost.
2018-07-03 20:01:57 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1, 9, 9, 3905d5be2ec8d67f41435dab5e0dcda3ae47455d, [undef]), strKey = pg_data/changesize.txt
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/changecontent.txt, strQueueIdx = 0
P01 INFO: backup file [TEST_PATH]/db-master/db/base/changesize.txt (9B, 99%) checksum 3905d5be2ec8d67f41435dab5e0dcda3ae47455d
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1, 7, 7, a094d94583e209556d03c3c5da33131a065f1689, [undef]), strKey = pg_data/changecontent.txt
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/pg_stat/global.stat, strQueueIdx = 0
P01 INFO: backup file [TEST_PATH]/db-master/db/base/changecontent.txt (7B, 99%) checksum a094d94583e209556d03c3c5da33131a065f1689
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (4, 5, [undef], e350d5ce0153f3e22d5db21cf2a4eff00f3ee877, [undef]), strKey = pg_data/pg_stat/global.stat
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/changetime.txt, strQueueIdx = 0
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base/pg_stat/global.stat (5B, 99%) checksum e350d5ce0153f3e22d5db21cf2a4eff00f3ee877
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (4, 4, [undef], 88087292ed82e26f3eb824d0bffc05ccf7a30f8d, [undef]), strKey = pg_data/changetime.txt
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/base/32768/PG_VERSION, strQueueIdx = 0
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base/changetime.txt (4B, 99%) checksum 88087292ed82e26f3eb824d0bffc05ccf7a30f8d
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (4, 3, [undef], 184473f470864e067ee3a22e64b47b0a1c356f29, [undef]), strKey = pg_data/base/32768/PG_VERSION
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/base/16384/PG_VERSION, strQueueIdx = 0
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base/base/32768/PG_VERSION (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (4, 3, [undef], 184473f470864e067ee3a22e64b47b0a1c356f29, [undef]), strKey = pg_data/base/16384/PG_VERSION
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/base/1/PG_VERSION, strQueueIdx = 0
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base/base/16384/PG_VERSION (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (4, 3, [undef], 184473f470864e067ee3a22e64b47b0a1c356f29, [undef]), strKey = pg_data/base/1/PG_VERSION
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/PG_VERSION, strQueueIdx = 0
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base/base/1/PG_VERSION (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (4, 3, [undef], 184473f470864e067ee3a22e64b47b0a1c356f29, [undef]), strKey = pg_data/PG_VERSION
Add zero-length file to mock/all test.
2018-07-05 21:40:50 +02:00
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/zerosize.txt, strQueueIdx = 0
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base/PG_VERSION (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
Add zero-length file to mock/all test.
2018-07-05 21:40:50 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1, 0, 0, da39a3ee5e6b4b0d3255bfef95601890afd80709, [undef]), strKey = pg_data/zerosize.txt
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_tblspc/2/[TS_PATH-1]/32768/tablespace2.txt, strQueueIdx = 1
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P01 INFO: backup file [TEST_PATH]/db-master/db/base/zerosize.txt (0B, 99%)
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1, 7, 7, dc7f76e43c46101b47acc55ae4d593a9e6983578, {bAlign => 0, bValid => 0}), strKey = pg_tblspc/2/[TS_PATH-1]/32768/tablespace2.txt
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_tblspc/1/[TS_PATH-1]/16384/tablespace1.txt, strQueueIdx = 2
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P01 INFO: backup file [TEST_PATH]/db-master/db/base/pg_tblspc/2/[TS_PATH-1]/32768/tablespace2.txt (7B, 99%) checksum dc7f76e43c46101b47acc55ae4d593a9e6983578
Additional backup exclusions. * Exclude contents of pg_snapshots, pg_serial, pg_notify, and pg_dynshmem from backup since they are rebuilt on startup. * Exclude pg_internal.init files from backup since they are rebuilt on startup.
2017-09-04 14:26:57 +02:00
P00 WARN: page misalignment in file [TEST_PATH]/db-master/db/base/pg_tblspc/2/[TS_PATH-1]/32768/tablespace2.txt: file size 7 is not divisible by page size 8192
Fix critical bug in resume that resulted in inconsistent backups. A regression in v0.82 removed the timestamp comparison when deciding which files from the aborted backup to keep on resume. All resumed backups should be considered inconsistent. A resumed backup can be identified by checking the log for the message "aborted backup of same type exists, will be cleaned to remove invalid files and resumed". Reported by David Youatt, Yogesh Sharma, Stephen Frost.
2018-07-03 20:01:57 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1, 7, 7, d85de07d6421d90aa9191c11c889bfde43680f0f, {bAlign => 0, bValid => 0}), strKey = pg_tblspc/1/[TS_PATH-1]/16384/tablespace1.txt
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->process: no jobs found, stop local: iHostConfigIdx = [undef], iHostIdx = 0, iProcessId = 1, strHostType = [undef]
P00 DEBUG: Protocol::Command::Master->close=>: iExitStatus = 0
Fix critical bug in resume that resulted in inconsistent backups. A regression in v0.82 removed the timestamp comparison when deciding which files from the aborted backup to keep on resume. All resumed backups should be considered inconsistent. A resumed backup can be identified by checking the log for the message "aborted backup of same type exists, will be cleaned to remove invalid files and resumed". Reported by David Youatt, Yogesh Sharma, Stephen Frost.
2018-07-03 20:01:57 +02:00
P01 INFO: backup file [TEST_PATH]/db-master/db/base/pg_tblspc/1/[TS_PATH-1]/16384/tablespace1.txt (7B, 100%) checksum d85de07d6421d90aa9191c11c889bfde43680f0f
P00 WARN: page misalignment in file [TEST_PATH]/db-master/db/base/pg_tblspc/1/[TS_PATH-1]/16384/tablespace1.txt: file size 7 is not divisible by page size 8192
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Local::Process->process: all jobs complete
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P00 DEBUG: Backup::Backup->processManifest: reference pg_data/PG_VERSION to [BACKUP-FULL-2]
P00 DEBUG: Backup::Backup->processManifest: reference pg_data/base/1/12000 to [BACKUP-FULL-2]
P00 DEBUG: Backup::Backup->processManifest: reference pg_data/base/1/PG_VERSION to [BACKUP-FULL-2]
P00 DEBUG: Backup::Backup->processManifest: reference pg_data/base/16384/17000 to [BACKUP-FULL-2]
P00 DEBUG: Backup::Backup->processManifest: reference pg_data/base/16384/PG_VERSION to [BACKUP-FULL-2]
P00 DEBUG: Backup::Backup->processManifest: reference pg_data/base/32768/33000 to [BACKUP-FULL-2]
P00 DEBUG: Backup::Backup->processManifest: reference pg_data/base/32768/33000.32767 to [BACKUP-FULL-2]
P00 DEBUG: Backup::Backup->processManifest: reference pg_data/base/32768/33001 to [BACKUP-FULL-2]
P00 DEBUG: Backup::Backup->processManifest: reference pg_data/base/32768/44000_init to [BACKUP-FULL-2]
P00 DEBUG: Backup::Backup->processManifest: reference pg_data/base/32768/PG_VERSION to [BACKUP-FULL-2]
P00 DEBUG: Backup::Backup->processManifest: reference pg_data/changetime.txt to [BACKUP-FULL-2]
P00 DEBUG: Backup::Backup->processManifest: reference pg_data/global/pg_control to [BACKUP-FULL-2]
P00 DEBUG: Backup::Backup->processManifest: reference pg_data/pg_stat/global.stat to [BACKUP-FULL-2]
P00 DEBUG: Backup::Backup->processManifest: reference pg_data/postgresql.conf to [BACKUP-FULL-2]
P00 DEBUG: Backup::Backup->processManifest: reference pg_data/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?% to [BACKUP-FULL-2]
P00 DEBUG: Backup::Backup->processManifest: reference pg_data/zero_from_start to [BACKUP-FULL-2]
P00 DEBUG: Backup::Backup->processManifest=>: lSizeTotal = 196691
P00 INFO: incr backup size = 192KB
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Helper::protocolDestroy(): bComplete = true, iRemoteIdx = [undef], strRemoteType = [undef]
P00 DEBUG: Protocol::Helper::protocolDestroy=>: iExitStatus = 0
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Storage::Local->pathSync(): strPathExp = <REPO:BACKUP>/[BACKUP-INCR-2]
Make backup directory sync more efficient. Scanning the entire backup directory can be very expensive if there are a lot of small tables. The backup manifest contains the backup directory list so use it to perform syncs instead of scanning the backup directory.
2018-04-04 03:30:15 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]/pg_data
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]/pg_data
Make backup directory sync more efficient. Scanning the entire backup directory can be very expensive if there are a lot of small tables. The backup manifest contains the backup directory list so use it to perform syncs instead of scanning the backup directory.
2018-04-04 03:30:15 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]/pg_data/base
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]/pg_data/base/1
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]/pg_data/base/16384
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]/pg_data/base/32768
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]/pg_data/global
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]/pg_data/pg_clog
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]/pg_data/pg_dynshmem
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]/pg_data/pg_log
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
Make backup directory sync more efficient. Scanning the entire backup directory can be very expensive if there are a lot of small tables. The backup manifest contains the backup directory list so use it to perform syncs instead of scanning the backup directory.
2018-04-04 03:30:15 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]/pg_data/pg_notify
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]/pg_data/pg_replslot
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]/pg_data/pg_serial
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]/pg_data/pg_snapshots
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]/pg_data/pg_stat
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]/pg_data/pg_stat_tmp
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]/pg_data/pg_subtrans
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]/pg_data/pg_tblspc
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]/pg_tblspc
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]/pg_tblspc
Make backup directory sync more efficient. Scanning the entire backup directory can be very expensive if there are a lot of small tables. The backup manifest contains the backup directory list so use it to perform syncs instead of scanning the backup directory.
2018-04-04 03:30:15 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]/pg_tblspc/1
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]/pg_tblspc/1
Make backup directory sync more efficient. Scanning the entire backup directory can be very expensive if there are a lot of small tables. The backup manifest contains the backup directory list so use it to perform syncs instead of scanning the backup directory.
2018-04-04 03:30:15 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]/pg_tblspc/1/[TS_PATH-1]
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]/pg_tblspc/1/[TS_PATH-1]
Make backup directory sync more efficient. Scanning the entire backup directory can be very expensive if there are a lot of small tables. The backup manifest contains the backup directory list so use it to perform syncs instead of scanning the backup directory.
2018-04-04 03:30:15 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]/pg_tblspc/1/[TS_PATH-1]/16384
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]/pg_tblspc/1/[TS_PATH-1]/16384
Make backup directory sync more efficient. Scanning the entire backup directory can be very expensive if there are a lot of small tables. The backup manifest contains the backup directory list so use it to perform syncs instead of scanning the backup directory.
2018-04-04 03:30:15 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]/pg_tblspc/11
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]/pg_tblspc/11/[TS_PATH-1]
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]/pg_tblspc/2
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]/pg_tblspc/2
Make backup directory sync more efficient. Scanning the entire backup directory can be very expensive if there are a lot of small tables. The backup manifest contains the backup directory list so use it to perform syncs instead of scanning the backup directory.
2018-04-04 03:30:15 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]/pg_tblspc/2/[TS_PATH-1]
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]/pg_tblspc/2/[TS_PATH-1]
Make backup directory sync more efficient. Scanning the entire backup directory can be very expensive if there are a lot of small tables. The backup manifest contains the backup directory list so use it to perform syncs instead of scanning the backup directory.
2018-04-04 03:30:15 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]/pg_tblspc/2/[TS_PATH-1]/32768
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]/pg_tblspc/2/[TS_PATH-1]/32768
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->openWrite(): bAtomic = <false>, bPathCreate = <false>, lTimestamp = [undef], rhyFilter = [undef], strCipherPass = [undef], strGroup = [undef], strMode = <0640>, strUser = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]/backup.manifest
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->openWrite(): bAtomic = <false>, bPathCreate = <false>, lTimestamp = [undef], rhyFilter = [undef], strCipherPass = [undef], strGroup = [undef], strMode = <0640>, strUser = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]/backup.manifest.copy
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: new backup label = [BACKUP-INCR-2]
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->openRead(): bIgnoreMissing = <false>, rhyFilter = [undef], strCipherPass = [undef], xFileExp = <REPO:BACKUP>/[BACKUP-INCR-2]/backup.manifest
Make backup.history sync more efficient. Only the backup.history/[year] directory was being synced, so check if the backup.history is newly created and sync it as well.
2018-04-04 02:54:18 +02:00
P00 DEBUG: Storage::Local->openWrite(): bAtomic = true, bPathCreate = true, lTimestamp = [undef], rhyFilter = ({strClass => pgBackRest::Storage::Filter::Gzip}), strCipherPass = [undef], strGroup = [undef], strMode = <0640>, strUser = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/backup.history/[YEAR-1]/[BACKUP-INCR-2].manifest.gz
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Base->copy(): xDestinationFile = [object], xSourceFile = [object]
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Storage::Local->pathSync(): strPathExp = <REPO:BACKUP>/backup.history
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/backup.history/[YEAR-1]
Add bIgnoreMissing parameter to Local->manifest().
2017-07-25 18:44:38 +02:00
P00 DEBUG: Storage::Local->remove(): bIgnoreMissing = <true>, bRecurse = <false>, xstryPathFileExp = <REPO:BACKUP>/latest
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->remove=>: bRemoved = true
P00 DEBUG: Storage::Local->linkCreate(): bHard = <false>, bIgnoreExists = <false>, bPathCreate = <true>, bRelative = true, strDestinationLinkExp = <REPO:BACKUP>/latest, strSourcePathFileExp = <REPO:BACKUP>/[BACKUP-INCR-2]
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->openWrite(): bAtomic = <false>, bPathCreate = <false>, lTimestamp = [undef], rhyFilter = [undef], strCipherPass = [undef], strGroup = [undef], strMode = <0640>, strUser = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/backup.info
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/repo/backup/db
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Storage::Local->openWrite(): bAtomic = <false>, bPathCreate = <false>, lTimestamp = [undef], rhyFilter = [undef], strCipherPass = [undef], strGroup = [undef], strMode = <0640>, strUser = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/backup.info.copy
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/repo/backup/db
P00 DEBUG: Storage::Local->pathSync(): strPathExp = <REPO:BACKUP>
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: perl/exec::perlExec: => 0
Added a predictable log entry to signal that a command has completed successfully. For example a backup ends successfully with: INFO: backup command end: completed successfully.
2016-12-04 21:44:53 +02:00
P00 INFO: backup command end: completed successfully
Move async forking and more error handling to C. The Perl process was exiting directly when called but that interfered with proper locking for the forked async process. Now Perl returns results to the C process which handles all errors, including signals.
2018-04-13 02:42:26 +02:00
P00 INFO: expire command begin
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: perl/exec::perlExec: (void)
Move async forking and more error handling to C. The Perl process was exiting directly when called but that interfered with proper locking for the forked async process. Now Perl returns results to the C process which handles all errors, including signals.
2018-04-13 02:42:26 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp =
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = true, bIgnoreExists = true, strMode = 0770, strPathExp = [TEST_PATH]/db-master/log
P00 DEBUG: Common::Lock::lockStopTest(): bStanzaStopRequired = <false>
P00 DEBUG: Common::Lock::lockStopTest=>: bStopExists = false
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
P00 DEBUG: Backup::Info->new(): bIgnoreMissing = <false>, bLoad = <true>, bRequired = <true>, bValidate = <true>, oStorage = <[object]>, strBackupClusterPath = [TEST_PATH]/db-master/repo/backup/db, strCipherPassSub = [undef]
P00 DEBUG: Storage::Local->encrypted(): bIgnoreMissing = true, strFileName = [TEST_PATH]/db-master/repo/backup/db/backup.info
P00 DEBUG: Storage::Local->encrypted=>: bEncrypted = false
P00 DEBUG: Storage::Local->encryptionValid(): bEncrypted = false
P00 DEBUG: Storage::Local->encryptionValid=>: bValid = true
P00 DEBUG: Storage::Local->openRead(): bIgnoreMissing = true, rhyFilter = [undef], strCipherPass = [undef], xFileExp = [TEST_PATH]/db-master/repo/backup/db/backup.info
P00 DEBUG: Storage::Base->get(): strCipherPass = [undef], xFile = [object]
Move backup modules into Backup directory.
2017-05-15 22:01:00 +02:00
P00 DEBUG: Backup::Info->reconstruct(): bRequired = <true>, bSave = <true>, iCatalogVersion = [undef], iControlVersion = [undef], strDbVersion = [undef], ullDbSysId = [undef]
P00 DEBUG: Backup::Common::backupRegExpGet(): bAnchor = <true>, bDifferential = true, bFull = true, bIncremental = true
P00 DEBUG: Backup::Common::backupRegExpGet=>: strRegExp = ^[0-9]{8}\-[0-9]{6}F(\_[0-9]{8}\-[0-9]{6}(D|I)){0,1}$
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->list(): bIgnoreMissing = <false>, strExpression = ^[0-9]{8}\-[0-9]{6}F(\_[0-9]{8}\-[0-9]{6}(D|I)){0,1}$, strPathExp = [TEST_PATH]/db-master/repo/backup/db, strSortOrder = <forward>
P00 DEBUG: Storage::Local->list=>: stryFileList = ([BACKUP-FULL-2], [BACKUP-INCR-2])
Move backup modules into Backup directory.
2017-05-15 22:01:00 +02:00
P00 DEBUG: Backup::Info->current(): strBackup = [BACKUP-FULL-2]
P00 DEBUG: Backup::Info->current=>: bTest = true
P00 DEBUG: Backup::Info->current(): strBackup = [BACKUP-INCR-2]
P00 DEBUG: Backup::Info->current=>: bTest = true
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-2]/backup.manifest
P00 DEBUG: Storage::Local->exists=>: bExists = true
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]/backup.manifest
P00 DEBUG: Storage::Local->exists=>: bExists = true
Move backup modules into Backup directory.
2017-05-15 22:01:00 +02:00
P00 DEBUG: Backup::Common::backupRegExpGet(): bAnchor = <true>, bDifferential = true, bFull = true, bIncremental = true
P00 DEBUG: Backup::Common::backupRegExpGet=>: strRegExp = ^[0-9]{8}\-[0-9]{6}F(\_[0-9]{8}\-[0-9]{6}(D|I)){0,1}$
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 DEBUG: Storage::Local->list(): bIgnoreMissing = <false>, strExpression = ^[0-9]{8}\-[0-9]{6}F(\_[0-9]{8}\-[0-9]{6}(D|I)){0,1}$, strPathExp = <REPO:BACKUP>, strSortOrder = reverse
P00 DEBUG: Storage::Local->list=>: stryFileList = ([BACKUP-INCR-2], [BACKUP-FULL-2])
Move backup modules into Backup directory.
2017-05-15 22:01:00 +02:00
P00 DEBUG: Backup::Info->current(): strBackup = [BACKUP-INCR-2]
P00 DEBUG: Backup::Info->current=>: bTest = true
P00 DEBUG: Backup::Info->current(): strBackup = [BACKUP-FULL-2]
P00 DEBUG: Backup::Info->current=>: bTest = true
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
P00 INFO: option 'repo1-retention-archive' is not set - archive logs will not be expired
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: perl/exec::perlExec: => 0
P00 DEBUG: common/exit::exitSafe: (result: 0, error: false, signalType: 0)
P00 DEBUG: common/lock::lockRelease: (failOnNoLock: false)
P00 DEBUG: common/lock::lockRelease: => true
Move async forking and more error handling to C. The Perl process was exiting directly when called but that interfered with proper locking for the forked async process. Now Perl returns results to the C process which handles all errors, including signals.
2018-04-13 02:42:26 +02:00
P00 DEBUG: Main::mainCleanup(): iExitCode = 0
Move modules in Protocol directory in subdirectories.
2017-05-15 17:12:14 +02:00
P00 DEBUG: Protocol::Helper::protocolDestroy(): bComplete = true, iRemoteIdx = [undef], strRemoteType = [undef]
P00 DEBUG: Protocol::Helper::protocolDestroy=>: iExitStatus = 0
Added a predictable log entry to signal that a command has completed successfully. For example a backup ends successfully with: INFO: backup command end: completed successfully.
2016-12-04 21:44:53 +02:00
P00 INFO: expire command end: completed successfully
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: common/exit::exitSafe: => 0
P00 DEBUG: main::main: => 0
Log testing can now be enabled for certain deterministic tests. This works by comparing the generated logs against a previous copy. Currently only enabled for the backup/synthetic tests.
2015-05-05 19:08:48 +02:00
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
+ supplemental file: [TEST_PATH]/db-master/pgbackrest.conf
----------------------------------------------------------
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[db]
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
pg1-path=[TEST_PATH]/db-master/db/base
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
New simpler configuration and consistent project/exe/path naming. * The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required. * Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case. * The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally. * The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest. * Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory. * Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory. * Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 15:30:54 +02:00
[global]
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
compress=n
Fix custom settings for compress-level option being ignored. Reported by Jens Wilke.
2017-10-23 00:05:46 +02:00
compress-level=3
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
db-timeout=45
lock-path=[TEST_PATH]/db-master/lock
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
log-level-console=debug
log-level-file=trace
Added the log-level-stderr option. Controls whether console log messages are sent to stderr or stdout. By default this is set to warn which represents a change in behavior from previous versions, even though it may be more intuitive. Setting log-level-stderr=off will preserve the old behavior. Suggested by Sascha Biberhofer.
2016-10-05 15:09:30 +02:00
log-level-stderr=off
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
log-path=[TEST_PATH]/db-master/log
Add log-subprocess option to allow file logging for local and remote subprocesses.
2018-08-23 02:05:49 +02:00
log-subprocess=y
Set log-timestamp=n for integration tests. This means less filtering of logs needs to be done and new timestamps can be added without adding new filters.
2018-05-11 17:24:38 +02:00
log-timestamp=n
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
protocol-timeout=60
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
repo1-path=[TEST_PATH]/db-master/repo
Consolidate archive-push tests into new archive test.
2017-06-21 14:38:53 +02:00
spool-path=[TEST_PATH]/db-master/spool
New simpler configuration and consistent project/exe/path naming. * The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required. * Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case. * The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally. * The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest. * Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory. * Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory. * Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 15:30:54 +02:00
[global:backup]
archive-copy=y
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
exclude=postgresql.auto.conf
exclude=pg_log/
exclude=pg_log2
New simpler configuration and consistent project/exe/path naming. * The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required. * Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case. * The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally. * The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest. * Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory. * Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory. * Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 15:30:54 +02:00
start-fast=y
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
+ supplemental file: [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-2]/backup.manifest
-----------------------------------------------------------------------------------------
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[backrest]
backrest-checksum="[CHECKSUM]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
backrest-format=5
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
backrest-version="[VERSION-1]"
[backup]
backup-label="[BACKUP-INCR-2]"
backup-prior="[BACKUP-FULL-2]"
backup-timestamp-copy-start=[TIMESTAMP]
backup-timestamp-start=[TIMESTAMP]
backup-timestamp-stop=[TIMESTAMP]
backup-type="incr"
[backup:db]
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-catalog-version=201409291
db-control-version=942
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
db-id=1
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
db-system-id=1000000000000000094
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-version="9.4"
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[backup:option]
option-archive-check=true
option-archive-copy=true
Backup from a standby cluster. A connection to the primary cluster is still required to start/stop the backup and copy files that are not replicated, but the vast majority of files are copied from the standby in order to reduce load on the master.
2016-08-25 17:25:46 +02:00
option-backup-standby=false
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
option-checksum-page=true
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
option-compress=false
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
option-delta=true
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
option-hardlink=false
Closed #167: Rename `--no-start-stop` option to `--no-online`.
2016-02-20 02:32:02 +02:00
option-online=false
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[backup:target]
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
pg_data={"path":"[TEST_PATH]/db-master/db/base","type":"path"}
pg_tblspc/1={"path":"[TEST_PATH]/db-master/db/tablespace/ts1","tablespace-id":"1","tablespace-name":"ts1","type":"link"}
Fixed an issue where tablespace paths with the same prefix would cause an invalid link error. Reported by Nikhilchandra Kulkarni.
2016-11-07 16:37:16 +02:00
pg_tblspc/11={"path":"[TEST_PATH]/db-master/db/tablespace/ts11","tablespace-id":"11","tablespace-name":"ts11","type":"link"}
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
pg_tblspc/2={"path":"[TEST_PATH]/db-master/db/tablespace/ts2","tablespace-id":"2","tablespace-name":"ts2","type":"link"}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:file]
Fixed a issue where tablespaces were copied from the master during standby backup.
2016-09-04 15:19:44 +02:00
pg_data/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/badchecksum.txt={"checksum":"f927212cd08d11a42a666b2f04235398e9ceeb51","master":true,"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
pg_data/base/1/12000={"checksum":"22c98d248ff548311eda88559e4a8405ed77c003","checksum-page":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
pg_data/base/1/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","mode":"0660","reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
pg_data/base/16384/17000={"checksum":"e0101dd8ffb910c9c202ca35b5f828bcb9697bed","checksum-page":false,"checksum-page-error":[1],"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
pg_data/base/16384/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Reduced the likelihood of torn pages causing a false positive in page checksums by filtering on start backup LSN.
2017-01-30 20:59:00 +02:00
pg_data/base/32768/33000={"checksum":"4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f","checksum-page":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Fixed an issue where newly initialized (but unused) pages would cause page checksum warnings. Reported by Stephen Frost.
2017-03-13 21:50:13 +02:00
pg_data/base/32768/33000.32767={"checksum":"21e2c7c1a326682c07053b7d6a5a40dbd49c2ec5","checksum-page":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
pg_data/base/32768/33001={"checksum":"6bf316f11d28c28914ea9be92c00de9bea6d9a6b","checksum-page":false,"checksum-page-error":[0,[3,5],7],"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
pg_data/base/32768/44000_init={"checksum":"4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f","checksum-page":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
pg_data/base/32768/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
pg_data/changecontent.txt={"checksum":"a094d94583e209556d03c3c5da33131a065f1689","master":true,"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Fix critical bug in resume that resulted in inconsistent backups. A regression in v0.82 removed the timestamp comparison when deciding which files from the aborted backup to keep on resume. All resumed backups should be considered inconsistent. A resumed backup can be identified by checking the log for the message "aborted backup of same type exists, will be cleaned to remove invalid files and resumed". Reported by David Youatt, Yogesh Sharma, Stephen Frost.
2018-07-03 20:01:57 +02:00
pg_data/changesize.txt={"checksum":"3905d5be2ec8d67f41435dab5e0dcda3ae47455d","master":true,"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
pg_data/changetime.txt={"checksum":"88087292ed82e26f3eb824d0bffc05ccf7a30f8d","master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-2]}
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 11:24:42 +02:00
pg_data/global/pg_control={"checksum":"4c77c900f7af0d9ab13fa9982051a42e0b637f6c","master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-2]}
Fixed a issue where tablespaces were copied from the master during standby backup.
2016-09-04 15:19:44 +02:00
pg_data/pg_stat/global.stat={"checksum":"e350d5ce0153f3e22d5db21cf2a4eff00f3ee877","master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-2]}
pg_data/postgresql.conf={"checksum":"6721d92c9fcdf4248acff1f9a1377127d9064807","master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-2]}
Improve support for special characters in filenames. % characters caused issues in backup/restore due to filenames being appended directly into a format string. Reserved XML characters (<>&') caused issues in the S3 driver due to improper escaping. Add a file with all common special characters to regression testing.
2018-09-10 16:54:34 +02:00
pg_data/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?%={"master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Allow zero-size files in backup manifest to reference a prior manifest regardless of timestamp delta. Contributed by Cynthia Shang.
2018-08-24 22:50:33 +02:00
pg_data/zero_from_start={"master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Add zero-length file to mock/all test.
2018-07-05 21:40:50 +02:00
pg_data/zerosize.txt={"master":true,"size":[SIZE],"timestamp":[TIMESTAMP-2]}
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
pg_tblspc/1/[TS_PATH-1]/16384/tablespace1.txt={"checksum":"d85de07d6421d90aa9191c11c889bfde43680f0f","checksum-page":false,"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_tblspc/2/[TS_PATH-1]/32768/tablespace2.txt={"checksum":"dc7f76e43c46101b47acc55ae4d593a9e6983578","checksum-page":false,"size":[SIZE],"timestamp":[TIMESTAMP-1]}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:file:default]
Continuous integration using travis-ci.
2017-02-21 15:59:23 +02:00
group="[GROUP-1]"
Fixed a issue where tablespaces were copied from the master during standby backup.
2016-09-04 15:19:44 +02:00
master=false
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
mode="0600"
Regression tests can now be run as any properly-configured user, not just vagrant.
2016-10-01 19:39:44 +02:00
user="[USER-1]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:link]
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
pg_data/pg_tblspc/1={"destination":"[TEST_PATH]/db-master/db/tablespace/ts1"}
Fixed an issue where tablespace paths with the same prefix would cause an invalid link error. Reported by Nikhilchandra Kulkarni.
2016-11-07 16:37:16 +02:00
pg_data/pg_tblspc/11={"destination":"[TEST_PATH]/db-master/db/tablespace/ts11"}
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
pg_data/pg_tblspc/2={"destination":"[TEST_PATH]/db-master/db/tablespace/ts2"}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:link:default]
Continuous integration using travis-ci.
2017-02-21 15:59:23 +02:00
group="[GROUP-1]"
Regression tests can now be run as any properly-configured user, not just vagrant.
2016-10-01 19:39:44 +02:00
user="[USER-1]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:path]
Fixed an issue with invalid backup groups being set correctly on restore. If the backup cannot map a group to a name it stores the group in the manifest as false then uses either the owner of $PGDATA to set the group during restore or failing that the group of the current user. This logic was not working correctly because the selected group was overwriting the user on restore leaving the group undefined and the user incorrectly set to the group. (Reported by Jeff McCormick.)
2017-11-20 19:46:35 +02:00
pg_data={"mode":"0777"}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_data/base={}
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
pg_data/base/1={}
pg_data/base/16384={}
pg_data/base/32768={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_data/global={}
pg_data/pg_clog={}
Exclude directories during backup that are cleaned, recreated, or zeroed by PostgreSQL at startup. These include (depending on the version where they were introduced): pgsql_tmp, pg_dynshmem, pg_notify, pg_replslot, pg_serial, pg_snapshots, pg_stat_tmp, pg_subtrans. The postgresql.auto.conf.tmp file is now excluded in addition to files that were already excluded: backup_label.old, postmaster.opts, postmaster.pid, recovery.conf, recovery.done.
2016-08-16 15:35:16 +02:00
pg_data/pg_dynshmem={}
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
pg_data/pg_log={}
Exclude directories during backup that are cleaned, recreated, or zeroed by PostgreSQL at startup. These include (depending on the version where they were introduced): pgsql_tmp, pg_dynshmem, pg_notify, pg_replslot, pg_serial, pg_snapshots, pg_stat_tmp, pg_subtrans. The postgresql.auto.conf.tmp file is now excluded in addition to files that were already excluded: backup_label.old, postmaster.opts, postmaster.pid, recovery.conf, recovery.done.
2016-08-16 15:35:16 +02:00
pg_data/pg_notify={}
pg_data/pg_replslot={}
pg_data/pg_serial={}
pg_data/pg_snapshots={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_data/pg_stat={}
Exclude directories during backup that are cleaned, recreated, or zeroed by PostgreSQL at startup. These include (depending on the version where they were introduced): pgsql_tmp, pg_dynshmem, pg_notify, pg_replslot, pg_serial, pg_snapshots, pg_stat_tmp, pg_subtrans. The postgresql.auto.conf.tmp file is now excluded in addition to files that were already excluded: backup_label.old, postmaster.opts, postmaster.pid, recovery.conf, recovery.done.
2016-08-16 15:35:16 +02:00
pg_data/pg_stat_tmp={}
pg_data/pg_subtrans={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_data/pg_tblspc={}
Fixed an issue with invalid backup groups being set correctly on restore. If the backup cannot map a group to a name it stores the group in the manifest as false then uses either the owner of $PGDATA to set the group during restore or failing that the group of the current user. This logic was not working correctly because the selected group was overwriting the user on restore leaving the group undefined and the user incorrectly set to the group. (Reported by Jeff McCormick.)
2017-11-20 19:46:35 +02:00
pg_tblspc={"mode":"0777"}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_tblspc/1={}
pg_tblspc/1/[TS_PATH-1]={}
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
pg_tblspc/1/[TS_PATH-1]/16384={}
Fixed an issue where tablespace paths with the same prefix would cause an invalid link error. Reported by Nikhilchandra Kulkarni.
2016-11-07 16:37:16 +02:00
pg_tblspc/11={}
pg_tblspc/11/[TS_PATH-1]={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_tblspc/2={}
pg_tblspc/2/[TS_PATH-1]={}
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
pg_tblspc/2/[TS_PATH-1]/32768={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:path:default]
Continuous integration using travis-ci.
2017-02-21 15:59:23 +02:00
group="[GROUP-1]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
mode="0700"
Regression tests can now be run as any properly-configured user, not just vagrant.
2016-10-01 19:39:44 +02:00
user="[USER-1]"
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
+ supplemental file: [TEST_PATH]/db-master/repo/backup/db/backup.info
---------------------------------------------------------------------
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[backrest]
backrest-checksum="[CHECKSUM]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
backrest-format=5
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
backrest-version="[VERSION-1]"
[backup:current]
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
[BACKUP-FULL-2]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"full","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":true,"option-compress":false,"option-hardlink":false,"option-online":false}
[BACKUP-INCR-2]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-prior":"[BACKUP-FULL-2]","backup-reference":["[BACKUP-FULL-2]"],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"incr","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":true,"option-compress":false,"option-hardlink":false,"option-online":false}
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[db]
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-catalog-version=201409291
db-control-version=942
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
db-id=1
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
db-system-id=1000000000000000094
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-version="9.4"
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[db:history]
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
1={"db-catalog-version":201409291,"db-control-version":942,"db-system-id":1000000000000000094,"db-version":"9.4"}
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
diff backup - cannot resume - new diff (db-master host)
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --no-online --log-level-console=detail --process-max=1 --delta --type=diff --stanza=db backup --test --test-delay=0.2 --test-point=backup-noresume=y
Log testing can now be enabled for certain deterministic tests. This works by comparing the generated logs against a previous copy. Currently only enabled for the backup/synthetic tests.
2015-05-05 19:08:48 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P00 INFO: backup command begin [BACKREST-VERSION]: --no-compress --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --db-timeout=45 --delta --exclude=postgresql.auto.conf --exclude=pg_log/ --exclude=pg_log2 --lock-path=[TEST_PATH]/db-master/lock --log-level-console=detail --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --no-online --pg1-path=[TEST_PATH]/db-master/db/base --process-max=1 --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --stanza=db --start-fast --test --test-delay=0.2 --test-point=backup-noresume=y --type=diff
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
P00 WARN: option repo1-retention-full is not set, the repository may run out of space
HINT: to retain full backups indefinitely (without warning), set option 'repo1-retention-full' to the maximum.
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 WARN: backup [BACKUP-INCR-2] missing manifest removed from backup.info
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: last backup label = [BACKUP-FULL-2], version = [VERSION-1]
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 WARN: aborted backup [BACKUP-INCR-2] cannot be resumed: new backup-type 'diff' does not match aborted backup-type 'incr'
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 TEST: PgBaCkReStTeSt-BACKUP-NORESUME-PgBaCkReStTeSt
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 WARN: diff backup cannot alter 'checksum-page' option to 'false', reset to 'true' from [BACKUP-FULL-2]
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base/base/32768/33001 (64KB, 33%) checksum 6bf316f11d28c28914ea9be92c00de9bea6d9a6b
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base/base/32768/44000_init (32KB, 49%) checksum 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base/base/32768/33000.32767 (32KB, 66%) checksum 21e2c7c1a326682c07053b7d6a5a40dbd49c2ec5
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base/base/32768/33000 (32KB, 83%) checksum 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base/base/16384/17000 (16KB, 91%) checksum e0101dd8ffb910c9c202ca35b5f828bcb9697bed
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 11:24:42 +02:00
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base/global/pg_control (8KB, 95%) checksum 4c77c900f7af0d9ab13fa9982051a42e0b637f6c
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base/base/1/12000 (8KB, 99%) checksum 22c98d248ff548311eda88559e4a8405ed77c003
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base/postgresql.conf (21B, 99%) checksum 6721d92c9fcdf4248acff1f9a1377127d9064807
P01 INFO: backup file [TEST_PATH]/db-master/db/base/badchecksum.txt (11B, 99%) checksum f927212cd08d11a42a666b2f04235398e9ceeb51
P01 INFO: backup file [TEST_PATH]/db-master/db/base/changecontent.txt (7B, 99%) checksum a094d94583e209556d03c3c5da33131a065f1689
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base/pg_stat/global.stat (5B, 99%) checksum e350d5ce0153f3e22d5db21cf2a4eff00f3ee877
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base/changetime.txt (4B, 99%) checksum 88087292ed82e26f3eb824d0bffc05ccf7a30f8d
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base/base/32768/PG_VERSION (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base/base/16384/PG_VERSION (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base/base/1/PG_VERSION (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base/PG_VERSION (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P01 INFO: backup file [TEST_PATH]/db-master/db/base/zerosize.txt (0B, 99%)
P01 INFO: backup file [TEST_PATH]/db-master/db/base/pg_tblspc/2/[TS_PATH-1]/32768/tablespace2.txt (7B, 99%) checksum dc7f76e43c46101b47acc55ae4d593a9e6983578
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
P00 WARN: page misalignment in file [TEST_PATH]/db-master/db/base/pg_tblspc/2/[TS_PATH-1]/32768/tablespace2.txt: file size 7 is not divisible by page size 8192
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P01 INFO: backup file [TEST_PATH]/db-master/db/base/pg_tblspc/1/[TS_PATH-1]/16384/tablespace1.txt (7B, 100%) checksum d85de07d6421d90aa9191c11c889bfde43680f0f
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
P00 WARN: page misalignment in file [TEST_PATH]/db-master/db/base/pg_tblspc/1/[TS_PATH-1]/16384/tablespace1.txt: file size 7 is not divisible by page size 8192
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P00 INFO: diff backup size = 192KB
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: new backup label = [BACKUP-DIFF-1]
Added a predictable log entry to signal that a command has completed successfully. For example a backup ends successfully with: INFO: backup command end: completed successfully.
2016-12-04 21:44:53 +02:00
P00 INFO: backup command end: completed successfully
Move async forking and more error handling to C. The Perl process was exiting directly when called but that interfered with proper locking for the forked async process. Now Perl returns results to the C process which handles all errors, including signals.
2018-04-13 02:42:26 +02:00
P00 INFO: expire command begin
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
P00 INFO: option 'repo1-retention-archive' is not set - archive logs will not be expired
Added a predictable log entry to signal that a command has completed successfully. For example a backup ends successfully with: INFO: backup command end: completed successfully.
2016-12-04 21:44:53 +02:00
P00 INFO: expire command end: completed successfully
Added resume setting to allow disabling resume feature.
2015-05-07 18:29:30 +02:00
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
+ supplemental file: [TEST_PATH]/db-master/pgbackrest.conf
----------------------------------------------------------
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[db]
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
pg1-path=[TEST_PATH]/db-master/db/base
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
New simpler configuration and consistent project/exe/path naming. * The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required. * Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case. * The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally. * The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest. * Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory. * Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory. * Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 15:30:54 +02:00
[global]
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
compress=n
Fix custom settings for compress-level option being ignored. Reported by Jens Wilke.
2017-10-23 00:05:46 +02:00
compress-level=3
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
db-timeout=45
lock-path=[TEST_PATH]/db-master/lock
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
log-level-console=debug
log-level-file=trace
Added the log-level-stderr option. Controls whether console log messages are sent to stderr or stdout. By default this is set to warn which represents a change in behavior from previous versions, even though it may be more intuitive. Setting log-level-stderr=off will preserve the old behavior. Suggested by Sascha Biberhofer.
2016-10-05 15:09:30 +02:00
log-level-stderr=off
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
log-path=[TEST_PATH]/db-master/log
Add log-subprocess option to allow file logging for local and remote subprocesses.
2018-08-23 02:05:49 +02:00
log-subprocess=y
Set log-timestamp=n for integration tests. This means less filtering of logs needs to be done and new timestamps can be added without adding new filters.
2018-05-11 17:24:38 +02:00
log-timestamp=n
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
protocol-timeout=60
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
repo1-path=[TEST_PATH]/db-master/repo
Consolidate archive-push tests into new archive test.
2017-06-21 14:38:53 +02:00
spool-path=[TEST_PATH]/db-master/spool
New simpler configuration and consistent project/exe/path naming. * The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required. * Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case. * The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally. * The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest. * Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory. * Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory. * Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 15:30:54 +02:00
[global:backup]
archive-copy=y
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
exclude=postgresql.auto.conf
exclude=pg_log/
exclude=pg_log2
New simpler configuration and consistent project/exe/path naming. * The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required. * Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case. * The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally. * The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest. * Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory. * Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory. * Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 15:30:54 +02:00
start-fast=y
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
+ supplemental file: [TEST_PATH]/db-master/repo/backup/db/[BACKUP-DIFF-1]/backup.manifest
-----------------------------------------------------------------------------------------
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[backrest]
backrest-checksum="[CHECKSUM]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
backrest-format=5
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
backrest-version="[VERSION-1]"
[backup]
backup-label="[BACKUP-DIFF-1]"
backup-prior="[BACKUP-FULL-2]"
backup-timestamp-copy-start=[TIMESTAMP]
backup-timestamp-start=[TIMESTAMP]
backup-timestamp-stop=[TIMESTAMP]
backup-type="diff"
[backup:db]
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-catalog-version=201409291
db-control-version=942
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
db-id=1
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
db-system-id=1000000000000000094
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-version="9.4"
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[backup:option]
option-archive-check=true
option-archive-copy=true
Backup from a standby cluster. A connection to the primary cluster is still required to start/stop the backup and copy files that are not replicated, but the vast majority of files are copied from the standby in order to reduce load on the master.
2016-08-25 17:25:46 +02:00
option-backup-standby=false
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
option-checksum-page=true
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
option-compress=false
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
option-delta=true
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
option-hardlink=false
Closed #167: Rename `--no-start-stop` option to `--no-online`.
2016-02-20 02:32:02 +02:00
option-online=false
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[backup:target]
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
pg_data={"path":"[TEST_PATH]/db-master/db/base","type":"path"}
pg_tblspc/1={"path":"[TEST_PATH]/db-master/db/tablespace/ts1","tablespace-id":"1","tablespace-name":"ts1","type":"link"}
pg_tblspc/2={"path":"[TEST_PATH]/db-master/db/tablespace/ts2","tablespace-id":"2","tablespace-name":"ts2","type":"link"}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:file]
Fixed a issue where tablespaces were copied from the master during standby backup.
2016-09-04 15:19:44 +02:00
pg_data/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/badchecksum.txt={"checksum":"f927212cd08d11a42a666b2f04235398e9ceeb51","master":true,"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
pg_data/base/1/12000={"checksum":"22c98d248ff548311eda88559e4a8405ed77c003","checksum-page":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
pg_data/base/1/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","mode":"0660","reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
pg_data/base/16384/17000={"checksum":"e0101dd8ffb910c9c202ca35b5f828bcb9697bed","checksum-page":false,"checksum-page-error":[1],"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
pg_data/base/16384/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Reduced the likelihood of torn pages causing a false positive in page checksums by filtering on start backup LSN.
2017-01-30 20:59:00 +02:00
pg_data/base/32768/33000={"checksum":"4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f","checksum-page":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Fixed an issue where newly initialized (but unused) pages would cause page checksum warnings. Reported by Stephen Frost.
2017-03-13 21:50:13 +02:00
pg_data/base/32768/33000.32767={"checksum":"21e2c7c1a326682c07053b7d6a5a40dbd49c2ec5","checksum-page":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
pg_data/base/32768/33001={"checksum":"6bf316f11d28c28914ea9be92c00de9bea6d9a6b","checksum-page":false,"checksum-page-error":[0,[3,5],7],"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
pg_data/base/32768/44000_init={"checksum":"4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f","checksum-page":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
pg_data/base/32768/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
pg_data/changecontent.txt={"checksum":"a094d94583e209556d03c3c5da33131a065f1689","master":true,"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/changetime.txt={"checksum":"88087292ed82e26f3eb824d0bffc05ccf7a30f8d","master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-2]}
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 11:24:42 +02:00
pg_data/global/pg_control={"checksum":"4c77c900f7af0d9ab13fa9982051a42e0b637f6c","master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-2]}
Fixed a issue where tablespaces were copied from the master during standby backup.
2016-09-04 15:19:44 +02:00
pg_data/pg_stat/global.stat={"checksum":"e350d5ce0153f3e22d5db21cf2a4eff00f3ee877","master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-2]}
pg_data/postgresql.conf={"checksum":"6721d92c9fcdf4248acff1f9a1377127d9064807","master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-2]}
Improve support for special characters in filenames. % characters caused issues in backup/restore due to filenames being appended directly into a format string. Reserved XML characters (<>&') caused issues in the S3 driver due to improper escaping. Add a file with all common special characters to regression testing.
2018-09-10 16:54:34 +02:00
pg_data/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?%={"master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Allow zero-size files in backup manifest to reference a prior manifest regardless of timestamp delta. Contributed by Cynthia Shang.
2018-08-24 22:50:33 +02:00
pg_data/zero_from_start={"master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Add zero-length file to mock/all test.
2018-07-05 21:40:50 +02:00
pg_data/zerosize.txt={"master":true,"size":[SIZE],"timestamp":[TIMESTAMP-2]}
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
pg_tblspc/1/[TS_PATH-1]/16384/tablespace1.txt={"checksum":"d85de07d6421d90aa9191c11c889bfde43680f0f","checksum-page":false,"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_tblspc/2/[TS_PATH-1]/32768/tablespace2.txt={"checksum":"dc7f76e43c46101b47acc55ae4d593a9e6983578","checksum-page":false,"size":[SIZE],"timestamp":[TIMESTAMP-1]}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:file:default]
Continuous integration using travis-ci.
2017-02-21 15:59:23 +02:00
group="[GROUP-1]"
Fixed a issue where tablespaces were copied from the master during standby backup.
2016-09-04 15:19:44 +02:00
master=false
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
mode="0600"
Regression tests can now be run as any properly-configured user, not just vagrant.
2016-10-01 19:39:44 +02:00
user="[USER-1]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:link]
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
pg_data/pg_tblspc/1={"destination":"[TEST_PATH]/db-master/db/tablespace/ts1"}
pg_data/pg_tblspc/2={"destination":"[TEST_PATH]/db-master/db/tablespace/ts2"}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:link:default]
Continuous integration using travis-ci.
2017-02-21 15:59:23 +02:00
group="[GROUP-1]"
Regression tests can now be run as any properly-configured user, not just vagrant.
2016-10-01 19:39:44 +02:00
user="[USER-1]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:path]
Fixed an issue with invalid backup groups being set correctly on restore. If the backup cannot map a group to a name it stores the group in the manifest as false then uses either the owner of $PGDATA to set the group during restore or failing that the group of the current user. This logic was not working correctly because the selected group was overwriting the user on restore leaving the group undefined and the user incorrectly set to the group. (Reported by Jeff McCormick.)
2017-11-20 19:46:35 +02:00
pg_data={"mode":"0777"}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_data/base={}
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
pg_data/base/1={}
pg_data/base/16384={}
pg_data/base/32768={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_data/global={}
pg_data/pg_clog={}
Exclude directories during backup that are cleaned, recreated, or zeroed by PostgreSQL at startup. These include (depending on the version where they were introduced): pgsql_tmp, pg_dynshmem, pg_notify, pg_replslot, pg_serial, pg_snapshots, pg_stat_tmp, pg_subtrans. The postgresql.auto.conf.tmp file is now excluded in addition to files that were already excluded: backup_label.old, postmaster.opts, postmaster.pid, recovery.conf, recovery.done.
2016-08-16 15:35:16 +02:00
pg_data/pg_dynshmem={}
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
pg_data/pg_log={}
Exclude directories during backup that are cleaned, recreated, or zeroed by PostgreSQL at startup. These include (depending on the version where they were introduced): pgsql_tmp, pg_dynshmem, pg_notify, pg_replslot, pg_serial, pg_snapshots, pg_stat_tmp, pg_subtrans. The postgresql.auto.conf.tmp file is now excluded in addition to files that were already excluded: backup_label.old, postmaster.opts, postmaster.pid, recovery.conf, recovery.done.
2016-08-16 15:35:16 +02:00
pg_data/pg_notify={}
pg_data/pg_replslot={}
pg_data/pg_serial={}
pg_data/pg_snapshots={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_data/pg_stat={}
Exclude directories during backup that are cleaned, recreated, or zeroed by PostgreSQL at startup. These include (depending on the version where they were introduced): pgsql_tmp, pg_dynshmem, pg_notify, pg_replslot, pg_serial, pg_snapshots, pg_stat_tmp, pg_subtrans. The postgresql.auto.conf.tmp file is now excluded in addition to files that were already excluded: backup_label.old, postmaster.opts, postmaster.pid, recovery.conf, recovery.done.
2016-08-16 15:35:16 +02:00
pg_data/pg_stat_tmp={}
pg_data/pg_subtrans={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_data/pg_tblspc={}
Fixed an issue with invalid backup groups being set correctly on restore. If the backup cannot map a group to a name it stores the group in the manifest as false then uses either the owner of $PGDATA to set the group during restore or failing that the group of the current user. This logic was not working correctly because the selected group was overwriting the user on restore leaving the group undefined and the user incorrectly set to the group. (Reported by Jeff McCormick.)
2017-11-20 19:46:35 +02:00
pg_tblspc={"mode":"0777"}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_tblspc/1={}
pg_tblspc/1/[TS_PATH-1]={}
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
pg_tblspc/1/[TS_PATH-1]/16384={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_tblspc/2={}
pg_tblspc/2/[TS_PATH-1]={}
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
pg_tblspc/2/[TS_PATH-1]/32768={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:path:default]
Continuous integration using travis-ci.
2017-02-21 15:59:23 +02:00
group="[GROUP-1]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
mode="0700"
Regression tests can now be run as any properly-configured user, not just vagrant.
2016-10-01 19:39:44 +02:00
user="[USER-1]"
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
+ supplemental file: [TEST_PATH]/db-master/repo/backup/db/backup.info
---------------------------------------------------------------------
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[backrest]
backrest-checksum="[CHECKSUM]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
backrest-format=5
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
backrest-version="[VERSION-1]"
[backup:current]
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
[BACKUP-FULL-2]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"full","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":true,"option-compress":false,"option-hardlink":false,"option-online":false}
[BACKUP-DIFF-1]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-prior":"[BACKUP-FULL-2]","backup-reference":["[BACKUP-FULL-2]"],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"diff","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":true,"option-compress":false,"option-hardlink":false,"option-online":false}
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[db]
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-catalog-version=201409291
db-control-version=942
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
db-id=1
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
db-system-id=1000000000000000094
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-version="9.4"
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[db:history]
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
1={"db-catalog-version":201409291,"db-control-version":942,"db-system-id":1000000000000000094,"db-version":"9.4"}
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
Added the --repo-link option. Allow internal symlinks to be suppressed when the repository is located on a filesystem that does not support symlinks. This does not affect any pgBackRest functionality, but the convenience link latest will not be created and neither will internal tablespace symlinks, which will affect the ability to bring up clusters in-place manually using filesystem snapshots.
2016-12-03 17:23:26 +02:00
diff backup - cannot resume - disabled / no repo link (db-master host)
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --no-online --no-resume --log-level-console=detail --process-max=1 --delta --type=diff --stanza=db backup --test --test-delay=0.2 --test-point=backup-noresume=y
Added resume setting to allow disabling resume feature.
2015-05-07 18:29:30 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P00 INFO: backup command begin [BACKREST-VERSION]: --no-compress --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --db-timeout=45 --delta --exclude=postgresql.auto.conf --exclude=pg_log/ --exclude=pg_log2 --lock-path=[TEST_PATH]/db-master/lock --log-level-console=detail --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --no-online --pg1-path=[TEST_PATH]/db-master/db/base --process-max=1 --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --no-resume --stanza=db --start-fast --test --test-delay=0.2 --test-point=backup-noresume=y --type=diff
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
P00 WARN: option repo1-retention-full is not set, the repository may run out of space
HINT: to retain full backups indefinitely (without warning), set option 'repo1-retention-full' to the maximum.
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 WARN: backup [BACKUP-DIFF-1] missing manifest removed from backup.info
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: last backup label = [BACKUP-FULL-2], version = [VERSION-1]
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 WARN: aborted backup [BACKUP-DIFF-1] cannot be resumed: resume is disabled
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 TEST: PgBaCkReStTeSt-BACKUP-NORESUME-PgBaCkReStTeSt
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 WARN: diff backup cannot alter 'checksum-page' option to 'false', reset to 'true' from [BACKUP-FULL-2]
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base/base/32768/33001 (64KB, 33%) checksum 6bf316f11d28c28914ea9be92c00de9bea6d9a6b
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base/base/32768/44000_init (32KB, 49%) checksum 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base/base/32768/33000.32767 (32KB, 66%) checksum 21e2c7c1a326682c07053b7d6a5a40dbd49c2ec5
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base/base/32768/33000 (32KB, 83%) checksum 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base/base/16384/17000 (16KB, 91%) checksum e0101dd8ffb910c9c202ca35b5f828bcb9697bed
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 11:24:42 +02:00
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base/global/pg_control (8KB, 95%) checksum 4c77c900f7af0d9ab13fa9982051a42e0b637f6c
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base/base/1/12000 (8KB, 99%) checksum 22c98d248ff548311eda88559e4a8405ed77c003
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base/postgresql.conf (21B, 99%) checksum 6721d92c9fcdf4248acff1f9a1377127d9064807
P01 INFO: backup file [TEST_PATH]/db-master/db/base/badchecksum.txt (11B, 99%) checksum f927212cd08d11a42a666b2f04235398e9ceeb51
P01 INFO: backup file [TEST_PATH]/db-master/db/base/changecontent.txt (7B, 99%) checksum a094d94583e209556d03c3c5da33131a065f1689
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base/pg_stat/global.stat (5B, 99%) checksum e350d5ce0153f3e22d5db21cf2a4eff00f3ee877
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base/changetime.txt (4B, 99%) checksum 88087292ed82e26f3eb824d0bffc05ccf7a30f8d
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base/base/32768/PG_VERSION (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base/base/16384/PG_VERSION (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base/base/1/PG_VERSION (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base/PG_VERSION (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P01 INFO: backup file [TEST_PATH]/db-master/db/base/zerosize.txt (0B, 99%)
P01 INFO: backup file [TEST_PATH]/db-master/db/base/pg_tblspc/2/[TS_PATH-1]/32768/tablespace2.txt (7B, 99%) checksum dc7f76e43c46101b47acc55ae4d593a9e6983578
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
P00 WARN: page misalignment in file [TEST_PATH]/db-master/db/base/pg_tblspc/2/[TS_PATH-1]/32768/tablespace2.txt: file size 7 is not divisible by page size 8192
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P01 INFO: backup file [TEST_PATH]/db-master/db/base/pg_tblspc/1/[TS_PATH-1]/16384/tablespace1.txt (7B, 100%) checksum d85de07d6421d90aa9191c11c889bfde43680f0f
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
P00 WARN: page misalignment in file [TEST_PATH]/db-master/db/base/pg_tblspc/1/[TS_PATH-1]/16384/tablespace1.txt: file size 7 is not divisible by page size 8192
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P00 INFO: diff backup size = 192KB
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: new backup label = [BACKUP-DIFF-2]
Added a predictable log entry to signal that a command has completed successfully. For example a backup ends successfully with: INFO: backup command end: completed successfully.
2016-12-04 21:44:53 +02:00
P00 INFO: backup command end: completed successfully
Move async forking and more error handling to C. The Perl process was exiting directly when called but that interfered with proper locking for the forked async process. Now Perl returns results to the C process which handles all errors, including signals.
2018-04-13 02:42:26 +02:00
P00 INFO: expire command begin
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
P00 INFO: option 'repo1-retention-archive' is not set - archive logs will not be expired
Added a predictable log entry to signal that a command has completed successfully. For example a backup ends successfully with: INFO: backup command end: completed successfully.
2016-12-04 21:44:53 +02:00
P00 INFO: expire command end: completed successfully
Log testing can now be enabled for certain deterministic tests. This works by comparing the generated logs against a previous copy. Currently only enabled for the backup/synthetic tests.
2015-05-05 19:08:48 +02:00
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
+ supplemental file: [TEST_PATH]/db-master/pgbackrest.conf
----------------------------------------------------------
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[db]
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
pg1-path=[TEST_PATH]/db-master/db/base
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
New simpler configuration and consistent project/exe/path naming. * The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required. * Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case. * The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally. * The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest. * Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory. * Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory. * Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 15:30:54 +02:00
[global]
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
compress=n
Fix custom settings for compress-level option being ignored. Reported by Jens Wilke.
2017-10-23 00:05:46 +02:00
compress-level=3
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
db-timeout=45
lock-path=[TEST_PATH]/db-master/lock
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
log-level-console=debug
log-level-file=trace
Added the log-level-stderr option. Controls whether console log messages are sent to stderr or stdout. By default this is set to warn which represents a change in behavior from previous versions, even though it may be more intuitive. Setting log-level-stderr=off will preserve the old behavior. Suggested by Sascha Biberhofer.
2016-10-05 15:09:30 +02:00
log-level-stderr=off
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
log-path=[TEST_PATH]/db-master/log
Add log-subprocess option to allow file logging for local and remote subprocesses.
2018-08-23 02:05:49 +02:00
log-subprocess=y
Set log-timestamp=n for integration tests. This means less filtering of logs needs to be done and new timestamps can be added without adding new filters.
2018-05-11 17:24:38 +02:00
log-timestamp=n
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
protocol-timeout=60
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
repo1-path=[TEST_PATH]/db-master/repo
Consolidate archive-push tests into new archive test.
2017-06-21 14:38:53 +02:00
spool-path=[TEST_PATH]/db-master/spool
New simpler configuration and consistent project/exe/path naming. * The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required. * Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case. * The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally. * The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest. * Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory. * Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory. * Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 15:30:54 +02:00
[global:backup]
archive-copy=y
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
exclude=postgresql.auto.conf
exclude=pg_log/
exclude=pg_log2
New simpler configuration and consistent project/exe/path naming. * The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required. * Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case. * The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally. * The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest. * Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory. * Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory. * Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 15:30:54 +02:00
start-fast=y
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
+ supplemental file: [TEST_PATH]/db-master/repo/backup/db/[BACKUP-DIFF-2]/backup.manifest
-----------------------------------------------------------------------------------------
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[backrest]
backrest-checksum="[CHECKSUM]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
backrest-format=5
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
backrest-version="[VERSION-1]"
[backup]
backup-label="[BACKUP-DIFF-2]"
backup-prior="[BACKUP-FULL-2]"
backup-timestamp-copy-start=[TIMESTAMP]
backup-timestamp-start=[TIMESTAMP]
backup-timestamp-stop=[TIMESTAMP]
backup-type="diff"
[backup:db]
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-catalog-version=201409291
db-control-version=942
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
db-id=1
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
db-system-id=1000000000000000094
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-version="9.4"
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[backup:option]
option-archive-check=true
option-archive-copy=true
Backup from a standby cluster. A connection to the primary cluster is still required to start/stop the backup and copy files that are not replicated, but the vast majority of files are copied from the standby in order to reduce load on the master.
2016-08-25 17:25:46 +02:00
option-backup-standby=false
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
option-checksum-page=true
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
option-compress=false
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
option-delta=true
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
option-hardlink=false
Closed #167: Rename `--no-start-stop` option to `--no-online`.
2016-02-20 02:32:02 +02:00
option-online=false
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[backup:target]
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
pg_data={"path":"[TEST_PATH]/db-master/db/base","type":"path"}
pg_tblspc/1={"path":"[TEST_PATH]/db-master/db/tablespace/ts1","tablespace-id":"1","tablespace-name":"ts1","type":"link"}
pg_tblspc/2={"path":"[TEST_PATH]/db-master/db/tablespace/ts2","tablespace-id":"2","tablespace-name":"ts2","type":"link"}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:file]
Fixed a issue where tablespaces were copied from the master during standby backup.
2016-09-04 15:19:44 +02:00
pg_data/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/badchecksum.txt={"checksum":"f927212cd08d11a42a666b2f04235398e9ceeb51","master":true,"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
pg_data/base/1/12000={"checksum":"22c98d248ff548311eda88559e4a8405ed77c003","checksum-page":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
pg_data/base/1/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","mode":"0660","reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
pg_data/base/16384/17000={"checksum":"e0101dd8ffb910c9c202ca35b5f828bcb9697bed","checksum-page":false,"checksum-page-error":[1],"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
pg_data/base/16384/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Reduced the likelihood of torn pages causing a false positive in page checksums by filtering on start backup LSN.
2017-01-30 20:59:00 +02:00
pg_data/base/32768/33000={"checksum":"4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f","checksum-page":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Fixed an issue where newly initialized (but unused) pages would cause page checksum warnings. Reported by Stephen Frost.
2017-03-13 21:50:13 +02:00
pg_data/base/32768/33000.32767={"checksum":"21e2c7c1a326682c07053b7d6a5a40dbd49c2ec5","checksum-page":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
pg_data/base/32768/33001={"checksum":"6bf316f11d28c28914ea9be92c00de9bea6d9a6b","checksum-page":false,"checksum-page-error":[0,[3,5],7],"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
pg_data/base/32768/44000_init={"checksum":"4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f","checksum-page":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
pg_data/base/32768/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
pg_data/changecontent.txt={"checksum":"a094d94583e209556d03c3c5da33131a065f1689","master":true,"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/changetime.txt={"checksum":"88087292ed82e26f3eb824d0bffc05ccf7a30f8d","master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-2]}
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 11:24:42 +02:00
pg_data/global/pg_control={"checksum":"4c77c900f7af0d9ab13fa9982051a42e0b637f6c","master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-2]}
Fixed a issue where tablespaces were copied from the master during standby backup.
2016-09-04 15:19:44 +02:00
pg_data/pg_stat/global.stat={"checksum":"e350d5ce0153f3e22d5db21cf2a4eff00f3ee877","master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-2]}
pg_data/postgresql.conf={"checksum":"6721d92c9fcdf4248acff1f9a1377127d9064807","master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-2]}
Improve support for special characters in filenames. % characters caused issues in backup/restore due to filenames being appended directly into a format string. Reserved XML characters (<>&') caused issues in the S3 driver due to improper escaping. Add a file with all common special characters to regression testing.
2018-09-10 16:54:34 +02:00
pg_data/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?%={"master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Allow zero-size files in backup manifest to reference a prior manifest regardless of timestamp delta. Contributed by Cynthia Shang.
2018-08-24 22:50:33 +02:00
pg_data/zero_from_start={"master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Add zero-length file to mock/all test.
2018-07-05 21:40:50 +02:00
pg_data/zerosize.txt={"master":true,"size":[SIZE],"timestamp":[TIMESTAMP-2]}
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
pg_tblspc/1/[TS_PATH-1]/16384/tablespace1.txt={"checksum":"d85de07d6421d90aa9191c11c889bfde43680f0f","checksum-page":false,"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_tblspc/2/[TS_PATH-1]/32768/tablespace2.txt={"checksum":"dc7f76e43c46101b47acc55ae4d593a9e6983578","checksum-page":false,"size":[SIZE],"timestamp":[TIMESTAMP-1]}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:file:default]
Continuous integration using travis-ci.
2017-02-21 15:59:23 +02:00
group="[GROUP-1]"
Fixed a issue where tablespaces were copied from the master during standby backup.
2016-09-04 15:19:44 +02:00
master=false
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
mode="0600"
Regression tests can now be run as any properly-configured user, not just vagrant.
2016-10-01 19:39:44 +02:00
user="[USER-1]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:link]
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
pg_data/pg_tblspc/1={"destination":"[TEST_PATH]/db-master/db/tablespace/ts1"}
pg_data/pg_tblspc/2={"destination":"[TEST_PATH]/db-master/db/tablespace/ts2"}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:link:default]
Continuous integration using travis-ci.
2017-02-21 15:59:23 +02:00
group="[GROUP-1]"
Regression tests can now be run as any properly-configured user, not just vagrant.
2016-10-01 19:39:44 +02:00
user="[USER-1]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:path]
Fixed an issue with invalid backup groups being set correctly on restore. If the backup cannot map a group to a name it stores the group in the manifest as false then uses either the owner of $PGDATA to set the group during restore or failing that the group of the current user. This logic was not working correctly because the selected group was overwriting the user on restore leaving the group undefined and the user incorrectly set to the group. (Reported by Jeff McCormick.)
2017-11-20 19:46:35 +02:00
pg_data={"mode":"0777"}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_data/base={}
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
pg_data/base/1={}
pg_data/base/16384={}
pg_data/base/32768={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_data/global={}
pg_data/pg_clog={}
Exclude directories during backup that are cleaned, recreated, or zeroed by PostgreSQL at startup. These include (depending on the version where they were introduced): pgsql_tmp, pg_dynshmem, pg_notify, pg_replslot, pg_serial, pg_snapshots, pg_stat_tmp, pg_subtrans. The postgresql.auto.conf.tmp file is now excluded in addition to files that were already excluded: backup_label.old, postmaster.opts, postmaster.pid, recovery.conf, recovery.done.
2016-08-16 15:35:16 +02:00
pg_data/pg_dynshmem={}
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
pg_data/pg_log={}
Exclude directories during backup that are cleaned, recreated, or zeroed by PostgreSQL at startup. These include (depending on the version where they were introduced): pgsql_tmp, pg_dynshmem, pg_notify, pg_replslot, pg_serial, pg_snapshots, pg_stat_tmp, pg_subtrans. The postgresql.auto.conf.tmp file is now excluded in addition to files that were already excluded: backup_label.old, postmaster.opts, postmaster.pid, recovery.conf, recovery.done.
2016-08-16 15:35:16 +02:00
pg_data/pg_notify={}
pg_data/pg_replslot={}
pg_data/pg_serial={}
pg_data/pg_snapshots={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_data/pg_stat={}
Exclude directories during backup that are cleaned, recreated, or zeroed by PostgreSQL at startup. These include (depending on the version where they were introduced): pgsql_tmp, pg_dynshmem, pg_notify, pg_replslot, pg_serial, pg_snapshots, pg_stat_tmp, pg_subtrans. The postgresql.auto.conf.tmp file is now excluded in addition to files that were already excluded: backup_label.old, postmaster.opts, postmaster.pid, recovery.conf, recovery.done.
2016-08-16 15:35:16 +02:00
pg_data/pg_stat_tmp={}
pg_data/pg_subtrans={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_data/pg_tblspc={}
Fixed an issue with invalid backup groups being set correctly on restore. If the backup cannot map a group to a name it stores the group in the manifest as false then uses either the owner of $PGDATA to set the group during restore or failing that the group of the current user. This logic was not working correctly because the selected group was overwriting the user on restore leaving the group undefined and the user incorrectly set to the group. (Reported by Jeff McCormick.)
2017-11-20 19:46:35 +02:00
pg_tblspc={"mode":"0777"}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_tblspc/1={}
pg_tblspc/1/[TS_PATH-1]={}
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
pg_tblspc/1/[TS_PATH-1]/16384={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_tblspc/2={}
pg_tblspc/2/[TS_PATH-1]={}
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
pg_tblspc/2/[TS_PATH-1]/32768={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:path:default]
Continuous integration using travis-ci.
2017-02-21 15:59:23 +02:00
group="[GROUP-1]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
mode="0700"
Regression tests can now be run as any properly-configured user, not just vagrant.
2016-10-01 19:39:44 +02:00
user="[USER-1]"
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
+ supplemental file: [TEST_PATH]/db-master/repo/backup/db/backup.info
---------------------------------------------------------------------
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[backrest]
backrest-checksum="[CHECKSUM]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
backrest-format=5
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
backrest-version="[VERSION-1]"
[backup:current]
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
[BACKUP-FULL-2]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"full","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":true,"option-compress":false,"option-hardlink":false,"option-online":false}
[BACKUP-DIFF-2]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-prior":"[BACKUP-FULL-2]","backup-reference":["[BACKUP-FULL-2]"],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"diff","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":true,"option-compress":false,"option-hardlink":false,"option-online":false}
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[db]
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-catalog-version=201409291
db-control-version=942
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
db-id=1
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
db-system-id=1000000000000000094
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-version="9.4"
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[db:history]
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
1={"db-catalog-version":201409291,"db-control-version":942,"db-system-id":1000000000000000094,"db-version":"9.4"}
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
Renumber return codes between 25 and 125 to avoid PostgreSQL interpreting some as fatal signal exceptions. Suggested by Yogesh Sharma.
2017-04-12 18:41:34 +02:00
restore, backup '[BACKUP-DIFF-2]', expect exit 40 - fail on used path (db-master host)
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --set=[BACKUP-DIFF-2] --log-level-console=detail --stanza=db restore
Log testing can now be enabled for certain deterministic tests. This works by comparing the generated logs against a previous copy. Currently only enabled for the backup/synthetic tests.
2015-05-05 19:08:48 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Add log-subprocess option to allow file logging for local and remote subprocesses.
2018-08-23 02:05:49 +02:00
P00 INFO: restore command begin [BACKREST-VERSION]: --no-compress --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --lock-path=[TEST_PATH]/db-master/lock --log-level-console=detail --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --pg1-path=[TEST_PATH]/db-master/db/base --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --set=[BACKUP-DIFF-2] --stanza=db
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: restore backup set [BACKUP-DIFF-2]
P00 DETAIL: check [TEST_PATH]/db-master/db/base exists
Renumber return codes between 25 and 125 to avoid PostgreSQL interpreting some as fatal signal exceptions. Suggested by Yogesh Sharma.
2017-04-12 18:41:34 +02:00
P00 ERROR: [040]: cannot restore to path '[TEST_PATH]/db-master/db/base' that contains files - try using --delta if this is what you intended
P00 INFO: restore command end: aborted with exception [040]
Log testing can now be enabled for certain deterministic tests. This works by comparing the generated logs against a previous copy. Currently only enabled for the backup/synthetic tests.
2015-05-05 19:08:48 +02:00
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
restore, backup '[BACKUP-DIFF-2]', remap - remap all paths (db-master host)
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --set=[BACKUP-DIFF-2] --stanza=db restore
Log testing can now be enabled for certain deterministic tests. This works by comparing the generated logs against a previous copy. Currently only enabled for the backup/synthetic tests.
2015-05-05 19:08:48 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Add log-subprocess option to allow file logging for local and remote subprocesses.
2018-08-23 02:05:49 +02:00
P00 INFO: restore command begin [BACKREST-VERSION]: --no-compress --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --lock-path=[TEST_PATH]/db-master/lock --log-level-console=debug --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --pg1-path=[TEST_PATH]/db-master/db/base-2 --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --set=[BACKUP-DIFF-2] --stanza=db --tablespace-map=1=[TEST_PATH]/db-master/db/tablespace/ts1-2 --tablespace-map=2=[TEST_PATH]/db-master/db/tablespace/ts2-2
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: config/load::cfgLoad: => void
P00 DEBUG: perl/exec::perlExec: (void)
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Storage::Posix::Driver->new(): bFileSync = <true>, bPathSync = <true>
P00 DEBUG: Storage::Local->new(): bAllowTemp = <true>, hRule = [hash], lBufferMax = 4194304, oDriver = [object], strCipherPassUser = [undef], strCipherType = [undef], strDefaultFileMode = <0640>, strDefaultPathMode = <0750>, strPathBase = [TEST_PATH]/db-master/repo, strTempExtension = pgbackrest.tmp
P00 DEBUG: Storage::Local->pathExists(): strPathExp =
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
P00 DEBUG: Storage::Posix::Driver->new(): bFileSync = <true>, bPathSync = <true>
P00 DEBUG: Storage::Local->new(): bAllowTemp = <true>, hRule = [undef], lBufferMax = 4194304, oDriver = [object], strCipherPassUser = [undef], strCipherType = [undef], strDefaultFileMode = <0640>, strDefaultPathMode = <0750>, strPathBase = /, strTempExtension = pgbackrest.tmp
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = true, bIgnoreExists = true, strMode = 0770, strPathExp = [TEST_PATH]/db-master/log
P00 DEBUG: Storage::Posix::Driver->new(): bFileSync = <true>, bPathSync = <true>
P00 DEBUG: Storage::Local->new(): bAllowTemp = <true>, hRule = [undef], lBufferMax = 4194304, oDriver = [object], strCipherPassUser = [undef], strCipherType = [undef], strDefaultFileMode = <0640>, strDefaultPathMode = <0750>, strPathBase = [TEST_PATH]/db-master/db/base-2, strTempExtension = pgbackrest.tmp
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/base-2
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/db/base-2/postmaster.pid
P00 DEBUG: Storage::Local->exists=>: bExists = false
P00 DEBUG: Storage::Local->openRead(): bIgnoreMissing = <false>, rhyFilter = [undef], strCipherPass = [undef], xFileExp = <REPO:BACKUP>/backup.info
P00 DEBUG: Storage::Base->copy(): xDestinationFile = [TEST_PATH]/db-master/db/base-2/backup.info, xSourceFile = [object]
P00 DEBUG: Storage::Local->openWrite(): bAtomic = <false>, bPathCreate = <false>, lTimestamp = [undef], rhyFilter = [undef], strCipherPass = [undef], strGroup = [undef], strMode = <0640>, strUser = [undef], xFileExp = [TEST_PATH]/db-master/db/base-2/backup.info
P00 DEBUG: Backup::Info->new(): bIgnoreMissing = <false>, bLoad = <true>, bRequired = <true>, bValidate = false, oStorage = [object], strBackupClusterPath = [TEST_PATH]/db-master/db/base-2, strCipherPassSub = [undef]
P00 DEBUG: Storage::Local->encrypted(): bIgnoreMissing = true, strFileName = [TEST_PATH]/db-master/db/base-2/backup.info
P00 DEBUG: Storage::Local->encrypted=>: bEncrypted = false
P00 DEBUG: Storage::Local->encryptionValid(): bEncrypted = false
P00 DEBUG: Storage::Local->encryptionValid=>: bValid = true
P00 DEBUG: Storage::Local->openRead(): bIgnoreMissing = true, rhyFilter = [undef], strCipherPass = [undef], xFileExp = [TEST_PATH]/db-master/db/base-2/backup.info
P00 DEBUG: Storage::Base->get(): strCipherPass = [undef], xFile = [object]
P00 DEBUG: Storage::Local->remove(): bIgnoreMissing = <true>, bRecurse = <false>, xstryPathFileExp = [TEST_PATH]/db-master/db/base-2/backup.info
P00 DEBUG: Storage::Local->remove=>: bRemoved = true
P00 DEBUG: Backup::Info->current(): strBackup = [BACKUP-DIFF-2]
P00 DEBUG: Backup::Info->current=>: bTest = true
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: restore backup set [BACKUP-DIFF-2]
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Restore->manifestLoad(): strCipherPass = [undef]
P00 DEBUG: Storage::Local->exists(): strFileExp = <REPO:BACKUP>/[BACKUP-DIFF-2]/backup.manifest
P00 DEBUG: Storage::Local->exists=>: bExists = true
P00 DEBUG: Storage::Local->openRead(): bIgnoreMissing = <false>, rhyFilter = [undef], strCipherPass = [undef], xFileExp = <REPO:BACKUP>/[BACKUP-DIFF-2]/backup.manifest
P00 DEBUG: Storage::Base->copy(): xDestinationFile = [TEST_PATH]/db-master/db/base-2/backup.manifest, xSourceFile = [object]
P00 DEBUG: Storage::Local->openWrite(): bAtomic = <false>, bPathCreate = <false>, lTimestamp = [undef], rhyFilter = [undef], strCipherPass = [undef], strGroup = [undef], strMode = <0640>, strUser = [undef], xFileExp = [TEST_PATH]/db-master/db/base-2/backup.manifest
Require PostgreSQL catalog version when instantiating a Manifest object (and not loading it from disk). Contributed by Cynthia Shang.
2018-07-16 23:25:15 +02:00
P00 DEBUG: Manifest->new(): bLoad = <true>, iDbCatalogVersion = [undef], oStorage = [object], strCipherPass = [undef], strCipherPassSub = [undef], strDbVersion = [undef], strFileName = [TEST_PATH]/db-master/db/base-2/backup.manifest
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Storage::Local->encrypted(): bIgnoreMissing = true, strFileName = [TEST_PATH]/db-master/db/base-2/backup.manifest
P00 DEBUG: Storage::Local->encrypted=>: bEncrypted = false
P00 DEBUG: Storage::Local->encryptionValid(): bEncrypted = false
P00 DEBUG: Storage::Local->encryptionValid=>: bValid = true
P00 DEBUG: Storage::Local->openRead(): bIgnoreMissing = true, rhyFilter = [undef], strCipherPass = [undef], xFileExp = [TEST_PATH]/db-master/db/base-2/backup.manifest
P00 DEBUG: Storage::Base->get(): strCipherPass = [undef], xFile = [object]
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: remap $PGDATA directory to [TEST_PATH]/db-master/db/base-2
P00 INFO: remap tablespace pg_tblspc/1 directory to [TEST_PATH]/db-master/db/tablespace/ts1-2
P00 INFO: remap tablespace pg_tblspc/2 directory to [TEST_PATH]/db-master/db/tablespace/ts2-2
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Restore->manifestOwnershipCheck(): oManifest = [object]
P00 DEBUG: Storage::Local->remove(): bIgnoreMissing = <true>, bRecurse = <false>, xstryPathFileExp = [TEST_PATH]/db-master/db/base-2/global/pg_control
P00 DEBUG: Storage::Local->remove=>: bRemoved = false
P00 DEBUG: Restore->clean(): oManifest = [object]
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 DETAIL: check [TEST_PATH]/db-master/db/base-2 exists
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/base-2
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
Ignore all files in a linked tablespace directory except the subdirectory for the current version of PostgreSQL. Previously an error would be generated if other files were present and not owned by the PostgreSQL user. This hasn't been a big deal in practice but it could cause issues. Also add tests to make sure the same logic applies with links to files, i.e. all other files in the directory should be ignored. This was actually working correctly, but there were no tests for it before.
2018-08-31 22:06:40 +02:00
P00 DEBUG: Storage::Local->manifest(): strFilter = [undef], strPathExp = [TEST_PATH]/db-master/db/base-2
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 DETAIL: check [TEST_PATH]/db-master/db/tablespace/ts1-2 exists
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/tablespace/ts1-2
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/tablespace/ts1-2/[TS_PATH-1]
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 DETAIL: check [TEST_PATH]/db-master/db/tablespace/ts2-2 exists
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/tablespace/ts2-2
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/tablespace/ts2-2/[TS_PATH-1]
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Restore->build(): oManifest = [object]
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/tablespace/ts1-2
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/tablespace/ts2-2
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
P00 DEBUG: build level 1 paths/links
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/base-2/base
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/db/base-2/base
P00 DEBUG: Storage::Local->exists=>: bExists = false
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = <false>, strMode = 0700, strPathExp = [TEST_PATH]/db-master/db/base-2/base
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/base-2/global
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/db/base-2/global
P00 DEBUG: Storage::Local->exists=>: bExists = false
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = <false>, strMode = 0700, strPathExp = [TEST_PATH]/db-master/db/base-2/global
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/base-2/pg_clog
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/db/base-2/pg_clog
P00 DEBUG: Storage::Local->exists=>: bExists = false
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = <false>, strMode = 0700, strPathExp = [TEST_PATH]/db-master/db/base-2/pg_clog
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/base-2/pg_dynshmem
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/db/base-2/pg_dynshmem
P00 DEBUG: Storage::Local->exists=>: bExists = false
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = <false>, strMode = 0700, strPathExp = [TEST_PATH]/db-master/db/base-2/pg_dynshmem
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/base-2/pg_log
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/db/base-2/pg_log
P00 DEBUG: Storage::Local->exists=>: bExists = false
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = <false>, strMode = 0700, strPathExp = [TEST_PATH]/db-master/db/base-2/pg_log
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/base-2/pg_notify
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/db/base-2/pg_notify
P00 DEBUG: Storage::Local->exists=>: bExists = false
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = <false>, strMode = 0700, strPathExp = [TEST_PATH]/db-master/db/base-2/pg_notify
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/base-2/pg_replslot
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/db/base-2/pg_replslot
P00 DEBUG: Storage::Local->exists=>: bExists = false
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = <false>, strMode = 0700, strPathExp = [TEST_PATH]/db-master/db/base-2/pg_replslot
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/base-2/pg_serial
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/db/base-2/pg_serial
P00 DEBUG: Storage::Local->exists=>: bExists = false
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = <false>, strMode = 0700, strPathExp = [TEST_PATH]/db-master/db/base-2/pg_serial
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/base-2/pg_snapshots
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/db/base-2/pg_snapshots
P00 DEBUG: Storage::Local->exists=>: bExists = false
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = <false>, strMode = 0700, strPathExp = [TEST_PATH]/db-master/db/base-2/pg_snapshots
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/base-2/pg_stat
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/db/base-2/pg_stat
P00 DEBUG: Storage::Local->exists=>: bExists = false
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = <false>, strMode = 0700, strPathExp = [TEST_PATH]/db-master/db/base-2/pg_stat
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/base-2/pg_stat_tmp
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/db/base-2/pg_stat_tmp
P00 DEBUG: Storage::Local->exists=>: bExists = false
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = <false>, strMode = 0700, strPathExp = [TEST_PATH]/db-master/db/base-2/pg_stat_tmp
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/base-2/pg_subtrans
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/db/base-2/pg_subtrans
P00 DEBUG: Storage::Local->exists=>: bExists = false
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = <false>, strMode = 0700, strPathExp = [TEST_PATH]/db-master/db/base-2/pg_subtrans
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/base-2/pg_tblspc
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/db/base-2/pg_tblspc
P00 DEBUG: Storage::Local->exists=>: bExists = false
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = <false>, strMode = 0700, strPathExp = [TEST_PATH]/db-master/db/base-2/pg_tblspc
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/base-2/pg_tblspc
P00 DEBUG: Storage::Local->pathExists=>: bExists = true
P00 DEBUG: build level 2 paths/links
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/base-2/base/1
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/db/base-2/base/1
P00 DEBUG: Storage::Local->exists=>: bExists = false
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = <false>, strMode = 0700, strPathExp = [TEST_PATH]/db-master/db/base-2/base/1
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/base-2/base/16384
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/db/base-2/base/16384
P00 DEBUG: Storage::Local->exists=>: bExists = false
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = <false>, strMode = 0700, strPathExp = [TEST_PATH]/db-master/db/base-2/base/16384
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/base-2/base/32768
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/db/base-2/base/32768
P00 DEBUG: Storage::Local->exists=>: bExists = false
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = <false>, strMode = 0700, strPathExp = [TEST_PATH]/db-master/db/base-2/base/32768
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/base-2/pg_tblspc/1
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/db/base-2/pg_tblspc/1
P00 DEBUG: Storage::Local->exists=>: bExists = false
P00 DEBUG: Storage::Local->linkCreate(): bHard = <false>, bIgnoreExists = true, bPathCreate = <true>, bRelative = false, strDestinationLinkExp = [TEST_PATH]/db-master/db/base-2/pg_tblspc/1, strSourcePathFileExp = [TEST_PATH]/db-master/db/tablespace/ts1-2
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/base-2/pg_tblspc/2
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/db/base-2/pg_tblspc/2
P00 DEBUG: Storage::Local->exists=>: bExists = false
P00 DEBUG: Storage::Local->linkCreate(): bHard = <false>, bIgnoreExists = true, bPathCreate = <true>, bRelative = false, strDestinationLinkExp = [TEST_PATH]/db-master/db/base-2/pg_tblspc/2, strSourcePathFileExp = [TEST_PATH]/db-master/db/tablespace/ts2-2
P00 DEBUG: build level 3 paths/links
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/base-2/pg_tblspc/1/[TS_PATH-1]
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/db/base-2/pg_tblspc/1/[TS_PATH-1]
P00 DEBUG: Storage::Local->exists=>: bExists = false
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = <false>, strMode = 0700, strPathExp = [TEST_PATH]/db-master/db/base-2/pg_tblspc/1/[TS_PATH-1]
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/base-2/pg_tblspc/2/[TS_PATH-1]
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/db/base-2/pg_tblspc/2/[TS_PATH-1]
P00 DEBUG: Storage::Local->exists=>: bExists = false
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = <false>, strMode = 0700, strPathExp = [TEST_PATH]/db-master/db/base-2/pg_tblspc/2/[TS_PATH-1]
P00 DEBUG: build level 4 paths/links
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/base-2/pg_tblspc/1/[TS_PATH-1]/16384
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/db/base-2/pg_tblspc/1/[TS_PATH-1]/16384
P00 DEBUG: Storage::Local->exists=>: bExists = false
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = <false>, strMode = 0700, strPathExp = [TEST_PATH]/db-master/db/base-2/pg_tblspc/1/[TS_PATH-1]/16384
P00 DEBUG: Storage::Local->pathExists(): strPathExp = [TEST_PATH]/db-master/db/base-2/pg_tblspc/2/[TS_PATH-1]/32768
P00 DEBUG: Storage::Local->pathExists=>: bExists = false
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/db/base-2/pg_tblspc/2/[TS_PATH-1]/32768
P00 DEBUG: Storage::Local->exists=>: bExists = false
P00 DEBUG: Storage::Local->pathCreate(): bCreateParent = <false>, bIgnoreExists = <false>, strMode = 0700, strPathExp = [TEST_PATH]/db-master/db/base-2/pg_tblspc/2/[TS_PATH-1]/32768
P00 DEBUG: build level 5 paths/links
P00 DEBUG: Protocol::Local::Process->new(): bConfessError = <true>, iSelectTimeout = <30>, strBackRestBin = <[BACKREST-BIN]>, strHostType = backup
P00 DEBUG: Protocol::Local::Process->hostAdd(): iHostConfigIdx = 1, iProcessMax = 1
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base-2/base/32768/33001, 65536, [MODIFICATION-TIME-2], 6bf316f11d28c28914ea9be92c00de9bea6d9a6b, 0, 0, pg_data/base/32768/33001, [BACKUP-FULL-2], 0600, [USER-1], [GROUP-1], [MODIFICATION-TIME-7], 0, [BACKUP-DIFF-2], 0), rParamSecure = [undef], strKey = pg_data/base/32768/33001, strOp = restoreFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base-2/base/32768/44000_init, 32768, [MODIFICATION-TIME-2], 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f, 0, 0, pg_data/base/32768/44000_init, [BACKUP-FULL-2], 0600, [USER-1], [GROUP-1], [MODIFICATION-TIME-7], 0, [BACKUP-DIFF-2], 0), rParamSecure = [undef], strKey = pg_data/base/32768/44000_init, strOp = restoreFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base-2/base/32768/33000.32767, 32768, [MODIFICATION-TIME-2], 21e2c7c1a326682c07053b7d6a5a40dbd49c2ec5, 0, 0, pg_data/base/32768/33000.32767, [BACKUP-FULL-2], 0600, [USER-1], [GROUP-1], [MODIFICATION-TIME-7], 0, [BACKUP-DIFF-2], 0), rParamSecure = [undef], strKey = pg_data/base/32768/33000.32767, strOp = restoreFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base-2/base/32768/33000, 32768, [MODIFICATION-TIME-2], 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f, 0, 0, pg_data/base/32768/33000, [BACKUP-FULL-2], 0600, [USER-1], [GROUP-1], [MODIFICATION-TIME-7], 0, [BACKUP-DIFF-2], 0), rParamSecure = [undef], strKey = pg_data/base/32768/33000, strOp = restoreFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base-2/base/16384/17000, 16384, [MODIFICATION-TIME-2], e0101dd8ffb910c9c202ca35b5f828bcb9697bed, 0, 0, pg_data/base/16384/17000, [BACKUP-FULL-2], 0600, [USER-1], [GROUP-1], [MODIFICATION-TIME-7], 0, [BACKUP-DIFF-2], 0), rParamSecure = [undef], strKey = pg_data/base/16384/17000, strOp = restoreFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base-2/global/pg_control.pgbackrest.tmp, 8192, [MODIFICATION-TIME-3], 4c77c900f7af0d9ab13fa9982051a42e0b637f6c, 0, 0, pg_data/global/pg_control, [BACKUP-FULL-2], 0600, [USER-1], [GROUP-1], [MODIFICATION-TIME-7], 0, [BACKUP-DIFF-2], 0), rParamSecure = [undef], strKey = pg_data/global/pg_control, strOp = restoreFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base-2/base/1/12000, 8192, [MODIFICATION-TIME-2], 22c98d248ff548311eda88559e4a8405ed77c003, 0, 0, pg_data/base/1/12000, [BACKUP-FULL-2], 0600, [USER-1], [GROUP-1], [MODIFICATION-TIME-7], 0, [BACKUP-DIFF-2], 0), rParamSecure = [undef], strKey = pg_data/base/1/12000, strOp = restoreFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base-2/postgresql.conf, 21, [MODIFICATION-TIME-3], 6721d92c9fcdf4248acff1f9a1377127d9064807, 0, 0, pg_data/postgresql.conf, [BACKUP-FULL-2], 0600, [USER-1], [GROUP-1], [MODIFICATION-TIME-7], 0, [BACKUP-DIFF-2], 0), rParamSecure = [undef], strKey = pg_data/postgresql.conf, strOp = restoreFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base-2/badchecksum.txt, 11, [MODIFICATION-TIME-2], f927212cd08d11a42a666b2f04235398e9ceeb51, 0, 0, pg_data/badchecksum.txt, [undef], 0600, [USER-1], [GROUP-1], [MODIFICATION-TIME-7], 0, [BACKUP-DIFF-2], 0), rParamSecure = [undef], strKey = pg_data/badchecksum.txt, strOp = restoreFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base-2/pg_tblspc/2/[TS_PATH-1]/32768/tablespace2.txt, 7, [MODIFICATION-TIME-2], dc7f76e43c46101b47acc55ae4d593a9e6983578, 0, 0, pg_tblspc/2/[TS_PATH-1]/32768/tablespace2.txt, [undef], 0600, [USER-1], [GROUP-1], [MODIFICATION-TIME-7], 0, [BACKUP-DIFF-2], 0), rParamSecure = [undef], strKey = pg_tblspc/2/[TS_PATH-1]/32768/tablespace2.txt, strOp = restoreFile, strQueue = pg_tblspc/2
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base-2/pg_tblspc/1/[TS_PATH-1]/16384/tablespace1.txt, 7, [MODIFICATION-TIME-2], d85de07d6421d90aa9191c11c889bfde43680f0f, 0, 0, pg_tblspc/1/[TS_PATH-1]/16384/tablespace1.txt, [undef], 0600, [USER-1], [GROUP-1], [MODIFICATION-TIME-7], 0, [BACKUP-DIFF-2], 0), rParamSecure = [undef], strKey = pg_tblspc/1/[TS_PATH-1]/16384/tablespace1.txt, strOp = restoreFile, strQueue = pg_tblspc/1
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base-2/changecontent.txt, 7, [MODIFICATION-TIME-2], a094d94583e209556d03c3c5da33131a065f1689, 0, 0, pg_data/changecontent.txt, [undef], 0600, [USER-1], [GROUP-1], [MODIFICATION-TIME-7], 0, [BACKUP-DIFF-2], 0), rParamSecure = [undef], strKey = pg_data/changecontent.txt, strOp = restoreFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base-2/pg_stat/global.stat, 5, [MODIFICATION-TIME-3], e350d5ce0153f3e22d5db21cf2a4eff00f3ee877, 0, 0, pg_data/pg_stat/global.stat, [BACKUP-FULL-2], 0600, [USER-1], [GROUP-1], [MODIFICATION-TIME-7], 0, [BACKUP-DIFF-2], 0), rParamSecure = [undef], strKey = pg_data/pg_stat/global.stat, strOp = restoreFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base-2/changetime.txt, 4, [MODIFICATION-TIME-3], 88087292ed82e26f3eb824d0bffc05ccf7a30f8d, 0, 0, pg_data/changetime.txt, [BACKUP-FULL-2], 0600, [USER-1], [GROUP-1], [MODIFICATION-TIME-7], 0, [BACKUP-DIFF-2], 0), rParamSecure = [undef], strKey = pg_data/changetime.txt, strOp = restoreFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base-2/base/32768/PG_VERSION, 3, [MODIFICATION-TIME-2], 184473f470864e067ee3a22e64b47b0a1c356f29, 0, 0, pg_data/base/32768/PG_VERSION, [BACKUP-FULL-2], 0600, [USER-1], [GROUP-1], [MODIFICATION-TIME-7], 0, [BACKUP-DIFF-2], 0), rParamSecure = [undef], strKey = pg_data/base/32768/PG_VERSION, strOp = restoreFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base-2/base/16384/PG_VERSION, 3, [MODIFICATION-TIME-2], 184473f470864e067ee3a22e64b47b0a1c356f29, 0, 0, pg_data/base/16384/PG_VERSION, [BACKUP-FULL-2], 0600, [USER-1], [GROUP-1], [MODIFICATION-TIME-7], 0, [BACKUP-DIFF-2], 0), rParamSecure = [undef], strKey = pg_data/base/16384/PG_VERSION, strOp = restoreFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base-2/base/1/PG_VERSION, 3, [MODIFICATION-TIME-2], 184473f470864e067ee3a22e64b47b0a1c356f29, 0, 0, pg_data/base/1/PG_VERSION, [BACKUP-FULL-2], 0660, [USER-1], [GROUP-1], [MODIFICATION-TIME-7], 0, [BACKUP-DIFF-2], 0), rParamSecure = [undef], strKey = pg_data/base/1/PG_VERSION, strOp = restoreFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base-2/PG_VERSION, 3, [MODIFICATION-TIME-2], 184473f470864e067ee3a22e64b47b0a1c356f29, 0, 0, pg_data/PG_VERSION, [BACKUP-FULL-2], 0600, [USER-1], [GROUP-1], [MODIFICATION-TIME-7], 0, [BACKUP-DIFF-2], 0), rParamSecure = [undef], strKey = pg_data/PG_VERSION, strOp = restoreFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base-2/zerosize.txt, 0, [MODIFICATION-TIME-3], [undef], 0, 0, pg_data/zerosize.txt, [undef], 0600, [USER-1], [GROUP-1], [MODIFICATION-TIME-7], 0, [BACKUP-DIFF-2], 0), rParamSecure = [undef], strKey = pg_data/zerosize.txt, strOp = restoreFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base-2/zero_from_start, 0, [MODIFICATION-TIME-2], [undef], 0, 0, pg_data/zero_from_start, [BACKUP-FULL-2], 0600, [USER-1], [GROUP-1], [MODIFICATION-TIME-7], 0, [BACKUP-DIFF-2], 0), rParamSecure = [undef], strKey = pg_data/zero_from_start, strOp = restoreFile, strQueue = pg_data
P00 DEBUG: Protocol::Local::Process->queueJob(): iHostConfigIdx = 1, rParam = ([TEST_PATH]/db-master/db/base-2/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?%, 0, [MODIFICATION-TIME-2], [undef], 0, 0, pg_data/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?%, [BACKUP-FULL-2], 0600, [USER-1], [GROUP-1], [MODIFICATION-TIME-7], 0, [BACKUP-DIFF-2], 0), rParamSecure = [undef], strKey = pg_data/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?%, strOp = restoreFile, strQueue = pg_data
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Protocol::Local::Process->hostConnect: start local process: iHostConfigIdx = 1, iHostIdx = 0, iHostProcessIdx = 0, iProcessId = 1, strHostType = backup
Add log-subprocess option to allow file logging for local and remote subprocesses.
2018-08-23 02:05:49 +02:00
P00 DEBUG: Protocol::Local::Master->new(): iProcessIdx = 1, strCommand = [BACKREST-BIN] --command=restore --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --host-id=1 --lock-path=[TEST_PATH]/db-master/lock --log-level-file=trace --log-path=[TEST_PATH]/db-master/log --log-subprocess --pg1-path=[TEST_PATH]/db-master/db/base-2 --process=1 --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --stanza=db --type=backup local
P00 DEBUG: Protocol::Command::Master->new(): iBufferMax = 4194304, iCompressLevel = 3, iCompressLevelNetwork = 3, iProtocolTimeout = 60, strCommand = [BACKREST-BIN] --command=restore --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --host-id=1 --lock-path=[TEST_PATH]/db-master/lock --log-level-file=trace --log-path=[TEST_PATH]/db-master/log --log-subprocess --pg1-path=[TEST_PATH]/db-master/db/base-2 --process=1 --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --stanza=db --type=backup local, strId = local-1 process, strName = local
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Protocol::Local::Process->hostConnect=>: bResult = true
P00 DEBUG: Protocol::Local::Process->init: init local process: iDirection = 1, iHostIdx = 0, iProcessId = 1, iQueueIdx = 0, iQueueLastIdx = 2
P00 DEBUG: Protocol::Local::Process->init=>: bResult = true
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/base/32768/33001, strQueueIdx = 0
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1), strKey = pg_data/base/32768/33001
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/base/32768/44000_init, strQueueIdx = 0
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: RestoreFile::restoreLog(): bCopy = true, bForce = false, bZero = false, iLocalId = 1, lModificationTime = [MODIFICATION-TIME-2], lSize = 65536, lSizeCurrent = 0, lSizeTotal = 196682, strChecksum = 6bf316f11d28c28914ea9be92c00de9bea6d9a6b, strDbFile = [TEST_PATH]/db-master/db/base-2/base/32768/33001
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/base/32768/33001 (64KB, 33%) checksum 6bf316f11d28c28914ea9be92c00de9bea6d9a6b
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1), strKey = pg_data/base/32768/44000_init
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/base/32768/33000.32767, strQueueIdx = 0
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: RestoreFile::restoreLog(): bCopy = true, bForce = false, bZero = false, iLocalId = 1, lModificationTime = [MODIFICATION-TIME-2], lSize = 32768, lSizeCurrent = 65536, lSizeTotal = 196682, strChecksum = 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f, strDbFile = [TEST_PATH]/db-master/db/base-2/base/32768/44000_init
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/base/32768/44000_init (32KB, 49%) checksum 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1), strKey = pg_data/base/32768/33000.32767
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/base/32768/33000, strQueueIdx = 0
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: RestoreFile::restoreLog(): bCopy = true, bForce = false, bZero = false, iLocalId = 1, lModificationTime = [MODIFICATION-TIME-2], lSize = 32768, lSizeCurrent = 98304, lSizeTotal = 196682, strChecksum = 21e2c7c1a326682c07053b7d6a5a40dbd49c2ec5, strDbFile = [TEST_PATH]/db-master/db/base-2/base/32768/33000.32767
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/base/32768/33000.32767 (32KB, 66%) checksum 21e2c7c1a326682c07053b7d6a5a40dbd49c2ec5
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1), strKey = pg_data/base/32768/33000
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/base/16384/17000, strQueueIdx = 0
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: RestoreFile::restoreLog(): bCopy = true, bForce = false, bZero = false, iLocalId = 1, lModificationTime = [MODIFICATION-TIME-2], lSize = 32768, lSizeCurrent = 131072, lSizeTotal = 196682, strChecksum = 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f, strDbFile = [TEST_PATH]/db-master/db/base-2/base/32768/33000
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/base/32768/33000 (32KB, 83%) checksum 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1), strKey = pg_data/base/16384/17000
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/global/pg_control, strQueueIdx = 0
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: RestoreFile::restoreLog(): bCopy = true, bForce = false, bZero = false, iLocalId = 1, lModificationTime = [MODIFICATION-TIME-2], lSize = 16384, lSizeCurrent = 163840, lSizeTotal = 196682, strChecksum = e0101dd8ffb910c9c202ca35b5f828bcb9697bed, strDbFile = [TEST_PATH]/db-master/db/base-2/base/16384/17000
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/base/16384/17000 (16KB, 91%) checksum e0101dd8ffb910c9c202ca35b5f828bcb9697bed
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1), strKey = pg_data/global/pg_control
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/base/1/12000, strQueueIdx = 0
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: RestoreFile::restoreLog(): bCopy = true, bForce = false, bZero = false, iLocalId = 1, lModificationTime = [MODIFICATION-TIME-3], lSize = 8192, lSizeCurrent = 180224, lSizeTotal = 196682, strChecksum = 4c77c900f7af0d9ab13fa9982051a42e0b637f6c, strDbFile = [TEST_PATH]/db-master/db/base-2/global/pg_control.pgbackrest.tmp
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 11:24:42 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/global/pg_control.pgbackrest.tmp (8KB, 95%) checksum 4c77c900f7af0d9ab13fa9982051a42e0b637f6c
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1), strKey = pg_data/base/1/12000
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/postgresql.conf, strQueueIdx = 0
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: RestoreFile::restoreLog(): bCopy = true, bForce = false, bZero = false, iLocalId = 1, lModificationTime = [MODIFICATION-TIME-2], lSize = 8192, lSizeCurrent = 188416, lSizeTotal = 196682, strChecksum = 22c98d248ff548311eda88559e4a8405ed77c003, strDbFile = [TEST_PATH]/db-master/db/base-2/base/1/12000
Add more realistic data files to synthetic backup and restore tests.
2016-12-10 16:13:55 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/base/1/12000 (8KB, 99%) checksum 22c98d248ff548311eda88559e4a8405ed77c003
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1), strKey = pg_data/postgresql.conf
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/badchecksum.txt, strQueueIdx = 0
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: RestoreFile::restoreLog(): bCopy = true, bForce = false, bZero = false, iLocalId = 1, lModificationTime = [MODIFICATION-TIME-3], lSize = 21, lSizeCurrent = 196608, lSizeTotal = 196682, strChecksum = 6721d92c9fcdf4248acff1f9a1377127d9064807, strDbFile = [TEST_PATH]/db-master/db/base-2/postgresql.conf
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/postgresql.conf (21B, 99%) checksum 6721d92c9fcdf4248acff1f9a1377127d9064807
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1), strKey = pg_data/badchecksum.txt
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/changecontent.txt, strQueueIdx = 0
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: RestoreFile::restoreLog(): bCopy = true, bForce = false, bZero = false, iLocalId = 1, lModificationTime = [MODIFICATION-TIME-2], lSize = 11, lSizeCurrent = 196629, lSizeTotal = 196682, strChecksum = f927212cd08d11a42a666b2f04235398e9ceeb51, strDbFile = [TEST_PATH]/db-master/db/base-2/badchecksum.txt
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/badchecksum.txt (11B, 99%) checksum f927212cd08d11a42a666b2f04235398e9ceeb51
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1), strKey = pg_data/changecontent.txt
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/pg_stat/global.stat, strQueueIdx = 0
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: RestoreFile::restoreLog(): bCopy = true, bForce = false, bZero = false, iLocalId = 1, lModificationTime = [MODIFICATION-TIME-2], lSize = 7, lSizeCurrent = 196640, lSizeTotal = 196682, strChecksum = a094d94583e209556d03c3c5da33131a065f1689, strDbFile = [TEST_PATH]/db-master/db/base-2/changecontent.txt
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/changecontent.txt (7B, 99%) checksum a094d94583e209556d03c3c5da33131a065f1689
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1), strKey = pg_data/pg_stat/global.stat
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/changetime.txt, strQueueIdx = 0
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: RestoreFile::restoreLog(): bCopy = true, bForce = false, bZero = false, iLocalId = 1, lModificationTime = [MODIFICATION-TIME-3], lSize = 5, lSizeCurrent = 196647, lSizeTotal = 196682, strChecksum = e350d5ce0153f3e22d5db21cf2a4eff00f3ee877, strDbFile = [TEST_PATH]/db-master/db/base-2/pg_stat/global.stat
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/pg_stat/global.stat (5B, 99%) checksum e350d5ce0153f3e22d5db21cf2a4eff00f3ee877
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1), strKey = pg_data/changetime.txt
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/base/32768/PG_VERSION, strQueueIdx = 0
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: RestoreFile::restoreLog(): bCopy = true, bForce = false, bZero = false, iLocalId = 1, lModificationTime = [MODIFICATION-TIME-3], lSize = 4, lSizeCurrent = 196652, lSizeTotal = 196682, strChecksum = 88087292ed82e26f3eb824d0bffc05ccf7a30f8d, strDbFile = [TEST_PATH]/db-master/db/base-2/changetime.txt
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/changetime.txt (4B, 99%) checksum 88087292ed82e26f3eb824d0bffc05ccf7a30f8d
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1), strKey = pg_data/base/32768/PG_VERSION
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/base/16384/PG_VERSION, strQueueIdx = 0
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: RestoreFile::restoreLog(): bCopy = true, bForce = false, bZero = false, iLocalId = 1, lModificationTime = [MODIFICATION-TIME-2], lSize = 3, lSizeCurrent = 196656, lSizeTotal = 196682, strChecksum = 184473f470864e067ee3a22e64b47b0a1c356f29, strDbFile = [TEST_PATH]/db-master/db/base-2/base/32768/PG_VERSION
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/base/32768/PG_VERSION (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1), strKey = pg_data/base/16384/PG_VERSION
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/base/1/PG_VERSION, strQueueIdx = 0
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: RestoreFile::restoreLog(): bCopy = true, bForce = false, bZero = false, iLocalId = 1, lModificationTime = [MODIFICATION-TIME-2], lSize = 3, lSizeCurrent = 196659, lSizeTotal = 196682, strChecksum = 184473f470864e067ee3a22e64b47b0a1c356f29, strDbFile = [TEST_PATH]/db-master/db/base-2/base/16384/PG_VERSION
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/base/16384/PG_VERSION (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1), strKey = pg_data/base/1/PG_VERSION
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/PG_VERSION, strQueueIdx = 0
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: RestoreFile::restoreLog(): bCopy = true, bForce = false, bZero = false, iLocalId = 1, lModificationTime = [MODIFICATION-TIME-2], lSize = 3, lSizeCurrent = 196662, lSizeTotal = 196682, strChecksum = 184473f470864e067ee3a22e64b47b0a1c356f29, strDbFile = [TEST_PATH]/db-master/db/base-2/base/1/PG_VERSION
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/base/1/PG_VERSION (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1), strKey = pg_data/PG_VERSION
Add zero-length file to mock/all test.
2018-07-05 21:40:50 +02:00
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/zerosize.txt, strQueueIdx = 0
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: RestoreFile::restoreLog(): bCopy = true, bForce = false, bZero = false, iLocalId = 1, lModificationTime = [MODIFICATION-TIME-2], lSize = 3, lSizeCurrent = 196665, lSizeTotal = 196682, strChecksum = 184473f470864e067ee3a22e64b47b0a1c356f29, strDbFile = [TEST_PATH]/db-master/db/base-2/PG_VERSION
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/PG_VERSION (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
Add zero-length file to mock/all test.
2018-07-05 21:40:50 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1), strKey = pg_data/zerosize.txt
Allow zero-size files in backup manifest to reference a prior manifest regardless of timestamp delta. Contributed by Cynthia Shang.
2018-08-24 22:50:33 +02:00
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/zero_from_start, strQueueIdx = 0
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: RestoreFile::restoreLog(): bCopy = true, bForce = false, bZero = false, iLocalId = 1, lModificationTime = [MODIFICATION-TIME-3], lSize = 0, lSizeCurrent = 196668, lSizeTotal = 196682, strChecksum = [undef], strDbFile = [TEST_PATH]/db-master/db/base-2/zerosize.txt
Add zero-length file to mock/all test.
2018-07-05 21:40:50 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/zerosize.txt (0B, 99%)
Allow zero-size files in backup manifest to reference a prior manifest regardless of timestamp delta. Contributed by Cynthia Shang.
2018-08-24 22:50:33 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1), strKey = pg_data/zero_from_start
Improve support for special characters in filenames. % characters caused issues in backup/restore due to filenames being appended directly into a format string. Reserved XML characters (<>&') caused issues in the S3 driver due to improper escaping. Add a file with all common special characters to regression testing.
2018-09-10 16:54:34 +02:00
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_data/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?%, strQueueIdx = 0
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: RestoreFile::restoreLog(): bCopy = true, bForce = false, bZero = false, iLocalId = 1, lModificationTime = [MODIFICATION-TIME-2], lSize = 0, lSizeCurrent = 196668, lSizeTotal = 196682, strChecksum = [undef], strDbFile = [TEST_PATH]/db-master/db/base-2/zero_from_start
Allow zero-size files in backup manifest to reference a prior manifest regardless of timestamp delta. Contributed by Cynthia Shang.
2018-08-24 22:50:33 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/zero_from_start (0B, 99%)
Improve support for special characters in filenames. % characters caused issues in backup/restore due to filenames being appended directly into a format string. Reserved XML characters (<>&') caused issues in the S3 driver due to improper escaping. Add a file with all common special characters to regression testing.
2018-09-10 16:54:34 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1), strKey = pg_data/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?%
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_tblspc/2/[TS_PATH-1]/32768/tablespace2.txt, strQueueIdx = 1
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: RestoreFile::restoreLog(): bCopy = true, bForce = false, bZero = false, iLocalId = 1, lModificationTime = [MODIFICATION-TIME-2], lSize = 0, lSizeCurrent = 196668, lSizeTotal = 196682, strChecksum = [undef], strDbFile = [TEST_PATH]/db-master/db/base-2/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?%
Improve support for special characters in filenames. % characters caused issues in backup/restore due to filenames being appended directly into a format string. Reserved XML characters (<>&') caused issues in the S3 driver due to improper escaping. Add a file with all common special characters to regression testing.
2018-09-10 16:54:34 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?% (0B, 99%)
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1), strKey = pg_tblspc/2/[TS_PATH-1]/32768/tablespace2.txt
P00 DEBUG: Protocol::Local::Process->process: get job from queue: iHostIdx = 0, iProcessId = 1, strKey = pg_tblspc/1/[TS_PATH-1]/16384/tablespace1.txt, strQueueIdx = 2
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: RestoreFile::restoreLog(): bCopy = true, bForce = false, bZero = false, iLocalId = 1, lModificationTime = [MODIFICATION-TIME-2], lSize = 7, lSizeCurrent = 196668, lSizeTotal = 196682, strChecksum = dc7f76e43c46101b47acc55ae4d593a9e6983578, strDbFile = [TEST_PATH]/db-master/db/base-2/pg_tblspc/2/[TS_PATH-1]/32768/tablespace2.txt
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/pg_tblspc/2/[TS_PATH-1]/32768/tablespace2.txt (7B, 99%) checksum dc7f76e43c46101b47acc55ae4d593a9e6983578
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Protocol::Local::Process->process: job complete: iProcessId = 1, rResult = (1), strKey = pg_tblspc/1/[TS_PATH-1]/16384/tablespace1.txt
P00 DEBUG: Protocol::Local::Process->process: no jobs found, stop local: iHostConfigIdx = [undef], iHostIdx = 0, iProcessId = 1, strHostType = [undef]
P00 DEBUG: Protocol::Command::Master->close=>: iExitStatus = 0
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 DEBUG: RestoreFile::restoreLog(): bCopy = true, bForce = false, bZero = false, iLocalId = 1, lModificationTime = [MODIFICATION-TIME-2], lSize = 7, lSizeCurrent = 196675, lSizeTotal = 196682, strChecksum = d85de07d6421d90aa9191c11c889bfde43680f0f, strDbFile = [TEST_PATH]/db-master/db/base-2/pg_tblspc/1/[TS_PATH-1]/16384/tablespace1.txt
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/pg_tblspc/1/[TS_PATH-1]/16384/tablespace1.txt (7B, 100%) checksum d85de07d6421d90aa9191c11c889bfde43680f0f
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Protocol::Local::Process->process: all jobs complete
P00 DEBUG: Storage::Local->exists(): strFileExp = [TEST_PATH]/db-master/db/base-2/recovery.conf
P00 DEBUG: Storage::Local->exists=>: bExists = false
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P00 INFO: write [TEST_PATH]/db-master/db/base-2/recovery.conf
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/db/base-2/base
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/db/base-2/base/1
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/db/base-2/base/16384
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/db/base-2/base/32768
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/db/base-2/global
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/db/base-2/pg_clog
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/db/base-2/pg_dynshmem
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/db/base-2/pg_log
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/db/base-2/pg_notify
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/db/base-2/pg_replslot
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/db/base-2/pg_serial
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/db/base-2/pg_snapshots
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/db/base-2/pg_stat
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/db/base-2/pg_stat_tmp
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/db/base-2/pg_subtrans
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/db/base-2/pg_tblspc
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/db/base-2/pg_tblspc/1/[TS_PATH-1]
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/db/base-2/pg_tblspc/1/[TS_PATH-1]/16384
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/db/base-2/pg_tblspc/2/[TS_PATH-1]
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/db/base-2/pg_tblspc/2/[TS_PATH-1]/32768
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/db/tablespace/ts1-2
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/db/tablespace/ts2-2
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 INFO: restore global/pg_control (performed last to ensure aborted restores cannot be started)
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Storage::Local->move(): bPathCreate = <false>, strDestinationPathExp = [TEST_PATH]/db-master/db/base-2/global/pg_control, strSourcePathExp = [TEST_PATH]/db-master/db/base-2/global/pg_control.pgbackrest.tmp
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/db/base-2
P00 DEBUG: Storage::Local->remove(): bIgnoreMissing = <true>, bRecurse = <false>, xstryPathFileExp = [TEST_PATH]/db-master/db/base-2/backup.manifest
P00 DEBUG: Storage::Local->remove=>: bRemoved = true
P00 DEBUG: Storage::Local->pathSync(): strPathExp = [TEST_PATH]/db-master/db/base-2
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: perl/exec::perlExec: => 0
P00 DEBUG: common/exit::exitSafe: (result: 0, error: false, signalType: 0)
P00 DEBUG: common/lock::lockRelease: (failOnNoLock: false)
P00 DEBUG: common/lock::lockRelease: => false
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
P00 DEBUG: Main::mainCleanup(): iExitCode = 0
P00 DEBUG: Protocol::Helper::protocolDestroy(): bComplete = true, iRemoteIdx = [undef], strRemoteType = [undef]
P00 DEBUG: Protocol::Helper::protocolDestroy=>: iExitStatus = 0
Added a predictable log entry to signal that a command has completed successfully. For example a backup ends successfully with: INFO: backup command end: completed successfully.
2016-12-04 21:44:53 +02:00
P00 INFO: restore command end: completed successfully
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 17:57:32 +02:00
P00 DEBUG: common/exit::exitSafe: => 0
P00 DEBUG: main::main: => 0
Log testing can now be enabled for certain deterministic tests. This works by comparing the generated logs against a previous copy. Currently only enabled for the backup/synthetic tests.
2015-05-05 19:08:48 +02:00
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
+ supplemental file: [TEST_PATH]/db-master/db/base-2/recovery.conf
------------------------------------------------------------------
Use path list in the backup manifest to do restore path syncs. Remove recursive path sync functionality since it is no longer used.
2018-05-01 17:05:37 +02:00
restore_command = '[BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --stanza=db archive-get %f "%p"'
Added vagrant configuration for Ubuntu 12.04. Changes to regression tests to work better with VMs.
2015-07-07 03:00:48 +02:00
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
restore delta, backup '[BACKUP-DIFF-2]', remap - ensure file in tblspc root remains after --delta (db-master host)
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --delta --set=[BACKUP-DIFF-2] --log-level-console=detail --stanza=db restore
Closed #58: Get catalog number for better tablespace copying When backing up and restoring tablespaces pgBackRest only operates on the subdirectory created for the version of PostgreSQL being run against. Since multiple versions can live in a tablespace (especially during a binary upgrade) this prevents too many files from being copied during a backup and other versions possibly being wiped out during a `--delta` restore. This only applies to PostgreSQL >= 9.0 -- before that only one PostgreSQL version could use a tablespace.
2016-02-02 21:33:15 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Add log-subprocess option to allow file logging for local and remote subprocesses.
2018-08-23 02:05:49 +02:00
P00 INFO: restore command begin [BACKREST-VERSION]: --no-compress --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --delta --lock-path=[TEST_PATH]/db-master/lock --log-level-console=detail --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --pg1-path=[TEST_PATH]/db-master/db/base-2 --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --set=[BACKUP-DIFF-2] --stanza=db --tablespace-map=1=[TEST_PATH]/db-master/db/tablespace/ts1-2 --tablespace-map=2=[TEST_PATH]/db-master/db/tablespace/ts2-2
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: restore backup set [BACKUP-DIFF-2]
P00 INFO: remap $PGDATA directory to [TEST_PATH]/db-master/db/base-2
P00 INFO: remap tablespace pg_tblspc/1 directory to [TEST_PATH]/db-master/db/tablespace/ts1-2
P00 INFO: remap tablespace pg_tblspc/2 directory to [TEST_PATH]/db-master/db/tablespace/ts2-2
P00 DETAIL: check [TEST_PATH]/db-master/db/base-2 exists
P00 DETAIL: check [TEST_PATH]/db-master/db/tablespace/ts1-2 exists
P00 DETAIL: check [TEST_PATH]/db-master/db/tablespace/ts2-2 exists
P00 INFO: remove invalid files/paths/links from [TEST_PATH]/db-master/db/tablespace/ts2-2/[TS_PATH-1]
P00 INFO: remove invalid files/paths/links from [TEST_PATH]/db-master/db/tablespace/ts1-2/[TS_PATH-1]
P00 INFO: remove invalid files/paths/links from [TEST_PATH]/db-master/db/base-2
P00 DETAIL: preserve file [TEST_PATH]/db-master/db/base-2/recovery.conf
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base-2/base/32768/33001 - exists and matches backup (64KB, 33%) checksum 6bf316f11d28c28914ea9be92c00de9bea6d9a6b
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base-2/base/32768/44000_init - exists and matches backup (32KB, 49%) checksum 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base-2/base/32768/33000.32767 - exists and matches backup (32KB, 66%) checksum 21e2c7c1a326682c07053b7d6a5a40dbd49c2ec5
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base-2/base/32768/33000 - exists and matches backup (32KB, 83%) checksum 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base-2/base/16384/17000 - exists and matches backup (16KB, 91%) checksum e0101dd8ffb910c9c202ca35b5f828bcb9697bed
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 11:24:42 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/global/pg_control.pgbackrest.tmp (8KB, 95%) checksum 4c77c900f7af0d9ab13fa9982051a42e0b637f6c
Add more realistic data files to synthetic backup and restore tests.
2016-12-10 16:13:55 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base-2/base/1/12000 - exists and matches backup (8KB, 99%) checksum 22c98d248ff548311eda88559e4a8405ed77c003
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base-2/postgresql.conf - exists and matches backup (21B, 99%) checksum 6721d92c9fcdf4248acff1f9a1377127d9064807
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base-2/badchecksum.txt - exists and matches backup (11B, 99%) checksum f927212cd08d11a42a666b2f04235398e9ceeb51
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base-2/changecontent.txt - exists and matches backup (7B, 99%) checksum a094d94583e209556d03c3c5da33131a065f1689
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base-2/pg_stat/global.stat - exists and matches backup (5B, 99%) checksum e350d5ce0153f3e22d5db21cf2a4eff00f3ee877
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base-2/changetime.txt - exists and matches backup (4B, 99%) checksum 88087292ed82e26f3eb824d0bffc05ccf7a30f8d
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base-2/base/32768/PG_VERSION - exists and matches backup (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base-2/base/16384/PG_VERSION - exists and matches backup (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base-2/base/1/PG_VERSION - exists and matches backup (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base-2/PG_VERSION - exists and matches backup (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
Add zero-length file to mock/all test.
2018-07-05 21:40:50 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base-2/zerosize.txt - exists and is zero size (0B, 99%)
Allow zero-size files in backup manifest to reference a prior manifest regardless of timestamp delta. Contributed by Cynthia Shang.
2018-08-24 22:50:33 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base-2/zero_from_start - exists and is zero size (0B, 99%)
Improve support for special characters in filenames. % characters caused issues in backup/restore due to filenames being appended directly into a format string. Reserved XML characters (<>&') caused issues in the S3 driver due to improper escaping. Add a file with all common special characters to regression testing.
2018-09-10 16:54:34 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base-2/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?% - exists and is zero size (0B, 99%)
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base-2/pg_tblspc/2/[TS_PATH-1]/32768/tablespace2.txt - exists and matches backup (7B, 99%) checksum dc7f76e43c46101b47acc55ae4d593a9e6983578
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base-2/pg_tblspc/1/[TS_PATH-1]/16384/tablespace1.txt - exists and matches backup (7B, 100%) checksum d85de07d6421d90aa9191c11c889bfde43680f0f
P00 INFO: write [TEST_PATH]/db-master/db/base-2/recovery.conf
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 INFO: restore global/pg_control (performed last to ensure aborted restores cannot be started)
Added a predictable log entry to signal that a command has completed successfully. For example a backup ends successfully with: INFO: backup command end: completed successfully.
2016-12-04 21:44:53 +02:00
P00 INFO: restore command end: completed successfully
Closed #58: Get catalog number for better tablespace copying When backing up and restoring tablespaces pgBackRest only operates on the subdirectory created for the version of PostgreSQL being run against. Since multiple versions can live in a tablespace (especially during a binary upgrade) this prevents too many files from being copied during a backup and other versions possibly being wiped out during a `--delta` restore. This only applies to PostgreSQL >= 9.0 -- before that only one PostgreSQL version could use a tablespace.
2016-02-02 21:33:15 +02:00
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
+ supplemental file: [TEST_PATH]/db-master/db/base-2/recovery.conf
------------------------------------------------------------------
restore_command = '[BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --log-level-console=detail --stanza=db archive-get %f "%p"'
Closed #58: Get catalog number for better tablespace copying When backing up and restoring tablespaces pgBackRest only operates on the subdirectory created for the version of PostgreSQL being run against. Since multiple versions can live in a tablespace (especially during a binary upgrade) this prevents too many files from being copied during a backup and other versions possibly being wiped out during a `--delta` restore. This only applies to PostgreSQL >= 9.0 -- before that only one PostgreSQL version could use a tablespace.
2016-02-02 21:33:15 +02:00
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
incr backup - add files and remove tablespace 2 (db-master host)
Reduce process redundancy in mock tests.
2017-06-22 00:47:25 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --no-online --log-level-console=detail --process-max=1 --stanza=db backup
Log testing can now be enabled for certain deterministic tests. This works by comparing the generated logs against a previous copy. Currently only enabled for the backup/synthetic tests.
2015-05-05 19:08:48 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
P00 INFO: backup command begin [BACKREST-VERSION]: --no-compress --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --db-timeout=45 --exclude=postgresql.auto.conf --exclude=pg_log/ --exclude=pg_log2 --lock-path=[TEST_PATH]/db-master/lock --log-level-console=detail --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --no-online --pg1-path=[TEST_PATH]/db-master/db/base-2 --process-max=1 --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --stanza=db --start-fast
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
P00 WARN: option repo1-retention-full is not set, the repository may run out of space
HINT: to retain full backups indefinitely (without warning), set option 'repo1-retention-full' to the maximum.
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: last backup label = [BACKUP-DIFF-2], version = [VERSION-1]
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
P00 WARN: incr backup cannot alter 'checksum-page' option to 'false', reset to 'true' from [BACKUP-DIFF-2]
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P01 INFO: backup file [TEST_PATH]/db-master/db/base-2/pg_tblspc/2/[TS_PATH-1]/32768/tablespace2b.txt (8B, 61%) checksum e324463005236d83e6e54795dbddd20a74533bf3
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
P00 WARN: page misalignment in file [TEST_PATH]/db-master/db/base-2/pg_tblspc/2/[TS_PATH-1]/32768/tablespace2b.txt: file size 8 is not divisible by page size 8192
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P01 INFO: backup file [TEST_PATH]/db-master/db/base-2/base/base2.txt (5B, 100%) checksum 09b5e31766be1dba1ec27de82f975c1b6eea2a92
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
P00 WARN: page misalignment in file [TEST_PATH]/db-master/db/base-2/base/base2.txt: file size 5 is not divisible by page size 8192
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: incr backup size = 13B
P00 INFO: new backup label = [BACKUP-INCR-3]
Added a predictable log entry to signal that a command has completed successfully. For example a backup ends successfully with: INFO: backup command end: completed successfully.
2016-12-04 21:44:53 +02:00
P00 INFO: backup command end: completed successfully
Move async forking and more error handling to C. The Perl process was exiting directly when called but that interfered with proper locking for the forked async process. Now Perl returns results to the C process which handles all errors, including signals.
2018-04-13 02:42:26 +02:00
P00 INFO: expire command begin
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
P00 INFO: option 'repo1-retention-archive' is not set - archive logs will not be expired
Added a predictable log entry to signal that a command has completed successfully. For example a backup ends successfully with: INFO: backup command end: completed successfully.
2016-12-04 21:44:53 +02:00
P00 INFO: expire command end: completed successfully
Log testing can now be enabled for certain deterministic tests. This works by comparing the generated logs against a previous copy. Currently only enabled for the backup/synthetic tests.
2015-05-05 19:08:48 +02:00
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
+ supplemental file: [TEST_PATH]/db-master/pgbackrest.conf
----------------------------------------------------------
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[db]
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
pg1-path=[TEST_PATH]/db-master/db/base-2
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
New simpler configuration and consistent project/exe/path naming. * The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required. * Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case. * The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally. * The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest. * Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory. * Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory. * Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 15:30:54 +02:00
[db:restore]
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
tablespace-map=1=[TEST_PATH]/db-master/db/tablespace/ts1-2
tablespace-map=2=[TEST_PATH]/db-master/db/tablespace/ts2-2
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
New simpler configuration and consistent project/exe/path naming. * The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required. * Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case. * The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally. * The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest. * Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory. * Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory. * Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 15:30:54 +02:00
[global]
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
compress=n
Fix custom settings for compress-level option being ignored. Reported by Jens Wilke.
2017-10-23 00:05:46 +02:00
compress-level=3
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
db-timeout=45
lock-path=[TEST_PATH]/db-master/lock
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
log-level-console=debug
log-level-file=trace
Added the log-level-stderr option. Controls whether console log messages are sent to stderr or stdout. By default this is set to warn which represents a change in behavior from previous versions, even though it may be more intuitive. Setting log-level-stderr=off will preserve the old behavior. Suggested by Sascha Biberhofer.
2016-10-05 15:09:30 +02:00
log-level-stderr=off
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
log-path=[TEST_PATH]/db-master/log
Add log-subprocess option to allow file logging for local and remote subprocesses.
2018-08-23 02:05:49 +02:00
log-subprocess=y
Set log-timestamp=n for integration tests. This means less filtering of logs needs to be done and new timestamps can be added without adding new filters.
2018-05-11 17:24:38 +02:00
log-timestamp=n
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
protocol-timeout=60
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
repo1-path=[TEST_PATH]/db-master/repo
Consolidate archive-push tests into new archive test.
2017-06-21 14:38:53 +02:00
spool-path=[TEST_PATH]/db-master/spool
New simpler configuration and consistent project/exe/path naming. * The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required. * Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case. * The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally. * The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest. * Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory. * Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory. * Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 15:30:54 +02:00
[global:backup]
archive-copy=y
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
exclude=postgresql.auto.conf
exclude=pg_log/
exclude=pg_log2
New simpler configuration and consistent project/exe/path naming. * The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required. * Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case. * The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally. * The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest. * Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory. * Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory. * Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 15:30:54 +02:00
start-fast=y
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
+ supplemental file: [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-3]/backup.manifest
-----------------------------------------------------------------------------------------
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[backrest]
backrest-checksum="[CHECKSUM]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
backrest-format=5
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
backrest-version="[VERSION-1]"
[backup]
backup-label="[BACKUP-INCR-3]"
backup-prior="[BACKUP-DIFF-2]"
backup-timestamp-copy-start=[TIMESTAMP]
backup-timestamp-start=[TIMESTAMP]
backup-timestamp-stop=[TIMESTAMP]
backup-type="incr"
[backup:db]
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-catalog-version=201409291
db-control-version=942
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
db-id=1
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
db-system-id=1000000000000000094
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-version="9.4"
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[backup:option]
option-archive-check=true
option-archive-copy=true
Backup from a standby cluster. A connection to the primary cluster is still required to start/stop the backup and copy files that are not replicated, but the vast majority of files are copied from the standby in order to reduce load on the master.
2016-08-25 17:25:46 +02:00
option-backup-standby=false
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
option-checksum-page=true
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
option-compress=false
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
option-delta=false
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
option-hardlink=false
Closed #167: Rename `--no-start-stop` option to `--no-online`.
2016-02-20 02:32:02 +02:00
option-online=false
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[backup:target]
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
pg_data={"path":"[TEST_PATH]/db-master/db/base-2","type":"path"}
pg_tblspc/2={"path":"[TEST_PATH]/db-master/db/tablespace/ts2-2","tablespace-id":"2","tablespace-name":"ts2","type":"link"}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:file]
Fixed a issue where tablespaces were copied from the master during standby backup.
2016-09-04 15:19:44 +02:00
pg_data/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/badchecksum.txt={"checksum":"f927212cd08d11a42a666b2f04235398e9ceeb51","master":true,"reference":"[BACKUP-DIFF-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
pg_data/base/1/12000={"checksum":"22c98d248ff548311eda88559e4a8405ed77c003","checksum-page":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
pg_data/base/1/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","mode":"0660","reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
pg_data/base/16384/17000={"checksum":"e0101dd8ffb910c9c202ca35b5f828bcb9697bed","checksum-page":false,"checksum-page-error":[1],"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
pg_data/base/16384/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Reduced the likelihood of torn pages causing a false positive in page checksums by filtering on start backup LSN.
2017-01-30 20:59:00 +02:00
pg_data/base/32768/33000={"checksum":"4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f","checksum-page":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Fixed an issue where newly initialized (but unused) pages would cause page checksum warnings. Reported by Stephen Frost.
2017-03-13 21:50:13 +02:00
pg_data/base/32768/33000.32767={"checksum":"21e2c7c1a326682c07053b7d6a5a40dbd49c2ec5","checksum-page":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
pg_data/base/32768/33001={"checksum":"6bf316f11d28c28914ea9be92c00de9bea6d9a6b","checksum-page":false,"checksum-page-error":[0,[3,5],7],"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
pg_data/base/32768/44000_init={"checksum":"4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f","checksum-page":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
pg_data/base/32768/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
pg_data/base/base2.txt={"checksum":"09b5e31766be1dba1ec27de82f975c1b6eea2a92","checksum-page":false,"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
pg_data/changecontent.txt={"checksum":"a094d94583e209556d03c3c5da33131a065f1689","master":true,"reference":"[BACKUP-DIFF-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/changetime.txt={"checksum":"88087292ed82e26f3eb824d0bffc05ccf7a30f8d","master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-2]}
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 11:24:42 +02:00
pg_data/global/pg_control={"checksum":"4c77c900f7af0d9ab13fa9982051a42e0b637f6c","master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-2]}
Fixed a issue where tablespaces were copied from the master during standby backup.
2016-09-04 15:19:44 +02:00
pg_data/pg_stat/global.stat={"checksum":"e350d5ce0153f3e22d5db21cf2a4eff00f3ee877","master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-2]}
pg_data/postgresql.conf={"checksum":"6721d92c9fcdf4248acff1f9a1377127d9064807","master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-2]}
Improve support for special characters in filenames. % characters caused issues in backup/restore due to filenames being appended directly into a format string. Reserved XML characters (<>&') caused issues in the S3 driver due to improper escaping. Add a file with all common special characters to regression testing.
2018-09-10 16:54:34 +02:00
pg_data/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?%={"master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Allow zero-size files in backup manifest to reference a prior manifest regardless of timestamp delta. Contributed by Cynthia Shang.
2018-08-24 22:50:33 +02:00
pg_data/zero_from_start={"master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Add zero-length file to mock/all test.
2018-07-05 21:40:50 +02:00
pg_data/zerosize.txt={"master":true,"reference":"[BACKUP-DIFF-2]","size":[SIZE],"timestamp":[TIMESTAMP-2]}
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
pg_tblspc/2/[TS_PATH-1]/32768/tablespace2.txt={"checksum":"dc7f76e43c46101b47acc55ae4d593a9e6983578","checksum-page":false,"reference":"[BACKUP-DIFF-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_tblspc/2/[TS_PATH-1]/32768/tablespace2b.txt={"checksum":"e324463005236d83e6e54795dbddd20a74533bf3","checksum-page":false,"size":[SIZE],"timestamp":[TIMESTAMP-1]}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:file:default]
Continuous integration using travis-ci.
2017-02-21 15:59:23 +02:00
group="[GROUP-1]"
Fixed a issue where tablespaces were copied from the master during standby backup.
2016-09-04 15:19:44 +02:00
master=false
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
mode="0600"
Regression tests can now be run as any properly-configured user, not just vagrant.
2016-10-01 19:39:44 +02:00
user="[USER-1]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:link]
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
pg_data/pg_tblspc/2={"destination":"[TEST_PATH]/db-master/db/tablespace/ts2-2"}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:link:default]
Continuous integration using travis-ci.
2017-02-21 15:59:23 +02:00
group="[GROUP-1]"
Regression tests can now be run as any properly-configured user, not just vagrant.
2016-10-01 19:39:44 +02:00
user="[USER-1]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:path]
pg_data={}
pg_data/base={}
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
pg_data/base/1={}
pg_data/base/16384={}
pg_data/base/32768={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_data/global={}
pg_data/pg_clog={}
Exclude directories during backup that are cleaned, recreated, or zeroed by PostgreSQL at startup. These include (depending on the version where they were introduced): pgsql_tmp, pg_dynshmem, pg_notify, pg_replslot, pg_serial, pg_snapshots, pg_stat_tmp, pg_subtrans. The postgresql.auto.conf.tmp file is now excluded in addition to files that were already excluded: backup_label.old, postmaster.opts, postmaster.pid, recovery.conf, recovery.done.
2016-08-16 15:35:16 +02:00
pg_data/pg_dynshmem={}
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
pg_data/pg_log={}
Exclude directories during backup that are cleaned, recreated, or zeroed by PostgreSQL at startup. These include (depending on the version where they were introduced): pgsql_tmp, pg_dynshmem, pg_notify, pg_replslot, pg_serial, pg_snapshots, pg_stat_tmp, pg_subtrans. The postgresql.auto.conf.tmp file is now excluded in addition to files that were already excluded: backup_label.old, postmaster.opts, postmaster.pid, recovery.conf, recovery.done.
2016-08-16 15:35:16 +02:00
pg_data/pg_notify={}
pg_data/pg_replslot={}
pg_data/pg_serial={}
pg_data/pg_snapshots={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_data/pg_stat={}
Exclude directories during backup that are cleaned, recreated, or zeroed by PostgreSQL at startup. These include (depending on the version where they were introduced): pgsql_tmp, pg_dynshmem, pg_notify, pg_replslot, pg_serial, pg_snapshots, pg_stat_tmp, pg_subtrans. The postgresql.auto.conf.tmp file is now excluded in addition to files that were already excluded: backup_label.old, postmaster.opts, postmaster.pid, recovery.conf, recovery.done.
2016-08-16 15:35:16 +02:00
pg_data/pg_stat_tmp={}
pg_data/pg_subtrans={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_data/pg_tblspc={}
pg_tblspc={}
pg_tblspc/2={}
pg_tblspc/2/[TS_PATH-1]={}
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
pg_tblspc/2/[TS_PATH-1]/32768={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:path:default]
Continuous integration using travis-ci.
2017-02-21 15:59:23 +02:00
group="[GROUP-1]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
mode="0700"
Regression tests can now be run as any properly-configured user, not just vagrant.
2016-10-01 19:39:44 +02:00
user="[USER-1]"
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
+ supplemental file: [TEST_PATH]/db-master/repo/backup/db/backup.info
---------------------------------------------------------------------
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[backrest]
backrest-checksum="[CHECKSUM]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
backrest-format=5
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
backrest-version="[VERSION-1]"
[backup:current]
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
[BACKUP-FULL-2]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"full","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":true,"option-compress":false,"option-hardlink":false,"option-online":false}
[BACKUP-DIFF-2]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-prior":"[BACKUP-FULL-2]","backup-reference":["[BACKUP-FULL-2]"],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"diff","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":true,"option-compress":false,"option-hardlink":false,"option-online":false}
[BACKUP-INCR-3]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-prior":"[BACKUP-DIFF-2]","backup-reference":["[BACKUP-FULL-2]","[BACKUP-DIFF-2]"],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"incr","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":true,"option-compress":false,"option-hardlink":false,"option-online":false}
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[db]
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-catalog-version=201409291
db-control-version=942
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
db-id=1
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
db-system-id=1000000000000000094
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-version="9.4"
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[db:history]
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
1={"db-catalog-version":201409291,"db-control-version":942,"db-system-id":1000000000000000094,"db-version":"9.4"}
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
Improved stanza-create command so that it can repair broken repositories in most cases and is robust enough to be made mandatory. Contributed by Cynthia Shang.
2016-12-20 23:52:20 +02:00
incr backup - update files - fail on missing backup.info (db-master host)
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --no-online --log-level-console=detail --stanza=db backup
------------------------------------------------------------------------------------------------------------------------------------
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
P00 INFO: backup command begin [BACKREST-VERSION]: --no-compress --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --db-timeout=45 --exclude=postgresql.auto.conf --exclude=pg_log/ --exclude=pg_log2 --lock-path=[TEST_PATH]/db-master/lock --log-level-console=detail --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --no-online --pg1-path=[TEST_PATH]/db-master/db/base-2 --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --stanza=db --start-fast
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
P00 WARN: option repo1-retention-full is not set, the repository may run out of space
HINT: to retain full backups indefinitely (without warning), set option 'repo1-retention-full' to the maximum.
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 ERROR: [055]: unable to open [TEST_PATH]/db-master/repo/backup/db/backup.info or [TEST_PATH]/db-master/repo/backup/db/backup.info.copy
Renumber return codes between 25 and 125 to avoid PostgreSQL interpreting some as fatal signal exceptions. Suggested by Yogesh Sharma.
2017-04-12 18:41:34 +02:00
P00 ERROR: [055]: [TEST_PATH]/db-master/repo/backup/db/backup.info does not exist and is required to perform a backup.
Fixed alignment issues with multiline logging.
2017-02-06 02:58:52 +02:00
HINT: has a stanza-create been performed?
Renumber return codes between 25 and 125 to avoid PostgreSQL interpreting some as fatal signal exceptions. Suggested by Yogesh Sharma.
2017-04-12 18:41:34 +02:00
P00 INFO: backup command end: aborted with exception [055]
Improved stanza-create command so that it can repair broken repositories in most cases and is robust enough to be made mandatory. Contributed by Cynthia Shang.
2016-12-20 23:52:20 +02:00
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
stanza-create db - fail on backup directory missing backup.info (db-master host)
Improved stanza-create command so that it can repair broken repositories in most cases and is robust enough to be made mandatory. Contributed by Cynthia Shang.
2016-12-20 23:52:20 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --stanza=db --log-level-console=detail --no-online stanza-create
------------------------------------------------------------------------------------------------------------------------------------
Add log-subprocess option to allow file logging for local and remote subprocesses.
2018-08-23 02:05:49 +02:00
P00 INFO: stanza-create command begin [BACKREST-VERSION]: --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --db-timeout=45 --lock-path=[TEST_PATH]/db-master/lock --log-level-console=detail --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --no-online --pg1-path=[TEST_PATH]/db-master/db/base-2 --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --stanza=db
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 ERROR: [055]: backup information missing
Improve stanza-create command so that it does not error when the stanza already exists. Contributed by Cynthia Shang.
2018-02-01 21:12:03 +02:00
HINT: use stanza-create --force to force the stanza data to be recreated.
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 INFO: stanza-create command end: aborted with exception [055]
Improved stanza-create command so that it can repair broken repositories in most cases and is robust enough to be made mandatory. Contributed by Cynthia Shang.
2016-12-20 23:52:20 +02:00
+ supplemental file: [TEST_PATH]/db-master/repo/archive/db/archive.info
-----------------------------------------------------------------------
[backrest]
backrest-checksum="[CHECKSUM]"
backrest-format=5
backrest-version="[VERSION-1]"
[db]
db-id=1
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
db-system-id=1000000000000000094
Improved stanza-create command so that it can repair broken repositories in most cases and is robust enough to be made mandatory. Contributed by Cynthia Shang.
2016-12-20 23:52:20 +02:00
db-version="9.4"
[db:history]
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
1={"db-id":1000000000000000094,"db-version":"9.4"}
Improved stanza-create command so that it can repair broken repositories in most cases and is robust enough to be made mandatory. Contributed by Cynthia Shang.
2016-12-20 23:52:20 +02:00
stanza-create db - create required data for stanza (db-master host)
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --stanza=db --log-level-console=detail --no-online --force stanza-create
------------------------------------------------------------------------------------------------------------------------------------
Add log-subprocess option to allow file logging for local and remote subprocesses.
2018-08-23 02:05:49 +02:00
P00 INFO: stanza-create command begin [BACKREST-VERSION]: --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --db-timeout=45 --force --lock-path=[TEST_PATH]/db-master/lock --log-level-console=detail --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --no-online --pg1-path=[TEST_PATH]/db-master/db/base-2 --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --stanza=db
Improved stanza-create command so that it can repair broken repositories in most cases and is robust enough to be made mandatory. Contributed by Cynthia Shang.
2016-12-20 23:52:20 +02:00
P00 INFO: stanza-create command end: completed successfully
+ supplemental file: [TEST_PATH]/db-master/repo/backup/db/backup.info
---------------------------------------------------------------------
[backrest]
backrest-checksum="[CHECKSUM]"
backrest-format=5
backrest-version="[VERSION-1]"
[backup:current]
[BACKUP-FULL-2]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"full","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":true,"option-compress":false,"option-hardlink":false,"option-online":false}
[BACKUP-DIFF-2]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-prior":"[BACKUP-FULL-2]","backup-reference":["[BACKUP-FULL-2]"],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"diff","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":true,"option-compress":false,"option-hardlink":false,"option-online":false}
[BACKUP-INCR-3]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-prior":"[BACKUP-DIFF-2]","backup-reference":["[BACKUP-FULL-2]","[BACKUP-DIFF-2]"],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"incr","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":true,"option-compress":false,"option-hardlink":false,"option-online":false}
[db]
db-catalog-version=201409291
db-control-version=942
db-id=1
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
db-system-id=1000000000000000094
Improved stanza-create command so that it can repair broken repositories in most cases and is robust enough to be made mandatory. Contributed by Cynthia Shang.
2016-12-20 23:52:20 +02:00
db-version="9.4"
[db:history]
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
1={"db-catalog-version":201409291,"db-control-version":942,"db-system-id":1000000000000000094,"db-version":"9.4"}
Improved stanza-create command so that it can repair broken repositories in most cases and is robust enough to be made mandatory. Contributed by Cynthia Shang.
2016-12-20 23:52:20 +02:00
+ supplemental file: [TEST_PATH]/db-master/repo/archive/db/archive.info
-----------------------------------------------------------------------
[backrest]
backrest-checksum="[CHECKSUM]"
backrest-format=5
backrest-version="[VERSION-1]"
[db]
db-id=1
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
db-system-id=1000000000000000094
Improved stanza-create command so that it can repair broken repositories in most cases and is robust enough to be made mandatory. Contributed by Cynthia Shang.
2016-12-20 23:52:20 +02:00
db-version="9.4"
[db:history]
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
1={"db-id":1000000000000000094,"db-version":"9.4"}
Improved stanza-create command so that it can repair broken repositories in most cases and is robust enough to be made mandatory. Contributed by Cynthia Shang.
2016-12-20 23:52:20 +02:00
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
incr backup - update files (db-master host)
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --no-online --log-level-console=detail --stanza=db backup
Log testing can now be enabled for certain deterministic tests. This works by comparing the generated logs against a previous copy. Currently only enabled for the backup/synthetic tests.
2015-05-05 19:08:48 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
P00 INFO: backup command begin [BACKREST-VERSION]: --no-compress --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --db-timeout=45 --exclude=postgresql.auto.conf --exclude=pg_log/ --exclude=pg_log2 --lock-path=[TEST_PATH]/db-master/lock --log-level-console=detail --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --no-online --pg1-path=[TEST_PATH]/db-master/db/base-2 --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --stanza=db --start-fast
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
P00 WARN: option repo1-retention-full is not set, the repository may run out of space
HINT: to retain full backups indefinitely (without warning), set option 'repo1-retention-full' to the maximum.
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: last backup label = [BACKUP-INCR-3], version = [VERSION-1]
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
P00 WARN: incr backup cannot alter 'checksum-page' option to 'false', reset to 'true' from [BACKUP-INCR-3]
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P00 WARN: file pg_data/base/16384/17000 timestamp in the past or size changed but timestamp did not, enabling delta checksum
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base-2/base/32768/33001 (64KB, 36%) checksum 6bf316f11d28c28914ea9be92c00de9bea6d9a6b
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base-2/base/32768/44000_init (32KB, 54%) checksum 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base-2/base/32768/33000.32767 (32KB, 72%) checksum 21e2c7c1a326682c07053b7d6a5a40dbd49c2ec5
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base-2/base/32768/33000 (32KB, 90%) checksum 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base-2/global/pg_control (8KB, 95%) checksum 4c77c900f7af0d9ab13fa9982051a42e0b637f6c
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base-2/base/1/12000 (8KB, 99%) checksum 22c98d248ff548311eda88559e4a8405ed77c003
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base-2/postgresql.conf (21B, 99%) checksum 6721d92c9fcdf4248acff1f9a1377127d9064807
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base-2/badchecksum.txt (11B, 99%) checksum f927212cd08d11a42a666b2f04235398e9ceeb51
P01 INFO: backup file [TEST_PATH]/db-master/db/base-2/base/16384/17000 (8B, 99%) checksum 9a53d532e27785e681766c98516a5e93f096a501
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
P00 WARN: page misalignment in file [TEST_PATH]/db-master/db/base-2/base/16384/17000: file size 8 is not divisible by page size 8192
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base-2/changecontent.txt (7B, 99%) checksum a094d94583e209556d03c3c5da33131a065f1689
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base-2/pg_stat/global.stat (5B, 99%) checksum e350d5ce0153f3e22d5db21cf2a4eff00f3ee877
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base-2/base/base2.txt (5B, 99%) checksum 09b5e31766be1dba1ec27de82f975c1b6eea2a92
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base-2/changetime.txt (4B, 99%) checksum 88087292ed82e26f3eb824d0bffc05ccf7a30f8d
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base-2/base/32768/PG_VERSION (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base-2/base/16384/PG_VERSION (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base-2/base/1/PG_VERSION (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base-2/PG_VERSION (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base-2/pg_tblspc/2/[TS_PATH-1]/32768/tablespace2b.txt (8B, 99%) checksum e324463005236d83e6e54795dbddd20a74533bf3
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base-2/pg_tblspc/2/[TS_PATH-1]/32768/tablespace2.txt (7B, 100%) checksum dc7f76e43c46101b47acc55ae4d593a9e6983578
P00 INFO: incr backup size = 176KB
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: new backup label = [BACKUP-INCR-4]
Added a predictable log entry to signal that a command has completed successfully. For example a backup ends successfully with: INFO: backup command end: completed successfully.
2016-12-04 21:44:53 +02:00
P00 INFO: backup command end: completed successfully
Move async forking and more error handling to C. The Perl process was exiting directly when called but that interfered with proper locking for the forked async process. Now Perl returns results to the C process which handles all errors, including signals.
2018-04-13 02:42:26 +02:00
P00 INFO: expire command begin
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
P00 INFO: option 'repo1-retention-archive' is not set - archive logs will not be expired
Added a predictable log entry to signal that a command has completed successfully. For example a backup ends successfully with: INFO: backup command end: completed successfully.
2016-12-04 21:44:53 +02:00
P00 INFO: expire command end: completed successfully
Log testing can now be enabled for certain deterministic tests. This works by comparing the generated logs against a previous copy. Currently only enabled for the backup/synthetic tests.
2015-05-05 19:08:48 +02:00
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
+ supplemental file: [TEST_PATH]/db-master/pgbackrest.conf
----------------------------------------------------------
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[db]
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
pg1-path=[TEST_PATH]/db-master/db/base-2
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
New simpler configuration and consistent project/exe/path naming. * The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required. * Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case. * The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally. * The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest. * Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory. * Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory. * Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 15:30:54 +02:00
[db:restore]
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
tablespace-map=1=[TEST_PATH]/db-master/db/tablespace/ts1-2
tablespace-map=2=[TEST_PATH]/db-master/db/tablespace/ts2-2
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
New simpler configuration and consistent project/exe/path naming. * The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required. * Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case. * The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally. * The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest. * Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory. * Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory. * Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 15:30:54 +02:00
[global]
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
compress=n
Fix custom settings for compress-level option being ignored. Reported by Jens Wilke.
2017-10-23 00:05:46 +02:00
compress-level=3
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
db-timeout=45
lock-path=[TEST_PATH]/db-master/lock
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
log-level-console=debug
log-level-file=trace
Added the log-level-stderr option. Controls whether console log messages are sent to stderr or stdout. By default this is set to warn which represents a change in behavior from previous versions, even though it may be more intuitive. Setting log-level-stderr=off will preserve the old behavior. Suggested by Sascha Biberhofer.
2016-10-05 15:09:30 +02:00
log-level-stderr=off
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
log-path=[TEST_PATH]/db-master/log
Add log-subprocess option to allow file logging for local and remote subprocesses.
2018-08-23 02:05:49 +02:00
log-subprocess=y
Set log-timestamp=n for integration tests. This means less filtering of logs needs to be done and new timestamps can be added without adding new filters.
2018-05-11 17:24:38 +02:00
log-timestamp=n
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
protocol-timeout=60
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
repo1-path=[TEST_PATH]/db-master/repo
Consolidate archive-push tests into new archive test.
2017-06-21 14:38:53 +02:00
spool-path=[TEST_PATH]/db-master/spool
New simpler configuration and consistent project/exe/path naming. * The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required. * Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case. * The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally. * The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest. * Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory. * Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory. * Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 15:30:54 +02:00
[global:backup]
archive-copy=y
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
exclude=postgresql.auto.conf
exclude=pg_log/
exclude=pg_log2
New simpler configuration and consistent project/exe/path naming. * The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required. * Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case. * The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally. * The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest. * Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory. * Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory. * Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 15:30:54 +02:00
start-fast=y
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
+ supplemental file: [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-4]/backup.manifest
-----------------------------------------------------------------------------------------
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[backrest]
backrest-checksum="[CHECKSUM]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
backrest-format=5
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
backrest-version="[VERSION-1]"
[backup]
backup-label="[BACKUP-INCR-4]"
backup-prior="[BACKUP-INCR-3]"
backup-timestamp-copy-start=[TIMESTAMP]
backup-timestamp-start=[TIMESTAMP]
backup-timestamp-stop=[TIMESTAMP]
backup-type="incr"
[backup:db]
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-catalog-version=201409291
db-control-version=942
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
db-id=1
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
db-system-id=1000000000000000094
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-version="9.4"
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[backup:option]
option-archive-check=true
option-archive-copy=true
Backup from a standby cluster. A connection to the primary cluster is still required to start/stop the backup and copy files that are not replicated, but the vast majority of files are copied from the standby in order to reduce load on the master.
2016-08-25 17:25:46 +02:00
option-backup-standby=false
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
option-checksum-page=true
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
option-compress=false
Automatically enable backup checksum delta when anomalies (e.g. timeline switch) are detected. There are a number of cases where a checksum delta is more appropriate than the default time-based delta: * Timeline has switched since the prior backup * File timestamp is older than recorded in the prior backup * File size changed but timestamp did not * File timestamp is in the future compared to the start of the backup * Online option has changed since the prior backup A practical example is that checksum delta will be enabled after a failover to standby due to the timeline switch. In this case, timestamps can't be trusted and our recommendation has been to run a full backup, which can impact the retention schedule and requires manual intervention. Now, a checksum delta will be performed if the backup type is incr/diff. This means more CPU will be used during the backup but the backup size will be smaller and the retention schedule will not be impacted. Contributed by Cynthia Shang.
2018-11-01 17:31:25 +02:00
option-delta=true
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
option-hardlink=false
Closed #167: Rename `--no-start-stop` option to `--no-online`.
2016-02-20 02:32:02 +02:00
option-online=false
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[backup:target]
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
pg_data={"path":"[TEST_PATH]/db-master/db/base-2","type":"path"}
pg_tblspc/2={"path":"[TEST_PATH]/db-master/db/tablespace/ts2-2","tablespace-id":"2","tablespace-name":"ts2","type":"link"}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:file]
Fixed a issue where tablespaces were copied from the master during standby backup.
2016-09-04 15:19:44 +02:00
pg_data/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/badchecksum.txt={"checksum":"f927212cd08d11a42a666b2f04235398e9ceeb51","master":true,"reference":"[BACKUP-DIFF-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
pg_data/base/1/12000={"checksum":"22c98d248ff548311eda88559e4a8405ed77c003","checksum-page":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
pg_data/base/1/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","mode":"0660","reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
pg_data/base/16384/17000={"checksum":"9a53d532e27785e681766c98516a5e93f096a501","checksum-page":false,"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
pg_data/base/16384/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Reduced the likelihood of torn pages causing a false positive in page checksums by filtering on start backup LSN.
2017-01-30 20:59:00 +02:00
pg_data/base/32768/33000={"checksum":"4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f","checksum-page":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Fixed an issue where newly initialized (but unused) pages would cause page checksum warnings. Reported by Stephen Frost.
2017-03-13 21:50:13 +02:00
pg_data/base/32768/33000.32767={"checksum":"21e2c7c1a326682c07053b7d6a5a40dbd49c2ec5","checksum-page":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
pg_data/base/32768/33001={"checksum":"6bf316f11d28c28914ea9be92c00de9bea6d9a6b","checksum-page":false,"checksum-page-error":[0,[3,5],7],"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
pg_data/base/32768/44000_init={"checksum":"4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f","checksum-page":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
pg_data/base/32768/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
pg_data/base/base2.txt={"checksum":"09b5e31766be1dba1ec27de82f975c1b6eea2a92","checksum-page":false,"reference":"[BACKUP-INCR-3]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
pg_data/changecontent.txt={"checksum":"a094d94583e209556d03c3c5da33131a065f1689","master":true,"reference":"[BACKUP-DIFF-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/changetime.txt={"checksum":"88087292ed82e26f3eb824d0bffc05ccf7a30f8d","master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-2]}
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 11:24:42 +02:00
pg_data/global/pg_control={"checksum":"4c77c900f7af0d9ab13fa9982051a42e0b637f6c","master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-2]}
Fixed a issue where tablespaces were copied from the master during standby backup.
2016-09-04 15:19:44 +02:00
pg_data/pg_stat/global.stat={"checksum":"e350d5ce0153f3e22d5db21cf2a4eff00f3ee877","master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-2]}
pg_data/postgresql.conf={"checksum":"6721d92c9fcdf4248acff1f9a1377127d9064807","master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-2]}
Improve support for special characters in filenames. % characters caused issues in backup/restore due to filenames being appended directly into a format string. Reserved XML characters (<>&') caused issues in the S3 driver due to improper escaping. Add a file with all common special characters to regression testing.
2018-09-10 16:54:34 +02:00
pg_data/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?%={"master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Allow zero-size files in backup manifest to reference a prior manifest regardless of timestamp delta. Contributed by Cynthia Shang.
2018-08-24 22:50:33 +02:00
pg_data/zero_from_start={"master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Add zero-length file to mock/all test.
2018-07-05 21:40:50 +02:00
pg_data/zerosize.txt={"master":true,"reference":"[BACKUP-DIFF-2]","size":[SIZE],"timestamp":[TIMESTAMP-2]}
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
pg_tblspc/2/[TS_PATH-1]/32768/tablespace2.txt={"checksum":"dc7f76e43c46101b47acc55ae4d593a9e6983578","checksum-page":false,"reference":"[BACKUP-DIFF-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_tblspc/2/[TS_PATH-1]/32768/tablespace2b.txt={"checksum":"e324463005236d83e6e54795dbddd20a74533bf3","checksum-page":false,"reference":"[BACKUP-INCR-3]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:file:default]
Continuous integration using travis-ci.
2017-02-21 15:59:23 +02:00
group="[GROUP-1]"
Fixed a issue where tablespaces were copied from the master during standby backup.
2016-09-04 15:19:44 +02:00
master=false
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
mode="0600"
Regression tests can now be run as any properly-configured user, not just vagrant.
2016-10-01 19:39:44 +02:00
user="[USER-1]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:link]
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
pg_data/pg_tblspc/2={"destination":"[TEST_PATH]/db-master/db/tablespace/ts2-2"}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:link:default]
Continuous integration using travis-ci.
2017-02-21 15:59:23 +02:00
group="[GROUP-1]"
Regression tests can now be run as any properly-configured user, not just vagrant.
2016-10-01 19:39:44 +02:00
user="[USER-1]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:path]
pg_data={}
pg_data/base={}
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
pg_data/base/1={}
pg_data/base/16384={}
pg_data/base/32768={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_data/global={}
pg_data/pg_clog={}
Exclude directories during backup that are cleaned, recreated, or zeroed by PostgreSQL at startup. These include (depending on the version where they were introduced): pgsql_tmp, pg_dynshmem, pg_notify, pg_replslot, pg_serial, pg_snapshots, pg_stat_tmp, pg_subtrans. The postgresql.auto.conf.tmp file is now excluded in addition to files that were already excluded: backup_label.old, postmaster.opts, postmaster.pid, recovery.conf, recovery.done.
2016-08-16 15:35:16 +02:00
pg_data/pg_dynshmem={}
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
pg_data/pg_log={}
Exclude directories during backup that are cleaned, recreated, or zeroed by PostgreSQL at startup. These include (depending on the version where they were introduced): pgsql_tmp, pg_dynshmem, pg_notify, pg_replslot, pg_serial, pg_snapshots, pg_stat_tmp, pg_subtrans. The postgresql.auto.conf.tmp file is now excluded in addition to files that were already excluded: backup_label.old, postmaster.opts, postmaster.pid, recovery.conf, recovery.done.
2016-08-16 15:35:16 +02:00
pg_data/pg_notify={}
pg_data/pg_replslot={}
pg_data/pg_serial={}
pg_data/pg_snapshots={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_data/pg_stat={}
Exclude directories during backup that are cleaned, recreated, or zeroed by PostgreSQL at startup. These include (depending on the version where they were introduced): pgsql_tmp, pg_dynshmem, pg_notify, pg_replslot, pg_serial, pg_snapshots, pg_stat_tmp, pg_subtrans. The postgresql.auto.conf.tmp file is now excluded in addition to files that were already excluded: backup_label.old, postmaster.opts, postmaster.pid, recovery.conf, recovery.done.
2016-08-16 15:35:16 +02:00
pg_data/pg_stat_tmp={}
pg_data/pg_subtrans={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_data/pg_tblspc={}
pg_tblspc={}
pg_tblspc/2={}
pg_tblspc/2/[TS_PATH-1]={}
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
pg_tblspc/2/[TS_PATH-1]/32768={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:path:default]
Continuous integration using travis-ci.
2017-02-21 15:59:23 +02:00
group="[GROUP-1]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
mode="0700"
Regression tests can now be run as any properly-configured user, not just vagrant.
2016-10-01 19:39:44 +02:00
user="[USER-1]"
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
+ supplemental file: [TEST_PATH]/db-master/repo/backup/db/backup.info
---------------------------------------------------------------------
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[backrest]
backrest-checksum="[CHECKSUM]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
backrest-format=5
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
backrest-version="[VERSION-1]"
[backup:current]
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
[BACKUP-FULL-2]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"full","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":true,"option-compress":false,"option-hardlink":false,"option-online":false}
[BACKUP-DIFF-2]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-prior":"[BACKUP-FULL-2]","backup-reference":["[BACKUP-FULL-2]"],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"diff","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":true,"option-compress":false,"option-hardlink":false,"option-online":false}
[BACKUP-INCR-3]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-prior":"[BACKUP-DIFF-2]","backup-reference":["[BACKUP-FULL-2]","[BACKUP-DIFF-2]"],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"incr","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":true,"option-compress":false,"option-hardlink":false,"option-online":false}
[BACKUP-INCR-4]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-prior":"[BACKUP-INCR-3]","backup-reference":["[BACKUP-FULL-2]","[BACKUP-DIFF-2]","[BACKUP-INCR-3]"],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"incr","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":true,"option-compress":false,"option-hardlink":false,"option-online":false}
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[db]
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-catalog-version=201409291
db-control-version=942
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
db-id=1
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
db-system-id=1000000000000000094
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-version="9.4"
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[db:history]
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
1={"db-catalog-version":201409291,"db-control-version":942,"db-system-id":1000000000000000094,"db-version":"9.4"}
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
diff backup - updates since last full (db-master host)
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --no-online --log-level-console=detail --process-max=1 --delta --type=diff --stanza=db backup
Log testing can now be enabled for certain deterministic tests. This works by comparing the generated logs against a previous copy. Currently only enabled for the backup/synthetic tests.
2015-05-05 19:08:48 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P00 INFO: backup command begin [BACKREST-VERSION]: --no-compress --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --db-timeout=45 --delta --exclude=postgresql.auto.conf --exclude=pg_log/ --exclude=pg_log2 --lock-path=[TEST_PATH]/db-master/lock --log-level-console=detail --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --no-online --pg1-path=[TEST_PATH]/db-master/db/base-2 --process-max=1 --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --stanza=db --start-fast --type=diff
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
P00 WARN: option repo1-retention-full is not set, the repository may run out of space
HINT: to retain full backups indefinitely (without warning), set option 'repo1-retention-full' to the maximum.
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: last backup label = [BACKUP-FULL-2], version = [VERSION-1]
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
P00 WARN: diff backup cannot alter 'checksum-page' option to 'false', reset to 'true' from [BACKUP-FULL-2]
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base-2/base/32768/33001 (64KB, 36%) checksum 6bf316f11d28c28914ea9be92c00de9bea6d9a6b
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base-2/base/32768/44000_init (32KB, 54%) checksum 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base-2/base/32768/33000.32767 (32KB, 72%) checksum 21e2c7c1a326682c07053b7d6a5a40dbd49c2ec5
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base-2/base/32768/33000 (32KB, 90%) checksum 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 11:24:42 +02:00
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base-2/global/pg_control (8KB, 95%) checksum 4c77c900f7af0d9ab13fa9982051a42e0b637f6c
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base-2/base/1/12000 (8KB, 99%) checksum 22c98d248ff548311eda88559e4a8405ed77c003
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base-2/postgresql.conf (21B, 99%) checksum 6721d92c9fcdf4248acff1f9a1377127d9064807
P01 INFO: backup file [TEST_PATH]/db-master/db/base-2/badchecksum.txt (11B, 99%) checksum f927212cd08d11a42a666b2f04235398e9ceeb51
P01 INFO: backup file [TEST_PATH]/db-master/db/base-2/base/16384/17000 (8B, 99%) checksum 9a53d532e27785e681766c98516a5e93f096a501
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
P00 WARN: page misalignment in file [TEST_PATH]/db-master/db/base-2/base/16384/17000: file size 8 is not divisible by page size 8192
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P01 INFO: backup file [TEST_PATH]/db-master/db/base-2/changecontent.txt (7B, 99%) checksum a094d94583e209556d03c3c5da33131a065f1689
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base-2/pg_stat/global.stat (5B, 99%) checksum e350d5ce0153f3e22d5db21cf2a4eff00f3ee877
P01 INFO: backup file [TEST_PATH]/db-master/db/base-2/base/base2.txt (5B, 99%) checksum 09b5e31766be1dba1ec27de82f975c1b6eea2a92
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
P00 WARN: page misalignment in file [TEST_PATH]/db-master/db/base-2/base/base2.txt: file size 5 is not divisible by page size 8192
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base-2/changetime.txt (4B, 99%) checksum 88087292ed82e26f3eb824d0bffc05ccf7a30f8d
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base-2/base/32768/PG_VERSION (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base-2/base/16384/PG_VERSION (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base-2/base/1/PG_VERSION (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base-2/PG_VERSION (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P01 INFO: backup file [TEST_PATH]/db-master/db/base-2/zerosize.txt (0B, 99%)
P01 INFO: backup file [TEST_PATH]/db-master/db/base-2/pg_tblspc/2/[TS_PATH-1]/32768/tablespace2b.txt (8B, 99%) checksum e324463005236d83e6e54795dbddd20a74533bf3
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
P00 WARN: page misalignment in file [TEST_PATH]/db-master/db/base-2/pg_tblspc/2/[TS_PATH-1]/32768/tablespace2b.txt: file size 8 is not divisible by page size 8192
Additional backup exclusions. * Exclude contents of pg_snapshots, pg_serial, pg_notify, and pg_dynshmem from backup since they are rebuilt on startup. * Exclude pg_internal.init files from backup since they are rebuilt on startup.
2017-09-04 14:26:57 +02:00
P01 INFO: backup file [TEST_PATH]/db-master/db/base-2/pg_tblspc/2/[TS_PATH-1]/32768/tablespace2.txt (7B, 100%) checksum dc7f76e43c46101b47acc55ae4d593a9e6983578
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
P00 WARN: page misalignment in file [TEST_PATH]/db-master/db/base-2/pg_tblspc/2/[TS_PATH-1]/32768/tablespace2.txt: file size 7 is not divisible by page size 8192
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P00 INFO: diff backup size = 176KB
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: new backup label = [BACKUP-DIFF-3]
Added a predictable log entry to signal that a command has completed successfully. For example a backup ends successfully with: INFO: backup command end: completed successfully.
2016-12-04 21:44:53 +02:00
P00 INFO: backup command end: completed successfully
Move async forking and more error handling to C. The Perl process was exiting directly when called but that interfered with proper locking for the forked async process. Now Perl returns results to the C process which handles all errors, including signals.
2018-04-13 02:42:26 +02:00
P00 INFO: expire command begin
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
P00 INFO: option 'repo1-retention-archive' is not set - archive logs will not be expired
Added a predictable log entry to signal that a command has completed successfully. For example a backup ends successfully with: INFO: backup command end: completed successfully.
2016-12-04 21:44:53 +02:00
P00 INFO: expire command end: completed successfully
Log testing can now be enabled for certain deterministic tests. This works by comparing the generated logs against a previous copy. Currently only enabled for the backup/synthetic tests.
2015-05-05 19:08:48 +02:00
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
+ supplemental file: [TEST_PATH]/db-master/pgbackrest.conf
----------------------------------------------------------
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[db]
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
pg1-path=[TEST_PATH]/db-master/db/base-2
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
New simpler configuration and consistent project/exe/path naming. * The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required. * Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case. * The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally. * The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest. * Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory. * Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory. * Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 15:30:54 +02:00
[db:restore]
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
tablespace-map=1=[TEST_PATH]/db-master/db/tablespace/ts1-2
tablespace-map=2=[TEST_PATH]/db-master/db/tablespace/ts2-2
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
New simpler configuration and consistent project/exe/path naming. * The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required. * Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case. * The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally. * The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest. * Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory. * Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory. * Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 15:30:54 +02:00
[global]
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
compress=n
Fix custom settings for compress-level option being ignored. Reported by Jens Wilke.
2017-10-23 00:05:46 +02:00
compress-level=3
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
db-timeout=45
lock-path=[TEST_PATH]/db-master/lock
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
log-level-console=debug
log-level-file=trace
Added the log-level-stderr option. Controls whether console log messages are sent to stderr or stdout. By default this is set to warn which represents a change in behavior from previous versions, even though it may be more intuitive. Setting log-level-stderr=off will preserve the old behavior. Suggested by Sascha Biberhofer.
2016-10-05 15:09:30 +02:00
log-level-stderr=off
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
log-path=[TEST_PATH]/db-master/log
Add log-subprocess option to allow file logging for local and remote subprocesses.
2018-08-23 02:05:49 +02:00
log-subprocess=y
Set log-timestamp=n for integration tests. This means less filtering of logs needs to be done and new timestamps can be added without adding new filters.
2018-05-11 17:24:38 +02:00
log-timestamp=n
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
protocol-timeout=60
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
repo1-path=[TEST_PATH]/db-master/repo
Consolidate archive-push tests into new archive test.
2017-06-21 14:38:53 +02:00
spool-path=[TEST_PATH]/db-master/spool
New simpler configuration and consistent project/exe/path naming. * The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required. * Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case. * The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally. * The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest. * Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory. * Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory. * Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 15:30:54 +02:00
[global:backup]
archive-copy=y
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
exclude=postgresql.auto.conf
exclude=pg_log/
exclude=pg_log2
New simpler configuration and consistent project/exe/path naming. * The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required. * Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case. * The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally. * The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest. * Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory. * Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory. * Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 15:30:54 +02:00
start-fast=y
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
+ supplemental file: [TEST_PATH]/db-master/repo/backup/db/[BACKUP-DIFF-3]/backup.manifest
-----------------------------------------------------------------------------------------
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[backrest]
backrest-checksum="[CHECKSUM]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
backrest-format=5
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
backrest-version="[VERSION-1]"
[backup]
backup-label="[BACKUP-DIFF-3]"
backup-prior="[BACKUP-FULL-2]"
backup-timestamp-copy-start=[TIMESTAMP]
backup-timestamp-start=[TIMESTAMP]
backup-timestamp-stop=[TIMESTAMP]
backup-type="diff"
[backup:db]
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-catalog-version=201409291
db-control-version=942
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
db-id=1
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
db-system-id=1000000000000000094
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-version="9.4"
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[backup:option]
option-archive-check=true
option-archive-copy=true
Backup from a standby cluster. A connection to the primary cluster is still required to start/stop the backup and copy files that are not replicated, but the vast majority of files are copied from the standby in order to reduce load on the master.
2016-08-25 17:25:46 +02:00
option-backup-standby=false
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
option-checksum-page=true
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
option-compress=false
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
option-delta=true
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
option-hardlink=false
Closed #167: Rename `--no-start-stop` option to `--no-online`.
2016-02-20 02:32:02 +02:00
option-online=false
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[backup:target]
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
pg_data={"path":"[TEST_PATH]/db-master/db/base-2","type":"path"}
pg_tblspc/2={"path":"[TEST_PATH]/db-master/db/tablespace/ts2-2","tablespace-id":"2","tablespace-name":"ts2","type":"link"}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:file]
Fixed a issue where tablespaces were copied from the master during standby backup.
2016-09-04 15:19:44 +02:00
pg_data/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/badchecksum.txt={"checksum":"f927212cd08d11a42a666b2f04235398e9ceeb51","master":true,"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
pg_data/base/1/12000={"checksum":"22c98d248ff548311eda88559e4a8405ed77c003","checksum-page":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
pg_data/base/1/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","mode":"0660","reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
pg_data/base/16384/17000={"checksum":"9a53d532e27785e681766c98516a5e93f096a501","checksum-page":false,"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
pg_data/base/16384/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Reduced the likelihood of torn pages causing a false positive in page checksums by filtering on start backup LSN.
2017-01-30 20:59:00 +02:00
pg_data/base/32768/33000={"checksum":"4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f","checksum-page":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Fixed an issue where newly initialized (but unused) pages would cause page checksum warnings. Reported by Stephen Frost.
2017-03-13 21:50:13 +02:00
pg_data/base/32768/33000.32767={"checksum":"21e2c7c1a326682c07053b7d6a5a40dbd49c2ec5","checksum-page":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
pg_data/base/32768/33001={"checksum":"6bf316f11d28c28914ea9be92c00de9bea6d9a6b","checksum-page":false,"checksum-page-error":[0,[3,5],7],"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
pg_data/base/32768/44000_init={"checksum":"4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f","checksum-page":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
pg_data/base/32768/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
pg_data/base/base2.txt={"checksum":"09b5e31766be1dba1ec27de82f975c1b6eea2a92","checksum-page":false,"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
pg_data/changecontent.txt={"checksum":"a094d94583e209556d03c3c5da33131a065f1689","master":true,"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/changetime.txt={"checksum":"88087292ed82e26f3eb824d0bffc05ccf7a30f8d","master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-2]}
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 11:24:42 +02:00
pg_data/global/pg_control={"checksum":"4c77c900f7af0d9ab13fa9982051a42e0b637f6c","master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-2]}
Fixed a issue where tablespaces were copied from the master during standby backup.
2016-09-04 15:19:44 +02:00
pg_data/pg_stat/global.stat={"checksum":"e350d5ce0153f3e22d5db21cf2a4eff00f3ee877","master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-2]}
pg_data/postgresql.conf={"checksum":"6721d92c9fcdf4248acff1f9a1377127d9064807","master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-2]}
Improve support for special characters in filenames. % characters caused issues in backup/restore due to filenames being appended directly into a format string. Reserved XML characters (<>&') caused issues in the S3 driver due to improper escaping. Add a file with all common special characters to regression testing.
2018-09-10 16:54:34 +02:00
pg_data/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?%={"master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Allow zero-size files in backup manifest to reference a prior manifest regardless of timestamp delta. Contributed by Cynthia Shang.
2018-08-24 22:50:33 +02:00
pg_data/zero_from_start={"master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Add zero-length file to mock/all test.
2018-07-05 21:40:50 +02:00
pg_data/zerosize.txt={"master":true,"size":[SIZE],"timestamp":[TIMESTAMP-2]}
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
pg_tblspc/2/[TS_PATH-1]/32768/tablespace2.txt={"checksum":"dc7f76e43c46101b47acc55ae4d593a9e6983578","checksum-page":false,"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_tblspc/2/[TS_PATH-1]/32768/tablespace2b.txt={"checksum":"e324463005236d83e6e54795dbddd20a74533bf3","checksum-page":false,"size":[SIZE],"timestamp":[TIMESTAMP-1]}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:file:default]
Continuous integration using travis-ci.
2017-02-21 15:59:23 +02:00
group="[GROUP-1]"
Fixed a issue where tablespaces were copied from the master during standby backup.
2016-09-04 15:19:44 +02:00
master=false
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
mode="0600"
Regression tests can now be run as any properly-configured user, not just vagrant.
2016-10-01 19:39:44 +02:00
user="[USER-1]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:link]
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
pg_data/pg_tblspc/2={"destination":"[TEST_PATH]/db-master/db/tablespace/ts2-2"}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:link:default]
Continuous integration using travis-ci.
2017-02-21 15:59:23 +02:00
group="[GROUP-1]"
Regression tests can now be run as any properly-configured user, not just vagrant.
2016-10-01 19:39:44 +02:00
user="[USER-1]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:path]
pg_data={}
pg_data/base={}
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
pg_data/base/1={}
pg_data/base/16384={}
pg_data/base/32768={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_data/global={}
pg_data/pg_clog={}
Exclude directories during backup that are cleaned, recreated, or zeroed by PostgreSQL at startup. These include (depending on the version where they were introduced): pgsql_tmp, pg_dynshmem, pg_notify, pg_replslot, pg_serial, pg_snapshots, pg_stat_tmp, pg_subtrans. The postgresql.auto.conf.tmp file is now excluded in addition to files that were already excluded: backup_label.old, postmaster.opts, postmaster.pid, recovery.conf, recovery.done.
2016-08-16 15:35:16 +02:00
pg_data/pg_dynshmem={}
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
pg_data/pg_log={}
Exclude directories during backup that are cleaned, recreated, or zeroed by PostgreSQL at startup. These include (depending on the version where they were introduced): pgsql_tmp, pg_dynshmem, pg_notify, pg_replslot, pg_serial, pg_snapshots, pg_stat_tmp, pg_subtrans. The postgresql.auto.conf.tmp file is now excluded in addition to files that were already excluded: backup_label.old, postmaster.opts, postmaster.pid, recovery.conf, recovery.done.
2016-08-16 15:35:16 +02:00
pg_data/pg_notify={}
pg_data/pg_replslot={}
pg_data/pg_serial={}
pg_data/pg_snapshots={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_data/pg_stat={}
Exclude directories during backup that are cleaned, recreated, or zeroed by PostgreSQL at startup. These include (depending on the version where they were introduced): pgsql_tmp, pg_dynshmem, pg_notify, pg_replslot, pg_serial, pg_snapshots, pg_stat_tmp, pg_subtrans. The postgresql.auto.conf.tmp file is now excluded in addition to files that were already excluded: backup_label.old, postmaster.opts, postmaster.pid, recovery.conf, recovery.done.
2016-08-16 15:35:16 +02:00
pg_data/pg_stat_tmp={}
pg_data/pg_subtrans={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_data/pg_tblspc={}
pg_tblspc={}
pg_tblspc/2={}
pg_tblspc/2/[TS_PATH-1]={}
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
pg_tblspc/2/[TS_PATH-1]/32768={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:path:default]
Continuous integration using travis-ci.
2017-02-21 15:59:23 +02:00
group="[GROUP-1]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
mode="0700"
Regression tests can now be run as any properly-configured user, not just vagrant.
2016-10-01 19:39:44 +02:00
user="[USER-1]"
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
+ supplemental file: [TEST_PATH]/db-master/repo/backup/db/backup.info
---------------------------------------------------------------------
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[backrest]
backrest-checksum="[CHECKSUM]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
backrest-format=5
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
backrest-version="[VERSION-1]"
[backup:current]
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
[BACKUP-FULL-2]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"full","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":true,"option-compress":false,"option-hardlink":false,"option-online":false}
[BACKUP-DIFF-2]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-prior":"[BACKUP-FULL-2]","backup-reference":["[BACKUP-FULL-2]"],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"diff","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":true,"option-compress":false,"option-hardlink":false,"option-online":false}
[BACKUP-INCR-3]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-prior":"[BACKUP-DIFF-2]","backup-reference":["[BACKUP-FULL-2]","[BACKUP-DIFF-2]"],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"incr","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":true,"option-compress":false,"option-hardlink":false,"option-online":false}
[BACKUP-INCR-4]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-prior":"[BACKUP-INCR-3]","backup-reference":["[BACKUP-FULL-2]","[BACKUP-DIFF-2]","[BACKUP-INCR-3]"],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"incr","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":true,"option-compress":false,"option-hardlink":false,"option-online":false}
[BACKUP-DIFF-3]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-prior":"[BACKUP-FULL-2]","backup-reference":["[BACKUP-FULL-2]"],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"diff","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":true,"option-compress":false,"option-hardlink":false,"option-online":false}
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[db]
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-catalog-version=201409291
db-control-version=942
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
db-id=1
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
db-system-id=1000000000000000094
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-version="9.4"
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[db:history]
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
1={"db-catalog-version":201409291,"db-control-version":942,"db-system-id":1000000000000000094,"db-version":"9.4"}
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
incr backup - remove files - but won't affect manifest (db-master host)
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --no-online --log-level-console=detail --stanza=db backup --test --test-delay=1 --test-point=manifest-build=y
Log testing can now be enabled for certain deterministic tests. This works by comparing the generated logs against a previous copy. Currently only enabled for the backup/synthetic tests.
2015-05-05 19:08:48 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
P00 INFO: backup command begin [BACKREST-VERSION]: --compress --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --db-timeout=45 --exclude=postgresql.auto.conf --exclude=pg_log/ --exclude=pg_log2 --lock-path=[TEST_PATH]/db-master/lock --log-level-console=detail --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --no-online --pg1-path=[TEST_PATH]/db-master/db/base-2 --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --stanza=db --start-fast --test --test-delay=1 --test-point=manifest-build=y
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
P00 WARN: option repo1-retention-full is not set, the repository may run out of space
HINT: to retain full backups indefinitely (without warning), set option 'repo1-retention-full' to the maximum.
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: last backup label = [BACKUP-DIFF-3], version = [VERSION-1]
Full/Synthetic test refactor. * Combine hardlink and non/compressed in synthetic tests to reduce test time and improve coverage. * Change log level of hardlink logging to detail. * Cast size in S3 manifest to integer.
2017-06-15 21:32:10 +02:00
P00 WARN: incr backup cannot alter compress option to 'true', reset to value in [BACKUP-DIFF-3]
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
P00 WARN: incr backup cannot alter 'checksum-page' option to 'false', reset to 'true' from [BACKUP-DIFF-3]
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 TEST: PgBaCkReStTeSt-MANIFEST-BUILD-PgBaCkReStTeSt
P00 INFO: incr backup size = 0B
P00 INFO: new backup label = [BACKUP-INCR-5]
Added a predictable log entry to signal that a command has completed successfully. For example a backup ends successfully with: INFO: backup command end: completed successfully.
2016-12-04 21:44:53 +02:00
P00 INFO: backup command end: completed successfully
Move async forking and more error handling to C. The Perl process was exiting directly when called but that interfered with proper locking for the forked async process. Now Perl returns results to the C process which handles all errors, including signals.
2018-04-13 02:42:26 +02:00
P00 INFO: expire command begin
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
P00 INFO: option 'repo1-retention-archive' is not set - archive logs will not be expired
Added a predictable log entry to signal that a command has completed successfully. For example a backup ends successfully with: INFO: backup command end: completed successfully.
2016-12-04 21:44:53 +02:00
P00 INFO: expire command end: completed successfully
Log testing can now be enabled for certain deterministic tests. This works by comparing the generated logs against a previous copy. Currently only enabled for the backup/synthetic tests.
2015-05-05 19:08:48 +02:00
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
+ supplemental file: [TEST_PATH]/db-master/pgbackrest.conf
----------------------------------------------------------
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[db]
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
pg1-path=[TEST_PATH]/db-master/db/base-2
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
New simpler configuration and consistent project/exe/path naming. * The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required. * Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case. * The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally. * The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest. * Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory. * Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory. * Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 15:30:54 +02:00
[db:restore]
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
tablespace-map=1=[TEST_PATH]/db-master/db/tablespace/ts1-2
tablespace-map=2=[TEST_PATH]/db-master/db/tablespace/ts2-2
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
New simpler configuration and consistent project/exe/path naming. * The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required. * Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case. * The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally. * The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest. * Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory. * Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory. * Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 15:30:54 +02:00
[global]
Full/Synthetic test refactor. * Combine hardlink and non/compressed in synthetic tests to reduce test time and improve coverage. * Change log level of hardlink logging to detail. * Cast size in S3 manifest to integer.
2017-06-15 21:32:10 +02:00
compress=y
Fix custom settings for compress-level option being ignored. Reported by Jens Wilke.
2017-10-23 00:05:46 +02:00
compress-level=3
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
db-timeout=45
lock-path=[TEST_PATH]/db-master/lock
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
log-level-console=debug
log-level-file=trace
Added the log-level-stderr option. Controls whether console log messages are sent to stderr or stdout. By default this is set to warn which represents a change in behavior from previous versions, even though it may be more intuitive. Setting log-level-stderr=off will preserve the old behavior. Suggested by Sascha Biberhofer.
2016-10-05 15:09:30 +02:00
log-level-stderr=off
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
log-path=[TEST_PATH]/db-master/log
Add log-subprocess option to allow file logging for local and remote subprocesses.
2018-08-23 02:05:49 +02:00
log-subprocess=y
Set log-timestamp=n for integration tests. This means less filtering of logs needs to be done and new timestamps can be added without adding new filters.
2018-05-11 17:24:38 +02:00
log-timestamp=n
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
protocol-timeout=60
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
repo1-path=[TEST_PATH]/db-master/repo
Consolidate archive-push tests into new archive test.
2017-06-21 14:38:53 +02:00
spool-path=[TEST_PATH]/db-master/spool
New simpler configuration and consistent project/exe/path naming. * The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required. * Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case. * The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally. * The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest. * Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory. * Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory. * Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 15:30:54 +02:00
[global:backup]
archive-copy=y
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
exclude=postgresql.auto.conf
exclude=pg_log/
exclude=pg_log2
New simpler configuration and consistent project/exe/path naming. * The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required. * Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case. * The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally. * The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest. * Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory. * Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory. * Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 15:30:54 +02:00
start-fast=y
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
+ supplemental file: [TEST_PATH]/db-master/repo/backup/db/[BACKUP-INCR-5]/backup.manifest
-----------------------------------------------------------------------------------------
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[backrest]
backrest-checksum="[CHECKSUM]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
backrest-format=5
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
backrest-version="[VERSION-1]"
[backup]
backup-label="[BACKUP-INCR-5]"
backup-prior="[BACKUP-DIFF-3]"
backup-timestamp-copy-start=[TIMESTAMP]
backup-timestamp-start=[TIMESTAMP]
backup-timestamp-stop=[TIMESTAMP]
backup-type="incr"
[backup:db]
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-catalog-version=201409291
db-control-version=942
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
db-id=1
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
db-system-id=1000000000000000094
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-version="9.4"
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[backup:option]
option-archive-check=true
option-archive-copy=true
Backup from a standby cluster. A connection to the primary cluster is still required to start/stop the backup and copy files that are not replicated, but the vast majority of files are copied from the standby in order to reduce load on the master.
2016-08-25 17:25:46 +02:00
option-backup-standby=false
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
option-checksum-page=true
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
option-compress=false
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
option-delta=false
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
option-hardlink=false
Closed #167: Rename `--no-start-stop` option to `--no-online`.
2016-02-20 02:32:02 +02:00
option-online=false
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[backup:target]
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
pg_data={"path":"[TEST_PATH]/db-master/db/base-2","type":"path"}
pg_tblspc/2={"path":"[TEST_PATH]/db-master/db/tablespace/ts2-2","tablespace-id":"2","tablespace-name":"ts2","type":"link"}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:file]
Fixed a issue where tablespaces were copied from the master during standby backup.
2016-09-04 15:19:44 +02:00
pg_data/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/badchecksum.txt={"checksum":"f927212cd08d11a42a666b2f04235398e9ceeb51","master":true,"reference":"[BACKUP-DIFF-3]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
pg_data/base/1/12000={"checksum":"22c98d248ff548311eda88559e4a8405ed77c003","checksum-page":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
pg_data/base/1/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","mode":"0660","reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
pg_data/base/16384/17000={"checksum":"9a53d532e27785e681766c98516a5e93f096a501","checksum-page":false,"reference":"[BACKUP-DIFF-3]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
pg_data/base/16384/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Reduced the likelihood of torn pages causing a false positive in page checksums by filtering on start backup LSN.
2017-01-30 20:59:00 +02:00
pg_data/base/32768/33000={"checksum":"4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f","checksum-page":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Fixed an issue where newly initialized (but unused) pages would cause page checksum warnings. Reported by Stephen Frost.
2017-03-13 21:50:13 +02:00
pg_data/base/32768/33000.32767={"checksum":"21e2c7c1a326682c07053b7d6a5a40dbd49c2ec5","checksum-page":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
pg_data/base/32768/33001={"checksum":"6bf316f11d28c28914ea9be92c00de9bea6d9a6b","checksum-page":false,"checksum-page-error":[0,[3,5],7],"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
pg_data/base/32768/44000_init={"checksum":"4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f","checksum-page":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
pg_data/base/32768/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
pg_data/base/base2.txt={"checksum":"09b5e31766be1dba1ec27de82f975c1b6eea2a92","checksum-page":false,"reference":"[BACKUP-DIFF-3]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
pg_data/changecontent.txt={"checksum":"a094d94583e209556d03c3c5da33131a065f1689","master":true,"reference":"[BACKUP-DIFF-3]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/changetime.txt={"checksum":"88087292ed82e26f3eb824d0bffc05ccf7a30f8d","master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-2]}
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 11:24:42 +02:00
pg_data/global/pg_control={"checksum":"4c77c900f7af0d9ab13fa9982051a42e0b637f6c","master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-2]}
Fixed a issue where tablespaces were copied from the master during standby backup.
2016-09-04 15:19:44 +02:00
pg_data/pg_stat/global.stat={"checksum":"e350d5ce0153f3e22d5db21cf2a4eff00f3ee877","master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-2]}
pg_data/postgresql.conf={"checksum":"6721d92c9fcdf4248acff1f9a1377127d9064807","master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-2]}
Improve support for special characters in filenames. % characters caused issues in backup/restore due to filenames being appended directly into a format string. Reserved XML characters (<>&') caused issues in the S3 driver due to improper escaping. Add a file with all common special characters to regression testing.
2018-09-10 16:54:34 +02:00
pg_data/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?%={"master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Allow zero-size files in backup manifest to reference a prior manifest regardless of timestamp delta. Contributed by Cynthia Shang.
2018-08-24 22:50:33 +02:00
pg_data/zero_from_start={"master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Add zero-length file to mock/all test.
2018-07-05 21:40:50 +02:00
pg_data/zerosize.txt={"master":true,"reference":"[BACKUP-DIFF-3]","size":[SIZE],"timestamp":[TIMESTAMP-2]}
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
pg_tblspc/2/[TS_PATH-1]/32768/tablespace2.txt={"checksum":"dc7f76e43c46101b47acc55ae4d593a9e6983578","checksum-page":false,"reference":"[BACKUP-DIFF-3]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_tblspc/2/[TS_PATH-1]/32768/tablespace2b.txt={"checksum":"e324463005236d83e6e54795dbddd20a74533bf3","checksum-page":false,"reference":"[BACKUP-DIFF-3]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:file:default]
Continuous integration using travis-ci.
2017-02-21 15:59:23 +02:00
group="[GROUP-1]"
Fixed a issue where tablespaces were copied from the master during standby backup.
2016-09-04 15:19:44 +02:00
master=false
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
mode="0600"
Regression tests can now be run as any properly-configured user, not just vagrant.
2016-10-01 19:39:44 +02:00
user="[USER-1]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:link]
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
pg_data/pg_tblspc/2={"destination":"[TEST_PATH]/db-master/db/tablespace/ts2-2"}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:link:default]
Continuous integration using travis-ci.
2017-02-21 15:59:23 +02:00
group="[GROUP-1]"
Regression tests can now be run as any properly-configured user, not just vagrant.
2016-10-01 19:39:44 +02:00
user="[USER-1]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:path]
pg_data={}
pg_data/base={}
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
pg_data/base/1={}
pg_data/base/16384={}
pg_data/base/32768={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_data/global={}
pg_data/pg_clog={}
Exclude directories during backup that are cleaned, recreated, or zeroed by PostgreSQL at startup. These include (depending on the version where they were introduced): pgsql_tmp, pg_dynshmem, pg_notify, pg_replslot, pg_serial, pg_snapshots, pg_stat_tmp, pg_subtrans. The postgresql.auto.conf.tmp file is now excluded in addition to files that were already excluded: backup_label.old, postmaster.opts, postmaster.pid, recovery.conf, recovery.done.
2016-08-16 15:35:16 +02:00
pg_data/pg_dynshmem={}
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
pg_data/pg_log={}
Exclude directories during backup that are cleaned, recreated, or zeroed by PostgreSQL at startup. These include (depending on the version where they were introduced): pgsql_tmp, pg_dynshmem, pg_notify, pg_replslot, pg_serial, pg_snapshots, pg_stat_tmp, pg_subtrans. The postgresql.auto.conf.tmp file is now excluded in addition to files that were already excluded: backup_label.old, postmaster.opts, postmaster.pid, recovery.conf, recovery.done.
2016-08-16 15:35:16 +02:00
pg_data/pg_notify={}
pg_data/pg_replslot={}
pg_data/pg_serial={}
pg_data/pg_snapshots={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_data/pg_stat={}
Exclude directories during backup that are cleaned, recreated, or zeroed by PostgreSQL at startup. These include (depending on the version where they were introduced): pgsql_tmp, pg_dynshmem, pg_notify, pg_replslot, pg_serial, pg_snapshots, pg_stat_tmp, pg_subtrans. The postgresql.auto.conf.tmp file is now excluded in addition to files that were already excluded: backup_label.old, postmaster.opts, postmaster.pid, recovery.conf, recovery.done.
2016-08-16 15:35:16 +02:00
pg_data/pg_stat_tmp={}
pg_data/pg_subtrans={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_data/pg_tblspc={}
pg_tblspc={}
pg_tblspc/2={}
pg_tblspc/2/[TS_PATH-1]={}
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
pg_tblspc/2/[TS_PATH-1]/32768={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:path:default]
Continuous integration using travis-ci.
2017-02-21 15:59:23 +02:00
group="[GROUP-1]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
mode="0700"
Regression tests can now be run as any properly-configured user, not just vagrant.
2016-10-01 19:39:44 +02:00
user="[USER-1]"
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
+ supplemental file: [TEST_PATH]/db-master/repo/backup/db/backup.info
---------------------------------------------------------------------
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[backrest]
backrest-checksum="[CHECKSUM]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
backrest-format=5
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
backrest-version="[VERSION-1]"
[backup:current]
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
[BACKUP-FULL-2]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"full","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":true,"option-compress":false,"option-hardlink":false,"option-online":false}
[BACKUP-DIFF-2]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-prior":"[BACKUP-FULL-2]","backup-reference":["[BACKUP-FULL-2]"],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"diff","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":true,"option-compress":false,"option-hardlink":false,"option-online":false}
[BACKUP-INCR-3]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-prior":"[BACKUP-DIFF-2]","backup-reference":["[BACKUP-FULL-2]","[BACKUP-DIFF-2]"],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"incr","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":true,"option-compress":false,"option-hardlink":false,"option-online":false}
[BACKUP-INCR-4]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-prior":"[BACKUP-INCR-3]","backup-reference":["[BACKUP-FULL-2]","[BACKUP-DIFF-2]","[BACKUP-INCR-3]"],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"incr","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":true,"option-compress":false,"option-hardlink":false,"option-online":false}
[BACKUP-DIFF-3]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-prior":"[BACKUP-FULL-2]","backup-reference":["[BACKUP-FULL-2]"],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"diff","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":true,"option-compress":false,"option-hardlink":false,"option-online":false}
[BACKUP-INCR-5]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-prior":"[BACKUP-DIFF-3]","backup-reference":["[BACKUP-FULL-2]","[BACKUP-DIFF-3]"],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"incr","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":true,"option-compress":false,"option-hardlink":false,"option-online":false}
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[db]
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-catalog-version=201409291
db-control-version=942
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
db-id=1
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
db-system-id=1000000000000000094
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-version="9.4"
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[db:history]
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
1={"db-catalog-version":201409291,"db-control-version":942,"db-system-id":1000000000000000094,"db-version":"9.4"}
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
diff backup - remove files during backup (db-master host)
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --no-online --log-level-console=detail --process-max=1 --delta --type=diff --stanza=db backup --test --test-delay=1 --test-point=manifest-build=y
Log testing can now be enabled for certain deterministic tests. This works by comparing the generated logs against a previous copy. Currently only enabled for the backup/synthetic tests.
2015-05-05 19:08:48 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P00 INFO: backup command begin [BACKREST-VERSION]: --compress --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --db-timeout=45 --delta --exclude=postgresql.auto.conf --exclude=pg_log/ --exclude=pg_log2 --lock-path=[TEST_PATH]/db-master/lock --log-level-console=detail --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --no-online --pg1-path=[TEST_PATH]/db-master/db/base-2 --process-max=1 --protocol-timeout=60 --repo1-hardlink --repo1-path=[TEST_PATH]/db-master/repo --stanza=db --start-fast --test --test-delay=1 --test-point=manifest-build=y --type=diff
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
P00 WARN: option repo1-retention-full is not set, the repository may run out of space
HINT: to retain full backups indefinitely (without warning), set option 'repo1-retention-full' to the maximum.
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: last backup label = [BACKUP-FULL-2], version = [VERSION-1]
Full/Synthetic test refactor. * Combine hardlink and non/compressed in synthetic tests to reduce test time and improve coverage. * Change log level of hardlink logging to detail. * Cast size in S3 manifest to integer.
2017-06-15 21:32:10 +02:00
P00 WARN: diff backup cannot alter compress option to 'true', reset to value in [BACKUP-FULL-2]
P00 WARN: diff backup cannot alter hardlink option to 'true', reset to value in [BACKUP-FULL-2]
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
P00 WARN: diff backup cannot alter 'checksum-page' option to 'false', reset to 'true' from [BACKUP-FULL-2]
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 TEST: PgBaCkReStTeSt-MANIFEST-BUILD-PgBaCkReStTeSt
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base-2/base/32768/33001 (64KB, 36%) checksum 6bf316f11d28c28914ea9be92c00de9bea6d9a6b
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base-2/base/32768/44000_init (32KB, 54%) checksum 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base-2/base/32768/33000.32767 (32KB, 72%) checksum 21e2c7c1a326682c07053b7d6a5a40dbd49c2ec5
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base-2/base/32768/33000 (32KB, 90%) checksum 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 11:24:42 +02:00
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base-2/global/pg_control (8KB, 95%) checksum 4c77c900f7af0d9ab13fa9982051a42e0b637f6c
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base-2/base/1/12000 (8KB, 99%) checksum 22c98d248ff548311eda88559e4a8405ed77c003
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base-2/postgresql.conf (21B, 99%) checksum 6721d92c9fcdf4248acff1f9a1377127d9064807
P01 INFO: backup file [TEST_PATH]/db-master/db/base-2/badchecksum.txt (11B, 99%) checksum f927212cd08d11a42a666b2f04235398e9ceeb51
P01 INFO: backup file [TEST_PATH]/db-master/db/base-2/changecontent.txt (7B, 99%) checksum a094d94583e209556d03c3c5da33131a065f1689
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base-2/pg_stat/global.stat (5B, 99%) checksum e350d5ce0153f3e22d5db21cf2a4eff00f3ee877
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 DETAIL: skip file removed by database [TEST_PATH]/db-master/db/base-2/base/base2.txt
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base-2/changetime.txt (4B, 99%) checksum 88087292ed82e26f3eb824d0bffc05ccf7a30f8d
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base-2/base/32768/PG_VERSION (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base-2/base/16384/PG_VERSION (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base-2/base/1/PG_VERSION (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P01 DETAIL: match file from prior backup [TEST_PATH]/db-master/db/base-2/PG_VERSION (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P01 INFO: backup file [TEST_PATH]/db-master/db/base-2/zerosize.txt (0B, 99%)
P01 INFO: backup file [TEST_PATH]/db-master/db/base-2/pg_tblspc/2/[TS_PATH-1]/32768/tablespace2c.txt (12B, 99%) checksum dfcb8679956b734706cf87259d50c88f83e80e66
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
P00 WARN: page misalignment in file [TEST_PATH]/db-master/db/base-2/pg_tblspc/2/[TS_PATH-1]/32768/tablespace2c.txt: file size 12 is not divisible by page size 8192
Additional backup exclusions. * Exclude contents of pg_snapshots, pg_serial, pg_notify, and pg_dynshmem from backup since they are rebuilt on startup. * Exclude pg_internal.init files from backup since they are rebuilt on startup.
2017-09-04 14:26:57 +02:00
P01 INFO: backup file [TEST_PATH]/db-master/db/base-2/pg_tblspc/2/[TS_PATH-1]/32768/tablespace2.txt (7B, 100%) checksum dc7f76e43c46101b47acc55ae4d593a9e6983578
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
P00 WARN: page misalignment in file [TEST_PATH]/db-master/db/base-2/pg_tblspc/2/[TS_PATH-1]/32768/tablespace2.txt: file size 7 is not divisible by page size 8192
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P00 INFO: diff backup size = 176KB
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: new backup label = [BACKUP-DIFF-4]
Added a predictable log entry to signal that a command has completed successfully. For example a backup ends successfully with: INFO: backup command end: completed successfully.
2016-12-04 21:44:53 +02:00
P00 INFO: backup command end: completed successfully
Move async forking and more error handling to C. The Perl process was exiting directly when called but that interfered with proper locking for the forked async process. Now Perl returns results to the C process which handles all errors, including signals.
2018-04-13 02:42:26 +02:00
P00 INFO: expire command begin
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
P00 INFO: option 'repo1-retention-archive' is not set - archive logs will not be expired
Added a predictable log entry to signal that a command has completed successfully. For example a backup ends successfully with: INFO: backup command end: completed successfully.
2016-12-04 21:44:53 +02:00
P00 INFO: expire command end: completed successfully
Log testing can now be enabled for certain deterministic tests. This works by comparing the generated logs against a previous copy. Currently only enabled for the backup/synthetic tests.
2015-05-05 19:08:48 +02:00
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
+ supplemental file: [TEST_PATH]/db-master/pgbackrest.conf
----------------------------------------------------------
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[db]
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
pg1-path=[TEST_PATH]/db-master/db/base-2
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
New simpler configuration and consistent project/exe/path naming. * The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required. * Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case. * The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally. * The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest. * Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory. * Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory. * Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 15:30:54 +02:00
[db:restore]
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
tablespace-map=1=[TEST_PATH]/db-master/db/tablespace/ts1-2
tablespace-map=2=[TEST_PATH]/db-master/db/tablespace/ts2-2
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
New simpler configuration and consistent project/exe/path naming. * The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required. * Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case. * The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally. * The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest. * Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory. * Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory. * Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 15:30:54 +02:00
[global]
Full/Synthetic test refactor. * Combine hardlink and non/compressed in synthetic tests to reduce test time and improve coverage. * Change log level of hardlink logging to detail. * Cast size in S3 manifest to integer.
2017-06-15 21:32:10 +02:00
compress=y
Fix custom settings for compress-level option being ignored. Reported by Jens Wilke.
2017-10-23 00:05:46 +02:00
compress-level=3
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
db-timeout=45
lock-path=[TEST_PATH]/db-master/lock
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
log-level-console=debug
log-level-file=trace
Added the log-level-stderr option. Controls whether console log messages are sent to stderr or stdout. By default this is set to warn which represents a change in behavior from previous versions, even though it may be more intuitive. Setting log-level-stderr=off will preserve the old behavior. Suggested by Sascha Biberhofer.
2016-10-05 15:09:30 +02:00
log-level-stderr=off
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
log-path=[TEST_PATH]/db-master/log
Add log-subprocess option to allow file logging for local and remote subprocesses.
2018-08-23 02:05:49 +02:00
log-subprocess=y
Set log-timestamp=n for integration tests. This means less filtering of logs needs to be done and new timestamps can be added without adding new filters.
2018-05-11 17:24:38 +02:00
log-timestamp=n
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
protocol-timeout=60
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
repo1-hardlink=y
repo1-path=[TEST_PATH]/db-master/repo
Consolidate archive-push tests into new archive test.
2017-06-21 14:38:53 +02:00
spool-path=[TEST_PATH]/db-master/spool
New simpler configuration and consistent project/exe/path naming. * The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required. * Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case. * The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally. * The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest. * Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory. * Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory. * Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 15:30:54 +02:00
[global:backup]
archive-copy=y
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
exclude=postgresql.auto.conf
exclude=pg_log/
exclude=pg_log2
New simpler configuration and consistent project/exe/path naming. * The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required. * Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case. * The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally. * The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest. * Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory. * Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory. * Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 15:30:54 +02:00
start-fast=y
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
+ supplemental file: [TEST_PATH]/db-master/repo/backup/db/[BACKUP-DIFF-4]/backup.manifest
-----------------------------------------------------------------------------------------
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[backrest]
backrest-checksum="[CHECKSUM]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
backrest-format=5
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
backrest-version="[VERSION-1]"
[backup]
backup-label="[BACKUP-DIFF-4]"
backup-prior="[BACKUP-FULL-2]"
backup-timestamp-copy-start=[TIMESTAMP]
backup-timestamp-start=[TIMESTAMP]
backup-timestamp-stop=[TIMESTAMP]
backup-type="diff"
[backup:db]
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-catalog-version=201409291
db-control-version=942
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
db-id=1
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
db-system-id=1000000000000000094
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-version="9.4"
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[backup:option]
option-archive-check=true
option-archive-copy=true
Backup from a standby cluster. A connection to the primary cluster is still required to start/stop the backup and copy files that are not replicated, but the vast majority of files are copied from the standby in order to reduce load on the master.
2016-08-25 17:25:46 +02:00
option-backup-standby=false
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
option-checksum-page=true
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
option-compress=false
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
option-delta=true
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
option-hardlink=false
Closed #167: Rename `--no-start-stop` option to `--no-online`.
2016-02-20 02:32:02 +02:00
option-online=false
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[backup:target]
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
pg_data={"path":"[TEST_PATH]/db-master/db/base-2","type":"path"}
pg_tblspc/2={"path":"[TEST_PATH]/db-master/db/tablespace/ts2-2","tablespace-id":"2","tablespace-name":"ts2","type":"link"}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:file]
Fixed a issue where tablespaces were copied from the master during standby backup.
2016-09-04 15:19:44 +02:00
pg_data/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/badchecksum.txt={"checksum":"f927212cd08d11a42a666b2f04235398e9ceeb51","master":true,"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
pg_data/base/1/12000={"checksum":"22c98d248ff548311eda88559e4a8405ed77c003","checksum-page":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
pg_data/base/1/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","mode":"0660","reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/base/16384/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Reduced the likelihood of torn pages causing a false positive in page checksums by filtering on start backup LSN.
2017-01-30 20:59:00 +02:00
pg_data/base/32768/33000={"checksum":"4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f","checksum-page":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Fixed an issue where newly initialized (but unused) pages would cause page checksum warnings. Reported by Stephen Frost.
2017-03-13 21:50:13 +02:00
pg_data/base/32768/33000.32767={"checksum":"21e2c7c1a326682c07053b7d6a5a40dbd49c2ec5","checksum-page":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
pg_data/base/32768/33001={"checksum":"6bf316f11d28c28914ea9be92c00de9bea6d9a6b","checksum-page":false,"checksum-page-error":[0,[3,5],7],"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
pg_data/base/32768/44000_init={"checksum":"4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f","checksum-page":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
pg_data/base/32768/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
pg_data/changecontent.txt={"checksum":"a094d94583e209556d03c3c5da33131a065f1689","master":true,"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/changetime.txt={"checksum":"88087292ed82e26f3eb824d0bffc05ccf7a30f8d","master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-2]}
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 11:24:42 +02:00
pg_data/global/pg_control={"checksum":"4c77c900f7af0d9ab13fa9982051a42e0b637f6c","master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-2]}
Fixed a issue where tablespaces were copied from the master during standby backup.
2016-09-04 15:19:44 +02:00
pg_data/pg_stat/global.stat={"checksum":"e350d5ce0153f3e22d5db21cf2a4eff00f3ee877","master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-2]}
pg_data/postgresql.conf={"checksum":"6721d92c9fcdf4248acff1f9a1377127d9064807","master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-2]}
Improve support for special characters in filenames. % characters caused issues in backup/restore due to filenames being appended directly into a format string. Reserved XML characters (<>&') caused issues in the S3 driver due to improper escaping. Add a file with all common special characters to regression testing.
2018-09-10 16:54:34 +02:00
pg_data/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?%={"master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Allow zero-size files in backup manifest to reference a prior manifest regardless of timestamp delta. Contributed by Cynthia Shang.
2018-08-24 22:50:33 +02:00
pg_data/zero_from_start={"master":true,"reference":"[BACKUP-FULL-2]","size":[SIZE],"timestamp":[TIMESTAMP-1]}
Add zero-length file to mock/all test.
2018-07-05 21:40:50 +02:00
pg_data/zerosize.txt={"master":true,"size":[SIZE],"timestamp":[TIMESTAMP-2]}
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
pg_tblspc/2/[TS_PATH-1]/32768/tablespace2.txt={"checksum":"dc7f76e43c46101b47acc55ae4d593a9e6983578","checksum-page":false,"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_tblspc/2/[TS_PATH-1]/32768/tablespace2c.txt={"checksum":"dfcb8679956b734706cf87259d50c88f83e80e66","checksum-page":false,"size":[SIZE],"timestamp":[TIMESTAMP-1]}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:file:default]
Continuous integration using travis-ci.
2017-02-21 15:59:23 +02:00
group="[GROUP-1]"
Fixed a issue where tablespaces were copied from the master during standby backup.
2016-09-04 15:19:44 +02:00
master=false
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
mode="0600"
Regression tests can now be run as any properly-configured user, not just vagrant.
2016-10-01 19:39:44 +02:00
user="[USER-1]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:link]
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
pg_data/pg_tblspc/2={"destination":"[TEST_PATH]/db-master/db/tablespace/ts2-2"}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:link:default]
Continuous integration using travis-ci.
2017-02-21 15:59:23 +02:00
group="[GROUP-1]"
Regression tests can now be run as any properly-configured user, not just vagrant.
2016-10-01 19:39:44 +02:00
user="[USER-1]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:path]
pg_data={}
pg_data/base={}
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
pg_data/base/1={}
pg_data/base/16384={}
pg_data/base/32768={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_data/global={}
pg_data/pg_clog={}
Exclude directories during backup that are cleaned, recreated, or zeroed by PostgreSQL at startup. These include (depending on the version where they were introduced): pgsql_tmp, pg_dynshmem, pg_notify, pg_replslot, pg_serial, pg_snapshots, pg_stat_tmp, pg_subtrans. The postgresql.auto.conf.tmp file is now excluded in addition to files that were already excluded: backup_label.old, postmaster.opts, postmaster.pid, recovery.conf, recovery.done.
2016-08-16 15:35:16 +02:00
pg_data/pg_dynshmem={}
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
pg_data/pg_log={}
Exclude directories during backup that are cleaned, recreated, or zeroed by PostgreSQL at startup. These include (depending on the version where they were introduced): pgsql_tmp, pg_dynshmem, pg_notify, pg_replslot, pg_serial, pg_snapshots, pg_stat_tmp, pg_subtrans. The postgresql.auto.conf.tmp file is now excluded in addition to files that were already excluded: backup_label.old, postmaster.opts, postmaster.pid, recovery.conf, recovery.done.
2016-08-16 15:35:16 +02:00
pg_data/pg_notify={}
pg_data/pg_replslot={}
pg_data/pg_serial={}
pg_data/pg_snapshots={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_data/pg_stat={}
Exclude directories during backup that are cleaned, recreated, or zeroed by PostgreSQL at startup. These include (depending on the version where they were introduced): pgsql_tmp, pg_dynshmem, pg_notify, pg_replslot, pg_serial, pg_snapshots, pg_stat_tmp, pg_subtrans. The postgresql.auto.conf.tmp file is now excluded in addition to files that were already excluded: backup_label.old, postmaster.opts, postmaster.pid, recovery.conf, recovery.done.
2016-08-16 15:35:16 +02:00
pg_data/pg_stat_tmp={}
pg_data/pg_subtrans={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_data/pg_tblspc={}
pg_tblspc={}
pg_tblspc/2={}
pg_tblspc/2/[TS_PATH-1]={}
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
pg_tblspc/2/[TS_PATH-1]/32768={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:path:default]
Continuous integration using travis-ci.
2017-02-21 15:59:23 +02:00
group="[GROUP-1]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
mode="0700"
Regression tests can now be run as any properly-configured user, not just vagrant.
2016-10-01 19:39:44 +02:00
user="[USER-1]"
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
+ supplemental file: [TEST_PATH]/db-master/repo/backup/db/backup.info
---------------------------------------------------------------------
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[backrest]
backrest-checksum="[CHECKSUM]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
backrest-format=5
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
backrest-version="[VERSION-1]"
[backup:current]
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
[BACKUP-FULL-2]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"full","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":true,"option-compress":false,"option-hardlink":false,"option-online":false}
[BACKUP-DIFF-2]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-prior":"[BACKUP-FULL-2]","backup-reference":["[BACKUP-FULL-2]"],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"diff","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":true,"option-compress":false,"option-hardlink":false,"option-online":false}
[BACKUP-INCR-3]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-prior":"[BACKUP-DIFF-2]","backup-reference":["[BACKUP-FULL-2]","[BACKUP-DIFF-2]"],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"incr","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":true,"option-compress":false,"option-hardlink":false,"option-online":false}
[BACKUP-INCR-4]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-prior":"[BACKUP-INCR-3]","backup-reference":["[BACKUP-FULL-2]","[BACKUP-DIFF-2]","[BACKUP-INCR-3]"],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"incr","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":true,"option-compress":false,"option-hardlink":false,"option-online":false}
[BACKUP-DIFF-3]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-prior":"[BACKUP-FULL-2]","backup-reference":["[BACKUP-FULL-2]"],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"diff","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":true,"option-compress":false,"option-hardlink":false,"option-online":false}
[BACKUP-INCR-5]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-prior":"[BACKUP-DIFF-3]","backup-reference":["[BACKUP-FULL-2]","[BACKUP-DIFF-3]"],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"incr","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":true,"option-compress":false,"option-hardlink":false,"option-online":false}
[BACKUP-DIFF-4]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-prior":"[BACKUP-FULL-2]","backup-reference":["[BACKUP-FULL-2]"],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"diff","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":true,"option-compress":false,"option-hardlink":false,"option-online":false}
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[db]
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-catalog-version=201409291
db-control-version=942
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
db-id=1
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
db-system-id=1000000000000000094
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-version="9.4"
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[db:history]
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
1={"db-catalog-version":201409291,"db-control-version":942,"db-system-id":1000000000000000094,"db-version":"9.4"}
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
full backup - update file (db-master host)
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --no-online --log-level-console=detail --type=full --stanza=db backup
Log testing can now be enabled for certain deterministic tests. This works by comparing the generated logs against a previous copy. Currently only enabled for the backup/synthetic tests.
2015-05-05 19:08:48 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
P00 INFO: backup command begin [BACKREST-VERSION]: --compress --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --db-timeout=45 --exclude=postgresql.auto.conf --exclude=pg_log/ --exclude=pg_log2 --lock-path=[TEST_PATH]/db-master/lock --log-level-console=detail --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --no-online --pg1-path=[TEST_PATH]/db-master/db/base-2 --protocol-timeout=60 --repo1-hardlink --repo1-path=[TEST_PATH]/db-master/repo --stanza=db --start-fast --type=full
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
P00 WARN: option repo1-retention-full is not set, the repository may run out of space
HINT: to retain full backups indefinitely (without warning), set option 'repo1-retention-full' to the maximum.
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
P01 INFO: backup file [TEST_PATH]/db-master/db/base-2/base/32768/33001 (64KB, 36%) checksum 6bf316f11d28c28914ea9be92c00de9bea6d9a6b
P01 INFO: backup file [TEST_PATH]/db-master/db/base-2/base/32768/44000_init (32KB, 54%) checksum 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f
P01 INFO: backup file [TEST_PATH]/db-master/db/base-2/base/32768/33000.32767 (32KB, 72%) checksum 21e2c7c1a326682c07053b7d6a5a40dbd49c2ec5
P01 INFO: backup file [TEST_PATH]/db-master/db/base-2/base/32768/33000 (32KB, 90%) checksum 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 11:24:42 +02:00
P01 INFO: backup file [TEST_PATH]/db-master/db/base-2/global/pg_control (8KB, 95%) checksum 4c77c900f7af0d9ab13fa9982051a42e0b637f6c
Add more realistic data files to synthetic backup and restore tests.
2016-12-10 16:13:55 +02:00
P01 INFO: backup file [TEST_PATH]/db-master/db/base-2/base/1/12000 (8KB, 99%) checksum 22c98d248ff548311eda88559e4a8405ed77c003
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 INFO: backup file [TEST_PATH]/db-master/db/base-2/postgresql.conf (21B, 99%) checksum 6721d92c9fcdf4248acff1f9a1377127d9064807
P01 INFO: backup file [TEST_PATH]/db-master/db/base-2/badchecksum.txt (11B, 99%) checksum f927212cd08d11a42a666b2f04235398e9ceeb51
P01 INFO: backup file [TEST_PATH]/db-master/db/base-2/base/16384/17000 (9B, 99%) checksum 7579ada0808d7f98087a0a586d0df9de009cdc33
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P01 INFO: backup file [TEST_PATH]/db-master/db/base-2/changecontent.txt (7B, 99%) checksum a094d94583e209556d03c3c5da33131a065f1689
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 INFO: backup file [TEST_PATH]/db-master/db/base-2/pg_stat/global.stat (5B, 99%) checksum e350d5ce0153f3e22d5db21cf2a4eff00f3ee877
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P01 INFO: backup file [TEST_PATH]/db-master/db/base-2/changetime.txt (4B, 99%) checksum 88087292ed82e26f3eb824d0bffc05ccf7a30f8d
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 INFO: backup file [TEST_PATH]/db-master/db/base-2/base/32768/PG_VERSION (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P01 INFO: backup file [TEST_PATH]/db-master/db/base-2/base/16384/PG_VERSION (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P01 INFO: backup file [TEST_PATH]/db-master/db/base-2/base/1/PG_VERSION (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P01 INFO: backup file [TEST_PATH]/db-master/db/base-2/PG_VERSION (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
Add zero-length file to mock/all test.
2018-07-05 21:40:50 +02:00
P01 INFO: backup file [TEST_PATH]/db-master/db/base-2/zerosize.txt (0B, 99%)
Allow zero-size files in backup manifest to reference a prior manifest regardless of timestamp delta. Contributed by Cynthia Shang.
2018-08-24 22:50:33 +02:00
P01 INFO: backup file [TEST_PATH]/db-master/db/base-2/zero_from_start (0B, 99%)
Improve support for special characters in filenames. % characters caused issues in backup/restore due to filenames being appended directly into a format string. Reserved XML characters (<>&') caused issues in the S3 driver due to improper escaping. Add a file with all common special characters to regression testing.
2018-09-10 16:54:34 +02:00
P01 INFO: backup file [TEST_PATH]/db-master/db/base-2/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?% (0B, 99%)
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 INFO: backup file [TEST_PATH]/db-master/db/base-2/pg_tblspc/2/[TS_PATH-1]/32768/tablespace2c.txt (12B, 99%) checksum dfcb8679956b734706cf87259d50c88f83e80e66
Additional backup exclusions. * Exclude contents of pg_snapshots, pg_serial, pg_notify, and pg_dynshmem from backup since they are rebuilt on startup. * Exclude pg_internal.init files from backup since they are rebuilt on startup.
2017-09-04 14:26:57 +02:00
P01 INFO: backup file [TEST_PATH]/db-master/db/base-2/pg_tblspc/2/[TS_PATH-1]/32768/tablespace2.txt (7B, 100%) checksum dc7f76e43c46101b47acc55ae4d593a9e6983578
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
P00 INFO: full backup size = 176KB
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: new backup label = [BACKUP-FULL-3]
Added a predictable log entry to signal that a command has completed successfully. For example a backup ends successfully with: INFO: backup command end: completed successfully.
2016-12-04 21:44:53 +02:00
P00 INFO: backup command end: completed successfully
Move async forking and more error handling to C. The Perl process was exiting directly when called but that interfered with proper locking for the forked async process. Now Perl returns results to the C process which handles all errors, including signals.
2018-04-13 02:42:26 +02:00
P00 INFO: expire command begin
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
P00 INFO: option 'repo1-retention-archive' is not set - archive logs will not be expired
Added a predictable log entry to signal that a command has completed successfully. For example a backup ends successfully with: INFO: backup command end: completed successfully.
2016-12-04 21:44:53 +02:00
P00 INFO: expire command end: completed successfully
Log testing can now be enabled for certain deterministic tests. This works by comparing the generated logs against a previous copy. Currently only enabled for the backup/synthetic tests.
2015-05-05 19:08:48 +02:00
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
+ supplemental file: [TEST_PATH]/db-master/pgbackrest.conf
----------------------------------------------------------
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[db]
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
pg1-path=[TEST_PATH]/db-master/db/base-2
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
New simpler configuration and consistent project/exe/path naming. * The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required. * Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case. * The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally. * The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest. * Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory. * Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory. * Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 15:30:54 +02:00
[db:restore]
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
tablespace-map=1=[TEST_PATH]/db-master/db/tablespace/ts1-2
tablespace-map=2=[TEST_PATH]/db-master/db/tablespace/ts2-2
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
New simpler configuration and consistent project/exe/path naming. * The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required. * Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case. * The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally. * The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest. * Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory. * Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory. * Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 15:30:54 +02:00
[global]
Full/Synthetic test refactor. * Combine hardlink and non/compressed in synthetic tests to reduce test time and improve coverage. * Change log level of hardlink logging to detail. * Cast size in S3 manifest to integer.
2017-06-15 21:32:10 +02:00
compress=y
Fix custom settings for compress-level option being ignored. Reported by Jens Wilke.
2017-10-23 00:05:46 +02:00
compress-level=3
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
db-timeout=45
lock-path=[TEST_PATH]/db-master/lock
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
log-level-console=debug
log-level-file=trace
Added the log-level-stderr option. Controls whether console log messages are sent to stderr or stdout. By default this is set to warn which represents a change in behavior from previous versions, even though it may be more intuitive. Setting log-level-stderr=off will preserve the old behavior. Suggested by Sascha Biberhofer.
2016-10-05 15:09:30 +02:00
log-level-stderr=off
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
log-path=[TEST_PATH]/db-master/log
Add log-subprocess option to allow file logging for local and remote subprocesses.
2018-08-23 02:05:49 +02:00
log-subprocess=y
Set log-timestamp=n for integration tests. This means less filtering of logs needs to be done and new timestamps can be added without adding new filters.
2018-05-11 17:24:38 +02:00
log-timestamp=n
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
protocol-timeout=60
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
repo1-hardlink=y
repo1-path=[TEST_PATH]/db-master/repo
Consolidate archive-push tests into new archive test.
2017-06-21 14:38:53 +02:00
spool-path=[TEST_PATH]/db-master/spool
New simpler configuration and consistent project/exe/path naming. * The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required. * Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case. * The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally. * The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest. * Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory. * Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory. * Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 15:30:54 +02:00
[global:backup]
archive-copy=y
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
exclude=postgresql.auto.conf
exclude=pg_log/
exclude=pg_log2
New simpler configuration and consistent project/exe/path naming. * The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required. * Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case. * The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally. * The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest. * Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory. * Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory. * Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 15:30:54 +02:00
start-fast=y
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
+ supplemental file: [TEST_PATH]/db-master/repo/backup/db/[BACKUP-FULL-3]/backup.manifest
-----------------------------------------------------------------------------------------
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[backrest]
backrest-checksum="[CHECKSUM]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
backrest-format=5
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
backrest-version="[VERSION-1]"
[backup]
backup-label="[BACKUP-FULL-3]"
backup-timestamp-copy-start=[TIMESTAMP]
backup-timestamp-start=[TIMESTAMP]
backup-timestamp-stop=[TIMESTAMP]
backup-type="full"
[backup:db]
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-catalog-version=201409291
db-control-version=942
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
db-id=1
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
db-system-id=1000000000000000094
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-version="9.4"
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[backup:option]
option-archive-check=true
option-archive-copy=true
Backup from a standby cluster. A connection to the primary cluster is still required to start/stop the backup and copy files that are not replicated, but the vast majority of files are copied from the standby in order to reduce load on the master.
2016-08-25 17:25:46 +02:00
option-backup-standby=false
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
option-checksum-page=false
Full/Synthetic test refactor. * Combine hardlink and non/compressed in synthetic tests to reduce test time and improve coverage. * Change log level of hardlink logging to detail. * Cast size in S3 manifest to integer.
2017-06-15 21:32:10 +02:00
option-compress=true
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
option-delta=false
Full/Synthetic test refactor. * Combine hardlink and non/compressed in synthetic tests to reduce test time and improve coverage. * Change log level of hardlink logging to detail. * Cast size in S3 manifest to integer.
2017-06-15 21:32:10 +02:00
option-hardlink=true
Closed #167: Rename `--no-start-stop` option to `--no-online`.
2016-02-20 02:32:02 +02:00
option-online=false
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[backup:target]
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
pg_data={"path":"[TEST_PATH]/db-master/db/base-2","type":"path"}
pg_tblspc/2={"path":"[TEST_PATH]/db-master/db/tablespace/ts2-2","tablespace-id":"2","tablespace-name":"ts2","type":"link"}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:file]
Full/Synthetic test refactor. * Combine hardlink and non/compressed in synthetic tests to reduce test time and improve coverage. * Change log level of hardlink logging to detail. * Cast size in S3 manifest to integer.
2017-06-15 21:32:10 +02:00
pg_data/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","master":true,"repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/badchecksum.txt={"checksum":"f927212cd08d11a42a666b2f04235398e9ceeb51","master":true,"repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/base/1/12000={"checksum":"22c98d248ff548311eda88559e4a8405ed77c003","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/base/1/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","mode":"0660","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/base/16384/17000={"checksum":"7579ada0808d7f98087a0a586d0df9de009cdc33","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/base/16384/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/base/32768/33000={"checksum":"4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/base/32768/33000.32767={"checksum":"21e2c7c1a326682c07053b7d6a5a40dbd49c2ec5","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/base/32768/33001={"checksum":"6bf316f11d28c28914ea9be92c00de9bea6d9a6b","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
pg_data/base/32768/44000_init={"checksum":"4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Full/Synthetic test refactor. * Combine hardlink and non/compressed in synthetic tests to reduce test time and improve coverage. * Change log level of hardlink logging to detail. * Cast size in S3 manifest to integer.
2017-06-15 21:32:10 +02:00
pg_data/base/32768/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
pg_data/changecontent.txt={"checksum":"a094d94583e209556d03c3c5da33131a065f1689","master":true,"repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/changetime.txt={"checksum":"88087292ed82e26f3eb824d0bffc05ccf7a30f8d","master":true,"repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-2]}
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 11:24:42 +02:00
pg_data/global/pg_control={"checksum":"4c77c900f7af0d9ab13fa9982051a42e0b637f6c","master":true,"repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-2]}
Full/Synthetic test refactor. * Combine hardlink and non/compressed in synthetic tests to reduce test time and improve coverage. * Change log level of hardlink logging to detail. * Cast size in S3 manifest to integer.
2017-06-15 21:32:10 +02:00
pg_data/pg_stat/global.stat={"checksum":"e350d5ce0153f3e22d5db21cf2a4eff00f3ee877","master":true,"repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-2]}
pg_data/postgresql.conf={"checksum":"6721d92c9fcdf4248acff1f9a1377127d9064807","master":true,"repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-2]}
Improve support for special characters in filenames. % characters caused issues in backup/restore due to filenames being appended directly into a format string. Reserved XML characters (<>&') caused issues in the S3 driver due to improper escaping. Add a file with all common special characters to regression testing.
2018-09-10 16:54:34 +02:00
pg_data/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?%={"master":true,"repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Allow zero-size files in backup manifest to reference a prior manifest regardless of timestamp delta. Contributed by Cynthia Shang.
2018-08-24 22:50:33 +02:00
pg_data/zero_from_start={"master":true,"repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Add zero-length file to mock/all test.
2018-07-05 21:40:50 +02:00
pg_data/zerosize.txt={"master":true,"repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-2]}
Full/Synthetic test refactor. * Combine hardlink and non/compressed in synthetic tests to reduce test time and improve coverage. * Change log level of hardlink logging to detail. * Cast size in S3 manifest to integer.
2017-06-15 21:32:10 +02:00
pg_tblspc/2/[TS_PATH-1]/32768/tablespace2.txt={"checksum":"dc7f76e43c46101b47acc55ae4d593a9e6983578","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_tblspc/2/[TS_PATH-1]/32768/tablespace2c.txt={"checksum":"dfcb8679956b734706cf87259d50c88f83e80e66","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:file:default]
Continuous integration using travis-ci.
2017-02-21 15:59:23 +02:00
group="[GROUP-1]"
Fixed a issue where tablespaces were copied from the master during standby backup.
2016-09-04 15:19:44 +02:00
master=false
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
mode="0600"
Regression tests can now be run as any properly-configured user, not just vagrant.
2016-10-01 19:39:44 +02:00
user="[USER-1]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:link]
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
pg_data/pg_tblspc/2={"destination":"[TEST_PATH]/db-master/db/tablespace/ts2-2"}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:link:default]
Continuous integration using travis-ci.
2017-02-21 15:59:23 +02:00
group="[GROUP-1]"
Regression tests can now be run as any properly-configured user, not just vagrant.
2016-10-01 19:39:44 +02:00
user="[USER-1]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:path]
pg_data={}
pg_data/base={}
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
pg_data/base/1={}
pg_data/base/16384={}
pg_data/base/32768={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_data/global={}
pg_data/pg_clog={}
Exclude directories during backup that are cleaned, recreated, or zeroed by PostgreSQL at startup. These include (depending on the version where they were introduced): pgsql_tmp, pg_dynshmem, pg_notify, pg_replslot, pg_serial, pg_snapshots, pg_stat_tmp, pg_subtrans. The postgresql.auto.conf.tmp file is now excluded in addition to files that were already excluded: backup_label.old, postmaster.opts, postmaster.pid, recovery.conf, recovery.done.
2016-08-16 15:35:16 +02:00
pg_data/pg_dynshmem={}
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
pg_data/pg_log={}
Exclude directories during backup that are cleaned, recreated, or zeroed by PostgreSQL at startup. These include (depending on the version where they were introduced): pgsql_tmp, pg_dynshmem, pg_notify, pg_replslot, pg_serial, pg_snapshots, pg_stat_tmp, pg_subtrans. The postgresql.auto.conf.tmp file is now excluded in addition to files that were already excluded: backup_label.old, postmaster.opts, postmaster.pid, recovery.conf, recovery.done.
2016-08-16 15:35:16 +02:00
pg_data/pg_notify={}
pg_data/pg_replslot={}
pg_data/pg_serial={}
pg_data/pg_snapshots={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_data/pg_stat={}
Exclude directories during backup that are cleaned, recreated, or zeroed by PostgreSQL at startup. These include (depending on the version where they were introduced): pgsql_tmp, pg_dynshmem, pg_notify, pg_replslot, pg_serial, pg_snapshots, pg_stat_tmp, pg_subtrans. The postgresql.auto.conf.tmp file is now excluded in addition to files that were already excluded: backup_label.old, postmaster.opts, postmaster.pid, recovery.conf, recovery.done.
2016-08-16 15:35:16 +02:00
pg_data/pg_stat_tmp={}
pg_data/pg_subtrans={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_data/pg_tblspc={}
pg_tblspc={}
pg_tblspc/2={}
pg_tblspc/2/[TS_PATH-1]={}
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
pg_tblspc/2/[TS_PATH-1]/32768={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:path:default]
Continuous integration using travis-ci.
2017-02-21 15:59:23 +02:00
group="[GROUP-1]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
mode="0700"
Regression tests can now be run as any properly-configured user, not just vagrant.
2016-10-01 19:39:44 +02:00
user="[USER-1]"
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
+ supplemental file: [TEST_PATH]/db-master/repo/backup/db/backup.info
---------------------------------------------------------------------
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[backrest]
backrest-checksum="[CHECKSUM]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
backrest-format=5
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
backrest-version="[VERSION-1]"
[backup:current]
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
[BACKUP-FULL-2]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"full","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":true,"option-compress":false,"option-hardlink":false,"option-online":false}
[BACKUP-DIFF-2]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-prior":"[BACKUP-FULL-2]","backup-reference":["[BACKUP-FULL-2]"],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"diff","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":true,"option-compress":false,"option-hardlink":false,"option-online":false}
[BACKUP-INCR-3]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-prior":"[BACKUP-DIFF-2]","backup-reference":["[BACKUP-FULL-2]","[BACKUP-DIFF-2]"],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"incr","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":true,"option-compress":false,"option-hardlink":false,"option-online":false}
[BACKUP-INCR-4]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-prior":"[BACKUP-INCR-3]","backup-reference":["[BACKUP-FULL-2]","[BACKUP-DIFF-2]","[BACKUP-INCR-3]"],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"incr","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":true,"option-compress":false,"option-hardlink":false,"option-online":false}
[BACKUP-DIFF-3]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-prior":"[BACKUP-FULL-2]","backup-reference":["[BACKUP-FULL-2]"],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"diff","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":true,"option-compress":false,"option-hardlink":false,"option-online":false}
[BACKUP-INCR-5]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-prior":"[BACKUP-DIFF-3]","backup-reference":["[BACKUP-FULL-2]","[BACKUP-DIFF-3]"],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"incr","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":true,"option-compress":false,"option-hardlink":false,"option-online":false}
[BACKUP-DIFF-4]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-prior":"[BACKUP-FULL-2]","backup-reference":["[BACKUP-FULL-2]"],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"diff","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":true,"option-compress":false,"option-hardlink":false,"option-online":false}
Full/Synthetic test refactor. * Combine hardlink and non/compressed in synthetic tests to reduce test time and improve coverage. * Change log level of hardlink logging to detail. * Cast size in S3 manifest to integer.
2017-06-15 21:32:10 +02:00
[BACKUP-FULL-3]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"full","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":false,"option-compress":true,"option-hardlink":true,"option-online":false}
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[db]
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-catalog-version=201409291
db-control-version=942
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
db-id=1
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
db-system-id=1000000000000000094
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-version="9.4"
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[db:history]
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
1={"db-catalog-version":201409291,"db-control-version":942,"db-system-id":1000000000000000094,"db-version":"9.4"}
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
info db stanza - normal output (db-master host)
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --log-level-console=warn --stanza=db info
Enhanced text output of info command to include timestamps, sizes, and the reference list for all backups.
2016-05-06 16:25:46 +02:00
------------------------------------------------------------------------------------------------------------------------------------
stanza: db
status: ok
Improve info command to display the stanza cipher type. Contributed by Cynthia Shang. Suggested by Douglas J Hunley.
2018-09-10 19:09:45 +02:00
cipher: none
Improved WAL data returned by info command. Modified the info command (both text and JSON output) to display the archive ID and minimum/maximum WAL currently present in the archive for the current and prior, if any, database cluster version. Contributed by Cynthia Shang.
2017-07-21 23:32:41 +02:00
db (current)
wal archive min/max (9.4-1): none present
full backup: [BACKUP-FULL-2]
timestamp start/stop: [TIMESTAMP-STR]
wal start/stop: n/a
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
database size: 192KB, backup size: 192KB
repository size: 192KB, repository backup size: 192KB
Improved WAL data returned by info command. Modified the info command (both text and JSON output) to display the archive ID and minimum/maximum WAL currently present in the archive for the current and prior, if any, database cluster version. Contributed by Cynthia Shang.
2017-07-21 23:32:41 +02:00
diff backup: [BACKUP-DIFF-2]
timestamp start/stop: [TIMESTAMP-STR]
wal start/stop: n/a
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
database size: 192KB, backup size: 32B
repository size: 192KB, repository backup size: 32B
Improved WAL data returned by info command. Modified the info command (both text and JSON output) to display the archive ID and minimum/maximum WAL currently present in the archive for the current and prior, if any, database cluster version. Contributed by Cynthia Shang.
2017-07-21 23:32:41 +02:00
backup reference list: [BACKUP-FULL-2]
incr backup: [BACKUP-INCR-3]
timestamp start/stop: [TIMESTAMP-STR]
wal start/stop: n/a
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
database size: 192KB, backup size: 13B
repository size: 192KB, repository backup size: 13B
Improved WAL data returned by info command. Modified the info command (both text and JSON output) to display the archive ID and minimum/maximum WAL currently present in the archive for the current and prior, if any, database cluster version. Contributed by Cynthia Shang.
2017-07-21 23:32:41 +02:00
backup reference list: [BACKUP-FULL-2], [BACKUP-DIFF-2]
incr backup: [BACKUP-INCR-4]
timestamp start/stop: [TIMESTAMP-STR]
wal start/stop: n/a
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
database size: 176KB, backup size: 8B
repository size: 176KB, repository backup size: 8B
Improved WAL data returned by info command. Modified the info command (both text and JSON output) to display the archive ID and minimum/maximum WAL currently present in the archive for the current and prior, if any, database cluster version. Contributed by Cynthia Shang.
2017-07-21 23:32:41 +02:00
backup reference list: [BACKUP-FULL-2], [BACKUP-DIFF-2], [BACKUP-INCR-3]
diff backup: [BACKUP-DIFF-3]
timestamp start/stop: [TIMESTAMP-STR]
wal start/stop: n/a
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
database size: 176KB, backup size: 46B
repository size: 176KB, repository backup size: 46B
Improved WAL data returned by info command. Modified the info command (both text and JSON output) to display the archive ID and minimum/maximum WAL currently present in the archive for the current and prior, if any, database cluster version. Contributed by Cynthia Shang.
2017-07-21 23:32:41 +02:00
backup reference list: [BACKUP-FULL-2]
incr backup: [BACKUP-INCR-5]
timestamp start/stop: [TIMESTAMP-STR]
wal start/stop: n/a
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
database size: 176KB, backup size: 0B
repository size: 176KB, repository backup size: 0B
Improved WAL data returned by info command. Modified the info command (both text and JSON output) to display the archive ID and minimum/maximum WAL currently present in the archive for the current and prior, if any, database cluster version. Contributed by Cynthia Shang.
2017-07-21 23:32:41 +02:00
backup reference list: [BACKUP-FULL-2], [BACKUP-DIFF-3]
diff backup: [BACKUP-DIFF-4]
timestamp start/stop: [TIMESTAMP-STR]
wal start/stop: n/a
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
database size: 176KB, backup size: 37B
repository size: 176KB, repository backup size: 37B
Improved WAL data returned by info command. Modified the info command (both text and JSON output) to display the archive ID and minimum/maximum WAL currently present in the archive for the current and prior, if any, database cluster version. Contributed by Cynthia Shang.
2017-07-21 23:32:41 +02:00
backup reference list: [BACKUP-FULL-2]
full backup: [BACKUP-FULL-3]
timestamp start/stop: [TIMESTAMP-STR]
wal start/stop: n/a
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
database size: 176KB, backup size: 176KB
The info command is implemented entirely in C. The C info code has already been committed but this commit wires it into main. Also remove the info Perl code and tests since they are no longer called.
2019-01-21 13:51:45 +02:00
repository size: 2.4KB, repository backup size: 2.4KB
Enhanced text output of info command to include timestamps, sizes, and the reference list for all backups.
2016-05-06 16:25:46 +02:00
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
info db stanza - normal output (db-master host)
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --log-level-console=warn --stanza=db --output=json info
Enhanced text output of info command to include timestamps, sizes, and the reference list for all backups.
2016-05-06 16:25:46 +02:00
------------------------------------------------------------------------------------------------------------------------------------
[
{
Improved WAL data returned by info command. Modified the info command (both text and JSON output) to display the archive ID and minimum/maximum WAL currently present in the archive for the current and prior, if any, database cluster version. Contributed by Cynthia Shang.
2017-07-21 23:32:41 +02:00
"archive" : [
{
"database" : {
Fix instances where database-id was not rendered as an integer in JSON info output. Fixed by Cynthia Shang. Reported by Jason O'Donnell.
2017-10-18 14:42:32 +02:00
"id" : 1
Improved WAL data returned by info command. Modified the info command (both text and JSON output) to display the archive ID and minimum/maximum WAL currently present in the archive for the current and prior, if any, database cluster version. Contributed by Cynthia Shang.
2017-07-21 23:32:41 +02:00
},
"id" : "9.4-1",
"max" : null,
"min" : null
}
],
Enhanced text output of info command to include timestamps, sizes, and the reference list for all backups.
2016-05-06 16:25:46 +02:00
"backup" : [
{
"archive" : {
"start" : null,
"stop" : null
},
"backrest" : {
"format" : 5,
"version" : "[VERSION-1]"
},
"database" : {
"id" : 1
},
"info" : {
"delta" : [DELTA],
"repository" : {
"delta" : [DELTA],
"size" : [SIZE]
},
"size" : [SIZE]
},
"label" : "[BACKUP-FULL-2]",
"prior" : null,
"reference" : null,
"timestamp" : {
"start" : [TIMESTAMP],
"stop" : [TIMESTAMP]
},
"type" : "full"
},
{
"archive" : {
"start" : null,
"stop" : null
},
"backrest" : {
"format" : 5,
"version" : "[VERSION-1]"
},
"database" : {
"id" : 1
},
"info" : {
"delta" : [DELTA],
"repository" : {
"delta" : [DELTA],
"size" : [SIZE]
},
"size" : [SIZE]
},
"label" : "[BACKUP-DIFF-2]",
"prior" : "[BACKUP-FULL-2]",
"reference" : [
"[BACKUP-FULL-2]"
],
"timestamp" : {
"start" : [TIMESTAMP],
"stop" : [TIMESTAMP]
},
"type" : "diff"
},
{
"archive" : {
"start" : null,
"stop" : null
},
"backrest" : {
"format" : 5,
"version" : "[VERSION-1]"
},
"database" : {
"id" : 1
},
"info" : {
"delta" : [DELTA],
"repository" : {
"delta" : [DELTA],
"size" : [SIZE]
},
"size" : [SIZE]
},
"label" : "[BACKUP-INCR-3]",
"prior" : "[BACKUP-DIFF-2]",
"reference" : [
"[BACKUP-FULL-2]",
"[BACKUP-DIFF-2]"
],
"timestamp" : {
"start" : [TIMESTAMP],
"stop" : [TIMESTAMP]
},
"type" : "incr"
},
{
"archive" : {
"start" : null,
"stop" : null
},
"backrest" : {
"format" : 5,
"version" : "[VERSION-1]"
},
"database" : {
"id" : 1
},
"info" : {
"delta" : [DELTA],
"repository" : {
"delta" : [DELTA],
"size" : [SIZE]
},
"size" : [SIZE]
},
"label" : "[BACKUP-INCR-4]",
"prior" : "[BACKUP-INCR-3]",
"reference" : [
"[BACKUP-FULL-2]",
"[BACKUP-DIFF-2]",
"[BACKUP-INCR-3]"
],
"timestamp" : {
"start" : [TIMESTAMP],
"stop" : [TIMESTAMP]
},
"type" : "incr"
},
{
"archive" : {
"start" : null,
"stop" : null
},
"backrest" : {
"format" : 5,
"version" : "[VERSION-1]"
},
"database" : {
"id" : 1
},
"info" : {
"delta" : [DELTA],
"repository" : {
"delta" : [DELTA],
"size" : [SIZE]
},
"size" : [SIZE]
},
"label" : "[BACKUP-DIFF-3]",
"prior" : "[BACKUP-FULL-2]",
"reference" : [
"[BACKUP-FULL-2]"
],
"timestamp" : {
"start" : [TIMESTAMP],
"stop" : [TIMESTAMP]
},
"type" : "diff"
},
{
"archive" : {
"start" : null,
"stop" : null
},
"backrest" : {
"format" : 5,
"version" : "[VERSION-1]"
},
"database" : {
"id" : 1
},
"info" : {
"delta" : [DELTA],
"repository" : {
"delta" : [DELTA],
"size" : [SIZE]
},
"size" : [SIZE]
},
"label" : "[BACKUP-INCR-5]",
"prior" : "[BACKUP-DIFF-3]",
"reference" : [
"[BACKUP-FULL-2]",
"[BACKUP-DIFF-3]"
],
"timestamp" : {
"start" : [TIMESTAMP],
"stop" : [TIMESTAMP]
},
"type" : "incr"
},
{
"archive" : {
"start" : null,
"stop" : null
},
"backrest" : {
"format" : 5,
"version" : "[VERSION-1]"
},
"database" : {
"id" : 1
},
"info" : {
"delta" : [DELTA],
"repository" : {
"delta" : [DELTA],
"size" : [SIZE]
},
"size" : [SIZE]
},
"label" : "[BACKUP-DIFF-4]",
"prior" : "[BACKUP-FULL-2]",
"reference" : [
"[BACKUP-FULL-2]"
],
"timestamp" : {
"start" : [TIMESTAMP],
"stop" : [TIMESTAMP]
},
"type" : "diff"
},
{
"archive" : {
"start" : null,
"stop" : null
},
"backrest" : {
"format" : 5,
"version" : "[VERSION-1]"
},
"database" : {
"id" : 1
},
"info" : {
"delta" : [DELTA],
"repository" : {
"delta" : [DELTA],
"size" : [SIZE]
},
"size" : [SIZE]
},
"label" : "[BACKUP-FULL-3]",
"prior" : null,
"reference" : null,
"timestamp" : {
"start" : [TIMESTAMP],
"stop" : [TIMESTAMP]
},
"type" : "full"
}
],
Improve info command to display the stanza cipher type. Contributed by Cynthia Shang. Suggested by Douglas J Hunley.
2018-09-10 19:09:45 +02:00
"cipher" : "none",
Enhanced text output of info command to include timestamps, sizes, and the reference list for all backups.
2016-05-06 16:25:46 +02:00
"db" : [
{
Fix instances where database-id was not rendered as an integer in JSON info output. Fixed by Cynthia Shang. Reported by Jason O'Donnell.
2017-10-18 14:42:32 +02:00
"id" : 1,
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
"system-id" : 1000000000000000094,
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
"version" : "9.4"
Enhanced text output of info command to include timestamps, sizes, and the reference list for all backups.
2016-05-06 16:25:46 +02:00
}
],
"name" : "db",
"status" : {
"code" : 0,
"message" : "ok"
}
}
]
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
expire full=1 (db-master host)
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --log-level-console=detail --repo1-retention-full=1 --stanza=db expire
Closed #131: Offline backups make expire command fail.
2016-01-16 06:37:17 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Add log-subprocess option to allow file logging for local and remote subprocesses.
2018-08-23 02:05:49 +02:00
P00 INFO: expire command begin [BACKREST-VERSION]: --config=[TEST_PATH]/db-master/pgbackrest.conf --lock-path=[TEST_PATH]/db-master/lock --log-level-console=detail --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --repo1-path=[TEST_PATH]/db-master/repo --repo1-retention-full=1 --stanza=db
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: expire full backup set: [BACKUP-FULL-2], [BACKUP-DIFF-2], [BACKUP-INCR-3], [BACKUP-INCR-4], [BACKUP-DIFF-3], [BACKUP-INCR-5], [BACKUP-DIFF-4]
P00 INFO: remove expired backup [BACKUP-DIFF-4]
P00 INFO: remove expired backup [BACKUP-INCR-5]
P00 INFO: remove expired backup [BACKUP-DIFF-3]
P00 INFO: remove expired backup [BACKUP-INCR-4]
P00 INFO: remove expired backup [BACKUP-INCR-3]
P00 INFO: remove expired backup [BACKUP-DIFF-2]
P00 INFO: remove expired backup [BACKUP-FULL-2]
Added a predictable log entry to signal that a command has completed successfully. For example a backup ends successfully with: INFO: backup command end: completed successfully.
2016-12-04 21:44:53 +02:00
P00 INFO: expire command end: completed successfully
Closed #131: Offline backups make expire command fail.
2016-01-16 06:37:17 +02:00
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
diff backup - add file (db-master host)
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --no-online --log-level-console=detail --checksum-page --type=diff --stanza=db backup
Log testing can now be enabled for certain deterministic tests. This works by comparing the generated logs against a previous copy. Currently only enabled for the backup/synthetic tests.
2015-05-05 19:08:48 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
P00 INFO: backup command begin [BACKREST-VERSION]: --checksum-page --compress --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --db-timeout=45 --exclude=postgresql.auto.conf --exclude=pg_log/ --exclude=pg_log2 --lock-path=[TEST_PATH]/db-master/lock --log-level-console=detail --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --no-online --pg1-path=[TEST_PATH]/db-master/db/base-2 --protocol-timeout=60 --repo1-hardlink --repo1-path=[TEST_PATH]/db-master/repo --stanza=db --start-fast --type=diff
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
P00 WARN: option repo1-retention-full is not set, the repository may run out of space
HINT: to retain full backups indefinitely (without warning), set option 'repo1-retention-full' to the maximum.
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: last backup label = [BACKUP-FULL-3], version = [VERSION-1]
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P01 INFO: backup file [TEST_PATH]/db-master/db/base-2/base/base2.txt (9B, 100%) checksum cafac3c59553f2cfde41ce2e62e7662295f108c0
P00 DETAIL: hardlink pg_data/PG_VERSION to [BACKUP-FULL-3]
P00 DETAIL: hardlink pg_data/badchecksum.txt to [BACKUP-FULL-3]
P00 DETAIL: hardlink pg_data/base/1/12000 to [BACKUP-FULL-3]
P00 DETAIL: hardlink pg_data/base/1/PG_VERSION to [BACKUP-FULL-3]
P00 DETAIL: hardlink pg_data/base/16384/17000 to [BACKUP-FULL-3]
P00 DETAIL: hardlink pg_data/base/16384/PG_VERSION to [BACKUP-FULL-3]
P00 DETAIL: hardlink pg_data/base/32768/33000 to [BACKUP-FULL-3]
P00 DETAIL: hardlink pg_data/base/32768/33000.32767 to [BACKUP-FULL-3]
Full/Synthetic test refactor. * Combine hardlink and non/compressed in synthetic tests to reduce test time and improve coverage. * Change log level of hardlink logging to detail. * Cast size in S3 manifest to integer.
2017-06-15 21:32:10 +02:00
P00 DETAIL: hardlink pg_data/base/32768/33001 to [BACKUP-FULL-3]
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
P00 DETAIL: hardlink pg_data/base/32768/44000_init to [BACKUP-FULL-3]
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P00 DETAIL: hardlink pg_data/base/32768/PG_VERSION to [BACKUP-FULL-3]
P00 DETAIL: hardlink pg_data/changecontent.txt to [BACKUP-FULL-3]
P00 DETAIL: hardlink pg_data/changetime.txt to [BACKUP-FULL-3]
Full/Synthetic test refactor. * Combine hardlink and non/compressed in synthetic tests to reduce test time and improve coverage. * Change log level of hardlink logging to detail. * Cast size in S3 manifest to integer.
2017-06-15 21:32:10 +02:00
P00 DETAIL: hardlink pg_data/global/pg_control to [BACKUP-FULL-3]
P00 DETAIL: hardlink pg_data/pg_stat/global.stat to [BACKUP-FULL-3]
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P00 DETAIL: hardlink pg_data/postgresql.conf to [BACKUP-FULL-3]
Improve support for special characters in filenames. % characters caused issues in backup/restore due to filenames being appended directly into a format string. Reserved XML characters (<>&') caused issues in the S3 driver due to improper escaping. Add a file with all common special characters to regression testing.
2018-09-10 16:54:34 +02:00
P00 DETAIL: hardlink pg_data/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?% to [BACKUP-FULL-3]
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P00 DETAIL: hardlink pg_data/zero_from_start to [BACKUP-FULL-3]
P00 DETAIL: hardlink pg_data/zerosize.txt to [BACKUP-FULL-3]
P00 DETAIL: hardlink pg_tblspc/2/[TS_PATH-1]/32768/tablespace2.txt to [BACKUP-FULL-3]
P00 DETAIL: hardlink pg_tblspc/2/[TS_PATH-1]/32768/tablespace2c.txt to [BACKUP-FULL-3]
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: diff backup size = 9B
P00 INFO: new backup label = [BACKUP-DIFF-5]
Added a predictable log entry to signal that a command has completed successfully. For example a backup ends successfully with: INFO: backup command end: completed successfully.
2016-12-04 21:44:53 +02:00
P00 INFO: backup command end: completed successfully
Move async forking and more error handling to C. The Perl process was exiting directly when called but that interfered with proper locking for the forked async process. Now Perl returns results to the C process which handles all errors, including signals.
2018-04-13 02:42:26 +02:00
P00 INFO: expire command begin
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
P00 INFO: option 'repo1-retention-archive' is not set - archive logs will not be expired
Added a predictable log entry to signal that a command has completed successfully. For example a backup ends successfully with: INFO: backup command end: completed successfully.
2016-12-04 21:44:53 +02:00
P00 INFO: expire command end: completed successfully
Added tablespace setting to allow compact database restores for development, staging, etc.
2015-05-09 00:34:27 +02:00
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
+ supplemental file: [TEST_PATH]/db-master/pgbackrest.conf
----------------------------------------------------------
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[db]
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
pg1-path=[TEST_PATH]/db-master/db/base-2
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
New simpler configuration and consistent project/exe/path naming. * The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required. * Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case. * The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally. * The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest. * Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory. * Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory. * Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 15:30:54 +02:00
[db:restore]
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
tablespace-map=1=[TEST_PATH]/db-master/db/tablespace/ts1-2
tablespace-map=2=[TEST_PATH]/db-master/db/tablespace/ts2-2
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
New simpler configuration and consistent project/exe/path naming. * The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required. * Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case. * The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally. * The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest. * Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory. * Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory. * Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 15:30:54 +02:00
[global]
Full/Synthetic test refactor. * Combine hardlink and non/compressed in synthetic tests to reduce test time and improve coverage. * Change log level of hardlink logging to detail. * Cast size in S3 manifest to integer.
2017-06-15 21:32:10 +02:00
compress=y
Fix custom settings for compress-level option being ignored. Reported by Jens Wilke.
2017-10-23 00:05:46 +02:00
compress-level=3
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
db-timeout=45
lock-path=[TEST_PATH]/db-master/lock
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
log-level-console=debug
log-level-file=trace
Added the log-level-stderr option. Controls whether console log messages are sent to stderr or stdout. By default this is set to warn which represents a change in behavior from previous versions, even though it may be more intuitive. Setting log-level-stderr=off will preserve the old behavior. Suggested by Sascha Biberhofer.
2016-10-05 15:09:30 +02:00
log-level-stderr=off
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
log-path=[TEST_PATH]/db-master/log
Add log-subprocess option to allow file logging for local and remote subprocesses.
2018-08-23 02:05:49 +02:00
log-subprocess=y
Set log-timestamp=n for integration tests. This means less filtering of logs needs to be done and new timestamps can be added without adding new filters.
2018-05-11 17:24:38 +02:00
log-timestamp=n
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
protocol-timeout=60
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
repo1-hardlink=y
repo1-path=[TEST_PATH]/db-master/repo
Consolidate archive-push tests into new archive test.
2017-06-21 14:38:53 +02:00
spool-path=[TEST_PATH]/db-master/spool
New simpler configuration and consistent project/exe/path naming. * The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required. * Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case. * The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally. * The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest. * Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory. * Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory. * Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 15:30:54 +02:00
[global:backup]
archive-copy=y
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
exclude=postgresql.auto.conf
exclude=pg_log/
exclude=pg_log2
New simpler configuration and consistent project/exe/path naming. * The repo-path option now always refers to the repository where backups and archive are stored, whether local or remote, so the repo-remote-path option has been removed. The new spool-path option can be used to define a location for queueing WAL segments when archiving asynchronously. Otherwise, a local repository is no longer required. * Implemented a new config format which should be far simpler to use. See the User Guide and Configuration Reference for details but for a simple configuration all options can now be placed in the stanza section. Options that are shared between stanzas can be placed in the [global] section. More complex configurations can still make use of command sections though this should be a rare use case. * The default configuration filename is now pgbackrest.conf instead of pg_backrest.conf. This was done for consistency with other naming changes but also to prevent old config files from being loaded accidentally. * The default repository name was changed from /var/lib/backup to /var/lib/pgbackrest. * Lock files are now stored in /tmp/pgbackrest by default. These days /run/pgbackrest would be the preferred location but that would require init scripts which are not part of this release. The lock-path option can be used to configure the lock directory. * Log files are now stored in /var/log/pgbackrest by default and no longer have the date appended so they can be managed with logrotate. The log-path option can be used to configure the lock directory. * Executable filename changed from pg_backrest to pgbackrest.
2016-04-14 15:30:54 +02:00
start-fast=y
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
+ supplemental file: [TEST_PATH]/db-master/repo/backup/db/[BACKUP-DIFF-5]/backup.manifest
-----------------------------------------------------------------------------------------
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[backrest]
backrest-checksum="[CHECKSUM]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
backrest-format=5
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
backrest-version="[VERSION-1]"
[backup]
backup-label="[BACKUP-DIFF-5]"
backup-prior="[BACKUP-FULL-3]"
backup-timestamp-copy-start=[TIMESTAMP]
backup-timestamp-start=[TIMESTAMP]
backup-timestamp-stop=[TIMESTAMP]
backup-type="diff"
[backup:db]
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-catalog-version=201409291
db-control-version=942
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
db-id=1
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
db-system-id=1000000000000000094
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-version="9.4"
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[backup:option]
option-archive-check=true
option-archive-copy=true
Backup from a standby cluster. A connection to the primary cluster is still required to start/stop the backup and copy files that are not replicated, but the vast majority of files are copied from the standby in order to reduce load on the master.
2016-08-25 17:25:46 +02:00
option-backup-standby=false
Added the --checksum-page option. This option allows pgBackRest to validate page checksums in data files when checksums are enabled on PostgreSQL >= 9.3. Note that this functionality requires a C library which may not initially be available in OS packages. The option will automatically be enabled when the library is present and checksums are enabled on the cluster.
2016-12-13 01:54:07 +02:00
option-checksum-page=false
Full/Synthetic test refactor. * Combine hardlink and non/compressed in synthetic tests to reduce test time and improve coverage. * Change log level of hardlink logging to detail. * Cast size in S3 manifest to integer.
2017-06-15 21:32:10 +02:00
option-compress=true
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
option-delta=false
Full/Synthetic test refactor. * Combine hardlink and non/compressed in synthetic tests to reduce test time and improve coverage. * Change log level of hardlink logging to detail. * Cast size in S3 manifest to integer.
2017-06-15 21:32:10 +02:00
option-hardlink=true
Closed #167: Rename `--no-start-stop` option to `--no-online`.
2016-02-20 02:32:02 +02:00
option-online=false
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[backup:target]
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
pg_data={"path":"[TEST_PATH]/db-master/db/base-2","type":"path"}
pg_tblspc/2={"path":"[TEST_PATH]/db-master/db/tablespace/ts2-2","tablespace-id":"2","tablespace-name":"ts2","type":"link"}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:file]
Full/Synthetic test refactor. * Combine hardlink and non/compressed in synthetic tests to reduce test time and improve coverage. * Change log level of hardlink logging to detail. * Cast size in S3 manifest to integer.
2017-06-15 21:32:10 +02:00
pg_data/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","master":true,"reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/badchecksum.txt={"checksum":"f927212cd08d11a42a666b2f04235398e9ceeb51","master":true,"reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/base/1/12000={"checksum":"22c98d248ff548311eda88559e4a8405ed77c003","reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/base/1/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","mode":"0660","reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/base/16384/17000={"checksum":"7579ada0808d7f98087a0a586d0df9de009cdc33","reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/base/16384/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/base/32768/33000={"checksum":"4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f","reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/base/32768/33000.32767={"checksum":"21e2c7c1a326682c07053b7d6a5a40dbd49c2ec5","reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/base/32768/33001={"checksum":"6bf316f11d28c28914ea9be92c00de9bea6d9a6b","reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
pg_data/base/32768/44000_init={"checksum":"4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f","reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Full/Synthetic test refactor. * Combine hardlink and non/compressed in synthetic tests to reduce test time and improve coverage. * Change log level of hardlink logging to detail. * Cast size in S3 manifest to integer.
2017-06-15 21:32:10 +02:00
pg_data/base/32768/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/base/base2.txt={"checksum":"cafac3c59553f2cfde41ce2e62e7662295f108c0","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
pg_data/changecontent.txt={"checksum":"a094d94583e209556d03c3c5da33131a065f1689","master":true,"reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/changetime.txt={"checksum":"88087292ed82e26f3eb824d0bffc05ccf7a30f8d","master":true,"reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-2]}
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 11:24:42 +02:00
pg_data/global/pg_control={"checksum":"4c77c900f7af0d9ab13fa9982051a42e0b637f6c","master":true,"reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-2]}
Full/Synthetic test refactor. * Combine hardlink and non/compressed in synthetic tests to reduce test time and improve coverage. * Change log level of hardlink logging to detail. * Cast size in S3 manifest to integer.
2017-06-15 21:32:10 +02:00
pg_data/pg_stat/global.stat={"checksum":"e350d5ce0153f3e22d5db21cf2a4eff00f3ee877","master":true,"reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-2]}
pg_data/postgresql.conf={"checksum":"6721d92c9fcdf4248acff1f9a1377127d9064807","master":true,"reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-2]}
Improve support for special characters in filenames. % characters caused issues in backup/restore due to filenames being appended directly into a format string. Reserved XML characters (<>&') caused issues in the S3 driver due to improper escaping. Add a file with all common special characters to regression testing.
2018-09-10 16:54:34 +02:00
pg_data/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?%={"master":true,"reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Allow zero-size files in backup manifest to reference a prior manifest regardless of timestamp delta. Contributed by Cynthia Shang.
2018-08-24 22:50:33 +02:00
pg_data/zero_from_start={"master":true,"reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Add zero-length file to mock/all test.
2018-07-05 21:40:50 +02:00
pg_data/zerosize.txt={"master":true,"reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-2]}
Full/Synthetic test refactor. * Combine hardlink and non/compressed in synthetic tests to reduce test time and improve coverage. * Change log level of hardlink logging to detail. * Cast size in S3 manifest to integer.
2017-06-15 21:32:10 +02:00
pg_tblspc/2/[TS_PATH-1]/32768/tablespace2.txt={"checksum":"dc7f76e43c46101b47acc55ae4d593a9e6983578","reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_tblspc/2/[TS_PATH-1]/32768/tablespace2c.txt={"checksum":"dfcb8679956b734706cf87259d50c88f83e80e66","reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:file:default]
Continuous integration using travis-ci.
2017-02-21 15:59:23 +02:00
group="[GROUP-1]"
Fixed a issue where tablespaces were copied from the master during standby backup.
2016-09-04 15:19:44 +02:00
master=false
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
mode="0600"
Regression tests can now be run as any properly-configured user, not just vagrant.
2016-10-01 19:39:44 +02:00
user="[USER-1]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:link]
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
pg_data/pg_tblspc/2={"destination":"[TEST_PATH]/db-master/db/tablespace/ts2-2"}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:link:default]
Continuous integration using travis-ci.
2017-02-21 15:59:23 +02:00
group="[GROUP-1]"
Regression tests can now be run as any properly-configured user, not just vagrant.
2016-10-01 19:39:44 +02:00
user="[USER-1]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:path]
pg_data={}
pg_data/base={}
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
pg_data/base/1={}
pg_data/base/16384={}
pg_data/base/32768={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_data/global={}
pg_data/pg_clog={}
Exclude directories during backup that are cleaned, recreated, or zeroed by PostgreSQL at startup. These include (depending on the version where they were introduced): pgsql_tmp, pg_dynshmem, pg_notify, pg_replslot, pg_serial, pg_snapshots, pg_stat_tmp, pg_subtrans. The postgresql.auto.conf.tmp file is now excluded in addition to files that were already excluded: backup_label.old, postmaster.opts, postmaster.pid, recovery.conf, recovery.done.
2016-08-16 15:35:16 +02:00
pg_data/pg_dynshmem={}
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
pg_data/pg_log={}
Exclude directories during backup that are cleaned, recreated, or zeroed by PostgreSQL at startup. These include (depending on the version where they were introduced): pgsql_tmp, pg_dynshmem, pg_notify, pg_replslot, pg_serial, pg_snapshots, pg_stat_tmp, pg_subtrans. The postgresql.auto.conf.tmp file is now excluded in addition to files that were already excluded: backup_label.old, postmaster.opts, postmaster.pid, recovery.conf, recovery.done.
2016-08-16 15:35:16 +02:00
pg_data/pg_notify={}
pg_data/pg_replslot={}
pg_data/pg_serial={}
pg_data/pg_snapshots={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_data/pg_stat={}
Exclude directories during backup that are cleaned, recreated, or zeroed by PostgreSQL at startup. These include (depending on the version where they were introduced): pgsql_tmp, pg_dynshmem, pg_notify, pg_replslot, pg_serial, pg_snapshots, pg_stat_tmp, pg_subtrans. The postgresql.auto.conf.tmp file is now excluded in addition to files that were already excluded: backup_label.old, postmaster.opts, postmaster.pid, recovery.conf, recovery.done.
2016-08-16 15:35:16 +02:00
pg_data/pg_stat_tmp={}
pg_data/pg_subtrans={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
pg_data/pg_tblspc={}
pg_tblspc={}
pg_tblspc/2={}
pg_tblspc/2/[TS_PATH-1]={}
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
pg_tblspc/2/[TS_PATH-1]/32768={}
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[target:path:default]
Continuous integration using travis-ci.
2017-02-21 15:59:23 +02:00
group="[GROUP-1]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
mode="0700"
Regression tests can now be run as any properly-configured user, not just vagrant.
2016-10-01 19:39:44 +02:00
user="[USER-1]"
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
+ supplemental file: [TEST_PATH]/db-master/repo/backup/db/backup.info
---------------------------------------------------------------------
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[backrest]
backrest-checksum="[CHECKSUM]"
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
backrest-format=5
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
backrest-version="[VERSION-1]"
[backup:current]
Full/Synthetic test refactor. * Combine hardlink and non/compressed in synthetic tests to reduce test time and improve coverage. * Change log level of hardlink logging to detail. * Cast size in S3 manifest to integer.
2017-06-15 21:32:10 +02:00
[BACKUP-FULL-3]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"full","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":false,"option-compress":true,"option-hardlink":true,"option-online":false}
[BACKUP-DIFF-5]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-prior":"[BACKUP-FULL-3]","backup-reference":["[BACKUP-FULL-3]"],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"diff","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":false,"option-compress":true,"option-hardlink":true,"option-online":false}
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[db]
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-catalog-version=201409291
db-control-version=942
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
db-id=1
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
db-system-id=1000000000000000094
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
db-version="9.4"
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
[db:history]
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
1={"db-catalog-version":201409291,"db-control-version":942,"db-system-id":1000000000000000094,"db-version":"9.4"}
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
restore delta, remap - selective restore 16384 (db-master host)
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --delta --log-level-console=detail --db-include=16384 --stanza=db restore
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Add log-subprocess option to allow file logging for local and remote subprocesses.
2018-08-23 02:05:49 +02:00
P00 INFO: restore command begin [BACKREST-VERSION]: --compress --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --db-include=16384 --delta --lock-path=[TEST_PATH]/db-master/lock --log-level-console=detail --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --pg1-path=[TEST_PATH]/db-master/db/base-2 --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --stanza=db --tablespace-map=2=[TEST_PATH]/db-master/db/tablespace/ts2-2
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P00 INFO: restore backup set [BACKUP-DIFF-5]
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: remap tablespace pg_tblspc/2 directory to [TEST_PATH]/db-master/db/tablespace/ts2-2
P00 DETAIL: check [TEST_PATH]/db-master/db/base-2 exists
P00 DETAIL: check [TEST_PATH]/db-master/db/tablespace/ts2-2 exists
P00 INFO: remove invalid files/paths/links from [TEST_PATH]/db-master/db/tablespace/ts2-2/[TS_PATH-1]
P00 INFO: remove invalid files/paths/links from [TEST_PATH]/db-master/db/base-2
P00 DETAIL: preserve file [TEST_PATH]/db-master/db/base-2/recovery.conf
P00 DETAIL: databases for include/exclude (1, 16384, 32768)
P00 DETAIL: database filter: (^pg_data\/base\/32768\/)|(^pg_tblspc/2\/[TS_PATH-1]\/32768\/)
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
P01 DETAIL: restore zeroed file [TEST_PATH]/db-master/db/base-2/base/32768/33001 (64KB, 36%)
P01 DETAIL: restore zeroed file [TEST_PATH]/db-master/db/base-2/base/32768/44000_init (32KB, 54%)
P01 DETAIL: restore zeroed file [TEST_PATH]/db-master/db/base-2/base/32768/33000.32767 (32KB, 72%)
P01 DETAIL: restore zeroed file [TEST_PATH]/db-master/db/base-2/base/32768/33000 (32KB, 90%)
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 11:24:42 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/global/pg_control.pgbackrest.tmp (8KB, 95%) checksum 4c77c900f7af0d9ab13fa9982051a42e0b637f6c
Add more realistic data files to synthetic backup and restore tests.
2016-12-10 16:13:55 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base-2/base/1/12000 - exists and matches backup (8KB, 99%) checksum 22c98d248ff548311eda88559e4a8405ed77c003
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base-2/postgresql.conf - exists and matches backup (21B, 99%) checksum 6721d92c9fcdf4248acff1f9a1377127d9064807
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base-2/badchecksum.txt - exists and matches backup (11B, 99%) checksum f927212cd08d11a42a666b2f04235398e9ceeb51
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base-2/base/base2.txt - exists and matches backup (9B, 99%) checksum cafac3c59553f2cfde41ce2e62e7662295f108c0
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base-2/base/16384/17000 - exists and matches backup (9B, 99%) checksum 7579ada0808d7f98087a0a586d0df9de009cdc33
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base-2/changecontent.txt - exists and matches backup (7B, 99%) checksum a094d94583e209556d03c3c5da33131a065f1689
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base-2/pg_stat/global.stat - exists and matches backup (5B, 99%) checksum e350d5ce0153f3e22d5db21cf2a4eff00f3ee877
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base-2/changetime.txt - exists and matches backup (4B, 99%) checksum 88087292ed82e26f3eb824d0bffc05ccf7a30f8d
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base-2/base/32768/PG_VERSION - exists and matches backup (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base-2/base/16384/PG_VERSION - exists and matches backup (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base-2/base/1/PG_VERSION - exists and matches backup (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base-2/PG_VERSION - exists and matches backup (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
Add zero-length file to mock/all test.
2018-07-05 21:40:50 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base-2/zerosize.txt - exists and is zero size (0B, 99%)
Allow zero-size files in backup manifest to reference a prior manifest regardless of timestamp delta. Contributed by Cynthia Shang.
2018-08-24 22:50:33 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base-2/zero_from_start - exists and is zero size (0B, 99%)
Improve support for special characters in filenames. % characters caused issues in backup/restore due to filenames being appended directly into a format string. Reserved XML characters (<>&') caused issues in the S3 driver due to improper escaping. Add a file with all common special characters to regression testing.
2018-09-10 16:54:34 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base-2/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?% - exists and is zero size (0B, 99%)
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 DETAIL: restore zeroed file [TEST_PATH]/db-master/db/base-2/pg_tblspc/2/[TS_PATH-1]/32768/tablespace2c.txt (12B, 99%)
Additional backup exclusions. * Exclude contents of pg_snapshots, pg_serial, pg_notify, and pg_dynshmem from backup since they are rebuilt on startup. * Exclude pg_internal.init files from backup since they are rebuilt on startup.
2017-09-04 14:26:57 +02:00
P01 DETAIL: restore zeroed file [TEST_PATH]/db-master/db/base-2/pg_tblspc/2/[TS_PATH-1]/32768/tablespace2.txt (7B, 100%)
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P00 INFO: write [TEST_PATH]/db-master/db/base-2/recovery.conf
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 INFO: restore global/pg_control (performed last to ensure aborted restores cannot be started)
Added a predictable log entry to signal that a command has completed successfully. For example a backup ends successfully with: INFO: backup command end: completed successfully.
2016-12-04 21:44:53 +02:00
P00 INFO: restore command end: completed successfully
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
+ supplemental file: [TEST_PATH]/db-master/db/base-2/recovery.conf
------------------------------------------------------------------
restore_command = '[BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --log-level-console=detail --stanza=db archive-get %f "%p"'
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
restore delta, remap - selective restore 32768 (db-master host)
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --delta --log-level-console=detail --db-include=32768 --stanza=db restore
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Add log-subprocess option to allow file logging for local and remote subprocesses.
2018-08-23 02:05:49 +02:00
P00 INFO: restore command begin [BACKREST-VERSION]: --compress --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --db-include=32768 --delta --lock-path=[TEST_PATH]/db-master/lock --log-level-console=detail --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --pg1-path=[TEST_PATH]/db-master/db/base-2 --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --stanza=db --tablespace-map=2=[TEST_PATH]/db-master/db/tablespace/ts2-2
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P00 INFO: restore backup set [BACKUP-DIFF-5]
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: remap tablespace pg_tblspc/2 directory to [TEST_PATH]/db-master/db/tablespace/ts2-2
P00 DETAIL: check [TEST_PATH]/db-master/db/base-2 exists
P00 DETAIL: check [TEST_PATH]/db-master/db/tablespace/ts2-2 exists
P00 INFO: remove invalid files/paths/links from [TEST_PATH]/db-master/db/tablespace/ts2-2/[TS_PATH-1]
P00 INFO: remove invalid files/paths/links from [TEST_PATH]/db-master/db/base-2
P00 DETAIL: preserve file [TEST_PATH]/db-master/db/base-2/recovery.conf
P00 DETAIL: databases for include/exclude (1, 16384, 32768)
P00 DETAIL: database filter: (^pg_data\/base\/16384\/)|(^pg_tblspc/2\/[TS_PATH-1]\/16384\/)
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/base/32768/33001 (64KB, 36%) checksum 6bf316f11d28c28914ea9be92c00de9bea6d9a6b
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/base/32768/44000_init (32KB, 54%) checksum 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/base/32768/33000.32767 (32KB, 72%) checksum 21e2c7c1a326682c07053b7d6a5a40dbd49c2ec5
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/base/32768/33000 (32KB, 90%) checksum 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 11:24:42 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/global/pg_control.pgbackrest.tmp (8KB, 95%) checksum 4c77c900f7af0d9ab13fa9982051a42e0b637f6c
Add more realistic data files to synthetic backup and restore tests.
2016-12-10 16:13:55 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base-2/base/1/12000 - exists and matches backup (8KB, 99%) checksum 22c98d248ff548311eda88559e4a8405ed77c003
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base-2/postgresql.conf - exists and matches backup (21B, 99%) checksum 6721d92c9fcdf4248acff1f9a1377127d9064807
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base-2/badchecksum.txt - exists and matches backup (11B, 99%) checksum f927212cd08d11a42a666b2f04235398e9ceeb51
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base-2/base/base2.txt - exists and matches backup (9B, 99%) checksum cafac3c59553f2cfde41ce2e62e7662295f108c0
P01 DETAIL: restore zeroed file [TEST_PATH]/db-master/db/base-2/base/16384/17000 (9B, 99%)
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base-2/changecontent.txt - exists and matches backup (7B, 99%) checksum a094d94583e209556d03c3c5da33131a065f1689
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base-2/pg_stat/global.stat - exists and matches backup (5B, 99%) checksum e350d5ce0153f3e22d5db21cf2a4eff00f3ee877
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base-2/changetime.txt - exists and matches backup (4B, 99%) checksum 88087292ed82e26f3eb824d0bffc05ccf7a30f8d
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base-2/base/32768/PG_VERSION - exists and matches backup (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base-2/base/16384/PG_VERSION - exists and matches backup (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base-2/base/1/PG_VERSION - exists and matches backup (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base-2/PG_VERSION - exists and matches backup (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
Add zero-length file to mock/all test.
2018-07-05 21:40:50 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base-2/zerosize.txt - exists and is zero size (0B, 99%)
Allow zero-size files in backup manifest to reference a prior manifest regardless of timestamp delta. Contributed by Cynthia Shang.
2018-08-24 22:50:33 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base-2/zero_from_start - exists and is zero size (0B, 99%)
Improve support for special characters in filenames. % characters caused issues in backup/restore due to filenames being appended directly into a format string. Reserved XML characters (<>&') caused issues in the S3 driver due to improper escaping. Add a file with all common special characters to regression testing.
2018-09-10 16:54:34 +02:00
P01 DETAIL: restore file [TEST_PATH]/db-master/db/base-2/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?% - exists and is zero size (0B, 99%)
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/pg_tblspc/2/[TS_PATH-1]/32768/tablespace2c.txt (12B, 99%) checksum dfcb8679956b734706cf87259d50c88f83e80e66
Additional backup exclusions. * Exclude contents of pg_snapshots, pg_serial, pg_notify, and pg_dynshmem from backup since they are rebuilt on startup. * Exclude pg_internal.init files from backup since they are rebuilt on startup.
2017-09-04 14:26:57 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/pg_tblspc/2/[TS_PATH-1]/32768/tablespace2.txt (7B, 100%) checksum dc7f76e43c46101b47acc55ae4d593a9e6983578
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P00 INFO: write [TEST_PATH]/db-master/db/base-2/recovery.conf
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 INFO: restore global/pg_control (performed last to ensure aborted restores cannot be started)
Added a predictable log entry to signal that a command has completed successfully. For example a backup ends successfully with: INFO: backup command end: completed successfully.
2016-12-04 21:44:53 +02:00
P00 INFO: restore command end: completed successfully
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
+ supplemental file: [TEST_PATH]/db-master/db/base-2/recovery.conf
------------------------------------------------------------------
restore_command = '[BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --log-level-console=detail --stanza=db archive-get %f "%p"'
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
Renumber return codes between 25 and 125 to avoid PostgreSQL interpreting some as fatal signal exceptions. Suggested by Yogesh Sharma.
2017-04-12 18:41:34 +02:00
restore delta, remap, expect exit 80 - error on invalid id (db-master host)
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --delta --log-level-console=warn --db-include=7777 --stanza=db restore
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Renumber return codes between 25 and 125 to avoid PostgreSQL interpreting some as fatal signal exceptions. Suggested by Yogesh Sharma.
2017-04-12 18:41:34 +02:00
P00 ERROR: [080]: database to include '7777' does not exist
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
Renumber return codes between 25 and 125 to avoid PostgreSQL interpreting some as fatal signal exceptions. Suggested by Yogesh Sharma.
2017-04-12 18:41:34 +02:00
restore delta, remap, expect exit 81 - error on system id (db-master host)
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --delta --log-level-console=warn --db-include=1 --stanza=db restore
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Renumber return codes between 25 and 125 to avoid PostgreSQL interpreting some as fatal signal exceptions. Suggested by Yogesh Sharma.
2017-04-12 18:41:34 +02:00
P00 ERROR: [081]: system databases (template0, postgres, etc.) are included by default
Allow selective restore of databases from a cluster backup. This feature can result in major space and time savings when only specific databases are restored. Unrestored databases will not be accessible but must be manually dropped before they will be removed from the shared catalogue.
2016-05-11 15:21:39 +02:00
Renumber return codes between 25 and 125 to avoid PostgreSQL interpreting some as fatal signal exceptions. Suggested by Yogesh Sharma.
2017-04-12 18:41:34 +02:00
restore, remap, expect exit 73 - no tablespace remap - error when tablespace dir does not exist (db-master host)
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --log-level-console=detail --tablespace-map-all=../../tablespace --stanza=db restore
Added tablespace setting to allow compact database restores for development, staging, etc.
2015-05-09 00:34:27 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Add log-subprocess option to allow file logging for local and remote subprocesses.
2018-08-23 02:05:49 +02:00
P00 INFO: restore command begin [BACKREST-VERSION]: --compress --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --lock-path=[TEST_PATH]/db-master/lock --log-level-console=detail --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --pg1-path=[TEST_PATH]/db-master/db/base-2/base --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --stanza=db --tablespace-map-all=../../tablespace
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P00 INFO: restore backup set [BACKUP-DIFF-5]
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: remap $PGDATA directory to [TEST_PATH]/db-master/db/base-2/base
P00 INFO: remap tablespace pg_tblspc/2 directory to ../../tablespace/ts2
P00 DETAIL: check [TEST_PATH]/db-master/db/base-2/base exists
P00 DETAIL: check [TEST_PATH]/db-master/db/base-2/tablespace exists
Renumber return codes between 25 and 125 to avoid PostgreSQL interpreting some as fatal signal exceptions. Suggested by Yogesh Sharma.
2017-04-12 18:41:34 +02:00
P00 ERROR: [073]: cannot restore to missing path [TEST_PATH]/db-master/db/base-2/tablespace
P00 INFO: restore command end: aborted with exception [073]
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
restore - no tablespace remap (db-master host)
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --tablespace-map-all=../../tablespace --log-level-console=detail --stanza=db restore
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Add log-subprocess option to allow file logging for local and remote subprocesses.
2018-08-23 02:05:49 +02:00
P00 INFO: restore command begin [BACKREST-VERSION]: --compress --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --lock-path=[TEST_PATH]/db-master/lock --log-level-console=detail --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --pg1-path=[TEST_PATH]/db-master/db/base-2/base --protocol-timeout=60 --repo1-path=[TEST_PATH]/db-master/repo --stanza=db --tablespace-map-all=../../tablespace
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P00 INFO: restore backup set [BACKUP-DIFF-5]
Show Process ID in log output instead of filtering it out with the timestamp.
2016-09-14 23:30:57 +02:00
P00 INFO: remap $PGDATA directory to [TEST_PATH]/db-master/db/base-2/base
P00 INFO: remap tablespace pg_tblspc/2 directory to ../../tablespace/ts2
P00 DETAIL: check [TEST_PATH]/db-master/db/base-2/base exists
P00 DETAIL: check [TEST_PATH]/db-master/db/base-2/tablespace exists
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/base/base/32768/33001 (64KB, 36%) checksum 6bf316f11d28c28914ea9be92c00de9bea6d9a6b
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/base/base/32768/44000_init (32KB, 54%) checksum 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/base/base/32768/33000.32767 (32KB, 72%) checksum 21e2c7c1a326682c07053b7d6a5a40dbd49c2ec5
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/base/base/32768/33000 (32KB, 90%) checksum 4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 11:24:42 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/base/global/pg_control.pgbackrest.tmp (8KB, 95%) checksum 4c77c900f7af0d9ab13fa9982051a42e0b637f6c
Add more realistic data files to synthetic backup and restore tests.
2016-12-10 16:13:55 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/base/base/1/12000 (8KB, 99%) checksum 22c98d248ff548311eda88559e4a8405ed77c003
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/base/postgresql.conf (21B, 99%) checksum 6721d92c9fcdf4248acff1f9a1377127d9064807
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/base/badchecksum.txt (11B, 99%) checksum f927212cd08d11a42a666b2f04235398e9ceeb51
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/base/base/base2.txt (9B, 99%) checksum cafac3c59553f2cfde41ce2e62e7662295f108c0
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/base/base/16384/17000 (9B, 99%) checksum 7579ada0808d7f98087a0a586d0df9de009cdc33
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/base/changecontent.txt (7B, 99%) checksum a094d94583e209556d03c3c5da33131a065f1689
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/base/pg_stat/global.stat (5B, 99%) checksum e350d5ce0153f3e22d5db21cf2a4eff00f3ee877
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/base/changetime.txt (4B, 99%) checksum 88087292ed82e26f3eb824d0bffc05ccf7a30f8d
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/base/base/32768/PG_VERSION (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/base/base/16384/PG_VERSION (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/base/base/1/PG_VERSION (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/base/PG_VERSION (3B, 99%) checksum 184473f470864e067ee3a22e64b47b0a1c356f29
Add zero-length file to mock/all test.
2018-07-05 21:40:50 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/base/zerosize.txt (0B, 99%)
Allow zero-size files in backup manifest to reference a prior manifest regardless of timestamp delta. Contributed by Cynthia Shang.
2018-08-24 22:50:33 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/base/zero_from_start (0B, 99%)
Improve support for special characters in filenames. % characters caused issues in backup/restore due to filenames being appended directly into a format string. Reserved XML characters (<>&') caused issues in the S3 driver due to improper escaping. Add a file with all common special characters to regression testing.
2018-09-10 16:54:34 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/base/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?% (0B, 99%)
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/base/pg_tblspc/2/[TS_PATH-1]/32768/tablespace2c.txt (12B, 99%) checksum dfcb8679956b734706cf87259d50c88f83e80e66
Additional backup exclusions. * Exclude contents of pg_snapshots, pg_serial, pg_notify, and pg_dynshmem from backup since they are rebuilt on startup. * Exclude pg_internal.init files from backup since they are rebuilt on startup.
2017-09-04 14:26:57 +02:00
P01 INFO: restore file [TEST_PATH]/db-master/db/base-2/base/pg_tblspc/2/[TS_PATH-1]/32768/tablespace2.txt (7B, 100%) checksum dc7f76e43c46101b47acc55ae4d593a9e6983578
For simplicity, the pg_control file is now copied with the rest of the files instead of by itself of at the end of the process. The backup command does not require this behavior and the restore copies to a temporary file which is renamed at the end of the restore.
2016-11-30 21:23:05 +02:00
P00 INFO: write [TEST_PATH]/db-master/db/base-2/base/recovery.conf
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
P00 INFO: restore global/pg_control (performed last to ensure aborted restores cannot be started)
Added a predictable log entry to signal that a command has completed successfully. For example a backup ends successfully with: INFO: backup command end: completed successfully.
2016-12-04 21:44:53 +02:00
P00 INFO: restore command end: completed successfully
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
+ supplemental file: [TEST_PATH]/db-master/db/base-2/base/recovery.conf
-----------------------------------------------------------------------
restore_command = '[BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --log-level-console=detail --stanza=db archive-get %f "%p"'
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
info all stanzas - normal output (db-master host)
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --log-level-console=warn info
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Enhanced text output of info command to include timestamps, sizes, and the reference list for all backups.
2016-05-06 16:25:46 +02:00
stanza: db
Change spacing in expect logs to accommodate DETAIL level.
2016-04-14 00:55:34 +02:00
status: ok
Improve info command to display the stanza cipher type. Contributed by Cynthia Shang. Suggested by Douglas J Hunley.
2018-09-10 19:09:45 +02:00
cipher: none
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
Improved WAL data returned by info command. Modified the info command (both text and JSON output) to display the archive ID and minimum/maximum WAL currently present in the archive for the current and prior, if any, database cluster version. Contributed by Cynthia Shang.
2017-07-21 23:32:41 +02:00
db (current)
wal archive min/max (9.4-1): none present
Enhanced text output of info command to include timestamps, sizes, and the reference list for all backups.
2016-05-06 16:25:46 +02:00
Improved WAL data returned by info command. Modified the info command (both text and JSON output) to display the archive ID and minimum/maximum WAL currently present in the archive for the current and prior, if any, database cluster version. Contributed by Cynthia Shang.
2017-07-21 23:32:41 +02:00
full backup: [BACKUP-FULL-3]
timestamp start/stop: [TIMESTAMP-STR]
wal start/stop: n/a
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
database size: 176KB, backup size: 176KB
The info command is implemented entirely in C. The C info code has already been committed but this commit wires it into main. Also remove the info Perl code and tests since they are no longer called.
2019-01-21 13:51:45 +02:00
repository size: 2.4KB, repository backup size: 2.4KB
Improved WAL data returned by info command. Modified the info command (both text and JSON output) to display the archive ID and minimum/maximum WAL currently present in the archive for the current and prior, if any, database cluster version. Contributed by Cynthia Shang.
2017-07-21 23:32:41 +02:00
diff backup: [BACKUP-DIFF-5]
timestamp start/stop: [TIMESTAMP-STR]
wal start/stop: n/a
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
database size: 176KB, backup size: 9B
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
repository size: 2.4KB, repository backup size: 29B
Improved WAL data returned by info command. Modified the info command (both text and JSON output) to display the archive ID and minimum/maximum WAL currently present in the archive for the current and prior, if any, database cluster version. Contributed by Cynthia Shang.
2017-07-21 23:32:41 +02:00
backup reference list: [BACKUP-FULL-3]
Enhanced text output of info command to include timestamps, sizes, and the reference list for all backups.
2016-05-06 16:25:46 +02:00
stanza: db_empty
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
status: error (missing stanza data)
Improve info command to display the stanza cipher type. Contributed by Cynthia Shang. Suggested by Douglas J Hunley.
2018-09-10 19:09:45 +02:00
cipher: none
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
info all stanzas - normal output (db-master host)
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --log-level-console=warn --output=json info
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
------------------------------------------------------------------------------------------------------------------------------------
[
Change spacing in expect logs to accommodate DETAIL level.
2016-04-14 00:55:34 +02:00
{
Improved WAL data returned by info command. Modified the info command (both text and JSON output) to display the archive ID and minimum/maximum WAL currently present in the archive for the current and prior, if any, database cluster version. Contributed by Cynthia Shang.
2017-07-21 23:32:41 +02:00
"archive" : [
{
"database" : {
Fix instances where database-id was not rendered as an integer in JSON info output. Fixed by Cynthia Shang. Reported by Jason O'Donnell.
2017-10-18 14:42:32 +02:00
"id" : 1
Improved WAL data returned by info command. Modified the info command (both text and JSON output) to display the archive ID and minimum/maximum WAL currently present in the archive for the current and prior, if any, database cluster version. Contributed by Cynthia Shang.
2017-07-21 23:32:41 +02:00
},
"id" : "9.4-1",
"max" : null,
"min" : null
}
],
Change spacing in expect logs to accommodate DETAIL level.
2016-04-14 00:55:34 +02:00
"backup" : [
{
"archive" : {
"start" : null,
"stop" : null
},
"backrest" : {
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
"format" : 5,
Change spacing in expect logs to accommodate DETAIL level.
2016-04-14 00:55:34 +02:00
"version" : "[VERSION-1]"
},
"database" : {
"id" : 1
},
"info" : {
"delta" : [DELTA],
"repository" : {
"delta" : [DELTA],
"size" : [SIZE]
},
"size" : [SIZE]
},
"label" : "[BACKUP-FULL-3]",
"prior" : null,
"reference" : null,
"timestamp" : {
"start" : [TIMESTAMP],
"stop" : [TIMESTAMP]
},
"type" : "full"
},
{
"archive" : {
"start" : null,
"stop" : null
},
"backrest" : {
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
"format" : 5,
Change spacing in expect logs to accommodate DETAIL level.
2016-04-14 00:55:34 +02:00
"version" : "[VERSION-1]"
},
"database" : {
"id" : 1
},
"info" : {
"delta" : [DELTA],
"repository" : {
"delta" : [DELTA],
"size" : [SIZE]
},
"size" : [SIZE]
},
"label" : "[BACKUP-DIFF-5]",
"prior" : "[BACKUP-FULL-3]",
"reference" : [
"[BACKUP-FULL-3]"
],
"timestamp" : {
"start" : [TIMESTAMP],
"stop" : [TIMESTAMP]
},
"type" : "diff"
}
],
Improve info command to display the stanza cipher type. Contributed by Cynthia Shang. Suggested by Douglas J Hunley.
2018-09-10 19:09:45 +02:00
"cipher" : "none",
Change spacing in expect logs to accommodate DETAIL level.
2016-04-14 00:55:34 +02:00
"db" : [
{
Fix instances where database-id was not rendered as an integer in JSON info output. Fixed by Cynthia Shang. Reported by Jason O'Donnell.
2017-10-18 14:42:32 +02:00
"id" : 1,
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
"system-id" : 1000000000000000094,
Database version refactoring. * Refactor db version constants into a separate module. * Update synthetic backup tests to PostgreSQL 9.4.
2016-08-12 04:35:24 +02:00
"version" : "9.4"
Change spacing in expect logs to accommodate DETAIL level.
2016-04-14 00:55:34 +02:00
}
],
"name" : "db",
"status" : {
"code" : 0,
"message" : "ok"
}
},
{
Improved WAL data returned by info command. Modified the info command (both text and JSON output) to display the archive ID and minimum/maximum WAL currently present in the archive for the current and prior, if any, database cluster version. Contributed by Cynthia Shang.
2017-07-21 23:32:41 +02:00
"archive" : [],
Change spacing in expect logs to accommodate DETAIL level.
2016-04-14 00:55:34 +02:00
"backup" : [],
Improve info command to display the stanza cipher type. Contributed by Cynthia Shang. Suggested by Douglas J Hunley.
2018-09-10 19:09:45 +02:00
"cipher" : "none",
Change spacing in expect logs to accommodate DETAIL level.
2016-04-14 00:55:34 +02:00
"db" : [],
"name" : "db_empty",
"status" : {
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
"code" : 3,
"message" : "missing stanza data"
Change spacing in expect logs to accommodate DETAIL level.
2016-04-14 00:55:34 +02:00
}
}
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
]
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
info bogus stanza - bogus stanza (db-master host)
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --log-level-console=warn --stanza=bogus info
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Enhanced text output of info command to include timestamps, sizes, and the reference list for all backups.
2016-05-06 16:25:46 +02:00
stanza: bogus
Change spacing in expect logs to accommodate DETAIL level.
2016-04-14 00:55:34 +02:00
status: error (missing stanza path)
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
info bogus stanza - bogus stanza (db-master host)
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --log-level-console=warn --stanza=bogus --output=json info
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
------------------------------------------------------------------------------------------------------------------------------------
[
Change spacing in expect logs to accommodate DETAIL level.
2016-04-14 00:55:34 +02:00
{
"backup" : [],
"db" : [],
"name" : "bogus",
"status" : {
"code" : 1,
"message" : "missing stanza path"
}
}
Implemented issue #26: Info command. * Includes updating the manifest to format 4. It turns out the manifest and .info files were not very good for providing information. A format update was required anyway so worked through the backlog of changes that would require a format change. * Multiple database versions are now supported in the archive. Does't actually work yet but the structure should be good. * Tests use more constants now that test logs can catch name regressions.
2015-06-14 00:25:49 +02:00
]
Fix issue #137: Info command throws an exception if no stanzas exist.
2015-09-08 22:44:47 +02:00
Make ls ordering deterministic in mock/all test.
2018-07-02 02:02:37 +02:00
> ls -1Rtr [TEST_PATH]/db-master/repo/backup/db/backup.history
Fix issue #137: Info command throws an exception if no stanzas exist.
2015-09-08 22:44:47 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
[TEST_PATH]/db-master/repo/backup/db/backup.history:
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[YEAR-1]
Rename test paths for clarity. This was worked out as part of the test suite refactor [c8f806a] but not committed with it because of the large number of expect logs changes involved. Keeping them separate made it easier to audit the changes in the refactor.
2016-06-24 16:54:31 +02:00
[TEST_PATH]/db-master/repo/backup/db/backup.history/[YEAR-1]:
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[BACKUP-FULL-1].manifest.gz
Make ls ordering deterministic in mock/all test.
2018-07-02 02:02:37 +02:00
[BACKUP-FULL-2].manifest.gz
New manifest format. * All files and directories linked from PGDATA are now included in the backup. By default links will be restored directly into PGDATA as files or directories. The --link-all option can be used to restore all links to their original locations. The --link-map option can be used to remap a link to a new location. * Removed --tablespace option and replaced with --tablespace-map-all option which should more clearly indicate its function. * Added detail log level which will output more information than info without being as verbose as debug.
2016-04-15 04:50:02 +02:00
[BACKUP-INCR-1].manifest.gz
[BACKUP-INCR-2].manifest.gz
[BACKUP-DIFF-1].manifest.gz
[BACKUP-DIFF-2].manifest.gz
[BACKUP-INCR-3].manifest.gz
[BACKUP-INCR-4].manifest.gz
[BACKUP-DIFF-3].manifest.gz
[BACKUP-INCR-5].manifest.gz
[BACKUP-DIFF-4].manifest.gz
Major refactor of the test suite. * Make the code more modular and object-oriented. * Multiple Docker containers can now be created for a single test to simulate more realistic environments.
2016-06-24 14:12:58 +02:00
[BACKUP-FULL-3].manifest.gz
Make ls ordering deterministic in mock/all test.
2018-07-02 02:02:37 +02:00
[BACKUP-DIFF-5].manifest.gz
Fixed the backup command so the backup-standby option is reset (and the backup proceeds on the master) if the standby is not configured and/or reachable. Fixed by Cynthia Shang.
2017-05-12 22:05:44 +02:00
Fixed config warnings raised from a remote process causing errors in the master process. Fixed by Cynthia Shang.
2017-05-16 17:28:51 +02:00
diff backup - config file warning on local (db-master host)
Full/Synthetic test refactor. * Combine hardlink and non/compressed in synthetic tests to reduce test time and improve coverage. * Change log level of hardlink logging to detail. * Cast size in S3 manifest to integer.
2017-06-15 21:32:10 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --no-online --log-level-console=info 2>&1 --type=diff --stanza=db backup
Fixed config warnings raised from a remote process causing errors in the master process. Fixed by Cynthia Shang.
2017-05-16 17:28:51 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Add support for additional pgBackRest configuration files. Configuration files are loaded from the directory specified by the --config-include-path option. Add --config-path option for overriding the default base path of the --config and --config-include-path option. Contributed by Cynthia Shang.
2018-04-19 17:09:39 +02:00
WARN: configuration file contains invalid option 'bogus'
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
P00 INFO: backup command begin [BACKREST-VERSION]: --compress --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --db-timeout=45 --exclude=postgresql.auto.conf --exclude=pg_log/ --exclude=pg_log2 --lock-path=[TEST_PATH]/db-master/lock --log-level-console=info --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --no-online --pg1-path=[TEST_PATH]/db-master/db/base-2/base --protocol-timeout=60 --repo1-hardlink --repo1-path=[TEST_PATH]/db-master/repo --stanza=db --start-fast --type=diff
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
P00 WARN: option repo1-retention-full is not set, the repository may run out of space
HINT: to retain full backups indefinitely (without warning), set option 'repo1-retention-full' to the maximum.
Fixed config warnings raised from a remote process causing errors in the master process. Fixed by Cynthia Shang.
2017-05-16 17:28:51 +02:00
P00 INFO: last backup label = [BACKUP-FULL-3], version = [VERSION-1]
P01 INFO: backup file [TEST_PATH]/db-master/db/base-2/base/base/base2.txt (9B, 100%) checksum cafac3c59553f2cfde41ce2e62e7662295f108c0
P00 INFO: diff backup size = 9B
P00 INFO: new backup label = [BACKUP-DIFF-6]
P00 INFO: backup command end: completed successfully
Move async forking and more error handling to C. The Perl process was exiting directly when called but that interfered with proper locking for the forked async process. Now Perl returns results to the C process which handles all errors, including signals.
2018-04-13 02:42:26 +02:00
P00 INFO: expire command begin
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
P00 INFO: option 'repo1-retention-archive' is not set - archive logs will not be expired
Fixed config warnings raised from a remote process causing errors in the master process. Fixed by Cynthia Shang.
2017-05-16 17:28:51 +02:00
P00 INFO: expire command end: completed successfully
+ supplemental file: [TEST_PATH]/db-master/pgbackrest.conf
----------------------------------------------------------
[db]
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
pg1-path=[TEST_PATH]/db-master/db/base-2/base
Fixed config warnings raised from a remote process causing errors in the master process. Fixed by Cynthia Shang.
2017-05-16 17:28:51 +02:00
[db:restore]
[global]
Full/Synthetic test refactor. * Combine hardlink and non/compressed in synthetic tests to reduce test time and improve coverage. * Change log level of hardlink logging to detail. * Cast size in S3 manifest to integer.
2017-06-15 21:32:10 +02:00
compress=y
Fix custom settings for compress-level option being ignored. Reported by Jens Wilke.
2017-10-23 00:05:46 +02:00
compress-level=3
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
db-timeout=45
lock-path=[TEST_PATH]/db-master/lock
Fixed config warnings raised from a remote process causing errors in the master process. Fixed by Cynthia Shang.
2017-05-16 17:28:51 +02:00
log-level-console=debug
log-level-file=trace
log-level-stderr=off
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
log-path=[TEST_PATH]/db-master/log
Add log-subprocess option to allow file logging for local and remote subprocesses.
2018-08-23 02:05:49 +02:00
log-subprocess=y
Set log-timestamp=n for integration tests. This means less filtering of logs needs to be done and new timestamps can be added without adding new filters.
2018-05-11 17:24:38 +02:00
log-timestamp=n
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
protocol-timeout=60
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
repo1-hardlink=y
repo1-path=[TEST_PATH]/db-master/repo
Consolidate archive-push tests into new archive test.
2017-06-21 14:38:53 +02:00
spool-path=[TEST_PATH]/db-master/spool
Fixed config warnings raised from a remote process causing errors in the master process. Fixed by Cynthia Shang.
2017-05-16 17:28:51 +02:00
[global:backup]
archive-copy=y
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
exclude=postgresql.auto.conf
exclude=pg_log/
exclude=pg_log2
Fixed config warnings raised from a remote process causing errors in the master process. Fixed by Cynthia Shang.
2017-05-16 17:28:51 +02:00
start-fast=y
bogus=bogus
+ supplemental file: [TEST_PATH]/db-master/repo/backup/db/[BACKUP-DIFF-6]/backup.manifest
-----------------------------------------------------------------------------------------
[backrest]
backrest-checksum="[CHECKSUM]"
backrest-format=5
backrest-version="[VERSION-1]"
[backup]
backup-label="[BACKUP-DIFF-6]"
backup-prior="[BACKUP-FULL-3]"
backup-timestamp-copy-start=[TIMESTAMP]
backup-timestamp-start=[TIMESTAMP]
backup-timestamp-stop=[TIMESTAMP]
backup-type="diff"
[backup:db]
db-catalog-version=201409291
db-control-version=942
db-id=1
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
db-system-id=1000000000000000094
Fixed config warnings raised from a remote process causing errors in the master process. Fixed by Cynthia Shang.
2017-05-16 17:28:51 +02:00
db-version="9.4"
[backup:option]
option-archive-check=true
option-archive-copy=true
option-backup-standby=false
option-checksum-page=false
Full/Synthetic test refactor. * Combine hardlink and non/compressed in synthetic tests to reduce test time and improve coverage. * Change log level of hardlink logging to detail. * Cast size in S3 manifest to integer.
2017-06-15 21:32:10 +02:00
option-compress=true
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
option-delta=false
Full/Synthetic test refactor. * Combine hardlink and non/compressed in synthetic tests to reduce test time and improve coverage. * Change log level of hardlink logging to detail. * Cast size in S3 manifest to integer.
2017-06-15 21:32:10 +02:00
option-hardlink=true
Fixed config warnings raised from a remote process causing errors in the master process. Fixed by Cynthia Shang.
2017-05-16 17:28:51 +02:00
option-online=false
[backup:target]
pg_data={"path":"[TEST_PATH]/db-master/db/base-2/base","type":"path"}
pg_tblspc/2={"path":"../../tablespace/ts2","tablespace-id":"2","tablespace-name":"ts2","type":"link"}
[target:file]
Full/Synthetic test refactor. * Combine hardlink and non/compressed in synthetic tests to reduce test time and improve coverage. * Change log level of hardlink logging to detail. * Cast size in S3 manifest to integer.
2017-06-15 21:32:10 +02:00
pg_data/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","master":true,"reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/badchecksum.txt={"checksum":"f927212cd08d11a42a666b2f04235398e9ceeb51","master":true,"reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/base/1/12000={"checksum":"22c98d248ff548311eda88559e4a8405ed77c003","reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/base/1/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","mode":"0660","reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/base/16384/17000={"checksum":"7579ada0808d7f98087a0a586d0df9de009cdc33","reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/base/16384/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/base/32768/33000={"checksum":"4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f","reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/base/32768/33000.32767={"checksum":"21e2c7c1a326682c07053b7d6a5a40dbd49c2ec5","reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/base/32768/33001={"checksum":"6bf316f11d28c28914ea9be92c00de9bea6d9a6b","reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
pg_data/base/32768/44000_init={"checksum":"4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f","reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Full/Synthetic test refactor. * Combine hardlink and non/compressed in synthetic tests to reduce test time and improve coverage. * Change log level of hardlink logging to detail. * Cast size in S3 manifest to integer.
2017-06-15 21:32:10 +02:00
pg_data/base/32768/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/base/base2.txt={"checksum":"cafac3c59553f2cfde41ce2e62e7662295f108c0","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
pg_data/changecontent.txt={"checksum":"a094d94583e209556d03c3c5da33131a065f1689","master":true,"reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/changetime.txt={"checksum":"88087292ed82e26f3eb824d0bffc05ccf7a30f8d","master":true,"reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-2]}
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 11:24:42 +02:00
pg_data/global/pg_control={"checksum":"4c77c900f7af0d9ab13fa9982051a42e0b637f6c","master":true,"reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-2]}
Full/Synthetic test refactor. * Combine hardlink and non/compressed in synthetic tests to reduce test time and improve coverage. * Change log level of hardlink logging to detail. * Cast size in S3 manifest to integer.
2017-06-15 21:32:10 +02:00
pg_data/pg_stat/global.stat={"checksum":"e350d5ce0153f3e22d5db21cf2a4eff00f3ee877","master":true,"reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-2]}
pg_data/postgresql.conf={"checksum":"6721d92c9fcdf4248acff1f9a1377127d9064807","master":true,"reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-2]}
Improve support for special characters in filenames. % characters caused issues in backup/restore due to filenames being appended directly into a format string. Reserved XML characters (<>&') caused issues in the S3 driver due to improper escaping. Add a file with all common special characters to regression testing.
2018-09-10 16:54:34 +02:00
pg_data/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?%={"master":true,"reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Allow zero-size files in backup manifest to reference a prior manifest regardless of timestamp delta. Contributed by Cynthia Shang.
2018-08-24 22:50:33 +02:00
pg_data/zero_from_start={"master":true,"reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Add zero-length file to mock/all test.
2018-07-05 21:40:50 +02:00
pg_data/zerosize.txt={"master":true,"reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-2]}
Full/Synthetic test refactor. * Combine hardlink and non/compressed in synthetic tests to reduce test time and improve coverage. * Change log level of hardlink logging to detail. * Cast size in S3 manifest to integer.
2017-06-15 21:32:10 +02:00
pg_tblspc/2/[TS_PATH-1]/32768/tablespace2.txt={"checksum":"dc7f76e43c46101b47acc55ae4d593a9e6983578","reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_tblspc/2/[TS_PATH-1]/32768/tablespace2c.txt={"checksum":"dfcb8679956b734706cf87259d50c88f83e80e66","reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Fixed config warnings raised from a remote process causing errors in the master process. Fixed by Cynthia Shang.
2017-05-16 17:28:51 +02:00
[target:file:default]
group="[GROUP-1]"
master=false
mode="0600"
user="[USER-1]"
[target:link]
pg_data/pg_tblspc/2={"destination":"../../tablespace/ts2"}
[target:link:default]
group="[GROUP-1]"
user="[USER-1]"
[target:path]
pg_data={}
pg_data/base={}
pg_data/base/1={}
pg_data/base/16384={}
pg_data/base/32768={}
pg_data/global={}
pg_data/pg_clog={}
pg_data/pg_dynshmem={}
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
pg_data/pg_log={}
Fixed config warnings raised from a remote process causing errors in the master process. Fixed by Cynthia Shang.
2017-05-16 17:28:51 +02:00
pg_data/pg_notify={}
pg_data/pg_replslot={}
pg_data/pg_serial={}
pg_data/pg_snapshots={}
pg_data/pg_stat={}
pg_data/pg_stat_tmp={}
pg_data/pg_subtrans={}
pg_data/pg_tblspc={}
pg_tblspc={}
pg_tblspc/2={}
pg_tblspc/2/[TS_PATH-1]={}
pg_tblspc/2/[TS_PATH-1]/32768={}
[target:path:default]
group="[GROUP-1]"
mode="0700"
user="[USER-1]"
+ supplemental file: [TEST_PATH]/db-master/repo/backup/db/backup.info
---------------------------------------------------------------------
[backrest]
backrest-checksum="[CHECKSUM]"
backrest-format=5
backrest-version="[VERSION-1]"
[backup:current]
Full/Synthetic test refactor. * Combine hardlink and non/compressed in synthetic tests to reduce test time and improve coverage. * Change log level of hardlink logging to detail. * Cast size in S3 manifest to integer.
2017-06-15 21:32:10 +02:00
[BACKUP-FULL-3]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"full","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":false,"option-compress":true,"option-hardlink":true,"option-online":false}
[BACKUP-DIFF-5]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-prior":"[BACKUP-FULL-3]","backup-reference":["[BACKUP-FULL-3]"],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"diff","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":false,"option-compress":true,"option-hardlink":true,"option-online":false}
[BACKUP-DIFF-6]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-prior":"[BACKUP-FULL-3]","backup-reference":["[BACKUP-FULL-3]"],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"diff","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":false,"option-compress":true,"option-hardlink":true,"option-online":false}
Fixed config warnings raised from a remote process causing errors in the master process. Fixed by Cynthia Shang.
2017-05-16 17:28:51 +02:00
[db]
db-catalog-version=201409291
db-control-version=942
db-id=1
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
db-system-id=1000000000000000094
Fixed config warnings raised from a remote process causing errors in the master process. Fixed by Cynthia Shang.
2017-05-16 17:28:51 +02:00
db-version="9.4"
[db:history]
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
1={"db-catalog-version":201409291,"db-control-version":942,"db-system-id":1000000000000000094,"db-version":"9.4"}
Fixed config warnings raised from a remote process causing errors in the master process. Fixed by Cynthia Shang.
2017-05-16 17:28:51 +02:00
Fixed the backup command so the backup-standby option is reset (and the backup proceeds on the master) if the standby is not configured and/or reachable. Fixed by Cynthia Shang.
2017-05-12 22:05:44 +02:00
diff backup - option backup-standby reset - backup performed from master (db-master host)
Full/Synthetic test refactor. * Combine hardlink and non/compressed in synthetic tests to reduce test time and improve coverage. * Change log level of hardlink logging to detail. * Cast size in S3 manifest to integer.
2017-06-15 21:32:10 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --no-online --log-level-console=info --backup-standby --type=diff --stanza=db backup
Fixed the backup command so the backup-standby option is reset (and the backup proceeds on the master) if the standby is not configured and/or reachable. Fixed by Cynthia Shang.
2017-05-12 22:05:44 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
P00 INFO: backup command begin [BACKREST-VERSION]: --backup-standby --compress --compress-level=3 --config=[TEST_PATH]/db-master/pgbackrest.conf --db-timeout=45 --exclude=postgresql.auto.conf --exclude=pg_log/ --exclude=pg_log2 --lock-path=[TEST_PATH]/db-master/lock --log-level-console=info --log-level-file=trace --log-level-stderr=off --log-path=[TEST_PATH]/db-master/log --log-subprocess --no-log-timestamp --no-online --pg1-path=[TEST_PATH]/db-master/db/base-2/base --protocol-timeout=60 --repo1-hardlink --repo1-path=[TEST_PATH]/db-master/repo --stanza=db --start-fast --type=diff
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
P00 WARN: option repo1-retention-full is not set, the repository may run out of space
HINT: to retain full backups indefinitely (without warning), set option 'repo1-retention-full' to the maximum.
Fixed the backup command so the backup-standby option is reset (and the backup proceeds on the master) if the standby is not configured and/or reachable. Fixed by Cynthia Shang.
2017-05-12 22:05:44 +02:00
P00 WARN: option backup-standby is enabled but standby is not properly configured - backups will be performed from the master
P00 INFO: last backup label = [BACKUP-FULL-3], version = [VERSION-1]
P01 INFO: backup file [TEST_PATH]/db-master/db/base-2/base/base/base2.txt (9B, 100%) checksum cafac3c59553f2cfde41ce2e62e7662295f108c0
P00 INFO: diff backup size = 9B
Fixed config warnings raised from a remote process causing errors in the master process. Fixed by Cynthia Shang.
2017-05-16 17:28:51 +02:00
P00 INFO: new backup label = [BACKUP-DIFF-7]
Fixed the backup command so the backup-standby option is reset (and the backup proceeds on the master) if the standby is not configured and/or reachable. Fixed by Cynthia Shang.
2017-05-12 22:05:44 +02:00
P00 INFO: backup command end: completed successfully
Move async forking and more error handling to C. The Perl process was exiting directly when called but that interfered with proper locking for the forked async process. Now Perl returns results to the C process which handles all errors, including signals.
2018-04-13 02:42:26 +02:00
P00 INFO: expire command begin
Rename retention-* options to repo-retention-*.
2018-02-19 22:07:24 +02:00
P00 INFO: option 'repo1-retention-archive' is not set - archive logs will not be expired
Fixed the backup command so the backup-standby option is reset (and the backup proceeds on the master) if the standby is not configured and/or reachable. Fixed by Cynthia Shang.
2017-05-12 22:05:44 +02:00
P00 INFO: expire command end: completed successfully
+ supplemental file: [TEST_PATH]/db-master/pgbackrest.conf
----------------------------------------------------------
[db]
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
pg1-path=[TEST_PATH]/db-master/db/base-2/base
Fixed the backup command so the backup-standby option is reset (and the backup proceeds on the master) if the standby is not configured and/or reachable. Fixed by Cynthia Shang.
2017-05-12 22:05:44 +02:00
[db:restore]
[global]
Full/Synthetic test refactor. * Combine hardlink and non/compressed in synthetic tests to reduce test time and improve coverage. * Change log level of hardlink logging to detail. * Cast size in S3 manifest to integer.
2017-06-15 21:32:10 +02:00
compress=y
Fix custom settings for compress-level option being ignored. Reported by Jens Wilke.
2017-10-23 00:05:46 +02:00
compress-level=3
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
db-timeout=45
lock-path=[TEST_PATH]/db-master/lock
Fixed the backup command so the backup-standby option is reset (and the backup proceeds on the master) if the standby is not configured and/or reachable. Fixed by Cynthia Shang.
2017-05-12 22:05:44 +02:00
log-level-console=debug
log-level-file=trace
log-level-stderr=off
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
log-path=[TEST_PATH]/db-master/log
Add log-subprocess option to allow file logging for local and remote subprocesses.
2018-08-23 02:05:49 +02:00
log-subprocess=y
Set log-timestamp=n for integration tests. This means less filtering of logs needs to be done and new timestamps can be added without adding new filters.
2018-05-11 17:24:38 +02:00
log-timestamp=n
Storage and IO layer refactor: Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.) Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 23:51:41 +02:00
protocol-timeout=60
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-04 01:27:38 +02:00
repo1-hardlink=y
repo1-path=[TEST_PATH]/db-master/repo
Consolidate archive-push tests into new archive test.
2017-06-21 14:38:53 +02:00
spool-path=[TEST_PATH]/db-master/spool
Fixed the backup command so the backup-standby option is reset (and the backup proceeds on the master) if the standby is not configured and/or reachable. Fixed by Cynthia Shang.
2017-05-12 22:05:44 +02:00
[global:backup]
archive-copy=y
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
exclude=postgresql.auto.conf
exclude=pg_log/
exclude=pg_log2
Fixed the backup command so the backup-standby option is reset (and the backup proceeds on the master) if the standby is not configured and/or reachable. Fixed by Cynthia Shang.
2017-05-12 22:05:44 +02:00
start-fast=y
Fixed config warnings raised from a remote process causing errors in the master process. Fixed by Cynthia Shang.
2017-05-16 17:28:51 +02:00
+ supplemental file: [TEST_PATH]/db-master/repo/backup/db/[BACKUP-DIFF-7]/backup.manifest
Fixed the backup command so the backup-standby option is reset (and the backup proceeds on the master) if the standby is not configured and/or reachable. Fixed by Cynthia Shang.
2017-05-12 22:05:44 +02:00
-----------------------------------------------------------------------------------------
[backrest]
backrest-checksum="[CHECKSUM]"
backrest-format=5
backrest-version="[VERSION-1]"
[backup]
Fixed config warnings raised from a remote process causing errors in the master process. Fixed by Cynthia Shang.
2017-05-16 17:28:51 +02:00
backup-label="[BACKUP-DIFF-7]"
Fixed the backup command so the backup-standby option is reset (and the backup proceeds on the master) if the standby is not configured and/or reachable. Fixed by Cynthia Shang.
2017-05-12 22:05:44 +02:00
backup-prior="[BACKUP-FULL-3]"
backup-timestamp-copy-start=[TIMESTAMP]
backup-timestamp-start=[TIMESTAMP]
backup-timestamp-stop=[TIMESTAMP]
backup-type="diff"
[backup:db]
db-catalog-version=201409291
db-control-version=942
db-id=1
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
db-system-id=1000000000000000094
Fixed the backup command so the backup-standby option is reset (and the backup proceeds on the master) if the standby is not configured and/or reachable. Fixed by Cynthia Shang.
2017-05-12 22:05:44 +02:00
db-version="9.4"
[backup:option]
option-archive-check=true
option-archive-copy=true
option-backup-standby=false
option-checksum-page=false
Full/Synthetic test refactor. * Combine hardlink and non/compressed in synthetic tests to reduce test time and improve coverage. * Change log level of hardlink logging to detail. * Cast size in S3 manifest to integer.
2017-06-15 21:32:10 +02:00
option-compress=true
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
option-delta=false
Full/Synthetic test refactor. * Combine hardlink and non/compressed in synthetic tests to reduce test time and improve coverage. * Change log level of hardlink logging to detail. * Cast size in S3 manifest to integer.
2017-06-15 21:32:10 +02:00
option-hardlink=true
Fixed the backup command so the backup-standby option is reset (and the backup proceeds on the master) if the standby is not configured and/or reachable. Fixed by Cynthia Shang.
2017-05-12 22:05:44 +02:00
option-online=false
[backup:target]
pg_data={"path":"[TEST_PATH]/db-master/db/base-2/base","type":"path"}
pg_tblspc/2={"path":"../../tablespace/ts2","tablespace-id":"2","tablespace-name":"ts2","type":"link"}
[target:file]
Full/Synthetic test refactor. * Combine hardlink and non/compressed in synthetic tests to reduce test time and improve coverage. * Change log level of hardlink logging to detail. * Cast size in S3 manifest to integer.
2017-06-15 21:32:10 +02:00
pg_data/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","master":true,"reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/badchecksum.txt={"checksum":"f927212cd08d11a42a666b2f04235398e9ceeb51","master":true,"reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/base/1/12000={"checksum":"22c98d248ff548311eda88559e4a8405ed77c003","reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/base/1/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","mode":"0660","reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/base/16384/17000={"checksum":"7579ada0808d7f98087a0a586d0df9de009cdc33","reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/base/16384/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/base/32768/33000={"checksum":"4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f","reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/base/32768/33000.32767={"checksum":"21e2c7c1a326682c07053b7d6a5a40dbd49c2ec5","reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/base/32768/33001={"checksum":"6bf316f11d28c28914ea9be92c00de9bea6d9a6b","reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. Contributed by Cynthia Shang.
2018-07-31 00:53:34 +02:00
pg_data/base/32768/44000_init={"checksum":"4a383e4fb8b5cd2a4e8fab91ef63dce48e532a2f","reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Full/Synthetic test refactor. * Combine hardlink and non/compressed in synthetic tests to reduce test time and improve coverage. * Change log level of hardlink logging to detail. * Cast size in S3 manifest to integer.
2017-06-15 21:32:10 +02:00
pg_data/base/32768/PG_VERSION={"checksum":"184473f470864e067ee3a22e64b47b0a1c356f29","reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/base/base2.txt={"checksum":"cafac3c59553f2cfde41ce2e62e7662295f108c0","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Add checksum delta for incremental backups. Use checksums rather than timestamps to determine if files have changed. This is useful in cases where the timestamps may not be trustworthy, e.g. when performing an incremental after failing over to a standby. If checksum delta is enabled then checksums will be used for verification of resumed backups, even if they are full. Resumes have always used checksums to verify the files in the repository, enabling delta performs checksums on the database files as well. Note that the user must manually enable this feature in cases were it would be useful or just keep in enabled all the time. A future commit will address automatically enabling the feature in cases where it seems likely to be useful. Contributed by Cynthia Shang.
2018-09-19 17:12:45 +02:00
pg_data/changecontent.txt={"checksum":"a094d94583e209556d03c3c5da33131a065f1689","master":true,"reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_data/changetime.txt={"checksum":"88087292ed82e26f3eb824d0bffc05ccf7a30f8d","master":true,"reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-2]}
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 11:24:42 +02:00
pg_data/global/pg_control={"checksum":"4c77c900f7af0d9ab13fa9982051a42e0b637f6c","master":true,"reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-2]}
Full/Synthetic test refactor. * Combine hardlink and non/compressed in synthetic tests to reduce test time and improve coverage. * Change log level of hardlink logging to detail. * Cast size in S3 manifest to integer.
2017-06-15 21:32:10 +02:00
pg_data/pg_stat/global.stat={"checksum":"e350d5ce0153f3e22d5db21cf2a4eff00f3ee877","master":true,"reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-2]}
pg_data/postgresql.conf={"checksum":"6721d92c9fcdf4248acff1f9a1377127d9064807","master":true,"reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-2]}
Improve support for special characters in filenames. % characters caused issues in backup/restore due to filenames being appended directly into a format string. Reserved XML characters (<>&') caused issues in the S3 driver due to improper escaping. Add a file with all common special characters to regression testing.
2018-09-10 16:54:34 +02:00
pg_data/special-@!#$^&*()-_+~`{}[]\|:;"<>',.?%={"master":true,"reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Allow zero-size files in backup manifest to reference a prior manifest regardless of timestamp delta. Contributed by Cynthia Shang.
2018-08-24 22:50:33 +02:00
pg_data/zero_from_start={"master":true,"reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Add zero-length file to mock/all test.
2018-07-05 21:40:50 +02:00
pg_data/zerosize.txt={"master":true,"reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-2]}
Full/Synthetic test refactor. * Combine hardlink and non/compressed in synthetic tests to reduce test time and improve coverage. * Change log level of hardlink logging to detail. * Cast size in S3 manifest to integer.
2017-06-15 21:32:10 +02:00
pg_tblspc/2/[TS_PATH-1]/32768/tablespace2.txt={"checksum":"dc7f76e43c46101b47acc55ae4d593a9e6983578","reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
pg_tblspc/2/[TS_PATH-1]/32768/tablespace2c.txt={"checksum":"dfcb8679956b734706cf87259d50c88f83e80e66","reference":"[BACKUP-FULL-3]","repo-size":[SIZE],"size":[SIZE],"timestamp":[TIMESTAMP-1]}
Fixed the backup command so the backup-standby option is reset (and the backup proceeds on the master) if the standby is not configured and/or reachable. Fixed by Cynthia Shang.
2017-05-12 22:05:44 +02:00
[target:file:default]
group="[GROUP-1]"
master=false
mode="0600"
user="[USER-1]"
[target:link]
pg_data/pg_tblspc/2={"destination":"../../tablespace/ts2"}
[target:link:default]
group="[GROUP-1]"
user="[USER-1]"
[target:path]
pg_data={}
pg_data/base={}
pg_data/base/1={}
pg_data/base/16384={}
pg_data/base/32768={}
pg_data/global={}
pg_data/pg_clog={}
pg_data/pg_dynshmem={}
Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using.
2018-08-27 21:51:05 +02:00
pg_data/pg_log={}
Fixed the backup command so the backup-standby option is reset (and the backup proceeds on the master) if the standby is not configured and/or reachable. Fixed by Cynthia Shang.
2017-05-12 22:05:44 +02:00
pg_data/pg_notify={}
pg_data/pg_replslot={}
pg_data/pg_serial={}
pg_data/pg_snapshots={}
pg_data/pg_stat={}
pg_data/pg_stat_tmp={}
pg_data/pg_subtrans={}
pg_data/pg_tblspc={}
pg_tblspc={}
pg_tblspc/2={}
pg_tblspc/2/[TS_PATH-1]={}
pg_tblspc/2/[TS_PATH-1]/32768={}
[target:path:default]
group="[GROUP-1]"
mode="0700"
user="[USER-1]"
+ supplemental file: [TEST_PATH]/db-master/repo/backup/db/backup.info
---------------------------------------------------------------------
[backrest]
backrest-checksum="[CHECKSUM]"
backrest-format=5
backrest-version="[VERSION-1]"
[backup:current]
Full/Synthetic test refactor. * Combine hardlink and non/compressed in synthetic tests to reduce test time and improve coverage. * Change log level of hardlink logging to detail. * Cast size in S3 manifest to integer.
2017-06-15 21:32:10 +02:00
[BACKUP-FULL-3]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"full","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":false,"option-compress":true,"option-hardlink":true,"option-online":false}
[BACKUP-DIFF-5]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-prior":"[BACKUP-FULL-3]","backup-reference":["[BACKUP-FULL-3]"],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"diff","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":false,"option-compress":true,"option-hardlink":true,"option-online":false}
[BACKUP-DIFF-6]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-prior":"[BACKUP-FULL-3]","backup-reference":["[BACKUP-FULL-3]"],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"diff","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":false,"option-compress":true,"option-hardlink":true,"option-online":false}
[BACKUP-DIFF-7]={"backrest-format":5,"backrest-version":"[VERSION-1]","backup-archive-start":null,"backup-archive-stop":null,"backup-info-repo-size":[SIZE],"backup-info-repo-size-delta":[DELTA],"backup-info-size":[SIZE],"backup-info-size-delta":[DELTA],"backup-prior":"[BACKUP-FULL-3]","backup-reference":["[BACKUP-FULL-3]"],"backup-timestamp-start":[TIMESTAMP],"backup-timestamp-stop":[TIMESTAMP],"backup-type":"diff","db-id":1,"option-archive-check":true,"option-archive-copy":true,"option-backup-standby":false,"option-checksum-page":false,"option-compress":true,"option-hardlink":true,"option-online":false}
Fixed the backup command so the backup-standby option is reset (and the backup proceeds on the master) if the standby is not configured and/or reachable. Fixed by Cynthia Shang.
2017-05-12 22:05:44 +02:00
[db]
db-catalog-version=201409291
db-control-version=942
db-id=1
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
db-system-id=1000000000000000094
Fixed the backup command so the backup-standby option is reset (and the backup proceeds on the master) if the standby is not configured and/or reachable. Fixed by Cynthia Shang.
2017-05-12 22:05:44 +02:00
db-version="9.4"
[db:history]
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
1={"db-catalog-version":201409291,"db-control-version":942,"db-system-id":1000000000000000094,"db-version":"9.4"}
Reference in New Issue Copy Permalink