db/pgbackrest
1
0
Fork 0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2024-12-14 10:13:05 +02:00
Code Issues Releases Activity
be7b8a485b
pgbackrest/test/expect/mock-archive-stop-002.log

67 lines
4.5 KiB
Plaintext
Raw Normal View History

Azure support for repository storage. Azure and Azure-compatible object stores can now be used for repository storage. Currently only shared key authentication is supported but SAS will be added soon.
2020-07-02 22:24:34 +02:00
run 002 - rmt 1, cmp lz4, error connect, storage azure, enc 0
=============================================================
Fix for issue #83: Provide more helpful error message during archive-stop situations. Fix for issue #84: archive-async in combination with archive-max-mb doesn't work as documented. Unit tests for archive stop.
2015-05-28 19:17:34 +02:00
Create test matrix for mock/archive-stop to increase coverage and reduce tests. The same test configurations are run on all four test VMs, which seems a real waste of resources. Vary the tests per VM to increase coverage while reducing the total number of tests. Be sure to include each major feature (remote, s3, encryption) in each VM at least once.
2019-03-01 17:12:41 +02:00
stanza-create db - create required data for stanza (backup host)
Reduce log level for all expect tests to detail. The C code is designed to be efficient rather than deterministic at the debug log level. As we move more testing from integration to unit tests it makes less sense to try and maintain the expect logs at this log level. Most of the expect logs have already been moved to detail level but mock/all still had tests at debug level. Change the logging defaults in the config file and remove as many references to log-level-console as possible.
2019-05-23 00:23:44 +02:00
> [CONTAINER-EXEC] backup [BACKREST-BIN] --config=[TEST_PATH]/backup/pgbackrest.conf --stanza=db --no-online stanza-create
Improved stanza-create command so that it can repair broken repositories in most cases and is robust enough to be made mandatory. Contributed by Cynthia Shang.
2016-12-20 23:52:20 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Use lower-case names from Azure identifiers in integration tests. Azurite, which is used for testing, did not enforce this before so the capital letters were not a problem. Now Azurite enforces the same rules as Azure so use lower-case identifiers instead. These names were only used in integration tests so there was no production impact.
2021-06-09 18:28:40 +02:00
P00 INFO: stanza-create command begin [BACKREST-VERSION]: --buffer-size=[BUFFER-SIZE] --compress-level-network=1 --config=[TEST_PATH]/backup/pgbackrest.conf --db-timeout=45 --exec-id=[EXEC-ID] --lock-path=[TEST_PATH]/backup/lock --log-level-console=detail --log-level-file=[LOG-LEVEL-FILE] --log-level-stderr=off --log-path=[TEST_PATH]/backup/log[] --no-log-timestamp --no-online --pg1-host=db-primary --pg1-host-cmd=[BACKREST-BIN] --pg1-host-config=[TEST_PATH]/db-primary/pgbackrest.conf --pg1-host-user=[USER-1] --pg1-path=[TEST_PATH]/db-primary/db/base --protocol-timeout=60 --repo1-azure-account=<redacted> --repo1-azure-container=azcontainer --repo1-azure-key=<redacted> --repo1-path=/ --repo1-storage-host=azure --no-repo1-storage-verify-tls --repo1-type=azure --stanza=db
Partial multi-repository implementation. Multi-repository implementations for the archive-push, check, info, stanza-create, stanza-upgrade, and stanza-delete commands. Multi-repo configuration is disabled so there should be no behavioral changes between these commands and their current single-repo implementations. Multi-repo documentation and integration tests are still in the multi-repo development branch. All unit tests work as multi-repo since they are able to bypass the configuration restrictions.
2021-01-21 22:21:50 +02:00
P00 INFO: stanza-create for stanza 'db' on repo1
Add general-purpose statistics collector. Currently each module that needs to collect statistics implements custom code to do so. This is cumbersome. Create a general purpose module for collecting and reporting statistics. Statistics are output in the log at detail level, but there are other uses they could be put to eventually. No new functionality is added. This is just a drop-in replacement for the current statistics, with the advantage of being more flexible. The new stats are slower because they involve a list lookup, but performance testing shows stats can be updated at about 40,000/ms which seems fast enough for our purposes.
2020-08-20 20:04:26 +02:00
P00 DETAIL: statistics: STATISTICS
Improved stanza-create command so that it can repair broken repositories in most cases and is robust enough to be made mandatory. Contributed by Cynthia Shang.
2016-12-20 23:52:20 +02:00
P00 INFO: stanza-create command end: completed successfully
Create test matrix for mock/archive-stop to increase coverage and reduce tests. The same test configurations are run on all four test VMs, which seems a real waste of resources. Vary the tests per VM to increase coverage while reducing the total number of tests. Be sure to include each major feature (remote, s3, encryption) in each VM at least once.
2019-03-01 17:12:41 +02:00
+ supplemental file: /backup/db/backup.info
-------------------------------------------
Improved stanza-create command so that it can repair broken repositories in most cases and is robust enough to be made mandatory. Contributed by Cynthia Shang.
2016-12-20 23:52:20 +02:00
[backrest]
backrest-format=5
backrest-version="[VERSION-1]"
[db]
db-catalog-version=201409291
db-control-version=942
db-id=1
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
db-system-id=1000000000000000094
Improved stanza-create command so that it can repair broken repositories in most cases and is robust enough to be made mandatory. Contributed by Cynthia Shang.
2016-12-20 23:52:20 +02:00
db-version="9.4"
[db:history]
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
1={"db-catalog-version":201409291,"db-control-version":942,"db-system-id":1000000000000000094,"db-version":"9.4"}
Improved stanza-create command so that it can repair broken repositories in most cases and is robust enough to be made mandatory. Contributed by Cynthia Shang.
2016-12-20 23:52:20 +02:00
Improve performance of info file load/save. Info files required three copies in memory to be loaded (the original string, an ini representation, and the final info object). Not only was this memory inefficient but the Ini object does sequential scans when searching for keys making large files very slow to load. This has not been an issue since archive.info and backup.info are very small, but it becomes a big deal when loading manifests with hundreds of thousands of files. Instead of holding copies of the data in memory, use a callback to deliver the ini data directly to the object when loading. Use a similar method for save to avoid having an intermediate copy. Save is a bit complex because sections/keys must be written in alpha order or older versions of pgBackRest will not calculate the correct checksum. Also move the load retry logic to helper functions rather than embedding it in the Info object. This allows for more flexibility in loading and ensures that stack traces will be available when developing unit tests. Reviewed by Cynthia Shang.
2019-09-06 19:48:28 +02:00
[backrest]
backrest-checksum="[CHECKSUM]"
Create test matrix for mock/archive-stop to increase coverage and reduce tests. The same test configurations are run on all four test VMs, which seems a real waste of resources. Vary the tests per VM to increase coverage while reducing the total number of tests. Be sure to include each major feature (remote, s3, encryption) in each VM at least once.
2019-03-01 17:12:41 +02:00
+ supplemental file: /archive/db/archive.info
---------------------------------------------
Improved stanza-create command so that it can repair broken repositories in most cases and is robust enough to be made mandatory. Contributed by Cynthia Shang.
2016-12-20 23:52:20 +02:00
[backrest]
backrest-format=5
backrest-version="[VERSION-1]"
[db]
db-id=1
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
db-system-id=1000000000000000094
Improved stanza-create command so that it can repair broken repositories in most cases and is robust enough to be made mandatory. Contributed by Cynthia Shang.
2016-12-20 23:52:20 +02:00
db-version="9.4"
[db:history]
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
1={"db-id":1000000000000000094,"db-version":"9.4"}
Improved stanza-create command so that it can repair broken repositories in most cases and is robust enough to be made mandatory. Contributed by Cynthia Shang.
2016-12-20 23:52:20 +02:00
Improve performance of info file load/save. Info files required three copies in memory to be loaded (the original string, an ini representation, and the final info object). Not only was this memory inefficient but the Ini object does sequential scans when searching for keys making large files very slow to load. This has not been an issue since archive.info and backup.info are very small, but it becomes a big deal when loading manifests with hundreds of thousands of files. Instead of holding copies of the data in memory, use a callback to deliver the ini data directly to the object when loading. Use a similar method for save to avoid having an intermediate copy. Save is a bit complex because sections/keys must be written in alpha order or older versions of pgBackRest will not calculate the correct checksum. Also move the load retry logic to helper functions rather than embedding it in the Info object. This allows for more flexibility in loading and ensures that stack traces will be available when developing unit tests. Reviewed by Cynthia Shang.
2019-09-06 19:48:28 +02:00
[backrest]
backrest-checksum="[CHECKSUM]"
Rename most instances of master to primary in tests. This aligns better with general PostgreSQL usage and our own documentation (updated in 4bcef702). Usage in the backup.manifest tests has not been updated since it might break the file format.
2020-06-16 20:06:38 +02:00
> [CONTAINER-EXEC] db-primary [BACKREST-BIN] --config=[TEST_PATH]/db-primary/pgbackrest.conf --log-level-console=warn --archive-push-queue-max=33554432 --stanza=db archive-push [TEST_PATH]/db-primary/db/base/pg_xlog/000000010000000100000001
Fix for issue #83: Provide more helpful error message during archive-stop situations. Fix for issue #84: archive-async in combination with archive-max-mb doesn't work as documented. Unit tests for archive stop.
2015-05-28 19:17:34 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Improve archive-push command fault tolerance. 3b8f0ef missed some cases that could cause archive-push to fail: * Checking archive info. * Checking to see if a WAL segment already exists. These cases are now handled so archive-push can succeed on any valid repos.
2021-03-25 18:54:49 +02:00
> [CONTAINER-EXEC] db-primary [BACKREST-BIN] --config=[TEST_PATH]/db-primary/pgbackrest.conf --log-level-console=warn --archive-push-queue-max=33554432 --stanza=db archive-push [TEST_PATH]/db-primary/db/base/pg_xlog/000000010000000100000002 --repo1-host=bogus
Fix for issue #83: Provide more helpful error message during archive-stop situations. Fix for issue #84: archive-async in combination with archive-max-mb doesn't work as documented. Unit tests for archive stop.
2015-05-28 19:17:34 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Improve archive-push command fault tolerance. 3b8f0ef missed some cases that could cause archive-push to fail: * Checking archive info. * Checking to see if a WAL segment already exists. These cases are now handled so archive-push can succeed on any valid repos.
2021-03-25 18:54:49 +02:00
P00 ERROR: [103]: unable to find a valid repository:
repo1: [UnknownError] remote-0 process on 'bogus' terminated unexpectedly [255]: ssh: Could not resolve hostname bogus: Name or service not known
Fix for issue #83: Provide more helpful error message during archive-stop situations. Fix for issue #84: archive-async in combination with archive-max-mb doesn't work as documented. Unit tests for archive stop.
2015-05-28 19:17:34 +02:00
Improve archive-push command fault tolerance. 3b8f0ef missed some cases that could cause archive-push to fail: * Checking archive info. * Checking to see if a WAL segment already exists. These cases are now handled so archive-push can succeed on any valid repos.
2021-03-25 18:54:49 +02:00
> [CONTAINER-EXEC] db-primary [BACKREST-BIN] --config=[TEST_PATH]/db-primary/pgbackrest.conf --log-level-console=warn --archive-push-queue-max=33554432 --stanza=db archive-push [TEST_PATH]/db-primary/db/base/pg_xlog/000000010000000100000003 --repo1-host=bogus
Fix for issue #83: Provide more helpful error message during archive-stop situations. Fix for issue #84: archive-async in combination with archive-max-mb doesn't work as documented. Unit tests for archive stop.
2015-05-28 19:17:34 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Improve archive-push command fault tolerance. 3b8f0ef missed some cases that could cause archive-push to fail: * Checking archive info. * Checking to see if a WAL segment already exists. These cases are now handled so archive-push can succeed on any valid repos.
2021-03-25 18:54:49 +02:00
P00 ERROR: [103]: unable to find a valid repository:
repo1: [UnknownError] remote-0 process on 'bogus' terminated unexpectedly [255]: ssh: Could not resolve hostname bogus: Name or service not known
Fix for issue #83: Provide more helpful error message during archive-stop situations. Fix for issue #84: archive-async in combination with archive-max-mb doesn't work as documented. Unit tests for archive stop.
2015-05-28 19:17:34 +02:00
Rename most instances of master to primary in tests. This aligns better with general PostgreSQL usage and our own documentation (updated in 4bcef702). Usage in the backup.manifest tests has not been updated since it might break the file format.
2020-06-16 20:06:38 +02:00
> [CONTAINER-EXEC] db-primary [BACKREST-BIN] --config=[TEST_PATH]/db-primary/pgbackrest.conf --log-level-console=warn --archive-push-queue-max=33554432 --stanza=db archive-push [TEST_PATH]/db-primary/db/base/pg_xlog/000000010000000100000004 --repo1-host=bogus
Fix for issue #83: Provide more helpful error message during archive-stop situations. Fix for issue #84: archive-async in combination with archive-max-mb doesn't work as documented. Unit tests for archive stop.
2015-05-28 19:17:34 +02:00
------------------------------------------------------------------------------------------------------------------------------------
The archive-push command is implemented entirely in C. This new implementation should behave exactly like the old Perl code with the exception of updated log messages. Remove as much of the Perl code as possible without breaking other commands.
2019-03-29 15:26:33 +02:00
P00 WARN: dropped WAL file '000000010000000100000004' because archive queue exceeded 32MB
Fix for issue #83: Provide more helpful error message during archive-stop situations. Fix for issue #84: archive-async in combination with archive-max-mb doesn't work as documented. Unit tests for archive stop.
2015-05-28 19:17:34 +02:00
Rename most instances of master to primary in tests. This aligns better with general PostgreSQL usage and our own documentation (updated in 4bcef702). Usage in the backup.manifest tests has not been updated since it might break the file format.
2020-06-16 20:06:38 +02:00
> [CONTAINER-EXEC] db-primary [BACKREST-BIN] --config=[TEST_PATH]/db-primary/pgbackrest.conf --log-level-console=warn --archive-push-queue-max=33554432 --stanza=db archive-push [TEST_PATH]/db-primary/db/base/pg_xlog/000000010000000100000005
Fixed an issue where asynchronous archiving was transferring one file per execution instead of transferring files in batches. This regression was introduced in v1.09 and affected efficiency only, all WAL segments were correctly archived in asynchronous mode. Reported by Stephen Frost.
2016-11-17 16:39:21 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Reference in New Issue Copy Permalink