db/pgbackrest
1
0
Fork 0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2024-12-14 10:13:05 +02:00
Code Issues Releases Activity
d9efbc3698
pgbackrest/test/expect/mock-archive-stop-002.log

65 lines
4.3 KiB
Plaintext
Raw Normal View History

Create test matrix for mock/archive-stop to increase coverage and reduce tests. The same test configurations are run on all four test VMs, which seems a real waste of resources. Vary the tests per VM to increase coverage while reducing the total number of tests. Be sure to include each major feature (remote, s3, encryption) in each VM at least once.
2019-03-01 17:12:41 +02:00
run 002 - rmt 1, cmp 1, error connect, s3 1, enc 0
Repository encryption support. Contributed by Cynthia Shang.
2017-11-06 19:51:12 +02:00
==================================================
Fix for issue #83: Provide more helpful error message during archive-stop situations. Fix for issue #84: archive-async in combination with archive-max-mb doesn't work as documented. Unit tests for archive stop.
2015-05-28 19:17:34 +02:00
Create test matrix for mock/archive-stop to increase coverage and reduce tests. The same test configurations are run on all four test VMs, which seems a real waste of resources. Vary the tests per VM to increase coverage while reducing the total number of tests. Be sure to include each major feature (remote, s3, encryption) in each VM at least once.
2019-03-01 17:12:41 +02:00
stanza-create db - create required data for stanza (backup host)
Reduce log level for all expect tests to detail. The C code is designed to be efficient rather than deterministic at the debug log level. As we move more testing from integration to unit tests it makes less sense to try and maintain the expect logs at this log level. Most of the expect logs have already been moved to detail level but mock/all still had tests at debug level. Change the logging defaults in the config file and remove as many references to log-level-console as possible.
2019-05-23 00:23:44 +02:00
> [CONTAINER-EXEC] backup [BACKREST-BIN] --config=[TEST_PATH]/backup/pgbackrest.conf --stanza=db --no-online stanza-create
Improved stanza-create command so that it can repair broken repositories in most cases and is robust enough to be made mandatory. Contributed by Cynthia Shang.
2016-12-20 23:52:20 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Integration test improvements for disk and memory efficiency. Set log-level-file=off when more that one test will run. In this case is it impossible to see the logs anyway since they will be automatically cleaned up after the test. This improves performance pretty dramatically since trace-level logging is expensive. If a singe integration test is run then log-level-file is trace by default but can be changed with the --log-level-test-file option. Reduce buffer-size to 64k to save memory during testing and allow more processes to run in parallel. Update log replacement rules so that these options can change without affecting expect logs.
2019-12-17 22:23:07 +02:00
P00 INFO: stanza-create command begin [BACKREST-VERSION]: --buffer-size=[BUFFER-SIZE] --compress-level=3 --compress-level-network=1 --config=[TEST_PATH]/backup/pgbackrest.conf --db-timeout=45 --lock-path=[TEST_PATH]/backup/lock --log-level-console=detail --log-level-file=[LOG-LEVEL-FILE] --log-level-stderr=off --log-path=[TEST_PATH]/backup/log[] --no-log-timestamp --no-online --pg1-host=db-master --pg1-host-cmd=[BACKREST-BIN] --pg1-host-config=[TEST_PATH]/db-master/pgbackrest.conf --pg1-host-user=[USER-1] --pg1-path=[TEST_PATH]/db-master/db/base --protocol-timeout=60 --repo1-path=/ --repo1-s3-bucket=pgbackrest-dev --repo1-s3-endpoint=s3.amazonaws.com --repo1-s3-key=<redacted> --repo1-s3-key-secret=<redacted> --repo1-s3-region=us-east-1 --no-repo1-s3-verify-tls --repo1-type=s3 --stanza=db
Add Perl interface to C storage layer. Maintaining the storage layer/drivers in two languages is burdensome. Since the integration tests require the Perl storage layer/drivers we'll need them even after the core code is migrated to C. Create an interface layer so the Perl code can be removed and new storage drivers/features introduced without adding Perl equivalents. The goal is to move the integration tests to C so this interface will eventually be removed. That being the case, the interface was designed for maximum compatibility to ease the transition. The result looks a bit hacky but we'll improve it as needed until it can be retired.
2019-06-26 14:24:58 +02:00
P00 DETAIL: tls statistics:[TLS-STATISTICS]
P00 INFO: http statistics:[HTTP-STATISTICS]
Improved stanza-create command so that it can repair broken repositories in most cases and is robust enough to be made mandatory. Contributed by Cynthia Shang.
2016-12-20 23:52:20 +02:00
P00 INFO: stanza-create command end: completed successfully
Create test matrix for mock/archive-stop to increase coverage and reduce tests. The same test configurations are run on all four test VMs, which seems a real waste of resources. Vary the tests per VM to increase coverage while reducing the total number of tests. Be sure to include each major feature (remote, s3, encryption) in each VM at least once.
2019-03-01 17:12:41 +02:00
+ supplemental file: /backup/db/backup.info
-------------------------------------------
Improved stanza-create command so that it can repair broken repositories in most cases and is robust enough to be made mandatory. Contributed by Cynthia Shang.
2016-12-20 23:52:20 +02:00
[backrest]
backrest-format=5
backrest-version="[VERSION-1]"
[db]
db-catalog-version=201409291
db-control-version=942
db-id=1
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
db-system-id=1000000000000000094
Improved stanza-create command so that it can repair broken repositories in most cases and is robust enough to be made mandatory. Contributed by Cynthia Shang.
2016-12-20 23:52:20 +02:00
db-version="9.4"
[db:history]
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
1={"db-catalog-version":201409291,"db-control-version":942,"db-system-id":1000000000000000094,"db-version":"9.4"}
Improved stanza-create command so that it can repair broken repositories in most cases and is robust enough to be made mandatory. Contributed by Cynthia Shang.
2016-12-20 23:52:20 +02:00
Improve performance of info file load/save. Info files required three copies in memory to be loaded (the original string, an ini representation, and the final info object). Not only was this memory inefficient but the Ini object does sequential scans when searching for keys making large files very slow to load. This has not been an issue since archive.info and backup.info are very small, but it becomes a big deal when loading manifests with hundreds of thousands of files. Instead of holding copies of the data in memory, use a callback to deliver the ini data directly to the object when loading. Use a similar method for save to avoid having an intermediate copy. Save is a bit complex because sections/keys must be written in alpha order or older versions of pgBackRest will not calculate the correct checksum. Also move the load retry logic to helper functions rather than embedding it in the Info object. This allows for more flexibility in loading and ensures that stack traces will be available when developing unit tests. Reviewed by Cynthia Shang.
2019-09-06 19:48:28 +02:00
[backrest]
backrest-checksum="[CHECKSUM]"
Create test matrix for mock/archive-stop to increase coverage and reduce tests. The same test configurations are run on all four test VMs, which seems a real waste of resources. Vary the tests per VM to increase coverage while reducing the total number of tests. Be sure to include each major feature (remote, s3, encryption) in each VM at least once.
2019-03-01 17:12:41 +02:00
+ supplemental file: /archive/db/archive.info
---------------------------------------------
Improved stanza-create command so that it can repair broken repositories in most cases and is robust enough to be made mandatory. Contributed by Cynthia Shang.
2016-12-20 23:52:20 +02:00
[backrest]
backrest-format=5
backrest-version="[VERSION-1]"
[db]
db-id=1
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
db-system-id=1000000000000000094
Improved stanza-create command so that it can repair broken repositories in most cases and is robust enough to be made mandatory. Contributed by Cynthia Shang.
2016-12-20 23:52:20 +02:00
db-version="9.4"
[db:history]
Automate generation of WAL and pg_control test files. The existing static files would not work with 32-bit or big-endian systems so create functions to generate these files dynamically rather than creating a bunch of new static files.
2017-11-19 03:02:54 +02:00
1={"db-id":1000000000000000094,"db-version":"9.4"}
Improved stanza-create command so that it can repair broken repositories in most cases and is robust enough to be made mandatory. Contributed by Cynthia Shang.
2016-12-20 23:52:20 +02:00
Improve performance of info file load/save. Info files required three copies in memory to be loaded (the original string, an ini representation, and the final info object). Not only was this memory inefficient but the Ini object does sequential scans when searching for keys making large files very slow to load. This has not been an issue since archive.info and backup.info are very small, but it becomes a big deal when loading manifests with hundreds of thousands of files. Instead of holding copies of the data in memory, use a callback to deliver the ini data directly to the object when loading. Use a similar method for save to avoid having an intermediate copy. Save is a bit complex because sections/keys must be written in alpha order or older versions of pgBackRest will not calculate the correct checksum. Also move the load retry logic to helper functions rather than embedding it in the Info object. This allows for more flexibility in loading and ensures that stack traces will be available when developing unit tests. Reviewed by Cynthia Shang.
2019-09-06 19:48:28 +02:00
[backrest]
backrest-checksum="[CHECKSUM]"
Rename archive-queue-max option to archive-push-queue-max. This avoids confusion with the archive-get-queue-max option. The old option name will continue to be accepted.
2018-04-19 01:37:35 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --log-level-console=warn --archive-push-queue-max=33554432 --stanza=db archive-push [TEST_PATH]/db-master/db/base/pg_xlog/000000010000000100000001
Fix for issue #83: Provide more helpful error message during archive-stop situations. Fix for issue #84: archive-async in combination with archive-max-mb doesn't work as documented. Unit tests for archive stop.
2015-05-28 19:17:34 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Create test matrix for mock/archive-stop to increase coverage and reduce tests. The same test configurations are run on all four test VMs, which seems a real waste of resources. Vary the tests per VM to increase coverage while reducing the total number of tests. Be sure to include each major feature (remote, s3, encryption) in each VM at least once.
2019-03-01 17:12:41 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --log-level-console=warn --archive-push-queue-max=33554432 --stanza=db --repo1-host=bogus archive-push [TEST_PATH]/db-master/db/base/pg_xlog/000000010000000100000002
Fix for issue #83: Provide more helpful error message during archive-stop situations. Fix for issue #84: archive-async in combination with archive-max-mb doesn't work as documented. Unit tests for archive stop.
2015-05-28 19:17:34 +02:00
------------------------------------------------------------------------------------------------------------------------------------
The archive-push command is implemented entirely in C. This new implementation should behave exactly like the old Perl code with the exception of updated log messages. Remove as much of the Perl code as possible without breaking other commands.
2019-03-29 15:26:33 +02:00
P00 ERROR: [125]: remote-0 process on 'bogus' terminated unexpectedly [255]: ssh: Could not resolve hostname bogus: Name or service not known
Fix for issue #83: Provide more helpful error message during archive-stop situations. Fix for issue #84: archive-async in combination with archive-max-mb doesn't work as documented. Unit tests for archive stop.
2015-05-28 19:17:34 +02:00
Create test matrix for mock/archive-stop to increase coverage and reduce tests. The same test configurations are run on all four test VMs, which seems a real waste of resources. Vary the tests per VM to increase coverage while reducing the total number of tests. Be sure to include each major feature (remote, s3, encryption) in each VM at least once.
2019-03-01 17:12:41 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --log-level-console=warn --archive-push-queue-max=33554432 --stanza=db --repo1-host=bogus archive-push [TEST_PATH]/db-master/db/base/pg_xlog/000000010000000100000003
Fix for issue #83: Provide more helpful error message during archive-stop situations. Fix for issue #84: archive-async in combination with archive-max-mb doesn't work as documented. Unit tests for archive stop.
2015-05-28 19:17:34 +02:00
------------------------------------------------------------------------------------------------------------------------------------
The archive-push command is implemented entirely in C. This new implementation should behave exactly like the old Perl code with the exception of updated log messages. Remove as much of the Perl code as possible without breaking other commands.
2019-03-29 15:26:33 +02:00
P00 ERROR: [125]: remote-0 process on 'bogus' terminated unexpectedly [255]: ssh: Could not resolve hostname bogus: Name or service not known
Fix for issue #83: Provide more helpful error message during archive-stop situations. Fix for issue #84: archive-async in combination with archive-max-mb doesn't work as documented. Unit tests for archive stop.
2015-05-28 19:17:34 +02:00
Fix issue with archive-push-queue-max not being honored on connection error. If an error occurred while acquiring a lock on a remote server the error would be reported correctly, but the queue max detection code was not reached. The tests failed to detect this because they fixed the connection before queue max, allowing the ccde to be reached. Move the queue max code before the lock so it will run even when remote connections are not working. This means that no attempt will be made to transfer WAL once queue max has been exceeded, but it makes it much more likely that the code will be reach without error. Update tests to continue errors up to the point where queue max is exceeded. Reported by Lardière Sébastien.
2018-10-27 17:57:57 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --log-level-console=warn --archive-push-queue-max=33554432 --stanza=db archive-push [TEST_PATH]/db-master/db/base/pg_xlog/000000010000000100000004 --repo1-host=bogus
Fix for issue #83: Provide more helpful error message during archive-stop situations. Fix for issue #84: archive-async in combination with archive-max-mb doesn't work as documented. Unit tests for archive stop.
2015-05-28 19:17:34 +02:00
------------------------------------------------------------------------------------------------------------------------------------
The archive-push command is implemented entirely in C. This new implementation should behave exactly like the old Perl code with the exception of updated log messages. Remove as much of the Perl code as possible without breaking other commands.
2019-03-29 15:26:33 +02:00
P00 WARN: dropped WAL file '000000010000000100000004' because archive queue exceeded 32MB
Fix for issue #83: Provide more helpful error message during archive-stop situations. Fix for issue #84: archive-async in combination with archive-max-mb doesn't work as documented. Unit tests for archive stop.
2015-05-28 19:17:34 +02:00
Rename archive-queue-max option to archive-push-queue-max. This avoids confusion with the archive-get-queue-max option. The old option name will continue to be accepted.
2018-04-19 01:37:35 +02:00
> [CONTAINER-EXEC] db-master [BACKREST-BIN] --config=[TEST_PATH]/db-master/pgbackrest.conf --log-level-console=warn --archive-push-queue-max=33554432 --stanza=db archive-push [TEST_PATH]/db-master/db/base/pg_xlog/000000010000000100000005
Fixed an issue where asynchronous archiving was transferring one file per execution instead of transferring files in batches. This regression was introduced in v1.09 and affected efficiency only, all WAL segments were correctly archived in asynchronous mode. Reported by Stephen Frost.
2016-11-17 16:39:21 +02:00
------------------------------------------------------------------------------------------------------------------------------------
Reference in New Issue Copy Permalink