db/pgbackrest
1
0
Fork 0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2025-02-07 13:42:41 +02:00
Code Issues Releases Activity
pgbackrest/doc/xml/release/2023/2.48.xml

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

214 lines
8.4 KiB
XML
Raw Normal View History

v2.48: Repository Storage Tags Bug Fixes: * Fix issue restoring block incremental without a block list. (Reviewed by Stephen Frost, Burak Yurdakul. Reported by Burak Yurdakul.) Features: * Add --repo-storage-tag option to create object tags. (Reviewed by Stephen Frost, Stefan Fercot, Timothée Peignier.) * Add known hosts checking for SFTP storage driver. (Contributed by Reid Thompson. Reviewed by Stephen Frost, David Steele.) * Support for dual stack connections. (Reviewed by Stephen Frost.) * Add backup size completed/total to info command JSON output. (Contributed by Stefan Fercot. Reviewed by David Steele.) Improvements: * Multi-stanza check command. (Reviewed by Stephen Frost.) * Retry reads of pg_control until checksum is valid. (Reviewed by Stefan Fercot, Stephen Frost.) * Optimize WAL segment check after successful backup. (Reviewed by Stephen Frost.) * Improve GCS multi-part performance. (Reviewed by Reid Thompson.) * Allow archive-get command to run when stanza is stopped. (Reviewed by Tom Swartz, David Christensen, Reid Thompson.) * Accept leading tilde in paths for SFTP public/private keys. (Contributed by Reid Thompson. Reviewed by David Steele.) * Reload GCS credentials before renewing authentication token. (Reviewed by Stephen Frost. Suggested by Daniel Farina.) Documentation Bug Fixes: * Fix configuration reference example for the tls-server-address option. (Fixed by Hartmut Goebel. Reviewed by David Steele.) * Fix command reference example for the filter option. Test Suite Improvements: * Allow storage/sftp unit test to run without libssh2 installed. (Contributed by Reid Thompson. Reviewed by David Steele. Suggested by Wu Ning.)
2023-09-25 09:32:15 -04:00
<release date="2023-09-25" version="2.48" title="Repository Storage Tags">
Split release notes into individual files. The release.xml file was getting pretty unwieldy so break release notes into separate files. Also break contributors into a separate file. In theory most of release.xml could now be generated automatically but adding a new release does not represent a serious maintenance burden, so for the time being it does not seem worth it.
2023-09-04 12:00:06 -04:00
<release-core-list>
Fix issue restoring block incremental without a block list. It is currently possible for a block map to be written without any accompanying blocks. This happens when a file timestamp is updated but the file has not changed. On restore, this caused problems when encryption was enabled, the block map was bundled after a file that had been stored without block incremental, and both files were included in a single bundle read. In this case the block map would not be decrypted and the encrypted data was passed to blockMapNewRead() with unpredictable results. In many cases built-in retries would rectify this problem as long as delta was enabled since block maps would move to the beginning of the bundle read and be decrypted properly. If enough files were affected, however, it could overwhelm the retries and throw an error. Subsequent delta restores would eventually be able to produce a valid result. Fix this by moving block map decryption so it works correctly no matter where the block map is located in the read. This has the additional benefit of limiting how far the block map can read so it will error earlier if corrupt. Though in this case there was no repository corruption involved, it appeared that way to blockMapNewRead() since it was reading encrypted data. Arguably block maps without blocks should not be written at all, but it would be better to consider that as a separate change. This pattern clearly exists in the wild and needs to be handled, plus the new implementation has other benefits.
2023-09-19 11:30:29 -04:00
<release-bug-list>
<release-item>
<github-issue id="2182"/>
<github-pull-request id="2184"/>
<release-item-contributor-list>
<release-item-ideator id="burak.yurdakul"/>
<release-item-contributor id="david.steele"/>
<release-item-reviewer id="stephen.frost"/>
<!-- Actually tester, but we don't have a tag for that yet -->
<release-item-reviewer id="burak.yurdakul"/>
</release-item-contributor-list>
<p>Fix issue restoring block incremental without a block list.</p>
</release-item>
</release-bug-list>
Split release notes into individual files. The release.xml file was getting pretty unwieldy so break release notes into separate files. Also break contributors into a separate file. In theory most of release.xml could now be generated automatically but adding a new release does not represent a serious maintenance burden, so for the time being it does not seem worth it.
2023-09-04 12:00:06 -04:00
<release-feature-list>
Add --repo-storage-tag option to create object tags. This new option allows tags to be added to objects in S3, GCS, and Azure repositories. This was fairly straightforward for S3 and Azure, but GCS does not allow tags for a simple upload using the JSON interface. If tags are required then the resumable interface must be used even if the file falls below the limit that usually triggers a resumable upload (i.e. size < repo-storage-upload-chunk-size). This option is structured so that tags must be specified per-repo rather than globally for all repos. This seems logical since the tag keys and values may vary by service, e.g. S3 vs GCS. These storage tags are independent of backup annotations since they are likely to be used for different purposes, e.g. billing, while the backup annotations are primarily intended for monitoring.
2023-09-14 08:22:21 -04:00
<release-item>
<github-issue id="2150"/>
<github-pull-request id="2159"/>
<release-item-contributor-list>
<release-item-contributor id="david.steele"/>
<release-item-reviewer id="stephen.frost"/>
<release-item-reviewer id="stefan.fercot"/>
<release-item-reviewer id="timothee.peignier"/>
</release-item-contributor-list>
<p>Add <br-option>--repo-storage-tag</br-option> option to create object tags.</p>
</release-item>
Add known hosts checking for SFTP storage driver. By default require a known hosts match as part of the SFTP storage driver's authentication process, i.e. repo-sftp-host-key-check-type=strict. The match is expected to be found in the default list or in a list of known hosts files provided by the user. An exception is made if a fingerprint has been manually configured with repo-sftp-host-fingerprint or repo-sftp-host-key-check-type=accept-new can be used to automatically add new hosts. Also allow host key verification to be skipped, as before, but require the user to explicitly set this (repo-sftp-host-key-check-type=none) rather than it being the default.
2023-09-15 20:22:38 -04:00
<release-item>
<github-pull-request id="2125"/>
<release-item-contributor-list>
<release-item-contributor id="reid.thompson"/>
<release-item-reviewer id="stephen.frost"/>
<release-item-reviewer id="david.steele"/>
</release-item-contributor-list>
<p>Add known hosts checking for <proper>SFTP</proper> storage driver.</p>
</release-item>
Support for dual stack connections. The prior code would only connect to the first address provided by getaddrinfo(). Instead try each address in the list. If all connections fail then wait and try them all again until timeout. Currently a round robin approach is used where each connection attempt must fail before the next connection is attempted. This works fine, for example, when an ipv6 address has no route to the host, but will work less well when a host answers but doesn't respond in a timely fashion. We may consider a Happy Eyeballs approach in the future, but since pgBackRest is primarily a background process, it is not clear that slightly improved response time (in the case of failed connections) is worth the extra complexity.
2023-09-12 18:09:58 -04:00
<release-item>
<commit subject="Adjust Wait object to be more accurate when nested."/>
<commit subject="Aggregate error retries in ErrorRetry output."/>
<commit subject="Refactor address list lookup to include all returned addresses.">
<github-pull-request id="2174"/>
</commit>
<commit subject="Support for dual stack connections.">
<github-issue id="2154"/>
<github-pull-request id="2175"/>
</commit>
<release-item-contributor-list>
<release-item-contributor id="david.steele"/>
<release-item-reviewer id="stephen.frost"/>
</release-item-contributor-list>
<p>Support for dual stack connections.</p>
</release-item>
Split release notes into individual files. The release.xml file was getting pretty unwieldy so break release notes into separate files. Also break contributors into a separate file. In theory most of release.xml could now be generated automatically but adding a new release does not represent a serious maintenance burden, so for the time being it does not seem worth it.
2023-09-04 12:00:06 -04:00
<release-item>
<github-issue id="2007"/>
<github-pull-request id="2141"/>
<release-item-contributor-list>
<release-item-contributor id="stefan.fercot"/>
<release-item-reviewer id="david.steele"/>
</release-item-contributor-list>
<p>Add backup size completed/total to <cmd>info</cmd> command <proper>JSON</proper> output.</p>
</release-item>
</release-feature-list>
<release-improvement-list>
<release-item>
<github-pull-request id="2057"/>
<release-item-contributor-list>
<release-item-contributor id="david.steele"/>
<release-item-reviewer id="stephen.frost"/>
</release-item-contributor-list>
<p>Multi-stanza check command.</p>
</release-item>
Retry reads of pg_control until checksum is valid. On certain file systems (e.g. ext4) pg_control may appear torn if there is a concurrent write while reading the file. To prevent an invalid read, retry until the checksum matches the control data. Special handling is required for the pg-version-force feature since the offset of the checksum is not known. In this case, scan from the default position to the end of the data looking for a checksum match. This is a bit imprecise, but better than nothing, and the chance of a random collision in the control data seems very remote considering the ratio of data size (< 512 bytes) to checksum size (4 bytes). This was discovered and a possible solution proposed for PostgreSQL in [1]. The proposed solution may work for backup, but pgBackRest needs to be able to read pg_control reliably outside of backup. So no matter what fix is adopted for PostgreSQL, pgBackRest need retries. Further adjustment may be required as the PostgreSQL fix evolves. [1] https://www.postgresql.org/message-id/20221123014224.xisi44byq3cf5psi%40awork3.anarazel.de
2023-09-10 09:47:49 -04:00
<release-item>
<github-pull-request id="2131"/>
<release-item-contributor-list>
<release-item-contributor id="david.steele"/>
<release-item-reviewer id="stefan.fercot"/>
<release-item-reviewer id="stephen.frost"/>
</release-item-contributor-list>
<p>Retry reads of <file>pg_control</file> until checksum is valid.</p>
</release-item>
Optimize WAL segment check after successful backup. The prior code did one list command against the storage for each WAL segment. This led to a lot of lists and was especially inefficient when the WAL (or the majority of it) was already present. Optimize to keep the contents of a WAL directory and use them on a subsequent search. Leave the optimizations for a single WAL segment since other places still use that mode.
2023-09-10 12:45:58 -04:00
<release-item>
<github-pull-request id="2169"/>
<release-item-contributor-list>
<release-item-contributor id="david.steele"/>
<release-item-reviewer id="stephen.frost"/>
</release-item-contributor-list>
<p>Optimize WAL segment check after successful backup.</p>
</release-item>
Split release notes into individual files. The release.xml file was getting pretty unwieldy so break release notes into separate files. Also break contributors into a separate file. In theory most of release.xml could now be generated automatically but adding a new release does not represent a serious maintenance burden, so for the time being it does not seem worth it.
2023-09-04 12:00:06 -04:00
<release-item>
<github-pull-request id="2162"/>
<release-item-contributor-list>
<release-item-contributor id="david.steele"/>
<release-item-reviewer id="reid.thompson"/>
</release-item-contributor-list>
<p>Improve <proper>GCS</proper> multi-part performance.</p>
</release-item>
<release-item>
<release-item-contributor-list>
<release-item-contributor id="david.steele"/>
<release-item-reviewer id="tom.swartz"/>
<release-item-reviewer id="david.christensen"/>
<release-item-reviewer id="reid.thompson"/>
</release-item-contributor-list>
<p>Allow <cmd>archive-get</cmd> command to run when stanza is stopped.</p>
</release-item>
<release-item>
<github-pull-request id="2136"/>
<release-item-contributor-list>
<release-item-contributor id="reid.thompson"/>
<release-item-reviewer id="david.steele"/>
</release-item-contributor-list>
<p>Accept leading tilde in paths for SFTP public/private keys.</p>
</release-item>
<release-item>
<github-pull-request id="2072"/>
<release-item-contributor-list>
<release-item-ideator id="daniel.farina"/>
<release-item-contributor id="david.steele"/>
<release-item-reviewer id="stephen.frost"/>
</release-item-contributor-list>
<p>Reload GCS credentials before renewing authentication token.</p>
</release-item>
</release-improvement-list>
Add report option to check command. This option is intended to eventually create a comprehensive report about the state of the pgBackRest configuration based on the results of the check command. Implement a detailed report of the configuration options in the environment and configuration files. This should be useful information when debugging configuration errors, since invalid options and configurations are automatically noted. While custom config locations will not be found automatically, it will at least be clear that the config is not in a standard location. For now keep this option internal since there is a lot of work to be done, but commit it so that it can be used when needed and tested in various environments. Note that for now when --report is specified, the check command is not being run at all. Only the config report is generated. This behavior will be improved in the future.
2023-09-15 09:30:40 -04:00
<release-development-list>
<release-item>
<github-pull-request id="2176"/>
<release-item-contributor-list>
<release-item-contributor id="david.steele"/>
<release-item-reviewer id="stephen.frost"/>
</release-item-contributor-list>
<p>Add report option to <cmd>check</cmd> command.</p>
</release-item>
</release-development-list>
Split release notes into individual files. The release.xml file was getting pretty unwieldy so break release notes into separate files. Also break contributors into a separate file. In theory most of release.xml could now be generated automatically but adding a new release does not represent a serious maintenance burden, so for the time being it does not seem worth it.
2023-09-04 12:00:06 -04:00
</release-core-list>
Fix configuration reference example for the tls-server-address option. The prior example (::*) was not valid and would result in the following error: ERROR: [049]: unable to get address for '::*': [-2] Name or service not known Correct values are either * for IPv4 or :: for IPv6. The IPv4 value is used as the example since only one example is allowed.
2023-09-04 19:39:11 +02:00
<release-doc-list>
<release-bug-list>
<release-item>
<github-pull-request id="2165"/>
<release-item-contributor-list>
<release-item-contributor id="hartmut.goebel"/>
<release-item-reviewer id="david.steele"/>
</release-item-contributor-list>
<p>Fix configuration reference example for the <br-option>tls-server-address</br-option> option.</p>
</release-item>
Fix command reference example for the filter option. This example was broken by 24f7252. Revert to (almost) the prior code to fix this example until something better can be committed. The something better is in progress but it adds new build requirements so it is too late to include it for the release. Technically this breaks some other examples, but they are all internal and not visible in the user-facing documentation.
2023-09-23 13:41:03 -04:00
<release-item>
<release-item-contributor-list>
<release-item-contributor id="david.steele"/>
</release-item-contributor-list>
<p>Fix command reference example for the <br-option>filter</br-option> option.</p>
</release-item>
Fix configuration reference example for the tls-server-address option. The prior example (::*) was not valid and would result in the following error: ERROR: [049]: unable to get address for '::*': [-2] Name or service not known Correct values are either * for IPv4 or :: for IPv6. The IPv4 value is used as the example since only one example is allowed.
2023-09-04 19:39:11 +02:00
</release-bug-list>
</release-doc-list>
Allow storage/sftp unit test to run without libssh2 installed. Add missing #ifdefs and update tests to prevent test compilation failure and test run issues when libssh2 is not present.
2023-09-19 16:26:13 -04:00
<release-test-list>
<release-improvement-list>
<release-item>
<github-issue id="2163"/>
<github-pull-request id="2177"/>
<release-item-contributor-list>
<release-item-ideator id="wu.ning"/>
<release-item-contributor id="reid.thompson"/>
<release-item-reviewer id="david.steele"/>
</release-item-contributor-list>
<p>Allow <id>storage/sftp</id> unit test to run without <id>libssh2</id> installed.</p>
</release-item>
</release-improvement-list>
</release-test-list>
Split release notes into individual files. The release.xml file was getting pretty unwieldy so break release notes into separate files. Also break contributors into a separate file. In theory most of release.xml could now be generated automatically but adding a new release does not represent a serious maintenance burden, so for the time being it does not seem worth it.
2023-09-04 12:00:06 -04:00
</release>
Reference in New Issue Copy Permalink