BUFFER_EXTERN() provides a clean way to create buffer constants.
Convert HASH_TYPE_SHA256_ZERO_STR to HASH_TYPE_SHA256_ZERO_BUF to be consistent with HASH_TYPE_SHA1_ZERO_BUF.
This should make it a little clearer what the variable (VR) macros are doing since the declaration/definition cannot both be set to extern (but functions can).
Splitting the variable macros out also allows them to be changed in the future with little churn, while changing the function macro creates a large amount of churn.
CirrusCI stopped supporting Intel but the arm builds are not working, even with the same steps that work on an arm Mac.
Remove to unstick the build pipeline until this can be resolved.
This is immediately useful because it will detect any extern'd functions or variables that are not being used. It also detects functions or variables that are declared but not defined.
If a FV/VR_EXTERN macro is missing it will be detected either because of a mismatch in the declaration/definition or because a new defined symbol will appear in the nm test.
Eventually the unity build will be used to create a more optimized pgbackrest binary but that will need to wait.
In this case the destination will be large enough to hold the source so memcpy is more efficient.
Also, in highly optimized builds the compiler may warn for strncpy() when it can see that the source size won't include the terminator.
Similar to b9be4fa5, these functions are not used by the core code so move them to the build module. The new implementation is a little less efficient but that is much less of a worry in the build/test code.
Also remove regExpMatchSize() since it was not longer needed.
Neither of these functions were used by the core code. strReplace() is only used in the tests but it doesn't hurt to put it in build since the build code is not distributed.
This was done by checking the extension but it is possible to include a module that does not have a vendor or auto extension. Instead make it explicit that the module is included in another module.
Also change the variable from "include" to "included" to make it clearer what it indicates.
It is probably not a good idea to restore the latest backup when it was not made from the current PostgreSQL version. If there is no backup after a stanza-upgrade then replicas might be built with a prior version leading to failures.
Add an error in this case if the latest backup would be used, i.e. --set or --type=time/lsn is not specified.
In particular the section about other backup solutions not supporting parallel processing was no longer accurate, so reword it.
Also update some other sections that used older nomenclature, had awkward wording, or needed clarification.
The behavior of pause depends on the hot_standby parameter and the PostgreSQL version so mention both.
This behavior has been verified on PostgreSQL 9.6–15. PostgreSQL 12 is an inflection point because the behavior of an unset recovery_target_action with hot_standby=off changed in https://git.postgresql.org/gitweb/?p=postgresql.git;h=2dedf4d9a899b36d1a8ed29be5efbd1b31a8fe85.
The copy command was converting \n to a linefeed, which the json conversion did not like. In a healthy repository there won't be any linefeeds but certain errors can contain them.
Fix by loading into a text field and then replacing the linefeed when converting to jsonb.
The prior range checking was done based on the valid values for gz. While this worked it was a subset of what is available for lz4 and zst.
Allow the range to be specified for each compress-type. Adding this functionality to the parse module would be a better solution but that is a bigger project than this fix deserves, at least for now.
Calculate a checksum of the data stored in the repository when a file is transformed (e.g. compressed). This allows resume and verify to operate without needing to decompress/decrypt the data.
This can also be used to verify more complex formats such as block incremental and allow backups from the repository without needing to decompress the data to verify the checksum.
Add some basic encrypted tests to maintain coverage. These will be expanded in a future commit.
Manifest checksums were stored as hex-encoded strings due to legacy compatibility with Perl. Storing the checksums as binary in memory uses half the space and avoids many conversions.
There is no change to the on-disk manifest format which stores the checksum as a hex-encoded string.
Our new policy is to support ten versions of PostgreSQL, the five supported releases and the last five EOL releases. As of PostgreSQL 15, that means 9.0/9.1/9.2 are no longer supported by pgBackRest.
Remove all logic associated with 9.0/9.1/9.2 and update the tests.
Document the new support policy.
Update InfoPg to read/write control versions for the history in backup.info, since we can no longer rely on the mappings being available. In theory this could have been an issue after removing 8.3/8.4 if anybody was using a version that old.
This avoids constructs such as decodeToBin(encodeBase64, ...) which are confusing since decode and encode are used in the same function call. decodeToBin(encodingBase64, ...) makes it clearer what is happening.
The storageNewItrP() permissions test was running twice with the errorOnMissing flag set to false. Fix by setting to true for one test.
Also update the comments to be clearer about what the tests are doing and fix minor formatting.
This allows test backups to be run in other test modules.
It is likely that more logic will be moved here but for now this suffices to get test backups working in the restore module.
Bug Fixes:
* Fix missing reference in diff/incr backup. (Reviewed by Stefan Fercot. Reported by Marcel Borger, ulfedf, jaymefSO.)
Improvements:
* Add hint when an option is specified without an index. (Reviewed by Stefan Fercot.)
When loading prior manifests without the new reference list, the code failed to add the current backup to the reference list. Since the current backup is never explicitly referenced, building references from the file list was not sufficient to generate a complete list.
The main problem here was a bad test, fixed in 28f6604. This masked the issue and prevented it from being found. Now it is clear in the test that the current label is missing from the reference list.
Fix by adding the current label to the reference list if a reference list is not stored in the manifest.
Changing the label of a manifest that already had a label was not a good test and it ended up masking a bug where the current backup label was not being added to the reference list on manifest load, since manifestBackupLabelSet() added the label to the reference list. In fact, manifestBackupLabelSet() should never be called after a manifest load or even after the label has been set.
Add an assertion to prevent manifestBackupLabelSet() being called when the label is already set.
The bug exposed here will be fixed in a subsequent commit.
Hopefully this will make it a little clearer to the user what is wrong when they specify an indexed option without an index.
Also fix an ambiguous use of cfgParseOptionP(). The prior code worked in that it set prefixMatch = true but it was not very readable.
