1
0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2024-12-14 10:13:05 +02:00
Commit Graph

1797 Commits

Author SHA1 Message Date
David Steele
b6bfd9f99d
Strip extensions from history manifest before showing in error message.
In cases where clock skew or timezone issues are preventing backup label generation the user could see an error like this:

new backup label '20220504-152308F' is not later than latest backup label '20220504-222042F_20220504-222141I.manifest.gz'

This will happen if the most recent label is drawn from the history. It is cleaner (and probably less confusing) to strip off the extensions so the user sees:

new backup label '20220504-152308F' is not later than latest backup label '20220504-222042F_20220504-222141I'
2022-05-05 09:20:49 -04:00
David Steele
ef672c74ad
Prevent memContextFree() from freeing memory needed by callbacks.
The order of callbacks and frees meant that memory needed during a callback (for logging in all known cases) might end up being freed before a callback needed it.

Requiring callbacks and logging to check the validity of their allocations is pretty risky and it is not clear that all possible cases have been accounted for.

Instead recursively execute all the callbacks first and then come back and recursively free the context. This is safer and it removes the need to check if a context is freeing so a simple active flag (in debug builds) will do. The caller no longer needs this information at all so remove memContextFreeing() and objMemContextFreeing().
2022-05-04 14:53:05 -04:00
Reid Thompson
d9088b2e2b
Show backup percent complete in info output.
In the JSON output the percent complete is storage as an integer of the percent complete * 100. So, before display it should be converted to double and divided by 100, or split using integer mod and div.

Note that percent complete will only be displayed on the host where the backup was executed. Remote hosts will show a backup/expire running with no percent complete.
2022-05-04 12:52:05 -04:00
David Steele
20782c88bc
PostgreSQL 15 support.
PostgreSQL 15 drops support for exclusive backup and renames the start/stop backup commands.

This is based on the pgdg-testing repo since beta1 has not been released yet, but it seems unlikely that breaking changes will be made at this point. beta1 should be tagged just before our next release so we'll retest before the release.
2022-05-04 11:55:59 -04:00
David Steele
692fe496bd
Remove dependency on pg_database.datlastsysoid.
This column has been removed in PostgreSQL 15. Rather than add a lot of special handling, it seems better just to update all versions to not depend on this column.

Add centralized functions to identify the type of database (i.e. system or user) by name and use FirstNormalObjectId when a name is not available.

The new query in the db module will still return the prior result for PostgreSQL <= 15, which will be stored in the manifest. This is important to preserve behavior when downgrading pgBackRest. There are no concerns here for PostgreSQL 15 since older versions of pgBackRest won't be able to restore backups for PostgreSQL 15 anyway.
2022-05-04 08:22:45 -04:00
David Steele
9a271e925c
Fix error thrown from FINALLY() causing an infinite loop.
Any error thrown resets execution to the last setjmp(), which means that parts of the try block need to make sure they don't get run again. FINALLY() was not doing this so if it threw an error it would end up back in the FINALLY() block, where the error would likely be thrown again, causing an infinite loop.

Fix this by tracking the state of FINALLY() and only running it once. This requires cleaning the error stack like CATCH*() and clearing the error like TRY_END() depending on the order of execution.
2022-05-03 14:34:05 -04:00
David Steele
9629908694
Error on all lock failures except another process holding the lock.
The archive-get/archive-push commands would not error for, .e.g permissions errors, when attempting to get a lock before launching the async process. Since the async process was not launched there would be no error status file and the user would get a generic failure message. Also, there would be no async log.

Refactor lockAcquireFile() to throw an error when failOnNoLock = false unless the file is locked by another process. This seems to be the original intent of this parameter and there may have been a mistake when porting from Perl. In any case it looks wrong enough to be considered a bug.
2022-05-03 10:13:32 -04:00
David Steele
0055fa40fe Add user:group to lock permission error.
This will help debug permissions errors when the lock file cannot be created.
2022-05-02 09:45:57 -04:00
David Steele
03c71aa606 Add hint to check the log on archive-get/archive-push async error.
If this error is thrown rather than a specific error returned from the async process, it means the async process is unable to write the status files for some reason and the only way to get the error is out of the async log.

This hint includes the exact async log path and name to make finding errors easier.
2022-05-02 08:49:13 -04:00
David Steele
45c3f4d53c
Improve JSON handling.
Previously read/writing JSON required parsing/render via a variant, which add many more memory allocations and loops.

Instead allow JSON to be read/written serially to improve performance and simplify the code. This also allows us to get rid of many String and Variant constant which are no longer required.

The goal is to be able to read/write very large (e.g. gigabyte manifest) JSON structures, which would not be practical with the current code.

Note that external JSON (GCS, S3, etc) is still handled using variants. Converting these will require more consideration about key ordering since it cannot be guaranteed as in our own formats.
2022-04-25 09:06:26 -04:00
David Steele
c304fafd45
Refactor PgClient to return results in Pack format.
Packs support stronger typing than JSON and are more efficient. For the small result sets that we deal with efficiency is probably not very important, but this removes another place where we are using JSON instead of Pack.

Push checking for result struct (e.g. single row) down into PgClient since it has easy access to this information rather than needing to parse the result set to find out.

Refactor all code downstream that depends on PgClient results.
2022-04-20 08:36:53 -04:00
David Steele
214ee9eb0e Fix URL for apt.p.o archives.
A new archive repo was created in March of 2020: https://www.df7cb.de/blog/2020/apt-archive.postgresql.org.html
2022-04-17 09:41:22 -04:00
David Steele
fa40bcdc5c
Throw error when unable to read lock process.
Previously the process id was skipped if it did not exist. Instead, throw an error and handle the errors in downstream code.

This was probably ignored at some point to provide backward-compatibility, but that is no longer required, if it ever was.
2022-04-11 14:08:16 -04:00
David Steele
79b2041663
Add lockRead*() functions for reading locks from another process.
Sometimes we need to read a lock from another process. This was done two different ways and in the case of cmdStop() was definitely hacky.

Centralize the logic to make it easier to read the locks for another process. This will also make it easier to add new lock data.
2022-04-08 15:55:41 -04:00
Reid Thompson
aad7171940
Suppress existing WAL warning when archive-mode-check is disabled.
When archive-mode-check is disabled and archive-push is running from multiple hosts, it is very likely that the file will already exist with the same checksum, so disable the warning.

However, if the checksums do not match, an error will still be thrown.
2022-04-08 15:00:20 -04:00
David Steele
cff147a7d2
Add default for boolean options with unresolved dependencies.
If a boolean option had an unresolved dependency then the value would be NULL, which meant the dependency would need to be checked in the code to avoid an error. For example, cfgOptionBool(cfgOptOnline) needed to be checked before it was safe to call cfgOptionBool(cfgOptArchiveCheck).

Allow a default for boolean options when they are unresolved to simplify the code. This makes using the options easier and less prone to error. Not all boolean options get a dependency default in this commit, but more may be added in the future.
2022-04-06 14:45:51 -04:00
Reid Thompson
d8d4132118
Auto-select backup for restore command --type=lsn.
For PITR with --type=lsn, attempt to auto-select the appropriate backup set based on the --target LSN provided. Pick the most recent backup where backup-lsn-stop is less than or equal to the provided LSN.
2022-04-05 11:59:12 -04:00
David Steele
c222ce1a5f Fix tls-server-auth example and add clarifications. 2022-03-25 08:52:41 -06:00
Abubakar Mohammed
3dd7960451 Fix incorrect reference to stanza-update in the user guide.
This should be stanza-upgrade. Also fix in the git history cache since the comment was copied from the user guide.
2022-03-24 15:59:41 -06:00
David Steele
424008d293 Allow files that become zero-length after the backup manifest is built.
It is possible that a file will be be truncated to zero-length after the backup manifest has been built. We could build logic into backupFile() to handle this case but it is hard to test well because of the race condition so tests would need to written directly against backupFile() and backupJobResult(). It hardly seems worth all that effort for a condition that occurs rarely, if ever.

Instead just remove the manifest check and add tests to restore to make sure it handles bundled zero-length files correctly. Logging will show that the file was bundled so if it happens a lot (which seems very unlikely) then we can think about an alternate implementation.
2022-03-23 10:41:36 -06:00
David Steele
fe9fd2ff2d Disable repo-hardlink option when repo-bundle option is enabled.
Hardlinking will not work with bundles because files are not stored individually.
2022-03-22 09:02:33 -06:00
David Steele
7afaac0a3d
Allow repo-hardlink option to be changed after full backup.
This rule was added because there were not sufficient tests to demonstrate that the repo-hardlink option could be changed in a backup set.

Remove the restriction and add/update tests to show that it works.

This is necessary now because bundling requires that hardlinking be disabled. Rather than add code complexity, it seems better just to address this limitation.
2022-03-22 08:35:34 -06:00
Reid Thompson
5ae84d5e47
Improve path validation for repo-* commands.
Check for invalid path in repo-* commands. Perform path validation and throw an error when appropriate. Path may not contain '//'. Strip trailing '/' from path. Absolute path must fall under repo path.
2022-03-22 07:50:26 -06:00
nunopi
21cef09dfd
Add AWS IMDSv2 support.
IMDSv2 provides additional security to prevent instance metadata from being read by an attacker.

All AWS instances should provide IMDSv2 but still fail back to IMDSv1 if the IMDSv2 token request fails. This is in case there are any services outside AWS that are emulating IMDSv1 but have not implemented IMDSv2.
2022-03-16 11:02:29 -06:00
David Steele
3f66f42ef9
Rename bundle-* options to repo-bundle-*.
It seems best for these to be repo options so they can be configured per repo, rather than globally.

All clarify usage for repo-bundle-size and repo-bundle-limit.
2022-03-14 17:49:52 -06:00
Reid Thompson
7c9208ba85
Improve error message for invalid repo-azure-key.
Check that repo-azure-key is valid base64 when repo-azure-key-type = shared.
2022-03-11 10:10:02 -06:00
David Steele
0054677147 Add bundle logging to backup command.
This was added to the restore command so add it to the backup command as well.
2022-03-09 15:34:15 -06:00
David Steele
dca6da86bf
Optimize restore command for file bundling.
Since files are stored sequentially in a bundle, it is often possible to restore multiple files with a single read. Previously, each restored file required a separate read. Reducing the number of reads is particularly beneficial for object stores, but performance should benefit on any file system.

Currently if there is a gap then a new read is required. In the future we might set a limit for how large a gap we'll skip without starting a new read.
2022-03-09 15:03:28 -06:00
Reid Thompson
f7ab002aa7
Improve stop command to honor stanza option.
Improve the stop command, when force and stanza options are specified, to terminate only processes holding lock files for the given stanza. Prior to these changes, termination of all processes holding lock files regardless of stanza occurred.
2022-03-08 12:18:23 -06:00
David Steele
514137040e Add limit parameter to ioCopyP().
Allows the number of bytes copied to be limited.
2022-03-08 08:23:31 -06:00
David Steele
166039c0da Fix example for repo-gcs-key-type option in configuration reference.
This looks like a copy-paste error from another option.
2022-03-08 08:09:58 -06:00
Reid Thompson
330e19900e
Increase precision of percent complete logging for backup and restore.
For very large backups only getting an update per percent may not be often enough.

Add hundredths to the percent complete logging to provide more timely information.
2022-03-06 13:01:24 -06:00
David Steele
c242b966e0 Begin v2.39 development. 2022-03-06 11:14:01 -06:00
David Steele
5249b89a2e v2.38: Minor Bug Fixes and Improvements
IMPORTANT NOTE: Repository size reported by the info command is now entirely based on what pgBackRest has written to storage. Previously, in certain cases, pgBackRest could detect if additional compression was being applied by the storage but this is no longer supported.

Bug Fixes:

* Retry errors in S3 batch file delete. (Reviewed by Reid Thompson. Reported by Alex Richman.)
* Allow case-insensitive matching of HTTP connection header values. (Reviewed by Reid Thompson. Reported by Rémi Vidier.)

Features:

* Add support for AWS S3 server-side encryption using KMS. (Contributed by Christoph Berg. Reviewed by David Steele, Tharindu Amila.)
* Add archive-missing-retry option. (Reviewed by Stefan Fercot.)
* Add backup type filter to info command. (Contributed by Stefan Fercot. Reviewed by David Steele.)

Improvements:

* Retry on page validation failure during backup. (Reviewed by Stephen Frost, David Christensen.)
* Handle TLS servers that do not close connections gracefully. (Reviewed by Rémi Vidier, David Christensen, Stephen Frost.)
* Add backup LSNs to info command output. (Contributed by Stefan Fercot. Reviewed by David Steele.)
* Automatically strip trailing slashes for repo-ls paths. (Contributed by David Christensen. Reviewed by David Steele.)
* Do not retry fatal errors. (Reviewed by Reid Thompson.)
* Remove support for PostgreSQL 8.3/8.4. (Reviewed by Reid Thompson, Stefan Fercot.)
* Remove logic that tried to determine additional file system compression. (Reviewed by Reid Thompson, Stefan Fercot.)

Documentation Bug Fixes:

* Move repo options in TLS documentation to the global section. (Reported by Anton Kurochkin.)
* Remove unused backup-standby option from stanza commands. (Reported by Stefan Fercot.)
* Fix typos in help and release notes. (Fixed by Daniel Gustafsson. Reviewed by David Steele.)

Documentation Improvements:

* Add aliveness check to systemd service configuration. (Suggested by Yogesh Sharma.)
* Add FAQ explaining WAL archive suffix. (Contributed by Stefan Fercot. Reviewed by David Steele.)
* Note that replications slots are not restored. (Contributed by Reid Thompson. Reviewed by David Steele, Stefan Fercot. Suggested by Christophe Courtois.)
2022-03-06 10:30:59 -06:00
David Steele
59a5373cf8
Handle TLS servers that do not close connections gracefully.
Some TLS server implementations will simply close the socket rather than correctly closing the TLS connection. This causes problems when connection: close is specified with no content-length or chunked encoding and we are forced to read to EOF. It is hard to know if this is a real EOF or a network error.

In cases where we can parse the content and (hopefully) ensure it is correct, allow the closed socket to serve as EOF. This is not ideal, but the change in 8e1807c means that currently working servers with this issue will stop working after 2.35 is installed, which seems too risky.
2022-03-02 11:38:52 -06:00
David Steele
f1bdf3e04b Add aliveness check to systemd service configuration.
If the pgbackrest service service fails to start then the user will get an error. However, retries will continue in the background.
2022-03-02 10:59:06 -06:00
David Steele
a66ec8d549 Revert PGDG yum repo workaround for aarch64.
da0f3a855 used a workaround to get the documentation building on aarch64 but recent changes to the PGDG yum repo have broken this workaround. Installing the regular way still doesn't work, either.

Reverting for now to get the CI pipeline working again.
2022-03-02 10:22:15 -06:00
David Steele
53de3e3aeb Move repo options in TLS documentation to the global section.
These options were mistakenly added to the stanza section, which works in certain cases, but is not best practice.
2022-02-26 11:17:05 -06:00
Stefan Fercot
98d525dba4
Add FAQ explaining WAL archive suffix. 2022-02-25 14:03:09 -06:00
David Steele
b33cabe08c
Allow case-insensitive matching of HTTP connection header values.
The specification allows values for the connection header to be case-insensitive. See https://www.rfc-editor.org/rfc/rfc7230#section-6.1.
2022-02-25 10:51:40 -06:00
David Christensen
6320712323
Automatically strip trailing slashes for repo-ls paths.
Trailing slashes in at least some of the repository storage types were preventing repo-ls from displaying any content (presumably due to storage-specific behavior).

Since the path with the slash should be equivalent to the path without the slash, just remove it if provided by the user.
2022-02-23 13:53:02 -06:00
David Steele
53f1b25204
Improve validation of zero pages.
Checking that pd_upper == 0 is not enough since this field may be corrupted. Still use pd_upper as a quick check, but when it is zero proceed to check the rest of the page to ensure it is also all zeroes.
2022-02-23 13:17:14 -06:00
David Steele
9eec98c613
Retry on page checksum validation failure during backup.
Rather than attempting to filter page checksum failures by LSN, just retry when there is a page checksum failure. If the page has not changed since the last read report it as an error. If the page has changed, then PostgreSQL must be modifying the page so we can ignore the error because a full page write (and possibly updates) will be in the WAL.

Also remove tests made redundant by the test merge in b4897077.
2022-02-23 12:05:53 -06:00
David Steele
5c332a0b10 Fix release note attribution in cacfdd94 and f798458e. 2022-02-23 11:23:32 -06:00
Daniel Gustafsson
f798458e1d
Disconnect help.auto.c from build-help in Makefile.
When there was an issue with the system library path during building, the build-help rule would fail during executing ./build-help with the effect that main.c wouldn't build.

Break out help.auto.c generation from the build-help stage to allow it to be re-executed when the library path has been corrected.
2022-02-23 10:29:17 -06:00
David Steele
10038db9c9
Add archive-missing-retry option.
Retry a WAL segment that was previously reported as missing by the archive-get command. This prevents notifications in the spool path from a prior restore from being used and possibly causing a recovery failure if consistency has not been reached.

Disabling this option allows PostgreSQL to more reliably recognize when the end of the WAL in the archive has been reached, which permits it to switch over to streaming from the primary. With retries enabled, a steady stream of WAL being archived will cause PostgreSQL to continue getting WAL from the archive rather than switch to streaming.

When disabling this option it is important to ensure that the spool path for the stanza is empty. The restore command does this automatically if the spool path is configured at restore time. Otherwise, it is up to the user to ensure the spool path is empty.
2022-02-23 09:14:27 -06:00
Daniel Gustafsson
cacfdd94d7
Fix typos in help and release notes. 2022-02-23 08:01:38 -06:00
David Steele
e6e1122dbc Pass file by reference in manifestFileAdd().
Coverity complained that this pass by value was inefficient:

CID 376402:  Performance inefficiencies  (PASS_BY_VALUE)
Passing parameter file of type "ManifestFile" (size 136 bytes) by value.

This was completely intentional since it gives us a copy of the struct that we can change without bothering the caller. However, updating fields is fine and may benefit the caller at some future data, and in any case does no harm now.

And as usual it is easier not to fight with Coverity.
2022-02-20 16:45:07 -06:00
David Steele
efc09db7b9
Limit files that can be bundled.
Limit which files can be added to bundles, which allows resume to work reasonably well. On resume, the bundles are removed and any remaining file is eligible to be to be resumed.

Also reduce the bundle-size default to 20MiB. This is pretty arbitrary, but a smaller default seems better.
2022-02-17 07:25:12 -06:00
David Steele
34d649579e
Bundle files in the repository during backup.
Bundle (combine) smaller files during backup to reduce the number of files written to the repository (enable with --bundle). Reducing the number of files is a benefit on all file systems, but especially so on object stores such as S3 that have a high file creation cost. Another benefit is that zero-length files are only stored as metadata in the manifest.

Files are batched up to bundle-size and then compressed/encrypted individually and stored sequentially in the bundle. The bundle id and offset of each file is stored in the manifest so files can be retrieved randomly without needing to read the entire bundle. Files are ordered by timestamp descending when being assigned to bundles to reduce the amount of random access that needs to be done. The idea is that bundles with older files can be read in their entirety on restore and only bundles with newer files will get fragmented.

Bundles are a custom format with metadata stored in the manifest. Tar was considered but it is too limited a format, the major issue being that the size of the file must be known in advance and that is very contrary to how pgBackRest works, especially once we introduce page-level incremental backups.

Bundles are stored numbered in the bundle directory. Some files may still end up in pg_data if they are added after the backup is complete. backup_label is an example.

Currently, only the backup command works in batches. The restore and verify commands use the offsets to pull individual files out of the bundle. It seems better to finalize how this is going to work before optimizing the other commands. Even as is, this is a major step forward, and all commands function with bundling.

One caveat: resume is currently not supported when bundle is enabled.
2022-02-14 13:24:14 -06:00