1
0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2024-12-12 10:04:14 +02:00
Commit Graph

3182 Commits

Author SHA1 Message Date
Cynthia Shang
13dc8e68d7 Make --repo optional for backup command.
If there are multiple repos and the --repo option is not specified then backup will automatically select the highest priority repo.
2021-02-26 14:49:50 -05:00
Michael Schout
9243962b95
Allow custom config-path default with ./configure --with-configdir.
Add --with-confdir=DIR option to configure, which can be used to override the default configuration directory of /etc/pgbackrest.

Probably in the future it would be better to just leverage ${sysconfdir} which is based on prefix, but since previously the config directory was hard coded to /etc/pgbackrest, we retain that default value by not relying on sysconfdir for now.
2021-02-25 12:03:44 -05:00
Cynthia Shang
0ddc0380ff Remove restore default repo from integration tests.
The default is now to scan all repos so update the integration tests to reflect that.
2021-02-24 11:32:13 -05:00
David Steele
8f03c3574b Reduce default file log level for integration tests.
The real/all test could fill the ramdisk depending on which vm and pg version were selected.

Debug level should be fine for most purposes and the level can be increased when needed.
2021-02-24 08:27:58 -05:00
Cynthia Shang
065b2ff230 Refactor info command repoMin/Max. 2021-02-23 16:27:05 -05:00
Cynthia Shang
118d9e64fe Enhance restore command multi-repo support.
The restore command automatically defaults to selecting the latest backup from a single repository. With multiple repositories configured, the restore command will now default to selecting the latest backup from the first repository where backups exist. The order in which the repositories are checked is dictated by the pgbackrest.conf order.

To select from a specific repository, the --repo option can be passed (e.g. --repo=1). The --set option can be passed if a backup other than the latest is desired.
2021-02-23 16:17:27 -05:00
David Steele
bec3e20b2c Add archive-get command multi-repo support.
Repositories will be searched in order for the requested archive file.

Errors will be reported as warnings as long as a valid copy of the archive file is found.
2021-02-23 15:34:28 -05:00
Cynthia Shang
e28f6f11e9 Expire continues if an error occurs processing a repository.
Errors are logged to the log file rather than thrown. If, after processing all repos, one or more errors occurred, then a single error error will be thrown to indicate there were errors and the log file should be inspected.

Also update log messages to be more consistent with new patterns.
2021-02-23 12:20:02 -05:00
David Steele
e14036bf57 Remove unused header files in S3 and Azure storage modules. 2021-02-23 07:33:32 -05:00
David Steele
6fb9de9a48 Use list to search for WAL segments to preserve in queueNeed().
The regular expression predates strLstFind() on sorted lists. Using the list is both simpler and faster.
2021-02-23 06:35:45 -05:00
David Steele
00b60e564e Add base64url encoding.
For now only encoding is supported. Decoding is not needed and may never be.
2021-02-19 19:21:06 -05:00
David Steele
a1f4fd32a1 Add ASSERT_MSG().
Used when execution reaches an invalid location rather than an invalid condition.
2021-02-19 19:03:42 -05:00
David Steele
f6c3262861 Do not expose valid/validate functions from encode module.
These functions have never been used externally. Validation is always part of decoding so performing validation separately would be wasteful.
2021-02-19 18:25:50 -05:00
David Steele
edab2a0b89 Use switch rather than if-else for encoding types.
This is more efficient and the error case can be an assert rather than a runtime error.

For extra safety initialize destinationSize to SIZE_MAX to increase the chances of an error if the switch fails.
2021-02-19 17:57:13 -05:00
David Steele
abcbe0f9c1 Combine encode module files into a single file.
There is not enough code here to justify multiple files and declaring the functions for each encoding as static allows the compiler to inline where appropriate.
2021-02-19 17:25:00 -05:00
David Steele
d485609658 Add strNewEncode(), strCatEncode(), and bufNewDecode().
These constructors wrap encodeToStr() and decodeToBin(), making them convenient and safe by eliminating the need to create intermediate buffers. Encoding/decoding is performed directly into the target String/Buffer. Sizing of the destination buffer is handled by the new functions so it doesn't have to be done at each call site.
2021-02-19 17:05:15 -05:00
David Steele
5b98968605 Do not lower-case help summaries when first word is an acronym.
If the second letter is capital or a digit then the word is likely an acronym so don't lower-case the first letter.

For now only the digit case is checked since there are no summaries with a capital as the second letter.
2021-02-19 10:29:29 -05:00
David Steele
66a4ff496a Encode path before passing to HttpRequest.
GCS requires mixed encoding in the path so encoding inside HttpRequest does not work.

Instead, require the path to be correctly encoded before being passed to HttpRequest.
2021-02-19 09:05:32 -05:00
David Steele
1b4b3538cc Rename uri to path where appropriate in HTTP and storage modules.
The path was originally named uri due to the canonicalized path being called "canonicalized uri" in the S3 authentication documentation. The name got propagated everywhere from there.

This is not correct for general usage, however, so rename to path when describing the path component of an HTTP request.
2021-02-19 08:22:50 -05:00
David Steele
dcb79ab8fb Decode JSON \u escaped characters.
ASCII may occasionally be encoded (e.g. &) to prevent ambiguity depending on where the JSON is located.

Only ASCII can be decoded. In general Unicode should not be encoded in JSON.
2021-02-19 07:32:40 -05:00
David Steele
c4243331de Silence chmod warning of empty path when removing unit test data. 2021-02-19 07:24:59 -05:00
David Steele
3837e61a75
Fix option warnings breaking async archive-get/archive-push.
Option warnings will cause the async process to fail because a warning is logged but stdout is closed so the process aborts.

This bug has existed for quite some time, but it was made worse by abb8ebe because now the async role can have different valid options than the default role. Previously at least a warning would be emitted before the async process died.

Fix this by only allowing warnings for the default role. Warnings were already suppressed for local and remote roles so the logic already exists.
2021-02-18 13:29:09 -05:00
David Steele
d7befd4189 Fix tests that ensure log levels are not set for local/remote roles.
These tests were broken because they were being gated by resetLogLevel. So they were not setting the log levels, but not because of the role setting. Because resetLogLevel was being checked last coverage testing indicated that the tests were working.

Fix the resetLogLevel parameter in the tests and move resetLogLevel to be tested first so coverage reporting works as expected. This isn't perfect but it is an improvement.
2021-02-16 16:28:19 -05:00
David Steele
d29855bd0b
Fix stack overflow in cipher passphrase generation.
The destination buffer on the stack was not large enough to contain the zero-terminating character.

Increase the buffer size and add an assertion to prevent regressions.

Found on arm64 running musl libc. Other architectures and glibc do not seem to be affected though it is clearly a bug.
2021-02-12 10:08:47 -05:00
David Steele
920c746adb Better exclusions for configure help.
Exclude known unused options rather than trying to include used options. This works better when new options are added.
2021-02-10 15:54:37 -05:00
David Steele
6a717e032f Set config path in configure script.
This allows the config path to be modified with a parameter to the configure script, though this commit does not do that.

Update the Perl code generator to allow literals so that defaults can be C defines rather than static strings.
2021-02-10 14:46:26 -05:00
Cynthia Shang
3408f1ee2e Enhance expire command multi-repo support.
The expire command has been enhanced to expire backups and archives from all configured repositories by default.

In addition, it will accept the --repo option to expire backups and archives only from the specified repository. Using the --repo options the --set option can also be refined further to the specified repo. If --set is provided but the --repo option has not, then all repositories will be searched and retention settings will be applied on each whether the backup set has been found or not.
2021-02-10 12:03:52 -05:00
David Steele
26cbebbda7 Use latex sloppypar to fix monospace wrapping in PDF rendering.
Monospaced identifiers could end up running over if latex was not able to find a place to break the line. Using sloppypar forces breaks so monospaced identifiers don't run over or get broken up.

Also add vspace to admonitions so they have some separation from the prior text.
2021-02-10 09:10:51 -05:00
David Steele
c7d7280fa8 Ensure test user has permissions before removing test files.
This allows files to be deleted even when tests have limited the permissions.
2021-02-08 16:36:38 -05:00
David Steele
b2bba678a0 Add missing linefeeds. 2021-02-08 16:28:16 -05:00
David Steele
b9f0070d91 Update config.guess and config.sub to latest versions. 2021-02-08 13:44:50 -05:00
David Steele
00f06065e7 Begin v2.33 development. 2021-02-08 13:18:22 -05:00
David Steele
aadc9e2fe6 v2.32: Repository Commands
Bug Fixes:

* Fix resume after partial delete of backup by prior resume. (Reviewed by Cynthia Shang. Reported by Tom Swartz.)

Features:

* Add repo-ls command. (Reviewed by Cynthia Shang, Stefan Fercot.)
* Add repo-get command. (Contributed by Stefan Fercot, David Steele. Reviewed by Cynthia Shang.)
* Add archive-mode-check option. (Contributed by Stefan Fercot. Reviewed by David Steele, Michael Banck.)

Improvements:

* Improve archive-get performance. (Reviewed by Cynthia Shang.)
2021-02-08 09:08:16 -05:00
Cynthia Shang
d350d1cc21 Improve expire command documentation. 2021-02-05 11:48:07 -05:00
David Steele
b65c370346 Add repo-get command. 2021-02-05 10:39:03 -05:00
David Steele
218cd078a6 Add repo-ls command. 2021-02-05 10:07:43 -05:00
David Steele
73f61a1f60 Add missed comment. 2021-02-04 17:24:42 -05:00
David Steele
9154d73030 Add -g accidentally removed in 4e8d469f.
The tests all run fine without debug info but gdb and valgrind are a lot less useful without it.
2021-02-02 17:05:55 -05:00
Stefan Fercot
4b46115345
Add archive-mode-check option.
This option disallows the PostgreSQL archive_mode=always setting and disabling it allows the setting.
2021-02-02 13:43:14 -05:00
David Steele
101bf5d114 Log configuration parameters in config test harness.
This makes it easier to find the current configuration in tests.
2021-02-01 14:50:22 -05:00
David Steele
9baf5d7da7 Set dry-run before updating options in config test harness.
This is the same order that is used in cfgLoad() so apply it here as well.

Also remove vestigial HRN_FEATURE_STORAGE define.
2021-02-01 14:42:22 -05:00
David Steele
67d444b9e8 Add bufEmpty().
This seems more readable than bufUsed() == 0, just like 7d6c0319.
2021-02-01 09:22:01 -05:00
David Steele
7d6c0319f0 Add lstEmpty(), strLstEmpty(), and varLstEmpty().
This seems more readable than lst*Size() == 0.

Hopefully this will also eliminate usage of lst*Size() > 0/lst*Size() != 0 variants for the inverse.
2021-01-29 14:27:56 -05:00
David Steele
5281e31422 Add configurable error handlers.
The stackTrace and memContext error handlers were hard-coded which made testing the error module in isolation impossible.

Making the error handlers configurable also makes adding new ones in the future easier.
2021-01-27 17:25:13 -05:00
David Steele
8e9f04cc32 Add HRN_INTEST_* define to indicate when a test is being run.
This is useful for initialization that needs to be done for the test and all subsequent tests.

Use the new defines to implement initialization for sockets and statistics.
2021-01-27 16:54:41 -05:00
Cynthia Shang
d5b919e657
Update expire command log messages with repo prefix.
In preparation for multi-repo support, a repo tag is added in this commit to the expire command log and error messages. This change also affects the expect logs and the user-guide. The format of the tag is "repoX:" where X is the repo key used in the configuration.

Until multi-repo support has been completed, this tag will always be "repo1:".
2021-01-27 16:33:01 -05:00
David Steele
5d34bf3f38 Move cvtDoubleToStr() to strNewDbl().
This is a more logical location and it reduces the dependencies required to compile the common/convert module.
2021-01-27 11:50:10 -05:00
David Steele
87eb081a8f Make unit test builds incremental based on coverage in prior tests.
When building tests only include files covered by the current test or by prior tests. This increases performance (less compilation and linking) and also helps detect cross-dependencies in the code. Since there are currently cross-dependencies the depend option is used to document them and allow compilation. The idea is to resolve them incrementally over time.

Add the harness option to include harness modules when the minimum requirements for compilation are met.

Add the feature option to indicate which features are now available in the harness (based on source modules already tested). This allows conditional compilation in harness modules when some features are not yet available.
2021-01-27 10:57:42 -05:00
David Steele
73a8e91868 Disable DEBUG_TEST_TRACE when NDEBUG defined.
The underlying functions used by the FUNCTION_TEST*() macros will not be able when NDEBUG is defined so disable them.

It would be better to change #ifndef NDEBUG to #ifdef DEBUG_TEST_TRACE where needed to bring these into alignment but that will be a larger effort best undertaken separately.
2021-01-27 10:29:23 -05:00
David Steele
088df03355 Fix indentation in .cirrus.yml. 2021-01-26 19:59:14 -05:00