1
0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2024-12-14 10:13:05 +02:00
Commit Graph

3371 Commits

Author SHA1 Message Date
David Steele
a1f4fd32a1 Add ASSERT_MSG().
Used when execution reaches an invalid location rather than an invalid condition.
2021-02-19 19:03:42 -05:00
David Steele
f6c3262861 Do not expose valid/validate functions from encode module.
These functions have never been used externally. Validation is always part of decoding so performing validation separately would be wasteful.
2021-02-19 18:25:50 -05:00
David Steele
edab2a0b89 Use switch rather than if-else for encoding types.
This is more efficient and the error case can be an assert rather than a runtime error.

For extra safety initialize destinationSize to SIZE_MAX to increase the chances of an error if the switch fails.
2021-02-19 17:57:13 -05:00
David Steele
abcbe0f9c1 Combine encode module files into a single file.
There is not enough code here to justify multiple files and declaring the functions for each encoding as static allows the compiler to inline where appropriate.
2021-02-19 17:25:00 -05:00
David Steele
d485609658 Add strNewEncode(), strCatEncode(), and bufNewDecode().
These constructors wrap encodeToStr() and decodeToBin(), making them convenient and safe by eliminating the need to create intermediate buffers. Encoding/decoding is performed directly into the target String/Buffer. Sizing of the destination buffer is handled by the new functions so it doesn't have to be done at each call site.
2021-02-19 17:05:15 -05:00
David Steele
5b98968605 Do not lower-case help summaries when first word is an acronym.
If the second letter is capital or a digit then the word is likely an acronym so don't lower-case the first letter.

For now only the digit case is checked since there are no summaries with a capital as the second letter.
2021-02-19 10:29:29 -05:00
David Steele
66a4ff496a Encode path before passing to HttpRequest.
GCS requires mixed encoding in the path so encoding inside HttpRequest does not work.

Instead, require the path to be correctly encoded before being passed to HttpRequest.
2021-02-19 09:05:32 -05:00
David Steele
1b4b3538cc Rename uri to path where appropriate in HTTP and storage modules.
The path was originally named uri due to the canonicalized path being called "canonicalized uri" in the S3 authentication documentation. The name got propagated everywhere from there.

This is not correct for general usage, however, so rename to path when describing the path component of an HTTP request.
2021-02-19 08:22:50 -05:00
David Steele
dcb79ab8fb Decode JSON \u escaped characters.
ASCII may occasionally be encoded (e.g. &) to prevent ambiguity depending on where the JSON is located.

Only ASCII can be decoded. In general Unicode should not be encoded in JSON.
2021-02-19 07:32:40 -05:00
David Steele
c4243331de Silence chmod warning of empty path when removing unit test data. 2021-02-19 07:24:59 -05:00
David Steele
3837e61a75
Fix option warnings breaking async archive-get/archive-push.
Option warnings will cause the async process to fail because a warning is logged but stdout is closed so the process aborts.

This bug has existed for quite some time, but it was made worse by abb8ebe because now the async role can have different valid options than the default role. Previously at least a warning would be emitted before the async process died.

Fix this by only allowing warnings for the default role. Warnings were already suppressed for local and remote roles so the logic already exists.
2021-02-18 13:29:09 -05:00
David Steele
d7befd4189 Fix tests that ensure log levels are not set for local/remote roles.
These tests were broken because they were being gated by resetLogLevel. So they were not setting the log levels, but not because of the role setting. Because resetLogLevel was being checked last coverage testing indicated that the tests were working.

Fix the resetLogLevel parameter in the tests and move resetLogLevel to be tested first so coverage reporting works as expected. This isn't perfect but it is an improvement.
2021-02-16 16:28:19 -05:00
David Steele
d29855bd0b
Fix stack overflow in cipher passphrase generation.
The destination buffer on the stack was not large enough to contain the zero-terminating character.

Increase the buffer size and add an assertion to prevent regressions.

Found on arm64 running musl libc. Other architectures and glibc do not seem to be affected though it is clearly a bug.
2021-02-12 10:08:47 -05:00
David Steele
920c746adb Better exclusions for configure help.
Exclude known unused options rather than trying to include used options. This works better when new options are added.
2021-02-10 15:54:37 -05:00
David Steele
6a717e032f Set config path in configure script.
This allows the config path to be modified with a parameter to the configure script, though this commit does not do that.

Update the Perl code generator to allow literals so that defaults can be C defines rather than static strings.
2021-02-10 14:46:26 -05:00
Cynthia Shang
3408f1ee2e Enhance expire command multi-repo support.
The expire command has been enhanced to expire backups and archives from all configured repositories by default.

In addition, it will accept the --repo option to expire backups and archives only from the specified repository. Using the --repo options the --set option can also be refined further to the specified repo. If --set is provided but the --repo option has not, then all repositories will be searched and retention settings will be applied on each whether the backup set has been found or not.
2021-02-10 12:03:52 -05:00
David Steele
26cbebbda7 Use latex sloppypar to fix monospace wrapping in PDF rendering.
Monospaced identifiers could end up running over if latex was not able to find a place to break the line. Using sloppypar forces breaks so monospaced identifiers don't run over or get broken up.

Also add vspace to admonitions so they have some separation from the prior text.
2021-02-10 09:10:51 -05:00
David Steele
c7d7280fa8 Ensure test user has permissions before removing test files.
This allows files to be deleted even when tests have limited the permissions.
2021-02-08 16:36:38 -05:00
David Steele
b2bba678a0 Add missing linefeeds. 2021-02-08 16:28:16 -05:00
David Steele
b9f0070d91 Update config.guess and config.sub to latest versions. 2021-02-08 13:44:50 -05:00
David Steele
00f06065e7 Begin v2.33 development. 2021-02-08 13:18:22 -05:00
David Steele
aadc9e2fe6 v2.32: Repository Commands
Bug Fixes:

* Fix resume after partial delete of backup by prior resume. (Reviewed by Cynthia Shang. Reported by Tom Swartz.)

Features:

* Add repo-ls command. (Reviewed by Cynthia Shang, Stefan Fercot.)
* Add repo-get command. (Contributed by Stefan Fercot, David Steele. Reviewed by Cynthia Shang.)
* Add archive-mode-check option. (Contributed by Stefan Fercot. Reviewed by David Steele, Michael Banck.)

Improvements:

* Improve archive-get performance. (Reviewed by Cynthia Shang.)
2021-02-08 09:08:16 -05:00
Cynthia Shang
d350d1cc21 Improve expire command documentation. 2021-02-05 11:48:07 -05:00
David Steele
b65c370346 Add repo-get command. 2021-02-05 10:39:03 -05:00
David Steele
218cd078a6 Add repo-ls command. 2021-02-05 10:07:43 -05:00
David Steele
73f61a1f60 Add missed comment. 2021-02-04 17:24:42 -05:00
David Steele
9154d73030 Add -g accidentally removed in 4e8d469f.
The tests all run fine without debug info but gdb and valgrind are a lot less useful without it.
2021-02-02 17:05:55 -05:00
Stefan Fercot
4b46115345
Add archive-mode-check option.
This option disallows the PostgreSQL archive_mode=always setting and disabling it allows the setting.
2021-02-02 13:43:14 -05:00
David Steele
101bf5d114 Log configuration parameters in config test harness.
This makes it easier to find the current configuration in tests.
2021-02-01 14:50:22 -05:00
David Steele
9baf5d7da7 Set dry-run before updating options in config test harness.
This is the same order that is used in cfgLoad() so apply it here as well.

Also remove vestigial HRN_FEATURE_STORAGE define.
2021-02-01 14:42:22 -05:00
David Steele
67d444b9e8 Add bufEmpty().
This seems more readable than bufUsed() == 0, just like 7d6c0319.
2021-02-01 09:22:01 -05:00
David Steele
7d6c0319f0 Add lstEmpty(), strLstEmpty(), and varLstEmpty().
This seems more readable than lst*Size() == 0.

Hopefully this will also eliminate usage of lst*Size() > 0/lst*Size() != 0 variants for the inverse.
2021-01-29 14:27:56 -05:00
David Steele
5281e31422 Add configurable error handlers.
The stackTrace and memContext error handlers were hard-coded which made testing the error module in isolation impossible.

Making the error handlers configurable also makes adding new ones in the future easier.
2021-01-27 17:25:13 -05:00
David Steele
8e9f04cc32 Add HRN_INTEST_* define to indicate when a test is being run.
This is useful for initialization that needs to be done for the test and all subsequent tests.

Use the new defines to implement initialization for sockets and statistics.
2021-01-27 16:54:41 -05:00
Cynthia Shang
d5b919e657
Update expire command log messages with repo prefix.
In preparation for multi-repo support, a repo tag is added in this commit to the expire command log and error messages. This change also affects the expect logs and the user-guide. The format of the tag is "repoX:" where X is the repo key used in the configuration.

Until multi-repo support has been completed, this tag will always be "repo1:".
2021-01-27 16:33:01 -05:00
David Steele
5d34bf3f38 Move cvtDoubleToStr() to strNewDbl().
This is a more logical location and it reduces the dependencies required to compile the common/convert module.
2021-01-27 11:50:10 -05:00
David Steele
87eb081a8f Make unit test builds incremental based on coverage in prior tests.
When building tests only include files covered by the current test or by prior tests. This increases performance (less compilation and linking) and also helps detect cross-dependencies in the code. Since there are currently cross-dependencies the depend option is used to document them and allow compilation. The idea is to resolve them incrementally over time.

Add the harness option to include harness modules when the minimum requirements for compilation are met.

Add the feature option to indicate which features are now available in the harness (based on source modules already tested). This allows conditional compilation in harness modules when some features are not yet available.
2021-01-27 10:57:42 -05:00
David Steele
73a8e91868 Disable DEBUG_TEST_TRACE when NDEBUG defined.
The underlying functions used by the FUNCTION_TEST*() macros will not be able when NDEBUG is defined so disable them.

It would be better to change #ifndef NDEBUG to #ifdef DEBUG_TEST_TRACE where needed to bring these into alignment but that will be a larger effort best undertaken separately.
2021-01-27 10:29:23 -05:00
David Steele
088df03355 Fix indentation in .cirrus.yml. 2021-01-26 19:59:14 -05:00
David Steele
14f2b41bda Fix .editorconfig indent for yaml files. 2021-01-26 19:58:28 -05:00
David Steele
59e57c7231 Remove storageLocal() dependency from common/parse module.
storageLocal() requires the storage helper which is a huge dependency for this module.

Pass in a storage object instead.
2021-01-26 19:55:20 -05:00
David Steele
468507843f Replace storageLocal() with storageTest in postgres/interface test. 2021-01-26 19:34:39 -05:00
David Steele
77823af53d Add test in common/error module.
This is required for coverage when the common/error module is run with just the source files required to make it run, rather than all source files as we do now.

Likely something in the harness is providing coverage, but cover it explicitly so the coverage won't be lost if the harness changes.
2021-01-26 19:31:48 -05:00
David Steele
85ad1aa2be Remove version comment left over from Perl. 2021-01-26 12:47:52 -05:00
David Steele
456a300bb7 Remove too-verbose braces in switch statements.
The original intention was to enclose complex code in braces but somehow braces got propagated almost everywhere.

Document the standard for braces in switch statements and update the code to reflect the standard.
2021-01-26 12:10:24 -05:00
David Steele
f669da7dcc Use minio latest in documentation and integration tests.
At one time Minio had stability problems with latest but that appears to be resolved for the last year or so.

Use latest so we'll know if something breaks since Minio is frequently used in production.
2021-01-26 11:25:29 -05:00
Cynthia Shang
2e60b93709
Add backup verification to internal verify command.
This is phase 2 of verify command development (phase 1 was processing the archives and phase 3 will be reconciling the archives and backups). In this phase the backups are verified by verifying each file listed in the manifest for the backup and creating a result set with the list of invalid files, if any. A summary is then rendered.

Unit tests have been added and duplicate tests have been removed.
2021-01-26 11:21:36 -05:00
Cynthia Shang
5d48dd2fb3 Use explicit characters instead of Posix class in restore expression.
It is not clear how portable/supported the Posix character classes are. This way seems simpler and more portable.

Updated from 5c98157b.
2021-01-25 11:33:41 -05:00
Cynthia Shang
e251ec574a Add note about removing configuration to stanza-delete documentation. 2021-01-25 11:14:28 -05:00
Cynthia Shang
00fac1c0d1 Improve info command text output and --set handling.
The info command provides total sizes for files in the backup on the database as well as the repository. The text output and associated user documentation has been updated to provide more clarity regarding the sizes being displayed.

In addition, the info command is updated to allow a user to optionally specify the repository when requesting a specific backup set. In this case, the text output will reflect the status of the stanza, the cipher types and archive min/max over all the repositories instead of a single repository when the repo option is specified.
2021-01-25 09:19:05 -05:00