1
0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2024-12-16 10:20:02 +02:00
Commit Graph

2028 Commits

Author SHA1 Message Date
David Steele
a2ec1253e9 Add code classification exclusion missed in 3e1b06ac. 2019-05-30 10:44:35 -04:00
David Steele
64260b2e98 Build all docs with S3 using --var=s3-all=y
Force repo-type=s3 for all tests.  This is not currently the default for any OS builds.
2019-05-29 08:38:45 -04:00
David Steele
404284b90f Add internal flag for commands.
Allow commands to be skipped by default in the command help but still work if help is requested for the command directly.  There may be other uses for the flag in the future.

Update help for ls now that it is exposed.
2019-05-28 12:18:05 -04:00
David Steele
20e5b92f36 Add ls command.
Allows listing repo paths/files from the command-line, to be used primarily for testing and debugging.

This command is internal-only so the interface may change at any time without notice.
2019-05-28 10:03:48 -04:00
David Steele
3b3327eae6 Move tls/http statistics output to command/command.
This module already has the filtering required to keep these messages from being displayed by default for commands that output to stdout (e.g. info).
2019-05-28 09:50:59 -04:00
David Steele
3e1b06acaa Use minio as local S3 emulator in documentation.
The documentation was relying on a ScalityS3 container built for testing which wasn't very transparent.  Instead, use the stock minio container and configure it in the documentation.

Also, install certificates and CA so that TLS verification can be enabled.
2019-05-27 07:37:20 -04:00
David Steele
a474ba54c5 Refactoring path support in the storage module.
Not all storage types support paths as a physical thing that must be created/destroyed.  Add a feature to determine which drivers use paths and simplify the driver API as much as possible given that knowledge and by implementing as much path logic as possible in the Storage object.

Remove the ignoreMissing parameter from pathSync() since it is not used and makes little sense.

Create a standard list of error messages for the drivers to use and apply them where the code was modified -- there is plenty of work still to be done here.
2019-05-26 12:41:15 -04:00
David Steele
38f28bd520 Log TLS and HTTP statistics on exit.
These stats measure how efficiently TLS and HTTP are reusing connections (i.e. pipelining).
2019-05-26 12:32:49 -04:00
David Steele
819eda0818 Set log level of protocol processing functions to debug.
Setting these to trace effectively made debug level useless in local/remote processes since all debug messages were demoted to trace when called from these functions.
2019-05-26 12:29:29 -04:00
David Steele
d12d94c53c Make info(), pathCreate() and pathSync() optional for storage drivers.
These functions are not required for repository storage so make them optional and error if they are not implemented for non-repository storage, .e.g. pg or spool.

The goal is to simplify the drivers (e.g. S3) that are intended only for repository storage.
2019-05-24 18:09:16 -04:00
David Steele
39645fc1a9 Add pathRemove() and remove() to S3 storage.
These functions will be needed for the expire command.
2019-05-24 14:33:47 -04:00
David Steele
6c385dfa7a Add function to determine if HTTP response code is OK.
OK responses are a range rather than a single value, so simplify the check with a function.
2019-05-24 13:56:38 -04:00
David Steele
96770c529b storageList() returns an empty list by default for missing paths.
The prior behavior was to return NULL so the caller would know the path was missing, but this is rarely useful, complicates the calling code, and increases the chance of segfaults.

The .nullOnMissing param has been added to enable the prior behavior.
2019-05-24 13:12:56 -04:00
David Steele
707e4a29f0 Notify compiler that cryptoErrorCode() does not return. 2019-05-24 11:09:43 -04:00
David Steele
01f3024b5c Add missing const qualifier. 2019-05-24 11:06:11 -04:00
David Steele
39cb624831 Add missing menus to the new user guides.
Since the CentOS 6/7 user guides were generated as a single page they did not get menus.  Generate the entire site for each user guide so menus are included.
2019-05-24 07:45:03 -04:00
David Steele
04f8b4ea52 Use git url for website clone. 2019-05-24 07:41:55 -04:00
David Steele
ec9622cde8 Use the git log to ease release note management.
The release notes are generally a direct reflection of the git log.  So, ease the burden of maintaining the release notes by using the git log to determine what needs to be added.

Currently only non-dev items are required to be matched to a git commit but the goal is to account for all commits.

The git history cache is generated from the git log but can be modified to correct typos and match the release notes as they evolve.  The commit hash is used to identify commits that have already been added to the cache.

There's plenty more to do here.  For instance, links to the commits for each release item should be added to the release notes.
2019-05-22 18:54:49 -04:00
David Steele
86482c7db9 Reduce log level for all expect tests to detail.
The C code is designed to be efficient rather than deterministic at the debug log level.  As we move more testing from integration to unit tests it makes less sense to try and maintain the expect logs at this log level.

Most of the expect logs have already been moved to detail level but mock/all still had tests at debug level.  Change the logging defaults in the config file and remove as many references to log-level-console as possible.
2019-05-22 18:23:44 -04:00
David Steele
e4cc008b98 Remove duplicated hint.
At some point this hint got added to the underlying code so it is no longer needed here.
2019-05-22 11:31:38 -04:00
David Steele
ff9c1bed5d Save cipher-pass key/value missed in f492f057.
This value is required when encryption is enabled.

In passing simplify the expression used to skip the checksum when calculating the checksum.
2019-05-22 11:24:18 -04:00
David Steele
3da60f4b5e Don't escape / when rendering JSON.
/ is escaped in the spec but the Perl renderer we use does not escape it which leads to checksum mismatches between the two sets of code.

This particular escape seems to be a more recent addition to the spec and is targeted toward embedding JSON in JavaScript.

\/ is still allowed when parsing JSON.
2019-05-22 11:10:43 -04:00
David Steele
664054fea9 Remove RHEL package patch merged to upstream. 2019-05-22 06:55:31 -04:00
David Steele
e1cce9c614 Remove extraneous parens. 2019-05-21 14:07:35 -04:00
David Steele
936b8a289c Allow separate paragraphs in release items.
The first paragraph should match the first line of the commit message as closely as possible.  The following paragraphs add more information.

Release items have been updated back to 2.01.
2019-05-21 10:37:30 -04:00
David Steele
e3fe3434b4 Rename repo-s3-verify-ssl option to repo-s3-verify-tls.
The new name is preferred because pgBackRest does not support any SSL protocol versions (they are all considered to be insecure).

The old name will continue to be accepted.
2019-05-21 10:14:41 -04:00
David Steele
1bc84c6474 The local command for restore is implemented entirely in C.
This is just the part of restore run by the local helper processes, not the entire command.

Even so, various optimizations in the code (like pipelining and optimizations for zero-length files) should make the restore command faster on object stores.
2019-05-20 17:07:37 -04:00
Cynthia Shang
a839830333 Add most unimplemented functions to the remote storage driver.
Add pathCreate(), pathRemove(), pathSync(), and remove() to the driver.

Contributed by Cynthia Shang.
2019-05-20 16:19:14 -04:00
David Steele
bbf2e0d5b0 Remove extra linefeed. 2019-05-20 12:24:43 -04:00
David Steele
a9f589b87a Begin v2.15 development. 2019-05-20 12:23:40 -04:00
David Steele
0f685a33a0 v2.14: Bug Fix and Improvements
Bug Fixes:

* Fix segfault when process-max > 8 for archive-push/archive-get. (Reported by Jens Wilke.)

Improvements:

* Bypass database checks when stanza-delete issued with force. (Contributed by Cynthia Shang. Suggested by hatifnatt.)
* Add configure script for improved multi-platform support.

Documentation Features:

* Add user guides for CentOS/RHEL 6/7.
2019-05-20 11:51:58 -04:00
David Steele
51d67ce1ba Update contributor name. 2019-05-16 09:59:34 -04:00
Cynthia Shang
2b75f57232 Add linefeeds for clarity. 2019-05-16 09:56:48 -04:00
Cynthia Shang
19d8358cba Update mock/expire module test matrix so expect tests output.
Also add an error message to prevent regression.

Contributed by Cynthia Shang.
2019-05-16 09:53:55 -04:00
David Steele
c51274d1b6 Add user guides for CentOS/RHEL 6/7.
It would be better if the documentation could be generated on multiple operating systems all in one go, but the doc system currently does not allow vars to be changed once they are set.

The solution is to run the docs for each required OS and stitch the documentation together.  It's not pretty but it works and the automation in release.pl should at least make it easy to use.
2019-05-16 08:32:02 -04:00
David Steele
bc7b42e718 Restore index menu url default lost in b85e51d6.
The url for the menu item referring to the index (i.e. site root page) should use {[project-url-root]}.

This allows the url to be set to different values depending on the location of the index.
2019-05-16 07:35:45 -04:00
David Steele
a4561dc087 Add --out-preserve to preserve contents of output path.
Useful for building a single set of documentation over multiple doc.pl runs.
2019-05-16 07:29:17 -04:00
David Steele
bc30027d73 Fix typo in doc.pl command-line help. 2019-05-16 07:04:28 -04:00
Cynthia Shang
18d4cb5741 Bypass database checks when stanza-delete issued with force.
Previously it was not possible to delete a stanza if the PostgreSQL server could not be contacted.

Contributed by Cynthia Shang.
Suggested by Roman.
2019-05-15 13:14:58 -04:00
David Steele
5c1d4bcd0d Automate coverage summary report generation.
This report replaces the lcov report that was generated manually for each release.

The lcov report was overly verbose just to say that we have virtually 100% coverage.
2019-05-15 13:04:56 -04:00
David Steele
5bba72b874 Remove -Wswitch-enum compiler option.
The -Wswitch option included in -Wall provides the same level of coverage and allows enum options to be grouped into default.
2019-05-15 12:55:08 -04:00
David Steele
53f6232264 Improve coverage in config/parse module. 2019-05-14 10:35:39 -04:00
David Steele
1e5ab69cad Improve coverage in config/config module. 2019-05-13 19:10:24 -04:00
David Steele
57281d4792 Add MAX_ASSIGN() macro.
This improves coverage in cases where the compared values may be always ascending or descending.
2019-05-13 18:05:54 -04:00
David Steele
15a33bf74b Error on multiple option alternate names and simplify help command.
There are currently no options with multiple alternate (deprecated) names so the code to render them in the help command could not be covered.

Remove the uncovered code and add an error when multiple alternate names are configured.  It's not clear that the current code was handling this correctly, so it will need to be reviewed if it comes up again.
2019-05-13 17:10:41 -04:00
David Steele
2d2bec842a Improve coverage in perl/exec module. 2019-05-13 13:36:24 -04:00
David Steele
431532574b Add testUser() and testGroup().
Retrieve the name of the current user/group used for testing.
2019-05-13 07:51:11 -04:00
David Steele
31d0fe5f50 Improve log performance, simplify macros, rename logWill() to logAny().
Pre-calculate the value used by logAny() to improve performance and make it more likely to be inlined.

Move IF_LOG_ANY() into LOG_INTERNAL() to simplify the macros and improve performance of LOG() and LOG_PID().  If the message has no chance of being logged there's no reason to call logInternal().

Rename logWill() to logAny() because it seems more intuitive.
2019-05-11 18:20:57 -04:00
David Steele
87f36e814e Improve macros and coverage rules that were hiding missing coverage.
The branch coverage exclusion rules were overly broad and included functions that ended in a capital letter, which disabled all coverage for the statement.  Improve matching so that all characters in the name must be upper-case for a match.

Some macros with internal branches accepted parameters that might contain conditionals.  This made it impossible to tell which branches belonged to which, and in any case an overzealous exclusion rule was ignoring all branches in such cases.  Add the DEBUG_COVERAGE flag to build a modified version of the macros without any internal branches to be used for coverage testing.  In most cases, the branches were optimizations (like checking logWill()) that improve production performance but are not needed for testing.  In other cases, a parameter needed to be added to the underlying function to handle the branch during coverage testing.

Also tweak the coverage rules so that macros without conditionals are automatically excluded from branch coverage as long as they are not themselves a parameter.

Finally, update tests and code where missing coverage was exposed by these changes.  Some code was updated to remove existing coverage exclusions when it was a simple change.
2019-05-11 14:51:51 -04:00
David Steele
f819a32cdf Improve efficiency of FUNCTION_LOG*() macros.
Call stackTraceTestStop()/stackTraceTestStart() once per block instead of with every param call.  This was done to be cautious but is not necessary and slows down development.

These functions were never built into production so had no impact there.
2019-05-11 07:57:49 -04:00