1
0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2025-06-25 00:16:54 +02:00
Commit Graph

85 Commits

Author SHA1 Message Date
9070325760 Improve performance of HTTPS client.
Buffering now takes the pending bytes on the socket into account (when present) rather than relying entirely on select(). In some instances the final bytes would not be flushed until the connection was closed.
2018-02-18 14:53:27 -05:00
1659598cfe Create a master list of errors in build/error.yaml.
The C and Perl errors lists are created automatically by Build.pm so they stay up to date.
2018-02-08 16:11:47 -05:00
b58a45e067 Implement help command in C. 2018-01-23 13:34:24 -05:00
a4c058d070 Add 30 second wait loop to lockAcquire() when fail on no lock enabled.
This should help prevent processes that are shutting down from interfering with processes that are starting up.
2018-01-17 15:03:55 -05:00
1f39a34abc Move lock release later in exitSafe().
This reduce the chance of a new process starting and acquiring a lock before the old process has exited.
2018-01-17 10:31:21 -05:00
0e9ba98a50 Add stanza-delete command to cleanup unused stanzas.
Contributed by Cynthia Shang.
2018-01-03 12:23:33 -05:00
7cf955425e The C library is now required.
This eliminates conditional loading and eases development of new library features.
2017-11-26 17:45:00 -05:00
b03c26968a Repository encryption support.
Contributed by Cynthia Shang.
2017-11-06 12:51:12 -05:00
f07d2a3d0d Add eof to S3 file driver (required for encryption support). 2017-11-06 12:40:54 -05:00
8d6a08a32b Library code for repository encryption support. 2017-11-03 13:57:58 -04:00
8674a4f7ae Allow functions with sensitive options to be logged at debug level with redactions.
Previously, functions with sensitive options had to be logged at trace level to avoid exposing them. Trace level logging may still expose secrets so use with caution.
2017-10-24 12:35:36 -04:00
d989cf8ac2 Replace dynamically built class hierarchies in I/O layer with fixed parent() calls. 2017-10-22 19:07:17 -04:00
e1cb9ae86a Improve labeling for errors in helper processes. 2017-10-21 14:40:14 -04:00
44c7c0475a Improve protocol error handling.
In particular, "stop" errors are no longer reported as "unexpected".
2017-10-21 14:11:33 -04:00
1f120f3fce Improve performance of list requests on S3.
Any beginning literal portion of a filter expression is used to generate a search prefix which often helps keep the request small enough to avoid rate limiting.

Suggested by Mihail Shvein.
2017-10-20 14:10:16 -04:00
904b83747c Perl error handler recognizes errors thrown from the C library. 2017-10-16 10:47:31 -04:00
f1e739b4aa Add C error handler.
Basic try ... catch ... finally pattern to simplify error handling in C.
2017-10-16 10:09:56 -04:00
c857015a38 Fixed an issue where warnings were being emitted in place of lower priority log messages during backup from standby initialization. 2017-09-19 10:03:53 -04:00
eea2ccc3ab Add HTTP retries to harden against transient S3 network errors. 2017-09-03 16:48:41 -04:00
80e5e8db01 Increase HTTP timeout for S3. 2017-09-01 19:16:57 -04:00
75511d2481 Allow content-length (in addition to chunked encoding) when reading XML data to improve compatibility with third-party S3 gateways.
Suggested by Victor Gdalevich.
2017-09-01 15:07:20 -04:00
36e576b483 Up to seven standbys can be configured for backup from standby.`
Contributed by Cynthia Shang.
2017-08-31 19:15:44 -04:00
206415d4c7 Fixed an issue that could cause compression to abort on growing files.
Reported by Jesper St John, Aleksandr Rogozin.
2017-08-30 16:34:05 -04:00
1e0ed07455 Configuration rules are now pulled from the C library when present. 2017-08-25 16:47:47 -04:00
038d47bcc0 Retry when S3 returns an internal error (500). 2017-08-08 17:15:01 -04:00
918c1c6f49 Add s3-repo-ca-path and s3-repo-ca-file options.
The options accommodate systems where CAs are not automatically found by IO::Socket::SSL, i.e. RHEL7, or to load custom CAs.

Suggested by Scott Frazer.
2017-06-22 18:22:49 -04:00
f596702c5b Improve S3 error reporting. 2017-06-21 20:46:49 -04:00
051c961151 S3 repository support. 2017-06-12 10:52:32 -04:00
de7fc37f88 Storage and IO layer refactor:
Refactor storage layer to allow for new repository filesystems using drivers. (Reviewed by Cynthia Shang.)
Refactor IO layer to allow for new compression formats, checksum types, and other capabilities using filters. (Reviewed by Cynthia Shang.)
2017-06-09 17:51:41 -04:00
2a23157bf2 Move modules in Protocol directory in subdirectories. 2017-05-15 11:12:14 -04:00
f207dc7123 Renumber return codes between 25 and 125 to avoid PostgreSQL interpreting some as fatal signal exceptions.
Suggested by Yogesh Sharma.
2017-04-12 12:41:34 -04:00
21a000d002 Silence some perl critic warnings.
* Variables::RequireNegativeIndice
* Variables::ProhibitUnusedVariables
* Variables::ProhibitAugmentedAssignmentInDeclaration
* Modules::ProhibitConditionalUseStatements
2017-04-10 17:23:39 -04:00
b63ede5614 Added ability to test warning messages.
Contributed by Cynthia Shang.
2017-04-10 13:53:19 -04:00
3d84f2ce5e Improvements to Ini.pm.
* Refactor Ini.pm to facilitate testing.
* Complete statement/branch coverage for Ini.pm.
* Improved functions used to test/munge manifest and info files.
2017-04-10 13:24:45 -04:00
367e06f4be Coverage testing always enabled on Debian-based containers.
* Full coverage is verified when specified.
* Modules marked with partial coverage will error if they are actually fully covered.
* Simplified test representation is DefineTest.
* Added new representation for queries in DefineTest and added API functions.
* Update modules using DefineTest to use new API.
2017-04-10 12:31:30 -04:00
eb4918d018 Fixed locking bug and refactoring.
* Fixed an issue where read-only operations that used local worker processes (i.e. restore) were creating write locks that could interfere with parallel archive-push. (Reported by Jens Wilke.)
* Simplify locking scheme. Now, only the master process will hold write locks (archive-push, backup) and not all the local and remote worker processes as before.
2017-04-07 10:31:49 -04:00
b2f43b5601 Allow functions to accept optional parameters as a hash.
Refactor File->list() and fileList() to accept optional parameters.
2017-04-04 21:17:19 -04:00
e2ac7e1ea6 Fixed a regression introduced in v1.13 that could cause backups to fail.
This happened if files were removed (e.g. tables dropped) while the manifest was being built.

Reported by Navid Golpayegani.
2017-02-13 19:59:14 -05:00
0daeace1cd Fixed alignment issues with multiline logging. 2017-02-05 19:58:52 -05:00
cded35791d Added --log-timestamp option.
Allow timestamps to be suppressed in logging. This is primarily used to avoid filters in the automated documentation.
2017-02-05 10:59:49 -05:00
55feae645e Log file banner is not output until the first log entry is written.
Suggested by Jens Wilke.
2017-02-05 10:37:54 -05:00
3e54aad88f Removed --lock option.
This option was introduced before the lock directory could be located outside the repository and is now obsolete.
2017-01-31 10:38:45 -05:00
7418559a67 Fixed a spurious archive-get log message that indicated an exit code of 1 was an abnormal termination. 2017-01-31 10:36:59 -05:00
f3ccf3b897 Added Exception::exceptionCode() and Exception::exceptionMessage() to simplify error handling logic. 2017-01-27 09:56:26 -05:00
6620594b77 Various improvements in Lock.pm:
* Allow for locks to be taken more than once in the same process without error.
* Lock directories can be created when more than one directory level is required.
* Clean up optionValid()/optionTest() logic in Lock.pm.
2017-01-27 09:53:46 -05:00
18db178ac3 Various improvements to the test suite:
* Allow logging to be suppressed via logDisable() and logEnable().
* Added more flexibility in initializing and cleaning up after modules and tests.
* testResult() suppresses logging and reports exceptions.
* testException() allows messages to be matched with regular expressions.
* Refactor name/locations of common modules that setup test environments.
2017-01-27 09:42:30 -05:00
0e4f51c271 Added integration for testing coverage with Devel::Cover. 2017-01-09 20:49:04 -05:00
39744a4f1d Fixed an issue where a missing user/group on restore could cause an "uninitialized value" error in File->owner().
Reported by Leonardo Avellar.
2017-01-05 15:19:56 -05:00
974a02dbe4 Improved stanza-create command so that it can repair broken repositories in most cases and is robust enough to be made mandatory.
Contributed by Cynthia Shang.
2016-12-20 16:52:20 -05:00
aecc0f737e Fixed a few directory syncs that were missed for the --repo-sync option. 2016-12-19 14:40:22 -05:00