1
0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2024-12-14 10:13:05 +02:00
Commit Graph

1768 Commits

Author SHA1 Message Date
David Steele
33d39d248c Allow if condition in documentation lists and list items. 2019-01-24 16:45:28 +02:00
David Steele
bec52b6f41 Allow if in manifest variables.
The code supported this feature but the manifest DTD did now allow it.
2019-01-24 11:11:12 +02:00
David Steele
db08656537 Rename FUNCTION_DEBUG_* and consolidate ASSERT_* macros for consistency.
Rename FUNCTION_DEBUG_* macros to FUNCTION_LOG_* to more accurately reflect what they do.  Further rename FUNCTION_DEBUG_RESULT* macros to FUNCTION_LOG_RETURN* to make it clearer that they return from the function as well as logging.  Leave FUNCTION_TEST_* macros as they are.

Consolidate the various ASSERT* macros into a single ASSERT macro that is always compiled out of production builds.  It was difficult to figure out when an assert would be checked with all the different types in play.  When ASSERTs are compiled in they will always be checked regardless of the log level -- tying these two concepts together was not a good idea.
2019-01-21 17:41:59 +02:00
David Steele
d245f8eb42 The info command is implemented entirely in C.
The C info code has already been committed but this commit wires it into main.

Also remove the info Perl code and tests since they are no longer called.
2019-01-21 13:51:45 +02:00
David Steele
f79af47bd4 Increase timeout in storage/s3 module to improve reliability. 2019-01-19 20:44:40 +02:00
David Steele
e57f1b5293 Fix escaping in JSON string parser. 2019-01-19 12:57:40 +02:00
David Steele
7355248d6b Add remote storage objects.
This is a partial implementation of remote storage with just enough functionality to get the info command working.  The client is written in C but the server is still in Perl, which limits progress until a C server is written.
2019-01-18 22:04:37 +02:00
David Steele
88201f37a3 Add ProtocolClient object and helper functions.
This is a complete protocol client implementation in C.

Currently there is no C server implementation so the C client is talking to a Perl server.  This won't work very long, though, as the protocol format, even though in JSON, has a lot of language-specific structure.  While it would be possible to maintain compatibility between C and Perl it's probably not worth the effort in the long run.

Just as in Perl there are helper functions to make constructing protocol objects easier.  Currently only repository remotes are supported.
2019-01-18 21:32:51 +02:00
David Steele
0986db630c Fix comment typo. 2019-01-18 13:15:43 +02:00
David Steele
9cac403f61 Add Exec object.
Executes a child process and allows the calling process to communicate with it using read/write io.

This object is specially tailored to implement the protocol layer and may or may not be generally applicable to general purpose
execution.
2019-01-18 11:45:40 +02:00
David Steele
797f8098d1 Add ioReadBuf() to easily read into a buffer.
Moves some boilerplate into a function and makes it easier to get coverage in cases where a single buffer read captures all the data.
2019-01-18 11:14:44 +02:00
David Steele
06d41b4dc0 Add cfgExecParam() to generate parameters for executing commands.
Parameters for the local/remote commands are based on parameters that are passed to the current command.

Generate parameters for the new command based on the intersection of parameters between the current command and the command to be executed.
2019-01-17 22:29:19 +02:00
David Steele
ecd56105e6 Add IoHandleRead and IoHandleWrite objects.
General i/o objects for reading and writing file descriptors, in particular those that can block.  In other words, these are not generally to be used with file descriptors for actual files, but rather pipes, sockets, etc.
2019-01-17 22:08:31 +02:00
David Steele
bf0c41d9d6 Add const VariantList * debug type. 2019-01-16 22:23:25 +02:00
David Steele
7d4bbf290c Fix difference in cipher type reporting missed in 8304d452.
The C code can't get the cipher type from the storage object because the C storage object does not have encryption baked in like the Perl code does.

Instead, check backup.info to see if encryption is enabled.  This will need to rethought if another cipher type is added but for now it works fine.
2019-01-16 22:16:50 +02:00
David Steele
e68d1e7304 Simplify info command text message when no stanza are present.
Replace the repository path with just "the repository".  The path is not important in this context and it is clearer to state where the stanzas are missing from.
2019-01-16 19:23:10 +02:00
David Steele
ef9dc89e08 Update Storage::Local->list() to accept an undefined path.
The Perl code has a tendency to generate absolute paths even when they are not needed. This change helps the C and Perl storage work together via the protocol layer.
2019-01-16 18:49:12 +02:00
David Steele
b4146b6bff Update Perl repo rules to work when stanza is not specified.
The C storage object strives to use rules whenever possible instead of generating absolute paths.  This change helps the C and Perl storage work together via the protocol layer.
2019-01-16 18:45:19 +02:00
David Steele
0014e15944 Fix escaping in JSON string renderer. 2019-01-16 18:34:50 +02:00
David Steele
49bd8e0e3e Fix missing linefeed and param debug. 2019-01-16 12:04:35 +02:00
David Steele
c0a9048536 Fix null output in JSON renderer.
In some cases nulls were being rendered as "(null)" because no special logic existed to handle them.
2019-01-16 09:11:01 +02:00
David Steele
ec26a0594e Allow string Variant objects to contain null.
This is more useful than the variant itself being null as it saves conditionals when creating.
2019-01-16 09:05:13 +02:00
David Steele
1b334da87f Add kvMove() and varLstMove().
Allow these objects to be moved to another mem context.
2019-01-14 22:02:23 +02:00
David Steele
aab9e38b9a Return UnknownError from errorTypeFromCode() for invalid error codes.
The prior behavior was to throw an exception but this was not very helpful when something unexpected happened.  Better to at least emit the error message even if the error code is not very helpful.
2019-01-14 21:34:22 +02:00
David Steele
2b02d37602 Allow empty arrays in JSON parser. 2019-01-14 21:21:37 +02:00
David Steele
8304d452b3 Make the C version of the info command conform to the Perl version.
There were some small differences in ordering and how the C version handled missing directories.  It may be that the C version is more consistent, but for now it is more important to be compatible with the Perl version.

These differences were missed because the C info command was not wired into main.c so it was not being tested in regression.  This commit does not fix the wiring issue because there will likely be a release soon and it is too big a change to put in at the last moment.
2019-01-13 22:44:58 +02:00
David Steele
f314a1f8aa Improve accuracy of strSizeFormat().
Casting to int caused large values to be slightly inaccurate so cast to uint64_t instead.

Also, use multiplication where possible since the compiler should precompute multiplied values.
2019-01-13 22:22:15 +02:00
David Steele
cebcb3f027 Null-terminate list returned by strLstPtr().
The null-terminator is required by many C functions that accept a string list so just add it in all cases rather than making it optional.
2019-01-10 09:49:28 +02:00
Magnus Hagander
f8a9fc2039 Fix typos in monitoring examples. 2019-01-09 19:20:38 +02:00
David Steele
ade2c3102d Ignore SIGPIPE signals and check EPIPE result instead.
SIGPIPE immediately terminates the process but we would rather catch the EPIPE error and gracefully shutdown.

Ignore SIGPIPE and throw the EPIPE error via normal error handling.
2019-01-06 17:42:44 +02:00
Douglas J Hunley
7272d6e247 Add _DARWIN_C_SOURCE flag to Makefile for MacOS builds.
For some reason adding -D_POSIX_C_SOURCE=200112L caused MacOS builds to stop working.  Combining both flags seems to work fine for all tested systems.

Contributed by Douglas J Hunley.
2019-01-06 17:28:17 +02:00
David Steele
9560baf659 Move C module include in test.c above headers included for testing.
Including the C module after the headers required for testing meant that if headers were missing from the C module they were not caught while directly testing the C module.

The missing headers were caught in general testing, but it is frustrating to get an error in a module that has already passed while testing another module or running CI.

Move the C module include to the very top so missing headers cause immediate failures.
2019-01-06 15:52:59 +02:00
David Steele
0e0000f59c Add debug macros and extra space missed in 256b727a. 2019-01-06 15:27:30 +02:00
David Steele
1de22cac2b Rename common/io/handle module to common/io/handleWrite.
ioHandleWriteOneStr() will become a helper function for the IoHandleWrite object.
2019-01-06 14:37:39 +02:00
David Steele
f52e0187e6 Fix object struct comment to be conforming. 2019-01-05 18:10:30 +02:00
David Steele
842147321f Fix typo in error message. 2019-01-05 14:43:40 +02:00
David Steele
59d7352f31 Command-line help update missed in 200fd3aa. 2019-01-05 14:38:31 +02:00
David Steele
200fd3aa05 Clarify that encryption is always performed client-side.
Suggested by Bruce Burdick.
2019-01-04 12:25:58 +02:00
David Steele
50717aa846 Begin v2.09 development. 2019-01-04 11:00:59 +02:00
David Steele
db24ff8df4 v2.08: Minor Improvements and Bug Fixes
Bug Fixes:

* Remove request for S3 object info directly after putting it. (Reported by Matt Kunkel.)
* Correct archive-get-queue-max to be size type. (Reported by Ronan Dunklau.)
* Add error message when current user uid/gid does not map to a name. (Reported by Camilo Aguilar.)
* Error when --target-action=shutdown specified for PostgreSQL < 9.5.

Improvements:

* Set TCP keepalives on S3 connections. (Suggested by Ronan Dunklau.)
* Reorder info command text output so most recent backup is output last. (Contributed by Cynthia Shang. Suggested by Ryan Lambert.)
* Change file ownership only when required.
* Redact authentication header when throwing S3 errors. (Suggested by Brad Nicholson.)
2019-01-02 22:04:47 +02:00
David Steele
98ba7ece23 Update LICENSE.txt for 2019. 2019-01-01 18:46:27 +02:00
David Steele
26c888873e Merge common/typeVariantListTest module into common/typeVariantTest.
These modules are closely related so it makes sense for them to be merged.
2019-01-01 18:14:43 +02:00
David Steele
07b9176f25 Merge common/typeStringListTest module into common/typeStringTest.
These modules are closely related so it makes sense for them to be merged.
2019-01-01 18:05:13 +02:00
David Steele
493fb78787 Add strLstInsert() and strLstInsertZ() to StringList object.
Use lstInsert() to implement insertions into any position in a StringList.
2019-01-01 17:31:23 +02:00
Cynthia Shang
cc6376fdb6 Modify general document elements to allow any child element.
This allows for nesting of elements such as <b> and <i>.

Contributed by Cynthia Shang.
2018-12-31 09:19:38 +02:00
Cynthia Shang
2b7440ddf3 Allow custom logo for PDF documentation.
Contributed by Cynthia Shang.
2018-12-31 08:51:00 +02:00
Cynthia Shang
72865ca33b Add admonitions to documentation renderers.
Admonitions call out places where the user should take special care.

Support added for HTML, PDF, Markdown and help text renderers.  XML files have been updated accordingly.

Contributed by Cynthia Shang.
2018-12-30 16:40:20 +02:00
Cynthia Shang
3dc327fd05 Add optional table captions.
All renderers now support table captions, when requested.

Contributed by Cynthia Shang.
2018-12-20 23:20:54 +02:00
Cynthia Shang
8cf8d29f12 Escape special characters in latex when not in a code block.
A number of common characters are not allowed in latex without being escaped.

Also convert some HTML-specific codes that are used in the documentation.

Contributed by Cynthia Shang.
2018-12-19 19:13:26 +02:00
David Steele
23b583336f Set TCP keepalives on S3 connections.
Keepalives may help in situations where RST packets are being blocked by a firewall or otherwise do not arrive.

The C code uses select on all reads so it should never block, but add keepalives just in case.

Suggested by Ronan Dunklau.
2018-12-18 22:12:59 +02:00