1
0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2025-04-21 11:57:01 +02:00

44 Commits

Author SHA1 Message Date
David Steele
e513c52c09 Add macros to create constant Buffer objects.
These are more efficient than creating buffers in place when needed.

After replacement discovered that bufNewStr() and BufNewZ() were not being used in the core code so removed them.  This required using the macros in tests which is not the usual pattern.
2019-04-20 08:16:17 -04:00
David Steele
9382283586 Fix issues when a path option is / terminated.
This condition was not being properly checked for in the C code and it caused problems in the info command, at the very least.

Instead of applying a local fix, introduce a new path option type that will rigorously check the format of any incoming paths.

Reported by Marc Cousin.
2019-03-14 13:48:33 +04:00
David Steele
60fe5b7365 Error when parameters are passed to a command that does not accept parameters.
This behavior allowed a command like this to run without error:

pgbackrest backup --stanza=db full

Even though it actually performed an incremental backup in most circumstances because the `full` parameter was ignored.

Instead, output an error and exit.

Suggested by Jason O'Donnell.
2019-03-09 11:03:47 +02:00
David Steele
332a68ea8d Fix incorrect config constant introduced in 5e3b7cbe.
This commit introduced PGBACKREST_CONFIG_ORIG_PATH_FILE_STR as a String constant for PGBACKREST_CONFIG_ORIG_PATH_FILE but failed to get the value correct.

Also, no test was added for PGBACKREST_CONFIG_ORIG_PATH_FILE_STR to prevent regressions as there is for  PGBACKREST_CONFIG_ORIG_PATH_FILE.
2018-11-16 08:48:02 -05:00
David Steele
5404628148 Fix incorrect error message for duplicate options in configuration files.
Duplicating a non-multi-value option was not throwing the correct message when the option was a boolean.

The reason was that the option was being validated as a boolean before the multi-value check was being done.  The validation code assumed it was operating on a string but was instead operating on a string list causing an assertion to fail.

Since it's not safe to do the multi-value check so late, move it up to the command-line and configuration file parse phases instead.

Reported by Jesper St John.
2018-09-27 17:48:40 +01:00
David Steele
51484a008f Add bufNewZ() to Buffer object.
This constructor creates a Buffer object directly from a zero-terminated string.  The old way was to create a String object first, then convert that to a Buffer using bufNewStr().

Updated in all places that used the old pattern.
2018-09-26 18:46:52 +01:00
David Steele
c47fc28c8b Make comment blocks consistent across all tests. 2018-09-16 14:15:21 -04:00
David Steele
800afeef70 Posix file functions now differentiate between open and missing errors.
The Perl functions do so and the integration tests rely on checking for these errors.  This has been exposed as more functionality is moved into C.

Passing the errors types is now a bit complicated so instead use a flag to determine which errors to throw.
2018-09-04 17:47:23 -04:00
David Steele
c2d0a21d63 Allow secrets to be passed via environment variables.
When environment variables were added in d0b9f986 they were classified as cfgSourceParam, but one of the restrictions on this type is that they can't pass secrets because they might be exposed in the process list.

The solution is to reclassify environment variables as cfgSourceConfig.  This allows them to handle secrets because they will not pass values to subprocesses as parameters.  Instead, each subprocess is expected to check the environment directly during configuration parsing.

In passing, move the error about secrets being passed on the command-line up to command-line parsing and make the error more generic with respect to the configuration file now that multiple configuration files are allowed.
2018-08-30 18:44:40 -04:00
David Steele
ad989068d2 Fix issue when a boolean option (e.g. delta) was specified more than once.
Reported by Yogesh Sharma.
2018-08-22 16:45:25 -04:00
David Steele
d0b9f986a0 Allow any option to be set in an environment variable.
This includes options that previously could only be specified on the command line, e.g. stanza.
2018-08-15 10:52:53 -04:00
David Steele
4a822d3032 Correct OptionInvalidError to OptionInvalidValueError in boolean option parsing. 2018-08-15 10:32:18 -04:00
David Steele
cd5df3570b Validate configuration options in a single pass.
By pre-calculating and storing the option dependencies in parse.auto.c validation can be completed in a single pass, which is both simpler and faster.
2018-08-11 12:55:33 -04:00
David Steele
f06bf9e832 Improve error message when a command is missing the stanza option.
Suggested by Sarah Conway.
2018-08-11 09:47:07 -04:00
David Steele
429a356e33 Enable -Wstrict-prototypes and update all void functions to conform. 2018-08-03 19:19:14 -04:00
David Steele
666537cbde Update code count for new file types and exclusions. 2018-07-28 08:13:30 -04:00
David Steele
8568622a6f Add --log-level-test option.
This allows setting the test log level independently from the general test harness setting, but current only works for the C tests. It is useful for seeing log output from functions on the console while a test is running.
2018-07-20 19:03:46 -04:00
David Steele
58e9f1e50c Refactor the common/log tests to not depend on common/harnessLog.
common/harnessLog was not ideally suited for general testing and made all the tests quite awkward. Instead, move all code used to test the common/log module into the logTest module and repurpose common/harnessLog to do log expect testing for all other tests in a cleaner way.

Add a few exceptions for config testing since the log levels are reset by default in config/parse.
2018-07-20 18:51:42 -04:00
David Steele
52bc073234 Add stack trace macros to all functions.
Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 11:57:32 -04:00
Cynthia Shang
721432f6ae Update the archive-push-queue-max, manifest-save-threshold, and buffer-size options to accept values in KB, MB, GB, TB, or PB where the multiplier is a power of 1024.
Contributed by Cynthia Shang.
2018-04-25 15:46:05 -04:00
David Steele
bb8c315cad Storage object improvements.
* Add storageCopy(), storageMove(), and storagePathSync().
* Separate StorageFile object into separate read and write objects.
* Abstract out Posix file read/write objects.
2018-04-23 17:26:27 -04:00
Cynthia Shang
27aa5f756e Add support for additional pgBackRest configuration files.
Configuration files are loaded from the directory specified by the --config-include-path option.

Add --config-path option for overriding the default base path of the --config and --config-include-path option.

Contributed by Cynthia Shang.
2018-04-19 11:09:39 -04:00
David Steele
79b7552940 Rename archive-queue-max option to archive-push-queue-max.
This avoids confusion with the archive-get-queue-max option. The old option name will continue to be accepted.
2018-04-18 19:37:35 -04:00
David Steele
93fdb98d15 Storage object improvements.
* Convert all functions to variadic functions.
* Enforce read-only storage.
* Add storageLocalWrite() helper function. Add storageExists(), storagePathCreate(), storageRemove(), and storageStat().
* Add StorageFile object and storageOpenRead()/storageOpenWrite().
2018-04-03 12:25:21 -04:00
David Steele
c6457b070f Full branch coverage for config/parse module. 2018-03-21 13:46:08 -04:00
David Steele
8a1ce42c30 Immediately error when a secure option (e.g. repo1-s3-key) is passed on the command line.
Since pgBackRest would not pass secure options on to sub-processes an obscure error was thrown. The new error is much clearer and provides hints about how to fix the problem.

Update command documentation to omit secure options that cannot be specified on the command-line.

Reported by Brad Nicholson.
2018-03-15 12:02:09 -04:00
David Steele
dd3ce70810 Config test code writes secure options to a file instead of passing on the command-line. 2018-03-15 11:09:46 -04:00
Cynthia Shang
34898ce471 Error when multiple arguments are set in the config file for an option that does not accept multiple arguments.
Contributed by Cynthia Shang.
2018-03-08 14:15:05 -05:00
Cynthia Shang
45ffe37e4f Add cfgOptionTest().
Update cfgOption() calls that are better implemented as cfgOptionTest().

Contributed by Cynthia Shang.
2018-03-05 18:51:48 -05:00
David Steele
c07aa2ae19 Improve code documentation in config module. 2018-03-02 22:42:02 -05:00
David Steele
406acf9cba Enable -Wconversion and silence new warnings. 2018-03-02 12:07:12 -05:00
Cynthia Shang
00e9aca872 Fixed issue where --reset-* was not passing the default value to Perl.
Contributed by Cynthia Shang.
2018-02-13 15:58:14 -05:00
David Steele
7c58fe7451 Use 64-bit integers for config options.
In particular, archive-queue-max requires a greater range than is provided by a 32-bit integer.
2018-02-09 13:54:33 -05:00
David Steele
7d13087c4b Rename -no to -reset for non-boolean options. 2018-02-06 11:26:06 -05:00
David Steele
808b6000eb Fix some bugs in C config code:
1) Error when the same option is defined multiple times in a section using alternate names.
2) Fix logging of invalid command error.
3) Warn when a stanza-only option is in a global section.

Also, make a note to add validation of section names to the check command.

Per review by Cynthia Shang.
2018-02-05 15:48:26 -05:00
David Steele
db21b7a360 Cleanup usage of internal options.
Apply internal to options that need to be read to determine locality but should not appear in the help.
2018-02-05 11:50:09 -05:00
David Steele
be90028100 Rename db-* options to pg-* and backup-* options to repo-* to improve consistency.
* repo-* options are now indexed although only one is allowed.
* List deprecated option names in documentation and command-line help.
2018-02-03 18:27:38 -05:00
David Steele
298b33988c Fix non-compliant formatting for function declarations.
Per review by Cynthia Shang.
2018-01-31 18:22:25 -05:00
David Steele
a91a648019 Config parsing no longer implemented in Perl.
Options are passed to Perl as JSON from the C process.
2018-01-28 21:37:09 -05:00
David Steele
39cb971afb Add log module. 2018-01-16 13:52:20 -05:00
David Steele
526acca5bd Config parsing implemented in C. 2017-12-22 23:27:49 -05:00
David Steele
f89b027dec Improve String and StringList modules. 2017-12-22 19:30:15 -05:00
David Steele
8d4674031e More config parsing in C in preparation for all config parsing in C. 2017-12-05 10:09:07 -05:00
David Steele
915ae5662a Implement version command in C. 2017-11-28 21:44:05 -05:00