1
0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2024-12-14 10:13:05 +02:00
Commit Graph

2668 Commits

Author SHA1 Message Date
David Steele
ec173f12fb Add MEM_CONTEXT_PRIOR() block and update current call sites.
This macro block encapsulates the common pattern of switching to the prior (formerly called old) mem context to return results from a function.

Also rename MEM_CONTEXT_OLD() to memContextPrior().  This violates our convention of macros being in all caps but memContextPrior() will become a function very soon so this will reduce churn.
2020-01-17 13:29:49 -07:00
David Steele
b5fa9951e3 Use MEM_CONTEXT_BEGIN() block in varFree().
We probably arrived at this unusual construction because of the complexity of getting the mem context.  Whether or not this is a good way to store the mem context, it still makes sense to use the standard pattern for switching mem contexts.
2020-01-17 13:08:47 -07:00
David Steele
c6d6b7dbef Use MEM_CONTEXT_NEW_BEGIN() block instead of memContextNew().
A few places were using just memContextNew(), probably because they did not immediately need to create anything in the new context, but it's better if we use the same pattern everywhere, even if it results in a few extra mem context switches.
2020-01-17 11:58:41 -07:00
David Steele
d3be1e41a4 Add braces and spaces for clarity.
This should make it a little easier to see how the TRY macros work together.
2020-01-16 16:23:40 -07:00
David Steele
254f79803f Use MEM_CONTEXT_BEGIN() block instead of memContextSwitch().
This is the standard pattern but was missed here.
2020-01-16 16:20:59 -07:00
David Steele
8068a610d5 Use MEM_CONTEXT_NEW_BEGIN() block.
This pattern makes more sense.  The prior code was probably copy-pasted from code with slightly different requirements.
2020-01-16 16:19:45 -07:00
David Steele
4274fcbf6f Add missing semicolon.
This worked when the FUNCTION_TEST_RETURN_VOID() macro expanded to nothing because of the final semicolon.  If the FUNCTION_TEST_RETURN_VOID() macro expanded to something then there was one semicolon too few.
2020-01-16 14:42:01 -07:00
David Steele
731ffcfb3d Disable RHEL package builds since upstream is broken. 2020-01-16 14:29:04 -07:00
David Steele
e81629b442 Reclassify Perl and LibC code as test/harness.
These were still being included in the core totals but they are no longer used by core.
2020-01-15 13:53:30 -07:00
David Steele
193b78e150 Remove -s from make instructions.
It's a good option but not required so let the user make the choice.

Also combine configure and make into a single line.
2020-01-15 13:40:17 -07:00
David Steele
9414ad2300 Begin v2.22 development. 2020-01-15 13:29:52 -07:00
David Steele
2c0ba0820d v2.21: C Migration Complete
Bug Fixes:

* Fix options being ignored by asynchronous commands. The asynchronous archive-get/archive-push processes were not loading options configured in command configuration sections, e.g. [global:archive-get]. (Reviewed by Cynthia Shang. Reported by Urs Kramer.)
* Fix handling of \ in filenames. \ was not being properly escaped when calculating the manifest checksum which prevented the manifest from loading. Since instances of \ in cluster filenames should be rare to nonexistent this does not seem likely to be a serious problem in the field.

Features:

* pgBackRest is now pure C.
* Add pg-user option. Specifies the database user name when connecting to PostgreSQL. If not specified pgBackRest will connect with the local OS user or PGUSER, which was the previous behavior. (Contributed by Mike Palmiotto.)
* Allow path-style URIs in S3 driver.

Improvements:

* The backup command is implemented entirely in C. (Reviewed by Cynthia Shang.)
2020-01-15 13:21:52 -07:00
David Steele
8d3710b2fe Fix options being ignored by asynchronous commands.
The local, remote, archive-get-async, and archive-push-async commands were used to run functionality that was not directly available to the user. Unfortunately that meant they would not pick up options from the command that the user expected, e.g. backup, archive-get, etc.

Remove the internal commands and add roles which allow pgBackRest to determine what functionality is required without implementing special commands. This way the options are loaded from the expected command section.

Since remote is no longer a specific command with its own options, more manipulation is required when calling remote. This might be something we can improve in the config system but it may be worth leaving as is because it is a one-off, for now at least.
2020-01-15 12:24:58 -07:00
David Steele
c43ec9d38c Fix function return logging type.
This was working because the stack was being cleaned up by a try block at the call site.
2020-01-15 08:19:32 -07:00
David Steele
f8a29c266e Clarify comment. 2020-01-13 20:36:42 -07:00
David Steele
a7738ebba3 Update comments in command/remote module. 2020-01-13 13:21:28 -07:00
David Steele
a969a860ab Fix misaligned braces. 2020-01-13 11:13:37 -07:00
David Steele
fe263e87b1 Allow path-style URIs in S3 driver.
Although path-style URIs have been deprecated by AWS, they may still be used with products like Minio because no additional DNS configuration is required.

Path-style URIs must be explicitly enabled since it is not clear how they can be auto-detected reliably.  More importantly, faulty detection could cause regressions in current installations.
2020-01-12 11:31:06 -07:00
David Steele
069345d339 Add string constants. 2020-01-12 11:00:10 -07:00
David Steele
3f89ecf8d9 Add time to storage ls JSON output.
Time is supported in all drivers with the update to S3 at 61538f93, so it is now possible to add time to the ls command and have it work on all repo types.
2020-01-10 09:39:33 -07:00
David Steele
0fe7bb2ec4 Improve code that updates/removes pg options passed to a remote.
The prior code was updating/removing hard-coded options but new options are added all the time and there was no indication that this code needed to be updated.  For example, dc1e7ca2 added the pg-user option but this code was not updated.

Instead find the options to update/remove dynamically.  The new code uses prefixes, which is not perfect, but the patterns for pg options are pretty well established and this seems safer than the existing code.
2020-01-09 12:20:13 -07:00
David Steele
4c8653fc8b Update inaccurate comments. 2020-01-09 11:56:03 -07:00
Cynthia Shang
2e11389ed4 Fix comment typo. 2020-01-09 11:52:51 -07:00
David Steele
0c5c78e5e1 Make quoting in cfgExeParam() optional.
Parameter lists that are passed directly to exec*() do not need quoting when spaces are present.  Worse, the quotes will not be stripped and the option value will be garbled.

Unfortunately this still does not fix all issues with quoting since we don't know how it might need to be escaped to work with SSH command configuration.  The answer seems to be to pass the options in the protocol layer but that's beyond the scope of this commit.
2020-01-09 09:23:15 -07:00
David Steele
7de5ce23ad Add internal remote-type option.
This option was overloaded on the general type option but it makes sense to split this out since the meaning is pretty different.

Rename the values to conform to current standards, i.e. pg and repo, now that the Perl code won't care anymore.
2020-01-08 18:59:02 -07:00
David Steele
7a1871c341 Fix test log message to match pg-version parameter name.
It was confusing that this part of the log message did not match the parameter name, which made reproducing test failures from CI a little harder.
2020-01-08 09:54:44 -07:00
David Steele
8e1f78b4c8 Remove obsolete Perl code missed in f0ef73db. 2020-01-07 12:20:51 -07:00
David Steele
a84ae6be04 Fix comment typos. 2020-01-06 16:16:48 -07:00
David Steele
61538f932c Parse dates in storageS3InfoList() and storageS3Info().
Previously dates were not being filled by these functions which was fine since dates were not used.

We plan to use dates for the ls command plus it makes sense for the driver to be complete since it will be used as an example.
2020-01-06 15:53:53 -07:00
David Steele
d2fb4f977c Add httpLastModifiedToTime() to parse HTTP last-modified header. 2020-01-06 15:24:49 -07:00
David Steele
a08298ce1b Add basic time management functions.
These are similar to what mktime() and strptime() do but they ignore the local system timezone which saves having to munge the TZ env variable to do time conversions.
2020-01-06 15:18:52 -07:00
David Steele
fc47907fe3 Update LICENSE.txt for 2020. 2020-01-03 09:32:52 -07:00
David Steele
011bcb4869 Remove command option from Perl db application name.
The command option will be removed from the C code so it needs to be removed here as well.

This code is now used only for testing so it's not important that it be so precise.
2019-12-29 21:22:18 -07:00
David Steele
33e328abbf Remove unused LibC code.
The code was made obsolete by the migration to C.
2019-12-28 18:30:32 -07:00
David Steele
e72a9dd0d2 Add error parameter to cfgCommandId().
This allows commands to be checked for validity without generating an error.
2019-12-28 13:37:03 -07:00
David Steele
d28837a300 Remove obsolete function declaration. 2019-12-28 13:31:59 -07:00
David Steele
d41eea685a Change meaning of TEST_RESULT_STR() macro.
This macro was created before the String object existed so subsequent usage with String always included a lot of strPtr() wrapping.

TEST_RESULT_STR_Z() had already been introduced but a wholesale replacement of TEST_RESULT_STR() was not done since the priority was on the C migration.

Update all calls to (old) TEST_RESULT_STR() with one of the following variants: (new) TEST_RESULT_STR(), TEST_RESULT_STR_Z(), TEST_RESULT_Z(), TEST_RESULT_Z_STR().
2019-12-26 18:08:27 -07:00
David Steele
02d3918b32 Remove duplicate header and extra linefeed. 2019-12-19 19:37:55 -05:00
David Steele
74c3842595 Remove errant tabs and fix spacing. 2019-12-19 16:25:46 -05:00
Mike Palmiotto
7dfea6b717 Remove errant tab. 2019-12-19 16:20:11 -05:00
Mike Palmiotto
dc1e7ca22d Add pg-user option.
Specifies the database user name when connecting to PostgreSQL.

If not specified pgBackRest will connect with the local OS user or PGUSER, which was the previous behavior.
2019-12-19 11:26:38 -05:00
David Steele
9452084dd1 Fix misspellings of libpq. 2019-12-17 23:32:39 -05:00
David Steele
63a855e2f7 Fix misaligned continuation character. 2019-12-17 23:26:52 -05:00
David Steele
2f7e139534 Error if option prefix and index total are not both defined.
They are both required for indexed options.
2019-12-17 22:35:30 -05:00
David Steele
d780d084b7 Add comments about increasing Vagrantfile disk size. 2019-12-17 21:56:02 -05:00
Mike Palmiotto
d89d9f1c52 Skip vagrant disksize option if no plugin.
Previously, `vagrant up` would bail if no `vagrant-disksize` plugin was
installed. This option is just a nice-to-have, so skip it rather than
bailing.
2019-12-17 21:47:19 -05:00
David Steele
ac35dcac39 Add note about symlink required for contributing documentation to build. 2019-12-17 21:28:40 -05:00
Stephen Frost
3394a50170 Remove unused Perl library missed in f0ef73db. 2019-12-17 21:25:54 -05:00
David Steele
620386f034 Remove integration tests that are now covered in the unit tests.
Most of these tests are just checking that errors are thrown when required.  These are well covered in various unit tests.

The "cannot resume" tests are also well covered in the backup unit tests.

Finally, config warnings are well covered in the config unit tests.

There is more to be done here, but this accounts for the low-hanging fruit.
2019-12-17 20:14:45 -05:00
David Steele
977ec2e307 Integration test improvements for disk and memory efficiency.
Set log-level-file=off when more that one test will run.  In this case is it impossible to see the logs anyway since they will be automatically cleaned up after the test.  This improves performance pretty dramatically since trace-level logging is expensive.  If a singe integration test is run then log-level-file is trace by default but can be changed with the --log-level-test-file option.

Reduce buffer-size to 64k to save memory during testing and allow more processes to run in parallel.

Update log replacement rules so that these options can change without affecting expect logs.
2019-12-17 15:23:07 -05:00