1
0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2024-12-14 10:13:05 +02:00
Commit Graph

3589 Commits

Author SHA1 Message Date
David Steele
bd0081fec8 Update IoClient/IoSession to use StringIds.
Using StringId for the client/session type removes String constants and some awkward referencing/dereferencing needed to use a String constant in the interface.

Converting IoSessionRole to StringId removes a conditional in ioSessionToLog() and improves debug logging by outputting client/server instead of 0/1.
2021-04-28 12:37:22 -04:00
David Steele
0ec91f61c6 Convert ArchivePushFileIoType enum to StringId.
Improves debug logging by outputting open/write/close instead of 0/1/2.
2021-04-28 12:23:37 -04:00
David Steele
8394577c6a Convert ArchiveMode enum to StringId.
Improves debug logging by outputting get/push instead of 0/1.
2021-04-28 12:05:01 -04:00
David Steele
7dd01897fd Convert ProtocolStorageType enum to StringId.
Allows removal of protocolStorageTypeEnum()/protocolStorageTypeStr() and improves debug logging of the enum.
2021-04-28 11:59:04 -04:00
David Steele
bd68ed63ba Convert ProtocolParallelJobState enum to StringId.
Allows removal of protocolParallelJobToConstZ(), which was used only for debugging.
2021-04-28 11:43:08 -04:00
David Steele
85fc3da4c3
Update CipherType/CipherMode to StringId.
As in 6cc521b, this allows option values and enums to be easily mapped together.
2021-04-28 11:36:20 -04:00
Cynthia Shang
c3b15fc3bd Fix comments where 'output' was misspelled as 'ouput'. 2021-04-28 10:58:45 -04:00
David Steele
066fbcf268 Refactor String, Buffer, and Variant types with inline getters.
Extend the pattern introduced in 79a2d02c to the String, Buffer, and Variant types.
2021-04-27 15:25:10 -04:00
David Steele
1edcfde93e
Add cfgOptionDisplay()/cfgOptionIdxDisplay().
Centralize the formatting of the configuration value for display to the user or passing on a command line.

For the new functions, if the value was set by the user via the command line, config, etc., then that exact value will be displayed. This makes it easier for the user to recognize the value and saves having to format it into something reasonable, especially for time and size option types.

Note that cfgOptTypeHash and cfgOptTypeList option types are not supported by these functions, but they are generally not displayed to the user as a whole.

This also fixes a bug in config/load.c where time values where not being formatted correctly in an error message.
2021-04-27 12:12:43 -04:00
David Steele
95fb002b85 Preserve unused YAML fix for older Debian/Ubuntu distributions.
This fix was not used since backports worked for older Debians and the old affected Ubuntus are EOL.

Still, seems worth preserving just in case it comes up elsewhere.
2021-04-27 12:04:50 -04:00
David Steele
6cc521b6b2
Update storage module to use StringIds.
Use StringIds for the storage types (e.g. STORAGE_S3_TYPE) and configuration settings, e.g. cfgOptS3KeyType.

Also add new config functions and harness config functions to support StringIds.
2021-04-23 13:19:47 -04:00
David Steele
aa72c19a83
Do not write files atomically or sync paths during backup copy.
There is no need to write the file atomically (e.g. via a temp file on Posix) because checksums are tested on resume after a failed backup. The path does not need be synced for each file because all paths are synced at the end of the backup.

This functionality was not lost during the migration -- it never existed in the Perl code, though these settings are used in restore. See 59f1353 where backupFile() was migrated to C.
2021-04-23 12:33:25 -04:00
David Steele
aaa15b9709
Add help for all internal options valid for default roles.
Fix the segfault when getting help for an internal option is requested by adding help for all internal options that are valid for a default command role.

Also print warnings about internal options in code rather than putting in each command/option description.
2021-04-23 11:46:03 -04:00
David Steele
2ad497ea4c Remove documentation about enum truncation and add tests.
The enum truncation observed was due to the value getting passed via a protocol function which silently narrowed the enum.

Even so, add some tests to ensure tested platforms support 64-bit enums.
2021-04-23 08:04:03 -04:00
David Steele
bcc925b740 Replace misused kvAdd() with kvPut().
Although kvAdd() works like kvPut() on the first call, kvPut() is more efficient when a key has a single value.

Update the comment to clarify that kvAdd() is seldom required.
2021-04-22 20:04:27 -04:00
David Steele
a2f02f95ec Remove redundant CompressType conversions.
These seem to be the result of overzealous copy/paste.

The conversions do not even serve as a verification of locally available compression types since compressTypeEnum() and compressTypeStr() do not check that.
2021-04-22 19:28:11 -04:00
David Steele
468aa79ea8 Remove auto-generated option String constants.
These are rarely used and end up using more space than they save since most of them are never referenced.

Replace VARSTR() with VARSTRDEF() where these constants are being used.
2021-04-22 19:10:13 -04:00
David Steele
6a39c51f8f Test pull requests in Cirrus CI.
Both FreedBSD and MacOS use clang as their default compiler and clang catches some errors that gcc does not. Specifically, wide integers being assigned to short integers resulting in possible truncations.
2021-04-22 18:18:34 -04:00
David Steele
06fa18582a
Free no longer needed remotes so they do not timeout during restore.
The remotes are no longer needed in the main process after the manifest is loaded. If the restore is long enough the connection will timeout and WARN at the end of the restore. This is harmless for the restore but distracting for the user.

To prevent this, free the remotes once they are no longer needed.
2021-04-22 12:23:08 -04:00
David Steele
45f83558ea
Fix help when a valid option is invalid for the specified command.
Getting help for a valid option that was invalid for the command would segfault.

Add a check to ensure the option is valid for the command's default role.
2021-04-22 11:48:04 -04:00
David Steele
72dac5b10b Use option constants in warnings.
In once case the constant replaces a formatted statement, which is more efficient, and in the other case plain text, which is more maintainable.
2021-04-22 08:15:36 -04:00
David Steele
20a018a8ae Fix incorrect test title. 2021-04-21 18:27:18 -04:00
David Steele
c5b446d406 Fix indentation. 2021-04-21 10:09:07 -04:00
David Steele
fd69357302 Add const to inline functions where appropriate.
This lets the compiler know that these variables are not modified which should lead to better optimization.

Smart compilers should be able to figure this out on their own, but marking parameters const is still good for documentation.
2021-04-20 18:43:16 -04:00
David Steele
ed0d48f52c Add StringId type.
It is often useful to represent identifiers as strings when they cannot easily be represented as an enum/integer, e.g. because they are distributed among a number of unrelated modules or need to be passed to remote processes. Strings are also more helpful in debugging since they can be recognized without cross-referencing the source. However, strings are awkward to work with in C since they cannot be directly used in switch statements leading to less efficient if-else structures.

A StringId encodes a short string into an integer so it can be used in switch statements but may also be readily converted back into a string for debugging purposes. StringIds may also be suitable for matching user input providing the strings are short enough.

This patch includes a sample of StringId usage by converting protocol commands to StringIds. There are many other possible use cases. To list a few:

* All "types" in storage, filters. IO , etc. These types are primarily for identification and debugging so they fit well with this model.

* MemContext names would work well as StringIds since these are entirely for debugging.

* Option values could be represented as StringIds which would mean we could remove the functions that convert strings to enums, e.g. CipherType.

* There are a number of places where enums need to be converted back to strings for logging/debugging purposes. An example is protocolParallelJobToConstZ. If ProtocolParallelJobState were defined as:

typedef enum
{
    protocolParallelJobStatePending = STRID5("pend", ...),
    protocolParallelJobStateRunning = STRID5("run", ...),
    protocolParallelJobStateDone = STRID5("done", ...),
} ProtocolParallelJobState;

then protocolParallelJobToConstZ() could be replaced with strIdToZ(). This also applies to many enums that we don't covert to strings for logging, such as CipherMode.

As an example of usage, convert all protocol commands from strings to StringIds.
2021-04-20 15:22:42 -04:00
Stefan Fercot
292f836f12
Add db-exclude option.
Restore excluding the specified databases. Databases excluded will be restored as sparse, zeroed files to save space but still allow PostgreSQL to perform recovery. After recovery, those databases will not be accessible but can be removed with the drop database command. The --db-exclude option can be passed multiple times to specify more than one database to exclude.

When used in combination with the --db-include option, --db-exclude will only apply to standard system databases (template0, template1, and postgres).
2021-04-19 15:01:00 -04:00
David Steele
a75b413ac2 Add storageDriver() inlines missed in b715c70b. 2021-04-19 14:22:36 -04:00
David Steele
d859fe8c4d Update to remove files after archive-get test completes.
This cleans up the spool path for the next test. Currently there isn't one but there will be.
2021-04-14 15:48:04 -04:00
David Steele
040ad71f8c Remove lockClear().
This function has not been used since the switch to the fork/exec model.

lockClear() was still used in one test (other than the lock test) so update the test and remove the function.
2021-04-14 15:41:55 -04:00
David Steele
c2d4a0286e Define DEBUG in build.auto.c.
Both NDEBUG and DEBUG were used in the code, which was a bit confusing.

Define DEBUG in build.auto.c so it is available in all C and header files and stop using NDEBUG. This is preferable to using NDEBUG everywhere since there are multiple DEBUG* defines, e.g. DEBUG_COVERAGE.

Note that NDEBUG is still required since it is used by the C libraries.
2021-04-13 18:06:07 -04:00
David Steele
c1aae434ca Update Cirrus CI FreeBSD release to 12.2. 2021-04-13 17:03:28 -04:00
David Steele
a0e24d492f Add THIS_PUB() macro to simplify inline object accessors.
In combination with the thisPub() function, this macro simplifies accessing the public part of a private object struct.

thisPub() asserts this != NULL so the caller does not need to do it.
2021-04-13 16:58:02 -04:00
David Steele
9fec4ce98c Refactor remaining common/io modules with inline getters/setters.
Extend the pattern introduced in 79a2d02c to the remaining common/io modules.
2021-04-13 14:37:02 -04:00
Isaacwhyuenac
5bf160643b Clarify that repo-s3-role is not an ARN. 2021-04-13 14:02:20 -04:00
David Steele
2c9c59170f Add bash to Cirrus CI FreeBSD package install.
It appears bash was removed from the base image so install explicitly.
2021-04-13 12:38:17 -04:00
David Steele
8844ced384 Refactor common/io/filter module with inline getters/setters.
Extend the pattern introduced in 79a2d02c to the common/io/filter module.
2021-04-12 16:05:40 -04:00
David Steele
3b9bed9518 Refactor common/type/keyValue module with inline getters/setters.
Extend the pattern introduced in 79a2d02c to the common/type/keyValue module.
2021-04-09 17:29:14 -04:00
David Steele
c434b9ed88 Refactor common/type/xml module with inline getters/setters.
Extend the pattern introduced in 79a2d02c to the common/type/xml module.
2021-04-09 17:03:55 -04:00
David Steele
e18f3c1701 Remove unused XML constructors.
These alternate constructors may have been used in the Perl days but they are no longer used so remove them.
2021-04-09 15:56:42 -04:00
David Steele
ddd37ebf7d Refactor common/exec module with inline getters/setters.
Extend the pattern introduced in 79a2d02c to the common/exec module.
2021-04-09 15:31:31 -04:00
David Steele
1d674c352f Refactor db module with inline getters/setters.
Extend the pattern introduced in 79a2d02c to the db module.
2021-04-09 14:32:21 -04:00
David Steele
442b2e41b1 Refactor info modules with inline getters/setters.
Extend the pattern introduced in 79a2d02c to the info modules.
2021-04-09 13:48:40 -04:00
David Steele
4937653a3d Refactor protocol modules with inline getters/setters.
Extend the pattern introduced in 79a2d02c to the protocol modules.
2021-04-08 16:46:42 -04:00
David Steele
235e32b57d Add const keyword to module scope constants.
These pointers should never be modified, so mark them const.
2021-04-08 14:47:53 -04:00
David Steele
6c005afd5b Add Ubuntu 20.04 test VM definition. 2021-04-08 14:38:20 -04:00
David Steele
d30ec9c9ae Replace OBJECT_DEFINE_MOVE() and OBJECT_DEFINE_FREE() with inlines.
Inline functions are more efficient and if they are not used are automatically omitted from the binary.

This also makes the implementation of these functions easier to find and removes the need for a declaration. That is, the complete implementation is located in the header rather than being spread between the header and C file.
2021-04-08 10:04:57 -04:00
David Steele
351e7db4c4 Replace OBJECT_DEFINE_FREE_RESOURCE_BEGIN() with normal functions.
OBJECT_DEFINE_MOVE() and OBJECT_DEFINE_FREE() will be replaced with inlines so this would be the only macro left that is constructing functions.

It is not a great pattern anyway since it makes it hard to find the function implementation.
2021-04-07 16:27:55 -04:00
David Steele
cc85c4f03d Replace OBJECT_DEFINE_GET() with *Pub struct pattern.
This macro was originally intended to simplify the creation of simple getters but it has been superseded by the pattern introduced in 79a2d02c.

Remove instances of OBJECT_DEFINE_GET() to avoid confusion with the new pattern.
2021-04-07 14:27:57 -04:00
David Steele
b715c70b46 Refactor storage modules with inline getters/setters.
Extended the pattern introduced in 79a2d02c to the storage modules: Storage, StorageRead, StorageWrite.
2021-04-07 14:04:38 -04:00
David Steele
79a2d02c9c Refactor List, StringList, and VariantList for performance.
Introduce a standard pattern for exposing public struct members (as documented in CODING.md) and use it to inline lstSize() which should improve the performance of iterating large lists.

Since many functions in these modules are just thin wrappers of other functions, inline where appropriate.

Remove strLstExistsZ() and strLstInsertZ() since they were only used in tests, where the String version of the function is sufficient.

Move strLstNewSplitSizeZ() to command/help/help.c and remove strLstNewSplitSize(). This function has only ever been used by help and does not seem widely applicable.
2021-04-07 12:50:33 -04:00