1
0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2025-07-05 00:28:52 +02:00
Commit Graph

23 Commits

Author SHA1 Message Date
0e76ccb5b7 Convert filter param/result to Pack type.
The Pack type is more compact and flexible than the Variant type. The Pack type also allows binary data to be stored, which is useful for transferring the passphrase in the CipherBlock filter.

The primary purpose is to allow more (and more complex) result data to be returned efficiently from the PageChecksum filter. For now the PageChecksum filter still returns the original Variant. Converting the result data will be the subject of a future commit.

Also convert filter types to StringId.
2021-09-22 10:48:21 -04:00
4ad0bbda53 Update command/backup tests to use standard patterns.
Includes backup and backupCommon tests.

Some tests in backupTest were split out where they were originally combined into a single boolean check - which made it difficult to determine which part of the conditional failed.

String values were also removed where they were no longer needed.
2021-07-15 17:00:20 -04:00
aed3d468a1 Rename strNew() to strNewZ() and add parameter-less strNew().
Replace all instances of strNew("") with strNew() and use strNewZ() for non-empty zero-terminated strings. Besides saving a useless parameter, this will allow smarter memory allocation in a future commit by signaling intent, in general, to append or not.

In the tests use STRDEF() or VARSTRDEF() where more appropriate rather than blindly replacing with strNewZ(). Also replace strLstAdd() with strLstAddZ() where appropriate for the same reason.
2021-05-21 17:36:43 -04:00
87df6d7a58 Convert BackupType enum to StringId.
Allows removal of backupType()/backupTypeStr() and improves debug logging of the enum.

Move BackupType enum and string constants to info/infoBackup.h so they are available to more modules. Also convert InfoBackup to use BackupType instead of a String.
2021-05-03 12:15:39 -04:00
28301199eb Rename FUNCTION_HARNESS_RESULT*() macros to FUNCTION_HARNESS_RETURN*().
When the FUNCTION_*_RESULT*() macros were renamed to FUNCTION_*_RETURN_*() in the core code the test harness macros were missed.

Update them to make the naming consistent.
2021-03-10 18:42:22 -05:00
aafd2f528d Cast constants explicitly in command/backupCommon test.
Clang was not clear that these constants should be uint64_t.

Found on MacOS M1.
2021-01-24 15:29:10 -05:00
688ec2a8f5 Use an extension to denote vendorized code.
Vendorized code is copied from another project when a library is not available and a git subproject won't work. Currently all the vendorized code is copied from PostgreSQL but it makes sense to have a more general mechanism for indicating vendorized code.

The .vendor extension will be used to denote vendorized code in the same way that .auto is used to denote auto-generated code.
2020-05-18 19:11:26 -04:00
e55443c890 Move logic from postgres/pageChecksum to command/backup/pageChecksum().
The postgres/pageChecksum module was designed as an interface to the C structs for the Perl code.  The new C code can do this directly so no need for an interface.

Move the remaining test for pgPageChecksum() into the postgres/interface test module.
2020-03-05 16:12:54 -05:00
4ab8943ca8 Use PG_PAGE_SIZE_DEFAULT constant instead of pageSize variable.
Page size is passed around a lot but in fact it can only have one value, PG_PAGE_SIZE_DEFAULT, which is checked when pg_control is loaded. There may be an argument for supporting multiple page sizes in the future but for now just use the constant to simplify the code.

There is also a significant performance benefit.  Because pageSize was being used in pageChecksumBlock() the main loop was neither unrolled nor vectorized (-funroll-loops -ftree-vectorize) as it is now with a constant loop boundary.
2020-03-05 09:14:27 -05:00
9d48882268 Centralize PostgreSQL page header data structures.
These data structures were copied a few places (but only once in the core code) so put them in a place where everyone can use them.

To do this create a new file, static.auto.h, to contain data types and macros that have stayed the same through all the versions of PostgreSQL that we support.  This allows us to have single, non-versioned set of headers and code for stable data structures like page headers.

Migrate a few types from version.auto.h that are required for page header structures and pull the remaining types from PostgreSQL directly.

We had previously renamed xlog to wal so update those where required since we won't be modifying the PostgreSQL names anymore.
2020-03-04 13:31:27 -05:00
b134175fc7 Use designated initializers to initialize structs.
Previously memNew() used memset() to initialize all struct members to 0, NULL, false, etc.  While this appears to work in practice, it is a violation of the C specification.  For instance, NULL == 0 must be true but neither NULL nor 0 must be represented with all zero bits.

Instead use designated initializers to initialize structs.  These guarantee that struct members will be properly initialized even if they are not specified in the initializer.  Note that due to a quirk in the C99 specification at least one member must be explicitly initialized even if it needs to be the default value.

Since pre-zeroed memory is no longer required, adjust memAllocInternal()/memReallocInternal() to return raw memory and update dependent functions accordingly.  All instances of memset() have been removed except in debug/test code where needed.

Add memMewPtrArray() to allocate an array of pointers and automatically set all pointers to NULL.

Rename memGrowRaw() to the more logical memResize().
2020-01-23 14:15:58 -07:00
d41eea685a Change meaning of TEST_RESULT_STR() macro.
This macro was created before the String object existed so subsequent usage with String always included a lot of strPtr() wrapping.

TEST_RESULT_STR_Z() had already been introduced but a wholesale replacement of TEST_RESULT_STR() was not done since the priority was on the C migration.

Update all calls to (old) TEST_RESULT_STR() with one of the following variants: (new) TEST_RESULT_STR(), TEST_RESULT_STR_Z(), TEST_RESULT_Z(), TEST_RESULT_Z_STR().
2019-12-26 18:08:27 -07:00
74c3842595 Remove errant tabs and fix spacing. 2019-12-19 16:25:46 -05:00
e206093beb Allow end anchor to be excluded in backupRegExp().
This is useful for matching files in the backup history directory which have characters after the backup label.
2019-12-12 18:52:16 -05:00
db1dc4f275 Remove pretty-printing from jsonFromKv() and jsonFromVar().
Now that pretty-printing has been removed from the info command it no longer has a purpose, so remove it.
2019-10-11 13:03:52 -04:00
45881c74ae Allow most unit tests to run outside of a container.
Three major changes were required to get this working:

1) Provide the path to pgbackrest in the build directory when running outside a container.  Tests in a container will continue to install and run against /usr/bin/pgbackrest.

1) Set a per-test lock path so tests don't conflict on the default /tmp/pgbackrest path.  Also set a per-test log-path while we are at it.

2) Use localhost instead of a custom host for TLS test connections.  Tests in containers will continue to update /etc/hosts and use the custom host.

Add infrastructure and update harnessCfgLoad*() to get the correct exe and paths loaded for testing.

Since new tests are required to verify that running outside a container works, also rework the tests in Travis CI to provide coverage within a reasonable amount of time.  Mainly, break up to doc tests by VM and run an abbreviated unit test suite on co6 and co7.
2019-10-08 12:06:30 -04:00
8aa1e552b0 Add backup type conversion functions.
Convert back and forth between the string and enum representations of backup types.
2019-08-18 20:09:44 -04:00
3e1062825d Allow multiple filters to be pushed to the remote and return results.
Previously only a single filter could be pushed to the remote since order was not being maintained.  Now the filters are strictly ordered.

Results are returned from the remote and set in the local IoFilterGroup so they can be retrieved.

Expand remote filter support to include all filters.
2019-07-15 16:49:46 -04:00
039e515a31 Allow protocol compression when read/writing remote files.
If the file is compressible (i.e. not encrypted or already compressed) it can be marked as such in storageNewRead()/storageNewWrite().  If the file is being read from/written to a remote it will be compressed in transit using gzip.

Simplify filter group handling by having the IoRead/IoWrite objects create the filter group automatically.  This removes the need for a lot of NULL checking and has a negligible effect on performance since a filter group needs to be created eventually unless the source file is missing.

Allow filters to be created using a VariantList so filter parameters can be passed to the remote.
2019-06-24 10:20:47 -04:00
ceafd8e19d Migrate page checksum filter to C.
This filter exactly mimics the behavior of the Perl filter so is a drop-in replacement.

The filter is not integrated yet since it requires the Perl-to-C storage layer interface coming in a future commit.
2019-06-17 07:52:03 -04:00
2b75f57232 Add linefeeds for clarity. 2019-05-16 09:56:48 -04:00
32ca27a20b Simplify storage object names.
Remove "File" and "Driver" from object names so they are shorter and easier to keep consistent.

Also remove the "driver" directory so storage implementations are visible directly under "storage".
2019-05-03 15:46:15 -04:00
a7281878ac Migrate backupRegExp() to C.
Removed the "anchor" parameter because it was never used in any calls in the Perl code so it was just a dead parameter that always defaulted to true.

Contributed by Cynthia Shang.
2019-04-15 08:29:25 -04:00