1
0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2025-07-15 01:04:37 +02:00
Commit Graph

2410 Commits

Author SHA1 Message Date
1378d9c58b Fix bad arithmetic in pgLsnToWalSegment().
/ takes precedence over & but the appropriate parens were not provided.

By some bad luck the tests worked either way, so add a new test that only works the correct way to prevent a regression.
2019-12-12 16:21:51 -05:00
676be2c773 Add pgWalPath() to return version-specific WAL path.
Also update the manifest module to use the new function.
2019-12-12 16:11:09 -05:00
94cb9540f1 Begin v2.21 development. 2019-12-12 09:05:10 -05:00
39fc2b7ad6 v2.20: Bug Fixes
Bug Fixes:

* Fix archive-push/archive-get when PGDATA is symlinked. These commands tried to use cwd() as PGDATA but this would disagree with the path configured in pgBackRest if PGDATA was symlinked. If cwd() does not match the pgBackRest path then chdir() to the path and make sure the next cwd() matches the result from the first call. (Reported by Stephen Frost, Milosz Suchy.)
* Fix reference list when backup.info is reconstructed in expire command. Since the backup command is still using the Perl version of reconstruct this issue will not express unless 1) there is a backup missing from backup.info and 2) the expire command is run directly instead of running after backup as usual. This unlikely combination of events means this is probably not a problem in the field.
* Fix segfault on unexpected EOF in gzip decompression. (Reported by Stephen Frost.)
release/2.20
2019-12-12 08:20:21 -05:00
b031dbbcf8 Allow timezones to be explicitly set for testing.
The TZ environment variable was not reliably pushed down to the test processes.

Instead pass TZ via a command line parameter and set explicitly in the test process.
2019-12-11 22:11:04 -05:00
db5175915a Use localtime() to format time_t in cvtTimeToZ().
Using gmtime() produced output skewed by the local timezone.

Since this function is currently only used for debug logging this is not a live bug in the field.
2019-12-11 21:51:32 -05:00
811ecd758d Make type in comment more specific. 2019-12-11 14:57:32 -05:00
0194a98671 Fix archive-push/archive-get when PGDATA is symlinked.
Commit 7168e074 tried to use cwd() as PGDATA but this would disagree with the path configured in pgBackRest if PGDATA was symlinked.

If cwd() does not match the pgBackRest path then chdir() to the path and make sure the next cwd() matches the result from the first call.
2019-12-11 14:36:39 -05:00
8c840c28a6 Fix segfault on unexpected EOF in gzip decompression.
If the compressed stream terminated early then the decompression process would get a flush request (NULL input buffer) since the filter was not marked as done.  This could happen on a zero-length or truncated (i.e. invalid) compressed file.

Change the existing assertion to an error to catch this condition in production gracefully.
2019-12-11 08:48:46 -05:00
c933f12f9c Remove obsolete --perl-option option.
This option was used when Perl was executed instead of being embedded.

It has been obsolete for a long time so remove it.
2019-12-10 13:28:15 -05:00
d0ba8ff58c Remove test point infrastructure.
82df7e6f and 9856fef5 updated tests that used test points in preparation for the feature not being available in the C code.

Since tests points are no longer used remove the infrastructure.

Also remove one stray --test option in mock/all that was essentially a noop but no longer works now that the option has been removed.
2019-12-10 13:16:47 -05:00
d7d663c2b9 Make buildPutDiffers() work with empty files.
If the file was empty the timestamp was updated.  If the file is empty and there is no content then file should not be saved.
2019-12-10 13:02:36 -05:00
800d2972b0 Remove stray uint type.
This was probably copied from an example but some compilers don't like it.
2019-12-09 18:28:20 -05:00
471d54a738 Add stringz module to define some commonly used strings.
This module will eventually contain various useful zero-terminated string functions.

For now, using NULL_Z instead of strPtr(NULL_STR) avoids a strict aliasing warning on RHEL 6.  This is likely a compiler issue, but adding these constants seems like a good idea anyway and we are not going to get a fix in a gcc that old.
2019-12-09 17:55:20 -05:00
ca33545630 Remove redundant test and move another test. 2019-12-09 14:06:32 -05:00
61ad3a44c9 Note that next repo format should use UTC for backup labels. 2019-12-09 09:12:34 -05:00
d3132dae26 Add functions for building new manifests.
New manifests are built before a backup is performed.

Reviewed by Cynthia Shang.
2019-12-08 18:43:47 -05:00
2cfde18755 Add pgLsnFromStr(), pgLsnToStr(), and pgLsnToWalSegment(). 2019-12-08 14:19:47 -05:00
f517b141fb Update pq harness to play nicely with variable LSNs. 2019-12-08 14:15:23 -05:00
d2587250da Add backup functions to Db object.
These functions implement the database backup functionality for all supported versions.
2019-12-07 18:44:06 -05:00
8766326da8 Add protocolRemoteFree() to shutdown a specific remote.
Sometimes it is useful to shutdown remotes that are no longer needed instead of waiting for them to be shutdown at program exit.
2019-12-07 17:48:53 -05:00
5175d52e94 Add constant for pg_tblspc path. 2019-12-07 17:42:42 -05:00
78b1e05b7e Remove unused Perl module. 2019-12-07 17:39:25 -05:00
35a262951a Pq test harness usability and error reporting improvements.
Pq script errors are now printed in test output in case they are being masked by a later error.

Once a script error occurs, the same error will be thrown forever rather than throwing a new error on the next item in the script.

HRNPQ_MACRO_CLOSE() is not required in scripts unless harnessPqScriptStrictSet(true) is called.  Most higher-level tests should not need to run in strict mode.

The command/check test seems to require strict mode but there's no apparent reason why it should.  This would be a good thing to look into at some point.
2019-12-07 17:33:34 -05:00
d6479ddd0e Add log replacements to help test non-deterministic log output.
Some log output (e.g. time) is hard to test because the values can change between tests.

Add expressions to replace substrings in the log with predictable values to simplify testing.

This is similar to the log replacement facility available for Perl expect log testing.
2019-12-07 17:15:20 -05:00
e4716ee036 Improve diff output in tests.
Always compare expected vs actual (in that order) and give a hint in the error message to indicate what should be added and what removed.
2019-12-07 17:02:41 -05:00
8c47ee296a Improve storage harness test callback.
Add ability to omit the root (i.e. dot) path and get real size of compressed files.
2019-12-07 16:55:50 -05:00
1b3770e248 Recopy during backup when resumed file is missing or corrupt.
A recopy would occur if the size or checksum was invalid but on error the backup would terminate.

Instead, recopy the resumed file on any error.  If the error is systemic (e.g. network failure) then it should show up again during the recopy.
2019-12-07 09:48:33 -05:00
d3f717c892 Storage hardlink and symlink features require path feature.
Since there is only one driver that supports (or is likely to support) links (Posix), require the path feature to make logic in the code simpler.

The checks are added just in case another driver supports links.
2019-12-07 09:26:51 -05:00
e632c60525 Fix backup labels in mock/all resume integration tests.
These were not getting updated to match the directory name when the manifests were copied.

The Perl code didn't care but the C code expects labels to be set correctly.
2019-12-06 11:48:41 -05:00
5395aa0c92 Allocate one extra process for backup from standby.
The connection to the primary is process-max + 1 so make space for it.
2019-12-05 22:34:38 -05:00
b2d82bd248 Add functions to get the substring found by regExpMatch().
For now this is only used in testing but there are places where it could be useful in the core code.

Even if that turns out not to be true, it doesn't seem worth implementing a new version in testing just to capture a few values that we already have.
2019-12-04 19:43:26 -05:00
053af2f54d Add asserts to ensure hostId > 0.
hostId is 1-based (e.g. pg1-*) so it should always be > 0.
2019-12-04 19:31:39 -05:00
d1f49825b1 Add missing header. 2019-12-02 11:54:45 -05:00
8dfe0e48e2 Use more general error code when tablespace linked into PGDATA.
The specific error code was not that useful since we also test the error message which contains details of the link error.
2019-12-02 10:49:25 -05:00
33a63aae50 Add flag to dbGet() to require a standby.
This is needed from backup from standby functionality.
2019-12-02 07:39:42 -05:00
28116918ff Error in remote command when stop file exists.
This duplicates the Perl functionality.
2019-12-02 07:35:36 -05:00
fc291b6f28 Reduce the scope of mock/all exclusion tests.
Run exclusions only on the tests where they will have an effect to reduce churn in the expect logs when they change.
2019-12-01 17:47:47 -05:00
d15ed33821 Make MCV return false when a boolean tie.
This is to maintain compatibility with the older Perl code that returned the lowest sorted order item in a tie.

For other datatypes the C code returns the same value, often enough at least to not cause churn in the expect tests.
2019-12-01 16:32:21 -05:00
ab0974cc8e Add PostgreSQL version where backup from standby is supported. 2019-12-01 16:30:44 -05:00
218698088f Add PostgreSQL file and path names used for building manifests. 2019-12-01 16:28:33 -05:00
56ee321a95 Add pgLsnName() and pgXactPath(). 2019-12-01 15:49:34 -05:00
bc83354fb5 Add storage feature flags.
Even though storagePathSync() is a noop when path sync is not supported, it is useful to know in advance if the function will do anything so add the storageFeaturePathSync flag.

The storageFeatureSymLink and storageFeatureHardLink flags are currently informational only since links are not yet implemented in the storage interface.
2019-11-29 11:29:16 -05:00
0933aeb1ad Remove unused reference to use pgBackRest::Backup::File. 2019-11-29 10:56:13 -05:00
50eb062e0e Fix reference list when backup.info is reconstructed in expire command.
Adding a manifest to backup.info was migrated to C in 4e4d1f41 but deduplication of the references was missed leading to a reference for every file being added to backup.info.

Since the backup command is still using the Perl version of reconstruct this issue will not express unless 1) there is a backup missing from backup.info and 2) the expire command is run directly instead of running after backup as usual.

This unlikely combination of events means this is probably not a problem in the field.
2019-11-28 09:34:19 -05:00
686b6f91da Set archive-check option in manifest correctly when offline.
Archive check does not run when in offline backup mode but the option was set to true in the manifest.  It's harmless since these options are informational only but it could cause confusion when debugging.
2019-11-28 08:27:21 -05:00
5506e5de27 Remove obsolete call to perlExec() for stand-alone expire command.
This was missed in ecae5e34.
2019-11-27 15:09:07 -05:00
4683557c54 Remove unused reference to pgBackRest::Db. 2019-11-26 17:22:30 -05:00
158e439689 Remove obsolete Perl archive code.
This should have been removed in a1c13a50 but was missed.
2019-11-26 17:16:45 -05:00
82df7e6f3b Update integration tests in real/all that use test points.
Test points are not supported by the new C code so these will be replaced with unit tests.

The fact that the tests still pass even when the changes aren't made mid-backup (except application_name) shows how weak they were in the first place.

Even so, this does represent a regression in (soon to be be removed) Perl coverage.
2019-11-26 11:32:12 -05:00