db/pgbackrest
1
0
Fork 0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2025-07-05 00:28:52 +02:00
Code Issues Releases Activity
Files
a05bf6bb15ba7f7e4b610dbb84e87ebcd359dd38
pgbackrest/test/src/module/postgres/interfaceTest.c

309 lines
17 KiB
C
Raw Normal View History

Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 10:24:42 +01:00
/***********************************************************************************************************************************
Test PostgreSQL Interface
***********************************************************************************************************************************/
Simplify storage object names. Remove "File" and "Driver" from object names so they are shorter and easier to keep consistent. Also remove the "driver" directory so storage implementations are visible directly under "storage".
2019-05-03 15:46:15 -04:00
#include "storage/posix/storage.h"
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 10:24:42 +01:00
Move PostgreSQL version interface test functions to a test harness. Some version interface test functions were integrated into the core code because they relied on the PostgreSQL versioned interface. Even though they were compiled out for production builds they cluttered the core code and made it harder to determine what was required by core. Create a PostgreSQL version interface in a test harness to contain these functions. This does require some duplication but the cleaner core code seems a good tradeoff. It is possible for some of this code to be auto-generated but since it is only updated once per year the matter is not pressing.
2021-05-17 07:20:28 -04:00
#include "common/harnessPostgres.h"
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 10:24:42 +01:00
/***********************************************************************************************************************************
Test Run
***********************************************************************************************************************************/
Add missing static keywords in test modules.
2021-10-18 12:22:48 -04:00
static void
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 10:24:42 +01:00
testRun(void)
{
FUNCTION_HARNESS_VOID();
Add defines for many test*() getter functions. A define was already added for TEST_PATH but it was not widely used. Replace all occurrences of testPath() with TEST_PATH in the tests. Replace testUser() with TEST_USER, testGroup() with TEST_GROUP, testRepoPath() with HRN_PATH_REPO, testDataPath() with HRN_PATH, testProjectExe() with TEST_PROJECT_EXE, and testScale() with TEST_SCALE. Replace {[path]}, {[user]}, {[group]}, etc. with defines and remove hrnReplaceKey(). This is better than having two ways to deal with replacements. In some cases the original test*() getters were kept because they are used by the harness, which does not have access to the new defines. Move them to harnessTest.intern.h to indicate that the tests should no longer use them.
2021-05-22 09:30:54 -04:00
Storage *storageTest = storagePosixNewP(TEST_PATH_STR, .write = true);
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 10:24:42 +01:00
// *****************************************************************************************************************************
if (testBegin("pgVersionFromStr() and pgVersionToStr()"))
{
Rename strNew() to strNewZ() and add parameter-less strNew(). Replace all instances of strNew("") with strNew() and use strNewZ() for non-empty zero-terminated strings. Besides saving a useless parameter, this will allow smarter memory allocation in a future commit by signaling intent, in general, to append or not. In the tests use STRDEF() or VARSTRDEF() where more appropriate rather than blindly replacing with strNewZ(). Also replace strLstAdd() with strLstAddZ() where appropriate for the same reason.
2021-05-21 17:36:43 -04:00
TEST_ERROR(pgVersionFromStr(STRDEF("9.3.4")), AssertError, "version 9.3.4 format is invalid");
TEST_ERROR(pgVersionFromStr(STRDEF("abc")), AssertError, "version abc format is invalid");
Rename FUNCTION_DEBUG_* and consolidate ASSERT_* macros for consistency. Rename FUNCTION_DEBUG_* macros to FUNCTION_LOG_* to more accurately reflect what they do. Further rename FUNCTION_DEBUG_RESULT* macros to FUNCTION_LOG_RETURN* to make it clearer that they return from the function as well as logging. Leave FUNCTION_TEST_* macros as they are. Consolidate the various ASSERT* macros into a single ASSERT macro that is always compiled out of production builds. It was difficult to figure out when an assert would be checked with all the different types in play. When ASSERTs are compiled in they will always be checked regardless of the log level -- tying these two concepts together was not a good idea.
2019-01-21 17:41:59 +02:00
TEST_ERROR(pgVersionFromStr(NULL), AssertError, "assertion 'version != NULL' failed");
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 10:24:42 +01:00
Rename strNew() to strNewZ() and add parameter-less strNew(). Replace all instances of strNew("") with strNew() and use strNewZ() for non-empty zero-terminated strings. Besides saving a useless parameter, this will allow smarter memory allocation in a future commit by signaling intent, in general, to append or not. In the tests use STRDEF() or VARSTRDEF() where more appropriate rather than blindly replacing with strNewZ(). Also replace strLstAdd() with strLstAddZ() where appropriate for the same reason.
2021-05-21 17:36:43 -04:00
TEST_RESULT_INT(pgVersionFromStr(STRDEF("10")), PG_VERSION_10, "valid pg version 10");
TEST_RESULT_INT(pgVersionFromStr(STRDEF("9.6")), 90600, "valid pg version 9.6");
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 10:24:42 +01:00
Fix formatting errors. Errors in our current (manually-maintained) code format discovered by uncrustify.
2023-01-30 11:16:31 +07:00
// -------------------------------------------------------------------------------------------------------------------------
Change meaning of TEST_RESULT_STR() macro. This macro was created before the String object existed so subsequent usage with String always included a lot of strPtr() wrapping. TEST_RESULT_STR_Z() had already been introduced but a wholesale replacement of TEST_RESULT_STR() was not done since the priority was on the C migration. Update all calls to (old) TEST_RESULT_STR() with one of the following variants: (new) TEST_RESULT_STR(), TEST_RESULT_STR_Z(), TEST_RESULT_Z(), TEST_RESULT_Z_STR().
2019-12-26 18:08:27 -07:00
TEST_RESULT_STR_Z(pgVersionToStr(PG_VERSION_11), "11", "infoPgVersionToString 11");
TEST_RESULT_STR_Z(pgVersionToStr(PG_VERSION_96), "9.6", "infoPgVersionToString 9.6");
Remove support for PostgreSQL 8.3/8.4. There is no evidence that users need 8.3/8.4 anymore but it does cost us in terms of development and testing, especially now that we have a number of new backup/restore features planned. It seems to make sense to remove this support now. If there are users who need to use/migrate from these versions they can use an older version of pgBackRest.
2022-01-06 15:34:04 -05:00
TEST_RESULT_STR_Z(pgVersionToStr(93456), "9.34", "infoPgVersionToString 93456");
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 10:24:42 +01:00
}
Remove dependency on pg_database.datlastsysoid. This column has been removed in PostgreSQL 15. Rather than add a lot of special handling, it seems better just to update all versions to not depend on this column. Add centralized functions to identify the type of database (i.e. system or user) by name and use FirstNormalObjectId when a name is not available. The new query in the db module will still return the prior result for PostgreSQL <= 15, which will be stored in the manifest. This is important to preserve behavior when downgrading pgBackRest. There are no concerns here for PostgreSQL 15 since older versions of pgBackRest won't be able to restore backups for PostgreSQL 15 anyway.
2022-05-04 08:22:45 -04:00
// *****************************************************************************************************************************
if (testBegin("pgDbIs*()"))
{
TEST_RESULT_BOOL(pgDbIsTemplate(STRDEF("template0")), true, "template0 is template");
TEST_RESULT_BOOL(pgDbIsTemplate(STRDEF("template1")), true, "template1 is template");
TEST_RESULT_BOOL(pgDbIsTemplate(STRDEF("postgres")), false, "postgres is not template");
TEST_RESULT_BOOL(pgDbIsSystem(STRDEF("postgres")), true, "postgres is system");
TEST_RESULT_BOOL(pgDbIsSystem(STRDEF("template0")), true, "template0 is system");
TEST_RESULT_BOOL(pgDbIsSystem(STRDEF("app")), false, "app is not system");
TEST_RESULT_BOOL(pgDbIsSystemId(16383), true, "16383 is system");
TEST_RESULT_BOOL(pgDbIsSystemId(16384), false, "16384 is not system");
}
Remove most references to PostgreSQL control and catalog versions. The control and catalog versions were stored a variety of places in the optimistic hope that they would be useful. In fact they never were. We can't remove them from the backup.info and backup.manifest files due to backwards compatibility concerns, but we can at least avoid loading and storing them in C structures. Add functions to the PostgreSQL interface which will return the control and catalog versions for any supported version of PostgreSQL to allow backwards compatibility for backup.info and backup.manifest. These functions will be useful in other ways, e.g. generating the tablespace identifier in PostgreSQL >= 9.0.
2019-09-07 18:04:39 -04:00
// *****************************************************************************************************************************
Improve PostgreSQL version identification. Previously, catalog versions were fixed for all versions which made maintaining the catalog versions during PostgreSQL beta and release candidate cycles very painful. A version of pgBackRest which was functionally compatible was rendered useless by a catalog version bump in PostgreSQL. Instead use only the control version to identify a PostgreSQL version when possible. Some older versions require a catalog version to positively identify a PostgreSQL version, so include them when required. Since the catalog number is required to work with tablespaces it will need to be stored. There's already a copy of it in backup.info so use that (even though we have been ignoring it in the C versions).
2020-09-18 16:55:26 -04:00
if (testBegin("pgControlVersion()"))
Remove most references to PostgreSQL control and catalog versions. The control and catalog versions were stored a variety of places in the optimistic hope that they would be useful. In fact they never were. We can't remove them from the backup.info and backup.manifest files due to backwards compatibility concerns, but we can at least avoid loading and storing them in C structures. Add functions to the PostgreSQL interface which will return the control and catalog versions for any supported version of PostgreSQL to allow backwards compatibility for backup.info and backup.manifest. These functions will be useful in other ways, e.g. generating the tablespace identifier in PostgreSQL >= 9.0.
2019-09-07 18:04:39 -04:00
{
Add pg-version-force option for fork integration. Forks may update pg_control version or WAL magic without affecting the structures that pgBackRest depends on. This option forces pgBackRest to treat a cluster as the specified version when it cannot be automatically identified.
2023-03-09 02:23:15 +01:00
TEST_ERROR(
pgControlVersion(70300), VersionNotSupportedError,
"invalid PostgreSQL version 70300\n"
"HINT: is this version of PostgreSQL supported?");
Remove support for PostgreSQL 9.0/9.1/9.2. Our new policy is to support ten versions of PostgreSQL, the five supported releases and the last five EOL releases. As of PostgreSQL 15, that means 9.0/9.1/9.2 are no longer supported by pgBackRest. Remove all logic associated with 9.0/9.1/9.2 and update the tests. Document the new support policy. Update InfoPg to read/write control versions for the history in backup.info, since we can no longer rely on the mappings being available. In theory this could have been an issue after removing 8.3/8.4 if anybody was using a version that old.
2022-12-20 12:20:47 +07:00
TEST_RESULT_UINT(pgControlVersion(PG_VERSION_93), 937, "9.3 control version");
Remove most references to PostgreSQL control and catalog versions. The control and catalog versions were stored a variety of places in the optimistic hope that they would be useful. In fact they never were. We can't remove them from the backup.info and backup.manifest files due to backwards compatibility concerns, but we can at least avoid loading and storing them in C structures. Add functions to the PostgreSQL interface which will return the control and catalog versions for any supported version of PostgreSQL to allow backwards compatibility for backup.info and backup.manifest. These functions will be useful in other ways, e.g. generating the tablespace identifier in PostgreSQL >= 9.0.
2019-09-07 18:04:39 -04:00
TEST_RESULT_UINT(pgControlVersion(PG_VERSION_11), 1100, "11 control version");
}
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 10:24:42 +01:00
// *****************************************************************************************************************************
if (testBegin("pgControlFromBuffer() and pgControlFromFile()"))
{
Refactor PostgreSQL interface to remove most code duplication. Having a copy per version worked well until it was time to add new features or modify existing functions. Then it was necessary to modify every version and try to keep them all in sync. Consolidate all the PostgreSQL types into a single file using #if for type versions. Many types do not change or change infrequently so this cuts down on duplication. In addition, it is far easier to see what has changed when a new version is added. Use macros to write the interface functions. There is still duplication here since some changes require a new copy of the macro, but it is far less than before.
2019-03-21 21:11:36 +04:00
// Sanity test to ensure PG_VERSION_MAX has been updated
Fix formatting errors. Errors in our current (manually-maintained) code format discovered by uncrustify.
2023-01-30 11:16:31 +07:00
// -------------------------------------------------------------------------------------------------------------------------
Refactor PostgreSQL interface to remove most code duplication. Having a copy per version worked well until it was time to add new features or modify existing functions. Then it was necessary to modify every version and try to keep them all in sync. Consolidate all the PostgreSQL types into a single file using #if for type versions. Many types do not change or change infrequently so this cuts down on duplication. In addition, it is far easier to see what has changed when a new version is added. Use macros to write the interface functions. There is still duplication here since some changes require a new copy of the macro, but it is far less than before.
2019-03-21 21:11:36 +04:00
TEST_RESULT_UINT(pgInterface[0].version, PG_VERSION_MAX, "check max version");
Fix formatting errors. Errors in our current (manually-maintained) code format discovered by uncrustify.
2023-01-30 11:16:31 +07:00
// -------------------------------------------------------------------------------------------------------------------------
Allow control version and WAL magic to be overridden in test harness. This makes it easier to write tests for invalid control version/WAL magic. Also add HRN_PG_WAL_TO_BUFFER() to simplify generation of WAL headers.
2023-03-04 12:50:02 +07:00
TEST_TITLE("unknown control version");
Use designated initializers to initialize structs. Previously memNew() used memset() to initialize all struct members to 0, NULL, false, etc. While this appears to work in practice, it is a violation of the C specification. For instance, NULL == 0 must be true but neither NULL nor 0 must be represented with all zero bits. Instead use designated initializers to initialize structs. These guarantee that struct members will be properly initialized even if they are not specified in the initializer. Note that due to a quirk in the C99 specification at least one member must be explicitly initialized even if it needs to be the default value. Since pre-zeroed memory is no longer required, adjust memAllocInternal()/memReallocInternal() to return raw memory and update dependent functions accordingly. All instances of memset() have been removed except in debug/test code where needed. Add memMewPtrArray() to allocate an array of pointers and automatically set all pointers to NULL. Rename memGrowRaw() to the more logical memResize().
2020-01-23 14:15:58 -07:00
Add pg-version-force option for fork integration. Forks may update pg_control version or WAL magic without affecting the structures that pgBackRest depends on. This option forces pgBackRest to treat a cluster as the specified version when it cannot be automatically identified.
2023-03-09 02:23:15 +01:00
HRN_PG_CONTROL_OVERRIDE_PUT(storageTest, PG_VERSION_15, 1501, .catalogVersion = 202211111);
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 10:24:42 +01:00
TEST_ERROR(
Add pg-version-force option for fork integration. Forks may update pg_control version or WAL magic without affecting the structures that pgBackRest depends on. This option forces pgBackRest to treat a cluster as the specified version when it cannot be automatically identified.
2023-03-09 02:23:15 +01:00
pgControlFromFile(storageTest, NULL), VersionNotSupportedError,
"unexpected control version = 1501 and catalog version = 202211111\n"
Allow control version and WAL magic to be overridden in test harness. This makes it easier to write tests for invalid control version/WAL magic. Also add HRN_PG_WAL_TO_BUFFER() to simplify generation of WAL headers.
2023-03-04 12:50:02 +07:00
"HINT: is this version of PostgreSQL supported?");
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 10:24:42 +01:00
Fix formatting errors. Errors in our current (manually-maintained) code format discovered by uncrustify.
2023-01-30 11:16:31 +07:00
// -------------------------------------------------------------------------------------------------------------------------
Add timeline and checkpoint checks to backup. Add the following checks: * Checkpoint is updated in pg_control after pg_start_backup(). This helps ensure that PostgreSQL and pgBackRest have a consistent view of the storage and that PGDATA paths match. * Timeline of backup start WAL file matches pg_control. Hard to see how this one could get hit, but we have the power... * Standby is on the same timeline as the primary. If not, this standby is not following the primary. * Last standby checkpoint is not greater than the backup checkpoint. If so, this standby is not following the primary. This also requires some additional plumbing to read/write timeline/checkpoint from pg_control and parse timelines from WAL filenames. There were some changes in the backup tests caused by the fact that pg_control now has different contents for each backup. The check to ensure that the required checkpoint was reached on the standby should also be updated to use pg_control (it currently uses pg_control_checkpoint()), but that requires non-trivial changes to the test harness and will need to wait.
2021-12-07 09:21:07 -05:00
HRN_PG_CONTROL_PUT(
storageTest, PG_VERSION_11, .systemId = 0xFACEFACE, .checkpoint = 0xEEFFEEFFAABBAABB, .timeline = 47,
.walSegmentSize = 1024 * 1024);
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 10:24:42 +01:00
PgControl info = {0};
Add pg-version-force option for fork integration. Forks may update pg_control version or WAL magic without affecting the structures that pgBackRest depends on. This option forces pgBackRest to treat a cluster as the specified version when it cannot be automatically identified.
2023-03-09 02:23:15 +01:00
TEST_ASSIGN(info, pgControlFromFile(storageTest, NULL), "get control info v11");
Fix incorrect result types in unit tests. Upcoming changes to the TEST_RESULT_* macros are more type safe and identified that the wrong macros were being used to test results in many cases. Commit these changes separately to verify that they work with the current macro versions. Note that no core bugs were exposed by these changes.
2020-03-22 20:25:31 -04:00
TEST_RESULT_UINT(info.systemId, 0xFACEFACE, " check system id");
TEST_RESULT_UINT(info.version, PG_VERSION_11, " check version");
Improve PostgreSQL version identification. Previously, catalog versions were fixed for all versions which made maintaining the catalog versions during PostgreSQL beta and release candidate cycles very painful. A version of pgBackRest which was functionally compatible was rendered useless by a catalog version bump in PostgreSQL. Instead use only the control version to identify a PostgreSQL version when possible. Some older versions require a catalog version to positively identify a PostgreSQL version, so include them when required. Since the catalog number is required to work with tablespaces it will need to be stored. There's already a copy of it in backup.info so use that (even though we have been ignoring it in the C versions).
2020-09-18 16:55:26 -04:00
TEST_RESULT_UINT(info.catalogVersion, 201809051, " check catalog version");
Add timeline and checkpoint checks to backup. Add the following checks: * Checkpoint is updated in pg_control after pg_start_backup(). This helps ensure that PostgreSQL and pgBackRest have a consistent view of the storage and that PGDATA paths match. * Timeline of backup start WAL file matches pg_control. Hard to see how this one could get hit, but we have the power... * Standby is on the same timeline as the primary. If not, this standby is not following the primary. * Last standby checkpoint is not greater than the backup checkpoint. If so, this standby is not following the primary. This also requires some additional plumbing to read/write timeline/checkpoint from pg_control and parse timelines from WAL filenames. There were some changes in the backup tests caused by the fact that pg_control now has different contents for each backup. The check to ensure that the required checkpoint was reached on the standby should also be updated to use pg_control (it currently uses pg_control_checkpoint()), but that requires non-trivial changes to the test harness and will need to wait.
2021-12-07 09:21:07 -05:00
TEST_RESULT_UINT(info.checkpoint, 0xEEFFEEFFAABBAABB, "check checkpoint");
TEST_RESULT_UINT(info.timeline, 47, "check timeline");
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 10:24:42 +01:00
Fix formatting errors. Errors in our current (manually-maintained) code format discovered by uncrustify.
2023-01-30 11:16:31 +07:00
// -------------------------------------------------------------------------------------------------------------------------
Add HRN_PG_CONTROL_PUT() and HRN_PG_CONTROL_TIME(). These macros simplify management of pg_control test files. Centralize time updates for pg_control in the command/backup module. This caused some time updates in the logs. Finally, move the postgres module after the storage module so it can use storage macros.
2021-11-30 13:23:11 -05:00
HRN_PG_CONTROL_PUT(storageTest, PG_VERSION_93, .walSegmentSize = 1024 * 1024);
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 10:24:42 +01:00
TEST_ERROR(
Add pg-version-force option for fork integration. Forks may update pg_control version or WAL magic without affecting the structures that pgBackRest depends on. This option forces pgBackRest to treat a cluster as the specified version when it cannot be automatically identified.
2023-03-09 02:23:15 +01:00
pgControlFromFile(storageTest, NULL), FormatError,
"wal segment size is 1048576 but must be 16777216 for PostgreSQL <= 10");
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 10:24:42 +01:00
Fix formatting errors. Errors in our current (manually-maintained) code format discovered by uncrustify.
2023-01-30 11:16:31 +07:00
// -------------------------------------------------------------------------------------------------------------------------
Add HRN_PG_CONTROL_PUT() and HRN_PG_CONTROL_TIME(). These macros simplify management of pg_control test files. Centralize time updates for pg_control in the command/backup module. This caused some time updates in the logs. Finally, move the postgres module after the storage module so it can use storage macros.
2021-11-30 13:23:11 -05:00
HRN_PG_CONTROL_PUT(storageTest, PG_VERSION_95, .pageSize = 32 * 1024);
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 10:24:42 +01:00
Add pg-version-force option for fork integration. Forks may update pg_control version or WAL magic without affecting the structures that pgBackRest depends on. This option forces pgBackRest to treat a cluster as the specified version when it cannot be automatically identified.
2023-03-09 02:23:15 +01:00
TEST_ERROR(pgControlFromFile(storageTest, NULL), FormatError, "page size is 32768 but must be 8192");
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 10:24:42 +01:00
Fix formatting errors. Errors in our current (manually-maintained) code format discovered by uncrustify.
2023-01-30 11:16:31 +07:00
// -------------------------------------------------------------------------------------------------------------------------
Add HRN_PG_CONTROL_PUT() and HRN_PG_CONTROL_TIME(). These macros simplify management of pg_control test files. Centralize time updates for pg_control in the command/backup module. This caused some time updates in the logs. Finally, move the postgres module after the storage module so it can use storage macros.
2021-11-30 13:23:11 -05:00
HRN_PG_CONTROL_PUT(
Remove support for PostgreSQL 9.0/9.1/9.2. Our new policy is to support ten versions of PostgreSQL, the five supported releases and the last five EOL releases. As of PostgreSQL 15, that means 9.0/9.1/9.2 are no longer supported by pgBackRest. Remove all logic associated with 9.0/9.1/9.2 and update the tests. Document the new support policy. Update InfoPg to read/write control versions for the history in backup.info, since we can no longer rely on the mappings being available. In theory this could have been an issue after removing 8.3/8.4 if anybody was using a version that old.
2022-12-20 12:20:47 +07:00
storageTest, PG_VERSION_93, .systemId = 0xEFEFEFEFEF, .catalogVersion = hrnPgCatalogVersion(PG_VERSION_93),
Add timeline and checkpoint checks to backup. Add the following checks: * Checkpoint is updated in pg_control after pg_start_backup(). This helps ensure that PostgreSQL and pgBackRest have a consistent view of the storage and that PGDATA paths match. * Timeline of backup start WAL file matches pg_control. Hard to see how this one could get hit, but we have the power... * Standby is on the same timeline as the primary. If not, this standby is not following the primary. * Last standby checkpoint is not greater than the backup checkpoint. If so, this standby is not following the primary. This also requires some additional plumbing to read/write timeline/checkpoint from pg_control and parse timelines from WAL filenames. There were some changes in the backup tests caused by the fact that pg_control now has different contents for each backup. The check to ensure that the required checkpoint was reached on the standby should also be updated to use pg_control (it currently uses pg_control_checkpoint()), but that requires non-trivial changes to the test harness and will need to wait.
2021-12-07 09:21:07 -05:00
.checkpoint = 0xAABBAABBEEFFEEFF, .timeline = 88);
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 10:24:42 +01:00
Add pg-version-force option for fork integration. Forks may update pg_control version or WAL magic without affecting the structures that pgBackRest depends on. This option forces pgBackRest to treat a cluster as the specified version when it cannot be automatically identified.
2023-03-09 02:23:15 +01:00
TEST_ASSIGN(info, pgControlFromFile(storageTest, NULL), "get control info v90");
Fix incorrect result types in unit tests. Upcoming changes to the TEST_RESULT_* macros are more type safe and identified that the wrong macros were being used to test results in many cases. Commit these changes separately to verify that they work with the current macro versions. Note that no core bugs were exposed by these changes.
2020-03-22 20:25:31 -04:00
TEST_RESULT_UINT(info.systemId, 0xEFEFEFEFEF, " check system id");
Remove support for PostgreSQL 9.0/9.1/9.2. Our new policy is to support ten versions of PostgreSQL, the five supported releases and the last five EOL releases. As of PostgreSQL 15, that means 9.0/9.1/9.2 are no longer supported by pgBackRest. Remove all logic associated with 9.0/9.1/9.2 and update the tests. Document the new support policy. Update InfoPg to read/write control versions for the history in backup.info, since we can no longer rely on the mappings being available. In theory this could have been an issue after removing 8.3/8.4 if anybody was using a version that old.
2022-12-20 12:20:47 +07:00
TEST_RESULT_UINT(info.version, PG_VERSION_93, " check version");
TEST_RESULT_UINT(info.catalogVersion, 201306121, " check catalog version");
Add timeline and checkpoint checks to backup. Add the following checks: * Checkpoint is updated in pg_control after pg_start_backup(). This helps ensure that PostgreSQL and pgBackRest have a consistent view of the storage and that PGDATA paths match. * Timeline of backup start WAL file matches pg_control. Hard to see how this one could get hit, but we have the power... * Standby is on the same timeline as the primary. If not, this standby is not following the primary. * Last standby checkpoint is not greater than the backup checkpoint. If so, this standby is not following the primary. This also requires some additional plumbing to read/write timeline/checkpoint from pg_control and parse timelines from WAL filenames. There were some changes in the backup tests caused by the fact that pg_control now has different contents for each backup. The check to ensure that the required checkpoint was reached on the standby should also be updated to use pg_control (it currently uses pg_control_checkpoint()), but that requires non-trivial changes to the test harness and will need to wait.
2021-12-07 09:21:07 -05:00
TEST_RESULT_UINT(info.checkpoint, 0xAABBAABBEEFFEEFF, "check checkpoint");
TEST_RESULT_UINT(info.timeline, 88, "check timeline");
Add pg-version-force option for fork integration. Forks may update pg_control version or WAL magic without affecting the structures that pgBackRest depends on. This option forces pgBackRest to treat a cluster as the specified version when it cannot be automatically identified.
2023-03-09 02:23:15 +01:00
// -------------------------------------------------------------------------------------------------------------------------
TEST_TITLE("force control version");
HRN_PG_CONTROL_OVERRIDE_PUT(
storageTest, PG_VERSION_15, 1501, .systemId = 0xEFEFEFEFEF, .catalogVersion = 202211111,
.checkpoint = 0xAABBAABBEEFFEEFF, .timeline = 88);
TEST_ERROR(
pgControlFromFile(storageTest, NULL), VersionNotSupportedError,
"unexpected control version = 1501 and catalog version = 202211111\n"
"HINT: is this version of PostgreSQL supported?");
TEST_ERROR(
pgControlFromFile(storageTest, STRDEF("99")), VersionNotSupportedError,
"invalid PostgreSQL version 990000\n"
"HINT: is this version of PostgreSQL supported?");
Rename PG_VERSION_*_STR constants to PG_VERSION_*_Z. This is more consistent with other zero-terminated string constants and also has the benefit of being shorter.
2023-04-16 17:32:24 +03:00
TEST_ASSIGN(info, pgControlFromFile(storageTest, STRDEF(PG_VERSION_15_Z)), "get control info v90");
Add pg-version-force option for fork integration. Forks may update pg_control version or WAL magic without affecting the structures that pgBackRest depends on. This option forces pgBackRest to treat a cluster as the specified version when it cannot be automatically identified.
2023-03-09 02:23:15 +01:00
TEST_RESULT_UINT(info.systemId, 0xEFEFEFEFEF, "check system id");
TEST_RESULT_UINT(info.version, PG_VERSION_15, "check version");
TEST_RESULT_UINT(info.catalogVersion, 202211111, "check catalog version");
TEST_RESULT_UINT(info.checkpoint, 0xAABBAABBEEFFEEFF, "check checkpoint");
TEST_RESULT_UINT(info.timeline, 88, "check timeline");
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 10:24:42 +01:00
}
Add pgLsnFromStr(), pgLsnToStr(), and pgLsnToWalSegment().
2019-12-08 14:19:47 -05:00
// *****************************************************************************************************************************
Add timeline and checkpoint checks to backup. Add the following checks: * Checkpoint is updated in pg_control after pg_start_backup(). This helps ensure that PostgreSQL and pgBackRest have a consistent view of the storage and that PGDATA paths match. * Timeline of backup start WAL file matches pg_control. Hard to see how this one could get hit, but we have the power... * Standby is on the same timeline as the primary. If not, this standby is not following the primary. * Last standby checkpoint is not greater than the backup checkpoint. If so, this standby is not following the primary. This also requires some additional plumbing to read/write timeline/checkpoint from pg_control and parse timelines from WAL filenames. There were some changes in the backup tests caused by the fact that pg_control now has different contents for each backup. The check to ensure that the required checkpoint was reached on the standby should also be updated to use pg_control (it currently uses pg_control_checkpoint()), but that requires non-trivial changes to the test harness and will need to wait.
2021-12-07 09:21:07 -05:00
if (testBegin("pgLsnFromStr(), pgLsnToStr(), pgLsnToWalSegment(), pg*FromWalSegment(), and pgLsnRangeToWalSegmentList()"))
Add pgLsnFromStr(), pgLsnToStr(), and pgLsnToWalSegment().
2019-12-08 14:19:47 -05:00
{
TEST_RESULT_UINT(pgLsnFromStr(STRDEF("1/1")), 0x0000000100000001, "lsn to string");
TEST_RESULT_UINT(pgLsnFromStr(STRDEF("ffffffff/ffffffff")), 0xFFFFFFFFFFFFFFFF, "lsn to string");
TEST_RESULT_UINT(pgLsnFromStr(STRDEF("ffffffff/aaaaaaaa")), 0xFFFFFFFFAAAAAAAA, "lsn to string");
TEST_RESULT_STR_Z(pgLsnToStr(0xFFFFFFFFAAAAAAAA), "ffffffff/aaaaaaaa", "string to lsn");
TEST_RESULT_STR_Z(pgLsnToStr(0x0000000000000000), "0/0", "string to lsn");
TEST_RESULT_STR_Z(pgLsnToStr(0x0000000100000002), "1/2", "string to lsn");
TEST_RESULT_STR_Z(pgLsnToWalSegment(1, 0xFFFFFFFFAAAAAAAA, 0x1000000), "00000001FFFFFFFF000000AA", "lsn to wal segment");
TEST_RESULT_STR_Z(pgLsnToWalSegment(1, 0xFFFFFFFFAAAAAAAA, 0x40000000), "00000001FFFFFFFF00000002", "lsn to wal segment");
Fix bad arithmetic in pgLsnToWalSegment(). / takes precedence over & but the appropriate parens were not provided. By some bad luck the tests worked either way, so add a new test that only works the correct way to prevent a regression.
2019-12-12 16:21:51 -05:00
TEST_RESULT_STR_Z(pgLsnToWalSegment(1, 0xFFFFFFFF40000000, 0x40000000), "00000001FFFFFFFF00000001", "lsn to wal segment");
Add pgLsnRangeToWalSegmentList() to convert lsn range to wal segments.
2019-12-12 16:43:34 -05:00
Add timeline and checkpoint checks to backup. Add the following checks: * Checkpoint is updated in pg_control after pg_start_backup(). This helps ensure that PostgreSQL and pgBackRest have a consistent view of the storage and that PGDATA paths match. * Timeline of backup start WAL file matches pg_control. Hard to see how this one could get hit, but we have the power... * Standby is on the same timeline as the primary. If not, this standby is not following the primary. * Last standby checkpoint is not greater than the backup checkpoint. If so, this standby is not following the primary. This also requires some additional plumbing to read/write timeline/checkpoint from pg_control and parse timelines from WAL filenames. There were some changes in the backup tests caused by the fact that pg_control now has different contents for each backup. The check to ensure that the required checkpoint was reached on the standby should also be updated to use pg_control (it currently uses pg_control_checkpoint()), but that requires non-trivial changes to the test harness and will need to wait.
2021-12-07 09:21:07 -05:00
TEST_RESULT_UINT(pgTimelineFromWalSegment(STRDEF("00000001FFFFFFFF000000AA")), 1, "timeline 1");
TEST_RESULT_UINT(pgTimelineFromWalSegment(STRDEF("F000000FFFFFFFFF000000AA")), 0xF000000F, "timeline F000000F");
Add TEST_RESULT_STRLST_*() unit test macros. This avoids the need for strLstJoin() when testing lists. Lists are \n delimited (rather than command or pipe) so that non-trivial lists can be more easily diff'd.
2021-01-08 12:49:33 -05:00
TEST_RESULT_STRLST_Z(
pgLsnRangeToWalSegmentList(
Remove support for PostgreSQL 9.0/9.1/9.2. Our new policy is to support ten versions of PostgreSQL, the five supported releases and the last five EOL releases. As of PostgreSQL 15, that means 9.0/9.1/9.2 are no longer supported by pgBackRest. Remove all logic associated with 9.0/9.1/9.2 and update the tests. Document the new support policy. Update InfoPg to read/write control versions for the history in backup.info, since we can no longer rely on the mappings being available. In theory this could have been an issue after removing 8.3/8.4 if anybody was using a version that old.
2022-12-20 12:20:47 +07:00
2, pgLsnFromStr(STRDEF("1/FD000000")), pgLsnFromStr(STRDEF("2/60")), 16 * 1024 * 1024),
Add TEST_RESULT_STRLST_*() unit test macros. This avoids the need for strLstJoin() when testing lists. Lists are \n delimited (rather than command or pipe) so that non-trivial lists can be more easily diff'd.
2021-01-08 12:49:33 -05:00
"0000000200000001000000FD\n0000000200000001000000FE\n0000000200000001000000FF\n000000020000000200000000\n",
Remove support for PostgreSQL 9.0/9.1/9.2. Our new policy is to support ten versions of PostgreSQL, the five supported releases and the last five EOL releases. As of PostgreSQL 15, that means 9.0/9.1/9.2 are no longer supported by pgBackRest. Remove all logic associated with 9.0/9.1/9.2 and update the tests. Document the new support policy. Update InfoPg to read/write control versions for the history in backup.info, since we can no longer rely on the mappings being available. In theory this could have been an issue after removing 8.3/8.4 if anybody was using a version that old.
2022-12-20 12:20:47 +07:00
"get range");
Add TEST_RESULT_STRLST_*() unit test macros. This avoids the need for strLstJoin() when testing lists. Lists are \n delimited (rather than command or pipe) so that non-trivial lists can be more easily diff'd.
2021-01-08 12:49:33 -05:00
TEST_RESULT_STRLST_Z(
pgLsnRangeToWalSegmentList(
Remove support for PostgreSQL 9.0/9.1/9.2. Our new policy is to support ten versions of PostgreSQL, the five supported releases and the last five EOL releases. As of PostgreSQL 15, that means 9.0/9.1/9.2 are no longer supported by pgBackRest. Remove all logic associated with 9.0/9.1/9.2 and update the tests. Document the new support policy. Update InfoPg to read/write control versions for the history in backup.info, since we can no longer rely on the mappings being available. In theory this could have been an issue after removing 8.3/8.4 if anybody was using a version that old.
2022-12-20 12:20:47 +07:00
2, pgLsnFromStr(STRDEF("A/800")), pgLsnFromStr(STRDEF("B/C0000000")), 1024 * 1024 * 1024),
Add TEST_RESULT_STRLST_*() unit test macros. This avoids the need for strLstJoin() when testing lists. Lists are \n delimited (rather than command or pipe) so that non-trivial lists can be more easily diff'd.
2021-01-08 12:49:33 -05:00
"000000020000000A00000000\n000000020000000A00000001\n000000020000000A00000002\n000000020000000A00000003\n"
Reformat code with uncrustify. uncrustify has been configured to be as close to the current format as possible but the following changes were required: * Break long struct initializiers out of function calls. * Bit fields get extra spacing. * Strings that continue from the previous line no longer indented. * Ternary operators that do not fit on a single line moved to the next line first. * Align under parens for multi-line if statements. * Macros in header #if blocks are no longer indented. * Purposeful lack of function indentation in tests has been removed. Currently uncrustify does not completely reflow the code so there are some edge cases that might not be caught. However, this still represents a huge improvement and the formatting can be refined going forward. Support code for uncrustify will be in a followup commit.
2023-01-30 11:55:54 +07:00
"000000020000000B00000000\n000000020000000B00000001\n000000020000000B00000002\n000000020000000B00000003\n",
Add pgLsnRangeToWalSegmentList() to convert lsn range to wal segments.
2019-12-12 16:43:34 -05:00
"get range >= 11/1GB");
Add TEST_RESULT_STRLST_*() unit test macros. This avoids the need for strLstJoin() when testing lists. Lists are \n delimited (rather than command or pipe) so that non-trivial lists can be more easily diff'd.
2021-01-08 12:49:33 -05:00
TEST_RESULT_STRLST_Z(
pgLsnRangeToWalSegmentList(
Remove support for PostgreSQL 9.0/9.1/9.2. Our new policy is to support ten versions of PostgreSQL, the five supported releases and the last five EOL releases. As of PostgreSQL 15, that means 9.0/9.1/9.2 are no longer supported by pgBackRest. Remove all logic associated with 9.0/9.1/9.2 and update the tests. Document the new support policy. Update InfoPg to read/write control versions for the history in backup.info, since we can no longer rely on the mappings being available. In theory this could have been an issue after removing 8.3/8.4 if anybody was using a version that old.
2022-12-20 12:20:47 +07:00
3, pgLsnFromStr(STRDEF("7/FFEFFFFF")), pgLsnFromStr(STRDEF("8/001AAAAA")), 1024 * 1024),
Add TEST_RESULT_STRLST_*() unit test macros. This avoids the need for strLstJoin() when testing lists. Lists are \n delimited (rather than command or pipe) so that non-trivial lists can be more easily diff'd.
2021-01-08 12:49:33 -05:00
"000000030000000700000FFE\n000000030000000700000FFF\n000000030000000800000000\n000000030000000800000001\n",
Add pgLsnRangeToWalSegmentList() to convert lsn range to wal segments.
2019-12-12 16:43:34 -05:00
"get range >= 11/1MB");
Add pgLsnFromStr(), pgLsnToStr(), and pgLsnToWalSegment().
2019-12-08 14:19:47 -05:00
}
Add Db object to encapsulate PostgreSQL queries and commands. Migrate functionality from the Perl Db module to C. For now this is just enough to implement the WAL switch check. Add the dbGet() helper function to get Db objects easily. Create macros in harnessPq to make writing pq scripts easier by grouping commonly used functions together. Reviewed by Cynthia Shang.
2019-08-01 15:38:27 -04:00
// *****************************************************************************************************************************
Add pgWalPath() to return version-specific WAL path. Also update the manifest module to use the new function.
2019-12-12 16:11:09 -05:00
if (testBegin("pgLsnName(), pgTablespaceId(), pgWalName(), pgWalPath(), and pgXactPath()"))
Add Db object to encapsulate PostgreSQL queries and commands. Migrate functionality from the Perl Db module to C. For now this is just enough to implement the WAL switch check. Add the dbGet() helper function to get Db objects easily. Create macros in harnessPq to make writing pq scripts easier by grouping commonly used functions together. Reviewed by Cynthia Shang.
2019-08-01 15:38:27 -04:00
{
Change meaning of TEST_RESULT_STR() macro. This macro was created before the String object existed so subsequent usage with String always included a lot of strPtr() wrapping. TEST_RESULT_STR_Z() had already been introduced but a wholesale replacement of TEST_RESULT_STR() was not done since the priority was on the C migration. Update all calls to (old) TEST_RESULT_STR() with one of the following variants: (new) TEST_RESULT_STR(), TEST_RESULT_STR_Z(), TEST_RESULT_Z(), TEST_RESULT_Z_STR().
2019-12-26 18:08:27 -07:00
TEST_RESULT_STR_Z(pgLsnName(PG_VERSION_96), "location", "check location name");
TEST_RESULT_STR_Z(pgLsnName(PG_VERSION_10), "lsn", "check lsn name");
Add pgLsnName() and pgXactPath().
2019-12-01 15:49:34 -05:00
Remove support for PostgreSQL 9.0/9.1/9.2. Our new policy is to support ten versions of PostgreSQL, the five supported releases and the last five EOL releases. As of PostgreSQL 15, that means 9.0/9.1/9.2 are no longer supported by pgBackRest. Remove all logic associated with 9.0/9.1/9.2 and update the tests. Document the new support policy. Update InfoPg to read/write control versions for the history in backup.info, since we can no longer rely on the mappings being available. In theory this could have been an issue after removing 8.3/8.4 if anybody was using a version that old.
2022-12-20 12:20:47 +07:00
TEST_RESULT_STR_Z(pgTablespaceId(PG_VERSION_93, 201306121), "PG_9.3_201306121", "check 9.3 tablespace id");
Improve PostgreSQL version identification. Previously, catalog versions were fixed for all versions which made maintaining the catalog versions during PostgreSQL beta and release candidate cycles very painful. A version of pgBackRest which was functionally compatible was rendered useless by a catalog version bump in PostgreSQL. Instead use only the control version to identify a PostgreSQL version when possible. Some older versions require a catalog version to positively identify a PostgreSQL version, so include them when required. Since the catalog number is required to work with tablespaces it will need to be stored. There's already a copy of it in backup.info so use that (even though we have been ignoring it in the C versions).
2020-09-18 16:55:26 -04:00
TEST_RESULT_STR_Z(pgTablespaceId(PG_VERSION_94, 999999999), "PG_9.4_999999999", "check 9.4 tablespace id");
Add function to generate PostgreSQL tablespace identifier. In PostgreSQL >= 9.0 each tablespace data is stored in a specially named directory so different major versions can share the same tablespace path.
2019-09-08 06:53:23 -04:00
Change meaning of TEST_RESULT_STR() macro. This macro was created before the String object existed so subsequent usage with String always included a lot of strPtr() wrapping. TEST_RESULT_STR_Z() had already been introduced but a wholesale replacement of TEST_RESULT_STR() was not done since the priority was on the C migration. Update all calls to (old) TEST_RESULT_STR() with one of the following variants: (new) TEST_RESULT_STR(), TEST_RESULT_STR_Z(), TEST_RESULT_Z(), TEST_RESULT_Z_STR().
2019-12-26 18:08:27 -07:00
TEST_RESULT_STR_Z(pgWalName(PG_VERSION_96), "xlog", "check xlog name");
TEST_RESULT_STR_Z(pgWalName(PG_VERSION_10), "wal", "check wal name");
Add pgLsnName() and pgXactPath().
2019-12-01 15:49:34 -05:00
Add pgWalPath() to return version-specific WAL path. Also update the manifest module to use the new function.
2019-12-12 16:11:09 -05:00
TEST_RESULT_STR_Z(pgWalPath(PG_VERSION_96), "pg_xlog", "check xlog path");
TEST_RESULT_STR_Z(pgWalPath(PG_VERSION_10), "pg_wal", "check wal path");
Change meaning of TEST_RESULT_STR() macro. This macro was created before the String object existed so subsequent usage with String always included a lot of strPtr() wrapping. TEST_RESULT_STR_Z() had already been introduced but a wholesale replacement of TEST_RESULT_STR() was not done since the priority was on the C migration. Update all calls to (old) TEST_RESULT_STR() with one of the following variants: (new) TEST_RESULT_STR(), TEST_RESULT_STR_Z(), TEST_RESULT_Z(), TEST_RESULT_Z_STR().
2019-12-26 18:08:27 -07:00
TEST_RESULT_STR_Z(pgXactPath(PG_VERSION_96), "pg_clog", "check pg_clog name");
TEST_RESULT_STR_Z(pgXactPath(PG_VERSION_10), "pg_xact", "check pg_xact name");
Add Db object to encapsulate PostgreSQL queries and commands. Migrate functionality from the Perl Db module to C. For now this is just enough to implement the WAL switch check. Add the dbGet() helper function to get Db objects easily. Create macros in harnessPq to make writing pq scripts easier by grouping commonly used functions together. Reviewed by Cynthia Shang.
2019-08-01 15:38:27 -04:00
}
Move logic from postgres/pageChecksum to command/backup/pageChecksum(). The postgres/pageChecksum module was designed as an interface to the C structs for the Perl code. The new C code can do this directly so no need for an interface. Move the remaining test for pgPageChecksum() into the postgres/interface test module.
2020-03-05 16:12:54 -05:00
// *****************************************************************************************************************************
if (testBegin("pgPageChecksum()"))
{
unsigned char page[PG_PAGE_SIZE_DEFAULT];
memset(page, 0xFF, PG_PAGE_SIZE_DEFAULT);
Add support for testing on 64-bit big-endian architectures. In particular add support for s390x but we hope this will work for other 64-bit big-endian architectures. Run basic unit tests on Travis CI for 390x.
2020-07-20 09:59:16 -04:00
TEST_RESULT_UINT(pgPageChecksum(page, 0), TEST_BIG_ENDIAN() ? 0xF55E : 0x0E1C, "check 0xFF filled page, block 0");
TEST_RESULT_UINT(pgPageChecksum(page, 999), TEST_BIG_ENDIAN() ? 0xF1B9 : 0x0EC3, "check 0xFF filled page, block 999");
Move logic from postgres/pageChecksum to command/backup/pageChecksum(). The postgres/pageChecksum module was designed as an interface to the C structs for the Perl code. The new C code can do this directly so no need for an interface. Move the remaining test for pgPageChecksum() into the postgres/interface test module.
2020-03-05 16:12:54 -05:00
}
Add WAL info to PostgreSQL interface. This allows the WAL header to be read for any supported version on PostgreSQL.
2019-03-19 19:44:06 +04:00
// *****************************************************************************************************************************
if (testBegin("pgWalFromBuffer() and pgWalFromFile()"))
{
Add defines for many test*() getter functions. A define was already added for TEST_PATH but it was not widely used. Replace all occurrences of testPath() with TEST_PATH in the tests. Replace testUser() with TEST_USER, testGroup() with TEST_GROUP, testRepoPath() with HRN_PATH_REPO, testDataPath() with HRN_PATH, testProjectExe() with TEST_PROJECT_EXE, and testScale() with TEST_SCALE. Replace {[path]}, {[user]}, {[group]}, etc. with defines and remove hrnReplaceKey(). This is better than having two ways to deal with replacements. In some cases the original test*() getters were kept because they are used by the harness, which does not have access to the new defines. Move them to harnessTest.intern.h to indicate that the tests should no longer use them.
2021-05-22 09:30:54 -04:00
const String *walFile = STRDEF(TEST_PATH "/0000000F0000000F0000000F");
Add WAL info to PostgreSQL interface. This allows the WAL header to be read for any supported version on PostgreSQL.
2019-03-19 19:44:06 +04:00
Fix formatting errors. Errors in our current (manually-maintained) code format discovered by uncrustify.
2023-01-30 11:16:31 +07:00
// -------------------------------------------------------------------------------------------------------------------------
Allow control version and WAL magic to be overridden in test harness. This makes it easier to write tests for invalid control version/WAL magic. Also add HRN_PG_WAL_TO_BUFFER() to simplify generation of WAL headers.
2023-03-04 12:50:02 +07:00
TEST_TITLE("unknown WAL magic");
Add WAL info to PostgreSQL interface. This allows the WAL header to be read for any supported version on PostgreSQL.
2019-03-19 19:44:06 +04:00
Buffer *result = bufNew((size_t)16 * 1024 * 1024);
memset(bufPtr(result), 0, bufSize(result));
bufUsedSet(result, bufSize(result));
Allow control version and WAL magic to be overridden in test harness. This makes it easier to write tests for invalid control version/WAL magic. Also add HRN_PG_WAL_TO_BUFFER() to simplify generation of WAL headers.
2023-03-04 12:50:02 +07:00
HRN_PG_WAL_OVERRIDE_TO_BUFFER(result, PG_VERSION_15, 777);
Add WAL info to PostgreSQL interface. This allows the WAL header to be read for any supported version on PostgreSQL.
2019-03-19 19:44:06 +04:00
TEST_ERROR(
Add pg-version-force option for fork integration. Forks may update pg_control version or WAL magic without affecting the structures that pgBackRest depends on. This option forces pgBackRest to treat a cluster as the specified version when it cannot be automatically identified.
2023-03-09 02:23:15 +01:00
pgWalFromBuffer(result, NULL), VersionNotSupportedError,
Add WAL info to PostgreSQL interface. This allows the WAL header to be read for any supported version on PostgreSQL.
2019-03-19 19:44:06 +04:00
"unexpected WAL magic 777\n"
Reformat code with uncrustify. uncrustify has been configured to be as close to the current format as possible but the following changes were required: * Break long struct initializiers out of function calls. * Bit fields get extra spacing. * Strings that continue from the previous line no longer indented. * Ternary operators that do not fit on a single line moved to the next line first. * Align under parens for multi-line if statements. * Macros in header #if blocks are no longer indented. * Purposeful lack of function indentation in tests has been removed. Currently uncrustify does not completely reflow the code so there are some edge cases that might not be caught. However, this still represents a huge improvement and the formatting can be refined going forward. Support code for uncrustify will be in a followup commit.
2023-01-30 11:55:54 +07:00
"HINT: is this version of PostgreSQL supported?");
Add WAL info to PostgreSQL interface. This allows the WAL header to be read for any supported version on PostgreSQL.
2019-03-19 19:44:06 +04:00
Allow control version and WAL magic to be overridden in test harness. This makes it easier to write tests for invalid control version/WAL magic. Also add HRN_PG_WAL_TO_BUFFER() to simplify generation of WAL headers.
2023-03-04 12:50:02 +07:00
// -------------------------------------------------------------------------------------------------------------------------
TEST_TITLE("invalid wal flag");
((PgWalCommon *)bufPtr(result))->flag = 0;
Add pg-version-force option for fork integration. Forks may update pg_control version or WAL magic without affecting the structures that pgBackRest depends on. This option forces pgBackRest to treat a cluster as the specified version when it cannot be automatically identified.
2023-03-09 02:23:15 +01:00
TEST_ERROR(pgWalFromBuffer(result, NULL), FormatError, "first page header in WAL file is expected to be in long format");
Allow control version and WAL magic to be overridden in test harness. This makes it easier to write tests for invalid control version/WAL magic. Also add HRN_PG_WAL_TO_BUFFER() to simplify generation of WAL headers.
2023-03-04 12:50:02 +07:00
Fix formatting errors. Errors in our current (manually-maintained) code format discovered by uncrustify.
2023-01-30 11:16:31 +07:00
// -------------------------------------------------------------------------------------------------------------------------
Set WAL long header flag in test function missed in e938a892. This was missed because the unit tests were reusing a buffer without resetting it to zero, so this flag ended up still set when the test function was called. This was not a live issue since it only expressed in tests and this code is not used in master yet.
2019-03-26 18:46:44 +02:00
memset(bufPtr(result), 0, bufSize(result));
Allow control version and WAL magic to be overridden in test harness. This makes it easier to write tests for invalid control version/WAL magic. Also add HRN_PG_WAL_TO_BUFFER() to simplify generation of WAL headers.
2023-03-04 12:50:02 +07:00
HRN_PG_WAL_TO_BUFFER(result, PG_VERSION_11, .systemId = 0xECAFECAF, .size = PG_WAL_SEGMENT_SIZE_DEFAULT * 2);
Remove *MP() macros variants. Adding a dummy column which is always set by the P() macro allows a single macro to be used for parameters or no parameters without violating C's prohibition on the {} initializer. -Wmissing-field-initializers remains disabled because it still gives wildly different results between versions of gcc.
2019-11-17 15:10:40 -05:00
storagePutP(storageNewWriteP(storageTest, walFile), result);
Add WAL info to PostgreSQL interface. This allows the WAL header to be read for any supported version on PostgreSQL.
2019-03-19 19:44:06 +04:00
PgWal info = {0};
Add pg-version-force option for fork integration. Forks may update pg_control version or WAL magic without affecting the structures that pgBackRest depends on. This option forces pgBackRest to treat a cluster as the specified version when it cannot be automatically identified.
2023-03-09 02:23:15 +01:00
TEST_ASSIGN(info, pgWalFromFile(walFile, storageTest, NULL), "get wal info v11");
Fix incorrect result types in unit tests. Upcoming changes to the TEST_RESULT_* macros are more type safe and identified that the wrong macros were being used to test results in many cases. Commit these changes separately to verify that they work with the current macro versions. Note that no core bugs were exposed by these changes.
2020-03-22 20:25:31 -04:00
TEST_RESULT_UINT(info.systemId, 0xECAFECAF, " check system id");
TEST_RESULT_UINT(info.version, PG_VERSION_11, " check version");
Read segment size from WAL headers. This allows validation of the WAL segment size for PostgreSQL versions <= 10.
2020-07-09 17:32:36 -04:00
TEST_RESULT_UINT(info.size, PG_WAL_SEGMENT_SIZE_DEFAULT * 2, " check size");
Add WAL info to PostgreSQL interface. This allows the WAL header to be read for any supported version on PostgreSQL.
2019-03-19 19:44:06 +04:00
Fix formatting errors. Errors in our current (manually-maintained) code format discovered by uncrustify.
2023-01-30 11:16:31 +07:00
// -------------------------------------------------------------------------------------------------------------------------
Set WAL long header flag in test function missed in e938a892. This was missed because the unit tests were reusing a buffer without resetting it to zero, so this flag ended up still set when the test function was called. This was not a live issue since it only expressed in tests and this code is not used in master yet.
2019-03-26 18:46:44 +02:00
memset(bufPtr(result), 0, bufSize(result));
Allow control version and WAL magic to be overridden in test harness. This makes it easier to write tests for invalid control version/WAL magic. Also add HRN_PG_WAL_TO_BUFFER() to simplify generation of WAL headers.
2023-03-04 12:50:02 +07:00
HRN_PG_WAL_TO_BUFFER(result, PG_VERSION_96, .systemId = 0xEAEAEAEA, .size = PG_WAL_SEGMENT_SIZE_DEFAULT * 2);
Read segment size from WAL headers. This allows validation of the WAL segment size for PostgreSQL versions <= 10.
2020-07-09 17:32:36 -04:00
Add pg-version-force option for fork integration. Forks may update pg_control version or WAL magic without affecting the structures that pgBackRest depends on. This option forces pgBackRest to treat a cluster as the specified version when it cannot be automatically identified.
2023-03-09 02:23:15 +01:00
TEST_ERROR(
pgWalFromBuffer(result, NULL), FormatError, "wal segment size is 33554432 but must be 16777216 for PostgreSQL <= 10");
Read segment size from WAL headers. This allows validation of the WAL segment size for PostgreSQL versions <= 10.
2020-07-09 17:32:36 -04:00
Fix formatting errors. Errors in our current (manually-maintained) code format discovered by uncrustify.
2023-01-30 11:16:31 +07:00
// -------------------------------------------------------------------------------------------------------------------------
Read segment size from WAL headers. This allows validation of the WAL segment size for PostgreSQL versions <= 10.
2020-07-09 17:32:36 -04:00
memset(bufPtr(result), 0, bufSize(result));
Allow control version and WAL magic to be overridden in test harness. This makes it easier to write tests for invalid control version/WAL magic. Also add HRN_PG_WAL_TO_BUFFER() to simplify generation of WAL headers.
2023-03-04 12:50:02 +07:00
HRN_PG_WAL_TO_BUFFER(result, PG_VERSION_93, .systemId = 0xEAEAEAEA, .size = PG_WAL_SEGMENT_SIZE_DEFAULT);
Remove *MP() macros variants. Adding a dummy column which is always set by the P() macro allows a single macro to be used for parameters or no parameters without violating C's prohibition on the {} initializer. -Wmissing-field-initializers remains disabled because it still gives wildly different results between versions of gcc.
2019-11-17 15:10:40 -05:00
storagePutP(storageNewWriteP(storageTest, walFile), result);
Add WAL info to PostgreSQL interface. This allows the WAL header to be read for any supported version on PostgreSQL.
2019-03-19 19:44:06 +04:00
Add pg-version-force option for fork integration. Forks may update pg_control version or WAL magic without affecting the structures that pgBackRest depends on. This option forces pgBackRest to treat a cluster as the specified version when it cannot be automatically identified.
2023-03-09 02:23:15 +01:00
TEST_ASSIGN(info, pgWalFromFile(walFile, storageTest, NULL), "get wal info v9.3");
Fix incorrect result types in unit tests. Upcoming changes to the TEST_RESULT_* macros are more type safe and identified that the wrong macros were being used to test results in many cases. Commit these changes separately to verify that they work with the current macro versions. Note that no core bugs were exposed by these changes.
2020-03-22 20:25:31 -04:00
TEST_RESULT_UINT(info.systemId, 0xEAEAEAEA, " check system id");
Remove support for PostgreSQL 9.0/9.1/9.2. Our new policy is to support ten versions of PostgreSQL, the five supported releases and the last five EOL releases. As of PostgreSQL 15, that means 9.0/9.1/9.2 are no longer supported by pgBackRest. Remove all logic associated with 9.0/9.1/9.2 and update the tests. Document the new support policy. Update InfoPg to read/write control versions for the history in backup.info, since we can no longer rely on the mappings being available. In theory this could have been an issue after removing 8.3/8.4 if anybody was using a version that old.
2022-12-20 12:20:47 +07:00
TEST_RESULT_UINT(info.version, PG_VERSION_93, " check version");
Read segment size from WAL headers. This allows validation of the WAL segment size for PostgreSQL versions <= 10.
2020-07-09 17:32:36 -04:00
TEST_RESULT_UINT(info.size, PG_WAL_SEGMENT_SIZE_DEFAULT, " check size");
Add pg-version-force option for fork integration. Forks may update pg_control version or WAL magic without affecting the structures that pgBackRest depends on. This option forces pgBackRest to treat a cluster as the specified version when it cannot be automatically identified.
2023-03-09 02:23:15 +01:00
// -------------------------------------------------------------------------------------------------------------------------
TEST_TITLE("force WAL version");
memset(bufPtr(result), 0, bufSize(result));
HRN_PG_WAL_OVERRIDE_TO_BUFFER(result, PG_VERSION_15, 777, .systemId = 0xFAFAFAFA, .size = PG_WAL_SEGMENT_SIZE_DEFAULT);
storagePutP(storageNewWriteP(storageTest, walFile), result);
TEST_ERROR(
pgWalFromBuffer(result, NULL), VersionNotSupportedError,
"unexpected WAL magic 777\n"
"HINT: is this version of PostgreSQL supported?");
Rename PG_VERSION_*_STR constants to PG_VERSION_*_Z. This is more consistent with other zero-terminated string constants and also has the benefit of being shorter.
2023-04-16 17:32:24 +03:00
TEST_ASSIGN(info, pgWalFromFile(walFile, storageTest, STRDEF(PG_VERSION_15_Z)), "force wal info v15");
Add pg-version-force option for fork integration. Forks may update pg_control version or WAL magic without affecting the structures that pgBackRest depends on. This option forces pgBackRest to treat a cluster as the specified version when it cannot be automatically identified.
2023-03-09 02:23:15 +01:00
TEST_RESULT_UINT(info.systemId, 0xFAFAFAFA, "check system id");
TEST_RESULT_UINT(info.version, PG_VERSION_15, " check version");
TEST_RESULT_UINT(info.size, PG_WAL_SEGMENT_SIZE_DEFAULT, " check size");
Add WAL info to PostgreSQL interface. This allows the WAL header to be read for any supported version on PostgreSQL.
2019-03-19 19:44:06 +04:00
}
Add missing ToLog() coverage to String, List, and PgControl. Missing coverage is exposed in the next commit which disables test tracing by default.
2019-02-22 11:31:37 +02:00
// *****************************************************************************************************************************
if (testBegin("pgControlToLog()"))
{
Refactor logging functions to never allocate memory. Allocating memory made these functions simpler but it meant that memory was leaking into the calling context when logging was enabled. It is not clear that this was an issue but it seems that trace level logging could result it a lot of memory usage depending on the use case. This also makes it possible to audit allocations returned to the calling context, which will be done in a followup commit. Also rename objToLog() to objNameToLog() since it seemed logical to name the new function objToLog().
2023-01-12 17:14:36 +07:00
char logBuf[STACK_TRACE_PARAM_MAX];
Add missing ToLog() coverage to String, List, and PgControl. Missing coverage is exposed in the next commit which disables test tracing by default.
2019-02-22 11:31:37 +02:00
PgControl pgControl =
{
.version = PG_VERSION_11,
.systemId = 0xEFEFEFEFEF,
Fix formatting errors. Errors in our current (manually-maintained) code format discovered by uncrustify.
2023-01-30 11:16:31 +07:00
.walSegmentSize = 16 * 1024 * 1024,
Add missing ToLog() coverage to String, List, and PgControl. Missing coverage is exposed in the next commit which disables test tracing by default.
2019-02-22 11:31:37 +02:00
.pageChecksum = true
};
Refactor logging functions to never allocate memory. Allocating memory made these functions simpler but it meant that memory was leaking into the calling context when logging was enabled. It is not clear that this was an issue but it seems that trace level logging could result it a lot of memory usage depending on the use case. This also makes it possible to audit allocations returned to the calling context, which will be done in a followup commit. Also rename objToLog() to objNameToLog() since it seemed logical to name the new function objToLog().
2023-01-12 17:14:36 +07:00
TEST_RESULT_VOID(FUNCTION_LOG_OBJECT_FORMAT(&pgControl, pgControlToLog, logBuf, sizeof(logBuf)), "pgControlToLog");
TEST_RESULT_Z(
logBuf, "{version: 110000, systemId: 1030522662895, walSegmentSize: 16777216, pageChecksum: true}", "check log");
Add missing ToLog() coverage to String, List, and PgControl. Missing coverage is exposed in the next commit which disables test tracing by default.
2019-02-22 11:31:37 +02:00
}
Add WAL info to PostgreSQL interface. This allows the WAL header to be read for any supported version on PostgreSQL.
2019-03-19 19:44:06 +04:00
// *****************************************************************************************************************************
if (testBegin("pgWalToLog()"))
{
Refactor logging functions to never allocate memory. Allocating memory made these functions simpler but it meant that memory was leaking into the calling context when logging was enabled. It is not clear that this was an issue but it seems that trace level logging could result it a lot of memory usage depending on the use case. This also makes it possible to audit allocations returned to the calling context, which will be done in a followup commit. Also rename objToLog() to objNameToLog() since it seemed logical to name the new function objToLog().
2023-01-12 17:14:36 +07:00
char logBuf[STACK_TRACE_PARAM_MAX];
Add WAL info to PostgreSQL interface. This allows the WAL header to be read for any supported version on PostgreSQL.
2019-03-19 19:44:06 +04:00
PgWal pgWal =
{
.version = PG_VERSION_10,
.systemId = 0xFEFEFEFEFE
};
Refactor logging functions to never allocate memory. Allocating memory made these functions simpler but it meant that memory was leaking into the calling context when logging was enabled. It is not clear that this was an issue but it seems that trace level logging could result it a lot of memory usage depending on the use case. This also makes it possible to audit allocations returned to the calling context, which will be done in a followup commit. Also rename objToLog() to objNameToLog() since it seemed logical to name the new function objToLog().
2023-01-12 17:14:36 +07:00
TEST_RESULT_VOID(FUNCTION_LOG_OBJECT_FORMAT(&pgWal, pgWalToLog, logBuf, sizeof(logBuf)), "pgWalToLog");
TEST_RESULT_Z(logBuf, "{version: 100000, systemId: 1095199817470}", "check log");
Add WAL info to PostgreSQL interface. This allows the WAL header to be read for any supported version on PostgreSQL.
2019-03-19 19:44:06 +04:00
}
Rename FUNCTION_HARNESS_RESULT*() macros to FUNCTION_HARNESS_RETURN*(). When the FUNCTION_*_RESULT*() macros were renamed to FUNCTION_*_RETURN_*() in the core code the test harness macros were missed. Update them to make the naming consistent.
2021-03-10 18:42:22 -05:00
FUNCTION_HARNESS_RETURN_VOID();
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 10:24:42 +01:00
}
Reference in New Issue Copy Permalink