db/pgbackrest
1
0
Fork 0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2025-06-16 23:47:38 +02:00
Code Issues Releases Activity
Files
e938a89250bd6a6512d4d6b5217a9750c848ca49
pgbackrest/test/src/module/postgres/interfaceTest.c

169 lines
8.9 KiB
C
Raw Normal View History

Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 10:24:42 +01:00
/***********************************************************************************************************************************
Test PostgreSQL Interface
***********************************************************************************************************************************/
#include "storage/driver/posix/storage.h"
/***********************************************************************************************************************************
Test Run
***********************************************************************************************************************************/
void
testRun(void)
{
FUNCTION_HARNESS_VOID();
Storage *storageTest = storageDriverPosixInterface(
storageDriverPosixNew(strNew(testPath()), STORAGE_MODE_FILE_DEFAULT, STORAGE_MODE_PATH_DEFAULT, true, NULL));
// *****************************************************************************************************************************
if (testBegin("pgVersionFromStr() and pgVersionToStr()"))
{
TEST_ERROR(pgVersionFromStr(strNew("9.3.4")), AssertError, "version 9.3.4 format is invalid");
TEST_ERROR(pgVersionFromStr(strNew("abc")), AssertError, "version abc format is invalid");
Rename FUNCTION_DEBUG_* and consolidate ASSERT_* macros for consistency. Rename FUNCTION_DEBUG_* macros to FUNCTION_LOG_* to more accurately reflect what they do. Further rename FUNCTION_DEBUG_RESULT* macros to FUNCTION_LOG_RETURN* to make it clearer that they return from the function as well as logging. Leave FUNCTION_TEST_* macros as they are. Consolidate the various ASSERT* macros into a single ASSERT macro that is always compiled out of production builds. It was difficult to figure out when an assert would be checked with all the different types in play. When ASSERTs are compiled in they will always be checked regardless of the log level -- tying these two concepts together was not a good idea.
2019-01-21 17:41:59 +02:00
TEST_ERROR(pgVersionFromStr(NULL), AssertError, "assertion 'version != NULL' failed");
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 10:24:42 +01:00
TEST_RESULT_INT(pgVersionFromStr(strNew("10")), PG_VERSION_10, "valid pg version 10");
TEST_RESULT_INT(pgVersionFromStr(strNew("9.6")), 90600, "valid pg version 9.6");
//--------------------------------------------------------------------------------------------------------------------------
TEST_RESULT_STR(strPtr(pgVersionToStr(PG_VERSION_11)), "11", "infoPgVersionToString 11");
TEST_RESULT_STR(strPtr(pgVersionToStr(PG_VERSION_96)), "9.6", "infoPgVersionToString 9.6");
TEST_RESULT_STR(strPtr(pgVersionToStr(83456)), "8.34", "infoPgVersionToString 83456");
}
// *****************************************************************************************************************************
if (testBegin("pgControlFromBuffer() and pgControlFromFile()"))
{
String *controlFile = strNew(PG_PATH_GLOBAL "/" PG_FILE_PGCONTROL);
// Create a bogus control file
Buffer *result = bufNew(PG_CONTROL_SIZE);
memset(bufPtr(result), 0, bufSize(result));
bufUsedSet(result, bufSize(result));
((PgControlCommon *)bufPtr(result))->controlVersion = 501;
((PgControlCommon *)bufPtr(result))->catalogVersion = 19780101;
TEST_ERROR(
pgControlFromBuffer(result), VersionNotSupportedError,
"unexpected control version = 501 and catalog version = 19780101\nHINT: is this version of PostgreSQL supported?");
//--------------------------------------------------------------------------------------------------------------------------
TEST_ERROR(pgControlTestToBuffer((PgControl){.version = 0}), AssertError, "invalid version 0");
//--------------------------------------------------------------------------------------------------------------------------
storagePutNP(
storageNewWriteNP(storageTest, controlFile),
pgControlTestToBuffer((PgControl){.version = PG_VERSION_11, .systemId = 0xFACEFACE, .walSegmentSize = 1024 * 1024}));
PgControl info = {0};
TEST_ASSIGN(info, pgControlFromFile(strNew(testPath())), "get control info v11");
TEST_RESULT_INT(info.systemId, 0xFACEFACE, " check system id");
TEST_RESULT_INT(info.controlVersion, 1100, " check control version");
TEST_RESULT_INT(info.catalogVersion, 201809051, " check catalog version");
TEST_RESULT_INT(info.version, PG_VERSION_11, " check version");
//--------------------------------------------------------------------------------------------------------------------------
storagePutNP(
storageNewWriteNP(storageTest, controlFile),
pgControlTestToBuffer((PgControl){.version = PG_VERSION_93, .walSegmentSize = 1024 * 1024}));
TEST_ERROR(
pgControlFromFile(strNew(testPath())), FormatError,
"wal segment size is 1048576 but must be 16777216 for PostgreSQL <= 10");
//--------------------------------------------------------------------------------------------------------------------------
storagePutNP(
storageNewWriteNP(storageTest, controlFile),
pgControlTestToBuffer((PgControl){.version = PG_VERSION_95, .pageSize = 32 * 1024}));
TEST_ERROR(pgControlFromFile(strNew(testPath())), FormatError, "page size is 32768 but must be 8192");
//--------------------------------------------------------------------------------------------------------------------------
storagePutNP(
storageNewWriteNP(storageTest, controlFile),
pgControlTestToBuffer((PgControl){.version = PG_VERSION_83, .systemId = 0xEFEFEFEFEF}));
TEST_ASSIGN(info, pgControlFromFile(strNew(testPath())), "get control info v83");
TEST_RESULT_INT(info.systemId, 0xEFEFEFEFEF, " check system id");
TEST_RESULT_INT(info.controlVersion, 833, " check control version");
TEST_RESULT_INT(info.catalogVersion, 200711281, " check catalog version");
TEST_RESULT_INT(info.version, PG_VERSION_83, " check version");
}
Add WAL info to PostgreSQL interface. This allows the WAL header to be read for any supported version on PostgreSQL.
2019-03-19 19:44:06 +04:00
// *****************************************************************************************************************************
if (testBegin("pgWalFromBuffer() and pgWalFromFile()"))
{
String *walFile = strNewFmt("%s/0000000F0000000F0000000F", testPath());
// Create a bogus control file, initially not in long format)
// --------------------------------------------------------------------------------------------------------------------------
Buffer *result = bufNew((size_t)16 * 1024 * 1024);
memset(bufPtr(result), 0, bufSize(result));
bufUsedSet(result, bufSize(result));
((PgWalCommon *)bufPtr(result))->magic = 777;
TEST_ERROR(pgWalFromBuffer(result), FormatError, "first page header in WAL file is expected to be in long format");
// Add the long flag so that the version will now error
// --------------------------------------------------------------------------------------------------------------------------
((PgWalCommon *)bufPtr(result))->flag = PG_WAL_LONG_HEADER;
TEST_ERROR(
pgWalFromBuffer(result), VersionNotSupportedError,
"unexpected WAL magic 777\n"
"HINT: is this version of PostgreSQL supported?");
//--------------------------------------------------------------------------------------------------------------------------
TEST_ERROR(pgWalTestToBuffer((PgWal){.version = 0}, result), AssertError, "invalid version 0");
//--------------------------------------------------------------------------------------------------------------------------
pgWalTestToBuffer((PgWal){.version = PG_VERSION_11, .systemId = 0xECAFECAF}, result);
storagePutNP(storageNewWriteNP(storageTest, walFile), result);
PgWal info = {0};
TEST_ASSIGN(info, pgWalFromFile(walFile), "get wal info v11");
TEST_RESULT_INT(info.systemId, 0xECAFECAF, " check system id");
TEST_RESULT_INT(info.version, PG_VERSION_11, " check version");
//--------------------------------------------------------------------------------------------------------------------------
pgWalTestToBuffer((PgWal){.version = PG_VERSION_83, .systemId = 0xEAEAEAEA}, result);
storagePutNP(storageNewWriteNP(storageTest, walFile), result);
TEST_ASSIGN(info, pgWalFromFile(walFile), "get wal info v8.3");
TEST_RESULT_INT(info.systemId, 0xEAEAEAEA, " check system id");
TEST_RESULT_INT(info.version, PG_VERSION_83, " check version");
}
Add missing ToLog() coverage to String, List, and PgControl. Missing coverage is exposed in the next commit which disables test tracing by default.
2019-02-22 11:31:37 +02:00
// *****************************************************************************************************************************
if (testBegin("pgControlToLog()"))
{
PgControl pgControl =
{
.version = PG_VERSION_11,
.systemId = 0xEFEFEFEFEF,
.walSegmentSize= 16 * 1024 * 1024,
.pageChecksum = true
};
TEST_RESULT_STR(
strPtr(pgControlToLog(&pgControl)),
"{version: 110000, systemId: 1030522662895, walSegmentSize: 16777216, pageChecksum: true}", "check log");
}
Add WAL info to PostgreSQL interface. This allows the WAL header to be read for any supported version on PostgreSQL.
2019-03-19 19:44:06 +04:00
// *****************************************************************************************************************************
if (testBegin("pgWalToLog()"))
{
PgWal pgWal =
{
.version = PG_VERSION_10,
.systemId = 0xFEFEFEFEFE
};
TEST_RESULT_STR(
strPtr(pgWalToLog(&pgWal)),
"{version: 100000, systemId: 1095199817470}", "check log");
}
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 10:24:42 +01:00
FUNCTION_HARNESS_RESULT_VOID();
}
Reference in New Issue Copy Permalink