db/pgbackrest
1
0
Fork 0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2025-06-18 23:57:33 +02:00
Code Issues Releases Activity
Files
8c712d89ebe10e3d318952a0b4db8cd09f8a580c
pgbackrest/test/src/module/protocol/protocolTest.c

744 lines
37 KiB
C
Raw Normal View History

Add ProtocolClient object and helper functions. This is a complete protocol client implementation in C. Currently there is no C server implementation so the C client is talking to a Perl server. This won't work very long, though, as the protocol format, even though in JSON, has a lot of language-specific structure. While it would be possible to maintain compatibility between C and Perl it's probably not worth the effort in the long run. Just as in Perl there are helper functions to make constructing protocol objects easier. Currently only repository remotes are supported.
2019-01-18 21:32:51 +02:00
/***********************************************************************************************************************************
Test Protocol
***********************************************************************************************************************************/
#include "common/io/handleRead.h"
#include "common/io/handleWrite.h"
Add ProtocolParallel* objects for parallelizing commands. Allows commands to be easily parallelized if the jobs are broken up into discrete, non-overlapping chunks.
2019-02-27 21:10:52 +02:00
#include "common/io/bufferRead.h"
#include "common/io/bufferWrite.h"
Fix reliability of error reporting from local/remote processes. Asserts were only only reported on stderr rather than being returned through the protocol layer. This did not appear to be very reliable. Instead, report the assert through the protocol layer like any other error. Add a stack trace if an assert error or debug logging is enabled.
2019-04-18 10:36:21 -04:00
#include "common/regExp.h"
Migrate remote command to C. Prior to this the Perl remote was used to satisfy C requests. This worked fine but since the remote needed to be migrated to C anyway there was no reason to wait. Add the ProtocolServer object and tweak ProtocolClient to work with it. It was also necessary to add a mechanism to get option values from the remote so that encryption settings could be read and used in the storage object. Update the remote storage objects to comply with the protocol changes and add the storage protocol handler. Ideally this commit would have been broken up into smaller chunks but there are cross-dependencies in the protocol layer and it didn't seem worth the extra effort.
2019-02-19 20:57:38 +02:00
#include "storage/storage.h"
#include "storage/driver/posix/storage.h"
#include "version.h"
Add ProtocolClient object and helper functions. This is a complete protocol client implementation in C. Currently there is no C server implementation so the C client is talking to a Perl server. This won't work very long, though, as the protocol format, even though in JSON, has a lot of language-specific structure. While it would be possible to maintain compatibility between C and Perl it's probably not worth the effort in the long run. Just as in Perl there are helper functions to make constructing protocol objects easier. Currently only repository remotes are supported.
2019-01-18 21:32:51 +02:00
#include "common/harnessConfig.h"
#include "common/harnessFork.h"
Migrate remote command to C. Prior to this the Perl remote was used to satisfy C requests. This worked fine but since the remote needed to be migrated to C anyway there was no reason to wait. Add the ProtocolServer object and tweak ProtocolClient to work with it. It was also necessary to add a mechanism to get option values from the remote so that encryption settings could be read and used in the storage object. Update the remote storage objects to comply with the protocol changes and add the storage protocol handler. Ideally this commit would have been broken up into smaller chunks but there are cross-dependencies in the protocol layer and it didn't seem worth the extra effort.
2019-02-19 20:57:38 +02:00
/***********************************************************************************************************************************
Test protocol request handler
***********************************************************************************************************************************/
bool
testServerProtocol(const String *command, const VariantList *paramList, ProtocolServer *server)
{
FUNCTION_HARNESS_BEGIN();
FUNCTION_HARNESS_PARAM(STRING, command);
FUNCTION_HARNESS_PARAM(VARIANT_LIST, paramList);
FUNCTION_HARNESS_PARAM(PROTOCOL_SERVER, server);
FUNCTION_HARNESS_END();
ASSERT(command != NULL);
// Attempt to satisfy the request -- we may get requests that are meant for other handlers
bool found = true;
MEM_CONTEXT_TEMP_BEGIN()
{
if (strEq(command, strNew("assert")))
{
THROW(AssertError, "test assert");
}
else if (strEq(command, strNew("request-simple")))
{
protocolServerResponse(server, varNewBool(true));
}
else if (strEq(command, strNew("request-complex")))
{
protocolServerResponse(server, varNewBool(false));
Add ioWriteStr() and ioWriteStrLine(). These function names make it clearer what is being written. The old ioWriteLine() has been repurposed to write buffers.
2019-04-22 18:46:29 -04:00
ioWriteStrLine(protocolServerIoWrite(server), strNew("LINEOFTEXT"));
Migrate remote command to C. Prior to this the Perl remote was used to satisfy C requests. This worked fine but since the remote needed to be migrated to C anyway there was no reason to wait. Add the ProtocolServer object and tweak ProtocolClient to work with it. It was also necessary to add a mechanism to get option values from the remote so that encryption settings could be read and used in the storage object. Update the remote storage objects to comply with the protocol changes and add the storage protocol handler. Ideally this commit would have been broken up into smaller chunks but there are cross-dependencies in the protocol layer and it didn't seem worth the extra effort.
2019-02-19 20:57:38 +02:00
ioWriteFlush(protocolServerIoWrite(server));
}
else
found = false;
}
MEM_CONTEXT_TEMP_END();
FUNCTION_HARNESS_RESULT(BOOL, found);
}
Add ProtocolClient object and helper functions. This is a complete protocol client implementation in C. Currently there is no C server implementation so the C client is talking to a Perl server. This won't work very long, though, as the protocol format, even though in JSON, has a lot of language-specific structure. While it would be possible to maintain compatibility between C and Perl it's probably not worth the effort in the long run. Just as in Perl there are helper functions to make constructing protocol objects easier. Currently only repository remotes are supported.
2019-01-18 21:32:51 +02:00
/***********************************************************************************************************************************
Test Run
***********************************************************************************************************************************/
void
testRun(void)
{
FUNCTION_HARNESS_VOID();
Improve type safety of interfaces and drivers. The function pointer casting used when creating drivers made changing interfaces difficult and led to slightly divergent driver implementations. Unit testing caught production-level errors but there were a lot of small issues and the process was harder than it should have been. Use void pointers instead so that no casts are required. Introduce the THIS_VOID and THIS() macros to make dealing with void pointers a little safer. Since we don't want to expose void pointers in header files, driver functions have been removed from the headers and the various driver objects return their interface type. This cuts down on accessor methods and the vast majority of those functions were not being used. Move functions that are still required to .intern.h. Remove the special "C" crypto functions that were used in libc and instead use the standard interface.
2019-05-02 17:52:24 -04:00
Storage *storageTest = storageDriverPosixNew(
strNew(testPath()), STORAGE_MODE_FILE_DEFAULT, STORAGE_MODE_PATH_DEFAULT, true, NULL);
Migrate remote command to C. Prior to this the Perl remote was used to satisfy C requests. This worked fine but since the remote needed to be migrated to C anyway there was no reason to wait. Add the ProtocolServer object and tweak ProtocolClient to work with it. It was also necessary to add a mechanism to get option values from the remote so that encryption settings could be read and used in the storage object. Update the remote storage objects to comply with the protocol changes and add the storage protocol handler. Ideally this commit would have been broken up into smaller chunks but there are cross-dependencies in the protocol layer and it didn't seem worth the extra effort.
2019-02-19 20:57:38 +02:00
Add ProtocolClient object and helper functions. This is a complete protocol client implementation in C. Currently there is no C server implementation so the C client is talking to a Perl server. This won't work very long, though, as the protocol format, even though in JSON, has a lot of language-specific structure. While it would be possible to maintain compatibility between C and Perl it's probably not worth the effort in the long run. Just as in Perl there are helper functions to make constructing protocol objects easier. Currently only repository remotes are supported.
2019-01-18 21:32:51 +02:00
// *****************************************************************************************************************************
if (testBegin("repoIsLocal()"))
{
StringList *argList = strLstNew();
strLstAddZ(argList, "pgbackrest");
strLstAddZ(argList, "--stanza=test1");
strLstAddZ(argList, "archive-get");
harnessCfgLoad(strLstSize(argList), strLstPtr(argList));
TEST_RESULT_BOOL(repoIsLocal(), true, "repo is local");
// -------------------------------------------------------------------------------------------------------------------------
argList = strLstNew();
strLstAddZ(argList, "pgbackrest");
strLstAddZ(argList, "--stanza=test1");
strLstAddZ(argList, "--repo1-host=remote-host");
strLstAddZ(argList, "archive-get");
harnessCfgLoad(strLstSize(argList), strLstPtr(argList));
TEST_RESULT_BOOL(repoIsLocal(), false, "repo is remote");
}
Fix issues with remote/local command logging options. Logging was being enable on local/remote processes even if --log-subprocess was not specified, so fix that. Also, make sure that stderr is enabled at error level as it was on Perl. This helps expose error information for debugging. For remotes, suppress log and lock paths since these are not applicable on remote hosts. These options should be set in the local config if they need to be overridden.
2019-03-16 15:00:02 +04:00
// *****************************************************************************************************************************
if (testBegin("protocolLocalParam()"))
{
StringList *argList = strLstNew();
strLstAddZ(argList, "pgbackrest");
strLstAddZ(argList, "--stanza=test1");
strLstAddZ(argList, "archive-get");
harnessCfgLoad(strLstSize(argList), strLstPtr(argList));
TEST_RESULT_STR(
strPtr(strLstJoin(protocolLocalParam(protocolStorageTypeRepo, 0), "|")),
strPtr(
strNew(
"--command=archive-get|--host-id=1|--log-level-file=off|--log-level-stderr=error|--process=0|--stanza=test1"
"|--type=backup|local")),
"local protocol params");
// -------------------------------------------------------------------------------------------------------------------------
argList = strLstNew();
strLstAddZ(argList, "pgbackrest");
strLstAddZ(argList, "--stanza=test1");
strLstAddZ(argList, "--log-subprocess");
strLstAddZ(argList, "archive-get");
harnessCfgLoad(strLstSize(argList), strLstPtr(argList));
TEST_RESULT_STR(
strPtr(strLstJoin(protocolLocalParam(protocolStorageTypeRepo, 1), "|")),
strPtr(
strNew(
"--command=archive-get|--host-id=1|--log-level-file=info|--log-level-stderr=error|--log-subprocess|--process=1"
"|--stanza=test1|--type=backup|local")),
"local protocol params with replacements");
}
Add ProtocolClient object and helper functions. This is a complete protocol client implementation in C. Currently there is no C server implementation so the C client is talking to a Perl server. This won't work very long, though, as the protocol format, even though in JSON, has a lot of language-specific structure. While it would be possible to maintain compatibility between C and Perl it's probably not worth the effort in the long run. Just as in Perl there are helper functions to make constructing protocol objects easier. Currently only repository remotes are supported.
2019-01-18 21:32:51 +02:00
// *****************************************************************************************************************************
Cleanup local/remote protocol interaction from 9367cc46. The command option was not being set correctly when a remote was started from a local. It was being set as 'local' rather than the command that the local was running as. Also automatically select the remote protocol id based on whether it is started from a local (use the local protocol id) or from the main process (use 0). These were not live issues but could cause strange behaviors as new features are added that might be hard to diagnose.
2019-02-28 09:51:19 +02:00
if (testBegin("protocolRemoteParam()"))
Add ProtocolClient object and helper functions. This is a complete protocol client implementation in C. Currently there is no C server implementation so the C client is talking to a Perl server. This won't work very long, though, as the protocol format, even though in JSON, has a lot of language-specific structure. While it would be possible to maintain compatibility between C and Perl it's probably not worth the effort in the long run. Just as in Perl there are helper functions to make constructing protocol objects easier. Currently only repository remotes are supported.
2019-01-18 21:32:51 +02:00
{
StringList *argList = strLstNew();
strLstAddZ(argList, "pgbackrest");
strLstAddZ(argList, "--stanza=test1");
strLstAddZ(argList, "--repo1-host=repo-host");
strLstAddZ(argList, "--repo1-host-user=repo-host-user");
strLstAddZ(argList, "archive-get");
harnessCfgLoad(strLstSize(argList), strLstPtr(argList));
TEST_RESULT_STR(
Cleanup local/remote protocol interaction from 9367cc46. The command option was not being set correctly when a remote was started from a local. It was being set as 'local' rather than the command that the local was running as. Also automatically select the remote protocol id based on whether it is started from a local (use the local protocol id) or from the main process (use 0). These were not live issues but could cause strange behaviors as new features are added that might be hard to diagnose.
2019-02-28 09:51:19 +02:00
strPtr(strLstJoin(protocolRemoteParam(protocolStorageTypeRepo, 0), "|")),
Add ProtocolClient object and helper functions. This is a complete protocol client implementation in C. Currently there is no C server implementation so the C client is talking to a Perl server. This won't work very long, though, as the protocol format, even though in JSON, has a lot of language-specific structure. While it would be possible to maintain compatibility between C and Perl it's probably not worth the effort in the long run. Just as in Perl there are helper functions to make constructing protocol objects easier. Currently only repository remotes are supported.
2019-01-18 21:32:51 +02:00
strPtr(
strNew(
"-o|LogLevel=error|-o|Compression=no|-o|PasswordAuthentication=no|repo-host-user@repo-host"
Add locking capability to the remote command. When a repository server is configured, commands that modify the repository acquire a remote lock as well as a local lock for extra protection against multiple writers. Instead of the custom logic used in Perl, make remote locking part of the command configuration. This also means that the C remote needs the stanza since it is used to construct the lock name. We may need to revisit this at a later date.
2019-03-27 21:14:06 +00:00
"|pgbackrest --command=archive-get --log-level-file=off --log-level-stderr=error --process=0 --stanza=test1"
" --type=backup remote")),
Add ProtocolClient object and helper functions. This is a complete protocol client implementation in C. Currently there is no C server implementation so the C client is talking to a Perl server. This won't work very long, though, as the protocol format, even though in JSON, has a lot of language-specific structure. While it would be possible to maintain compatibility between C and Perl it's probably not worth the effort in the long run. Just as in Perl there are helper functions to make constructing protocol objects easier. Currently only repository remotes are supported.
2019-01-18 21:32:51 +02:00
"remote protocol params");
// -------------------------------------------------------------------------------------------------------------------------
argList = strLstNew();
strLstAddZ(argList, "pgbackrest");
strLstAddZ(argList, "--stanza=test1");
Fix issues with remote/local command logging options. Logging was being enable on local/remote processes even if --log-subprocess was not specified, so fix that. Also, make sure that stderr is enabled at error level as it was on Perl. This helps expose error information for debugging. For remotes, suppress log and lock paths since these are not applicable on remote hosts. These options should be set in the local config if they need to be overridden.
2019-03-16 15:00:02 +04:00
strLstAddZ(argList, "--log-subprocess");
Add ProtocolClient object and helper functions. This is a complete protocol client implementation in C. Currently there is no C server implementation so the C client is talking to a Perl server. This won't work very long, though, as the protocol format, even though in JSON, has a lot of language-specific structure. While it would be possible to maintain compatibility between C and Perl it's probably not worth the effort in the long run. Just as in Perl there are helper functions to make constructing protocol objects easier. Currently only repository remotes are supported.
2019-01-18 21:32:51 +02:00
strLstAddZ(argList, "--repo1-host=repo-host");
strLstAddZ(argList, "--repo1-host-port=444");
strLstAddZ(argList, "--repo1-host-config=/path/pgbackrest.conf");
strLstAddZ(argList, "--repo1-host-config-include-path=/path/include");
strLstAddZ(argList, "--repo1-host-config-path=/path/config");
strLstAddZ(argList, "--repo1-host-user=repo-host-user");
strLstAddZ(argList, "archive-get");
harnessCfgLoad(strLstSize(argList), strLstPtr(argList));
TEST_RESULT_STR(
Migrate local command to C. The C local is only used for C commands in the main process. Some tweaking of the existing protocolGet() command was required. Originally the idea was to share the function for local and remote requests but the differences (as in Perl) were too great to make that practical.
2019-02-27 22:34:21 +02:00
strPtr(strLstJoin(protocolRemoteParam(protocolStorageTypeRepo, 1), "|")),
Add ProtocolClient object and helper functions. This is a complete protocol client implementation in C. Currently there is no C server implementation so the C client is talking to a Perl server. This won't work very long, though, as the protocol format, even though in JSON, has a lot of language-specific structure. While it would be possible to maintain compatibility between C and Perl it's probably not worth the effort in the long run. Just as in Perl there are helper functions to make constructing protocol objects easier. Currently only repository remotes are supported.
2019-01-18 21:32:51 +02:00
strPtr(
strNew(
"-o|LogLevel=error|-o|Compression=no|-o|PasswordAuthentication=no|-p|444|repo-host-user@repo-host"
"|pgbackrest --command=archive-get --config=/path/pgbackrest.conf --config-include-path=/path/include"
Fix issues with remote/local command logging options. Logging was being enable on local/remote processes even if --log-subprocess was not specified, so fix that. Also, make sure that stderr is enabled at error level as it was on Perl. This helps expose error information for debugging. For remotes, suppress log and lock paths since these are not applicable on remote hosts. These options should be set in the local config if they need to be overridden.
2019-03-16 15:00:02 +04:00
" --config-path=/path/config --log-level-file=info --log-level-stderr=error --log-subprocess --process=1"
Add locking capability to the remote command. When a repository server is configured, commands that modify the repository acquire a remote lock as well as a local lock for extra protection against multiple writers. Instead of the custom logic used in Perl, make remote locking part of the command configuration. This also means that the C remote needs the stanza since it is used to construct the lock name. We may need to revisit this at a later date.
2019-03-27 21:14:06 +00:00
" --stanza=test1 --type=backup remote")),
Add ProtocolClient object and helper functions. This is a complete protocol client implementation in C. Currently there is no C server implementation so the C client is talking to a Perl server. This won't work very long, though, as the protocol format, even though in JSON, has a lot of language-specific structure. While it would be possible to maintain compatibility between C and Perl it's probably not worth the effort in the long run. Just as in Perl there are helper functions to make constructing protocol objects easier. Currently only repository remotes are supported.
2019-01-18 21:32:51 +02:00
"remote protocol params with replacements");
Cleanup local/remote protocol interaction from 9367cc46. The command option was not being set correctly when a remote was started from a local. It was being set as 'local' rather than the command that the local was running as. Also automatically select the remote protocol id based on whether it is started from a local (use the local protocol id) or from the main process (use 0). These were not live issues but could cause strange behaviors as new features are added that might be hard to diagnose.
2019-02-28 09:51:19 +02:00
// -------------------------------------------------------------------------------------------------------------------------
argList = strLstNew();
strLstAddZ(argList, "pgbackrest");
strLstAddZ(argList, "--stanza=test1");
strLstAddZ(argList, "--command=archive-get");
strLstAddZ(argList, "--process=3");
strLstAddZ(argList, "--host-id=1");
strLstAddZ(argList, "--type=db");
strLstAddZ(argList, "--repo1-host=repo-host");
strLstAddZ(argList, "local");
harnessCfgLoad(strLstSize(argList), strLstPtr(argList));
TEST_RESULT_STR(
strPtr(strLstJoin(protocolRemoteParam(protocolStorageTypeRepo, 66), "|")),
strPtr(
strNew(
"-o|LogLevel=error|-o|Compression=no|-o|PasswordAuthentication=no|pgbackrest@repo-host"
Add locking capability to the remote command. When a repository server is configured, commands that modify the repository acquire a remote lock as well as a local lock for extra protection against multiple writers. Instead of the custom logic used in Perl, make remote locking part of the command configuration. This also means that the C remote needs the stanza since it is used to construct the lock name. We may need to revisit this at a later date.
2019-03-27 21:14:06 +00:00
"|pgbackrest --command=archive-get --log-level-file=off --log-level-stderr=error --process=3 --stanza=test1"
" --type=backup remote")),
Cleanup local/remote protocol interaction from 9367cc46. The command option was not being set correctly when a remote was started from a local. It was being set as 'local' rather than the command that the local was running as. Also automatically select the remote protocol id based on whether it is started from a local (use the local protocol id) or from the main process (use 0). These were not live issues but could cause strange behaviors as new features are added that might be hard to diagnose.
2019-02-28 09:51:19 +02:00
"remote protocol params for local");
Add ProtocolClient object and helper functions. This is a complete protocol client implementation in C. Currently there is no C server implementation so the C client is talking to a Perl server. This won't work very long, though, as the protocol format, even though in JSON, has a lot of language-specific structure. While it would be possible to maintain compatibility between C and Perl it's probably not worth the effort in the long run. Just as in Perl there are helper functions to make constructing protocol objects easier. Currently only repository remotes are supported.
2019-01-18 21:32:51 +02:00
}
Add ProtocolCommand object. This formalizes the creation of protocol commands, which was previously done by creating KeyValue objects manually.
2019-02-27 19:48:30 +02:00
// *****************************************************************************************************************************
if (testBegin("ProtocolCommand"))
{
ProtocolCommand *command = NULL;
MEM_CONTEXT_TEMP_BEGIN()
{
TEST_ASSIGN(command, protocolCommandNew(strNew("command1")), "create command");
TEST_RESULT_PTR(protocolCommandParamAdd(command, varNewStr(strNew("param1"))), command, "add param");
TEST_RESULT_PTR(protocolCommandParamAdd(command, varNewStr(strNew("param2"))), command, "add param");
TEST_RESULT_PTR(protocolCommandMove(command, MEM_CONTEXT_OLD()), command, "move protocol command");
TEST_RESULT_PTR(protocolCommandMove(NULL, MEM_CONTEXT_OLD()), NULL, "move null protocol command");
}
MEM_CONTEXT_TEMP_END();
TEST_RESULT_STR(strPtr(protocolCommandToLog(command)), "{command: command1}", "check log");
TEST_RESULT_STR(
strPtr(protocolCommandJson(command)), "{\"cmd\":\"command1\",\"param\":[\"param1\",\"param2\"]}", "check json");
// -------------------------------------------------------------------------------------------------------------------------
TEST_ASSIGN(command, protocolCommandNew(strNew("command2")), "create command");
TEST_RESULT_STR(strPtr(protocolCommandToLog(command)), "{command: command2}", "check log");
TEST_RESULT_STR(strPtr(protocolCommandJson(command)), "{\"cmd\":\"command2\"}", "check json");
// -------------------------------------------------------------------------------------------------------------------------
TEST_RESULT_VOID(protocolCommandFree(command), "free command");
TEST_RESULT_VOID(protocolCommandFree(NULL), "free null command");
}
Add ProtocolClient object and helper functions. This is a complete protocol client implementation in C. Currently there is no C server implementation so the C client is talking to a Perl server. This won't work very long, though, as the protocol format, even though in JSON, has a lot of language-specific structure. While it would be possible to maintain compatibility between C and Perl it's probably not worth the effort in the long run. Just as in Perl there are helper functions to make constructing protocol objects easier. Currently only repository remotes are supported.
2019-01-18 21:32:51 +02:00
// *****************************************************************************************************************************
if (testBegin("ProtocolClient"))
{
HARNESS_FORK_BEGIN()
{
Improve fork harness to allow multiple children and setup pipes automatically. There was a lot of extra boilerplate involved in setting up pipes so that is now automated. In some cases testing with multiple children is useful so allow that as well.
2019-02-27 18:07:16 +02:00
HARNESS_FORK_CHILD_BEGIN(0, true)
Add ProtocolClient object and helper functions. This is a complete protocol client implementation in C. Currently there is no C server implementation so the C client is talking to a Perl server. This won't work very long, though, as the protocol format, even though in JSON, has a lot of language-specific structure. While it would be possible to maintain compatibility between C and Perl it's probably not worth the effort in the long run. Just as in Perl there are helper functions to make constructing protocol objects easier. Currently only repository remotes are supported.
2019-01-18 21:32:51 +02:00
{
Improve type safety of interfaces and drivers. The function pointer casting used when creating drivers made changing interfaces difficult and led to slightly divergent driver implementations. Unit testing caught production-level errors but there were a lot of small issues and the process was harder than it should have been. Use void pointers instead so that no casts are required. Introduce the THIS_VOID and THIS() macros to make dealing with void pointers a little safer. Since we don't want to expose void pointers in header files, driver functions have been removed from the headers and the various driver objects return their interface type. This cuts down on accessor methods and the vast majority of those functions were not being used. Move functions that are still required to .intern.h. Remove the special "C" crypto functions that were used in libc and instead use the standard interface.
2019-05-02 17:52:24 -04:00
IoRead *read = ioHandleReadNew(strNew("server read"), HARNESS_FORK_CHILD_READ(), 2000);
Add ProtocolClient object and helper functions. This is a complete protocol client implementation in C. Currently there is no C server implementation so the C client is talking to a Perl server. This won't work very long, though, as the protocol format, even though in JSON, has a lot of language-specific structure. While it would be possible to maintain compatibility between C and Perl it's probably not worth the effort in the long run. Just as in Perl there are helper functions to make constructing protocol objects easier. Currently only repository remotes are supported.
2019-01-18 21:32:51 +02:00
ioReadOpen(read);
Improve type safety of interfaces and drivers. The function pointer casting used when creating drivers made changing interfaces difficult and led to slightly divergent driver implementations. Unit testing caught production-level errors but there were a lot of small issues and the process was harder than it should have been. Use void pointers instead so that no casts are required. Introduce the THIS_VOID and THIS() macros to make dealing with void pointers a little safer. Since we don't want to expose void pointers in header files, driver functions have been removed from the headers and the various driver objects return their interface type. This cuts down on accessor methods and the vast majority of those functions were not being used. Move functions that are still required to .intern.h. Remove the special "C" crypto functions that were used in libc and instead use the standard interface.
2019-05-02 17:52:24 -04:00
IoWrite *write = ioHandleWriteNew(strNew("server write"), HARNESS_FORK_CHILD_WRITE());
Add ProtocolClient object and helper functions. This is a complete protocol client implementation in C. Currently there is no C server implementation so the C client is talking to a Perl server. This won't work very long, though, as the protocol format, even though in JSON, has a lot of language-specific structure. While it would be possible to maintain compatibility between C and Perl it's probably not worth the effort in the long run. Just as in Perl there are helper functions to make constructing protocol objects easier. Currently only repository remotes are supported.
2019-01-18 21:32:51 +02:00
ioWriteOpen(write);
// Various bogus greetings
Add ioWriteStr() and ioWriteStrLine(). These function names make it clearer what is being written. The old ioWriteLine() has been repurposed to write buffers.
2019-04-22 18:46:29 -04:00
ioWriteStrLine(write, strNew("bogus greeting"));
Add ProtocolClient object and helper functions. This is a complete protocol client implementation in C. Currently there is no C server implementation so the C client is talking to a Perl server. This won't work very long, though, as the protocol format, even though in JSON, has a lot of language-specific structure. While it would be possible to maintain compatibility between C and Perl it's probably not worth the effort in the long run. Just as in Perl there are helper functions to make constructing protocol objects easier. Currently only repository remotes are supported.
2019-01-18 21:32:51 +02:00
ioWriteFlush(write);
Add ioWriteStr() and ioWriteStrLine(). These function names make it clearer what is being written. The old ioWriteLine() has been repurposed to write buffers.
2019-04-22 18:46:29 -04:00
ioWriteStrLine(write, strNew("{\"name\":999}"));
Add ProtocolClient object and helper functions. This is a complete protocol client implementation in C. Currently there is no C server implementation so the C client is talking to a Perl server. This won't work very long, though, as the protocol format, even though in JSON, has a lot of language-specific structure. While it would be possible to maintain compatibility between C and Perl it's probably not worth the effort in the long run. Just as in Perl there are helper functions to make constructing protocol objects easier. Currently only repository remotes are supported.
2019-01-18 21:32:51 +02:00
ioWriteFlush(write);
Add ioWriteStr() and ioWriteStrLine(). These function names make it clearer what is being written. The old ioWriteLine() has been repurposed to write buffers.
2019-04-22 18:46:29 -04:00
ioWriteStrLine(write, strNew("{\"name\":null}"));
Add ProtocolClient object and helper functions. This is a complete protocol client implementation in C. Currently there is no C server implementation so the C client is talking to a Perl server. This won't work very long, though, as the protocol format, even though in JSON, has a lot of language-specific structure. While it would be possible to maintain compatibility between C and Perl it's probably not worth the effort in the long run. Just as in Perl there are helper functions to make constructing protocol objects easier. Currently only repository remotes are supported.
2019-01-18 21:32:51 +02:00
ioWriteFlush(write);
Add ioWriteStr() and ioWriteStrLine(). These function names make it clearer what is being written. The old ioWriteLine() has been repurposed to write buffers.
2019-04-22 18:46:29 -04:00
ioWriteStrLine(write, strNew("{\"name\":\"bogus\"}"));
Add ProtocolClient object and helper functions. This is a complete protocol client implementation in C. Currently there is no C server implementation so the C client is talking to a Perl server. This won't work very long, though, as the protocol format, even though in JSON, has a lot of language-specific structure. While it would be possible to maintain compatibility between C and Perl it's probably not worth the effort in the long run. Just as in Perl there are helper functions to make constructing protocol objects easier. Currently only repository remotes are supported.
2019-01-18 21:32:51 +02:00
ioWriteFlush(write);
Add ioWriteStr() and ioWriteStrLine(). These function names make it clearer what is being written. The old ioWriteLine() has been repurposed to write buffers.
2019-04-22 18:46:29 -04:00
ioWriteStrLine(write, strNew("{\"name\":\"pgBackRest\",\"service\":\"bogus\"}"));
Add ProtocolClient object and helper functions. This is a complete protocol client implementation in C. Currently there is no C server implementation so the C client is talking to a Perl server. This won't work very long, though, as the protocol format, even though in JSON, has a lot of language-specific structure. While it would be possible to maintain compatibility between C and Perl it's probably not worth the effort in the long run. Just as in Perl there are helper functions to make constructing protocol objects easier. Currently only repository remotes are supported.
2019-01-18 21:32:51 +02:00
ioWriteFlush(write);
Add ioWriteStr() and ioWriteStrLine(). These function names make it clearer what is being written. The old ioWriteLine() has been repurposed to write buffers.
2019-04-22 18:46:29 -04:00
ioWriteStrLine(write, strNew("{\"name\":\"pgBackRest\",\"service\":\"test\",\"version\":\"bogus\"}"));
Add ProtocolClient object and helper functions. This is a complete protocol client implementation in C. Currently there is no C server implementation so the C client is talking to a Perl server. This won't work very long, though, as the protocol format, even though in JSON, has a lot of language-specific structure. While it would be possible to maintain compatibility between C and Perl it's probably not worth the effort in the long run. Just as in Perl there are helper functions to make constructing protocol objects easier. Currently only repository remotes are supported.
2019-01-18 21:32:51 +02:00
ioWriteFlush(write);
// Correct greeting with noop
Add ioWriteStr() and ioWriteStrLine(). These function names make it clearer what is being written. The old ioWriteLine() has been repurposed to write buffers.
2019-04-22 18:46:29 -04:00
ioWriteStrLine(write, strNew("{\"name\":\"pgBackRest\",\"service\":\"test\",\"version\":\"" PROJECT_VERSION "\"}"));
Add ProtocolClient object and helper functions. This is a complete protocol client implementation in C. Currently there is no C server implementation so the C client is talking to a Perl server. This won't work very long, though, as the protocol format, even though in JSON, has a lot of language-specific structure. While it would be possible to maintain compatibility between C and Perl it's probably not worth the effort in the long run. Just as in Perl there are helper functions to make constructing protocol objects easier. Currently only repository remotes are supported.
2019-01-18 21:32:51 +02:00
ioWriteFlush(write);
TEST_RESULT_STR(strPtr(ioReadLine(read)), "{\"cmd\":\"noop\"}", "noop");
Add ioWriteStr() and ioWriteStrLine(). These function names make it clearer what is being written. The old ioWriteLine() has been repurposed to write buffers.
2019-04-22 18:46:29 -04:00
ioWriteStrLine(write, strNew("{}"));
Add ProtocolClient object and helper functions. This is a complete protocol client implementation in C. Currently there is no C server implementation so the C client is talking to a Perl server. This won't work very long, though, as the protocol format, even though in JSON, has a lot of language-specific structure. While it would be possible to maintain compatibility between C and Perl it's probably not worth the effort in the long run. Just as in Perl there are helper functions to make constructing protocol objects easier. Currently only repository remotes are supported.
2019-01-18 21:32:51 +02:00
ioWriteFlush(write);
// Throw errors
TEST_RESULT_STR(strPtr(ioReadLine(read)), "{\"cmd\":\"noop\"}", "noop with error text");
Add ioWriteStr() and ioWriteStrLine(). These function names make it clearer what is being written. The old ioWriteLine() has been repurposed to write buffers.
2019-04-22 18:46:29 -04:00
ioWriteStrLine(write, strNew("{\"err\":25,\"out\":\"sample error message\"}"));
Add ProtocolClient object and helper functions. This is a complete protocol client implementation in C. Currently there is no C server implementation so the C client is talking to a Perl server. This won't work very long, though, as the protocol format, even though in JSON, has a lot of language-specific structure. While it would be possible to maintain compatibility between C and Perl it's probably not worth the effort in the long run. Just as in Perl there are helper functions to make constructing protocol objects easier. Currently only repository remotes are supported.
2019-01-18 21:32:51 +02:00
ioWriteFlush(write);
TEST_RESULT_STR(strPtr(ioReadLine(read)), "{\"cmd\":\"noop\"}", "noop with no error text");
Add ioWriteStr() and ioWriteStrLine(). These function names make it clearer what is being written. The old ioWriteLine() has been repurposed to write buffers.
2019-04-22 18:46:29 -04:00
ioWriteStrLine(write, strNew("{\"err\":255}"));
Add ProtocolClient object and helper functions. This is a complete protocol client implementation in C. Currently there is no C server implementation so the C client is talking to a Perl server. This won't work very long, though, as the protocol format, even though in JSON, has a lot of language-specific structure. While it would be possible to maintain compatibility between C and Perl it's probably not worth the effort in the long run. Just as in Perl there are helper functions to make constructing protocol objects easier. Currently only repository remotes are supported.
2019-01-18 21:32:51 +02:00
ioWriteFlush(write);
// No output expected
TEST_RESULT_STR(strPtr(ioReadLine(read)), "{\"cmd\":\"noop\"}", "noop with parameters returned");
Add ioWriteStr() and ioWriteStrLine(). These function names make it clearer what is being written. The old ioWriteLine() has been repurposed to write buffers.
2019-04-22 18:46:29 -04:00
ioWriteStrLine(write, strNew("{\"out\":[\"bogus\"]}"));
Add ProtocolClient object and helper functions. This is a complete protocol client implementation in C. Currently there is no C server implementation so the C client is talking to a Perl server. This won't work very long, though, as the protocol format, even though in JSON, has a lot of language-specific structure. While it would be possible to maintain compatibility between C and Perl it's probably not worth the effort in the long run. Just as in Perl there are helper functions to make constructing protocol objects easier. Currently only repository remotes are supported.
2019-01-18 21:32:51 +02:00
ioWriteFlush(write);
// Send output
TEST_RESULT_STR(strPtr(ioReadLine(read)), "{\"cmd\":\"test\"}", "test command");
Add ioWriteStr() and ioWriteStrLine(). These function names make it clearer what is being written. The old ioWriteLine() has been repurposed to write buffers.
2019-04-22 18:46:29 -04:00
ioWriteStrLine(write, strNew("{\"out\":[\"value1\",\"value2\"]}"));
Add ProtocolClient object and helper functions. This is a complete protocol client implementation in C. Currently there is no C server implementation so the C client is talking to a Perl server. This won't work very long, though, as the protocol format, even though in JSON, has a lot of language-specific structure. While it would be possible to maintain compatibility between C and Perl it's probably not worth the effort in the long run. Just as in Perl there are helper functions to make constructing protocol objects easier. Currently only repository remotes are supported.
2019-01-18 21:32:51 +02:00
ioWriteFlush(write);
// Wait for exit
TEST_RESULT_STR(strPtr(ioReadLine(read)), "{\"cmd\":\"exit\"}", "exit command");
}
Improve fork harness to allow multiple children and setup pipes automatically. There was a lot of extra boilerplate involved in setting up pipes so that is now automated. In some cases testing with multiple children is useful so allow that as well.
2019-02-27 18:07:16 +02:00
HARNESS_FORK_CHILD_END();
Add ProtocolClient object and helper functions. This is a complete protocol client implementation in C. Currently there is no C server implementation so the C client is talking to a Perl server. This won't work very long, though, as the protocol format, even though in JSON, has a lot of language-specific structure. While it would be possible to maintain compatibility between C and Perl it's probably not worth the effort in the long run. Just as in Perl there are helper functions to make constructing protocol objects easier. Currently only repository remotes are supported.
2019-01-18 21:32:51 +02:00
Improve fork harness to allow multiple children and setup pipes automatically. There was a lot of extra boilerplate involved in setting up pipes so that is now automated. In some cases testing with multiple children is useful so allow that as well.
2019-02-27 18:07:16 +02:00
HARNESS_FORK_PARENT_BEGIN()
Add ProtocolClient object and helper functions. This is a complete protocol client implementation in C. Currently there is no C server implementation so the C client is talking to a Perl server. This won't work very long, though, as the protocol format, even though in JSON, has a lot of language-specific structure. While it would be possible to maintain compatibility between C and Perl it's probably not worth the effort in the long run. Just as in Perl there are helper functions to make constructing protocol objects easier. Currently only repository remotes are supported.
2019-01-18 21:32:51 +02:00
{
Improve type safety of interfaces and drivers. The function pointer casting used when creating drivers made changing interfaces difficult and led to slightly divergent driver implementations. Unit testing caught production-level errors but there were a lot of small issues and the process was harder than it should have been. Use void pointers instead so that no casts are required. Introduce the THIS_VOID and THIS() macros to make dealing with void pointers a little safer. Since we don't want to expose void pointers in header files, driver functions have been removed from the headers and the various driver objects return their interface type. This cuts down on accessor methods and the vast majority of those functions were not being used. Move functions that are still required to .intern.h. Remove the special "C" crypto functions that were used in libc and instead use the standard interface.
2019-05-02 17:52:24 -04:00
IoRead *read = ioHandleReadNew(strNew("client read"), HARNESS_FORK_PARENT_READ_PROCESS(0), 2000);
Add ProtocolClient object and helper functions. This is a complete protocol client implementation in C. Currently there is no C server implementation so the C client is talking to a Perl server. This won't work very long, though, as the protocol format, even though in JSON, has a lot of language-specific structure. While it would be possible to maintain compatibility between C and Perl it's probably not worth the effort in the long run. Just as in Perl there are helper functions to make constructing protocol objects easier. Currently only repository remotes are supported.
2019-01-18 21:32:51 +02:00
ioReadOpen(read);
Improve type safety of interfaces and drivers. The function pointer casting used when creating drivers made changing interfaces difficult and led to slightly divergent driver implementations. Unit testing caught production-level errors but there were a lot of small issues and the process was harder than it should have been. Use void pointers instead so that no casts are required. Introduce the THIS_VOID and THIS() macros to make dealing with void pointers a little safer. Since we don't want to expose void pointers in header files, driver functions have been removed from the headers and the various driver objects return their interface type. This cuts down on accessor methods and the vast majority of those functions were not being used. Move functions that are still required to .intern.h. Remove the special "C" crypto functions that were used in libc and instead use the standard interface.
2019-05-02 17:52:24 -04:00
IoWrite *write = ioHandleWriteNew(strNew("client write"), HARNESS_FORK_PARENT_WRITE_PROCESS(0));
Add ProtocolClient object and helper functions. This is a complete protocol client implementation in C. Currently there is no C server implementation so the C client is talking to a Perl server. This won't work very long, though, as the protocol format, even though in JSON, has a lot of language-specific structure. While it would be possible to maintain compatibility between C and Perl it's probably not worth the effort in the long run. Just as in Perl there are helper functions to make constructing protocol objects easier. Currently only repository remotes are supported.
2019-01-18 21:32:51 +02:00
ioWriteOpen(write);
// Various bogus greetings
TEST_ERROR(
protocolClientNew(strNew("test client"), strNew("test"), read, write), JsonFormatError,
Add separate functions to encode/decode each JSON type. In most cases the JSON type is known so this is more efficient than converting to Variant first, both in terms of memory and time. Also rename some of the existing functions for consistency.
2019-04-22 18:41:01 -04:00
"expected '{' at 'bogus greeting'");
Add ProtocolClient object and helper functions. This is a complete protocol client implementation in C. Currently there is no C server implementation so the C client is talking to a Perl server. This won't work very long, though, as the protocol format, even though in JSON, has a lot of language-specific structure. While it would be possible to maintain compatibility between C and Perl it's probably not worth the effort in the long run. Just as in Perl there are helper functions to make constructing protocol objects easier. Currently only repository remotes are supported.
2019-01-18 21:32:51 +02:00
TEST_ERROR(
protocolClientNew(strNew("test client"), strNew("test"), read, write), ProtocolError,
"greeting key 'name' must be string type");
TEST_ERROR(
protocolClientNew(strNew("test client"), strNew("test"), read, write), ProtocolError,
"unable to find greeting key 'name'");
TEST_ERROR(
protocolClientNew(strNew("test client"), strNew("test"), read, write), ProtocolError,
"expected value 'pgBackRest' for greeting key 'name' but got 'bogus'");
TEST_ERROR(
protocolClientNew(strNew("test client"), strNew("test"), read, write), ProtocolError,
"expected value 'test' for greeting key 'service' but got 'bogus'");
TEST_ERROR(
protocolClientNew(strNew("test client"), strNew("test"), read, write), ProtocolError,
"expected value '" PROJECT_VERSION "' for greeting key 'version' but got 'bogus'");
// Correct greeting
ProtocolClient *client = NULL;
MEM_CONTEXT_TEMP_BEGIN()
{
TEST_ASSIGN(
client,
protocolClientMove(
protocolClientNew(strNew("test client"), strNew("test"), read, write), MEM_CONTEXT_OLD()),
"create client");
TEST_RESULT_VOID(protocolClientMove(NULL, MEM_CONTEXT_OLD()), "move null client");
}
MEM_CONTEXT_TEMP_END();
TEST_RESULT_PTR(protocolClientIoRead(client), client->read, "get read io");
TEST_RESULT_PTR(protocolClientIoWrite(client), client->write, "get write io");
// Throw errors
Fix reliability of error reporting from local/remote processes. Asserts were only only reported on stderr rather than being returned through the protocol layer. This did not appear to be very reliable. Instead, report the assert through the protocol layer like any other error. Add a stack trace if an assert error or debug logging is enabled.
2019-04-18 10:36:21 -04:00
TEST_ERROR(
protocolClientNoOp(client), AssertError,
"raised from test client: sample error message\nno stack trace available");
Add ProtocolClient object and helper functions. This is a complete protocol client implementation in C. Currently there is no C server implementation so the C client is talking to a Perl server. This won't work very long, though, as the protocol format, even though in JSON, has a lot of language-specific structure. While it would be possible to maintain compatibility between C and Perl it's probably not worth the effort in the long run. Just as in Perl there are helper functions to make constructing protocol objects easier. Currently only repository remotes are supported.
2019-01-18 21:32:51 +02:00
TEST_ERROR(protocolClientNoOp(client), UnknownError, "raised from test client: no details available");
TEST_ERROR(protocolClientNoOp(client), AssertError, "no output required by command");
// Get command output
const VariantList *output = NULL;
Add ProtocolCommand object. This formalizes the creation of protocol commands, which was previously done by creating KeyValue objects manually.
2019-02-27 19:48:30 +02:00
TEST_ASSIGN(
output,
varVarLst(protocolClientExecute(client, protocolCommandNew(strNew("test")), true)),
"execute command with output");
Add ProtocolClient object and helper functions. This is a complete protocol client implementation in C. Currently there is no C server implementation so the C client is talking to a Perl server. This won't work very long, though, as the protocol format, even though in JSON, has a lot of language-specific structure. While it would be possible to maintain compatibility between C and Perl it's probably not worth the effort in the long run. Just as in Perl there are helper functions to make constructing protocol objects easier. Currently only repository remotes are supported.
2019-01-18 21:32:51 +02:00
TEST_RESULT_UINT(varLstSize(output), 2, "check output size");
TEST_RESULT_STR(strPtr(varStr(varLstGet(output, 0))), "value1", "check value1");
TEST_RESULT_STR(strPtr(varStr(varLstGet(output, 1))), "value2", "check value2");
// Free client
TEST_RESULT_VOID(protocolClientFree(client), "free client");
TEST_RESULT_VOID(protocolClientFree(NULL), "free null client");
}
Improve fork harness to allow multiple children and setup pipes automatically. There was a lot of extra boilerplate involved in setting up pipes so that is now automated. In some cases testing with multiple children is useful so allow that as well.
2019-02-27 18:07:16 +02:00
HARNESS_FORK_PARENT_END();
Add ProtocolClient object and helper functions. This is a complete protocol client implementation in C. Currently there is no C server implementation so the C client is talking to a Perl server. This won't work very long, though, as the protocol format, even though in JSON, has a lot of language-specific structure. While it would be possible to maintain compatibility between C and Perl it's probably not worth the effort in the long run. Just as in Perl there are helper functions to make constructing protocol objects easier. Currently only repository remotes are supported.
2019-01-18 21:32:51 +02:00
}
HARNESS_FORK_END();
}
Migrate remote command to C. Prior to this the Perl remote was used to satisfy C requests. This worked fine but since the remote needed to be migrated to C anyway there was no reason to wait. Add the ProtocolServer object and tweak ProtocolClient to work with it. It was also necessary to add a mechanism to get option values from the remote so that encryption settings could be read and used in the storage object. Update the remote storage objects to comply with the protocol changes and add the storage protocol handler. Ideally this commit would have been broken up into smaller chunks but there are cross-dependencies in the protocol layer and it didn't seem worth the extra effort.
2019-02-19 20:57:38 +02:00
// *****************************************************************************************************************************
if (testBegin("ProtocolServer"))
{
HARNESS_FORK_BEGIN()
{
Improve fork harness to allow multiple children and setup pipes automatically. There was a lot of extra boilerplate involved in setting up pipes so that is now automated. In some cases testing with multiple children is useful so allow that as well.
2019-02-27 18:07:16 +02:00
HARNESS_FORK_CHILD_BEGIN(0, true)
Migrate remote command to C. Prior to this the Perl remote was used to satisfy C requests. This worked fine but since the remote needed to be migrated to C anyway there was no reason to wait. Add the ProtocolServer object and tweak ProtocolClient to work with it. It was also necessary to add a mechanism to get option values from the remote so that encryption settings could be read and used in the storage object. Update the remote storage objects to comply with the protocol changes and add the storage protocol handler. Ideally this commit would have been broken up into smaller chunks but there are cross-dependencies in the protocol layer and it didn't seem worth the extra effort.
2019-02-19 20:57:38 +02:00
{
Improve type safety of interfaces and drivers. The function pointer casting used when creating drivers made changing interfaces difficult and led to slightly divergent driver implementations. Unit testing caught production-level errors but there were a lot of small issues and the process was harder than it should have been. Use void pointers instead so that no casts are required. Introduce the THIS_VOID and THIS() macros to make dealing with void pointers a little safer. Since we don't want to expose void pointers in header files, driver functions have been removed from the headers and the various driver objects return their interface type. This cuts down on accessor methods and the vast majority of those functions were not being used. Move functions that are still required to .intern.h. Remove the special "C" crypto functions that were used in libc and instead use the standard interface.
2019-05-02 17:52:24 -04:00
IoRead *read = ioHandleReadNew(strNew("client read"), HARNESS_FORK_CHILD_READ(), 2000);
Migrate remote command to C. Prior to this the Perl remote was used to satisfy C requests. This worked fine but since the remote needed to be migrated to C anyway there was no reason to wait. Add the ProtocolServer object and tweak ProtocolClient to work with it. It was also necessary to add a mechanism to get option values from the remote so that encryption settings could be read and used in the storage object. Update the remote storage objects to comply with the protocol changes and add the storage protocol handler. Ideally this commit would have been broken up into smaller chunks but there are cross-dependencies in the protocol layer and it didn't seem worth the extra effort.
2019-02-19 20:57:38 +02:00
ioReadOpen(read);
Improve type safety of interfaces and drivers. The function pointer casting used when creating drivers made changing interfaces difficult and led to slightly divergent driver implementations. Unit testing caught production-level errors but there were a lot of small issues and the process was harder than it should have been. Use void pointers instead so that no casts are required. Introduce the THIS_VOID and THIS() macros to make dealing with void pointers a little safer. Since we don't want to expose void pointers in header files, driver functions have been removed from the headers and the various driver objects return their interface type. This cuts down on accessor methods and the vast majority of those functions were not being used. Move functions that are still required to .intern.h. Remove the special "C" crypto functions that were used in libc and instead use the standard interface.
2019-05-02 17:52:24 -04:00
IoWrite *write = ioHandleWriteNew(strNew("client write"), HARNESS_FORK_CHILD_WRITE());
Migrate remote command to C. Prior to this the Perl remote was used to satisfy C requests. This worked fine but since the remote needed to be migrated to C anyway there was no reason to wait. Add the ProtocolServer object and tweak ProtocolClient to work with it. It was also necessary to add a mechanism to get option values from the remote so that encryption settings could be read and used in the storage object. Update the remote storage objects to comply with the protocol changes and add the storage protocol handler. Ideally this commit would have been broken up into smaller chunks but there are cross-dependencies in the protocol layer and it didn't seem worth the extra effort.
2019-02-19 20:57:38 +02:00
ioWriteOpen(write);
// Check greeting
TEST_RESULT_STR(
strPtr(ioReadLine(read)), "{\"name\":\"pgBackRest\",\"service\":\"test\",\"version\":\"" PROJECT_VERSION "\"}",
"check greeting");
// Noop
Add ioWriteStr() and ioWriteStrLine(). These function names make it clearer what is being written. The old ioWriteLine() has been repurposed to write buffers.
2019-04-22 18:46:29 -04:00
TEST_RESULT_VOID(ioWriteStrLine(write, strNew("{\"cmd\":\"noop\"}")), "write noop");
Migrate remote command to C. Prior to this the Perl remote was used to satisfy C requests. This worked fine but since the remote needed to be migrated to C anyway there was no reason to wait. Add the ProtocolServer object and tweak ProtocolClient to work with it. It was also necessary to add a mechanism to get option values from the remote so that encryption settings could be read and used in the storage object. Update the remote storage objects to comply with the protocol changes and add the storage protocol handler. Ideally this commit would have been broken up into smaller chunks but there are cross-dependencies in the protocol layer and it didn't seem worth the extra effort.
2019-02-19 20:57:38 +02:00
TEST_RESULT_VOID(ioWriteFlush(write), "flush noop");
TEST_RESULT_STR(strPtr(ioReadLine(read)), "{}", "noop result");
// Invalid command
Fix reliability of error reporting from local/remote processes. Asserts were only only reported on stderr rather than being returned through the protocol layer. This did not appear to be very reliable. Instead, report the assert through the protocol layer like any other error. Add a stack trace if an assert error or debug logging is enabled.
2019-04-18 10:36:21 -04:00
KeyValue *result = NULL;
Add ioWriteStr() and ioWriteStrLine(). These function names make it clearer what is being written. The old ioWriteLine() has been repurposed to write buffers.
2019-04-22 18:46:29 -04:00
TEST_RESULT_VOID(ioWriteStrLine(write, strNew("{\"cmd\":\"bogus\"}")), "write bogus");
Migrate remote command to C. Prior to this the Perl remote was used to satisfy C requests. This worked fine but since the remote needed to be migrated to C anyway there was no reason to wait. Add the ProtocolServer object and tweak ProtocolClient to work with it. It was also necessary to add a mechanism to get option values from the remote so that encryption settings could be read and used in the storage object. Update the remote storage objects to comply with the protocol changes and add the storage protocol handler. Ideally this commit would have been broken up into smaller chunks but there are cross-dependencies in the protocol layer and it didn't seem worth the extra effort.
2019-02-19 20:57:38 +02:00
TEST_RESULT_VOID(ioWriteFlush(write), "flush bogus");
Fix reliability of error reporting from local/remote processes. Asserts were only only reported on stderr rather than being returned through the protocol layer. This did not appear to be very reliable. Instead, report the assert through the protocol layer like any other error. Add a stack trace if an assert error or debug logging is enabled.
2019-04-18 10:36:21 -04:00
TEST_ASSIGN(result, varKv(jsonToVar(ioReadLine(read))), "parse error result");
TEST_RESULT_INT(varIntForce(kvGet(result, VARSTRDEF("err"))), 39, " check code");
TEST_RESULT_STR(strPtr(varStr(kvGet(result, VARSTRDEF("out")))), "invalid command 'bogus'", " check message");
TEST_RESULT_BOOL(kvGet(result, VARSTRDEF("errStack")) != NULL, true, " check stack exists");
Migrate remote command to C. Prior to this the Perl remote was used to satisfy C requests. This worked fine but since the remote needed to be migrated to C anyway there was no reason to wait. Add the ProtocolServer object and tweak ProtocolClient to work with it. It was also necessary to add a mechanism to get option values from the remote so that encryption settings could be read and used in the storage object. Update the remote storage objects to comply with the protocol changes and add the storage protocol handler. Ideally this commit would have been broken up into smaller chunks but there are cross-dependencies in the protocol layer and it didn't seem worth the extra effort.
2019-02-19 20:57:38 +02:00
// Simple request
Add ioWriteStr() and ioWriteStrLine(). These function names make it clearer what is being written. The old ioWriteLine() has been repurposed to write buffers.
2019-04-22 18:46:29 -04:00
TEST_RESULT_VOID(ioWriteStrLine(write, strNew("{\"cmd\":\"request-simple\"}")), "write simple request");
Migrate remote command to C. Prior to this the Perl remote was used to satisfy C requests. This worked fine but since the remote needed to be migrated to C anyway there was no reason to wait. Add the ProtocolServer object and tweak ProtocolClient to work with it. It was also necessary to add a mechanism to get option values from the remote so that encryption settings could be read and used in the storage object. Update the remote storage objects to comply with the protocol changes and add the storage protocol handler. Ideally this commit would have been broken up into smaller chunks but there are cross-dependencies in the protocol layer and it didn't seem worth the extra effort.
2019-02-19 20:57:38 +02:00
TEST_RESULT_VOID(ioWriteFlush(write), "flush simple request");
TEST_RESULT_STR(strPtr(ioReadLine(read)), "{\"out\":true}", "simple request result");
Fix reliability of error reporting from local/remote processes. Asserts were only only reported on stderr rather than being returned through the protocol layer. This did not appear to be very reliable. Instead, report the assert through the protocol layer like any other error. Add a stack trace if an assert error or debug logging is enabled.
2019-04-18 10:36:21 -04:00
// Throw an assert error which will include a stack trace
Add ioWriteStr() and ioWriteStrLine(). These function names make it clearer what is being written. The old ioWriteLine() has been repurposed to write buffers.
2019-04-22 18:46:29 -04:00
TEST_RESULT_VOID(ioWriteStrLine(write, strNew("{\"cmd\":\"assert\"}")), "write assert");
Fix reliability of error reporting from local/remote processes. Asserts were only only reported on stderr rather than being returned through the protocol layer. This did not appear to be very reliable. Instead, report the assert through the protocol layer like any other error. Add a stack trace if an assert error or debug logging is enabled.
2019-04-18 10:36:21 -04:00
TEST_RESULT_VOID(ioWriteFlush(write), "flush assert error");
TEST_ASSIGN(result, varKv(jsonToVar(ioReadLine(read))), "parse error result");
TEST_RESULT_INT(varIntForce(kvGet(result, VARSTRDEF("err"))), 25, " check code");
TEST_RESULT_STR(strPtr(varStr(kvGet(result, VARSTRDEF("out")))), "test assert", " check message");
TEST_RESULT_BOOL(kvGet(result, VARSTRDEF("errStack")) != NULL, true, " check stack exists");
Migrate remote command to C. Prior to this the Perl remote was used to satisfy C requests. This worked fine but since the remote needed to be migrated to C anyway there was no reason to wait. Add the ProtocolServer object and tweak ProtocolClient to work with it. It was also necessary to add a mechanism to get option values from the remote so that encryption settings could be read and used in the storage object. Update the remote storage objects to comply with the protocol changes and add the storage protocol handler. Ideally this commit would have been broken up into smaller chunks but there are cross-dependencies in the protocol layer and it didn't seem worth the extra effort.
2019-02-19 20:57:38 +02:00
// Complex request -- after process loop has been restarted
Add ioWriteStr() and ioWriteStrLine(). These function names make it clearer what is being written. The old ioWriteLine() has been repurposed to write buffers.
2019-04-22 18:46:29 -04:00
TEST_RESULT_VOID(ioWriteStrLine(write, strNew("{\"cmd\":\"request-complex\"}")), "write complex request");
Migrate remote command to C. Prior to this the Perl remote was used to satisfy C requests. This worked fine but since the remote needed to be migrated to C anyway there was no reason to wait. Add the ProtocolServer object and tweak ProtocolClient to work with it. It was also necessary to add a mechanism to get option values from the remote so that encryption settings could be read and used in the storage object. Update the remote storage objects to comply with the protocol changes and add the storage protocol handler. Ideally this commit would have been broken up into smaller chunks but there are cross-dependencies in the protocol layer and it didn't seem worth the extra effort.
2019-02-19 20:57:38 +02:00
TEST_RESULT_VOID(ioWriteFlush(write), "flush complex request");
TEST_RESULT_STR(strPtr(ioReadLine(read)), "{\"out\":false}", "complex request result");
TEST_RESULT_STR(strPtr(ioReadLine(read)), "LINEOFTEXT", "complex request result");
// Exit
Add ioWriteStr() and ioWriteStrLine(). These function names make it clearer what is being written. The old ioWriteLine() has been repurposed to write buffers.
2019-04-22 18:46:29 -04:00
TEST_RESULT_VOID(ioWriteStrLine(write, strNew("{\"cmd\":\"exit\"}")), "write exit");
Migrate remote command to C. Prior to this the Perl remote was used to satisfy C requests. This worked fine but since the remote needed to be migrated to C anyway there was no reason to wait. Add the ProtocolServer object and tweak ProtocolClient to work with it. It was also necessary to add a mechanism to get option values from the remote so that encryption settings could be read and used in the storage object. Update the remote storage objects to comply with the protocol changes and add the storage protocol handler. Ideally this commit would have been broken up into smaller chunks but there are cross-dependencies in the protocol layer and it didn't seem worth the extra effort.
2019-02-19 20:57:38 +02:00
TEST_RESULT_VOID(ioWriteFlush(write), "flush exit");
}
Improve fork harness to allow multiple children and setup pipes automatically. There was a lot of extra boilerplate involved in setting up pipes so that is now automated. In some cases testing with multiple children is useful so allow that as well.
2019-02-27 18:07:16 +02:00
HARNESS_FORK_CHILD_END();
Migrate remote command to C. Prior to this the Perl remote was used to satisfy C requests. This worked fine but since the remote needed to be migrated to C anyway there was no reason to wait. Add the ProtocolServer object and tweak ProtocolClient to work with it. It was also necessary to add a mechanism to get option values from the remote so that encryption settings could be read and used in the storage object. Update the remote storage objects to comply with the protocol changes and add the storage protocol handler. Ideally this commit would have been broken up into smaller chunks but there are cross-dependencies in the protocol layer and it didn't seem worth the extra effort.
2019-02-19 20:57:38 +02:00
Improve fork harness to allow multiple children and setup pipes automatically. There was a lot of extra boilerplate involved in setting up pipes so that is now automated. In some cases testing with multiple children is useful so allow that as well.
2019-02-27 18:07:16 +02:00
HARNESS_FORK_PARENT_BEGIN()
Migrate remote command to C. Prior to this the Perl remote was used to satisfy C requests. This worked fine but since the remote needed to be migrated to C anyway there was no reason to wait. Add the ProtocolServer object and tweak ProtocolClient to work with it. It was also necessary to add a mechanism to get option values from the remote so that encryption settings could be read and used in the storage object. Update the remote storage objects to comply with the protocol changes and add the storage protocol handler. Ideally this commit would have been broken up into smaller chunks but there are cross-dependencies in the protocol layer and it didn't seem worth the extra effort.
2019-02-19 20:57:38 +02:00
{
Improve type safety of interfaces and drivers. The function pointer casting used when creating drivers made changing interfaces difficult and led to slightly divergent driver implementations. Unit testing caught production-level errors but there were a lot of small issues and the process was harder than it should have been. Use void pointers instead so that no casts are required. Introduce the THIS_VOID and THIS() macros to make dealing with void pointers a little safer. Since we don't want to expose void pointers in header files, driver functions have been removed from the headers and the various driver objects return their interface type. This cuts down on accessor methods and the vast majority of those functions were not being used. Move functions that are still required to .intern.h. Remove the special "C" crypto functions that were used in libc and instead use the standard interface.
2019-05-02 17:52:24 -04:00
IoRead *read = ioHandleReadNew(strNew("server read"), HARNESS_FORK_PARENT_READ_PROCESS(0), 2000);
Migrate remote command to C. Prior to this the Perl remote was used to satisfy C requests. This worked fine but since the remote needed to be migrated to C anyway there was no reason to wait. Add the ProtocolServer object and tweak ProtocolClient to work with it. It was also necessary to add a mechanism to get option values from the remote so that encryption settings could be read and used in the storage object. Update the remote storage objects to comply with the protocol changes and add the storage protocol handler. Ideally this commit would have been broken up into smaller chunks but there are cross-dependencies in the protocol layer and it didn't seem worth the extra effort.
2019-02-19 20:57:38 +02:00
ioReadOpen(read);
Improve type safety of interfaces and drivers. The function pointer casting used when creating drivers made changing interfaces difficult and led to slightly divergent driver implementations. Unit testing caught production-level errors but there were a lot of small issues and the process was harder than it should have been. Use void pointers instead so that no casts are required. Introduce the THIS_VOID and THIS() macros to make dealing with void pointers a little safer. Since we don't want to expose void pointers in header files, driver functions have been removed from the headers and the various driver objects return their interface type. This cuts down on accessor methods and the vast majority of those functions were not being used. Move functions that are still required to .intern.h. Remove the special "C" crypto functions that were used in libc and instead use the standard interface.
2019-05-02 17:52:24 -04:00
IoWrite *write = ioHandleWriteNew(strNew("server write"), HARNESS_FORK_PARENT_WRITE_PROCESS(0));
Migrate remote command to C. Prior to this the Perl remote was used to satisfy C requests. This worked fine but since the remote needed to be migrated to C anyway there was no reason to wait. Add the ProtocolServer object and tweak ProtocolClient to work with it. It was also necessary to add a mechanism to get option values from the remote so that encryption settings could be read and used in the storage object. Update the remote storage objects to comply with the protocol changes and add the storage protocol handler. Ideally this commit would have been broken up into smaller chunks but there are cross-dependencies in the protocol layer and it didn't seem worth the extra effort.
2019-02-19 20:57:38 +02:00
ioWriteOpen(write);
// Send greeting
ProtocolServer *server = NULL;
MEM_CONTEXT_TEMP_BEGIN()
{
TEST_ASSIGN(
server,
protocolServerMove(
protocolServerNew(strNew("test server"), strNew("test"), read, write), MEM_CONTEXT_OLD()),
"create server");
TEST_RESULT_VOID(protocolServerMove(NULL, MEM_CONTEXT_OLD()), "move null server");
}
MEM_CONTEXT_TEMP_END();
TEST_RESULT_PTR(protocolServerIoRead(server), server->read, "get read io");
TEST_RESULT_PTR(protocolServerIoWrite(server), server->write, "get write io");
TEST_RESULT_VOID(protocolServerHandlerAdd(server, testServerProtocol), "add handler");
Fix reliability of error reporting from local/remote processes. Asserts were only only reported on stderr rather than being returned through the protocol layer. This did not appear to be very reliable. Instead, report the assert through the protocol layer like any other error. Add a stack trace if an assert error or debug logging is enabled.
2019-04-18 10:36:21 -04:00
TEST_RESULT_VOID(protocolServerProcess(server), "run process loop");
Migrate remote command to C. Prior to this the Perl remote was used to satisfy C requests. This worked fine but since the remote needed to be migrated to C anyway there was no reason to wait. Add the ProtocolServer object and tweak ProtocolClient to work with it. It was also necessary to add a mechanism to get option values from the remote so that encryption settings could be read and used in the storage object. Update the remote storage objects to comply with the protocol changes and add the storage protocol handler. Ideally this commit would have been broken up into smaller chunks but there are cross-dependencies in the protocol layer and it didn't seem worth the extra effort.
2019-02-19 20:57:38 +02:00
TEST_RESULT_VOID(protocolServerFree(server), "free server");
TEST_RESULT_VOID(protocolServerFree(NULL), "free null server");
}
Improve fork harness to allow multiple children and setup pipes automatically. There was a lot of extra boilerplate involved in setting up pipes so that is now automated. In some cases testing with multiple children is useful so allow that as well.
2019-02-27 18:07:16 +02:00
HARNESS_FORK_PARENT_END();
Migrate remote command to C. Prior to this the Perl remote was used to satisfy C requests. This worked fine but since the remote needed to be migrated to C anyway there was no reason to wait. Add the ProtocolServer object and tweak ProtocolClient to work with it. It was also necessary to add a mechanism to get option values from the remote so that encryption settings could be read and used in the storage object. Update the remote storage objects to comply with the protocol changes and add the storage protocol handler. Ideally this commit would have been broken up into smaller chunks but there are cross-dependencies in the protocol layer and it didn't seem worth the extra effort.
2019-02-19 20:57:38 +02:00
}
HARNESS_FORK_END();
}
Add ProtocolParallel* objects for parallelizing commands. Allows commands to be easily parallelized if the jobs are broken up into discrete, non-overlapping chunks.
2019-02-27 21:10:52 +02:00
// *****************************************************************************************************************************
if (testBegin("ProtocolParallel and ProtocolParallelJob"))
{
ProtocolParallelJob *job = NULL;
MEM_CONTEXT_TEMP_BEGIN()
{
TEST_ASSIGN(job, protocolParallelJobNew(varNewStr(strNew("test")), protocolCommandNew(strNew("command"))), "new job");
TEST_RESULT_PTR(protocolParallelJobMove(job, MEM_CONTEXT_OLD()), job, "move job");
TEST_RESULT_PTR(protocolParallelJobMove(NULL, MEM_CONTEXT_OLD()), NULL, "move null job");
}
MEM_CONTEXT_TEMP_END();
TEST_ERROR(
protocolParallelJobStateSet(job, protocolParallelJobStateDone), AssertError,
"invalid state transition from 'pending' to 'done'");
TEST_RESULT_VOID(protocolParallelJobStateSet(job, protocolParallelJobStateRunning), "transition to running");
TEST_ERROR(
protocolParallelJobStateSet(job, protocolParallelJobStatePending), AssertError,
"invalid state transition from 'running' to 'pending'");
// Free job
TEST_RESULT_VOID(protocolParallelJobFree(job), "free job");
TEST_RESULT_VOID(protocolParallelJobFree(NULL), "free null job");
// -------------------------------------------------------------------------------------------------------------------------
HARNESS_FORK_BEGIN()
{
// Local 1
HARNESS_FORK_CHILD_BEGIN(0, true)
{
Improve type safety of interfaces and drivers. The function pointer casting used when creating drivers made changing interfaces difficult and led to slightly divergent driver implementations. Unit testing caught production-level errors but there were a lot of small issues and the process was harder than it should have been. Use void pointers instead so that no casts are required. Introduce the THIS_VOID and THIS() macros to make dealing with void pointers a little safer. Since we don't want to expose void pointers in header files, driver functions have been removed from the headers and the various driver objects return their interface type. This cuts down on accessor methods and the vast majority of those functions were not being used. Move functions that are still required to .intern.h. Remove the special "C" crypto functions that were used in libc and instead use the standard interface.
2019-05-02 17:52:24 -04:00
IoRead *read = ioHandleReadNew(strNew("server read"), HARNESS_FORK_CHILD_READ(), 10000);
Add ProtocolParallel* objects for parallelizing commands. Allows commands to be easily parallelized if the jobs are broken up into discrete, non-overlapping chunks.
2019-02-27 21:10:52 +02:00
ioReadOpen(read);
Improve type safety of interfaces and drivers. The function pointer casting used when creating drivers made changing interfaces difficult and led to slightly divergent driver implementations. Unit testing caught production-level errors but there were a lot of small issues and the process was harder than it should have been. Use void pointers instead so that no casts are required. Introduce the THIS_VOID and THIS() macros to make dealing with void pointers a little safer. Since we don't want to expose void pointers in header files, driver functions have been removed from the headers and the various driver objects return their interface type. This cuts down on accessor methods and the vast majority of those functions were not being used. Move functions that are still required to .intern.h. Remove the special "C" crypto functions that were used in libc and instead use the standard interface.
2019-05-02 17:52:24 -04:00
IoWrite *write = ioHandleWriteNew(strNew("server write"), HARNESS_FORK_CHILD_WRITE());
Add ProtocolParallel* objects for parallelizing commands. Allows commands to be easily parallelized if the jobs are broken up into discrete, non-overlapping chunks.
2019-02-27 21:10:52 +02:00
ioWriteOpen(write);
// Greeting with noop
Add ioWriteStr() and ioWriteStrLine(). These function names make it clearer what is being written. The old ioWriteLine() has been repurposed to write buffers.
2019-04-22 18:46:29 -04:00
ioWriteStrLine(write, strNew("{\"name\":\"pgBackRest\",\"service\":\"test\",\"version\":\"" PROJECT_VERSION "\"}"));
Add ProtocolParallel* objects for parallelizing commands. Allows commands to be easily parallelized if the jobs are broken up into discrete, non-overlapping chunks.
2019-02-27 21:10:52 +02:00
ioWriteFlush(write);
TEST_RESULT_STR(strPtr(ioReadLine(read)), "{\"cmd\":\"noop\"}", "noop");
Add ioWriteStr() and ioWriteStrLine(). These function names make it clearer what is being written. The old ioWriteLine() has been repurposed to write buffers.
2019-04-22 18:46:29 -04:00
ioWriteStrLine(write, strNew("{}"));
Add ProtocolParallel* objects for parallelizing commands. Allows commands to be easily parallelized if the jobs are broken up into discrete, non-overlapping chunks.
2019-02-27 21:10:52 +02:00
ioWriteFlush(write);
TEST_RESULT_STR(strPtr(ioReadLine(read)), "{\"cmd\":\"command1\",\"param\":[\"param1\",\"param2\"]}", "command1");
sleepMSec(4000);
Add ioWriteStr() and ioWriteStrLine(). These function names make it clearer what is being written. The old ioWriteLine() has been repurposed to write buffers.
2019-04-22 18:46:29 -04:00
ioWriteStrLine(write, strNew("{\"out\":1}"));
Add ProtocolParallel* objects for parallelizing commands. Allows commands to be easily parallelized if the jobs are broken up into discrete, non-overlapping chunks.
2019-02-27 21:10:52 +02:00
ioWriteFlush(write);
// Wait for exit
TEST_RESULT_STR(strPtr(ioReadLine(read)), "{\"cmd\":\"exit\"}", "exit command");
}
HARNESS_FORK_CHILD_END();
// Local 2
HARNESS_FORK_CHILD_BEGIN(0, true)
{
Improve type safety of interfaces and drivers. The function pointer casting used when creating drivers made changing interfaces difficult and led to slightly divergent driver implementations. Unit testing caught production-level errors but there were a lot of small issues and the process was harder than it should have been. Use void pointers instead so that no casts are required. Introduce the THIS_VOID and THIS() macros to make dealing with void pointers a little safer. Since we don't want to expose void pointers in header files, driver functions have been removed from the headers and the various driver objects return their interface type. This cuts down on accessor methods and the vast majority of those functions were not being used. Move functions that are still required to .intern.h. Remove the special "C" crypto functions that were used in libc and instead use the standard interface.
2019-05-02 17:52:24 -04:00
IoRead *read = ioHandleReadNew(strNew("server read"), HARNESS_FORK_CHILD_READ(), 10000);
Add ProtocolParallel* objects for parallelizing commands. Allows commands to be easily parallelized if the jobs are broken up into discrete, non-overlapping chunks.
2019-02-27 21:10:52 +02:00
ioReadOpen(read);
Improve type safety of interfaces and drivers. The function pointer casting used when creating drivers made changing interfaces difficult and led to slightly divergent driver implementations. Unit testing caught production-level errors but there were a lot of small issues and the process was harder than it should have been. Use void pointers instead so that no casts are required. Introduce the THIS_VOID and THIS() macros to make dealing with void pointers a little safer. Since we don't want to expose void pointers in header files, driver functions have been removed from the headers and the various driver objects return their interface type. This cuts down on accessor methods and the vast majority of those functions were not being used. Move functions that are still required to .intern.h. Remove the special "C" crypto functions that were used in libc and instead use the standard interface.
2019-05-02 17:52:24 -04:00
IoWrite *write = ioHandleWriteNew(strNew("server write"), HARNESS_FORK_CHILD_WRITE());
Add ProtocolParallel* objects for parallelizing commands. Allows commands to be easily parallelized if the jobs are broken up into discrete, non-overlapping chunks.
2019-02-27 21:10:52 +02:00
ioWriteOpen(write);
// Greeting with noop
Add ioWriteStr() and ioWriteStrLine(). These function names make it clearer what is being written. The old ioWriteLine() has been repurposed to write buffers.
2019-04-22 18:46:29 -04:00
ioWriteStrLine(write, strNew("{\"name\":\"pgBackRest\",\"service\":\"test\",\"version\":\"" PROJECT_VERSION "\"}"));
Add ProtocolParallel* objects for parallelizing commands. Allows commands to be easily parallelized if the jobs are broken up into discrete, non-overlapping chunks.
2019-02-27 21:10:52 +02:00
ioWriteFlush(write);
TEST_RESULT_STR(strPtr(ioReadLine(read)), "{\"cmd\":\"noop\"}", "noop");
Add ioWriteStr() and ioWriteStrLine(). These function names make it clearer what is being written. The old ioWriteLine() has been repurposed to write buffers.
2019-04-22 18:46:29 -04:00
ioWriteStrLine(write, strNew("{}"));
Add ProtocolParallel* objects for parallelizing commands. Allows commands to be easily parallelized if the jobs are broken up into discrete, non-overlapping chunks.
2019-02-27 21:10:52 +02:00
ioWriteFlush(write);
TEST_RESULT_STR(strPtr(ioReadLine(read)), "{\"cmd\":\"command2\",\"param\":[\"param1\"]}", "command2");
sleepMSec(1000);
Add ioWriteStr() and ioWriteStrLine(). These function names make it clearer what is being written. The old ioWriteLine() has been repurposed to write buffers.
2019-04-22 18:46:29 -04:00
ioWriteStrLine(write, strNew("{\"out\":2}"));
Add ProtocolParallel* objects for parallelizing commands. Allows commands to be easily parallelized if the jobs are broken up into discrete, non-overlapping chunks.
2019-02-27 21:10:52 +02:00
ioWriteFlush(write);
TEST_RESULT_STR(strPtr(ioReadLine(read)), "{\"cmd\":\"command3\",\"param\":[\"param1\"]}", "command3");
Add ioWriteStr() and ioWriteStrLine(). These function names make it clearer what is being written. The old ioWriteLine() has been repurposed to write buffers.
2019-04-22 18:46:29 -04:00
ioWriteStrLine(write, strNew("{\"err\":39,\"out\":\"very serious error\"}"));
Add ProtocolParallel* objects for parallelizing commands. Allows commands to be easily parallelized if the jobs are broken up into discrete, non-overlapping chunks.
2019-02-27 21:10:52 +02:00
ioWriteFlush(write);
// Wait for exit
TEST_RESULT_STR(strPtr(ioReadLine(read)), "{\"cmd\":\"exit\"}", "exit command");
}
HARNESS_FORK_CHILD_END();
HARNESS_FORK_PARENT_BEGIN()
{
// -----------------------------------------------------------------------------------------------------------------
ProtocolParallel *parallel = NULL;
TEST_ASSIGN(parallel, protocolParallelNew(2000), "create parallel");
TEST_RESULT_STR(
strPtr(protocolParallelToLog(parallel)), "{state: pending, clientTotal: 0, jobTotal: 0}", "check log");
// Add client
unsigned int clientTotal = 2;
ProtocolClient *client[HARNESS_FORK_CHILD_MAX];
for (unsigned int clientIdx = 0; clientIdx < clientTotal; clientIdx++)
{
Improve type safety of interfaces and drivers. The function pointer casting used when creating drivers made changing interfaces difficult and led to slightly divergent driver implementations. Unit testing caught production-level errors but there were a lot of small issues and the process was harder than it should have been. Use void pointers instead so that no casts are required. Introduce the THIS_VOID and THIS() macros to make dealing with void pointers a little safer. Since we don't want to expose void pointers in header files, driver functions have been removed from the headers and the various driver objects return their interface type. This cuts down on accessor methods and the vast majority of those functions were not being used. Move functions that are still required to .intern.h. Remove the special "C" crypto functions that were used in libc and instead use the standard interface.
2019-05-02 17:52:24 -04:00
IoRead *read = ioHandleReadNew(
strNewFmt("client %u read", clientIdx), HARNESS_FORK_PARENT_READ_PROCESS(clientIdx), 2000);
Add ProtocolParallel* objects for parallelizing commands. Allows commands to be easily parallelized if the jobs are broken up into discrete, non-overlapping chunks.
2019-02-27 21:10:52 +02:00
ioReadOpen(read);
Improve type safety of interfaces and drivers. The function pointer casting used when creating drivers made changing interfaces difficult and led to slightly divergent driver implementations. Unit testing caught production-level errors but there were a lot of small issues and the process was harder than it should have been. Use void pointers instead so that no casts are required. Introduce the THIS_VOID and THIS() macros to make dealing with void pointers a little safer. Since we don't want to expose void pointers in header files, driver functions have been removed from the headers and the various driver objects return their interface type. This cuts down on accessor methods and the vast majority of those functions were not being used. Move functions that are still required to .intern.h. Remove the special "C" crypto functions that were used in libc and instead use the standard interface.
2019-05-02 17:52:24 -04:00
IoWrite *write = ioHandleWriteNew(
strNewFmt("client %u write", clientIdx), HARNESS_FORK_PARENT_WRITE_PROCESS(clientIdx));
Add ProtocolParallel* objects for parallelizing commands. Allows commands to be easily parallelized if the jobs are broken up into discrete, non-overlapping chunks.
2019-02-27 21:10:52 +02:00
ioWriteOpen(write);
TEST_ASSIGN(
client[clientIdx],
protocolClientNew(strNewFmt("test client %u", clientIdx), strNew("test"), read, write),
"create client %u", clientIdx);
TEST_RESULT_VOID(protocolParallelClientAdd(parallel, client[clientIdx]), "add client %u", clientIdx);
}
// Attempt to add client without handle io
String *protocolString = strNew(
"{\"name\":\"pgBackRest\",\"service\":\"error\",\"version\":\"" PROJECT_VERSION "\"}\n"
"{}\n");
Improve type safety of interfaces and drivers. The function pointer casting used when creating drivers made changing interfaces difficult and led to slightly divergent driver implementations. Unit testing caught production-level errors but there were a lot of small issues and the process was harder than it should have been. Use void pointers instead so that no casts are required. Introduce the THIS_VOID and THIS() macros to make dealing with void pointers a little safer. Since we don't want to expose void pointers in header files, driver functions have been removed from the headers and the various driver objects return their interface type. This cuts down on accessor methods and the vast majority of those functions were not being used. Move functions that are still required to .intern.h. Remove the special "C" crypto functions that were used in libc and instead use the standard interface.
2019-05-02 17:52:24 -04:00
IoRead *read = ioBufferReadNew(BUFSTR(protocolString));
Add ProtocolParallel* objects for parallelizing commands. Allows commands to be easily parallelized if the jobs are broken up into discrete, non-overlapping chunks.
2019-02-27 21:10:52 +02:00
ioReadOpen(read);
Improve type safety of interfaces and drivers. The function pointer casting used when creating drivers made changing interfaces difficult and led to slightly divergent driver implementations. Unit testing caught production-level errors but there were a lot of small issues and the process was harder than it should have been. Use void pointers instead so that no casts are required. Introduce the THIS_VOID and THIS() macros to make dealing with void pointers a little safer. Since we don't want to expose void pointers in header files, driver functions have been removed from the headers and the various driver objects return their interface type. This cuts down on accessor methods and the vast majority of those functions were not being used. Move functions that are still required to .intern.h. Remove the special "C" crypto functions that were used in libc and instead use the standard interface.
2019-05-02 17:52:24 -04:00
IoWrite *write = ioBufferWriteNew(bufNew(1024));
Add ProtocolParallel* objects for parallelizing commands. Allows commands to be easily parallelized if the jobs are broken up into discrete, non-overlapping chunks.
2019-02-27 21:10:52 +02:00
ioWriteOpen(write);
ProtocolClient *clientError = protocolClientNew(strNew("error"), strNew("error"), read, write);
TEST_ERROR(protocolParallelClientAdd(parallel, clientError), AssertError, "client with read handle is required");
protocolClientFree(clientError);
// Add jobs
ProtocolCommand *command = protocolCommandNew(strNew("command1"));
protocolCommandParamAdd(command, varNewStr(strNew("param1")));
protocolCommandParamAdd(command, varNewStr(strNew("param2")));
TEST_RESULT_VOID(
protocolParallelJobAdd(parallel, protocolParallelJobNew(varNewStr(strNew("job1")), command)), "add job");
command = protocolCommandNew(strNew("command2"));
protocolCommandParamAdd(command, varNewStr(strNew("param1")));
TEST_RESULT_VOID(
protocolParallelJobAdd(parallel, protocolParallelJobNew(varNewStr(strNew("job2")), command)), "add job");
command = protocolCommandNew(strNew("command3"));
protocolCommandParamAdd(command, varNewStr(strNew("param1")));
TEST_RESULT_VOID(
protocolParallelJobAdd(parallel, protocolParallelJobNew(varNewStr(strNew("job3")), command)), "add job");
// Process jobs
TEST_RESULT_INT(protocolParallelProcess(parallel), 0, "process jobs");
TEST_RESULT_PTR(protocolParallelResult(parallel), NULL, "check no result");
// Process jobs
TEST_RESULT_INT(protocolParallelProcess(parallel), 1, "process jobs");
TEST_ASSIGN(job, protocolParallelResult(parallel), "get result");
TEST_RESULT_STR(strPtr(varStr(protocolParallelJobKey(job))), "job2", "check key is job2");
Add process id to ProtocolParallelJob. This exposes the process that performed to job to the caller for logging purposes.
2019-04-09 11:01:35 -04:00
TEST_RESULT_BOOL(
protocolParallelJobProcessId(job) >= 1 && protocolParallelJobProcessId(job) <= 2, true,
"check process id is valid");
Add ProtocolParallel* objects for parallelizing commands. Allows commands to be easily parallelized if the jobs are broken up into discrete, non-overlapping chunks.
2019-02-27 21:10:52 +02:00
TEST_RESULT_INT(varIntForce(protocolParallelJobResult(job)), 2, "check result is 2");
TEST_RESULT_PTR(protocolParallelResult(parallel), NULL, "check no more results");
// Process jobs
TEST_RESULT_INT(protocolParallelProcess(parallel), 1, "process jobs");
TEST_ASSIGN(job, protocolParallelResult(parallel), "get result");
TEST_RESULT_STR(strPtr(varStr(protocolParallelJobKey(job))), "job3", "check key is job3");
TEST_RESULT_INT(protocolParallelJobErrorCode(job), 39, "check error code");
TEST_RESULT_STR(
strPtr(protocolParallelJobErrorMessage(job)), "raised from test client 1: very serious error",
"check error message");
TEST_RESULT_PTR(protocolParallelJobResult(job), NULL, "check result is null");
TEST_RESULT_PTR(protocolParallelResult(parallel), NULL, "check no more results");
// Process jobs
TEST_RESULT_INT(protocolParallelProcess(parallel), 0, "process jobs");
TEST_RESULT_PTR(protocolParallelResult(parallel), NULL, "check no result");
// Process jobs
TEST_RESULT_INT(protocolParallelProcess(parallel), 1, "process jobs");
TEST_ASSIGN(job, protocolParallelResult(parallel), "get result");
TEST_RESULT_STR(strPtr(varStr(protocolParallelJobKey(job))), "job1", "check key is job1");
TEST_RESULT_INT(varIntForce(protocolParallelJobResult(job)), 1, "check result is 1");
TEST_RESULT_BOOL(protocolParallelDone(parallel), true, "check done");
// Free client
for (unsigned int clientIdx = 0; clientIdx < clientTotal; clientIdx++)
TEST_RESULT_VOID(protocolClientFree(client[clientIdx]), "free client %u", clientIdx);
// Free parallel
TEST_RESULT_VOID(protocolParallelFree(parallel), "free parallel");
TEST_RESULT_VOID(protocolParallelFree(NULL), "free null parallel");
}
HARNESS_FORK_PARENT_END();
}
HARNESS_FORK_END();
}
Add ProtocolClient object and helper functions. This is a complete protocol client implementation in C. Currently there is no C server implementation so the C client is talking to a Perl server. This won't work very long, though, as the protocol format, even though in JSON, has a lot of language-specific structure. While it would be possible to maintain compatibility between C and Perl it's probably not worth the effort in the long run. Just as in Perl there are helper functions to make constructing protocol objects easier. Currently only repository remotes are supported.
2019-01-18 21:32:51 +02:00
// *****************************************************************************************************************************
if (testBegin("protocolGet()"))
{
Add protocolKeepAlive() to send noops to all remotes. While the local processes are doing their jobs the remote connection from the main process may timeout. Send occasional noops to ensure that doesn't happen.
2019-03-27 20:59:28 +00:00
// Call keep alive before any remotes exist
// -------------------------------------------------------------------------------------------------------------------------
TEST_RESULT_VOID(protocolKeepAlive(), "keep alive");
Migrate remote command to C. Prior to this the Perl remote was used to satisfy C requests. This worked fine but since the remote needed to be migrated to C anyway there was no reason to wait. Add the ProtocolServer object and tweak ProtocolClient to work with it. It was also necessary to add a mechanism to get option values from the remote so that encryption settings could be read and used in the storage object. Update the remote storage objects to comply with the protocol changes and add the storage protocol handler. Ideally this commit would have been broken up into smaller chunks but there are cross-dependencies in the protocol layer and it didn't seem worth the extra effort.
2019-02-19 20:57:38 +02:00
// Simple protocol start
// -------------------------------------------------------------------------------------------------------------------------
Add ProtocolClient object and helper functions. This is a complete protocol client implementation in C. Currently there is no C server implementation so the C client is talking to a Perl server. This won't work very long, though, as the protocol format, even though in JSON, has a lot of language-specific structure. While it would be possible to maintain compatibility between C and Perl it's probably not worth the effort in the long run. Just as in Perl there are helper functions to make constructing protocol objects easier. Currently only repository remotes are supported.
2019-01-18 21:32:51 +02:00
StringList *argList = strLstNew();
strLstAddZ(argList, "/usr/bin/pgbackrest");
strLstAddZ(argList, "--stanza=db");
strLstAddZ(argList, "--protocol-timeout=10");
strLstAddZ(argList, "--repo1-host=localhost");
strLstAdd(argList, strNewFmt("--repo1-path=%s", testPath()));
Migrate remote command to C. Prior to this the Perl remote was used to satisfy C requests. This worked fine but since the remote needed to be migrated to C anyway there was no reason to wait. Add the ProtocolServer object and tweak ProtocolClient to work with it. It was also necessary to add a mechanism to get option values from the remote so that encryption settings could be read and used in the storage object. Update the remote storage objects to comply with the protocol changes and add the storage protocol handler. Ideally this commit would have been broken up into smaller chunks but there are cross-dependencies in the protocol layer and it didn't seem worth the extra effort.
2019-02-19 20:57:38 +02:00
strLstAddZ(argList, "info");
Add ProtocolClient object and helper functions. This is a complete protocol client implementation in C. Currently there is no C server implementation so the C client is talking to a Perl server. This won't work very long, though, as the protocol format, even though in JSON, has a lot of language-specific structure. While it would be possible to maintain compatibility between C and Perl it's probably not worth the effort in the long run. Just as in Perl there are helper functions to make constructing protocol objects easier. Currently only repository remotes are supported.
2019-01-18 21:32:51 +02:00
harnessCfgLoad(strLstSize(argList), strLstPtr(argList));
ProtocolClient *client = NULL;
Migrate local command to C. The C local is only used for C commands in the main process. Some tweaking of the existing protocolGet() command was required. Originally the idea was to share the function for local and remote requests but the differences (as in Perl) were too great to make that practical.
2019-02-27 22:34:21 +02:00
TEST_RESULT_VOID(protocolFree(), "free protocol objects before anything has been created");
Cleanup local/remote protocol interaction from 9367cc46. The command option was not being set correctly when a remote was started from a local. It was being set as 'local' rather than the command that the local was running as. Also automatically select the remote protocol id based on whether it is started from a local (use the local protocol id) or from the main process (use 0). These were not live issues but could cause strange behaviors as new features are added that might be hard to diagnose.
2019-02-28 09:51:19 +02:00
TEST_ASSIGN(client, protocolRemoteGet(protocolStorageTypeRepo), "get remote protocol");
TEST_RESULT_PTR(protocolRemoteGet(protocolStorageTypeRepo), client, "get remote cached protocol");
TEST_RESULT_PTR(protocolHelper.clientRemote[0].client, client, "check position in cache");
Add protocolKeepAlive() to send noops to all remotes. While the local processes are doing their jobs the remote connection from the main process may timeout. Send occasional noops to ensure that doesn't happen.
2019-03-27 20:59:28 +00:00
TEST_RESULT_VOID(protocolKeepAlive(), "keep alive");
Migrate local command to C. The C local is only used for C commands in the main process. Some tweaking of the existing protocolGet() command was required. Originally the idea was to share the function for local and remote requests but the differences (as in Perl) were too great to make that practical.
2019-02-27 22:34:21 +02:00
TEST_RESULT_VOID(protocolFree(), "free remote protocol objects");
TEST_RESULT_VOID(protocolFree(), "free remote protocol objects again");
Migrate remote command to C. Prior to this the Perl remote was used to satisfy C requests. This worked fine but since the remote needed to be migrated to C anyway there was no reason to wait. Add the ProtocolServer object and tweak ProtocolClient to work with it. It was also necessary to add a mechanism to get option values from the remote so that encryption settings could be read and used in the storage object. Update the remote storage objects to comply with the protocol changes and add the storage protocol handler. Ideally this commit would have been broken up into smaller chunks but there are cross-dependencies in the protocol layer and it didn't seem worth the extra effort.
2019-02-19 20:57:38 +02:00
// Start protocol with local encryption settings
// -------------------------------------------------------------------------------------------------------------------------
storagePut(
storageNewWriteNP(storageTest, strNew("pgbackrest.conf")),
Add macros to create constant Buffer objects. These are more efficient than creating buffers in place when needed. After replacement discovered that bufNewStr() and BufNewZ() were not being used in the core code so removed them. This required using the macros in tests which is not the usual pattern.
2019-04-20 08:16:17 -04:00
BUFSTRDEF(
"[global]\n"
"repo1-cipher-type=aes-256-cbc\n"
"repo1-cipher-pass=acbd\n"));
Migrate remote command to C. Prior to this the Perl remote was used to satisfy C requests. This worked fine but since the remote needed to be migrated to C anyway there was no reason to wait. Add the ProtocolServer object and tweak ProtocolClient to work with it. It was also necessary to add a mechanism to get option values from the remote so that encryption settings could be read and used in the storage object. Update the remote storage objects to comply with the protocol changes and add the storage protocol handler. Ideally this commit would have been broken up into smaller chunks but there are cross-dependencies in the protocol layer and it didn't seem worth the extra effort.
2019-02-19 20:57:38 +02:00
argList = strLstNew();
strLstAddZ(argList, "/usr/bin/pgbackrest");
strLstAddZ(argList, "--stanza=db");
strLstAddZ(argList, "--protocol-timeout=10");
strLstAdd(argList, strNewFmt("--config=%s/pgbackrest.conf", testPath()));
strLstAddZ(argList, "--repo1-host=localhost");
strLstAdd(argList, strNewFmt("--repo1-path=%s", testPath()));
Fix segfault when process-max > 8 for archive-push/archive-get. The remote list was at most 9 (based on pg[1-8]-* max index) so anything over 8 wrote into unallocated memory. The remote for the main process is (currently) stored in position zero so do the same for remotes started from locals, since there should only be one. The main process will need to start more remotes in the future which is why there is extra space. Reported by Jens Wilke.
2019-04-29 16:10:27 -04:00
strLstAddZ(argList, "--process=999");
Cleanup local/remote protocol interaction from 9367cc46. The command option was not being set correctly when a remote was started from a local. It was being set as 'local' rather than the command that the local was running as. Also automatically select the remote protocol id based on whether it is started from a local (use the local protocol id) or from the main process (use 0). These were not live issues but could cause strange behaviors as new features are added that might be hard to diagnose.
2019-02-28 09:51:19 +02:00
strLstAddZ(argList, "--command=archive-get");
strLstAddZ(argList, "--host-id=1");
strLstAddZ(argList, "--type=db");
strLstAddZ(argList, "local");
Migrate remote command to C. Prior to this the Perl remote was used to satisfy C requests. This worked fine but since the remote needed to be migrated to C anyway there was no reason to wait. Add the ProtocolServer object and tweak ProtocolClient to work with it. It was also necessary to add a mechanism to get option values from the remote so that encryption settings could be read and used in the storage object. Update the remote storage objects to comply with the protocol changes and add the storage protocol handler. Ideally this commit would have been broken up into smaller chunks but there are cross-dependencies in the protocol layer and it didn't seem worth the extra effort.
2019-02-19 20:57:38 +02:00
harnessCfgLoad(strLstSize(argList), strLstPtr(argList));
TEST_RESULT_STR(strPtr(cfgOptionStr(cfgOptRepoCipherPass)), "acbd", "check cipher pass before");
Cleanup local/remote protocol interaction from 9367cc46. The command option was not being set correctly when a remote was started from a local. It was being set as 'local' rather than the command that the local was running as. Also automatically select the remote protocol id based on whether it is started from a local (use the local protocol id) or from the main process (use 0). These were not live issues but could cause strange behaviors as new features are added that might be hard to diagnose.
2019-02-28 09:51:19 +02:00
TEST_ASSIGN(client, protocolRemoteGet(protocolStorageTypeRepo), "get remote protocol");
Fix segfault when process-max > 8 for archive-push/archive-get. The remote list was at most 9 (based on pg[1-8]-* max index) so anything over 8 wrote into unallocated memory. The remote for the main process is (currently) stored in position zero so do the same for remotes started from locals, since there should only be one. The main process will need to start more remotes in the future which is why there is extra space. Reported by Jens Wilke.
2019-04-29 16:10:27 -04:00
TEST_RESULT_PTR(protocolHelper.clientRemote[0].client, client, "check position in cache");
Migrate remote command to C. Prior to this the Perl remote was used to satisfy C requests. This worked fine but since the remote needed to be migrated to C anyway there was no reason to wait. Add the ProtocolServer object and tweak ProtocolClient to work with it. It was also necessary to add a mechanism to get option values from the remote so that encryption settings could be read and used in the storage object. Update the remote storage objects to comply with the protocol changes and add the storage protocol handler. Ideally this commit would have been broken up into smaller chunks but there are cross-dependencies in the protocol layer and it didn't seem worth the extra effort.
2019-02-19 20:57:38 +02:00
TEST_RESULT_STR(strPtr(cfgOptionStr(cfgOptRepoCipherPass)), "acbd", "check cipher pass after");
Migrate local command to C. The C local is only used for C commands in the main process. Some tweaking of the existing protocolGet() command was required. Originally the idea was to share the function for local and remote requests but the differences (as in Perl) were too great to make that practical.
2019-02-27 22:34:21 +02:00
TEST_RESULT_VOID(protocolFree(), "free remote protocol objects");
Migrate remote command to C. Prior to this the Perl remote was used to satisfy C requests. This worked fine but since the remote needed to be migrated to C anyway there was no reason to wait. Add the ProtocolServer object and tweak ProtocolClient to work with it. It was also necessary to add a mechanism to get option values from the remote so that encryption settings could be read and used in the storage object. Update the remote storage objects to comply with the protocol changes and add the storage protocol handler. Ideally this commit would have been broken up into smaller chunks but there are cross-dependencies in the protocol layer and it didn't seem worth the extra effort.
2019-02-19 20:57:38 +02:00
// Start protocol with remote encryption settings
// -------------------------------------------------------------------------------------------------------------------------
storagePut(
storageNewWriteNP(storageTest, strNew("pgbackrest.conf")),
Add macros to create constant Buffer objects. These are more efficient than creating buffers in place when needed. After replacement discovered that bufNewStr() and BufNewZ() were not being used in the core code so removed them. This required using the macros in tests which is not the usual pattern.
2019-04-20 08:16:17 -04:00
BUFSTRDEF(
"[global]\n"
"repo1-cipher-type=aes-256-cbc\n"
"repo1-cipher-pass=dcba\n"));
Migrate remote command to C. Prior to this the Perl remote was used to satisfy C requests. This worked fine but since the remote needed to be migrated to C anyway there was no reason to wait. Add the ProtocolServer object and tweak ProtocolClient to work with it. It was also necessary to add a mechanism to get option values from the remote so that encryption settings could be read and used in the storage object. Update the remote storage objects to comply with the protocol changes and add the storage protocol handler. Ideally this commit would have been broken up into smaller chunks but there are cross-dependencies in the protocol layer and it didn't seem worth the extra effort.
2019-02-19 20:57:38 +02:00
argList = strLstNew();
strLstAddZ(argList, "/usr/bin/pgbackrest");
strLstAddZ(argList, "--stanza=db");
strLstAddZ(argList, "--protocol-timeout=10");
strLstAdd(argList, strNewFmt("--repo1-host-config=%s/pgbackrest.conf", testPath()));
strLstAddZ(argList, "--repo1-host=localhost");
strLstAdd(argList, strNewFmt("--repo1-path=%s", testPath()));
strLstAddZ(argList, "info");
harnessCfgLoad(strLstSize(argList), strLstPtr(argList));
TEST_RESULT_PTR(cfgOptionStr(cfgOptRepoCipherPass), NULL, "check cipher pass before");
Cleanup local/remote protocol interaction from 9367cc46. The command option was not being set correctly when a remote was started from a local. It was being set as 'local' rather than the command that the local was running as. Also automatically select the remote protocol id based on whether it is started from a local (use the local protocol id) or from the main process (use 0). These were not live issues but could cause strange behaviors as new features are added that might be hard to diagnose.
2019-02-28 09:51:19 +02:00
TEST_ASSIGN(client, protocolRemoteGet(protocolStorageTypeRepo), "get remote protocol");
Migrate remote command to C. Prior to this the Perl remote was used to satisfy C requests. This worked fine but since the remote needed to be migrated to C anyway there was no reason to wait. Add the ProtocolServer object and tweak ProtocolClient to work with it. It was also necessary to add a mechanism to get option values from the remote so that encryption settings could be read and used in the storage object. Update the remote storage objects to comply with the protocol changes and add the storage protocol handler. Ideally this commit would have been broken up into smaller chunks but there are cross-dependencies in the protocol layer and it didn't seem worth the extra effort.
2019-02-19 20:57:38 +02:00
TEST_RESULT_STR(strPtr(cfgOptionStr(cfgOptRepoCipherPass)), "dcba", "check cipher pass after");
Migrate local command to C. The C local is only used for C commands in the main process. Some tweaking of the existing protocolGet() command was required. Originally the idea was to share the function for local and remote requests but the differences (as in Perl) were too great to make that practical.
2019-02-27 22:34:21 +02:00
// Start local protocol
// -------------------------------------------------------------------------------------------------------------------------
argList = strLstNew();
strLstAddZ(argList, "/usr/bin/pgbackrest");
strLstAddZ(argList, "--stanza=db");
strLstAddZ(argList, "--protocol-timeout=10");
The archive-get command is implemented entirely in C. This new implementation should behave exactly like the old Perl code with the exception of a few updated log messages. Remove as much of the Perl code as possible without breaking other commands.
2019-02-27 23:03:02 +02:00
strLstAddZ(argList, "--process-max=2");
strLstAddZ(argList, "archive-get-async");
Migrate local command to C. The C local is only used for C commands in the main process. Some tweaking of the existing protocolGet() command was required. Originally the idea was to share the function for local and remote requests but the differences (as in Perl) were too great to make that practical.
2019-02-27 22:34:21 +02:00
harnessCfgLoad(strLstSize(argList), strLstPtr(argList));
TEST_ASSIGN(client, protocolLocalGet(protocolStorageTypeRepo, 1), "get local protocol");
TEST_RESULT_PTR(protocolLocalGet(protocolStorageTypeRepo, 1), client, "get local cached protocol");
TEST_RESULT_PTR(protocolHelper.clientLocal[0].client, client, "check location in cache");
TEST_RESULT_VOID(protocolFree(), "free local and remote protocol objects");
Add ProtocolClient object and helper functions. This is a complete protocol client implementation in C. Currently there is no C server implementation so the C client is talking to a Perl server. This won't work very long, though, as the protocol format, even though in JSON, has a lot of language-specific structure. While it would be possible to maintain compatibility between C and Perl it's probably not worth the effort in the long run. Just as in Perl there are helper functions to make constructing protocol objects easier. Currently only repository remotes are supported.
2019-01-18 21:32:51 +02:00
}
FUNCTION_HARNESS_RESULT_VOID();
}
Reference in New Issue Copy Permalink