db/pgbackrest
1
0
Fork 0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2025-07-05 00:28:52 +02:00
Code Issues Releases Activity
Files
5ea7b91bf9ab57a29778d2332c0971e84f14c4c6
pgbackrest/src/postgres/client.c

395 lines
14 KiB
C
Raw Normal View History

Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
/***********************************************************************************************************************************
Postgres Client
***********************************************************************************************************************************/
#include "build.auto.h"
#include <libpq-fe.h>
#include "common/debug.h"
#include "common/log.h"
#include "common/wait.h"
#include "postgres/client.h"
/***********************************************************************************************************************************
Object type
***********************************************************************************************************************************/
struct PgClient
{
Refactor postgres/client module with inline getters/setters. Extend the pattern introduced in 79a2d02 to the postgres/client module.
2021-11-10 07:53:46 -05:00
PgClientPub pub; // Publicly accessible variables
PGconn *connection; // Pg connection
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
};
/***********************************************************************************************************************************
Close protocol connection
***********************************************************************************************************************************/
Replace OBJECT_DEFINE_FREE_RESOURCE_BEGIN() with normal functions. OBJECT_DEFINE_MOVE() and OBJECT_DEFINE_FREE() will be replaced with inlines so this would be the only macro left that is constructing functions. It is not a great pattern anyway since it makes it hard to find the function implementation.
2021-04-07 16:27:55 -04:00
static void
pgClientFreeResource(THIS_VOID)
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
{
Replace OBJECT_DEFINE_FREE_RESOURCE_BEGIN() with normal functions. OBJECT_DEFINE_MOVE() and OBJECT_DEFINE_FREE() will be replaced with inlines so this would be the only macro left that is constructing functions. It is not a great pattern anyway since it makes it hard to find the function implementation.
2021-04-07 16:27:55 -04:00
THIS(PgClient);
FUNCTION_LOG_BEGIN(logLevelTrace);
FUNCTION_LOG_PARAM(PG_CLIENT, this);
FUNCTION_LOG_END();
ASSERT(this != NULL);
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
PQfinish(this->connection);
Replace OBJECT_DEFINE_FREE_RESOURCE_BEGIN() with normal functions. OBJECT_DEFINE_MOVE() and OBJECT_DEFINE_FREE() will be replaced with inlines so this would be the only macro left that is constructing functions. It is not a great pattern anyway since it makes it hard to find the function implementation.
2021-04-07 16:27:55 -04:00
FUNCTION_LOG_RETURN_VOID();
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
}
Move extern function comments to headers. This has been the policy for some time but due to migration pressure only new functions and refactors have been following this rule. Now it seems sensible to make a clean sweep and move all the comments that have not been moved already (i.e. most of them). Only obvious typos and gross inaccuracies in the comments have been fixed. For this most part this was a copy and paste operation. Useless comments, e.g. "New object", were not copied. Even so, there are surely many deficient comments left. Some rearranging was done where needed and functions were placed in the proper sections, e.g. "Constructors", "Functions", etc. A few function prototypes were found that not longer had an implementation. These were removed, but there may be more. The coding document has been updated to reflect this policy, which is not new but has never been documented.
2020-04-03 18:01:28 -04:00
/**********************************************************************************************************************************/
Split VR_EXTERN/FN_EXTERN macros from FV_EXTERN. This should make it a little clearer what the variable (VR) macros are doing since the declaration/definition cannot both be set to extern (but functions can). Splitting the variable macros out also allows them to be changed in the future with little churn, while changing the function macro creates a large amount of churn.
2023-01-02 15:24:51 +07:00
FN_EXTERN PgClient *
Refactor postgres/client module with inline getters/setters. Extend the pattern introduced in 79a2d02 to the postgres/client module.
2021-11-10 07:53:46 -05:00
pgClientNew(const String *host, const unsigned int port, const String *database, const String *user, const TimeMSec timeout)
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
{
FUNCTION_LOG_BEGIN(logLevelDebug);
FUNCTION_LOG_PARAM(STRING, host);
FUNCTION_LOG_PARAM(UINT, port);
FUNCTION_LOG_PARAM(STRING, database);
FUNCTION_LOG_PARAM(STRING, user);
Refactor postgres/client module with inline getters/setters. Extend the pattern introduced in 79a2d02 to the postgres/client module.
2021-11-10 07:53:46 -05:00
FUNCTION_LOG_PARAM(TIME_MSEC, timeout);
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
FUNCTION_LOG_END();
ASSERT(port >= 1 && port <= 65535);
ASSERT(database != NULL);
Improve memory usage of mem contexts. Each mem context can track child contexts, allocations, and a callback. Before this change memory was allocated for tracking all three even if they were not used for a particular context. This made mem contexts unsuitable for String and Variant objects since they are plentiful and need to be as small as possible. This change allows mem contexts to be configured to track any combination of child contexts, allocations, and a callback. In addition, the mem context can be configured to track a single child context and/or allocation, which saves memory and is a common use case. Another benefit is that Variants can own objects (e.g. KeyValue) that they encapsulate. All of this makes memory accounting simpler because mem contexts have names while allocations do not. No more memory is used than before since Variants and Strings still had to store the memory context they were originally allocated in so they could be easily freed. Update the String and Variant objects to use this new functionality. The custom strFree() and varFree() functions are no longer required and can now be a wrapper around objFree(). Lastly, this will allow strMove() and varMove() to be implemented and used in cases where strDup() and varDup() are being used to move a String or Variant to a new context. Since this will be a bit noisy it is saved for a future commit.
2022-05-18 10:52:01 -04:00
OBJ_NEW_BEGIN(PgClient, .childQty = MEM_CONTEXT_QTY_MAX, .callbackQty = 1)
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
{
Use designated initializers to initialize structs. Previously memNew() used memset() to initialize all struct members to 0, NULL, false, etc. While this appears to work in practice, it is a violation of the C specification. For instance, NULL == 0 must be true but neither NULL nor 0 must be represented with all zero bits. Instead use designated initializers to initialize structs. These guarantee that struct members will be properly initialized even if they are not specified in the initializer. Note that due to a quirk in the C99 specification at least one member must be explicitly initialized even if it needs to be the default value. Since pre-zeroed memory is no longer required, adjust memAllocInternal()/memReallocInternal() to return raw memory and update dependent functions accordingly. All instances of memset() have been removed except in debug/test code where needed. Add memMewPtrArray() to allocate an array of pointers and automatically set all pointers to NULL. Rename memGrowRaw() to the more logical memResize().
2020-01-23 14:15:58 -07:00
*this = (PgClient)
{
Refactor postgres/client module with inline getters/setters. Extend the pattern introduced in 79a2d02 to the postgres/client module.
2021-11-10 07:53:46 -05:00
.pub =
{
.host = strDup(host),
.port = port,
.database = strDup(database),
.user = strDup(user),
.timeout = timeout,
},
Use designated initializers to initialize structs. Previously memNew() used memset() to initialize all struct members to 0, NULL, false, etc. While this appears to work in practice, it is a violation of the C specification. For instance, NULL == 0 must be true but neither NULL nor 0 must be represented with all zero bits. Instead use designated initializers to initialize structs. These guarantee that struct members will be properly initialized even if they are not specified in the initializer. Note that due to a quirk in the C99 specification at least one member must be explicitly initialized even if it needs to be the default value. Since pre-zeroed memory is no longer required, adjust memAllocInternal()/memReallocInternal() to return raw memory and update dependent functions accordingly. All instances of memset() have been removed except in debug/test code where needed. Add memMewPtrArray() to allocate an array of pointers and automatically set all pointers to NULL. Rename memGrowRaw() to the more logical memResize().
2020-01-23 14:15:58 -07:00
};
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
}
Allow additional memory to be allocated with a mem context. The primary benefit is that objects can allocate memory for their struct with the context, which saves an additional allocation and makes it easier to read context/allocation dumps. Also, the memory context does not need to be stored with the object since it can be determined using the object pointer. Object pointers cannot be moved, so this means whatever additional memory is allocated cannot be resized. That makes the additional memory ideal for object structs, but not so much for allocating a list that might change size. Mem contexts can no longer be reused since they will probably be the wrong size so their memory is freed on memContextFree(). This still means fewer allocations and frees overall. Interfaces still need to be freed by mem context so the old objMove() and objFree() have been preserved as objMoveContext() and objFreeContext(). This will be addressed in a future commit.
2021-09-01 11:10:35 -04:00
OBJ_NEW_END();
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
FUNCTION_LOG_RETURN(PG_CLIENT, this);
}
/***********************************************************************************************************************************
Just ignore notices and warnings
***********************************************************************************************************************************/
static void
pgClientNoticeProcessor(void *arg, const char *message)
{
(void)arg;
(void)message;
}
/***********************************************************************************************************************************
Encode string to escape ' and \
***********************************************************************************************************************************/
static String *
pgClientEscape(const String *string)
{
FUNCTION_TEST_BEGIN();
FUNCTION_TEST_PARAM(STRING, string);
FUNCTION_TEST_END();
ASSERT(string != NULL);
More efficient memory allocation for Strings and String Variants. The vast majority of Strings are never modified so for most cases allocate memory for the string with the object. This results in one allocation in most cases instead of two. Use strNew() if strCat*() functions are needed. Update varNewStr() in the same way since String Variants can never be modified. This results in one allocation in all cases instead of three. Also update varNewStrZ() to use STR() instead of strNewZ() to save two more allocations.
2021-10-07 19:43:28 -04:00
String *result = strCatZ(strNew(), "'");
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
// Iterate all characters in the string
for (unsigned stringIdx = 0; stringIdx < strSize(string); stringIdx++)
{
Rename strPtr()/strPtrNull() to strZ()/strZNull(). We use the Z suffix in many functions to indicate that we are expecting a zero-terminated string so make this function conform to the pattern. As a bonus the new name is a bit shorter, which is a good quality in a commonly-used function.
2020-07-30 07:49:06 -04:00
char stringChar = strZ(string)[stringIdx];
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
// These characters are escaped
if (stringChar == '\'' || stringChar == '\\')
strCatChr(result, '\\');
strCatChr(result, stringChar);
}
strCatChr(result, '\'');
Require type for FUNCTION_TEST_RETURN*() macros. This allows code to run after the return type has been generated in the case where it is an expression. No new functionality here yet, but this will be used by a future commit that audits memory usage.
2022-04-24 19:19:46 -04:00
FUNCTION_TEST_RETURN(STRING, result);
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
}
Move extern function comments to headers. This has been the policy for some time but due to migration pressure only new functions and refactors have been following this rule. Now it seems sensible to make a clean sweep and move all the comments that have not been moved already (i.e. most of them). Only obvious typos and gross inaccuracies in the comments have been fixed. For this most part this was a copy and paste operation. Useless comments, e.g. "New object", were not copied. Even so, there are surely many deficient comments left. Some rearranging was done where needed and functions were placed in the proper sections, e.g. "Constructors", "Functions", etc. A few function prototypes were found that not longer had an implementation. These were removed, but there may be more. The coding document has been updated to reflect this policy, which is not new but has never been documented.
2020-04-03 18:01:28 -04:00
/**********************************************************************************************************************************/
Split VR_EXTERN/FN_EXTERN macros from FV_EXTERN. This should make it a little clearer what the variable (VR) macros are doing since the declaration/definition cannot both be set to extern (but functions can). Splitting the variable macros out also allows them to be changed in the future with little churn, while changing the function macro creates a large amount of churn.
2023-01-02 15:24:51 +07:00
FN_EXTERN PgClient *
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
pgClientOpen(PgClient *this)
{
FUNCTION_LOG_BEGIN(logLevelDebug);
FUNCTION_LOG_PARAM(PG_CLIENT, this);
FUNCTION_LOG_END();
ASSERT(this != NULL);
Add error type and message to CHECK() macro. A CHECK() worked exactly like ASSERT() except that it was compiled into production code. However, over time many checks have been added that should not throw AssertError, which should be reserved for probable coding errors. Allow the error code to be specified so other error types can be thrown. Also add a human-readable message since many of these could be seen by users even when there is no coding error. Update coverage exceptions for CHECK() to match ASSERT() since all conditions will never be covered.
2021-11-30 16:21:15 -05:00
CHECK(AssertError, this->connection == NULL, "invalid connection");
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
MEM_CONTEXT_TEMP_BEGIN()
{
// Base connection string
Refactor postgres/client module with inline getters/setters. Extend the pattern introduced in 79a2d02 to the postgres/client module.
2021-11-10 07:53:46 -05:00
String *connInfo = strCatFmt(
strNew(), "dbname=%s port=%u", strZ(pgClientEscape(pgClientDatabase(this))), pgClientPort(this));
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
// Add user if specified
Refactor postgres/client module with inline getters/setters. Extend the pattern introduced in 79a2d02 to the postgres/client module.
2021-11-10 07:53:46 -05:00
if (pgClientUser(this) != NULL)
strCatFmt(connInfo, " user=%s", strZ(pgClientEscape(pgClientUser(this))));
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
// Add host if specified
Refactor postgres/client module with inline getters/setters. Extend the pattern introduced in 79a2d02 to the postgres/client module.
2021-11-10 07:53:46 -05:00
if (pgClientHost(this) != NULL)
strCatFmt(connInfo, " host=%s", strZ(pgClientEscape(pgClientHost(this))));
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
// Make the connection
Rename strPtr()/strPtrNull() to strZ()/strZNull(). We use the Z suffix in many functions to indicate that we are expecting a zero-terminated string so make this function conform to the pattern. As a bonus the new name is a bit shorter, which is a good quality in a commonly-used function.
2020-07-30 07:49:06 -04:00
this->connection = PQconnectdb(strZ(connInfo));
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
// Set a callback to shutdown the connection
Allow additional memory to be allocated with a mem context. The primary benefit is that objects can allocate memory for their struct with the context, which saves an additional allocation and makes it easier to read context/allocation dumps. Also, the memory context does not need to be stored with the object since it can be determined using the object pointer. Object pointers cannot be moved, so this means whatever additional memory is allocated cannot be resized. That makes the additional memory ideal for object structs, but not so much for allocating a list that might change size. Mem contexts can no longer be reused since they will probably be the wrong size so their memory is freed on memContextFree(). This still means fewer allocations and frees overall. Interfaces still need to be freed by mem context so the old objMove() and objFree() have been preserved as objMoveContext() and objFreeContext(). This will be addressed in a future commit.
2021-09-01 11:10:35 -04:00
memContextCallbackSet(objMemContext(this), pgClientFreeResource, this);
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
// Handle errors
if (PQstatus(this->connection) != CONNECTION_OK)
{
THROW_FMT(
Rename strPtr()/strPtrNull() to strZ()/strZNull(). We use the Z suffix in many functions to indicate that we are expecting a zero-terminated string so make this function conform to the pattern. As a bonus the new name is a bit shorter, which is a good quality in a commonly-used function.
2020-07-30 07:49:06 -04:00
DbConnectError, "unable to connect to '%s': %s", strZ(connInfo),
Rename strNew() to strNewZ() and add parameter-less strNew(). Replace all instances of strNew("") with strNew() and use strNewZ() for non-empty zero-terminated strings. Besides saving a useless parameter, this will allow smarter memory allocation in a future commit by signaling intent, in general, to append or not. In the tests use STRDEF() or VARSTRDEF() where more appropriate rather than blindly replacing with strNewZ(). Also replace strLstAdd() with strLstAddZ() where appropriate for the same reason.
2021-05-21 17:36:43 -04:00
strZ(strTrim(strNewZ(PQerrorMessage(this->connection)))));
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
}
// Set notice and warning processor
PQsetNoticeProcessor(this->connection, pgClientNoticeProcessor, NULL);
}
MEM_CONTEXT_TEMP_END();
FUNCTION_LOG_RETURN(PG_CLIENT, this);
}
Move extern function comments to headers. This has been the policy for some time but due to migration pressure only new functions and refactors have been following this rule. Now it seems sensible to make a clean sweep and move all the comments that have not been moved already (i.e. most of them). Only obvious typos and gross inaccuracies in the comments have been fixed. For this most part this was a copy and paste operation. Useless comments, e.g. "New object", were not copied. Even so, there are surely many deficient comments left. Some rearranging was done where needed and functions were placed in the proper sections, e.g. "Constructors", "Functions", etc. A few function prototypes were found that not longer had an implementation. These were removed, but there may be more. The coding document has been updated to reflect this policy, which is not new but has never been documented.
2020-04-03 18:01:28 -04:00
/**********************************************************************************************************************************/
Split VR_EXTERN/FN_EXTERN macros from FV_EXTERN. This should make it a little clearer what the variable (VR) macros are doing since the declaration/definition cannot both be set to extern (but functions can). Splitting the variable macros out also allows them to be changed in the future with little churn, while changing the function macro creates a large amount of churn.
2023-01-02 15:24:51 +07:00
FN_EXTERN Pack *
Refactor PgClient to return results in Pack format. Packs support stronger typing than JSON and are more efficient. For the small result sets that we deal with efficiency is probably not very important, but this removes another place where we are using JSON instead of Pack. Push checking for result struct (e.g. single row) down into PgClient since it has easy access to this information rather than needing to parse the result set to find out. Refactor all code downstream that depends on PgClient results.
2022-04-20 08:36:53 -04:00
pgClientQuery(PgClient *const this, const String *const query, const PgClientQueryResult resultType)
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
{
FUNCTION_LOG_BEGIN(logLevelDebug);
FUNCTION_LOG_PARAM(PG_CLIENT, this);
FUNCTION_LOG_PARAM(STRING, query);
Refactor PgClient to return results in Pack format. Packs support stronger typing than JSON and are more efficient. For the small result sets that we deal with efficiency is probably not very important, but this removes another place where we are using JSON instead of Pack. Push checking for result struct (e.g. single row) down into PgClient since it has easy access to this information rather than needing to parse the result set to find out. Refactor all code downstream that depends on PgClient results.
2022-04-20 08:36:53 -04:00
FUNCTION_LOG_PARAM(STRING_ID, resultType);
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
FUNCTION_LOG_END();
ASSERT(this != NULL);
Add error type and message to CHECK() macro. A CHECK() worked exactly like ASSERT() except that it was compiled into production code. However, over time many checks have been added that should not throw AssertError, which should be reserved for probable coding errors. Allow the error code to be specified so other error types can be thrown. Also add a human-readable message since many of these could be seen by users even when there is no coding error. Update coverage exceptions for CHECK() to match ASSERT() since all conditions will never be covered.
2021-11-30 16:21:15 -05:00
CHECK(AssertError, this->connection != NULL, "invalid connection");
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
ASSERT(query != NULL);
Refactor PgClient to return results in Pack format. Packs support stronger typing than JSON and are more efficient. For the small result sets that we deal with efficiency is probably not very important, but this removes another place where we are using JSON instead of Pack. Push checking for result struct (e.g. single row) down into PgClient since it has easy access to this information rather than needing to parse the result set to find out. Refactor all code downstream that depends on PgClient results.
2022-04-20 08:36:53 -04:00
ASSERT(resultType != 0);
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
Refactor PgClient to return results in Pack format. Packs support stronger typing than JSON and are more efficient. For the small result sets that we deal with efficiency is probably not very important, but this removes another place where we are using JSON instead of Pack. Push checking for result struct (e.g. single row) down into PgClient since it has easy access to this information rather than needing to parse the result set to find out. Refactor all code downstream that depends on PgClient results.
2022-04-20 08:36:53 -04:00
Pack *result = NULL;
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
MEM_CONTEXT_TEMP_BEGIN()
{
// Send the query without waiting for results so we can timeout if needed
Rename strPtr()/strPtrNull() to strZ()/strZNull(). We use the Z suffix in many functions to indicate that we are expecting a zero-terminated string so make this function conform to the pattern. As a bonus the new name is a bit shorter, which is a good quality in a commonly-used function.
2020-07-30 07:49:06 -04:00
if (!PQsendQuery(this->connection, strZ(query)))
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
{
THROW_FMT(
Rename strPtr()/strPtrNull() to strZ()/strZNull(). We use the Z suffix in many functions to indicate that we are expecting a zero-terminated string so make this function conform to the pattern. As a bonus the new name is a bit shorter, which is a good quality in a commonly-used function.
2020-07-30 07:49:06 -04:00
DbQueryError, "unable to send query '%s': %s", strZ(query),
Rename strNew() to strNewZ() and add parameter-less strNew(). Replace all instances of strNew("") with strNew() and use strNewZ() for non-empty zero-terminated strings. Besides saving a useless parameter, this will allow smarter memory allocation in a future commit by signaling intent, in general, to append or not. In the tests use STRDEF() or VARSTRDEF() where more appropriate rather than blindly replacing with strNewZ(). Also replace strLstAdd() with strLstAddZ() where appropriate for the same reason.
2021-05-21 17:36:43 -04:00
strZ(strTrim(strNewZ(PQerrorMessage(this->connection)))));
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
}
// Wait for a result
Refactor postgres/client module with inline getters/setters. Extend the pattern introduced in 79a2d02 to the postgres/client module.
2021-11-10 07:53:46 -05:00
Wait *wait = waitNew(pgClientTimeout(this));
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
bool busy = false;
do
{
PQconsumeInput(this->connection);
busy = PQisBusy(this->connection);
}
while (busy && waitMore(wait));
// If the query is still busy after the timeout attempt to cancel
if (busy)
{
PGcancel *cancel = PQgetCancel(this->connection);
Improve error message when PQgetCancel() returns NULL. There is not a lot to be done in this case since it looks like PostgreSQL disconnected while the query was running, but at least improve the error message and remove the assert, which indicates a coding error.
2020-12-01 15:15:35 -05:00
// If cancel is NULL then more than likely the server process crashed or disconnected
if (cancel == NULL)
THROW_FMT(DbQueryError, "unable to cancel query '%s': connection was lost", strZ(query));
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
TRY_BEGIN()
{
char error[256];
if (!PQcancel(cancel, error, sizeof(error)))
Rename strNew() to strNewZ() and add parameter-less strNew(). Replace all instances of strNew("") with strNew() and use strNewZ() for non-empty zero-terminated strings. Besides saving a useless parameter, this will allow smarter memory allocation in a future commit by signaling intent, in general, to append or not. In the tests use STRDEF() or VARSTRDEF() where more appropriate rather than blindly replacing with strNewZ(). Also replace strLstAdd() with strLstAddZ() where appropriate for the same reason.
2021-05-21 17:36:43 -04:00
THROW_FMT(DbQueryError, "unable to cancel query '%s': %s", strZ(query), strZ(strTrim(strNewZ(error))));
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
}
FINALLY()
{
PQfreeCancel(cancel);
}
TRY_END();
}
// Get the result (even if query was cancelled -- to prevent the connection being left in a bad state)
PGresult *pgResult = PQgetResult(this->connection);
TRY_BEGIN()
{
// Throw timeout error if cancelled
if (busy)
Refactor postgres/client module with inline getters/setters. Extend the pattern introduced in 79a2d02 to the postgres/client module.
2021-11-10 07:53:46 -05:00
THROW_FMT(DbQueryError, "query '%s' timed out after %" PRIu64 "ms", strZ(query), pgClientTimeout(this));
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
// If this was a command that returned no results then we are done
Fix return type for PQresultStatus(). Found on MacOS M1.
2021-01-21 18:30:24 -05:00
ExecStatusType resultStatus = PQresultStatus(pgResult);
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
Refactor PgClient to return results in Pack format. Packs support stronger typing than JSON and are more efficient. For the small result sets that we deal with efficiency is probably not very important, but this removes another place where we are using JSON instead of Pack. Push checking for result struct (e.g. single row) down into PgClient since it has easy access to this information rather than needing to parse the result set to find out. Refactor all code downstream that depends on PgClient results.
2022-04-20 08:36:53 -04:00
if (resultStatus == PGRES_COMMAND_OK)
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
{
Refactor PgClient to return results in Pack format. Packs support stronger typing than JSON and are more efficient. For the small result sets that we deal with efficiency is probably not very important, but this removes another place where we are using JSON instead of Pack. Push checking for result struct (e.g. single row) down into PgClient since it has easy access to this information rather than needing to parse the result set to find out. Refactor all code downstream that depends on PgClient results.
2022-04-20 08:36:53 -04:00
if (resultType != pgClientQueryResultNone && resultType != pgClientQueryResultAny)
THROW_FMT(DbQueryError, "result expected from '%s'", strZ(query));
}
else
{
if (resultType == pgClientQueryResultNone)
THROW_FMT(DbQueryError, "no result expected from '%s'", strZ(query));
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
// Expect some rows to be returned
if (resultStatus != PGRES_TUPLES_OK)
{
THROW_FMT(
Rename strPtr()/strPtrNull() to strZ()/strZNull(). We use the Z suffix in many functions to indicate that we are expecting a zero-terminated string so make this function conform to the pattern. As a bonus the new name is a bit shorter, which is a good quality in a commonly-used function.
2020-07-30 07:49:06 -04:00
DbQueryError, "unable to execute query '%s': %s", strZ(query),
Rename strNew() to strNewZ() and add parameter-less strNew(). Replace all instances of strNew("") with strNew() and use strNewZ() for non-empty zero-terminated strings. Besides saving a useless parameter, this will allow smarter memory allocation in a future commit by signaling intent, in general, to append or not. In the tests use STRDEF() or VARSTRDEF() where more appropriate rather than blindly replacing with strNewZ(). Also replace strLstAdd() with strLstAddZ() where appropriate for the same reason.
2021-05-21 17:36:43 -04:00
strZ(strTrim(strNewZ(PQresultErrorMessage(pgResult)))));
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
}
// Fetch row and column values
Refactor PgClient to return results in Pack format. Packs support stronger typing than JSON and are more efficient. For the small result sets that we deal with efficiency is probably not very important, but this removes another place where we are using JSON instead of Pack. Push checking for result struct (e.g. single row) down into PgClient since it has easy access to this information rather than needing to parse the result set to find out. Refactor all code downstream that depends on PgClient results.
2022-04-20 08:36:53 -04:00
PackWrite *const pack = pckWriteNewP();
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
Refactor PgClient to return results in Pack format. Packs support stronger typing than JSON and are more efficient. For the small result sets that we deal with efficiency is probably not very important, but this removes another place where we are using JSON instead of Pack. Push checking for result struct (e.g. single row) down into PgClient since it has easy access to this information rather than needing to parse the result set to find out. Refactor all code downstream that depends on PgClient results.
2022-04-20 08:36:53 -04:00
int rowTotal = PQntuples(pgResult);
int columnTotal = PQnfields(pgResult);
if (resultType != pgClientQueryResultAny)
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
{
Refactor PgClient to return results in Pack format. Packs support stronger typing than JSON and are more efficient. For the small result sets that we deal with efficiency is probably not very important, but this removes another place where we are using JSON instead of Pack. Push checking for result struct (e.g. single row) down into PgClient since it has easy access to this information rather than needing to parse the result set to find out. Refactor all code downstream that depends on PgClient results.
2022-04-20 08:36:53 -04:00
if (rowTotal != 1)
THROW_FMT(DbQueryError, "expected one row from '%s'", strZ(query));
if (resultType == pgClientQueryResultColumn && columnTotal != 1)
THROW_FMT(DbQueryError, "expected one column from '%s'", strZ(query));
}
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
Refactor PgClient to return results in Pack format. Packs support stronger typing than JSON and are more efficient. For the small result sets that we deal with efficiency is probably not very important, but this removes another place where we are using JSON instead of Pack. Push checking for result struct (e.g. single row) down into PgClient since it has easy access to this information rather than needing to parse the result set to find out. Refactor all code downstream that depends on PgClient results.
2022-04-20 08:36:53 -04:00
// Get column types
Oid *columnType = memNew(sizeof(int) * (size_t)columnTotal);
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
Refactor PgClient to return results in Pack format. Packs support stronger typing than JSON and are more efficient. For the small result sets that we deal with efficiency is probably not very important, but this removes another place where we are using JSON instead of Pack. Push checking for result struct (e.g. single row) down into PgClient since it has easy access to this information rather than needing to parse the result set to find out. Refactor all code downstream that depends on PgClient results.
2022-04-20 08:36:53 -04:00
for (int columnIdx = 0; columnIdx < columnTotal; columnIdx++)
columnType[columnIdx] = PQftype(pgResult, columnIdx);
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
Refactor PgClient to return results in Pack format. Packs support stronger typing than JSON and are more efficient. For the small result sets that we deal with efficiency is probably not very important, but this removes another place where we are using JSON instead of Pack. Push checking for result struct (e.g. single row) down into PgClient since it has easy access to this information rather than needing to parse the result set to find out. Refactor all code downstream that depends on PgClient results.
2022-04-20 08:36:53 -04:00
// Get values
for (int rowIdx = 0; rowIdx < rowTotal; rowIdx++)
{
if (resultType == pgClientQueryResultAny)
pckWriteArrayBeginP(pack);
for (int columnIdx = 0; columnIdx < columnTotal; columnIdx++)
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
{
Refactor PgClient to return results in Pack format. Packs support stronger typing than JSON and are more efficient. For the small result sets that we deal with efficiency is probably not very important, but this removes another place where we are using JSON instead of Pack. Push checking for result struct (e.g. single row) down into PgClient since it has easy access to this information rather than needing to parse the result set to find out. Refactor all code downstream that depends on PgClient results.
2022-04-20 08:36:53 -04:00
char *value = PQgetvalue(pgResult, rowIdx, columnIdx);
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
Refactor PgClient to return results in Pack format. Packs support stronger typing than JSON and are more efficient. For the small result sets that we deal with efficiency is probably not very important, but this removes another place where we are using JSON instead of Pack. Push checking for result struct (e.g. single row) down into PgClient since it has easy access to this information rather than needing to parse the result set to find out. Refactor all code downstream that depends on PgClient results.
2022-04-20 08:36:53 -04:00
// If value is zero-length then check if it is null
if (value[0] == '\0' && PQgetisnull(pgResult, rowIdx, columnIdx))
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
{
Refactor PgClient to return results in Pack format. Packs support stronger typing than JSON and are more efficient. For the small result sets that we deal with efficiency is probably not very important, but this removes another place where we are using JSON instead of Pack. Push checking for result struct (e.g. single row) down into PgClient since it has easy access to this information rather than needing to parse the result set to find out. Refactor all code downstream that depends on PgClient results.
2022-04-20 08:36:53 -04:00
pckWriteNullP(pack);
}
// Else convert the value to a variant
else
{
// Convert column type. Not all PostgreSQL types are supported but these should suffice.
switch (columnType[columnIdx])
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
{
Refactor PgClient to return results in Pack format. Packs support stronger typing than JSON and are more efficient. For the small result sets that we deal with efficiency is probably not very important, but this removes another place where we are using JSON instead of Pack. Push checking for result struct (e.g. single row) down into PgClient since it has easy access to this information rather than needing to parse the result set to find out. Refactor all code downstream that depends on PgClient results.
2022-04-20 08:36:53 -04:00
// Boolean type
case 16: // bool
pckWriteBoolP(pack, varBoolForce(varNewStrZ(value)), .defaultWrite = true);
break;
// Text/char types
case 18: // char
case 19: // name
case 25: // text
pckWriteStrP(pack, STR(value), .defaultWrite = true);
break;
// 64-bit integer type
case 20: // int8
pckWriteI64P(pack, cvtZToInt64(value), .defaultWrite = true);
break;
// 32-bit integer type
case 23: // int4
pckWriteI32P(pack, cvtZToInt(value), .defaultWrite = true);
break;
// 32-bit unsigned integer type
case 26: // oid
pckWriteU32P(pack, cvtZToUInt(value), .defaultWrite = true);
break;
default:
THROW_FMT(
FormatError, "unable to parse type %u in column %d for query '%s'",
columnType[columnIdx], columnIdx, strZ(query));
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
}
}
}
Refactor PgClient to return results in Pack format. Packs support stronger typing than JSON and are more efficient. For the small result sets that we deal with efficiency is probably not very important, but this removes another place where we are using JSON instead of Pack. Push checking for result struct (e.g. single row) down into PgClient since it has easy access to this information rather than needing to parse the result set to find out. Refactor all code downstream that depends on PgClient results.
2022-04-20 08:36:53 -04:00
if (resultType == pgClientQueryResultAny)
pckWriteArrayEndP(pack);
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
}
Refactor PgClient to return results in Pack format. Packs support stronger typing than JSON and are more efficient. For the small result sets that we deal with efficiency is probably not very important, but this removes another place where we are using JSON instead of Pack. Push checking for result struct (e.g. single row) down into PgClient since it has easy access to this information rather than needing to parse the result set to find out. Refactor all code downstream that depends on PgClient results.
2022-04-20 08:36:53 -04:00
// End pack and return result
pckWriteEndP(pack);
result = pckMove(pckWriteResult(pack), memContextPrior());
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
}
}
FINALLY()
{
// Free the result
PQclear(pgResult);
Add error type and message to CHECK() macro. A CHECK() worked exactly like ASSERT() except that it was compiled into production code. However, over time many checks have been added that should not throw AssertError, which should be reserved for probable coding errors. Allow the error code to be specified so other error types can be thrown. Also add a human-readable message since many of these could be seen by users even when there is no coding error. Update coverage exceptions for CHECK() to match ASSERT() since all conditions will never be covered.
2021-11-30 16:21:15 -05:00
CHECK(ServiceError, PQgetResult(this->connection) == NULL, "NULL result required to complete request");
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
}
TRY_END();
}
MEM_CONTEXT_TEMP_END();
Refactor PgClient to return results in Pack format. Packs support stronger typing than JSON and are more efficient. For the small result sets that we deal with efficiency is probably not very important, but this removes another place where we are using JSON instead of Pack. Push checking for result struct (e.g. single row) down into PgClient since it has easy access to this information rather than needing to parse the result set to find out. Refactor all code downstream that depends on PgClient results.
2022-04-20 08:36:53 -04:00
FUNCTION_LOG_RETURN(PACK, result);
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
}
Move extern function comments to headers. This has been the policy for some time but due to migration pressure only new functions and refactors have been following this rule. Now it seems sensible to make a clean sweep and move all the comments that have not been moved already (i.e. most of them). Only obvious typos and gross inaccuracies in the comments have been fixed. For this most part this was a copy and paste operation. Useless comments, e.g. "New object", were not copied. Even so, there are surely many deficient comments left. Some rearranging was done where needed and functions were placed in the proper sections, e.g. "Constructors", "Functions", etc. A few function prototypes were found that not longer had an implementation. These were removed, but there may be more. The coding document has been updated to reflect this policy, which is not new but has never been documented.
2020-04-03 18:01:28 -04:00
/**********************************************************************************************************************************/
Split VR_EXTERN/FN_EXTERN macros from FV_EXTERN. This should make it a little clearer what the variable (VR) macros are doing since the declaration/definition cannot both be set to extern (but functions can). Splitting the variable macros out also allows them to be changed in the future with little churn, while changing the function macro creates a large amount of churn.
2023-01-02 15:24:51 +07:00
FN_EXTERN void
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
pgClientClose(PgClient *this)
{
FUNCTION_LOG_BEGIN(logLevelDebug);
FUNCTION_LOG_PARAM(PG_CLIENT, this);
FUNCTION_LOG_END();
ASSERT(this != NULL);
Add Db object to encapsulate PostgreSQL queries and commands. Migrate functionality from the Perl Db module to C. For now this is just enough to implement the WAL switch check. Add the dbGet() helper function to get Db objects easily. Create macros in harnessPq to make writing pq scripts easier by grouping commonly used functions together. Reviewed by Cynthia Shang.
2019-08-01 15:38:27 -04:00
if (this->connection != NULL)
{
Allow additional memory to be allocated with a mem context. The primary benefit is that objects can allocate memory for their struct with the context, which saves an additional allocation and makes it easier to read context/allocation dumps. Also, the memory context does not need to be stored with the object since it can be determined using the object pointer. Object pointers cannot be moved, so this means whatever additional memory is allocated cannot be resized. That makes the additional memory ideal for object structs, but not so much for allocating a list that might change size. Mem contexts can no longer be reused since they will probably be the wrong size so their memory is freed on memContextFree(). This still means fewer allocations and frees overall. Interfaces still need to be freed by mem context so the old objMove() and objFree() have been preserved as objMoveContext() and objFreeContext(). This will be addressed in a future commit.
2021-09-01 11:10:35 -04:00
memContextCallbackClear(objMemContext(this));
Add Db object to encapsulate PostgreSQL queries and commands. Migrate functionality from the Perl Db module to C. For now this is just enough to implement the WAL switch check. Add the dbGet() helper function to get Db objects easily. Create macros in harnessPq to make writing pq scripts easier by grouping commonly used functions together. Reviewed by Cynthia Shang.
2019-08-01 15:38:27 -04:00
PQfinish(this->connection);
this->connection = NULL;
}
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
FUNCTION_LOG_RETURN_VOID();
}
Move extern function comments to headers. This has been the policy for some time but due to migration pressure only new functions and refactors have been following this rule. Now it seems sensible to make a clean sweep and move all the comments that have not been moved already (i.e. most of them). Only obvious typos and gross inaccuracies in the comments have been fixed. For this most part this was a copy and paste operation. Useless comments, e.g. "New object", were not copied. Even so, there are surely many deficient comments left. Some rearranging was done where needed and functions were placed in the proper sections, e.g. "Constructors", "Functions", etc. A few function prototypes were found that not longer had an implementation. These were removed, but there may be more. The coding document has been updated to reflect this policy, which is not new but has never been documented.
2020-04-03 18:01:28 -04:00
/**********************************************************************************************************************************/
Refactor logging functions to never allocate memory. Allocating memory made these functions simpler but it meant that memory was leaking into the calling context when logging was enabled. It is not clear that this was an issue but it seems that trace level logging could result it a lot of memory usage depending on the use case. This also makes it possible to audit allocations returned to the calling context, which will be done in a followup commit. Also rename objToLog() to objNameToLog() since it seemed logical to name the new function objToLog().
2023-01-12 17:14:36 +07:00
FN_EXTERN void
pgClientToLog(const PgClient *const this, StringStatic *const debugLog)
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
{
Refactor logging functions to never allocate memory. Allocating memory made these functions simpler but it meant that memory was leaking into the calling context when logging was enabled. It is not clear that this was an issue but it seems that trace level logging could result it a lot of memory usage depending on the use case. This also makes it possible to audit allocations returned to the calling context, which will be done in a followup commit. Also rename objToLog() to objNameToLog() since it seemed logical to name the new function objToLog().
2023-01-12 17:14:36 +07:00
strStcCat(debugLog, "{host: ");
strStcResultSizeInc(
debugLog,
FUNCTION_LOG_OBJECT_FORMAT(pgClientHost(this), strToLog, strStcRemains(debugLog), strStcRemainsSize(debugLog)));
strStcCat(debugLog, ", database: ");
strToLog(pgClientDatabase(this), debugLog);
strStcCat(debugLog, ", user: ");
strStcResultSizeInc(
debugLog,
FUNCTION_LOG_OBJECT_FORMAT(pgClientUser(this), strToLog, strStcRemains(debugLog), strStcRemainsSize(debugLog)));
strStcFmt(
debugLog, ", port: %u, queryTimeout %" PRIu64 "}", pgClientPort(this), pgClientTimeout(this));
Add PostgreSQL query client. This direct interface to libpq allows simple queries to be run against PostgreSQL and supports timeouts. Testing is performed using a shim that can use scripted responses to test all aspects of the client code. The shim will be very useful for testing backup scenarios on complex topologies. Reviewed by Cynthia Shang.
2019-07-25 14:50:02 -04:00
}
Reference in New Issue Copy Permalink