db/pgbackrest
1
0
Fork 0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2025-07-07 00:35:37 +02:00
Code Issues Releases Activity
Files
d30ec9c9aeaaa454078b67a5998614f0b4c9c81d
pgbackrest/src/protocol/parallel.h

68 lines
3.7 KiB
C
Raw Normal View History

Add ProtocolParallel* objects for parallelizing commands. Allows commands to be easily parallelized if the jobs are broken up into discrete, non-overlapping chunks.
2019-02-27 21:10:52 +02:00
/***********************************************************************************************************************************
Protocol Parallel Executor
***********************************************************************************************************************************/
#ifndef PROTOCOL_PARALLEL_H
#define PROTOCOL_PARALLEL_H
/***********************************************************************************************************************************
Object type
***********************************************************************************************************************************/
typedef struct ProtocolParallel ProtocolParallel;
#include "common/time.h"
Replace OBJECT_DEFINE_MOVE() and OBJECT_DEFINE_FREE() with inlines. Inline functions are more efficient and if they are not used are automatically omitted from the binary. This also makes the implementation of these functions easier to find and removes the need for a declaration. That is, the complete implementation is located in the header rather than being spread between the header and C file.
2021-04-08 10:04:57 -04:00
#include "common/type/object.h"
Add ProtocolParallel* objects for parallelizing commands. Allows commands to be easily parallelized if the jobs are broken up into discrete, non-overlapping chunks.
2019-02-27 21:10:52 +02:00
#include "protocol/client.h"
#include "protocol/parallelJob.h"
Use a callback to feed jobs to ProtocolParallel. Loading jobs in advance uses a lot of memory in the case that there are millions of jobs to be performed. We haven't seen this yet, but with backup and restore on the horizon it will become the norm. Instead, use a callback so that jobs are only created as they are needed and can be freed as soon as they are completed.
2019-09-18 07:15:16 -04:00
/***********************************************************************************************************************************
Job request callback
Called whenever a new job is required for processing. If no more jobs are available then NULL is returned. Note that NULL must be
returned to each clientIdx in case job distribution varies by clientIdx.
***********************************************************************************************************************************/
typedef ProtocolParallelJob *ParallelJobCallback(void *data, unsigned int clientIdx);
Add ProtocolParallel* objects for parallelizing commands. Allows commands to be easily parallelized if the jobs are broken up into discrete, non-overlapping chunks.
2019-02-27 21:10:52 +02:00
/***********************************************************************************************************************************
Standardize some section names in headers. It's better to start out with plural forms rather than flip back and forth as functions are added and subtracted. So, use "Constructors" instead of "Constructor". Use "Getters/Setters" rather than "Getters" or "Setters" to avoid similar churn.
2020-04-03 18:15:32 -04:00
Constructors
Add ProtocolParallel* objects for parallelizing commands. Allows commands to be easily parallelized if the jobs are broken up into discrete, non-overlapping chunks.
2019-02-27 21:10:52 +02:00
***********************************************************************************************************************************/
Use a callback to feed jobs to ProtocolParallel. Loading jobs in advance uses a lot of memory in the case that there are millions of jobs to be performed. We haven't seen this yet, but with backup and restore on the horizon it will become the norm. Instead, use a callback so that jobs are only created as they are needed and can be freed as soon as they are completed.
2019-09-18 07:15:16 -04:00
ProtocolParallel *protocolParallelNew(TimeMSec timeout, ParallelJobCallback *callbackFunction, void *callbackData);
Add ProtocolParallel* objects for parallelizing commands. Allows commands to be easily parallelized if the jobs are broken up into discrete, non-overlapping chunks.
2019-02-27 21:10:52 +02:00
/***********************************************************************************************************************************
Functions
***********************************************************************************************************************************/
Move extern function comments to headers. This has been the policy for some time but due to migration pressure only new functions and refactors have been following this rule. Now it seems sensible to make a clean sweep and move all the comments that have not been moved already (i.e. most of them). Only obvious typos and gross inaccuracies in the comments have been fixed. For this most part this was a copy and paste operation. Useless comments, e.g. "New object", were not copied. Even so, there are surely many deficient comments left. Some rearranging was done where needed and functions were placed in the proper sections, e.g. "Constructors", "Functions", etc. A few function prototypes were found that not longer had an implementation. These were removed, but there may be more. The coding document has been updated to reflect this policy, which is not new but has never been documented.
2020-04-03 18:01:28 -04:00
// Add client
Add ProtocolParallel* objects for parallelizing commands. Allows commands to be easily parallelized if the jobs are broken up into discrete, non-overlapping chunks.
2019-02-27 21:10:52 +02:00
void protocolParallelClientAdd(ProtocolParallel *this, ProtocolClient *client);
Move extern function comments to headers. This has been the policy for some time but due to migration pressure only new functions and refactors have been following this rule. Now it seems sensible to make a clean sweep and move all the comments that have not been moved already (i.e. most of them). Only obvious typos and gross inaccuracies in the comments have been fixed. For this most part this was a copy and paste operation. Useless comments, e.g. "New object", were not copied. Even so, there are surely many deficient comments left. Some rearranging was done where needed and functions were placed in the proper sections, e.g. "Constructors", "Functions", etc. A few function prototypes were found that not longer had an implementation. These were removed, but there may be more. The coding document has been updated to reflect this policy, which is not new but has never been documented.
2020-04-03 18:01:28 -04:00
// Process jobs
Add ProtocolParallel* objects for parallelizing commands. Allows commands to be easily parallelized if the jobs are broken up into discrete, non-overlapping chunks.
2019-02-27 21:10:52 +02:00
unsigned int protocolParallelProcess(ProtocolParallel *this);
/***********************************************************************************************************************************
Standardize some section names in headers. It's better to start out with plural forms rather than flip back and forth as functions are added and subtracted. So, use "Constructors" instead of "Constructor". Use "Getters/Setters" rather than "Getters" or "Setters" to avoid similar churn.
2020-04-03 18:15:32 -04:00
Getters/Setters
Add ProtocolParallel* objects for parallelizing commands. Allows commands to be easily parallelized if the jobs are broken up into discrete, non-overlapping chunks.
2019-02-27 21:10:52 +02:00
***********************************************************************************************************************************/
Move extern function comments to headers. This has been the policy for some time but due to migration pressure only new functions and refactors have been following this rule. Now it seems sensible to make a clean sweep and move all the comments that have not been moved already (i.e. most of them). Only obvious typos and gross inaccuracies in the comments have been fixed. For this most part this was a copy and paste operation. Useless comments, e.g. "New object", were not copied. Even so, there are surely many deficient comments left. Some rearranging was done where needed and functions were placed in the proper sections, e.g. "Constructors", "Functions", etc. A few function prototypes were found that not longer had an implementation. These were removed, but there may be more. The coding document has been updated to reflect this policy, which is not new but has never been documented.
2020-04-03 18:01:28 -04:00
// Are all jobs done?
Allow ProtocolParallel to complete with no jobs. If the callback never returned any jobs then protocolParallelDone() would never be true. The reason is that the done state was being set in protocolParallelResult(), which never gets called if there are no results. Calling protocolParallelResult() doesn't make much sense in this case so instead move the done logic to protocolParallelDone(). For current usage of ProtocolParallel we ensure there are jobs before processing so this is not a live issue, but the new behavior is required for future development.
2020-09-02 11:03:49 -04:00
bool protocolParallelDone(ProtocolParallel *this);
Move extern function comments to headers. This has been the policy for some time but due to migration pressure only new functions and refactors have been following this rule. Now it seems sensible to make a clean sweep and move all the comments that have not been moved already (i.e. most of them). Only obvious typos and gross inaccuracies in the comments have been fixed. For this most part this was a copy and paste operation. Useless comments, e.g. "New object", were not copied. Even so, there are surely many deficient comments left. Some rearranging was done where needed and functions were placed in the proper sections, e.g. "Constructors", "Functions", etc. A few function prototypes were found that not longer had an implementation. These were removed, but there may be more. The coding document has been updated to reflect this policy, which is not new but has never been documented.
2020-04-03 18:01:28 -04:00
// Completed job result
Add ProtocolParallel* objects for parallelizing commands. Allows commands to be easily parallelized if the jobs are broken up into discrete, non-overlapping chunks.
2019-02-27 21:10:52 +02:00
ProtocolParallelJob *protocolParallelResult(ProtocolParallel *this);
/***********************************************************************************************************************************
Destructor
***********************************************************************************************************************************/
Replace OBJECT_DEFINE_MOVE() and OBJECT_DEFINE_FREE() with inlines. Inline functions are more efficient and if they are not used are automatically omitted from the binary. This also makes the implementation of these functions easier to find and removes the need for a declaration. That is, the complete implementation is located in the header rather than being spread between the header and C file.
2021-04-08 10:04:57 -04:00
__attribute__((always_inline)) static inline void
protocolParallelFree(ProtocolParallel *this)
{
objFree(this);
}
Add ProtocolParallel* objects for parallelizing commands. Allows commands to be easily parallelized if the jobs are broken up into discrete, non-overlapping chunks.
2019-02-27 21:10:52 +02:00
/***********************************************************************************************************************************
Macros for function logging
***********************************************************************************************************************************/
String *protocolParallelToLog(const ProtocolParallel *this);
#define FUNCTION_LOG_PROTOCOL_PARALLEL_TYPE \
ProtocolParallel *
#define FUNCTION_LOG_PROTOCOL_PARALLEL_FORMAT(value, buffer, bufferSize) \
FUNCTION_LOG_STRING_OBJECT_FORMAT(value, protocolParallelToLog, buffer, bufferSize)
#endif
Reference in New Issue Copy Permalink