db/pgbackrest
1
0
Fork 0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2025-03-05 15:05:48 +02:00
Code Issues Releases Activity
pgbackrest/src/main.c

308 lines
12 KiB
C
Raw Normal View History

The pgbackrest executable is now a C binary instead of Perl. This allows certain time-critical commands (like async archive-push) to run more quickly.
2017-11-26 18:43:51 -05:00
/***********************************************************************************************************************************
Main
***********************************************************************************************************************************/
Add configure script for improved multi-platform support. Use autoconf to provide a basic configure script. WITH_BACKTRACE is yet to be migrated to configure and the unit tests still use a custom Makefile. Each C file must include "build.auto.conf" before all other includes and defines. This is enforced by test.pl for includes, but it won't detect incorrect define ordering. Update packages to call configure and use standard flags to pass options.
2019-04-26 08:08:23 -04:00
#include "build.auto.h"
Implement version command in C.
2017-11-28 21:44:05 -05:00
#include <stdio.h>
#include <stdlib.h>
Migrate remote command to C. Prior to this the Perl remote was used to satisfy C requests. This worked fine but since the remote needed to be migrated to C anyway there was no reason to wait. Add the ProtocolServer object and tweak ProtocolClient to work with it. It was also necessary to add a mechanism to get option values from the remote so that encryption settings could be read and used in the storage object. Update the remote storage objects to comply with the protocol changes and add the storage protocol handler. Ideally this commit would have been broken up into smaller chunks but there are cross-dependencies in the protocol layer and it didn't seem worth the extra effort.
2019-02-19 20:57:38 +02:00
#include <unistd.h>
Implement version command in C.
2017-11-28 21:44:05 -05:00
Add asynchronous, parallel archive-get. This feature maintains a queue of WAL segments to help reduce latency when PostgreSQL requests a WAL segment with restore_command.
2018-04-30 17:27:39 -04:00
#include "command/archive/get/get.h"
The archive-push command is now partially coded in C. This allows the PostgreSQL archive_command to run significantly faster when processing status messages from the asynchronous archive process.
2018-01-17 15:52:00 -05:00
#include "command/archive/push/push.h"
The backup command is implemented entirely in C. For the most part this is a direct migration of the Perl code into C except as noted below. A backup can now be initiated from a linked directory. The link will not be stored in the manifest or recreated on restore. If a link or directory does not already exist in the restore location then a directory will be created. The logic for creating backup labels has been improved and it should no longer be possible to get a backup label earlier than the latest backup even with timezone changes or clock skew. This has never been an issue in the field that we know of, but we found it in testing. For online backups all times are fetched from the PostgreSQL primary host (before only copy start was). This doesn't affect backup integrity but it does prevent clock skew between hosts affecting backup duration reporting. Archive copy now works as expected when the archive and backup have different compression settings, i.e. when one is compressed and the other is not. This was a long-standing bug in the Perl code. Resume will now work even if hardlink settings have been changed. Reviewed by Cynthia Shang.
2019-12-13 17:14:26 -05:00
#include "command/backup/backup.h"
The check command is implemented partly in C. Implement switch WAL and archive check in C but leave the rest in Perl for now. The main idea was to have some real integration tests for the new database code so the rest of the migration can wait. Reviewed by Cynthia Shang.
2019-08-01 20:35:01 -04:00
#include "command/check/check.h"
Migrate local command to C. The C local is only used for C commands in the main process. Some tweaking of the existing protocolGet() command was required. Originally the idea was to share the function for local and remote requests but the differences (as in Perl) were too great to make that practical.
2019-02-27 22:34:21 +02:00
#include "command/command.h"
The start/stop commands are implemented entirely in C. The Perl versions remain because they are still being used by the Perl stanza commands. Once the stanza commands are migrated they can be removed. Contributed by Cynthia Shang.
2019-08-09 15:17:18 -04:00
#include "command/control/start.h"
#include "command/control/stop.h"
The expire command is implemented entirely in C. This implementation duplicates the functionality of the Perl code but does so with different logic and includes full unit tests. Along the way at least one bug was fixed, see issue #748. Contributed by Cynthia Shang.
2019-06-18 15:19:20 -04:00
#include "command/expire/expire.h"
Implement help command in C.
2018-01-23 13:34:24 -05:00
#include "command/help/help.h"
Make the C version of the info command conform to the Perl version. There were some small differences in ordering and how the C version handled missing directories. It may be that the C version is more consistent, but for now it is more important to be compatible with the Perl version. These differences were missed because the C info command was not wired into main.c so it was not being tested in regression. This commit does not fix the wiring issue because there will likely be a release soon and it is too big a change to put in at the last moment.
2019-01-13 22:44:58 +02:00
#include "command/info/info.h"
Migrate local command to C. The C local is only used for C commands in the main process. Some tweaking of the existing protocolGet() command was required. Originally the idea was to share the function for local and remote requests but the differences (as in Perl) were too great to make that practical.
2019-02-27 22:34:21 +02:00
#include "command/local/local.h"
Migrate remote command to C. Prior to this the Perl remote was used to satisfy C requests. This worked fine but since the remote needed to be migrated to C anyway there was no reason to wait. Add the ProtocolServer object and tweak ProtocolClient to work with it. It was also necessary to add a mechanism to get option values from the remote so that encryption settings could be read and used in the storage object. Update the remote storage objects to comply with the protocol changes and add the storage protocol handler. Ideally this commit would have been broken up into smaller chunks but there are cross-dependencies in the protocol layer and it didn't seem worth the extra effort.
2019-02-19 20:57:38 +02:00
#include "command/remote/remote.h"
Add repo-create, repo-get, repo-put, and repo-rm commands. These commands are generally useful but more importantly they allow removing LibC by providing the Perl integration tests an alternate way to work with repository storage. All the commands are currently internal only and should not be used on production repositories.
2020-03-09 17:15:03 -04:00
#include "command/repo/create.h"
#include "command/repo/get.h"
Rename ls command to repo-ls. This command only makes sense for the repository storage since other storage (e.g. pg and spool) must be located on a local Posix filesystem and can be listed using standard unix commands. Since the repo storage can be located lots of places having a common way to list it makes sense. Prefix with repo- to make the scope of this command clear. Update documentation to reflect this change.
2020-03-09 16:41:04 -04:00
#include "command/repo/ls.h"
Add repo-create, repo-get, repo-put, and repo-rm commands. These commands are generally useful but more importantly they allow removing LibC by providing the Perl integration tests an alternate way to work with repository storage. All the commands are currently internal only and should not be used on production repositories.
2020-03-09 17:15:03 -04:00
#include "command/repo/put.h"
#include "command/repo/rm.h"
The restore command is implemented entirely in C. For the most part this is a direct migration of the Perl code into C. There is one important behavioral change with regard to how file permissions are handled. The Perl code tried to set ownership as it was in the manifest even when running as an unprivileged user. This usually just led to errors and frustration. The C code works like this: If a restore is run as a non-root user (the typical scenario) then all files restored will belong to the user/group executing pgBackRest. If existing files are not owned by the executing user/group then an error will result if the ownership cannot be updated to the executing user/group. In that case the file ownership will need to be updated by a privileged user before the restore can be retried. If a restore is run as the root user then pgBackRest will attempt to recreate the ownership recorded in the manifest when the backup was made. Only user/group names are stored in the manifest so the same names must exist on the restore host for this to work. If the user/group name cannot be found locally then the user/group of the PostgreSQL data directory will be used and finally root if the data directory user/group cannot be mapped to a name. Reviewed by Cynthia Shang.
2019-09-26 07:52:02 -04:00
#include "command/restore/restore.h"
Add TLS Server. The TLS server is an alternative to using SSH for protocol connections to remote hosts. This command is currently experimental and intended only for trial and testing. As such, the new commands and options will not show up in the command-line help unless directly requested.
2021-10-18 14:32:41 -04:00
#include "command/server/ping.h"
#include "command/server/server.h"
The stanza-create/update/delete commands are implemented entirely in C. Contributed by Cynthia Shang.
2019-08-21 16:26:28 -04:00
#include "command/stanza/create.h"
#include "command/stanza/delete.h"
#include "command/stanza/upgrade.h"
Add internal verify command. Scan the WAL archive for missing or invalid files and build up ranges of WAL that will be used to verify backup integrity. A number of errors and warnings are currently emitted but they should not be considered authoritative (yet). The command is incomplete so is marked internal.
2020-09-22 11:57:38 -04:00
#include "command/verify/verify.h"
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 11:57:32 -04:00
#include "common/debug.h"
Implement version command in C.
2017-11-28 21:44:05 -05:00
#include "common/error.h"
Add command and exit modules.
2018-01-17 09:15:51 -05:00
#include "common/exit.h"
Refactor cmdLocal()/cmdRemote() to accept a ProtocolServer object. This allows a local/remote to be started independently of server initialization, which will be useful for implementing new transport types, e.g. TLS. Also remove some dead code in localTest.c.
2021-07-23 10:35:36 -04:00
#include "common/io/fdRead.h"
#include "common/io/fdWrite.h"
Add general-purpose statistics collector. Currently each module that needs to collect statistics implements custom code to do so. This is cumbersome. Create a general purpose module for collecting and reporting statistics. Statistics are output in the log at detail level, but there are other uses they could be put to eventually. No new functionality is added. This is just a drop-in replacement for the current statistics, with the advantage of being more flexible. The new stats are slower because they involve a list lookup, but performance testing shows stats can be updated at about 40,000/ms which seems fast enough for our purposes.
2020-08-20 14:04:26 -04:00
#include "common/stat.h"
Implement version command in C.
2017-11-28 21:44:05 -05:00
#include "config/config.h"
Add log module.
2018-01-16 13:52:20 -05:00
#include "config/load.h"
Support configurable WAL segment size. PostgreSQL 11 introduces configurable WAL segment sizes, from 1MB to 1GB. There are two areas that needed to be updated to support this: building the archive-get queue and checking that WAL has been archived after a backup. Both operations require the WAL segment size to properly build a list. Checking the archive after a backup is still implemented in Perl and has an active database connection, so just get the WAL segment size from the database. The archive-get command does not have a connection to the database, so get the WAL segment size from pg_control instead. This requires a deeper inspection of pg_control than has been done in the past, so it seemed best to copy the relevant data structures from each version of PostgreSQL and build a generic interface layer to address them. While this approach is a bit verbose, it has the advantage of being relatively simple, and can easily be updated for new versions of PostgreSQL. Since the integration tests generate pg_control files for testing, teach Perl how to generate files with the correct offsets for both 32-bit and 64-bit architectures.
2018-09-25 10:24:42 +01:00
#include "postgres/interface.h"
Refactor cmdLocal()/cmdRemote() to accept a ProtocolServer object. This allows a local/remote to be started independently of server initialization, which will be useful for implementing new transport types, e.g. TLS. Also remove some dead code in localTest.c.
2021-07-23 10:35:36 -04:00
#include "protocol/helper.h"
Add configurable storage helpers to create repository storage. Remove the hardcoded storage helpers from storageRepoGet() except for the the built-in Posix helper and the special remote helper. The goal is to make storage driver development a bit easier by isolating as much of the code as possible into the driver module. This also makes coverage reporting much simpler for additional drivers since they do not need to provide coverage for storage/helper. Consolidate the CIFS tests into the Posix tests since CIFS is just a special case of the Posix. Test all storage features in the Posix test so that other storage driver tests do not need to provide coverage for storage/storage. Remove some dead code in the storage/s3 test.
2021-10-06 19:27:04 -04:00
#include "storage/azure/helper.h"
#include "storage/cifs/helper.h"
#include "storage/gcs/helper.h"
Require storage when calling pgControlFromFile(). Previously storageLocal() was being used internally but loading pg_control from remote storage is often required. Also, storagePg() is more appropriate than storageLocal() for all current usage. Contributed by Cynthia Shang.
2019-08-21 11:29:30 -04:00
#include "storage/helper.h"
Add configurable storage helpers to create repository storage. Remove the hardcoded storage helpers from storageRepoGet() except for the the built-in Posix helper and the special remote helper. The goal is to make storage driver development a bit easier by isolating as much of the code as possible into the driver module. This also makes coverage reporting much simpler for additional drivers since they do not need to provide coverage for storage/helper. Consolidate the CIFS tests into the Posix tests since CIFS is just a special case of the Posix. Test all storage features in the Posix test so that other storage driver tests do not need to provide coverage for storage/storage. Remove some dead code in the storage/s3 test.
2021-10-06 19:27:04 -04:00
#include "storage/s3/helper.h"
Implement version command in C.
2017-11-28 21:44:05 -05:00
#include "version.h"
The pgbackrest executable is now a C binary instead of Perl. This allows certain time-critical commands (like async archive-push) to run more quickly.
2017-11-26 18:43:51 -05:00
Migrate command-line help generation to C. Command-line help is now generated at build time so it does not need to be committed. This reduces churn on commits that add configuration and/or update the help. Since churn is no longer an issue, help.auto.c is bzip2 compressed to save space in the binary. The Perl config parser (Data.pm) has been moved to doc/lib since the Perl build path is no longer required. Likewise doc/xml/reference.xml has been moved to src/build/help/help.xml since it is required at build time.
2021-09-08 18:16:06 -04:00
/***********************************************************************************************************************************
Include automatically generated help data
***********************************************************************************************************************************/
#include "command/help/help.auto.c"
Fix non-compliant formatting for function declarations. Per review by Cynthia Shang.
2018-01-31 18:22:25 -05:00
int
main(int argListSize, const char *argList[])
The pgbackrest executable is now a C binary instead of Perl. This allows certain time-critical commands (like async archive-push) to run more quickly.
2017-11-26 18:43:51 -05:00
{
Add configurable error handlers. The stackTrace and memContext error handlers were hard-coded which made testing the error module in isolation impossible. Making the error handlers configurable also makes adding new ones in the future easier.
2021-01-27 17:25:13 -05:00
// Set stack trace and mem context error cleanup handlers
static const ErrorHandlerFunction errorHandlerList[] = {stackTraceClean, memContextClean};
errorHandlerSet(errorHandlerList, sizeof(errorHandlerList) / sizeof(ErrorHandlerFunction));
Add configurable storage helpers to create repository storage. Remove the hardcoded storage helpers from storageRepoGet() except for the the built-in Posix helper and the special remote helper. The goal is to make storage driver development a bit easier by isolating as much of the code as possible into the driver module. This also makes coverage reporting much simpler for additional drivers since they do not need to provide coverage for storage/helper. Consolidate the CIFS tests into the Posix tests since CIFS is just a special case of the Posix. Test all storage features in the Posix test so that other storage driver tests do not need to provide coverage for storage/storage. Remove some dead code in the storage/s3 test.
2021-10-06 19:27:04 -04:00
// Set storage helpers
static const StorageHelper storageHelperList[] =
{
STORAGE_AZURE_HELPER,
STORAGE_CIFS_HELPER,
STORAGE_GCS_HELPER,
STORAGE_S3_HELPER,
STORAGE_END_HELPER
};
storageHelperInit(storageHelperList);
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 11:57:32 -04:00
#ifdef WITH_BACKTRACE
stackTraceInit(argList[0]);
#endif
Rename FUNCTION_DEBUG_* and consolidate ASSERT_* macros for consistency. Rename FUNCTION_DEBUG_* macros to FUNCTION_LOG_* to more accurately reflect what they do. Further rename FUNCTION_DEBUG_RESULT* macros to FUNCTION_LOG_RETURN* to make it clearer that they return from the function as well as logging. Leave FUNCTION_TEST_* macros as they are. Consolidate the various ASSERT* macros into a single ASSERT macro that is always compiled out of production builds. It was difficult to figure out when an assert would be checked with all the different types in play. When ASSERTs are compiled in they will always be checked regardless of the log level -- tying these two concepts together was not a good idea.
2019-01-21 17:41:59 +02:00
FUNCTION_LOG_BEGIN(logLevelDebug);
FUNCTION_LOG_PARAM(INT, argListSize);
FUNCTION_LOG_PARAM(CHARPY, argList);
FUNCTION_LOG_END();
Add stack trace macros to all functions. Low-level functions only include stack trace in test builds while higher-level functions ship with stack trace built-in. Stack traces include all parameters passed to the function but production builds only create the parameter list when the log level is set high enough, i.e. debug or trace depending on the function.
2018-05-18 11:57:32 -04:00
Print total runtime in milliseconds at command end.
2018-05-18 12:03:39 -04:00
// Initialize command with the start time
cmdInit();
Add general-purpose statistics collector. Currently each module that needs to collect statistics implements custom code to do so. This is cumbersome. Create a general purpose module for collecting and reporting statistics. Statistics are output in the log at detail level, but there are other uses they could be put to eventually. No new functionality is added. This is just a drop-in replacement for the current statistics, with the advantage of being more flexible. The new stats are slower because they involve a list lookup, but performance testing shows stats can be updated at about 40,000/ms which seems fast enough for our purposes.
2020-08-20 14:04:26 -04:00
// Initialize statistics collector
statInit();
Move async forking and more error handling to C. The Perl process was exiting directly when called but that interfered with proper locking for the forked async process. Now Perl returns results to the C process which handles all errors, including signals.
2018-04-12 20:42:26 -04:00
// Initialize exit handler
exitInit();
Remove exit() calls from main() to make it cleaner and more testable.
2018-02-18 15:45:32 -05:00
Clear error when a CATCH() block finishes. It is better to clear errors after the catch block completes rather than leave them set until the next error. This also make is possible to tell when a error is currently being handled, which a function further down the stack might use to modify its behavior. Currently this is only useful in testing, but clearing the error seems like a good idea in general. Two places used errors outside the CATCH() block. Mem context cleanup now uses a FINALLY() which is a better implementation anyway. The error handling in main() now calls exitSafe() from withing the CATCH() block.
2021-06-23 13:02:19 -04:00
// Process commands
volatile int result = 0;
volatile bool error = false;
Implement version command in C.
2017-11-28 21:44:05 -05:00
TRY_BEGIN()
{
Add log module.
2018-01-16 13:52:20 -05:00
// Load the configuration
// -------------------------------------------------------------------------------------------------------------------------
Enable -Wconversion and silence new warnings.
2018-03-02 12:07:12 -05:00
cfgLoad((unsigned int)argListSize, argList);
Fix options being ignored by asynchronous commands. The local, remote, archive-get-async, and archive-push-async commands were used to run functionality that was not directly available to the user. Unfortunately that meant they would not pick up options from the command that the user expected, e.g. backup, archive-get, etc. Remove the internal commands and add roles which allow pgBackRest to determine what functionality is required without implementing special commands. This way the options are loaded from the expected command section. Since remote is no longer a specific command with its own options, more manipulation is required when calling remote. This might be something we can improve in the config system but it may be worth leaving as is because it is a one-off, for now at least.
2020-01-15 12:24:58 -07:00
ConfigCommandRole commandRole = cfgCommandRole();
Implement version command in C.
2017-11-28 21:44:05 -05:00
Implement help command in C.
2018-01-23 13:34:24 -05:00
// Display help
// -------------------------------------------------------------------------------------------------------------------------
if (cfgCommandHelp())
{
Migrate command-line help generation to C. Command-line help is now generated at build time so it does not need to be committed. This reduces churn on commits that add configuration and/or update the help. Since churn is no longer an issue, help.auto.c is bzip2 compressed to save space in the binary. The Perl config parser (Data.pm) has been moved to doc/lib since the Perl build path is no longer required. Likewise doc/xml/reference.xml has been moved to src/build/help/help.xml since it is required at build time.
2021-09-08 18:16:06 -04:00
cmdHelp(BUF(helpData, sizeof(helpData)));
Implement help command in C.
2018-01-23 13:34:24 -05:00
}
Fix options being ignored by asynchronous commands. The local, remote, archive-get-async, and archive-push-async commands were used to run functionality that was not directly available to the user. Unfortunately that meant they would not pick up options from the command that the user expected, e.g. backup, archive-get, etc. Remove the internal commands and add roles which allow pgBackRest to determine what functionality is required without implementing special commands. This way the options are loaded from the expected command section. Since remote is no longer a specific command with its own options, more manipulation is required when calling remote. This might be something we can improve in the config system but it may be worth leaving as is because it is a one-off, for now at least.
2020-01-15 12:24:58 -07:00
// Local role
// -------------------------------------------------------------------------------------------------------------------------
else if (commandRole == cfgCmdRoleLocal)
{
Refactor cmdLocal()/cmdRemote() to accept a ProtocolServer object. This allows a local/remote to be started independently of server initialization, which will be useful for implementing new transport types, e.g. TLS. Also remove some dead code in localTest.c.
2021-07-23 10:35:36 -04:00
String *name = strNewFmt(PROTOCOL_SERVICE_LOCAL "-%s", strZ(cfgOptionDisplay(cfgOptProcess)));
cmdLocal(
protocolServerNew(
name, PROTOCOL_SERVICE_LOCAL_STR, ioFdReadNewOpen(name, STDIN_FILENO, cfgOptionUInt64(cfgOptProtocolTimeout)),
ioFdWriteNewOpen(name, STDOUT_FILENO, cfgOptionUInt64(cfgOptProtocolTimeout))));
Fix options being ignored by asynchronous commands. The local, remote, archive-get-async, and archive-push-async commands were used to run functionality that was not directly available to the user. Unfortunately that meant they would not pick up options from the command that the user expected, e.g. backup, archive-get, etc. Remove the internal commands and add roles which allow pgBackRest to determine what functionality is required without implementing special commands. This way the options are loaded from the expected command section. Since remote is no longer a specific command with its own options, more manipulation is required when calling remote. This might be something we can improve in the config system but it may be worth leaving as is because it is a one-off, for now at least.
2020-01-15 12:24:58 -07:00
}
// Remote role
// -------------------------------------------------------------------------------------------------------------------------
else if (commandRole == cfgCmdRoleRemote)
{
Refactor cmdLocal()/cmdRemote() to accept a ProtocolServer object. This allows a local/remote to be started independently of server initialization, which will be useful for implementing new transport types, e.g. TLS. Also remove some dead code in localTest.c.
2021-07-23 10:35:36 -04:00
String *name = strNewFmt(PROTOCOL_SERVICE_REMOTE "-%s", strZ(cfgOptionDisplay(cfgOptProcess)));
cmdRemote(
protocolServerNew(
name, PROTOCOL_SERVICE_REMOTE_STR, ioFdReadNewOpen(name, STDIN_FILENO, cfgOptionUInt64(cfgOptProtocolTimeout)),
ioFdWriteNewOpen(name, STDOUT_FILENO, cfgOptionUInt64(cfgOptProtocolTimeout))));
Fix options being ignored by asynchronous commands. The local, remote, archive-get-async, and archive-push-async commands were used to run functionality that was not directly available to the user. Unfortunately that meant they would not pick up options from the command that the user expected, e.g. backup, archive-get, etc. Remove the internal commands and add roles which allow pgBackRest to determine what functionality is required without implementing special commands. This way the options are loaded from the expected command section. Since remote is no longer a specific command with its own options, more manipulation is required when calling remote. This might be something we can improve in the config system but it may be worth leaving as is because it is a one-off, for now at least.
2020-01-15 12:24:58 -07:00
}
Refactor main() as a switch() statement. This makes it clearer what has been migrated and where the new C implementations should go.
2019-04-22 08:47:32 -04:00
else
Move async forking and more error handling to C. The Perl process was exiting directly when called but that interfered with proper locking for the forked async process. Now Perl returns results to the C process which handles all errors, including signals.
2018-04-12 20:42:26 -04:00
{
Refactor main() as a switch() statement. This makes it clearer what has been migrated and where the new C implementations should go.
2019-04-22 08:47:32 -04:00
switch (cfgCommand())
{
// Archive get command
// -----------------------------------------------------------------------------------------------------------------
case cfgCmdArchiveGet:
{
Fix options being ignored by asynchronous commands. The local, remote, archive-get-async, and archive-push-async commands were used to run functionality that was not directly available to the user. Unfortunately that meant they would not pick up options from the command that the user expected, e.g. backup, archive-get, etc. Remove the internal commands and add roles which allow pgBackRest to determine what functionality is required without implementing special commands. This way the options are loaded from the expected command section. Since remote is no longer a specific command with its own options, more manipulation is required when calling remote. This might be something we can improve in the config system but it may be worth leaving as is because it is a one-off, for now at least.
2020-01-15 12:24:58 -07:00
if (commandRole == cfgCmdRoleAsync)
cmdArchiveGetAsync();
else
result = cmdArchiveGet();
Refactor main() as a switch() statement. This makes it clearer what has been migrated and where the new C implementations should go.
2019-04-22 08:47:32 -04:00
break;
}
// Archive push command
// -----------------------------------------------------------------------------------------------------------------
case cfgCmdArchivePush:
{
Fix options being ignored by asynchronous commands. The local, remote, archive-get-async, and archive-push-async commands were used to run functionality that was not directly available to the user. Unfortunately that meant they would not pick up options from the command that the user expected, e.g. backup, archive-get, etc. Remove the internal commands and add roles which allow pgBackRest to determine what functionality is required without implementing special commands. This way the options are loaded from the expected command section. Since remote is no longer a specific command with its own options, more manipulation is required when calling remote. This might be something we can improve in the config system but it may be worth leaving as is because it is a one-off, for now at least.
2020-01-15 12:24:58 -07:00
if (commandRole == cfgCmdRoleAsync)
cmdArchivePushAsync();
else
cmdArchivePush();
Refactor main() as a switch() statement. This makes it clearer what has been migrated and where the new C implementations should go.
2019-04-22 08:47:32 -04:00
break;
}
// Backup command
// -----------------------------------------------------------------------------------------------------------------
case cfgCmdBackup:
{
// Run backup
The backup command is implemented entirely in C. For the most part this is a direct migration of the Perl code into C except as noted below. A backup can now be initiated from a linked directory. The link will not be stored in the manifest or recreated on restore. If a link or directory does not already exist in the restore location then a directory will be created. The logic for creating backup labels has been improved and it should no longer be possible to get a backup label earlier than the latest backup even with timezone changes or clock skew. This has never been an issue in the field that we know of, but we found it in testing. For online backups all times are fetched from the PostgreSQL primary host (before only copy start was). This doesn't affect backup integrity but it does prevent clock skew between hosts affecting backup duration reporting. Archive copy now works as expected when the archive and backup have different compression settings, i.e. when one is compressed and the other is not. This was a long-standing bug in the Perl code. Resume will now work even if hardlink settings have been changed. Reviewed by Cynthia Shang.
2019-12-13 17:14:26 -05:00
cmdBackup();
Refactor main() as a switch() statement. This makes it clearer what has been migrated and where the new C implementations should go.
2019-04-22 08:47:32 -04:00
Add expire-auto option. This allows automatic expiration after a successful backup to be disabled.
2020-07-14 14:12:25 +02:00
if (cfgOptionBool(cfgOptExpireAuto))
{
// Switch to expire command
cmdEnd(0, NULL);
Rename default command role to main. Main makes more sense because we refer to the main process in the code, not the default process. The word default is pretty overloaded anyway.
2021-05-20 14:39:47 -04:00
cfgCommandSet(cfgCmdExpire, cfgCmdRoleMain);
Add expire-auto option. This allows automatic expiration after a successful backup to be disabled.
2020-07-14 14:12:25 +02:00
cfgLoadLogFile();
Remove unused parameter from cmdBegin().
2020-08-20 14:16:36 -04:00
cmdBegin();
Add expire-auto option. This allows automatic expiration after a successful backup to be disabled.
2020-07-14 14:12:25 +02:00
// Run expire
cmdExpire();
}
Refactor main() as a switch() statement. This makes it clearer what has been migrated and where the new C implementations should go.
2019-04-22 08:47:32 -04:00
break;
}
// Check command
// -----------------------------------------------------------------------------------------------------------------
case cfgCmdCheck:
The check command is implemented partly in C. Implement switch WAL and archive check in C but leave the rest in Perl for now. The main idea was to have some real integration tests for the new database code so the rest of the migration can wait. Reviewed by Cynthia Shang.
2019-08-01 20:35:01 -04:00
cmdCheck();
Refactor main() as a switch() statement. This makes it clearer what has been migrated and where the new C implementations should go.
2019-04-22 08:47:32 -04:00
break;
// Expire command
// -----------------------------------------------------------------------------------------------------------------
case cfgCmdExpire:
The expire command is implemented entirely in C. This implementation duplicates the functionality of the Perl code but does so with different logic and includes full unit tests. Along the way at least one bug was fixed, see issue #748. Contributed by Cynthia Shang.
2019-06-18 15:19:20 -04:00
cmdExpire();
Refactor main() as a switch() statement. This makes it clearer what has been migrated and where the new C implementations should go.
2019-04-22 08:47:32 -04:00
break;
// Help command
// -----------------------------------------------------------------------------------------------------------------
case cfgCmdHelp:
case cfgCmdNone:
THROW(AssertError, "'help' and 'none' commands should have been handled already");
// Info command
// -----------------------------------------------------------------------------------------------------------------
case cfgCmdInfo:
cmdInfo();
break;
Add repo-create, repo-get, repo-put, and repo-rm commands. These commands are generally useful but more importantly they allow removing LibC by providing the Perl integration tests an alternate way to work with repository storage. All the commands are currently internal only and should not be used on production repositories.
2020-03-09 17:15:03 -04:00
// Repository create command
// -----------------------------------------------------------------------------------------------------------------
case cfgCmdRepoCreate:
cmdRepoCreate();
break;
// Repository get file command
// -----------------------------------------------------------------------------------------------------------------
case cfgCmdRepoGet:
result = cmdStorageGet();
break;
Rename ls command to repo-ls. This command only makes sense for the repository storage since other storage (e.g. pg and spool) must be located on a local Posix filesystem and can be listed using standard unix commands. Since the repo storage can be located lots of places having a common way to list it makes sense. Prefix with repo- to make the scope of this command clear. Update documentation to reflect this change.
2020-03-09 16:41:04 -04:00
// Repository list paths/files command
// -----------------------------------------------------------------------------------------------------------------
case cfgCmdRepoLs:
cmdStorageList();
break;
Add repo-create, repo-get, repo-put, and repo-rm commands. These commands are generally useful but more importantly they allow removing LibC by providing the Perl integration tests an alternate way to work with repository storage. All the commands are currently internal only and should not be used on production repositories.
2020-03-09 17:15:03 -04:00
// Repository put file command
// -----------------------------------------------------------------------------------------------------------------
case cfgCmdRepoPut:
cmdStoragePut();
break;
// Repository remove paths/files command
// -----------------------------------------------------------------------------------------------------------------
case cfgCmdRepoRm:
cmdStorageRemove();
break;
Rename server-start command to server.
2021-11-18 17:23:11 -05:00
// Server command
Add TLS Server. The TLS server is an alternative to using SSH for protocol connections to remote hosts. This command is currently experimental and intended only for trial and testing. As such, the new commands and options will not show up in the command-line help unless directly requested.
2021-10-18 14:32:41 -04:00
// -----------------------------------------------------------------------------------------------------------------
Rename server-start command to server.
2021-11-18 17:23:11 -05:00
case cfgCmdServer:
Add SIGTERM and SIGHUP handling to TLS server. SIGHUP allows the configuration to be reloaded. Note that the configuration will not be updated in child processes that have already started. SIGTERM terminates the server process gracefully and sends SIGTERM to all child processes. This also gives the tests an easy way to stop the server.
2021-12-07 18:18:43 -05:00
cmdServer((unsigned int)argListSize, argList);
Add TLS Server. The TLS server is an alternative to using SSH for protocol connections to remote hosts. This command is currently experimental and intended only for trial and testing. As such, the new commands and options will not show up in the command-line help unless directly requested.
2021-10-18 14:32:41 -04:00
break;
// Server ping command
// -----------------------------------------------------------------------------------------------------------------
case cfgCmdServerPing:
cmdServerPing();
break;
Refactor main() as a switch() statement. This makes it clearer what has been migrated and where the new C implementations should go.
2019-04-22 08:47:32 -04:00
// Restore command
// -----------------------------------------------------------------------------------------------------------------
case cfgCmdRestore:
The restore command is implemented entirely in C. For the most part this is a direct migration of the Perl code into C. There is one important behavioral change with regard to how file permissions are handled. The Perl code tried to set ownership as it was in the manifest even when running as an unprivileged user. This usually just led to errors and frustration. The C code works like this: If a restore is run as a non-root user (the typical scenario) then all files restored will belong to the user/group executing pgBackRest. If existing files are not owned by the executing user/group then an error will result if the ownership cannot be updated to the executing user/group. In that case the file ownership will need to be updated by a privileged user before the restore can be retried. If a restore is run as the root user then pgBackRest will attempt to recreate the ownership recorded in the manifest when the backup was made. Only user/group names are stored in the manifest so the same names must exist on the restore host for this to work. If the user/group name cannot be found locally then the user/group of the PostgreSQL data directory will be used and finally root if the data directory user/group cannot be mapped to a name. Reviewed by Cynthia Shang.
2019-09-26 07:52:02 -04:00
cmdRestore();
Refactor main() as a switch() statement. This makes it clearer what has been migrated and where the new C implementations should go.
2019-04-22 08:47:32 -04:00
break;
// Stanza create command
// -----------------------------------------------------------------------------------------------------------------
case cfgCmdStanzaCreate:
The stanza-create/update/delete commands are implemented entirely in C. Contributed by Cynthia Shang.
2019-08-21 16:26:28 -04:00
cmdStanzaCreate();
Refactor main() as a switch() statement. This makes it clearer what has been migrated and where the new C implementations should go.
2019-04-22 08:47:32 -04:00
break;
// Stanza delete command
// -----------------------------------------------------------------------------------------------------------------
case cfgCmdStanzaDelete:
The stanza-create/update/delete commands are implemented entirely in C. Contributed by Cynthia Shang.
2019-08-21 16:26:28 -04:00
cmdStanzaDelete();
Refactor main() as a switch() statement. This makes it clearer what has been migrated and where the new C implementations should go.
2019-04-22 08:47:32 -04:00
break;
// Stanza upgrade command
// -----------------------------------------------------------------------------------------------------------------
case cfgCmdStanzaUpgrade:
The stanza-create/update/delete commands are implemented entirely in C. Contributed by Cynthia Shang.
2019-08-21 16:26:28 -04:00
cmdStanzaUpgrade();
Refactor main() as a switch() statement. This makes it clearer what has been migrated and where the new C implementations should go.
2019-04-22 08:47:32 -04:00
break;
// Start command
// -----------------------------------------------------------------------------------------------------------------
case cfgCmdStart:
The start/stop commands are implemented entirely in C. The Perl versions remain because they are still being used by the Perl stanza commands. Once the stanza commands are migrated they can be removed. Contributed by Cynthia Shang.
2019-08-09 15:17:18 -04:00
cmdStart();
Refactor main() as a switch() statement. This makes it clearer what has been migrated and where the new C implementations should go.
2019-04-22 08:47:32 -04:00
break;
// Stop command
// -----------------------------------------------------------------------------------------------------------------
case cfgCmdStop:
The start/stop commands are implemented entirely in C. The Perl versions remain because they are still being used by the Perl stanza commands. Once the stanza commands are migrated they can be removed. Contributed by Cynthia Shang.
2019-08-09 15:17:18 -04:00
cmdStop();
Refactor main() as a switch() statement. This makes it clearer what has been migrated and where the new C implementations should go.
2019-04-22 08:47:32 -04:00
break;
Add internal verify command. Scan the WAL archive for missing or invalid files and build up ranges of WAL that will be used to verify backup integrity. A number of errors and warnings are currently emitted but they should not be considered authoritative (yet). The command is incomplete so is marked internal.
2020-09-22 11:57:38 -04:00
// Verify command
// -----------------------------------------------------------------------------------------------------------------
case cfgCmdVerify:
cmdVerify();
break;
Refactor main() as a switch() statement. This makes it clearer what has been migrated and where the new C implementations should go.
2019-04-22 08:47:32 -04:00
// Display version
// -----------------------------------------------------------------------------------------------------------------
case cfgCmdVersion:
printf(PROJECT_NAME " " PROJECT_VERSION "\n");
fflush(stdout);
break;
}
Remove exit() calls from main() to make it cleaner and more testable.
2018-02-18 15:45:32 -05:00
}
Implement version command in C.
2017-11-28 21:44:05 -05:00
}
CATCH_ANY()
{
Remove exit() calls from main() to make it cleaner and more testable.
2018-02-18 15:45:32 -05:00
error = true;
Clear error when a CATCH() block finishes. It is better to clear errors after the catch block completes rather than leave them set until the next error. This also make is possible to tell when a error is currently being handled, which a function further down the stack might use to modify its behavior. Currently this is only useful in testing, but clearing the error seems like a good idea in general. Two places used errors outside the CATCH() block. Mem context cleanup now uses a FINALLY() which is a better implementation anyway. The error handling in main() now calls exitSafe() from withing the CATCH() block.
2021-06-23 13:02:19 -04:00
result = exitSafe(result, true, 0);
Implement version command in C.
2017-11-28 21:44:05 -05:00
}
TRY_END();
Add command and exit modules.
2018-01-17 09:15:51 -05:00
Clear error when a CATCH() block finishes. It is better to clear errors after the catch block completes rather than leave them set until the next error. This also make is possible to tell when a error is currently being handled, which a function further down the stack might use to modify its behavior. Currently this is only useful in testing, but clearing the error seems like a good idea in general. Two places used errors outside the CATCH() block. Mem context cleanup now uses a FINALLY() which is a better implementation anyway. The error handling in main() now calls exitSafe() from withing the CATCH() block.
2021-06-23 13:02:19 -04:00
FUNCTION_LOG_RETURN(INT, error ? result : exitSafe(result, false, 0));
The pgbackrest executable is now a C binary instead of Perl. This allows certain time-critical commands (like async archive-push) to run more quickly.
2017-11-26 18:43:51 -05:00
}
Reference in New Issue Copy Permalink