db/pgbackrest
1
0
Fork 0
mirror of https://github.com/pgbackrest/pgbackrest.git synced 2025-03-03 14:52:21 +02:00
Code Issues Releases Activity
pgbackrest/build/lib/pgBackRestBuild/Config/BuildParse.pm

631 lines
23 KiB
Perl
Raw Normal View History

Implement version command in C.
2017-11-28 21:44:05 -05:00
####################################################################################################################################
# Auto-Generate Option Definition for Parsing with getopt_long().
####################################################################################################################################
package pgBackRestBuild::Config::BuildParse;
use strict;
use warnings FATAL => qw(all);
use Carp qw(confess);
use English '-no_match_vars';
use Cwd qw(abs_path);
use Exporter qw(import);
our @EXPORT = qw();
use File::Basename qw(dirname);
use Storable qw(dclone);
Rename BackRestDoc Perl module to pgBackRestDoc. This is consistent with the way BackRest and BackRest test were renamed way back in 18fd2523. More modules will be moving to pgBackRestDoc soon so renaming now reduces churn later.
2020-03-10 15:41:56 -04:00
use pgBackRestDoc::Common::Log;
use pgBackRestDoc::Common::String;
Move pgBackRest::Version module to pgBackRestDoc::ProjectInfo. The primary source for project info is now src/version.h. The pgBackRestDoc::ProjectInfo module loads the project info from src/version.h at runtime so there is no need to update it.
2020-03-10 17:57:02 -04:00
use pgBackRestDoc::ProjectInfo;
Move Perl modules out of lib directory. This directory was once the home of the production Perl code but since f0ef73db this is no longer true. Move the modules to test in most cases, except where the module is expected to be useful for the doc engine beyond the expected lifetime of the Perl test code (about a year if all goes well). The exception is pgBackRest::Version which requires more work to migrate since it is used to track pgBackRest versions.
2020-03-10 15:12:44 -04:00
Implement version command in C.
2017-11-28 21:44:05 -05:00
use pgBackRestBuild::Build::Common;
use pgBackRestBuild::Config::Build;
use pgBackRestBuild::Config::Data;
####################################################################################################################################
# Constants
####################################################################################################################################
use constant BLDLCL_FILE_DEFINE => 'parse';
Move all parse-related rules to parse module. Data required for parsing was spread between the config and defined modules, mostly for historical reasons because the same data was used by Perl. Requiring all the parse rules to be accessed with function interfaces makes the code more complicated and new rules harder to implement. Instead, move the data to the parse module so in the most complex cases no interface functions are needed. This reduces the total amount of code and paves the way for more complex parse rules.
2020-12-17 09:32:31 -05:00
use constant BLDLCL_ENUM_OPTION_TYPE => '01-enumOptionType';
use constant BLDLCL_DATA_COMMAND => '01-dataCommand';
use constant BLDLCL_DATA_OPTION_GROUP => '01-dataOptionGroup';
use constant BLDLCL_DATA_OPTION => '02-dataOption';
use constant BLDLCL_DATA_OPTION_GETOPT => '03-dataOptionGetOpt';
use constant BLDLCL_DATA_OPTION_RESOLVE => '04-dataOptionResolve';
Implement version command in C.
2017-11-28 21:44:05 -05:00
####################################################################################################################################
# Definitions for constants and data to build
####################################################################################################################################
Move all parse-related rules to parse module. Data required for parsing was spread between the config and defined modules, mostly for historical reasons because the same data was used by Perl. Requiring all the parse rules to be accessed with function interfaces makes the code more complicated and new rules harder to implement. Instead, move the data to the parse module so in the most complex cases no interface functions are needed. This reduces the total amount of code and paves the way for more complex parse rules.
2020-12-17 09:32:31 -05:00
my $strSummary = 'Config Parse Rules';
Implement version command in C.
2017-11-28 21:44:05 -05:00
my $rhBuild =
{
&BLD_FILE =>
{
&BLDLCL_FILE_DEFINE =>
{
&BLD_SUMMARY => $strSummary,
Move all parse-related rules to parse module. Data required for parsing was spread between the config and defined modules, mostly for historical reasons because the same data was used by Perl. Requiring all the parse rules to be accessed with function interfaces makes the code more complicated and new rules harder to implement. Instead, move the data to the parse module so in the most complex cases no interface functions are needed. This reduces the total amount of code and paves the way for more complex parse rules.
2020-12-17 09:32:31 -05:00
&BLD_ENUM =>
{
&BLDLCL_ENUM_OPTION_TYPE =>
{
&BLD_SUMMARY => 'Option type',
&BLD_NAME => 'ConfigOptionType',
},
},
Implement version command in C.
2017-11-28 21:44:05 -05:00
&BLD_DATA =>
{
Move all parse-related rules to parse module. Data required for parsing was spread between the config and defined modules, mostly for historical reasons because the same data was used by Perl. Requiring all the parse rules to be accessed with function interfaces makes the code more complicated and new rules harder to implement. Instead, move the data to the parse module so in the most complex cases no interface functions are needed. This reduces the total amount of code and paves the way for more complex parse rules.
2020-12-17 09:32:31 -05:00
&BLDLCL_DATA_COMMAND =>
{
&BLD_SUMMARY => 'Command parse data',
},
&BLDLCL_DATA_OPTION_GROUP =>
{
&BLD_SUMMARY => 'Option group parse data',
},
Implement version command in C.
2017-11-28 21:44:05 -05:00
&BLDLCL_DATA_OPTION =>
{
&BLD_SUMMARY => 'Option parse data',
},
Validate configuration options in a single pass. By pre-calculating and storing the option dependencies in parse.auto.c validation can be completed in a single pass, which is both simpler and faster.
2018-08-11 12:55:33 -04:00
Move all parse-related rules to parse module. Data required for parsing was spread between the config and defined modules, mostly for historical reasons because the same data was used by Perl. Requiring all the parse rules to be accessed with function interfaces makes the code more complicated and new rules harder to implement. Instead, move the data to the parse module so in the most complex cases no interface functions are needed. This reduces the total amount of code and paves the way for more complex parse rules.
2020-12-17 09:32:31 -05:00
&BLDLCL_DATA_OPTION_GETOPT =>
{
&BLD_SUMMARY => 'Option data for getopt_long()',
},
Validate configuration options in a single pass. By pre-calculating and storing the option dependencies in parse.auto.c validation can be completed in a single pass, which is both simpler and faster.
2018-08-11 12:55:33 -04:00
&BLDLCL_DATA_OPTION_RESOLVE =>
{
&BLD_SUMMARY => 'Order for option parse resolution',
},
Implement version command in C.
2017-11-28 21:44:05 -05:00
},
},
},
};
Move all parse-related rules to parse module. Data required for parsing was spread between the config and defined modules, mostly for historical reasons because the same data was used by Perl. Requiring all the parse rules to be accessed with function interfaces makes the code more complicated and new rules harder to implement. Instead, move the data to the parse module so in the most complex cases no interface functions are needed. This reduces the total amount of code and paves the way for more complex parse rules.
2020-12-17 09:32:31 -05:00
####################################################################################################################################
# Generate enum names
####################################################################################################################################
sub buildConfigDefineOptionTypeEnum
{
return bldEnum('cfgOptType', shift);
}
push @EXPORT, qw(buildConfigDefineOptionTypeEnum);
Allow option validity to be determined by command role. Validity by command was not granular enough so numerous options needed be marked internal so users would not stumble across them. Options were also needlessly being passed to roles that had no use for them. Introduce per-role validity lists that depend on what roles are valid per command. Also add a check to ensure that only valid roles are used with a command. This commit adds the functionality but does not introduce any new behavior, i.e. all options are valid for all roles that the command is valid for. A subsequent commit will introduce the new role restrictions to make the changes easier to audit.
2020-12-28 09:43:23 -05:00
sub buildConfigCommandRoleEnum
{
return bldEnum('cfgCmdRole', shift);
}
Move all parse-related rules to parse module. Data required for parsing was spread between the config and defined modules, mostly for historical reasons because the same data was used by Perl. Requiring all the parse rules to be accessed with function interfaces makes the code more complicated and new rules harder to implement. Instead, move the data to the parse module so in the most complex cases no interface functions are needed. This reduces the total amount of code and paves the way for more complex parse rules.
2020-12-17 09:32:31 -05:00
####################################################################################################################################
# Helper functions for building optional option data
####################################################################################################################################
sub renderAllowList
{
my $ryAllowList = shift;
my $bCommandIndent = shift;
my $strIndent = $bCommandIndent ? ' ' : '';
return
"${strIndent} PARSE_RULE_OPTION_OPTIONAL_ALLOW_LIST\n" .
"${strIndent} (\n" .
"${strIndent} " . join(",\n${strIndent} ", bldQuoteList($ryAllowList)) .
"\n" .
"${strIndent} ),\n";
}
sub renderDepend
{
my $rhDepend = shift;
my $bCommandIndent = shift;
my $strIndent = $bCommandIndent ? ' ' : '';
my $strDependOption = $rhDepend->{&CFGDEF_DEPEND_OPTION};
my $ryDependList = $rhDepend->{&CFGDEF_DEPEND_LIST};
if (defined($ryDependList))
{
my @stryQuoteList;
foreach my $strItem (@{$ryDependList})
{
push(@stryQuoteList, "\"${strItem}\"");
}
return
"${strIndent} PARSE_RULE_OPTION_OPTIONAL_DEPEND_LIST\n" .
"${strIndent} (\n" .
"${strIndent} " . buildConfigOptionEnum($strDependOption) . ",\n" .
"${strIndent} " . join(",\n${strIndent} ", bldQuoteList($ryDependList)) .
"\n" .
"${strIndent} ),\n";
}
return
"${strIndent} PARSE_RULE_OPTION_OPTIONAL_DEPEND(" . buildConfigOptionEnum($strDependOption) . "),\n";
}
sub renderOptional
{
my $rhOptional = shift;
my $bCommand = shift;
my $strCommand = shift;
my $strOption = shift;
my $strIndent = $bCommand ? ' ' : '';
my $strBuildSourceOptional;
my $bSingleLine = false;
if (defined($rhOptional->{&CFGDEF_ALLOW_LIST}))
{
$strBuildSourceOptional .=
(defined($strBuildSourceOptional) && !$bSingleLine ? "\n" : '') .
renderAllowList($rhOptional->{&CFGDEF_ALLOW_LIST}, $bCommand);
$bSingleLine = false;
}
if (defined($rhOptional->{&CFGDEF_ALLOW_RANGE}))
{
my @fyRange = @{$rhOptional->{&CFGDEF_ALLOW_RANGE}};
my $iMultiplier = $rhOptional->{&CFGDEF_TYPE} eq CFGDEF_TYPE_TIME ? 1000 : 1;
$strBuildSourceOptional =
(defined($strBuildSourceOptional) && !$bSingleLine ? "\n" : '') .
"${strIndent} PARSE_RULE_OPTION_OPTIONAL_ALLOW_RANGE(" . ($fyRange[0] * $iMultiplier) . ', ' .
($fyRange[1] * $iMultiplier) . "),\n";
$bSingleLine = true;
}
if (defined($rhOptional->{&CFGDEF_DEPEND}))
{
$strBuildSourceOptional .=
(defined($strBuildSourceOptional) && !$bSingleLine ? "\n" : '') .
renderDepend($rhOptional->{&CFGDEF_DEPEND}, $bCommand);
$bSingleLine = defined($rhOptional->{&CFGDEF_DEPEND}{&CFGDEF_DEPEND_LIST}) ? false : true;
}
if (defined($rhOptional->{&CFGDEF_DEFAULT}))
{
$strBuildSourceOptional .=
(defined($strBuildSourceOptional) && !$bSingleLine ? "\n" : '') .
Set config path in configure script. This allows the config path to be modified with a parameter to the configure script, though this commit does not do that. Update the Perl code generator to allow literals so that defaults can be C defines rather than static strings.
2021-02-10 14:46:26 -05:00
"${strIndent} PARSE_RULE_OPTION_OPTIONAL_DEFAULT(" .
($rhOptional->{&CFGDEF_DEFAULT_LITERAL} ? '' : '"') .
Move all parse-related rules to parse module. Data required for parsing was spread between the config and defined modules, mostly for historical reasons because the same data was used by Perl. Requiring all the parse rules to be accessed with function interfaces makes the code more complicated and new rules harder to implement. Instead, move the data to the parse module so in the most complex cases no interface functions are needed. This reduces the total amount of code and paves the way for more complex parse rules.
2020-12-17 09:32:31 -05:00
(defined($rhOptional->{&CFGDEF_TYPE}) && $rhOptional->{&CFGDEF_TYPE} eq CFGDEF_TYPE_TIME ?
$rhOptional->{&CFGDEF_DEFAULT} * 1000 : $rhOptional->{&CFGDEF_DEFAULT}) .
Set config path in configure script. This allows the config path to be modified with a parameter to the configure script, though this commit does not do that. Update the Perl code generator to allow literals so that defaults can be C defines rather than static strings.
2021-02-10 14:46:26 -05:00
($rhOptional->{&CFGDEF_DEFAULT_LITERAL} ? '' : '"') .
"),\n";
Move all parse-related rules to parse module. Data required for parsing was spread between the config and defined modules, mostly for historical reasons because the same data was used by Perl. Requiring all the parse rules to be accessed with function interfaces makes the code more complicated and new rules harder to implement. Instead, move the data to the parse module so in the most complex cases no interface functions are needed. This reduces the total amount of code and paves the way for more complex parse rules.
2020-12-17 09:32:31 -05:00
$bSingleLine = true;
}
if ($bCommand && defined($rhOptional->{&CFGDEF_REQUIRED}))
{
$strBuildSourceOptional .=
(defined($strBuildSourceOptional) && !$bSingleLine ? "\n" : '') .
"${strIndent} PARSE_RULE_OPTION_OPTIONAL_REQUIRED(" .
($rhOptional->{&CFGDEF_REQUIRED} ? 'true' : 'false') . "),\n";
$bSingleLine = true;
}
return $strBuildSourceOptional;
}
Implement version command in C.
2017-11-28 21:44:05 -05:00
####################################################################################################################################
# Build configuration constants and data
####################################################################################################################################
sub buildConfigParse
{
Move all parse-related rules to parse module. Data required for parsing was spread between the config and defined modules, mostly for historical reasons because the same data was used by Perl. Requiring all the parse rules to be accessed with function interfaces makes the code more complicated and new rules harder to implement. Instead, move the data to the parse module so in the most complex cases no interface functions are needed. This reduces the total amount of code and paves the way for more complex parse rules.
2020-12-17 09:32:31 -05:00
# Build option type enum
# ------------------------------------------------------------------------------------------------------------------------------
my $rhEnum = $rhBuild->{&BLD_FILE}{&BLDLCL_FILE_DEFINE}{&BLD_ENUM}{&BLDLCL_ENUM_OPTION_TYPE};
foreach my $strOptionType (cfgDefineOptionTypeList())
{
# Build C enum
my $strOptionTypeEnum = buildConfigDefineOptionTypeEnum($strOptionType);
push(@{$rhEnum->{&BLD_LIST}}, $strOptionTypeEnum);
};
# Build command parse data
# ------------------------------------------------------------------------------------------------------------------------------
my $rhCommandDefine = cfgDefineCommand();
Implement version command in C.
2017-11-28 21:44:05 -05:00
my $strBuildSource =
Move all parse-related rules to parse module. Data required for parsing was spread between the config and defined modules, mostly for historical reasons because the same data was used by Perl. Requiring all the parse rules to be accessed with function interfaces makes the code more complicated and new rules harder to implement. Instead, move the data to the parse module so in the most complex cases no interface functions are needed. This reduces the total amount of code and paves the way for more complex parse rules.
2020-12-17 09:32:31 -05:00
"static const ParseRuleCommand parseRuleCommand[CFG_COMMAND_TOTAL] =\n" .
"{";
foreach my $strCommand (sort(keys(%{$rhCommandDefine})))
{
my $rhCommand = $rhCommandDefine->{$strCommand};
# Build command data
$strBuildSource .=
"\n" .
" //" . (qw{-} x 126) . "\n" .
" PARSE_RULE_COMMAND\n" .
" (\n" .
" PARSE_RULE_COMMAND_NAME(\"${strCommand}\"),\n";
if ($rhCommand->{&CFGDEF_PARAMETER_ALLOWED})
{
$strBuildSource .=
" PARSE_RULE_COMMAND_PARAMETER_ALLOWED(true),\n";
}
Allow option validity to be determined by command role. Validity by command was not granular enough so numerous options needed be marked internal so users would not stumble across them. Options were also needlessly being passed to roles that had no use for them. Introduce per-role validity lists that depend on what roles are valid per command. Also add a check to ensure that only valid roles are used with a command. This commit adds the functionality but does not introduce any new behavior, i.e. all options are valid for all roles that the command is valid for. A subsequent commit will introduce the new role restrictions to make the changes easier to audit.
2020-12-28 09:43:23 -05:00
$strBuildSource .=
"\n" .
" PARSE_RULE_COMMAND_ROLE_VALID_LIST\n" .
" (\n";
foreach my $strCommandRole (sort(keys(%{$rhCommand->{&CFGDEF_COMMAND_ROLE}})))
{
$strBuildSource .=
" PARSE_RULE_COMMAND_ROLE(" . buildConfigCommandRoleEnum($strCommandRole) . ")\n";
}
$strBuildSource .=
" ),\n";
Move all parse-related rules to parse module. Data required for parsing was spread between the config and defined modules, mostly for historical reasons because the same data was used by Perl. Requiring all the parse rules to be accessed with function interfaces makes the code more complicated and new rules harder to implement. Instead, move the data to the parse module so in the most complex cases no interface functions are needed. This reduces the total amount of code and paves the way for more complex parse rules.
2020-12-17 09:32:31 -05:00
$strBuildSource .=
" ),\n";
};
$strBuildSource .=
"};\n";
$rhBuild->{&BLD_FILE}{&BLDLCL_FILE_DEFINE}{&BLD_DATA}{&BLDLCL_DATA_COMMAND}{&BLD_SOURCE} = $strBuildSource;
# Build option group parse data
# ------------------------------------------------------------------------------------------------------------------------------
my $rhOptionGroupDefine = cfgDefineOptionGroup();
$strBuildSource =
"static const ParseRuleOptionGroup parseRuleOptionGroup[CFG_OPTION_GROUP_TOTAL] =\n" .
"{";
foreach my $strGroup (sort(keys(%{$rhOptionGroupDefine})))
{
$strBuildSource .=
"\n" .
" //" . (qw{-} x 126) . "\n" .
" PARSE_RULE_OPTION_GROUP\n" .
" (\n" .
" PARSE_RULE_OPTION_GROUP_NAME(\"" . $strGroup . "\"),\n" .
" ),\n";
}
$strBuildSource .=
"};\n";
$rhBuild->{&BLD_FILE}{&BLDLCL_FILE_DEFINE}{&BLD_DATA}{&BLDLCL_DATA_OPTION_GROUP}{&BLD_SOURCE} = $strBuildSource;
# Build option parse data
# ------------------------------------------------------------------------------------------------------------------------------
my $rhConfigDefine = cfgDefine();
$strBuildSource =
"static const ParseRuleOption parseRuleOption[CFG_OPTION_TOTAL] =\n" .
"{";
foreach my $strOption (sort(keys(%{$rhConfigDefine})))
{
my $rhOption = $rhConfigDefine->{$strOption};
$strBuildSource .=
"\n" .
" // " . (qw{-} x 125) . "\n" .
" PARSE_RULE_OPTION\n" .
" (\n" .
" PARSE_RULE_OPTION_NAME(\"${strOption}\"),\n" .
" PARSE_RULE_OPTION_TYPE(" . buildConfigDefineOptionTypeEnum($rhOption->{&CFGDEF_TYPE}) . "),\n" .
" PARSE_RULE_OPTION_REQUIRED(" . ($rhOption->{&CFGDEF_REQUIRED} ? 'true' : 'false') . "),\n" .
" PARSE_RULE_OPTION_SECTION(cfgSection" .
(defined($rhOption->{&CFGDEF_SECTION}) ? ucfirst($rhOption->{&CFGDEF_SECTION}) : 'CommandLine') .
"),\n";
if ($rhOption->{&CFGDEF_SECURE})
{
$strBuildSource .=
" PARSE_RULE_OPTION_SECURE(true),\n";
}
if ($rhOption->{&CFGDEF_TYPE} eq CFGDEF_TYPE_HASH || $rhOption->{&CFGDEF_TYPE} eq CFGDEF_TYPE_LIST)
{
$strBuildSource .=
" PARSE_RULE_OPTION_MULTI(true),\n";
}
# Build group info
# --------------------------------------------------------------------------------------------------------------------------
if ($rhOption->{&CFGDEF_GROUP})
{
$strBuildSource .=
" PARSE_RULE_OPTION_GROUP_MEMBER(true),\n" .
" PARSE_RULE_OPTION_GROUP_ID(" . buildConfigOptionGroupEnum($rhOption->{&CFGDEF_GROUP}) . "),\n";
}
Allow option validity to be determined by command role. Validity by command was not granular enough so numerous options needed be marked internal so users would not stumble across them. Options were also needlessly being passed to roles that had no use for them. Introduce per-role validity lists that depend on what roles are valid per command. Also add a check to ensure that only valid roles are used with a command. This commit adds the functionality but does not introduce any new behavior, i.e. all options are valid for all roles that the command is valid for. A subsequent commit will introduce the new role restrictions to make the changes easier to audit.
2020-12-28 09:43:23 -05:00
# Build command role valid lists
#---------------------------------------------------------------------------------------------------------------------------
Move all parse-related rules to parse module. Data required for parsing was spread between the config and defined modules, mostly for historical reasons because the same data was used by Perl. Requiring all the parse rules to be accessed with function interfaces makes the code more complicated and new rules harder to implement. Instead, move the data to the parse module so in the most complex cases no interface functions are needed. This reduces the total amount of code and paves the way for more complex parse rules.
2020-12-17 09:32:31 -05:00
my $strBuildSourceSub = "";
Rename default command role to main. Main makes more sense because we refer to the main process in the code, not the default process. The word default is pretty overloaded anyway.
2021-05-20 14:39:47 -04:00
foreach my $strCommandRole (CFGCMD_ROLE_MAIN, CFGCMD_ROLE_ASYNC, CFGCMD_ROLE_LOCAL, CFGCMD_ROLE_REMOTE)
Move all parse-related rules to parse module. Data required for parsing was spread between the config and defined modules, mostly for historical reasons because the same data was used by Perl. Requiring all the parse rules to be accessed with function interfaces makes the code more complicated and new rules harder to implement. Instead, move the data to the parse module so in the most complex cases no interface functions are needed. This reduces the total amount of code and paves the way for more complex parse rules.
2020-12-17 09:32:31 -05:00
{
Allow option validity to be determined by command role. Validity by command was not granular enough so numerous options needed be marked internal so users would not stumble across them. Options were also needlessly being passed to roles that had no use for them. Introduce per-role validity lists that depend on what roles are valid per command. Also add a check to ensure that only valid roles are used with a command. This commit adds the functionality but does not introduce any new behavior, i.e. all options are valid for all roles that the command is valid for. A subsequent commit will introduce the new role restrictions to make the changes easier to audit.
2020-12-28 09:43:23 -05:00
$strBuildSourceSub = "";
foreach my $strCommand (cfgDefineCommandList())
Move all parse-related rules to parse module. Data required for parsing was spread between the config and defined modules, mostly for historical reasons because the same data was used by Perl. Requiring all the parse rules to be accessed with function interfaces makes the code more complicated and new rules harder to implement. Instead, move the data to the parse module so in the most complex cases no interface functions are needed. This reduces the total amount of code and paves the way for more complex parse rules.
2020-12-17 09:32:31 -05:00
{
Allow option validity to be determined by command role. Validity by command was not granular enough so numerous options needed be marked internal so users would not stumble across them. Options were also needlessly being passed to roles that had no use for them. Introduce per-role validity lists that depend on what roles are valid per command. Also add a check to ensure that only valid roles are used with a command. This commit adds the functionality but does not introduce any new behavior, i.e. all options are valid for all roles that the command is valid for. A subsequent commit will introduce the new role restrictions to make the changes easier to audit.
2020-12-28 09:43:23 -05:00
if (defined($rhOption->{&CFGDEF_COMMAND}{$strCommand}))
{
Limit option validity by command role. Building on 23f5712, limit option validity by role. This is mostly for options that weren't needed for certain roles but were harmless. However, the upcoming multi repository functionality requires the granularity implemented here. The remote role benefits since host options can automatically excluded when building the options. Also, many options that are only required for the default role (e.g. repo-retention-full) no longer need to be passed in tests for other roles.
2020-12-29 15:49:37 -05:00
if (defined($rhOption->{&CFGDEF_COMMAND}{$strCommand}{&CFGDEF_COMMAND_ROLE}{$strCommandRole}))
Allow option validity to be determined by command role. Validity by command was not granular enough so numerous options needed be marked internal so users would not stumble across them. Options were also needlessly being passed to roles that had no use for them. Introduce per-role validity lists that depend on what roles are valid per command. Also add a check to ensure that only valid roles are used with a command. This commit adds the functionality but does not introduce any new behavior, i.e. all options are valid for all roles that the command is valid for. A subsequent commit will introduce the new role restrictions to make the changes easier to audit.
2020-12-28 09:43:23 -05:00
{
$strBuildSourceSub .=
" PARSE_RULE_OPTION_COMMAND(" . buildConfigCommandEnum($strCommand) . ")\n";
}
}
Move all parse-related rules to parse module. Data required for parsing was spread between the config and defined modules, mostly for historical reasons because the same data was used by Perl. Requiring all the parse rules to be accessed with function interfaces makes the code more complicated and new rules harder to implement. Instead, move the data to the parse module so in the most complex cases no interface functions are needed. This reduces the total amount of code and paves the way for more complex parse rules.
2020-12-17 09:32:31 -05:00
}
Allow option validity to be determined by command role. Validity by command was not granular enough so numerous options needed be marked internal so users would not stumble across them. Options were also needlessly being passed to roles that had no use for them. Introduce per-role validity lists that depend on what roles are valid per command. Also add a check to ensure that only valid roles are used with a command. This commit adds the functionality but does not introduce any new behavior, i.e. all options are valid for all roles that the command is valid for. A subsequent commit will introduce the new role restrictions to make the changes easier to audit.
2020-12-28 09:43:23 -05:00
if ($strBuildSourceSub ne "")
{
$strBuildSource .=
"\n" .
" PARSE_RULE_OPTION_COMMAND_ROLE_" . uc($strCommandRole) . "_VALID_LIST\n" .
" (\n" .
$strBuildSourceSub .
" ),\n";
}
Move all parse-related rules to parse module. Data required for parsing was spread between the config and defined modules, mostly for historical reasons because the same data was used by Perl. Requiring all the parse rules to be accessed with function interfaces makes the code more complicated and new rules harder to implement. Instead, move the data to the parse module so in the most complex cases no interface functions are needed. This reduces the total amount of code and paves the way for more complex parse rules.
2020-12-17 09:32:31 -05:00
}
# Render optional data and command overrides
# --------------------------------------------------------------------------------------------------------------------------
my $strBuildSourceOptional = renderOptional($rhOption, false);
foreach my $strCommand (cfgDefineCommandList())
{
my $strBuildSourceOptionalCommand;
my $rhCommand = $rhOption->{&CFGDEF_COMMAND}{$strCommand};
if (defined($rhCommand))
{
$strBuildSourceOptionalCommand = renderOptional($rhCommand, true, $strCommand, $strOption);
if (defined($strBuildSourceOptionalCommand))
{
$strBuildSourceOptional .=
(defined($strBuildSourceOptional) ? "\n" : '') .
" PARSE_RULE_OPTION_OPTIONAL_COMMAND_OVERRIDE\n" .
" (\n" .
" PARSE_RULE_OPTION_OPTIONAL_COMMAND(" . buildConfigCommandEnum($strCommand) . "),\n" .
"\n" .
$strBuildSourceOptionalCommand .
" )\n";
}
}
}
if (defined($strBuildSourceOptional))
{
$strBuildSource .=
"\n" .
" PARSE_RULE_OPTION_OPTIONAL_LIST\n" .
" (\n" .
$strBuildSourceOptional .
" ),\n";
}
$strBuildSource .=
" ),\n";
}
$strBuildSource .=
"};\n";
$rhBuild->{&BLD_FILE}{&BLDLCL_FILE_DEFINE}{&BLD_DATA}{&BLDLCL_DATA_OPTION}{&BLD_SOURCE} = $strBuildSource;
# Build option list for getopt_long()
#-------------------------------------------------------------------------------------------------------------------------------
$strBuildSource =
Implement version command in C.
2017-11-28 21:44:05 -05:00
"static const struct option optionList[] =\n" .
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-03 18:27:38 -05:00
"{";
Implement version command in C.
2017-11-28 21:44:05 -05:00
foreach my $strOption (sort(keys(%{$rhConfigDefine})))
{
my $rhOption = $rhConfigDefine->{$strOption};
my $strOptionEnum = buildConfigOptionEnum($strOption);
my $strOptionArg = ($rhOption->{&CFGDEF_TYPE} ne CFGDEF_TYPE_BOOLEAN ? " .has_arg = required_argument,\n" : '');
my $strOptionPrefix = $rhConfigDefine->{$strOption}{&CFGDEF_PREFIX};
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-03 18:27:38 -05:00
my @stryOptionName = ($strOption);
Move configuration definition to src/build/config/config.yaml. Moving to YAML allows the configuration data to be read by C programs. Also go back to using YAML::XS since it is the only implementation that has proper boolean support.
2021-03-08 16:01:05 -05:00
if (defined($rhOption->{&CFGDEF_DEPRECATE}))
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-03 18:27:38 -05:00
{
Move configuration definition to src/build/config/config.yaml. Moving to YAML allows the configuration data to be read by C programs. Also go back to using YAML::XS since it is the only implementation that has proper boolean support.
2021-03-08 16:01:05 -05:00
foreach my $strOptionNameAlt (sort(keys(%{$rhOption->{&CFGDEF_DEPRECATE}})))
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-03 18:27:38 -05:00
{
push(@stryOptionName, $strOptionNameAlt);
}
}
$strBuildSource .=
"\n" .
" // ${strOption} option" . (@stryOptionName > 1 ? ' and deprecations' : '') . "\n" .
" // " . (qw{-} x 125) . "\n";
for (my $iOptionIdx = 1; $iOptionIdx <= $rhOption->{&CFGDEF_INDEX_TOTAL}; $iOptionIdx++)
Implement version command in C.
2017-11-28 21:44:05 -05:00
{
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-03 18:27:38 -05:00
for (my $iOptionNameIdx = 0; $iOptionNameIdx < @stryOptionName; $iOptionNameIdx++)
{
my $strOptionName = $stryOptionName[$iOptionNameIdx];
Move configuration definition to src/build/config/config.yaml. Moving to YAML allows the configuration data to be read by C programs. Also go back to using YAML::XS since it is the only implementation that has proper boolean support.
2021-03-08 16:01:05 -05:00
my $rhNameAlt = $rhOption->{&CFGDEF_DEPRECATE}{$strOptionName};
Implement version command in C.
2017-11-28 21:44:05 -05:00
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-03 18:27:38 -05:00
# Skip alt name if it is not valid for this option index
if ($iOptionNameIdx > 0 && defined($rhNameAlt->{&CFGDEF_INDEX}) && $rhNameAlt->{&CFGDEF_INDEX} != $iOptionIdx)
{
next;
}
Implement version command in C.
2017-11-28 21:44:05 -05:00
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-03 18:27:38 -05:00
# Generate output name
my $strOptionNameOut = $strOptionName;
Automatically generate constants for command and option names. These constants are easier than using cfgOptionName() and cfgCommandName() and lead to cleaner code and simpler to construct messages. String versions are provided. Eventually all the strings will be used in the config structures, but for now they are useful to avoid wrapping with strNew().
2019-04-12 09:03:34 -04:00
my $strOptionConst;
More config parsing in C in preparation for all config parsing in C.
2017-12-05 10:09:07 -05:00
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-03 18:27:38 -05:00
if (defined($strOptionPrefix))
{
if ($iOptionNameIdx == 0)
{
$strOptionNameOut =
"${strOptionPrefix}${iOptionIdx}-" . substr($strOptionName, length($strOptionPrefix) + 1);
}
else
{
$strOptionNameOut =~ s/\?/$iOptionIdx/g;
}
}
# Generate option value used for parsing (offset is added so options don't conflict with getopt_long return values)
my $strOptionFlag = 'PARSE_OPTION_FLAG |';
Automatically generate constants for command and option names. These constants are easier than using cfgOptionName() and cfgCommandName() and lead to cleaner code and simpler to construct messages. String versions are provided. Eventually all the strings will be used in the config structures, but for now they are useful to avoid wrapping with strNew().
2019-04-12 09:03:34 -04:00
# Build option constant name if this is the current name for the option
if ($iOptionNameIdx == 0)
{
Remove indexed option constants. These constants don't scale well as the index total is increased for an option. The core code rarely uses these options and they are easily replaced with cfgOptionName(). The tests had started to make use of the constants, so provide functions that build the option name from the optionId and, optionally, the optionKey.
2020-10-19 14:03:48 -04:00
if (!$rhConfigDefine->{$strOption}{&CFGDEF_GROUP})
{
$strOptionConst = "CFGOPT_" . uc($strOptionNameOut);
$strOptionConst =~ s/\-/_/g;
}
Automatically generate constants for command and option names. These constants are easier than using cfgOptionName() and cfgCommandName() and lead to cleaner code and simpler to construct messages. String versions are provided. Eventually all the strings will be used in the config structures, but for now they are useful to avoid wrapping with strNew().
2019-04-12 09:03:34 -04:00
}
# Else use bare string and mark as deprecated
else
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-03 18:27:38 -05:00
{
$strOptionFlag .= ' PARSE_DEPRECATE_FLAG |';
}
Implement version command in C.
2017-11-28 21:44:05 -05:00
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-03 18:27:38 -05:00
my $strOptionVal =
Remove cfgOptionId() and replace it with cfgParseOption(). cfgOptionId() did not recognize deprecated options which made the help command throw errors when they were specified on the command line. cfgParseOption() will correctly identify deprecated options. cfgParseOption() can also be used in cfgParse() to reduce code duplication when parsing info out of the option value returned by optionFind(). Finally, code the option key index separately in parse.auto.c. For now they are simply added back together but future code will need them separated.
2020-10-20 11:24:26 -04:00
($rhOption->{&CFGDEF_GROUP} ? "(" . ($iOptionIdx - 1) . " << PARSE_KEY_IDX_SHIFT) | " : '') . $strOptionEnum;
Implement version command in C.
2017-11-28 21:44:05 -05:00
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-03 18:27:38 -05:00
# Add option
Implement version command in C.
2017-11-28 21:44:05 -05:00
$strBuildSource .=
" {\n" .
Remove indexed option constants. These constants don't scale well as the index total is increased for an option. The core code rarely uses these options and they are easily replaced with cfgOptionName(). The tests had started to make use of the constants, so provide functions that build the option name from the optionId and, optionally, the optionKey.
2020-10-19 14:03:48 -04:00
" .name = \"${strOptionNameOut}\",\n" .
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-03 18:27:38 -05:00
$strOptionArg .
" .val = ${strOptionFlag} ${strOptionVal},\n" .
Implement version command in C.
2017-11-28 21:44:05 -05:00
" },\n";
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-03 18:27:38 -05:00
# Add negation when defined
if ($rhOption->{&CFGDEF_NEGATE} &&
!($iOptionNameIdx > 0 && defined($rhNameAlt->{&CFGDEF_NEGATE}) && !$rhNameAlt->{&CFGDEF_NEGATE}))
{
$strBuildSource .=
" {\n" .
Remove indexed option constants. These constants don't scale well as the index total is increased for an option. The core code rarely uses these options and they are easily replaced with cfgOptionName(). The tests had started to make use of the constants, so provide functions that build the option name from the optionId and, optionally, the optionKey.
2020-10-19 14:03:48 -04:00
" .name = \"no-${strOptionNameOut}\",\n" .
Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. * repo-* options are now indexed although only one is allowed. * List deprecated option names in documentation and command-line help.
2018-02-03 18:27:38 -05:00
" .val = ${strOptionFlag} PARSE_NEGATE_FLAG | ${strOptionVal},\n" .
" },\n";
}
Rename -no to -reset for non-boolean options.
2018-02-06 11:26:06 -05:00
# Add reset when defined
if ($rhOption->{&CFGDEF_RESET} &&
!($iOptionNameIdx > 0 && defined($rhNameAlt->{&CFGDEF_RESET}) && !$rhNameAlt->{&CFGDEF_RESET}))
{
$strBuildSource .=
" {\n" .
Remove indexed option constants. These constants don't scale well as the index total is increased for an option. The core code rarely uses these options and they are easily replaced with cfgOptionName(). The tests had started to make use of the constants, so provide functions that build the option name from the optionId and, optionally, the optionKey.
2020-10-19 14:03:48 -04:00
" .name = \"reset-${strOptionNameOut}\",\n" .
Rename -no to -reset for non-boolean options.
2018-02-06 11:26:06 -05:00
" .val = ${strOptionFlag} PARSE_RESET_FLAG | ${strOptionVal},\n" .
" },\n";
}
Implement version command in C.
2017-11-28 21:44:05 -05:00
}
}
}
# The option list needs to be terminated or getopt_long will just keep on reading
$strBuildSource .=
" // Terminate option list\n" .
" {\n" .
" .name = NULL\n" .
" }\n" .
"};\n";
Move all parse-related rules to parse module. Data required for parsing was spread between the config and defined modules, mostly for historical reasons because the same data was used by Perl. Requiring all the parse rules to be accessed with function interfaces makes the code more complicated and new rules harder to implement. Instead, move the data to the parse module so in the most complex cases no interface functions are needed. This reduces the total amount of code and paves the way for more complex parse rules.
2020-12-17 09:32:31 -05:00
$rhBuild->{&BLD_FILE}{&BLDLCL_FILE_DEFINE}{&BLD_DATA}{&BLDLCL_DATA_OPTION_GETOPT}{&BLD_SOURCE} = $strBuildSource;
Implement version command in C.
2017-11-28 21:44:05 -05:00
Validate configuration options in a single pass. By pre-calculating and storing the option dependencies in parse.auto.c validation can be completed in a single pass, which is both simpler and faster.
2018-08-11 12:55:33 -04:00
# Build option resolve order list. This allows the option validation in C to take place in a single pass.
#
# Always process the stanza option first since confusing error message are produced if it is missing.
#-------------------------------------------------------------------------------------------------------------------------------
my @stryResolveOrder = (buildConfigOptionEnum(CFGOPT_STANZA));
my $rhResolved = {&CFGOPT_STANZA => true};
my $bAllResolved;
do
{
# Assume that we will finish on this loop
$bAllResolved = true;
# Loop through all options
foreach my $strOption (sort(keys(%{$rhConfigDefine})))
{
my $bSkip = false;
# Check the default depend
my $strOptionDepend =
ref($rhConfigDefine->{$strOption}{&CFGDEF_DEPEND}) eq 'HASH' ?
$rhConfigDefine->{$strOption}{&CFGDEF_DEPEND}{&CFGDEF_DEPEND_OPTION} :
$rhConfigDefine->{$strOption}{&CFGDEF_DEPEND};
if (defined($strOptionDepend) && !$rhResolved->{$strOptionDepend})
{
# &log(WARN, "$strOptionDepend is not resolved");
$bSkip = true;
}
# Check the command depends
foreach my $strCommand (sort(keys(%{$rhConfigDefine->{$strOption}{&CFGDEF_COMMAND}})))
{
my $strOptionDepend =
ref($rhConfigDefine->{$strOption}{&CFGDEF_COMMAND}{$strCommand}{&CFGDEF_DEPEND}) eq 'HASH' ?
$rhConfigDefine->{$strOption}{&CFGDEF_COMMAND}{$strCommand}{&CFGDEF_DEPEND}{&CFGDEF_DEPEND_OPTION} :
$rhConfigDefine->{$strOption}{&CFGDEF_COMMAND}{$strCommand}{&CFGDEF_DEPEND};
if (defined($strOptionDepend) && !$rhResolved->{$strOptionDepend})
{
$bSkip = true;
}
}
# If dependency was not found try again on the next loop
if ($bSkip)
{
$bAllResolved = false;
}
# Else add option to resolve order list
elsif (!$rhResolved->{$strOption})
{
Prepare configuration module for multi-repository support. Refactor the code to allow a dynamic number of indexes for indexed options, e.g. pg-path. Our reliance on getopt_long() still limits the number of indexes we can have per group, but once this limitation is removed the rest of the code should be happy with dynamic numbers of indexes (with a reasonable maximum). Add an option to set a default in each group. This was previously handled by the host-id option but now there is a specific option for each group, pg and repo. These remain internal until they can be fully tested with multi-repo support. They are fully tested for internal usage. Remove the ConfigDefineOption enum and use the ConfigOption enum instead. They are now equal since the indexed options (e.g. cfgOptRepoHost2) have been removed from ConfigOption. Remove the config/config test module and add required tests to the config/parse test module. Parsing is now the only way to load a config so this removes some redundancy. Split new internal config structures and functions into a new header file, config.intern.h. More functions will need to be moved over from config.h but that will need to be done in a future commit to reduce churn. Add repoIdx to repoIsLocal() and storageRepo*(). Multi-repository support requires that repo locality and storage be accessible by index. This allows, for example, multiple repos to be iterated in a loop. This could be done in a separate commit but doesn't seem worth it since the code is related. Remove the type parameter from storageRepoGet(). This parameter existed solely to provide coverage for the case where the storage type was invalid. A better pattern is to check that the type is S3 once all other types have been ruled out.
2020-11-23 15:55:46 -05:00
push(@stryResolveOrder, buildConfigOptionEnum($strOption));
Validate configuration options in a single pass. By pre-calculating and storing the option dependencies in parse.auto.c validation can be completed in a single pass, which is both simpler and faster.
2018-08-11 12:55:33 -04:00
Prepare configuration module for multi-repository support. Refactor the code to allow a dynamic number of indexes for indexed options, e.g. pg-path. Our reliance on getopt_long() still limits the number of indexes we can have per group, but once this limitation is removed the rest of the code should be happy with dynamic numbers of indexes (with a reasonable maximum). Add an option to set a default in each group. This was previously handled by the host-id option but now there is a specific option for each group, pg and repo. These remain internal until they can be fully tested with multi-repo support. They are fully tested for internal usage. Remove the ConfigDefineOption enum and use the ConfigOption enum instead. They are now equal since the indexed options (e.g. cfgOptRepoHost2) have been removed from ConfigOption. Remove the config/config test module and add required tests to the config/parse test module. Parsing is now the only way to load a config so this removes some redundancy. Split new internal config structures and functions into a new header file, config.intern.h. More functions will need to be moved over from config.h but that will need to be done in a future commit to reduce churn. Add repoIdx to repoIsLocal() and storageRepo*(). Multi-repository support requires that repo locality and storage be accessible by index. This allows, for example, multiple repos to be iterated in a loop. This could be done in a separate commit but doesn't seem worth it since the code is related. Remove the type parameter from storageRepoGet(). This parameter existed solely to provide coverage for the case where the storage type was invalid. A better pattern is to check that the type is S3 once all other types have been ruled out.
2020-11-23 15:55:46 -05:00
$rhResolved->{$strOption} = true;
Validate configuration options in a single pass. By pre-calculating and storing the option dependencies in parse.auto.c validation can be completed in a single pass, which is both simpler and faster.
2018-08-11 12:55:33 -04:00
}
}
}
while (!$bAllResolved);
$rhBuild->{&BLD_FILE}{&BLDLCL_FILE_DEFINE}{&BLD_DATA}{&BLDLCL_DATA_OPTION_RESOLVE}{&BLD_SOURCE} =
Allow resolve array from cd5df357 to autosize to detect short arrays in testing.
2018-08-14 17:17:14 -04:00
"static const ConfigOption optionResolveOrder[] =\n" .
Validate configuration options in a single pass. By pre-calculating and storing the option dependencies in parse.auto.c validation can be completed in a single pass, which is both simpler and faster.
2018-08-11 12:55:33 -04:00
"{\n" .
" " . join(",\n ", @stryResolveOrder) . ",\n" .
"};\n";
Implement version command in C.
2017-11-28 21:44:05 -05:00
return $rhBuild;
}
push @EXPORT, qw(buildConfigParse);
1;
Reference in New Issue Copy Permalink