mirror of
https://github.com/SAP/jenkins-library.git
synced 2024-12-14 11:03:09 +02:00
acbcc5646b
* Add ans implementation * Remove todo comment * Rename test function Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com> * Better wording Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com> * Add reading of response body function * Use http pkg ReadResponseBody * Check read error * Better test case description * Fix formatting * Create own package for read response body * Omit empty nested resource struct * Separate Resource struct from Event struct * Merge and unmarshall instead of only unmarshalling * Improve status code error message * Remove unchangeable event fields * Separate event parts * Change log level setter function * Restructure ans send test * Revert exporting readResponseBody function Instead the code is duplicated in the xsuaa and ans package * Add check correct ans setup request * Add set options function for mocking * Review fixes * Correct function name * Use strict unmarshalling * Validate event * Move functions * Add documentation comments * improve test * Validate event * Add logrus hook for ans * Set defaults on new hook creation * Fix log level on error * Don't alter entry log level * Set severity fatal on 'fatal error' log message * Ensure that log entries don't affect each other * Remove unnecessary correlationID * Use file path instead of event template string * Improve warning messages * Add empty log message check * Allow configuration from file and string * Add sourceEventId to tags * Change resourceType to Pipeline * Use structured config approach * Use new log level set function * Check correct setup and return error * Mock http requests * Only send log level warning or higher * Use new function name * One-liner ifs * Improve test name * Fix tests * Prevent double firing * Reduce Fire test size * Add error message to test * Reduce newANSHook test size * Further check error * Rename to defaultEvent in hook struct * Reduce ifs further * Fix set error category test The ansHook Fire test cannot run in parallel, as it would affect the other tests that use the error category. * Change function name to SetServiceKey * Validate event * Rename to eventTemplate in hook struct * Move copy to event.go * Fix function mix * Remove unnecessary cleanup * Remove parallel test The translation fails now and again when parallel is on. * Remove prefix test * Remove unused copyEvent function * Fix ifs * Add docu comment * Register ans hook from pkg * register hook and setup event template seperately * Exclusively read eventTemplate from environment * setupEventTemplate tests * adjust hook levels test * sync tests- wlill still fail * migrate TestANSHook_registerANSHook test * fixes * Introduce necessary parameters * Setup hook test * Use file instead * Adapt helper for ans * Generate go files * Add ans config to general config * Change generator * Regenerate steps * Allow hook config from user config Merges with hook config from defaults * Remove ans flags from root command * Get environment variables * Generate files * Add test when calling merge twice * Update generator * Regenerate steps * Check two location for ans service key env var * Re-generate * Fix if * Generate files with fix * Duplicate config struct * Add type casting test for ans config * Fix helper * Fix format * Fix type casting of config * Revert "Allow hook config from user config" This reverts commit 4864499a4c497998c9ffc3e157ef491be955e68e. * Revert "Add test when calling merge twice" This reverts commit b82320fd07b82f5a597c5071049d918bcf62de00. * Add ans config tests * Improve helper code * Re-generate commands * Fix helper unit tests * Change to only one argument * Fix helper tests * Re-generate * Revert piper and config changes * Re-generate missing step * Generate new steps * [ANS] Add servicekey credential to environment (#3684) * Add ANS credential * Switch to hooks and remove comments * Add subsection for ans Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com> * Remove changes to piper.go * Remove formatting * Add test for ANS * Define hook credential seperately from step credential * Add test for retrieval from general section * Add comment * Get ans hook info from DefaultValueCache * [ANS] Add documentation (#3704) * Add ANS credential * Switch to hooks and remove comments * Add subsection for ans Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com> * Remove changes to piper.go * Remove formatting * Add test for ANS * Define hook credential seperately from step credential * Add test for retrieval from general section * Add comment * Add documentation * Review changes * Review comments * Improve documentation further * Add note of two event templates * Add log level destinction * Further improvements * Improve text * Remove unused things * Add ANS credential * Switch to hooks and remove comments * Add subsection for ans Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com> * Remove changes to piper.go * Remove formatting * Add test for ANS * Define hook credential seperately from step credential * Add test for retrieval from general section * Add comment * Get ans hook info from DefaultValueCache * Improvements Co-authored-by: Linda Siebert <linda.siebert@sap.com> Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com> Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com> * New lines Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com> Co-authored-by: Roland Stengel <r.stengel@sap.com> Co-authored-by: Thorsten Duda <thorsten.duda@sap.com>
290 lines
12 KiB
Groovy
290 lines
12 KiB
Groovy
import com.sap.piper.BashUtils
|
|
import com.sap.piper.DebugReport
|
|
import com.sap.piper.DefaultValueCache
|
|
import com.sap.piper.JenkinsUtils
|
|
import com.sap.piper.MapUtils
|
|
import com.sap.piper.PiperGoUtils
|
|
import com.sap.piper.Utils
|
|
import com.sap.piper.analytics.InfluxData
|
|
import groovy.transform.Field
|
|
|
|
import static com.sap.piper.Prerequisites.checkScript
|
|
|
|
@Field String STEP_NAME = getClass().getName()
|
|
|
|
void call(Map parameters = [:], String stepName, String metadataFile, List credentialInfo, boolean failOnMissingReports = false, boolean failOnMissingLinks = false, boolean failOnError = false) {
|
|
|
|
Map handlePipelineStepErrorsParameters = [stepName: stepName, stepParameters: parameters]
|
|
if (failOnError) {
|
|
handlePipelineStepErrorsParameters.failOnError = true
|
|
}
|
|
|
|
handlePipelineStepErrors(handlePipelineStepErrorsParameters) {
|
|
Script script = checkScript(this, parameters) ?: this
|
|
def jenkinsUtils = parameters.jenkinsUtilsStub ?: new JenkinsUtils()
|
|
def utils = parameters.juStabUtils ?: new Utils()
|
|
|
|
String piperGoPath = parameters.piperGoPath ?: './piper'
|
|
|
|
prepareExecution(script, utils, parameters)
|
|
prepareMetadataResource(script, metadataFile)
|
|
Map stepParameters = prepareStepParameters(parameters)
|
|
echo "Step params $stepParameters"
|
|
|
|
withEnv([
|
|
"PIPER_parametersJSON=${groovy.json.JsonOutput.toJson(stepParameters)}",
|
|
"PIPER_correlationID=${env.BUILD_URL}",
|
|
//ToDo: check if parameters make it into docker image on JaaS
|
|
]) {
|
|
String defaultConfigArgs = getCustomDefaultConfigsArg()
|
|
String customConfigArg = getCustomConfigArg(script)
|
|
|
|
echo "PIPER_parametersJSON: ${groovy.json.JsonOutput.toJson(stepParameters)}"
|
|
|
|
// get context configuration
|
|
Map config
|
|
handleErrorDetails(stepName) {
|
|
config = getStepContextConfig(script, piperGoPath, metadataFile, defaultConfigArgs, customConfigArg)
|
|
echo "Context Config: ${config}"
|
|
}
|
|
|
|
//Add ANS credential information to the config
|
|
ansHookServiceKeyCredentialsId =
|
|
DefaultValueCache.getInstance().getDefaultValues().hooks?.ans?.serviceKeyCredentialsId
|
|
config += ["ansHookServiceKeyCredentialsId": ansHookServiceKeyCredentialsId]
|
|
|
|
// prepare stashes
|
|
// first eliminate empty stashes
|
|
config.stashContent = utils.unstashAll(config.stashContent)
|
|
// then make sure that commonPipelineEnvironment, config, ... is also available when step stashing is active
|
|
if (config.stashContent?.size() > 0) {
|
|
config.stashContent.add('pipelineConfigAndTests')
|
|
config.stashContent.add('piper-bin')
|
|
config.stashContent.add('pipelineStepReports')
|
|
}
|
|
|
|
if (parameters.stashNoDefaultExcludes) {
|
|
// Merge this parameter which is only relevant in Jenkins context
|
|
// (for dockerExecuteOnKubernetes step) and go binary doesn't know about
|
|
config.stashNoDefaultExcludes = parameters.stashNoDefaultExcludes
|
|
}
|
|
|
|
dockerWrapper(script, stepName, config) {
|
|
handleErrorDetails(stepName) {
|
|
writePipelineEnv(script: script, piperGoPath: piperGoPath)
|
|
utils.unstash('pipelineStepReports')
|
|
try {
|
|
try {
|
|
try {
|
|
credentialWrapper(config, credentialInfo) {
|
|
sh "${piperGoPath} ${stepName}${defaultConfigArgs}${customConfigArg}"
|
|
}
|
|
} finally {
|
|
jenkinsUtils.handleStepResults(stepName, failOnMissingReports, failOnMissingLinks)
|
|
}
|
|
} finally {
|
|
readPipelineEnv(script: script, piperGoPath: piperGoPath)
|
|
}
|
|
} finally {
|
|
InfluxData.readFromDisk(script)
|
|
stash name: 'pipelineStepReports', includes: '.pipeline/stepReports/**', allowEmpty: true
|
|
}
|
|
}
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
// reused in sonarExecuteScan
|
|
static void prepareExecution(Script script, Utils utils, Map parameters = [:]) {
|
|
def piperGoUtils = parameters.piperGoUtils ?: new PiperGoUtils(script, utils)
|
|
piperGoUtils.unstashPiperBin()
|
|
utils.unstash('pipelineConfigAndTests')
|
|
}
|
|
|
|
// reused in sonarExecuteScan
|
|
static Map prepareStepParameters(Map parameters) {
|
|
Map stepParameters = [:].plus(parameters)
|
|
|
|
stepParameters.remove('script')
|
|
stepParameters.remove('jenkinsUtilsStub')
|
|
stepParameters.remove('piperGoPath')
|
|
stepParameters.remove('juStabUtils')
|
|
stepParameters.remove('piperGoUtils')
|
|
|
|
// When converting to JSON and back again, entries which had a 'null' value will now have a value
|
|
// of type 'net.sf.json.JSONNull', for which the Groovy Truth resolves to 'true' in for example if-conditions
|
|
return MapUtils.pruneNulls(stepParameters)
|
|
}
|
|
|
|
// reused in sonarExecuteScan
|
|
static void prepareMetadataResource(Script script, String metadataFile) {
|
|
script.writeFile(file: ".pipeline/tmp/${metadataFile}", text: script.libraryResource(metadataFile))
|
|
}
|
|
|
|
// reused in sonarExecuteScan
|
|
static Map getStepContextConfig(Script script, String piperGoPath, String metadataFile, String defaultConfigArgs, String customConfigArg) {
|
|
return script.readJSON(text: script.sh(returnStdout: true, script: "${piperGoPath} getConfig --contextConfig --stepMetadata '.pipeline/tmp/${metadataFile}'${defaultConfigArgs}${customConfigArg}"))
|
|
}
|
|
|
|
static String getCustomDefaultConfigs() {
|
|
// The default config files were extracted from merged library
|
|
// resources by setupCommonPipelineEnvironment.groovy into .pipeline/.
|
|
List customDefaults = DefaultValueCache.getInstance().getCustomDefaults()
|
|
for (int i = 0; i < customDefaults.size(); i++) {
|
|
customDefaults[i] = BashUtils.quoteAndEscape(".pipeline/${customDefaults[i]}")
|
|
}
|
|
return customDefaults.join(',')
|
|
}
|
|
|
|
// reused in sonarExecuteScan
|
|
static String getCustomDefaultConfigsArg() {
|
|
String customDefaults = getCustomDefaultConfigs()
|
|
if (customDefaults) {
|
|
return " --defaultConfig ${customDefaults} --ignoreCustomDefaults"
|
|
}
|
|
return ''
|
|
}
|
|
|
|
// reused in sonarExecuteScan
|
|
static String getCustomConfigArg(def script) {
|
|
if (script?.commonPipelineEnvironment?.configurationFile
|
|
&& script.commonPipelineEnvironment.configurationFile != '.pipeline/config.yml'
|
|
&& script.commonPipelineEnvironment.configurationFile != '.pipeline/config.yaml') {
|
|
return " --customConfig ${BashUtils.quoteAndEscape(script.commonPipelineEnvironment.configurationFile)}"
|
|
}
|
|
return ''
|
|
}
|
|
|
|
// reused in sonarExecuteScan
|
|
void dockerWrapper(script, stepName, config, body) {
|
|
if (config.dockerImage) {
|
|
echo "[INFO] executing pipeline step '${stepName}' with docker image '${config.dockerImage}'"
|
|
Map dockerExecuteParameters = [:].plus(config)
|
|
dockerExecuteParameters.script = script
|
|
dockerExecute(dockerExecuteParameters) {
|
|
body()
|
|
}
|
|
} else {
|
|
body()
|
|
}
|
|
}
|
|
|
|
// reused in sonarExecuteScan
|
|
void credentialWrapper(config, List credentialInfo, body) {
|
|
credentialInfo = handleVaultCredentials(config, credentialInfo)
|
|
credentialInfo = handleANSCredentials(config, credentialInfo)
|
|
if (credentialInfo.size() > 0) {
|
|
def creds = []
|
|
def sshCreds = []
|
|
credentialInfo.each { cred ->
|
|
def credentialsId
|
|
if (cred.resolveCredentialsId == false) {
|
|
credentialsId = cred.id
|
|
} else {
|
|
credentialsId = config[cred.id]
|
|
}
|
|
if (credentialsId) {
|
|
switch (cred.type) {
|
|
case "file":
|
|
creds.add(file(credentialsId: credentialsId, variable: cred.env[0]))
|
|
break
|
|
case "token":
|
|
creds.add(string(credentialsId: credentialsId, variable: cred.env[0]))
|
|
break
|
|
case "usernamePassword":
|
|
creds.add(usernamePassword(credentialsId: credentialsId, usernameVariable: cred.env[0], passwordVariable: cred.env[1]))
|
|
break
|
|
case "ssh":
|
|
sshCreds.add(credentialsId)
|
|
break
|
|
default:
|
|
error("invalid credential type: ${cred.type}")
|
|
}
|
|
}
|
|
}
|
|
|
|
// remove credentialIds that were probably defaulted and which are not present in jenkins
|
|
if (containsVaultConfig(config)) {
|
|
creds = removeMissingCredentials(creds, config)
|
|
sshCreds = removeMissingCredentials(sshCreds, config)
|
|
}
|
|
|
|
if (sshCreds.size() > 0) {
|
|
sshagent (sshCreds) {
|
|
withCredentials(creds) {
|
|
body()
|
|
}
|
|
}
|
|
} else {
|
|
withCredentials(creds) {
|
|
body()
|
|
}
|
|
}
|
|
} else {
|
|
body()
|
|
}
|
|
}
|
|
|
|
List removeMissingCredentials(List creds, Map config) {
|
|
return creds.findAll { credentialExists(it, config) }
|
|
}
|
|
|
|
boolean credentialExists(cred, Map config) {
|
|
try {
|
|
withCredentials([cred]) {
|
|
return true
|
|
}
|
|
} catch (e) {
|
|
return false
|
|
}
|
|
}
|
|
|
|
boolean containsVaultConfig(Map config) {
|
|
def approleIsUsed = config.containsKey('vaultAppRoleTokenCredentialsId') && config.containsKey('vaultAppRoleSecretTokenCredentialsId')
|
|
def tokenIsUsed = config.containsKey('vaultTokenCredentialsId')
|
|
|
|
return approleIsUsed || tokenIsUsed
|
|
}
|
|
|
|
// Injects vaultCredentials if steps supports resolving parameters from vault
|
|
List handleVaultCredentials(config, List credentialInfo) {
|
|
if (config.containsKey('vaultAppRoleTokenCredentialsId') && config.containsKey('vaultAppRoleSecretTokenCredentialsId')) {
|
|
credentialInfo += [[type: 'token', id: 'vaultAppRoleTokenCredentialsId', env: ['PIPER_vaultAppRoleID']],
|
|
[type: 'token', id: 'vaultAppRoleSecretTokenCredentialsId', env: ['PIPER_vaultAppRoleSecretID']]]
|
|
}
|
|
|
|
if (config.containsKey('vaultTokenCredentialsId')) {
|
|
credentialInfo += [[type: 'token', id: 'vaultTokenCredentialsId', env: ['PIPER_vaultToken']]]
|
|
}
|
|
|
|
return credentialInfo
|
|
}
|
|
|
|
List handleANSCredentials(config, List credentialInfo){
|
|
if (config.containsKey('ansHookServiceKeyCredentialsId')) {
|
|
credentialInfo += [[type: 'token', id: 'ansHookServiceKeyCredentialsId', env: ['PIPER_ansHookServiceKey']]]
|
|
}
|
|
|
|
return credentialInfo
|
|
}
|
|
|
|
// reused in sonarExecuteScan
|
|
void handleErrorDetails(String stepName, Closure body) {
|
|
try {
|
|
body()
|
|
} catch (ex) {
|
|
def errorDetailsFileName = "${stepName}_errorDetails.json"
|
|
if (fileExists(file: errorDetailsFileName)) {
|
|
def errorDetails = readJSON(file: errorDetailsFileName)
|
|
def errorCategory = ""
|
|
if (errorDetails.category) {
|
|
errorCategory = " (category: ${errorDetails.category})"
|
|
DebugReport.instance.failedBuild.category = errorDetails.category
|
|
}
|
|
error "[${stepName}] Step execution failed${errorCategory}. Error: ${errorDetails.error ?: errorDetails.message}"
|
|
}
|
|
error "[${stepName}] Step execution failed. Error: ${ex}, please see log file for more details."
|
|
}
|
|
}
|