1
0
mirror of https://github.com/SAP/jenkins-library.git synced 2024-12-14 11:03:09 +02:00
sap-jenkins-library/vars/piperExecuteBin.groovy
Linda Siebert acbcc5646b
[ANS] Change helper and re-generate steps (#3675)
* Add ans implementation

* Remove todo comment

* Rename test function

Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>

* Better wording

Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>

* Add reading of response body function

* Use http pkg ReadResponseBody

* Check read error

* Better test case description

* Fix formatting

* Create own package for read response body

* Omit empty nested resource struct

* Separate Resource struct from Event struct

* Merge and unmarshall instead of only unmarshalling

* Improve status code error message

* Remove unchangeable event fields

* Separate event parts

* Change log level setter function

* Restructure ans send test

* Revert exporting readResponseBody function

Instead the code is duplicated in the xsuaa and ans package

* Add check correct ans setup request

* Add set options function for mocking

* Review fixes

* Correct function name

* Use strict unmarshalling

* Validate event

* Move functions

* Add documentation comments

* improve test

* Validate event

* Add logrus hook for ans

* Set defaults on new hook creation

* Fix log level on error

* Don't alter entry log level

* Set severity fatal on 'fatal error' log message

* Ensure that log entries don't affect each other

* Remove unnecessary correlationID

* Use file path instead of event template string

* Improve warning messages

* Add empty log message check

* Allow configuration from file and string

* Add sourceEventId to tags

* Change resourceType to Pipeline

* Use structured config approach

* Use new log level set function

* Check correct setup and return error

* Mock http requests

* Only send log level warning or higher

* Use new function name

* One-liner ifs

* Improve test name

* Fix tests

* Prevent double firing

* Reduce Fire test size

* Add error message to test

* Reduce newANSHook test size

* Further check error

* Rename to defaultEvent in hook struct

* Reduce ifs further

* Fix set error category test

The ansHook Fire test cannot run in parallel, as it would affect the
other tests that use the error category.

* Change function name to SetServiceKey

* Validate event

* Rename to eventTemplate in hook struct

* Move copy to event.go

* Fix function mix

* Remove unnecessary cleanup

* Remove parallel test

The translation fails now and again when parallel is on.

* Remove prefix test

* Remove unused copyEvent function

* Fix ifs

* Add docu comment

* Register ans hook from pkg

* register hook and setup event template seperately

* Exclusively read eventTemplate from environment

* setupEventTemplate tests

* adjust hook levels test

* sync tests- wlill still fail

* migrate TestANSHook_registerANSHook test

* fixes

* Introduce necessary parameters

* Setup hook test

* Use file instead

* Adapt helper for ans

* Generate go files

* Add ans config to general config

* Change generator

* Regenerate steps

* Allow hook config from user config

Merges with hook config from defaults

* Remove ans flags from root command

* Get environment variables

* Generate files

* Add test when calling merge twice

* Update generator

* Regenerate steps

* Check two location for ans service key env var

* Re-generate

* Fix if

* Generate files with fix

* Duplicate config struct

* Add type casting test for ans config

* Fix helper

* Fix format

* Fix type casting of config

* Revert "Allow hook config from user config"

This reverts commit 4864499a4c497998c9ffc3e157ef491be955e68e.

* Revert "Add test when calling merge twice"

This reverts commit b82320fd07b82f5a597c5071049d918bcf62de00.

* Add ans config tests

* Improve helper code

* Re-generate commands

* Fix helper unit tests

* Change to only one argument

* Fix helper tests

* Re-generate

* Revert piper and config changes

* Re-generate missing step

* Generate new steps

* [ANS] Add servicekey credential to environment (#3684)

* Add ANS credential

* Switch to hooks and remove comments

* Add subsection for ans

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>

* Remove changes to piper.go

* Remove formatting

* Add test for ANS

* Define hook credential seperately from step credential

* Add test for retrieval from general section

* Add comment

* Get ans hook info from DefaultValueCache

* [ANS] Add documentation (#3704)

* Add ANS credential

* Switch to hooks and remove comments

* Add subsection for ans

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>

* Remove changes to piper.go

* Remove formatting

* Add test for ANS

* Define hook credential seperately from step credential

* Add test for retrieval from general section

* Add comment

* Add documentation

* Review changes

* Review comments

* Improve documentation further

* Add note of two event templates

* Add log level destinction

* Further improvements

* Improve text

* Remove unused things

* Add ANS credential

* Switch to hooks and remove comments

* Add subsection for ans

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>

* Remove changes to piper.go

* Remove formatting

* Add test for ANS

* Define hook credential seperately from step credential

* Add test for retrieval from general section

* Add comment

* Get ans hook info from DefaultValueCache

* Improvements

Co-authored-by: Linda Siebert <linda.siebert@sap.com>
Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>

* New lines

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
Co-authored-by: Roland Stengel <r.stengel@sap.com>
Co-authored-by: Thorsten Duda <thorsten.duda@sap.com>
2022-06-22 13:31:17 +02:00

290 lines
12 KiB
Groovy

import com.sap.piper.BashUtils
import com.sap.piper.DebugReport
import com.sap.piper.DefaultValueCache
import com.sap.piper.JenkinsUtils
import com.sap.piper.MapUtils
import com.sap.piper.PiperGoUtils
import com.sap.piper.Utils
import com.sap.piper.analytics.InfluxData
import groovy.transform.Field
import static com.sap.piper.Prerequisites.checkScript
@Field String STEP_NAME = getClass().getName()
void call(Map parameters = [:], String stepName, String metadataFile, List credentialInfo, boolean failOnMissingReports = false, boolean failOnMissingLinks = false, boolean failOnError = false) {
Map handlePipelineStepErrorsParameters = [stepName: stepName, stepParameters: parameters]
if (failOnError) {
handlePipelineStepErrorsParameters.failOnError = true
}
handlePipelineStepErrors(handlePipelineStepErrorsParameters) {
Script script = checkScript(this, parameters) ?: this
def jenkinsUtils = parameters.jenkinsUtilsStub ?: new JenkinsUtils()
def utils = parameters.juStabUtils ?: new Utils()
String piperGoPath = parameters.piperGoPath ?: './piper'
prepareExecution(script, utils, parameters)
prepareMetadataResource(script, metadataFile)
Map stepParameters = prepareStepParameters(parameters)
echo "Step params $stepParameters"
withEnv([
"PIPER_parametersJSON=${groovy.json.JsonOutput.toJson(stepParameters)}",
"PIPER_correlationID=${env.BUILD_URL}",
//ToDo: check if parameters make it into docker image on JaaS
]) {
String defaultConfigArgs = getCustomDefaultConfigsArg()
String customConfigArg = getCustomConfigArg(script)
echo "PIPER_parametersJSON: ${groovy.json.JsonOutput.toJson(stepParameters)}"
// get context configuration
Map config
handleErrorDetails(stepName) {
config = getStepContextConfig(script, piperGoPath, metadataFile, defaultConfigArgs, customConfigArg)
echo "Context Config: ${config}"
}
//Add ANS credential information to the config
ansHookServiceKeyCredentialsId =
DefaultValueCache.getInstance().getDefaultValues().hooks?.ans?.serviceKeyCredentialsId
config += ["ansHookServiceKeyCredentialsId": ansHookServiceKeyCredentialsId]
// prepare stashes
// first eliminate empty stashes
config.stashContent = utils.unstashAll(config.stashContent)
// then make sure that commonPipelineEnvironment, config, ... is also available when step stashing is active
if (config.stashContent?.size() > 0) {
config.stashContent.add('pipelineConfigAndTests')
config.stashContent.add('piper-bin')
config.stashContent.add('pipelineStepReports')
}
if (parameters.stashNoDefaultExcludes) {
// Merge this parameter which is only relevant in Jenkins context
// (for dockerExecuteOnKubernetes step) and go binary doesn't know about
config.stashNoDefaultExcludes = parameters.stashNoDefaultExcludes
}
dockerWrapper(script, stepName, config) {
handleErrorDetails(stepName) {
writePipelineEnv(script: script, piperGoPath: piperGoPath)
utils.unstash('pipelineStepReports')
try {
try {
try {
credentialWrapper(config, credentialInfo) {
sh "${piperGoPath} ${stepName}${defaultConfigArgs}${customConfigArg}"
}
} finally {
jenkinsUtils.handleStepResults(stepName, failOnMissingReports, failOnMissingLinks)
}
} finally {
readPipelineEnv(script: script, piperGoPath: piperGoPath)
}
} finally {
InfluxData.readFromDisk(script)
stash name: 'pipelineStepReports', includes: '.pipeline/stepReports/**', allowEmpty: true
}
}
}
}
}
}
// reused in sonarExecuteScan
static void prepareExecution(Script script, Utils utils, Map parameters = [:]) {
def piperGoUtils = parameters.piperGoUtils ?: new PiperGoUtils(script, utils)
piperGoUtils.unstashPiperBin()
utils.unstash('pipelineConfigAndTests')
}
// reused in sonarExecuteScan
static Map prepareStepParameters(Map parameters) {
Map stepParameters = [:].plus(parameters)
stepParameters.remove('script')
stepParameters.remove('jenkinsUtilsStub')
stepParameters.remove('piperGoPath')
stepParameters.remove('juStabUtils')
stepParameters.remove('piperGoUtils')
// When converting to JSON and back again, entries which had a 'null' value will now have a value
// of type 'net.sf.json.JSONNull', for which the Groovy Truth resolves to 'true' in for example if-conditions
return MapUtils.pruneNulls(stepParameters)
}
// reused in sonarExecuteScan
static void prepareMetadataResource(Script script, String metadataFile) {
script.writeFile(file: ".pipeline/tmp/${metadataFile}", text: script.libraryResource(metadataFile))
}
// reused in sonarExecuteScan
static Map getStepContextConfig(Script script, String piperGoPath, String metadataFile, String defaultConfigArgs, String customConfigArg) {
return script.readJSON(text: script.sh(returnStdout: true, script: "${piperGoPath} getConfig --contextConfig --stepMetadata '.pipeline/tmp/${metadataFile}'${defaultConfigArgs}${customConfigArg}"))
}
static String getCustomDefaultConfigs() {
// The default config files were extracted from merged library
// resources by setupCommonPipelineEnvironment.groovy into .pipeline/.
List customDefaults = DefaultValueCache.getInstance().getCustomDefaults()
for (int i = 0; i < customDefaults.size(); i++) {
customDefaults[i] = BashUtils.quoteAndEscape(".pipeline/${customDefaults[i]}")
}
return customDefaults.join(',')
}
// reused in sonarExecuteScan
static String getCustomDefaultConfigsArg() {
String customDefaults = getCustomDefaultConfigs()
if (customDefaults) {
return " --defaultConfig ${customDefaults} --ignoreCustomDefaults"
}
return ''
}
// reused in sonarExecuteScan
static String getCustomConfigArg(def script) {
if (script?.commonPipelineEnvironment?.configurationFile
&& script.commonPipelineEnvironment.configurationFile != '.pipeline/config.yml'
&& script.commonPipelineEnvironment.configurationFile != '.pipeline/config.yaml') {
return " --customConfig ${BashUtils.quoteAndEscape(script.commonPipelineEnvironment.configurationFile)}"
}
return ''
}
// reused in sonarExecuteScan
void dockerWrapper(script, stepName, config, body) {
if (config.dockerImage) {
echo "[INFO] executing pipeline step '${stepName}' with docker image '${config.dockerImage}'"
Map dockerExecuteParameters = [:].plus(config)
dockerExecuteParameters.script = script
dockerExecute(dockerExecuteParameters) {
body()
}
} else {
body()
}
}
// reused in sonarExecuteScan
void credentialWrapper(config, List credentialInfo, body) {
credentialInfo = handleVaultCredentials(config, credentialInfo)
credentialInfo = handleANSCredentials(config, credentialInfo)
if (credentialInfo.size() > 0) {
def creds = []
def sshCreds = []
credentialInfo.each { cred ->
def credentialsId
if (cred.resolveCredentialsId == false) {
credentialsId = cred.id
} else {
credentialsId = config[cred.id]
}
if (credentialsId) {
switch (cred.type) {
case "file":
creds.add(file(credentialsId: credentialsId, variable: cred.env[0]))
break
case "token":
creds.add(string(credentialsId: credentialsId, variable: cred.env[0]))
break
case "usernamePassword":
creds.add(usernamePassword(credentialsId: credentialsId, usernameVariable: cred.env[0], passwordVariable: cred.env[1]))
break
case "ssh":
sshCreds.add(credentialsId)
break
default:
error("invalid credential type: ${cred.type}")
}
}
}
// remove credentialIds that were probably defaulted and which are not present in jenkins
if (containsVaultConfig(config)) {
creds = removeMissingCredentials(creds, config)
sshCreds = removeMissingCredentials(sshCreds, config)
}
if (sshCreds.size() > 0) {
sshagent (sshCreds) {
withCredentials(creds) {
body()
}
}
} else {
withCredentials(creds) {
body()
}
}
} else {
body()
}
}
List removeMissingCredentials(List creds, Map config) {
return creds.findAll { credentialExists(it, config) }
}
boolean credentialExists(cred, Map config) {
try {
withCredentials([cred]) {
return true
}
} catch (e) {
return false
}
}
boolean containsVaultConfig(Map config) {
def approleIsUsed = config.containsKey('vaultAppRoleTokenCredentialsId') && config.containsKey('vaultAppRoleSecretTokenCredentialsId')
def tokenIsUsed = config.containsKey('vaultTokenCredentialsId')
return approleIsUsed || tokenIsUsed
}
// Injects vaultCredentials if steps supports resolving parameters from vault
List handleVaultCredentials(config, List credentialInfo) {
if (config.containsKey('vaultAppRoleTokenCredentialsId') && config.containsKey('vaultAppRoleSecretTokenCredentialsId')) {
credentialInfo += [[type: 'token', id: 'vaultAppRoleTokenCredentialsId', env: ['PIPER_vaultAppRoleID']],
[type: 'token', id: 'vaultAppRoleSecretTokenCredentialsId', env: ['PIPER_vaultAppRoleSecretID']]]
}
if (config.containsKey('vaultTokenCredentialsId')) {
credentialInfo += [[type: 'token', id: 'vaultTokenCredentialsId', env: ['PIPER_vaultToken']]]
}
return credentialInfo
}
List handleANSCredentials(config, List credentialInfo){
if (config.containsKey('ansHookServiceKeyCredentialsId')) {
credentialInfo += [[type: 'token', id: 'ansHookServiceKeyCredentialsId', env: ['PIPER_ansHookServiceKey']]]
}
return credentialInfo
}
// reused in sonarExecuteScan
void handleErrorDetails(String stepName, Closure body) {
try {
body()
} catch (ex) {
def errorDetailsFileName = "${stepName}_errorDetails.json"
if (fileExists(file: errorDetailsFileName)) {
def errorDetails = readJSON(file: errorDetailsFileName)
def errorCategory = ""
if (errorDetails.category) {
errorCategory = " (category: ${errorDetails.category})"
DebugReport.instance.failedBuild.category = errorDetails.category
}
error "[${stepName}] Step execution failed${errorCategory}. Error: ${errorDetails.error ?: errorDetails.message}"
}
error "[${stepName}] Step execution failed. Error: ${ex}, please see log file for more details."
}
}