Run artifactPrepareVersion in maven container if necessary (#1634)

* The metadata for artifactPrepareVersion-go specifies a container for when the buildTool is maven.
* The alias to 'mavenExecute' was removed. The problem with this is that when a section containers is included in the metadata, dockerImage will always be picked up from mavenExecute, the conditional dependency on buildTool will not even be considered. Parameters such as m2Path, projectSettingsFile and globalSettingsFile should be configured in general/maven if necessary.
* When the step ends up being executed within dockerExecuteOnKubernetes, we need to preserve the .git folder. This folder would normally be excluded by the default excludes of the stash step. There was already a comment that suppressing this behavior by passing useDefaultExcludes: false was problematic (unfortunately without going into details), so I've added a new parameter to dockerExecute and dockerExecuteOnKubernetes named stashNoDefaultExcludes (note the reverted meaning to ease preserving the default behavior when this parameter is not provided). This parameter is passed to piperExecuteBin from the artifactPreferVersion groovy wrapper.

cmd/artifactPrepareVersion_generated.go

@@ -102,7 +102,7 @@ Depending on the build tool used and thus the allowed versioning format the ` +
 * Not creating a tag would lead to a loss of the final artifact version in scm which often is not acceptable
 * You need to ensure that your CI/CD system can push back to your SCM (via providing ssh or HTTP(s) credentials)
 
-**This pattern is the default** behavior (` + "`" + `versioningType: cloud` + "`" + `) since this is suitable for for most cloud deliveries.
+**This pattern is the default** behavior (` + "`" + `versioningType: cloud` + "`" + `) since this is suitable for most cloud deliveries.
 
 It is possible to use ` + "`" + `versioningType: cloud_noTag` + "`" + ` which has a slighly different behavior than described above:
 
@@ -207,7 +207,7 @@ func addArtifactPrepareVersionFlags(cmd *cobra.Command, stepConfig *artifactPrep
 	cmd.Flags().BoolVar(&stepConfig.UnixTimestamp, "unixTimestamp", false, "Defines if the Unix timestamp number should be used as build number instead of the standard date format.")
 	cmd.Flags().StringVar(&stepConfig.Username, "username", os.Getenv("PIPER_username"), "User name for git authentication")
 	cmd.Flags().StringVar(&stepConfig.VersioningTemplate, "versioningTemplate", os.Getenv("PIPER_versioningTemplate"), "DEPRECATED: Defines the template for the automatic version which will be created")
-	cmd.Flags().StringVar(&stepConfig.VersioningType, "versioningType", `cloud`, "Defines the type of versioning (`cloud`: fully automatic, `cloud_noTag`: automatic but no tag created, `library`: manual)")
+	cmd.Flags().StringVar(&stepConfig.VersioningType, "versioningType", `cloud`, "Defines the type of versioning (`cloud`: fully automatic, `cloud_noTag`: automatic but no tag created, `library`: manual, i.e. the pipeline will pick up the version from the build descriptor, but not generate a new version)")
 
 	cmd.MarkFlagRequired("buildTool")
 }
@@ -217,7 +217,7 @@ func artifactPrepareVersionMetadata() config.StepData {
 	var theMetaData = config.StepData{
 		Metadata: config.StepMetadata{
 			Name:    "artifactPrepareVersion",
-			Aliases: []config.Alias{{Name: "artifactSetVersion", Deprecated: false}, {Name: "mavenExecute", Deprecated: false}, {Name: "setVersion", Deprecated: true}},
+			Aliases: []config.Alias{{Name: "artifactSetVersion", Deprecated: false}, {Name: "setVersion", Deprecated: true}},
 		},
 		Spec: config.StepSpec{
 			Inputs: config.StepInputs{

documentation/docs/pipelines/cloud-sdk/build-tools.md

@@ -23,7 +23,7 @@ The following table gives an overview over the features available per build tool
 
 | Feature                    | Maven | npm | MTA Maven | MTA npm |
 |----------------------------|-------|-----|-----------|---------|
-| Automatic Versioning       | x     |     | x         | x       |
+| Automatic Versioning       | x     | x   | x         | x       |
 | Build                      | x     | x   | x         | x       |
 | Backend Integration Tests  | x     | x   | x         | x       |
 | Frontend Integration Tests | x     | x   | x         | x       |

resources/metadata/versioning.yaml

@@ -2,7 +2,6 @@ metadata:
   name: artifactPrepareVersion
   aliases:
     - name: artifactSetVersion
-    - name: mavenExecute
     - name: setVersion
       deprecated: true
   description: Prepares and potentially updates the artifact's version before building the artifact.
@@ -27,7 +26,7 @@ metadata:
     * Not creating a tag would lead to a loss of the final artifact version in scm which often is not acceptable
     * You need to ensure that your CI/CD system can push back to your SCM (via providing ssh or HTTP(s) credentials)
 
-    **This pattern is the default** behavior (`versioningType: cloud`) since this is suitable for for most cloud deliveries.
+    **This pattern is the default** behavior (`versioningType: cloud`) since this is suitable for most cloud deliveries.
 
     It is possible to use `versioningType: cloud_noTag` which has a slighly different behavior than described above:
 
@@ -223,12 +222,18 @@ spec:
         - STEPS
       - name: versioningType
         type: string
-        description: "Defines the type of versioning (`cloud`: fully automatic, `cloud_noTag`: automatic but no tag created, `library`: manual)"
+        description: "Defines the type of versioning (`cloud`: fully automatic, `cloud_noTag`: automatic but no
+          tag created, `library`: manual, i.e. the pipeline will pick up the version from the build descriptor,
+          but not generate a new version)"
         scope:
         - PARAMETERS
         - STAGES
         - STEPS
         default: cloud
+        possibleValues:
+          - cloud
+          - cloud_noTag
+          - library
     secrets:
     - name: gitHttpsCredentialsId
       type: jenkins
@@ -246,3 +251,11 @@ spec:
           - name: originalArtifactVersion
           - name: git/commitId
           - name: git/commitMessage
+  containers:
+    - image: maven:3.6-jdk-8
+      imagePullPolicy: Never
+      conditions:
+        - conditionRef: strings-equal
+          params:
+            - name: buildTool
+              value: maven

vars/artifactPrepareVersion.groovy

@@ -14,6 +14,11 @@ void call(Map parameters = [:]) {
         [type: 'ssh', id: 'gitSshKeyCredentialsId'],
         [type: 'usernamePassword', id: 'gitHttpsCredentialsId', env: ['PIPER_username', 'PIPER_password']],
     ]
+
+    // Tell dockerExecuteOnKubernetes (if used) to stash also .-folders
+    // This preserves the '.git' folder into the pod and restores it from the pod with the created tag.
+    parameters['stashNoDefaultExcludes'] = true
+
     parameters = DownloadCacheUtils.injectDownloadCacheInParameters(script, parameters, BuildTool.MAVEN)
     piperExecuteBin(parameters, STEP_NAME, METADATA_FILE, credentials, false, false, true)
 }

vars/dockerExecute.groovy

@@ -103,7 +103,14 @@ import groovy.transform.Field
      */
     'stashContent'
 ])
-@Field Set PARAMETER_KEYS = STEP_CONFIG_KEYS
+@Field Set PARAMETER_KEYS = STEP_CONFIG_KEYS.plus([
+    /**
+     * In the Kubernetes case the workspace is only available to the respective Jenkins slave but not to the containers running inside the pod.<br />
+     * This flag controls whether the stashing does *not* use the default exclude patterns in addition to the patterns provided in `stashExcludes`.
+     * @possibleValues `true`, `false`
+     */
+    'stashNoDefaultExcludes',
+])
 
 /**
  * Executes a closure inside a docker container with the specified docker image.
@@ -163,7 +170,8 @@ void call(Map parameters = [:], body) {
                         dockerPullImage: config.dockerPullImage,
                         dockerEnvVars: config.dockerEnvVars,
                         dockerWorkspace: config.dockerWorkspace,
-                        stashContent: config.stashContent
+                        stashContent: config.stashContent,
+                        stashNoDefaultExcludes: config.stashNoDefaultExcludes,
                     ){
                         echo "[INFO][${STEP_NAME}] Executing inside a Kubernetes Pod"
                         body()
@@ -179,6 +187,7 @@ void call(Map parameters = [:], body) {
                         dockerEnvVars: config.dockerEnvVars,
                         dockerWorkspace: config.dockerWorkspace,
                         stashContent: config.stashContent,
+                        stashNoDefaultExcludes: config.stashNoDefaultExcludes,
                         containerPortMappings: config.containerPortMappings,
                         sidecarName: parameters.sidecarName,
                         sidecarImage: parameters.sidecarImage,

vars/dockerExecuteOnKubernetes.groovy

@@ -88,7 +88,7 @@ import hudson.AbortException
     'dockerImage',
     /**
      * Set this to 'false' to bypass a docker image pull.
-     * Usefull during development process. Allows testing of images which are available in the local registry only.
+     * Useful during development process. Allows testing of images which are available in the local registry only.
      */
     'dockerPullImage',
     /**
@@ -106,7 +106,7 @@ import hudson.AbortException
     'sidecarName',
     /**
      * Set this to 'false' to bypass a docker image pull.
-     * Usefull during development process. Allows testing of images which are available in the local registry only.
+     * Useful during development process. Allows testing of images which are available in the local registry only.
      */
     'sidecarPullImage',
     /**
@@ -151,7 +151,14 @@ import hudson.AbortException
      * * `workspace`: Pattern for stashing towards container
      * * `stashBack`: Pattern for bringing data from container back to Jenkins workspace. If not set: defaults to setting for `workspace`.
      */
-    'stashIncludes'
+    'stashIncludes',
+    /**
+     * In the Kubernetes case the workspace is only available to the respective Jenkins slave but not to the containers running inside the pod.<br />
+     * This configuration defines include pattern for stashing from Jenkins workspace to working directory in container and back.
+     * This flag controls whether the stashing does *not* use the default exclude patterns in addition to the patterns provided in `stashExcludes`.
+     * @possibleValues `true`, `false`
+     */
+    'stashNoDefaultExcludes'
 ])
 @Field Set PARAMETER_KEYS = STEP_CONFIG_KEYS.minus([
     'stashIncludes',
@@ -183,14 +190,14 @@ void call(Map parameters = [:], body) {
 
         def utils = parameters?.juStabUtils ?: new Utils()
 
-        ConfigurationHelper configHelper = ConfigurationHelper.newInstance(this)
+        Map config = ConfigurationHelper.newInstance(this)
             .loadStepDefaults()
             .mixinGeneralConfig(script.commonPipelineEnvironment, GENERAL_CONFIG_KEYS)
             .mixinStepConfig(script.commonPipelineEnvironment, STEP_CONFIG_KEYS)
             .mixinStageConfig(script.commonPipelineEnvironment, parameters.stageName ?: env.STAGE_NAME, STEP_CONFIG_KEYS)
             .mixin(parameters, PARAMETER_KEYS)
             .addIfEmpty('uniqueId', UUID.randomUUID().toString())
-        Map config = configHelper.use()
+            .use()
 
         new Utils().pushToSWA([
             step         : STEP_NAME,
@@ -322,10 +329,11 @@ chown -R ${runAsUser}:${fsGroup} ."""
         stash(
             name: stashName,
             includes: includes,
-            excludes: excludes
+            excludes: excludes,
+            // 'true' by default due to negative side-effects, but can be overwritten via parameters
+            // (as done by artifactPrepareVersion to preserve the .git folder)
+            useDefaultExcludes: !config.stashNoDefaultExcludes,
         )
-        //inactive due to negative side-effects, we may require a dedicated git stash to be used
-        //useDefaultExcludes: false)
         return stashName
     } catch (AbortException | IOException e) {
         echo "${e.getMessage()}"

vars/piperExecuteBin.groovy

@@ -64,6 +64,12 @@ void call(Map parameters = [:], stepName, metadataFile, List credentialInfo, fai
                 echo "Context Config: ${config}"
             }
 
+            if (parameters.stashNoDefaultExcludes) {
+                // Merge this parameter which is only relevant in Jenkins context
+                // (for dockerExecuteOnKubernetes step) and go binary doesn't know about
+                config.stashNoDefaultExcludes = parameters.stashNoDefaultExcludes
+            }
+
             dockerWrapper(script, config) {
                 handleErrorDetails(stepName) {
                     credentialWrapper(config, credentialInfo) {
@@ -111,6 +117,7 @@ void dockerWrapper(script, config, body) {
             dockerImage: config.dockerImage,
             dockerWorkspace: config.dockerWorkspace,
             dockerOptions: config.dockerOptions,
+            stashNoDefaultExcludes : config.stashNoDefaultExcludes,
             //ToDo: add additional dockerExecute parameters
         ) {
             body()