* initial commit of yaml file
* initial commit for HaDoLint in GO
* add helper function to load file from url
* load config file
* write report information to disk
* comment the code
* refactor groovy code
* remove download function from FileUtils
* use http.Downloader
* rename step files
* update generated files
* update generated files
* remove duplicate commands
* add credentials for config url
* add generated test file
* reuse piperExecuteBin functions
* correct step name
* update go step
* deactivate test
* fix import
* use differing go step name
* rename step
* correct result publishing
* correct command name
* expose tls insecure flag
* hand through error
* disable tls verification
* fix tls disabling
* use credentials
* mow
* reformat
* add qgate only if set
* correct report name
* remove old defaults
* add qgate to defaults
* handle report name
* restore default
* remove unused step config
* use piperExecuteBin
* remove obsolete type
* add test cases
* remove groovy tests
* move client parameter handling to run function
* use custom interfaces and mockery
* remove commented code
* correct struct names
* rename parameter dockerfile
* add further asserts
* cleanup
* change file permission to read/write
* remove tokenize
* add further comments
* init http client only if necessary
* add todo
* Revert "rename parameter dockerfile"
This reverts commit 2a570685b8.
* add alias for dockerfile parameter
* correct test case
* Apply suggestions from code review
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
* add comment about mock assertions
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
* fix checkmarx docs
* go generate
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
This change adds the steps mavenExecuteIntegration, mavenExecuteStaticCodeChecks,
npmExecuteLint and npmExecuteScripts to the stage piperPipelineStagePRVoting.
* new 'keepFailedDeployments' parameter which removes the --atomic flag
* Update resources/metadata/kubernetesdeploy.yaml
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* go generate
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* Make sure the UA scan is known to the scan object. Fixes downloading reports later on.
* Move polling into pkg/whitesource, add test for e2e scan
* Remove conditions from stash config resource
* Don't use version stored in CPE. This will prevent the versioningModel from being applied.
* Instead of letting the pipeline fail for the first config validation error, accumulate all errors and output them to the log, then fail.
* Beginnings of a resource for validating a config.yml migrated to the GPP. Can be configured in the general section:
```yaml
general:
legacyConfigSettings: 'com.sap.piper/pipeline/cloudSdkToGppConfigSettings.yml'
```
Without this change the default setting from dockerExecute
and dockerExecuteOnKubernetes are overruled.
With this change the user has to explicitly configure a step
if they want this behaviour.
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
This change extends the npmExecuteScripts step to support execution of
npm scripts for specific modules. Previously, it was not possible to
execute npm scripts only for specific modules. Now, if the parameter
buildDesriptorList is set the scripts defined by the runScripts
parameter will be executed for the modules defined by
buildDescriptorList. Note, in this case the buildDescriptorExcludeList
will be ignored.
* makes containerImage not mandatory
* Adds kubectl container
* Adds log statement to debug
* adds general to container image
* removes GENERAL again
Removes condition from Kubectl container
* removes workDir
* marks logs as debug
* adds workingdir again
* Adds author to commits
* Adds commit time now
* remove deprecated and reorder
* adds deprecated again to containerRegistryUrl
Adds GENERAL scope to containerImage
* updates generated file
* Renames containerImageNameTag
* adds else case
* adds debug log
* code cleanup
* adds debug log
* revert
* adds debug logs
* revert
* makes root path not hidden
* revert
* Read container properties
* Removes debug message
* Removes debug message
* Removes general scope again
* Fixes unit test
* Adds helm capabilities to the gitopsUpdateDeployment step
* Adds helm capabilities to gitopsUpdateDeployment step
* Removes condition from input field
* Adds test for invalid deploy tool
* Fixes typo
* Adds tests for git errors and file errors
Simplifies test setup
* Adds test for error on image name extraction
* fixes URL variable name
* adds workind directory to paths
* Refactors too long method
* Reverts refactoring method
* Adds repository name as parameter
* Adds glob method
* Test glob method
* Revert "Test glob method"
This reverts commit ac11b54c14.
* Revert "Adds glob method"
This reverts commit ddf47ddebe.
* Revert "Adds repository name as parameter"
This reverts commit 8fc471c909.
* Removes getWd
* Adds stash deployDescriptor
* removes = from paramters
* Revert "removes = from paramters"
This reverts commit 3ecb3665e2.
* Adds " around parameters
* adds logging of all files
* Updates helm to version 3.3.4
* Clean up debug logs
* Raise error if no branch name provided.
Defaulting should be handled by step configuration.
* clean code
* Updates fields and adds checks for required field for certain deploy tools
* Fixes default commit message
* Update long description
* Removes default parameter
* Update resources/metadata/gitopsUpdateDeployment.yaml
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
* Updates yaml file
* Add error category and removes too much wrapping
* Update generated file
* Checks all parameters before returning the error
* Introduces constant
* Renames constant
* Fixes unit tests
* unexpose constants
* Makes tests thread safe and resilient to failed deletion
* Remove methods that did not work properly with hash containers rather than tags.
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* makes containerImage not mandatory
* Adds kubectl container
* Adds log statement to debug
* adds general to container image
* removes GENERAL again
Removes condition from Kubectl container
* removes workDir
* marks logs as debug
* adds workingdir again
* Adds author to commits
* Adds commit time now
* remove deprecated and reorder
* adds deprecated again to containerRegistryUrl
Adds GENERAL scope to containerImage
* updates generated file
* Renames containerImageNameTag
* adds else case
* adds debug log
* code cleanup
* adds debug log
* revert
* adds debug logs
* revert
* makes root path not hidden
* revert
* Read container properties
* Removes debug message
* Removes debug message
* Removes general scope again
* Fixes unit test
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
The approach used up to now did not (... never) work since the uploaded content was always assigned to the `$TMP` package. In the meanwhile there is some fiori tooling available for managing such uploads.
Now we re-use this fiori tooling.
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
This change updates the stash settings, stage defaults and the containers map to
allow a single call of `piperPipelineStageIntegration` in SAP Cloud SDK Pipeline.
* kanikoExecute: improve user experience
* ensure proper tags
* update permissions
in case a container runs with a different user
we need to make sure that the orchestrator user
can work on the file
* update permissions
* ensure availablility of directories on Jenkins
* (fix) clean up tmp dir in test
* add resilience for incorrect step yaml
* incorporate PR feedback
* Adds piper step to update deployment configuration in external git repository.
https://github.wdf.sap.corp/ContinuousDelivery/piper-ita/issues/21
* Adds handling of branchName as an optional parameter
* Update resources/metadata/gitopsUpdateDeployment.yaml
Feedback about description
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
* Adapt to interface guide
* Refactors to GitopsExecRunner
* Refactors to GitopsExecRunner in test
* Removes unnecessary mocked methods
* Adds tests for git utils
* Adds new step to CommonStepsTest.groovy
* Updates description from yaml
* Restricts visibility of methods and interfaces
Adds comments where necessary
* Updates comments
* Fixes URL name
* updates description
* updates generated file
* Fixes compile issue in CommonStepsTest.groovy
* Updates long description
* Updates test to run green on all kind of OS
* Removes global variables from tests
* Default branch: master
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* Typo in Hierarchy
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* Refactors test to allow parallel execution
* Renames utility variable in gitopsUpdateDeployment.go
* Renames error variables in gitopsUpdateDeployment.go
* simplified parameters for kubectl
* Refactors util classes to use parameters rather than global variables
* makes username and password mandatory
* remove unnecessary mandatory flag
* remove new methods from mock that are not necessary
* replaces with EqualError
* replaces with NoError
* update generated file
* refactor tests
* refactor tests
* make tests parallel executable
* parallel execution of tests
* Refactors interfaces to stop exposing interfaces
* Feedback from PR
* Simplifies failing mocks
* Renames variables and interfaces
* Fixes error messages
* shorten variable names
* Renames unused parameters in tests
* Cleanup nil parameters
* Typo
* Wrap errors and remove unnecessary logs
* Remove containername and filePath from GENERAL scope
* correct generated file
* corrects expected error messages
Co-authored-by: OliverNocon <oliver.nocon@sap.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
This change fixes an issue with the default maven excludes where the step would fail when performance tests are present in the project, but are not a module in the root pom. In addition, it provides a small fix of a method where a wrong path was passed to an error.
* kanikoExecute: improve user experience
* ensure proper tags
* update permissions
in case a container runs with a different user
we need to make sure that the orchestrator user
can work on the file
* update permissions
* ensure availablility of directories on Jenkins
* (fix) clean up tmp dir in test
* add resilience for incorrect step yaml
* incorporate PR feedback
* docu update for issue 2157
docu update for issue 2157
* Docu updates for issue #2157 take 2
* Docu updates for issue #2157 take 3
* Docu updates for issue #2157 take 4
* add type to sonar field
* respect type of influx fields
* update generated code
* switch type
* copy changes from #1885
* log JSON data
* read simple values from json
* Update InfluxData.groovy
* Revert "Update InfluxData.groovy"
This reverts commit c8cfdf381f.
* Revert "read simple values from json"
This reverts commit 94b69866d2.
* Revert "copy changes from #1885"
This reverts commit 2471b4475e.
* update TODO
This reverts commit 32fd89c664.
Commit mentioned above has been merged in order to circumvent
the issue which has been resolved by commit 0a17ed56. Since this
issue is now resolved and since there is no other known use case
for running deployments explicitly in parallel we decided to remove
that flag again.
* make dockerUsername secret
* regenerate sources
Co-authored-by: Daniel Kurzynski <daniel.kurzynski@sap.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
This change fixes the stash settings for the security stage in Cloud
SDK Pipeline. Previously, the SOURCES stash was not unstashed, thus
potentially existing extensions could not be found.
* align parameters for Detect
* include feedback
* update generation and formatting
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Daniel Kurzynski <daniel.kurzynski@sap.com>
This change updates the Cloud SDK documentation with the changes of migrating to
the piperPipelineStageInit and update the legacy configuration settings accordingly.
In addition, it fixes the documentation of the step debugReportArchive and includes
it in the list of library steps.
This change adds necessary functionality to support initialization of
Cloud SDK pipelines. In addition, it adds the support to check the
pipeline configuration for legacy configuration parameters.
This change adds the capabilities to check if the user still uses legacy
configuration parameters and allows to fail the pipeline in the
piperPipelineStageInit in case incompatible configuration is found.
Thus, it is now possible to describe breaking changes in the
configuration with the use of a yaml resource file. In this file, one can
describe the changes in configuration, e.g., when a configuration parameter
was replaced or removed. A concrete example for such a file for the Cloud
SDK Pipeline is part of this PR.
* adding defult parameter to exclude test to detectExecute yml
* generating the modified cmd/detectExecuteScan_generated.go
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* Add projectKey and coverageExclusions params
* Also add binary, coverage exclusions and jacoco related options to sonar execution.
Co-authored-by: Daniel Kurzynski <daniel.kurzynski@sap.com>
Co-authored-by: Kevin Hudemann <kevin.hudemann@sap.com>
This change adds the support for executing maven static code checks and
linting using the mavenExecuteStaticCodeChecks and npmExecuteLint steps
as part of the piperPipelineStageBuild.
Co-authored-by: Florian Wilhelm <florian.wilhelm02@sap.com>
This change extends the testsPublishResults step to support publishing
of HTML reports using the HTML publisher plugin. This change is a
preperation for the migration of frontend unit tests as already existing
in the SAP Cloud SDK Pipeline to the
piperPipelineStageAdditionalUnitTests.
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
This change updates the Cloud SDK stage defaults, makes the piperPipelineStageMavenStaticCodeChecks use the step condition and adds the onlyRunInProductiveBranch configuration option to npmExecuteEndToEndTests step.
This change add the support for running integration tests using the
recently introduced step mavenExecuteIntegration in
piperPipelineStageIntegration. In addition, capabilities to provide
temporary credentials to these tests is added using the
writeTemporaryCredentials step.
The SAP NPM registry has been migrated to the default public registry,
thus the separate configuration with the sapNpmRegistry is not required
anymore.
All packages from npm.sap.com have been migrated to npmjs.org
and in the future SAP packages will only be available from the default
public registry.
* Add abaputils pkg and go files
* Add ReadServiceKeyAbapEnvironment function
* Fixes
* Add structs for SC, Pull and Branch
* Minor Improvements
* Adapt unit tests to new abaputils pkg
* Fixes
* Add adapted tests
* Fixes
* Fix cloudfoundry test
* Add check for host prefix (HTTPS)
* Fix tests + cleanup
* Fixes
* Fixes
* Fix
* Add mock for abaputils pkg unit tests
* Adapt abaputils comments
* Add abapEnvironmentCheckoutBranch step setup
* Change description of abapEnvCheckoutBranch step
* Add http client code
* Disable code due to missing interace
* Add coding for use of abaputils
* Adapt checkout branch step
* Adapt URL for checkout_branch function import
* Fixes
* Add unit test for missing params case
* Fix for missing mapping of CfSpace
* Fix for missing mapping of CfSpace
* Add working code for a Branch Checkout
* Fix host schema
* Remove LogoutOption param of unit tests and steps
* Fix unit test
* Fix unit test CF ReadServiceKey
* Add abapEnvironmentCheckoutBranch step setup
* Change description of abapEnvCheckoutBranch step
* Add http client code
* Disable code due to missing interace
* Add coding for use of abaputils
* Adapt checkout branch step
* Adapt URL for checkout_branch function import
* Fixes
* Fix for missing mapping of CfSpace
* Add working code for a Branch Checkout
* Adapt changes of abautils pkg
* Add test for polling
* Minor fix
* Fix yaml spacing
* Add longdescription to yaml
* Refactor abaputil methods
* Refactoring
* Refactoring
* Minor fix
* Minor fixeds
* Adapt to new abaputils.AUtilsMock
* Delete obsolete initial checks for params
* Fix manageGitRepoUtils_test.go
* Adjust pollEntity tests
* Add abapEnvironmentCheckoutBranch.groovy file
* Add documentation for abapEnvCheckoutBranch step
* Add step to whitelist in commonStepsTest.groovy
* Add GENERAL scope to host parameter
* Fix generated.go file
Co-authored-by: Daniel Mieg <56156797+DanielMieg@users.noreply.github.com>
* Add abaputils pkg and go files
* Add ReadServiceKeyAbapEnvironment function
* Fixes
* Add structs for SC, Pull and Branch
* Minor Improvements
* Adapt unit tests to new abaputils pkg
* Fixes
* Add adapted tests
* Fixes
* Fix cloudfoundry test
* Add check for host prefix (HTTPS)
* Fix tests + cleanup
* Fixes
* Fixes
* Fix
* Add mock for abaputils pkg unit tests
* Adapt abaputils comments
* Add abapEnvironmentCheckoutBranch step setup
* Change description of abapEnvCheckoutBranch step
* Add http client code
* Disable code due to missing interace
* Add coding for use of abaputils
* Adapt checkout branch step
* Adapt URL for checkout_branch function import
* Fixes
* Add unit test for missing params case
* Fix for missing mapping of CfSpace
* Fix for missing mapping of CfSpace
* Add working code for a Branch Checkout
* Fix host schema
* Remove LogoutOption param of unit tests and steps
* Fix unit test
* Fix unit test CF ReadServiceKey
* Add abapEnvironmentCheckoutBranch step setup
* Change description of abapEnvCheckoutBranch step
* Add http client code
* Disable code due to missing interace
* Add coding for use of abaputils
* Adapt checkout branch step
* Adapt URL for checkout_branch function import
* Fixes
* Fix for missing mapping of CfSpace
* Add working code for a Branch Checkout
* Adapt changes of abautils pkg
* Add test for polling
* Minor fix
* Fix yaml spacing
* Add longdescription to yaml
* Refactor abaputil methods
* Refactoring
* Refactoring
* Minor fix
* Minor fixeds
* Adapt to new abaputils.AUtilsMock
* Delete obsolete initial checks for params
* Fix manageGitRepoUtils_test.go
* Adjust pollEntity tests
Co-authored-by: Daniel Mieg <56156797+DanielMieg@users.noreply.github.com>
Provide cloudFoundryDeploy step in GO layer.
Groovy part untouched. Groovy-Stub needs to be provided later (with a feature toggle in order to be able to switch back)
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
* kubernetesDeploy: add step configuration alias
This is used for backward compatibility
* fix commonPipelineEnvironment references
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
* changes to detectExec before master merge
* changes for detectExecuteScan
* self generated code added
* fix syntax errors and update docu
* added unit tests for fail and Group
* fix failOn bug
* add Groups as string array
* add Groups as string array
* tests and validation for groups, failOn
* Updated docs and added more tests
* documentation md files should not be changed
* Handle merge conflicts from PR 1845
* fix merge errors
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
This change enables the use of the new npmExecuteEndToEndTests in the
stages release and acceptance if it is configured. If it is not
configured, nothing should change.
