* changes to detectExec before master merge
* changes for detectExecuteScan
* self generated code added
* fix syntax errors and update docu
* added unit tests for fail and Group
* fix failOn bug
* add Groups as string array
* add Groups as string array
* tests and validation for groups, failOn
* Updated docs and added more tests
* documentation md files should not be changed
* Handle merge conflicts from PR 1845
* fix merge errors
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
This change enables the use of the new npmExecuteEndToEndTests in the
stages release and acceptance if it is configured. If it is not
configured, nothing should change.
This change adds a buildDescriptorExcludeList parameter to
npmExecuteScripts, to enable the exclusion of certain directories when
executing npm scripts. Previously, npmExecuteScripts could only execute
scripts in all packages.
Now it is possible to provide paths or patterns as elements of the
buildDescriptorExcludeList to exclude packages when executing npm scripts.
This change adds the missing report handling to npmExecuteEndToEndTests
step. In addition, it extends the testsPublishResults step to support
cucumber test results.
The multicloudDeploy step is newer and more capable compared to the old steps. For pipelines that don't configure the multicloudDeploy step, nothing should change.
* Whitesource MVP for Gradle, Golang, and NPM/Yarn
* Refactoring
* Refactor and cleanup, better error checking
* publish stepResults, use pkg/versioning, bubble up errors, add gomod versioning support
* Run gofmt and cleanup comments
* Resolve PR comments
* Update resources/metadata/whitesource.yaml
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
* Only determine project coordinates if they are missing
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
* Gradle versioning artifact
* fix gradle artifact version regexp and refactor
* Fix token extraction from output buffer
* Fix some issues with pip and jsonfile versioning logic
* Remove useless spacing
* Remove unnecessary test file and fix naming style for JSONDescriptor
* Automatically download wss-unified-agent if file does not exist
* adds downloadVulnerabilityReport, checkSecurityViolations, minor refactoring
* adds config.ReportDirectoryName, improves readability
* Version-wide reporting for vulnerabilities and list of libraries.
* Refactor and improve build accuracy
* fix sed command
* Add includes file pattern config option
* Adds --exclude command line flag
* run go mod tidy and regenerate step framework
* Fix unit tests
* revert changes
* poll project status before downloading reports
* merge with master
* go mod tidy, go fmt, and fix whitesource unit test
* sync go.mod
* sync go.mod again
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* Fix alias cfServiceKeyName
* Adapt documentation
* Fix alias use in go files
Co-authored-by: Daniel Mieg <56156797+DanielMieg@users.noreply.github.com>
* add APIs for getNodes, getExtDescriptor, putExtDescriptor and uploadExtDescriptor
* add new params and extend the logic to enable upload or update extension descriptor to node
* extends unit tests
Co-authored-by: Zihe Cheng <zihe.cheng@sap.com>
Co-authored-by: Marcus Holl <marcus.holl@sap.com>
Ensure npm dev dependencies are available after mtaBuild as they are required by certain tests.
Co-authored-by: Kevin Hudemann <kevin.hudemann@sap.com>
Co-authored-by: Daniel Kurzynski <daniel.kurzynski@sap.com>
This change adds linting support for Java-/TypeScript projects based on the checkDefaultLint and checkUserLint steps from Cloud SDK Pipeline.
Following options are supported:
1. Define a linting script named ci-lint in the package.json file(s) of the project
2. Provide a configuration for ESLint as part of the project
3. If none of the above: ESLint is executed with a general purpose configuration.
Co-authored-by: Daniel Kurzynski <daniel.kurzynski@sap.com>
This change refactors the npm pkg and npmExecuteScripts implementations
to be reusable for future steps, e.g., npmExecuteLint.
In addition, it fixes few small bugs related to unit test execution on
Windows and the fileUtils mocking implementation.
Co-authored-by: Daniel Kurzynski <daniel.kurzynski@sap.com>
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
Extend mta build step to install maven artefacts after build to allow re-using them in later stages (additional unit or integration tests which might not be running as part of the "build" life-cycle).
* The metadata for artifactPrepareVersion-go specifies a container for when the buildTool is maven.
* The alias to 'mavenExecute' was removed. The problem with this is that when a section containers is included in the metadata, dockerImage will always be picked up from mavenExecute, the conditional dependency on buildTool will not even be considered. Parameters such as m2Path, projectSettingsFile and globalSettingsFile should be configured in general/maven if necessary.
* When the step ends up being executed within dockerExecuteOnKubernetes, we need to preserve the .git folder. This folder would normally be excluded by the default excludes of the stash step. There was already a comment that suppressing this behavior by passing useDefaultExcludes: false was problematic (unfortunately without going into details), so I've added a new parameter to dockerExecute and dockerExecuteOnKubernetes named stashNoDefaultExcludes (note the reverted meaning to ease preserving the default behavior when this parameter is not provided). This parameter is passed to piperExecuteBin from the artifactPreferVersion groovy wrapper.
Provide a timeout parameters to malwarescan step. This is forwarded to the piper http layer. The default used there is 10 seconds with is not useable for that use case for larger files.
* Update fortify.yaml
* src, exclude and pythonAdditionalPaths are now lists of strings
* Re-implement pythonIncludes and pythonExcludes as aliases of src and exclude
* Fix using the correct separator (; on windows, : on everything else)
* Tokenize also python "includes"
* mvnCustomArgs was removed
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
* also reduce code duplication in token fetching
* concatenate classpaths from multi-maven projects
Co-authored-by: Daniel Kurzynski <daniel.kurzynski@sap.com>
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
Co-authored-by: Kevin Hudemann <kevin.hudemann@sap.com>
Co-authored-by: Daniel Kurzynski <daniel.kurzynski@sap.com>
* added new step gctsDeployCommit
* suggested PR fixes applied
* fixed test
* Remove unused imports
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
* added URL encoding for 'request' parameter
* regenerate after change
* minor changes
* renamed step to 'gctsDeploy'
* changed name
* remove space
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* Create step to save container image
* add error handling for directory creation
* Update permissions
* remove indirection and add tests
* finalize tests
* Fix test to also run on linux environments
* Update cmd/containerSaveImage.go
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
* Update resources/metadata/savecontainer.yaml
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
* Update cmd/containerSaveImage.go
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
* Update cmd/containerSaveImage_test.go
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
* Update generated file
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
* Enable multiple repositories
* Improve readability
* Adapt documentation
* rerun generator
* Add initial info line
* Improve logging
* Enable configuration in GENERAL section
* Adapt yml abapEnvironmentPullGitRepo
* Add aliases for crendetialsId
* Change order of credentialsId aliases
* Change prio of different names of credentialsId
* Add docu to sidebar
* write flag for sonar execution to influx
* add missing step metadata file
* first attempt to read influx from disk
* add missing import
* correct glob pattern
* use file path
* correct type names
* cleanup
* fix code climate issue
* fix typo
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
* add test case
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
* artifactPrepareVersion: additional options
it is now possible to use
* a Unix timestamp number
* a short commit hash
* Add dedicated Docker versioning scheme
* fix docu
* enhanced step docu and metadata
* regenerate after changing metadata
* small wording change
Co-authored-by: Marcus Holl <marcus.holl@sap.com>
* add possible values for xsDeploy
* add possible values for protecodeExecuteScan
* add possible values for mtaBuild
* add possible values for artifactPrepareVersion
* add possible values for sonarExecuteScan
* add possible values for detectExecuteScan
* add possible values for checkmarxExecuteScan
* Update gctsCreateRepository.yaml
This change enables the use of the artifactPrepareVersion go step instead
of the deprecated artifactSetVersion groovy implementation for the Cloud
SDK Pipeline. The groovy wrapper for the artifactPrepareVersion step now
enables the use of the download cache and minor issues are fixed.
With the step gctsCreateRepository it is possible to create a local gCTS repository on an ABAP server
Co-authored-by: Marcus Holl <marcus.holl@sap.com>
* Extends kubernetesDeploy step to support Helm 3
Currently, the kubernetesDeploy step has no support to Helm 3 due to the fact that:
- the initialization command used works only for Helm 2
- the image used when running the helm CLI is based on Helm 2
The need for Helm 3 support comes from the fact that Helm 3 introduces major architectural changes,
more specifically, the removal of its server-side agent called Tiller - thus, being incompatible with
one another.
This commit adds this support by introducing a new configuration field (helmVersion).
By default, its values is set to 2 (Helm 2) to avoid breaking any existing functionalities.
* Use deployTool field to decide between Helm 2 or 3
* Remove helm init and replace wait for atomic in v3
* Update cmd/kubernetesDeploy.go
Nice catch!
Co-Authored-By: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
* Add documentation for kubernetesDeploy step
* Add helm3 example for kubernetesDeploy step using mandatory fields
* Add new line at the end of kubernetesDeploy documentation
* Link kubernetesDeploy step with docs generator
* Add possible values for deployTool in kubernetesDeploy
* dummy change
* Revert "dummy change"
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
* use sonar go in groovy
* use SONAR_TOKEN
* only use owner/repo if both are set
* trim version to major version digit
* fix code climate issues
* remove Sonar defaults
* use sonar go step
* use SONAR_TOKEN
* use certs from parameters
* use docker workspace & options
* add instance parameter
* implement branchName
* implement branchName
* remove duplicate default
* update docs
* fix TODOs
* remove merge mess
* fix code climate issue
* address comments
* respect custom defaults and custom config file name
* fix typo
* remove obsolete test cases
* disable CommonStepChecks
* check step config instead of context config
* remove TODOs
* respect jenkins PR envvars
* use value from stepConfig
* Update vars/sonarExecuteScan.groovy
* rename options to config
* correct type for options
* add test cases
* log sonar.options in debug message
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* iterate over found pom.xml files for maven projects, deploy found artifacts
* classifiers are found by testing prefix/suffix of artifact
* Remove additionalClassifiers option
* Improve documentation
* Fix fallback for finalBuildName
* Ignore mvn modules with packaging != pom but no target dir
* Fail when main artifact is missing, add more tests
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
* Add ReportAggregator to collect information about executed stages and create reports
* ReportAggregator is used by the SAP Cloud SDK to collect information about which stages were executed,
code coverage and many more measurements.
* based on the measurements a pdf report can be generated which can be used to get a certification by SAP
* Add ReportAggregator to collect information about executed stages and create reports
* ReportAggregator is used by the SAP Cloud SDK to collect information about which stages were executed,
code coverage and many more measurements.
* based on the measurements a pdf report can be generated which can be used to get a certification by SAP
Artifacts to upload are assembled for MTA projects and Maven projects with optional application sub-module. Then maven deploy:deploy-file is used as backend to upload bundles of artifacts plus sub-artifacts.
Co-authored-by: Florian Wilhelm <florian.wilhelm02@sap.com>
* Golang step metadata: Config aliases for steps
This will ease following scenarios:
* config migration due to step name changes
* re-use of more general config, e.g. `mavenExecute` in `mavenBuild`
* fix CodeClimate finding
* Fix panic if original stage config does not exist yet
* add static code checks for maven based projects as:
* pmd plugin
* spotBugs plugin
* test modules as unit-tests and integration-tests will be ignored by default. Additional modules to ignore are configurable
* for pmd: rulesets and excludes are configurable
* for spotBugs: includeFilter and excludeFilter are configurable
* Update schema and hcp deployer version in mta.yaml file
* Add name parameter in mta yaml file
* Rename template_mta.yml to template_mta.yaml
* Fix indentation
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
Co-authored-by: Marcus Holl <marcus.holl@sap.com>
* Protecode as go implementation
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
The DebugReport is a global instance where steps can store information relevant for diagnosing failed pipelines. In the SDK Pipeline, this is used to generate a debug report within the postActionArchiveDebugLog step. The reason for adding this to Piper is to feed information about extended or overwritten stages in piperStageWrapper into the DebugReport, as was done before in the SDK Pipeline's equivalent runAsStage step.
* Use commonPipelineEnvironment in go binary
* Update groovy part incl. tests
* Rework structure and naming
* Support influx resources in steps
* Update tests and some cleanups
* Add correct defer handling
* Address PR feedback
* Fix test
* Update resources.go
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
* chore(go): add schema for go metadata files
* chore(go): use k8s container definition in schema
* chore(go): add reusable schema definitions
* chore(go): add further schema definitions
* docs(schema): add description how to use metadata schema
