1
0
mirror of https://github.com/SAP/jenkins-library.git synced 2024-12-14 11:03:09 +02:00
Commit Graph

2780 Commits

Author SHA1 Message Date
Roland Stengel
f7015b562d
remove library statement from piperPipeline (#1595)
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-06-18 08:24:47 +02:00
Chris Bo
e9ad8c5e9c
Introducing new step 'gctsCloneRepository' (#1468)
* added new step gctsCloneRepository

* enhanced docu

* adjusted test case name

* enhanced docu

* refined error messages

* regenerated & remove unused Groovy imports

* small change

* remove execRunner

* Update documentation/docs/steps/gctsCloneRepository.md

* regenerated

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-06-18 07:45:22 +02:00
Florian Wilhelm
b5d4e7fd90
Avoid need of cd to install maven modules (#1677)
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-06-17 19:08:43 +02:00
Daniel Kurzynski
498e527c6c
Add npmExecuteScripts to integration stage (#1674)
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-06-17 18:17:05 +02:00
Daniel Kurzynski
743fca43e1
Remove projectVersion from fortify documentation (#1628)
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-06-17 17:47:59 +02:00
Marcus Holl
2bd15543d0
Introduce interface for manifest (#1676)
We need the interface in order to inject mocks for tests.

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-06-17 15:19:40 +02:00
Marcus Holl
39d256d33c
Introduce manifest utils for cf (#1608)
Provide a manifest util for transforming legacy manifests for cf deployments.

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-06-16 17:50:26 +02:00
Oliver Nocon
8e74952166
Switch Detect scan to golang implementation (#1673)
* Switch Detect scan to golang implementation

* update workingDir

* use universal container as default

* Switch to official curl image

* fix code climate issue

* switch default image

* update docker options

* update CommonStepsTest

* remove invalid groovy test

* clean up defaults

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-06-16 17:21:59 +02:00
Marcus Holl
dcd9972516
No parallel test exec for file tests (#1675) 2020-06-16 14:50:11 +02:00
Florian Wilhelm
1d3bd11621
Adapt mavenExecute example call to new api (#1672)
The old one won't work
2020-06-16 12:43:40 +02:00
Daniel Kurzynski
0222bf83d1
Run npm scripts in virtual frame buffer and extend command.go to run executable asynchronously (#1669)
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
Co-authored-by: Florian Wilhelm <florian.wilhelm02@sap.com>
2020-06-16 11:42:51 +02:00
Florian Wilhelm
b0329cd2c0
Use absolute path for m2path in mta install (#1671) 2020-06-16 09:48:47 +02:00
Daniel Kurzynski
9a30758115
Consider stage config from custom defaults (#1670) 2020-06-16 09:06:37 +02:00
Marcus Holl
7a132206a9
transport request upload file: Short notice about not working CTS upload (#1631)
Co-authored-by: Thorsten Duda <thorsten.duda@sap.com>
2020-06-16 08:13:49 +02:00
Dominik Lendle
0fd9c4be84
cf create/delete service: Changing to workingDir (#1560)
* minor changes

* Changed yaml with aliases

* minor changes

* minor changes

* minor changes

* minor changes

* minor changes

* minor changes

* minor changes

* minor changes

* minor changes

* minor changes

* Changed yaml aliases

* Adapted naming conventions

* Removed error code at the end

* Adapted configuration

* Minor changes

* Minor changes

* Minor changes

* Removed spaces

* Removed docker-related config from groovy file

* Minor changes

* Minor changes

* Removed container config

* Corrected testing function name

* Deleted unnecessary parts

* Changed service deletion message

* Changed service deletion message

* Logging out before throwing error service deletion step fails

* Minor changes

* Minor changes

* Minor changes

* Delete .DS_Store

* Delete .DS_Store

* Delete .DS_Store

* Delete .DS_Store

* Minor changes

* Minor changes

* Minor changes

* Added newline at end of file

* Added newline at end of file

* Changes for Pull request optimization

* added documentaion

* Adapted documentation

* Adapted documentation

* Adapted documentation

* Adapted documentation

* Adapted documentation

* Added CFDeleteServiceKeys

* Added ServiceKey deletion tests

* added cfServiceKeys flag explanation to documentation

* removed trailing spaces from documentation

* resolving conflicts

* Changed deletion message an variable naming

* Changed tests

* Changed tests

* Changed tests

* Changed tests

* Changed CloudFoundryDeleteServiceOptions to options

* Changed CloudFoundryDeleteServiceOptions to options

* Minor changes

* Minor changes

* Changed variable naming

* Changed error handling

* Changed error handling and logging

* Changed documentation

* Simplified code

* Fixed CodeClimate issues

* Changed from returning err to nil where no errur returned needed

* Add cloudFoundryCreateServiceKey Go Step

* Changed Groovy File

* Changed aliases

* Removed unneccessary parts

* Minor changes

* Minor changes

* Adapted documentation

* Adapted tests

* Adapted Groovy File

* Changed Groovy file

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Removed Groovy Tests for cfCreateServiceKey

* Minor changes

* Added ATC Check YAML

* Added ATC Check generated files

* Added test class

* Added abapEnvironmentRunATCCheck

* Minor changes

* Minor changes

* Changed groovy

* Minor changes

* Changed groovy

* Changed groovy

* Minor changes

* Adapted Groovy imports

* Adapted Groovy imports

* Adapted Groovy imports

* Adapted Groovy

* Getting ATC results

* Changed error message

* changed groovy

* removed trailing spaces

* Added login check

* Minor changes

* Added step to whitelistScriptReference

* Added ATC error message handling

* Added groovy file

* Added step to groovy tests

* corrected metadata file

* Debugging

* Debugging

* Added yaml config parameter for ATC run

* Adapted file location of ATC run config to jenkins specific location

* Implementing universal pipeline logic for finding yaml config regardless of pipeline

* Changed error handling for reading config yaml file

* Changed atcrunconfig alias

* minor changes

* Minor changes

* Minor changes

* Changed back to dynamic file reading

* Minor changes

* filepath changes

* Removing CF Login

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Removed whitespaces

* Added CF functions unit tests

* Added invalid parameter handling

* Removed package and SC flag

* Minor changes

* Changed tests

* Changed tests

* Changed tests

* Minor changes

* Changed tests

* removed unnecessary logout

* Added documentation

* Changed docu

* Changed docu

* Changed docu

* Changed docu

* Changed docu

* Changed docu

* Changed docu

* Changed docu

* Changed docu

* Changed docu

* Removed trailing spaces

* Added newline at end of file

* code climate fixes

* code climate fixes

* code climate fixes

* Minor changes

* Minor changes

* Minor changes

* Changed tests

* Test changes

* Splitted Cloud Foundry functions into two classes

* Removed two steps from whtielistScriptReference

* removed atcrunConfig alias

* issue fixes

* Changed docu

* Changed docu

* Changed docu

* Removed trailing spaced from docu

* Changed docu

* Go generator run

* Issue fixes

* Remove unnecessary imports

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Update whitelistScript

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Adding piperutils for writing xml file

* Persisting ATC Results with piperutils

* Set failonMissingReports to true

* Refactoring for CodeClimate

* Changed result file name

* Changed credentials aliases

* changing secret name

* Removing trailing spaces

* Added secret name and alias to docu

* Changing to workingDir

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-06-15 21:31:50 +02:00
Christopher Fenner
6c25c28801
chore(pipeline): add action to lint markdown files (#1593)
* add markdownlint action

* add markdownlint config

* add dummy change to MD file

* rename job

* run on master branch

* remove ignore files

* add dummy change

* revert dummy change

* Update markdown.yml

* Update .markdownlint.yml

* Update README.md

* Update .markdownlint.yml

* Update markdown.yml
2020-06-15 21:10:12 +02:00
lndrschlz
48f6a08433
GithubPublishRelease: Add "pre-release" flag (#1654)
* githubPublishRelease: added prerelease tag

* test case added

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-06-15 21:05:08 +02:00
Alexander Link
a864cf0f60
Fix docker.includes in WhitesourceConfigurationHelper (#1666)
* Fix docker.includes in WhitesourceConfigurationHelper

Not sure, but it seems the [`docker.includes` parameter](https://whitesource.atlassian.net/wiki/spaces/WD/pages/804814917/Unified+Agent+Configuration+File+and+Parameters#UnifiedAgentConfigurationFileandParameters-DockerImages) needs to be a regex.

Our pipeline is failing with:
```
10:22:33  [ERROR] [2020-06-15 08:22:33,740 +0000] - Resolve DockerEntity Exception Dangling meta character '*' near index 0
10:22:33  *.tar
10:22:33  ^
10:22:33  [DEBUG] [2020-06-15 08:22:33,743 +0000] - Resolve DockerEntity Exception {}
10:22:33  java.util.regex.PatternSyntaxException: Dangling meta character '*' near index 0
10:22:33  *.tar
10:22:33  ^
10:22:33  	at java.base/java.util.regex.Pattern.error(Unknown Source)
10:22:33  	at java.base/java.util.regex.Pattern.sequence(Unknown Source)
10:22:33  	at java.base/java.util.regex.Pattern.expr(Unknown Source)
10:22:33  	at java.base/java.util.regex.Pattern.compile(Unknown Source)
10:22:33  	at java.base/java.util.regex.Pattern.<init>(Unknown Source)
10:22:33  	at java.base/java.util.regex.Pattern.compile(Unknown Source)
10:22:33  	at org.whitesource.utils.WssStringUtils.isMatchingPattern(WssStringUtils.java:49)
10:22:33  	at org.whitesource.agent.dependency.resolver.docker.DockerResolver.filterTarImagesToScan(DockerResolver.java:296)
10:22:33  	at org.whitesource.agent.dependency.resolver.docker.DockerResolver.resolveDockerEntities(DockerResolver.java:186)
10:22:33  	at org.whitesource.fs.scanOrigins.DockerEntityScanOrigin.scan(DockerEntityScanOrigin.java:66)
10:22:33  	at org.whitesource.fs.scanOrigins.ScanOrigin.runOriginScan(ScanOrigin.java:36)
10:22:33  	at org.whitesource.fs.FileSystemAgent.createProjects(FileSystemAgent.java:132)
10:22:33  	at org.whitesource.fs.Main.scanAndSend(Main.java:157)
10:22:33  	at org.whitesource.fs.Main.main(Main.java:102)
10:22:33  [WARN] [2020-06-15 08:22:33,744 +0000] - Resolve DockerEntity Exception Dangling meta character '*' near index 0
10:22:33  *.tar
10:22:33  ^
```

* Switch docker.includes to slashy string

* Fix docker includes pattern in tests

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: D070410 <srinikitha.kondreddy@sap.com>
2020-06-15 17:02:28 +02:00
Oliver Nocon
f125ba5b60
adding groovy wrapper for containerSaveImage (#1663) 2020-06-15 16:44:49 +02:00
Oliver Nocon
6116fe5528
Decouple config reading and retrieval (#1668)
There are use cases where we need to run `GetStepConfig()` multiple times.
In such cases it is more efficient to load the files once and then resolve the
respective step configuration.
2020-06-15 14:17:59 +02:00
Florian Wilhelm
0fcf1654fe
Install maven artifacts after mta build (#1641)
Extend mta build step to install maven artefacts after build to allow re-using them in later stages (additional unit or integration tests which might not be running as part of the "build" life-cycle).
2020-06-15 12:46:54 +02:00
Stephan Aßmus
f855658e06
Enhance piperutils.Files and mock.FilesMock (#1664)
* Flesh out piperutils.Files and mock.FilesMock functionality
* Avoid a lot of code-duplication via embedding
2020-06-15 09:47:33 +02:00
Gabriel Borges
4e73a699c2
doc(kubernetes): Make plugin link clickable (#1657)
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-06-15 07:03:53 +02:00
Sven Merk
89f5ca9da1
whitesourceExecuteScan: Fix scanType: docker related issues (#1644)
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-06-12 16:55:16 +02:00
Oliver Nocon
50b2a2d6fc
Update deprecation message (#1656) 2020-06-12 15:19:13 +02:00
sudeepsukumar
702664645c
feat(newman): fetch xsuaa credentials in newman execution (#1325)
* fetch xsuaa credentials in newman execution

* refactor CfUtils and moving to integration package

* removing CfUtils from old package com.sap.piper

* avoid config.verbose as "null"

* variable renamed for consistent naming

* handling NPE and other comments incorp

* added test cases

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-06-12 10:04:03 +02:00
Daniel Kurzynski
41c1653a06
Fix checkmarx (#1655) 2020-06-12 09:22:22 +02:00
Daniel Kurzynski
d1cdf4449c
Fix reading download cache variables on node (#1660) 2020-06-11 21:59:55 +02:00
Stephan Aßmus
d558db9106
Refactor maven.Evaluate() to take options (#1659)
Co-authored-by: Florian Wilhelm <florian.wilhelm02@sap.com>
2020-06-11 14:02:54 +02:00
Stephan Aßmus
c693c4c7dc
Increase default transport timeout to 3 mins (#1648) 2020-06-10 11:14:55 +02:00
Daniel Kurzynski
d9a8537eb5
Revert to again use curl to download binary (#1637)
Revert "Remove curl for unstashing binary (#1580)"
This reverts commit 8b296f74

Improve error message

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-06-10 10:11:20 +02:00
Christopher Fenner
8e78199cce
chore: emit used project config & defaults to log (#1633)
* print used project config

* print used project defaults
2020-06-10 09:39:29 +02:00
Daniel Kurzynski
1ee13b358b
Store build tool in commonPipelineEnvironment (#1645) 2020-06-09 14:52:03 +02:00
Florian Wilhelm
4295f7a24e
Restore alphabetical order of docs (#1643) 2020-06-09 09:05:28 +02:00
Florian Wilhelm
edef158e61
Document new npmExecuteScripts and mavenBuild steps (#1632) 2020-06-09 08:46:54 +02:00
Stephan Aßmus
56699ab019
Run artifactPrepareVersion in maven container if necessary (#1634)
* The metadata for artifactPrepareVersion-go specifies a container for when the buildTool is maven.
* The alias to 'mavenExecute' was removed. The problem with this is that when a section containers is included in the metadata, dockerImage will always be picked up from mavenExecute, the conditional dependency on buildTool will not even be considered. Parameters such as m2Path, projectSettingsFile and globalSettingsFile should be configured in general/maven if necessary.
* When the step ends up being executed within dockerExecuteOnKubernetes, we need to preserve the .git folder. This folder would normally be excluded by the default excludes of the stash step. There was already a comment that suppressing this behavior by passing useDefaultExcludes: false was problematic (unfortunately without going into details), so I've added a new parameter to dockerExecute and dockerExecuteOnKubernetes named stashNoDefaultExcludes (note the reverted meaning to ease preserving the default behavior when this parameter is not provided). This parameter is passed to piperExecuteBin from the artifactPreferVersion groovy wrapper.
2020-06-08 17:08:05 +02:00
Florian Wilhelm
c490ce9211
Use go-based npmExecuteScripts, mavenBuild in buildExecute (#1622) 2020-06-04 17:53:06 +02:00
Florian Wilhelm
285537f6a4
Automated go dependency update (#1630)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2020-06-04 13:52:19 +02:00
Marcus Holl
f071b4490d
Fix: yaml util handling for boolean with value false (#1629)
* Provide test for yamlUtils substitution with boolean value false

* Fix: handle booleans with value 'false' in yaml substitution

* Improve variable naming
2020-06-04 06:03:12 +02:00
Marcus Holl
0ad38b8621
Timeout for malwarescan (#1623)
Provide a timeout parameters to malwarescan step. This is forwarded to the piper http layer. The default used there is 10 seconds with is not useable for that use case for larger files.
2020-06-03 11:08:34 +02:00
Stephan Aßmus
f90a4f9eae
Provide an ExecRunner implementation for running commands in docker (#1606)
* ExecRunner implementation for executing commands within docker
* Add whole-file example as documentation
2020-06-02 14:24:06 +02:00
Daniel Kurzynski
cf9a41850e
Needed CLI separator for Fortify tools depends on platform (#1616)
* Update fortify.yaml
* src, exclude and pythonAdditionalPaths are now lists of strings
* Re-implement pythonIncludes and pythonExcludes as aliases of src and exclude
* Fix using the correct separator (; on windows, : on everything else)
* Tokenize also python "includes"
* mvnCustomArgs was removed

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-06-02 13:47:07 +02:00
Florian Wilhelm
7fc9002ce1
Build go binary without debug symbols (#1618)
This produces a much smaller binary for faster downloads.

See https://golang.org/cmd/link/ for docs on -s and -w flag.
2020-06-02 12:32:29 +02:00
Srinikitha Kondreddy
c2de96ab43
Fix credential parameter in the docu (#1597)
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-06-02 10:04:56 +02:00
Stephan Aßmus
a24a7aad23
Fortify: Using mvn to auto-resolve classpath needs additional params (#1607)
* also reduce code duplication in token fetching
* concatenate classpaths from multi-maven projects

Co-authored-by: Daniel Kurzynski <daniel.kurzynski@sap.com>
2020-05-29 15:42:35 +02:00
Daniel Kurzynski
ae6853ee4d
Use fixed versions for docker images for mtaBuild (#1615) 2020-05-29 13:51:33 +02:00
Stephan Aßmus
c5f3b85037
Strip "./" from artifacts (#1611)
Co-authored-by: Daniel Kurzynski <daniel.kurzynski@sap.com>
2020-05-29 10:59:55 +02:00
Marcus Holl
c3c486e051
Reload default value cache also in case we have only customDefaultsFromFile present (#1612)
Co-authored-by: Kevin Hudemann <kevin.hudemann@sap.com>
2020-05-28 21:07:30 +02:00
Stephan Aßmus
6a96629d65
Provide a timeout of 20m for the go Integration Tests (#1613)
* Provide a timeout of 20m for the go IT
Default is 10m, see https://golang.org/cmd/go/#hdr-Testing_flags
Co-authored-by: Daniel Kurzynski <daniel.kurzynski@sap.com>
2020-05-28 18:28:38 +02:00
Daniel Mieg
6db5beb2ec
Increase timeout (#1609)
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-05-28 13:08:26 +02:00