1
0
mirror of https://github.com/SAP/jenkins-library.git synced 2024-12-14 11:03:09 +02:00
Commit Graph

947 Commits

Author SHA1 Message Date
Oliver Nocon
d602c6e2b9
extend security stage (#1687)
* extend security stage

Add new steps
* checkmarxExecuteScan
* fortifyExecuteScan

closes #1650

* update tests
2020-06-19 12:35:02 +02:00
Chris Bo
e9ad8c5e9c
Introducing new step 'gctsCloneRepository' (#1468)
* added new step gctsCloneRepository

* enhanced docu

* adjusted test case name

* enhanced docu

* refined error messages

* regenerated & remove unused Groovy imports

* small change

* remove execRunner

* Update documentation/docs/steps/gctsCloneRepository.md

* regenerated

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-06-18 07:45:22 +02:00
Daniel Kurzynski
498e527c6c
Add npmExecuteScripts to integration stage (#1674)
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-06-17 18:17:05 +02:00
Oliver Nocon
8e74952166
Switch Detect scan to golang implementation (#1673)
* Switch Detect scan to golang implementation

* update workingDir

* use universal container as default

* Switch to official curl image

* fix code climate issue

* switch default image

* update docker options

* update CommonStepsTest

* remove invalid groovy test

* clean up defaults

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-06-16 17:21:59 +02:00
Alexander Link
a864cf0f60
Fix docker.includes in WhitesourceConfigurationHelper (#1666)
* Fix docker.includes in WhitesourceConfigurationHelper

Not sure, but it seems the [`docker.includes` parameter](https://whitesource.atlassian.net/wiki/spaces/WD/pages/804814917/Unified+Agent+Configuration+File+and+Parameters#UnifiedAgentConfigurationFileandParameters-DockerImages) needs to be a regex.

Our pipeline is failing with:
```
10:22:33  [ERROR] [2020-06-15 08:22:33,740 +0000] - Resolve DockerEntity Exception Dangling meta character '*' near index 0
10:22:33  *.tar
10:22:33  ^
10:22:33  [DEBUG] [2020-06-15 08:22:33,743 +0000] - Resolve DockerEntity Exception {}
10:22:33  java.util.regex.PatternSyntaxException: Dangling meta character '*' near index 0
10:22:33  *.tar
10:22:33  ^
10:22:33  	at java.base/java.util.regex.Pattern.error(Unknown Source)
10:22:33  	at java.base/java.util.regex.Pattern.sequence(Unknown Source)
10:22:33  	at java.base/java.util.regex.Pattern.expr(Unknown Source)
10:22:33  	at java.base/java.util.regex.Pattern.compile(Unknown Source)
10:22:33  	at java.base/java.util.regex.Pattern.<init>(Unknown Source)
10:22:33  	at java.base/java.util.regex.Pattern.compile(Unknown Source)
10:22:33  	at org.whitesource.utils.WssStringUtils.isMatchingPattern(WssStringUtils.java:49)
10:22:33  	at org.whitesource.agent.dependency.resolver.docker.DockerResolver.filterTarImagesToScan(DockerResolver.java:296)
10:22:33  	at org.whitesource.agent.dependency.resolver.docker.DockerResolver.resolveDockerEntities(DockerResolver.java:186)
10:22:33  	at org.whitesource.fs.scanOrigins.DockerEntityScanOrigin.scan(DockerEntityScanOrigin.java:66)
10:22:33  	at org.whitesource.fs.scanOrigins.ScanOrigin.runOriginScan(ScanOrigin.java:36)
10:22:33  	at org.whitesource.fs.FileSystemAgent.createProjects(FileSystemAgent.java:132)
10:22:33  	at org.whitesource.fs.Main.scanAndSend(Main.java:157)
10:22:33  	at org.whitesource.fs.Main.main(Main.java:102)
10:22:33  [WARN] [2020-06-15 08:22:33,744 +0000] - Resolve DockerEntity Exception Dangling meta character '*' near index 0
10:22:33  *.tar
10:22:33  ^
```

* Switch docker.includes to slashy string

* Fix docker includes pattern in tests

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: D070410 <srinikitha.kondreddy@sap.com>
2020-06-15 17:02:28 +02:00
Oliver Nocon
f125ba5b60
adding groovy wrapper for containerSaveImage (#1663) 2020-06-15 16:44:49 +02:00
sudeepsukumar
702664645c
feat(newman): fetch xsuaa credentials in newman execution (#1325)
* fetch xsuaa credentials in newman execution

* refactor CfUtils and moving to integration package

* removing CfUtils from old package com.sap.piper

* avoid config.verbose as "null"

* variable renamed for consistent naming

* handling NPE and other comments incorp

* added test cases

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-06-12 10:04:03 +02:00
Daniel Kurzynski
d1cdf4449c
Fix reading download cache variables on node (#1660) 2020-06-11 21:59:55 +02:00
Daniel Kurzynski
d9a8537eb5
Revert to again use curl to download binary (#1637)
Revert "Remove curl for unstashing binary (#1580)"
This reverts commit 8b296f74

Improve error message

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-06-10 10:11:20 +02:00
Florian Wilhelm
c490ce9211
Use go-based npmExecuteScripts, mavenBuild in buildExecute (#1622) 2020-06-04 17:53:06 +02:00
Marcus Holl
f071b4490d
Fix: yaml util handling for boolean with value false (#1629)
* Provide test for yamlUtils substitution with boolean value false

* Fix: handle booleans with value 'false' in yaml substitution

* Improve variable naming
2020-06-04 06:03:12 +02:00
Maximilian Lenkeit
10e5bf98f8
artifactPrepareVersion: remember original artifact version (#1600)
* Update resources.go

* expose originalArtifactVersion parameter

* re-generate steps

* adjust golden files for generator changes
2020-05-28 10:05:22 +02:00
Daniel Kurzynski
3992f17b32
Fix injection of download cache parameters (#1605)
- NPMRegistry was always an empty string
- MTA did not use npm registry
2020-05-27 18:20:10 +02:00
Oliver Nocon
5e27d26a7c
whitesourceExecuteScan: add Docker image scanning (#1556)
* whitesourceExecuteScan: add Docker image scanning

* add test

* Update whitesourceExecuteScan.groovy

* Update WhitesourceConfigurationHelper.groovy
2020-05-27 16:27:36 +02:00
Daniel Kurzynski
8b296f7420
Remove curl for unstashing binary (#1580) 2020-05-27 13:30:04 +02:00
Marcus Holl
1836bb728b
Emit piper version to log (#1583)
* introduce variable for piper bin

* log the version of the piper bin

* don't fail with generic message from script execution (sh)
2020-05-26 09:38:47 +02:00
Oliver Nocon
d5d510719d
piperExecuteBin - prevent non-speaking exit code 1 error (#1584)
* piperExecuteBin - prevent non-speaking exit code 1 error

Make sure that error during context resolution gets more context.

* properly provide golang errors

* improve error message, when no details are available

* Add error category

* Update getConfig.go
2020-05-26 07:58:03 +02:00
lndrschlz
80f70aba4a
Error reporting: change category to standard term (#1587)
* streamlined error category with standard term

* remove unused import from test

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-05-25 22:38:26 +02:00
Sven Merk
af2a01c064
Fortify implementation in golang (#1428) 2020-05-25 19:48:59 +02:00
Marcus Holl
f47f2e6e90
Use mtaBuild step from go (#1536)
Co-authored-by: Your Name <kevin.hudemann@sap.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-05-19 08:51:18 +02:00
Chris Bo
abdc3f736b
Introducing new step 'gctsDeployCommit' (#1476)
* added new step gctsDeployCommit

* suggested PR fixes applied

* fixed test

* Remove unused imports

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* added URL encoding for 'request' parameter

* regenerate after change

* minor changes

* renamed step to 'gctsDeploy'

* changed name

* remove space

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-05-18 21:39:35 +02:00
lndrschlz
b9a357bab5
jenkinsMaterialzeLog: use log without binary annotations (#1542)
* jenkinsMaterialzeLog: use log without binary annotations

* test fixed
2020-05-18 21:19:30 +02:00
Florian Hoffmann
18f30c8068
Improve approval logging (#1513) 2020-05-18 12:21:05 +02:00
Stephan Aßmus
7d3f201609
Ignore invalid entries in custom defaults (#1558)
Extend tests, ignore invalid custom defaults entries
2020-05-18 10:59:02 +02:00
Christopher Fenner
0ce6473a98
fix(spinnaker): fix MissingPropertyException (#1543)
* add test case

* fix issue
2020-05-14 17:45:02 +02:00
Christopher Fenner
27c6320474
feat(sonar): write flag for sonar execution to influx (#1534)
* write flag for sonar execution to influx

* add missing step metadata file

* first attempt to read influx from disk

* add missing import

* correct glob pattern

* use file path

* correct type names

* cleanup

* fix code climate issue

* fix typo

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>

* add test case

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-05-14 13:46:40 +02:00
Stephan Aßmus
5ab4dedcb2
Optionally ignore custom defaults (#1538)
* Add parameter "--ignoreCustomDefaults"
* Pass to piper customDefaults from config also via --defaultConfig
... and add "--ignoreCustomDefaults".
* Log output when ignoring customDefaults

Co-authored-by: Daniel Kurzynski <daniel.kurzynski@sap.com>
2020-05-14 10:50:58 +02:00
philippthiele
4788b80f47
add -f option to kanikoExecute containerPreparationCommand remove (#1539)
* add -f option do kaniExecute containerPreparationCommand remove

* Update KanikoExecuteTest.groovy

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-05-13 17:41:32 +02:00
dominiklendle
ac732b3065
Add abapEnvironmentRunATCCheck step (#1454)
* Minor changes

* Changed groovy file

* Changed generated file

* Changed yaml with container config

* Changed groovy config

* minor changes

* minor changes

* Changed yaml with aliases

* minor changes

* minor changes

* minor changes

* minor changes

* minor changes

* minor changes

* minor changes

* minor changes

* minor changes

* minor changes

* Changed yaml aliases

* Adapted naming conventions

* Removed error code at the end

* Adapted configuration

* Minor changes

* Minor changes

* Minor changes

* Removed spaces

* Removed docker-related config from groovy file

* Minor changes

* Minor changes

* Removed container config

* Corrected testing function name

* Deleted unnecessary parts

* Changed service deletion message

* Changed service deletion message

* Logging out before throwing error service deletion step fails

* Minor changes

* Minor changes

* Minor changes

* Delete .DS_Store

* Delete .DS_Store

* Delete .DS_Store

* Delete .DS_Store

* Minor changes

* Minor changes

* Minor changes

* Added newline at end of file

* Added newline at end of file

* Changes for Pull request optimization

* added documentaion

* Adapted documentation

* Adapted documentation

* Adapted documentation

* Adapted documentation

* Adapted documentation

* Added CFDeleteServiceKeys

* Added ServiceKey deletion tests

* added cfServiceKeys flag explanation to documentation

* removed trailing spaces from documentation

* resolving conflicts

* Changed deletion message an variable naming

* Changed tests

* Changed tests

* Changed tests

* Changed tests

* Changed CloudFoundryDeleteServiceOptions to options

* Changed CloudFoundryDeleteServiceOptions to options

* Minor changes

* Minor changes

* Changed variable naming

* Changed error handling

* Changed error handling and logging

* Changed documentation

* Simplified code

* Fixed CodeClimate issues

* Changed from returning err to nil where no errur returned needed

* Add cloudFoundryCreateServiceKey Go Step

* Changed Groovy File

* Changed aliases

* Removed unneccessary parts

* Minor changes

* Minor changes

* Adapted documentation

* Adapted tests

* Adapted Groovy File

* Changed Groovy file

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Removed Groovy Tests for cfCreateServiceKey

* Minor changes

* Added ATC Check YAML

* Added ATC Check generated files

* Added test class

* Added abapEnvironmentRunATCCheck

* Minor changes

* Minor changes

* Changed groovy

* Minor changes

* Changed groovy

* Changed groovy

* Minor changes

* Adapted Groovy imports

* Adapted Groovy imports

* Adapted Groovy imports

* Adapted Groovy

* Getting ATC results

* Changed error message

* changed groovy

* removed trailing spaces

* Added login check

* Minor changes

* Added step to whitelistScriptReference

* Added ATC error message handling

* Added groovy file

* Added step to groovy tests

* corrected metadata file

* Debugging

* Debugging

* Added yaml config parameter for ATC run

* Adapted file location of ATC run config to jenkins specific location

* Implementing universal pipeline logic for finding yaml config regardless of pipeline

* Changed error handling for reading config yaml file

* Changed atcrunconfig alias

* minor changes

* Minor changes

* Minor changes

* Changed back to dynamic file reading

* Minor changes

* filepath changes

* Removing CF Login

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Removed whitespaces

* Added CF functions unit tests

* Added invalid parameter handling

* Removed package and SC flag

* Minor changes

* Changed tests

* Changed tests

* Changed tests

* Minor changes

* Changed tests

* removed unnecessary logout

* Added documentation

* Changed docu

* Changed docu

* Changed docu

* Changed docu

* Changed docu

* Changed docu

* Changed docu

* Changed docu

* Changed docu

* Changed docu

* Removed trailing spaces

* Added newline at end of file

* code climate fixes

* code climate fixes

* code climate fixes

* Minor changes

* Minor changes

* Minor changes

* Changed tests

* Test changes

* Splitted Cloud Foundry functions into two classes

* Removed two steps from whtielistScriptReference

* removed atcrunConfig alias

* issue fixes

* Changed docu

* Changed docu

* Changed docu

* Removed trailing spaced from docu

* Changed docu

* Go generator run

* Issue fixes

* Remove unnecessary imports

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Update whitelistScript

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Adding piperutils for writing xml file

* Persisting ATC Results with piperutils

* Set failonMissingReports to true

* Refactoring for CodeClimate

* Changed result file name

* Changed credentials aliases

* changing secret name

* Removing trailing spaces

* Added secret name and alias to docu

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-05-13 14:51:48 +02:00
Kevin Hudemann
d7985dd1b5
Enable use of customDefaults from Project Config also in Groovy (#1521)
This change enables the setupCommonPipelineEnvironment step to handle
custom default configurations defined in customDefaults parameter of the
project configuration.

Previously, only the getConfig Go step was able to incorporate custom
default configurations.

Update documentation on custom defaults and sharing between projects.

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-05-12 13:50:18 +02:00
dominiklendle
fd858dafcd
tests: remove test for step using piperExecuteBin() (#1506)
* Change Groovy implementation to piperExecuteBin()

This commit changes the old groovy implementation to the new way using the piperExecuteBin() function.

* Added abapEnvironemtPullGitRepo to whitelist for Tests

* Changed groovy implementation to using piperExecuteBin()

* Added cfDeleteService to test whitelist

* Removed Groovy unit tests

* Remove whitelist tests

* Update test/groovy/CommonStepsTest.groovy

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-05-07 14:06:31 +02:00
Florian Wilhelm
eaf5479e9c
Fix maven parameter handling (#1493)
Avoid maven error `Unknown lifecycle phase \"-\"` when the value of a define contains `-`.

Don't split and trim maven arguments. Expect they come in as a list, keep them as list.

This is a breaking change compared to the old Groovy implementation which relied on using a shell for calling maven.

As an example, consider this diff:

```diff
-        goals: 'org.apache.maven.plugins:maven-help-plugin:3.1.0:evaluate',
-        defines: "-Dexpression=$pomPathExpression -DforceStdout -q",
+        goals: ['org.apache.maven.plugins:maven-help-plugin:3.1.0:evaluate'],
+        defines: ["-Dexpression=$pomPathExpression", "-DforceStdout", "-q"],
```
2020-05-06 17:43:32 +02:00
Christopher Fenner
669dc91688
fix: prevent script to json conversion (#1495)
* prevent script to json conversion

* use piperExecuteBin

* correct imports

* fix trailing spaces
2020-05-05 15:21:59 +02:00
Christopher Fenner
d75dced2a6
test: activate common test cases using piperExecuteBin (#1501) 2020-05-05 14:31:12 +02:00
Marcus Holl
30850aec2d
Add groovy wrapper for malwarescanning (#1403) 2020-04-30 11:14:14 +02:00
Oliver Nocon
3f1a9d004a
Allow proper usage of resilience feature in golang steps (#1487)
* Allow proper usage of resilience feature in golang steps
2020-04-30 10:39:27 +02:00
Oliver Nocon
43947e6ef5
Pass golang error details to Jenkins pipeline (#1443)
Do not exit with os.Exit(1) but using log.Entry().Fatal() instead
* Golang: forward error details
* extend groovy wrapper to provide proper error message
* create closure for error handling
2020-04-28 07:42:02 +02:00
Oliver Nocon
b9450702af
piperExecuteBin: fix reading environment (#1452)
* piperExecuteBin: fix reading environment
* update tests

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-04-24 22:41:34 +02:00
Florian Wilhelm
3f5b9cc555
Implement npmExecuteScripts step (#1422) 2020-04-24 18:29:30 +02:00
Chris Bo
fb4cfd84ec
Introducing new step: "gctsCreateRepository" (#1424)
With the step gctsCreateRepository it is possible to create a local gCTS repository on an ABAP server

Co-authored-by: Marcus Holl <marcus.holl@sap.com>
2020-04-24 15:31:41 +02:00
Daniel Kurzynski
974327d16e
Use mavenExecute from go (#1388)
Also establish mavenExecute() via new JenkinsMavenExecuteRule in Groovy Unit Tests.

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-04-24 10:41:49 +02:00
artembannikov
8206fb9716
TMS: Print response content for status codes 3xx in addition to 4xx and 5xx (#1441)
* Extend JenkinsLoggingRule and TransportManagementServiceTest

* Print response content for status codes >= 300, also test that it's not
revealed in the case of unexpected status codes between 200 and 300
(both not included) for authentication

Co-authored-by: Marcus Holl <marcus.holl@sap.com>
2020-04-23 09:46:25 +02:00
Oliver Nocon
067eb0ba6a
Improve error messages for test failures (#1417) 2020-04-17 11:59:51 +02:00
Daniel Kurzynski
1c247de9d0
Add option to always run stages in a pod (#1414) 2020-04-17 10:31:04 +02:00
Oliver Nocon
4061881b53
UIVeri5 & Newman: provide better error messages (#1410) 2020-04-16 09:14:39 +02:00
Oliver Nocon
bd0eed30ff
stageWrapper: new error handling (#1401)
* handlePipelineStepErrors: allow disabling for stages
* update error handling in stages
* move file loading into try catch
* Cleanup DebugReport before test

Apparantly, the same DebugReport instance is loaded
during all Unit Tests. Avoid left-overs from previous
tests.

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-04-15 14:02:41 +02:00
Daniel Kurzynski
b32214ee78
Inherit from Kubernetes pod template (#1361) 2020-04-08 19:54:06 +02:00
Christopher Fenner
c190deee44
refactor(sonar): switch to Sonar scan in GO (#1320)
* use sonar go in groovy

* use SONAR_TOKEN

* only use owner/repo if both are set

* trim version to major version digit

* fix code climate issues

* remove Sonar defaults

* use sonar go step

* use SONAR_TOKEN

* use certs from parameters

* use docker workspace & options

* add instance parameter

* implement branchName

* implement branchName

* remove duplicate default

* update docs

* fix TODOs

* remove merge mess

* fix code climate issue

* address comments

* respect custom defaults and custom config file name

* fix typo

* remove obsolete test cases

* disable CommonStepChecks

* check step config instead of context config

* remove TODOs

* respect jenkins PR envvars

* use value from stepConfig

* Update vars/sonarExecuteScan.groovy

* rename options to config

* correct type for options

* add test cases

* log sonar.options in debug message

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-04-08 12:55:46 +02:00
artembannikov
4553c8f278
TMS: explicitly check in tests that there is no token in the logs (#1370)
* Check in the tests explicitly that token is not present in logs

* Address feedback

Co-authored-by: Marcus Holl <marcus.holl@sap.com>
2020-04-07 15:42:38 +02:00
artembannikov
20a8d3b656
Exchange dummy(.)com in tests with a more secure URL (#1371)
Co-authored-by: Marcus Holl <marcus.holl@sap.com>
2020-04-07 14:52:27 +02:00