1
0
mirror of https://github.com/SAP/jenkins-library.git synced 2024-12-14 11:03:09 +02:00
Commit Graph

608 Commits

Author SHA1 Message Date
Oliver Nocon
5096b3bfe1
fix(checkIfStepActive): respect explicit (de-)activation (#3055) 2021-08-17 11:20:04 +02:00
Mayur Belur Mohan
2d412d9f3c
IntegrationArtifactResource Command (#3027)
* IntegrationArtifactResource Command

* Remove unused code

* Formatting fix

* formatting fix

* formatting fix

* formatting fix

* formatting fix

* CodeReview Fixes

* Code Review Fix

* Code Review Fixes

* Code Review Fixes

* Format fix

* format fix

* format fix

* format fix

* Code Review Fix

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
2021-08-12 09:11:02 +02:00
Oliver Nocon
97b84429f1
feat(Vault): custom prefix for test credentials (#3043)
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2021-08-11 16:20:08 +02:00
Anil Keshav
c66c868d7c
remove schema parameter from cycloneDX for npm build (#3038)
Co-authored-by: Your Name <you@example.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-08-10 16:22:13 +02:00
Daniel Mieg
9c76b89fb8
Add telemetry for abapEnvironment steps (#3016)
* Remove explicit telemetry from signature

* Set log level

* ATC set error category
2021-08-04 17:31:16 +02:00
Oliver Nocon
f308fbfc0d
feat(getConfig) allow reading stage configuration (#3026)
* feat(getConfig) allow reading stage configuration

* update test

* use fix version of Selenium Image for Karma tests
2021-08-04 10:09:02 +02:00
Mayur Belur Mohan
c283b9319d
IntegrationArtifactUnDeploy Command (#3018)
* IntegrationArtifactUnDeploy Command

* formatting fix

* Code Review Fixex

* Code Review Fixes

* remove unused code

* Formating fixes

* formatting fixes

* formatting fix

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
2021-08-02 16:27:16 +02:00
Oliver Nocon
6f32b437be
fix(dockerOptions) allow proper usage of empty values (#3025)
This follows up on #3024
Setting emptyValue to s.th. like `--entrypoint=''` will break in case the argument is properly escaped.

Docker will return with
`container process caused: exec: "''": executable file not found in $PATH`
2021-08-02 14:57:37 +02:00
Giridhar Shenoy
4aa59dbdcb
Move BlackDuck API helper to OS version (#3008)
* feat(detectExecuteScan): generate ip result json

json will currently only be created in success cases.

No information about policy violation details available in the step yet.

* update report name

* Update cmd/detectExecuteScan.go

Co-authored-by: Giridhar Shenoy <giridhar.shenoy@sap.com>

* Update cmd/detectExecuteScan.go

Co-authored-by: Giridhar Shenoy <giridhar.shenoy@sap.com>

* Update cmd/detectExecuteScan_test.go

Co-authored-by: Giridhar Shenoy <giridhar.shenoy@sap.com>

* Update cmd/detectExecuteScan_test.go

Co-authored-by: Giridhar Shenoy <giridhar.shenoy@sap.com>

* move blackduck api package

* fix broken unit test

* added tests for new api calls

Co-authored-by: OliverNocon <oliver.nocon@sap.com>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-08-02 11:18:58 +02:00
Oliver Nocon
f455a8a4a0
fix(dockerOptions) properly pass empty values (#3024)
it is possible to overwrite the entrypoint for docker execution:
https://docs.docker.com/engine/reference/run/#entrypoint-default-command-to-execute-at-runtime

This is ideally done by passing `entrypoint=''` and not pass two options to the call.
This also helps with escaping issues of the empty value on other systems.

Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2021-08-02 10:21:14 +02:00
Martin Zuber
aa50cfb78d
Improve determining PR key in Azure DevOps provider. (#3007) 2021-08-02 10:08:33 +02:00
Mayur Belur Mohan
2f2fd84193
GetIntegrationArtifactMPLError function (#3000)
* GetIntegrationArtifactMPLError function

* formating fixes

* formating fixes

* formatting fixes

* formatting fixes

* Formatting fixes

* formatting fixes

* Code Review Fixes

* Code Review Fixes

* Code Review Fixes

* Code Review Fixes
2021-07-28 12:00:41 +02:00
Oliver Feldmann
7259ccc726
Remove version from deploy step (#2978) 2021-07-27 09:51:56 +02:00
Siarhei Pazdniakou
7a325e6fc8
Convert step/stage condition logic to golang (#2993)
* Added checkIfStepActive step

* Implemented npmScripts condition. Code was refactored

* Added some unit tests

* Fixed go modules

* Fixed go modules

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-07-26 07:47:43 +02:00
ffeldmann
39858cde2b
fix(splunkHook): Adds check if responseBody is nil (#3002)
* Improved logging of splunk connectivity errors

* Splunk logging

* Moved error logging message

* Bugfix for response body

* Moves response body check, logging of connectivity errors

* Reformatting

* Adds check if response body is nil
2021-07-23 16:02:27 +02:00
larsbrueckner
dbbbe1f0b3
Updates to toolrecord framework (#2986)
* Toolrecord framework -
provide a common entry point for post processing code scan results

Changes to be committed:
	new file:   pkg/toolrecord/REAMDE_toolrecord.md
	new file:   pkg/toolrecord/toolrecord_main.go
	new file:   pkg/toolrecord/toolrecord_test.go

* Add toolrecord file to Checkmarx results
modified:   cmd/checkmarxExecuteScan.go

* Add toolrecord file to Fortify results
	modified:   cmd/fortifyExecuteScan.go

* Add toolrecord file to Whitesource results
modified:   cmd/whitesourceExecuteScan.go

* unset umask (#2927)

* (feat) adds error logging output for downloading reports from whitesource (#2928)

* Add toolrecord file to Protecode results

* address code climate findings (1/2)

* address codeclimate findings (2/2)

* add comments to all methods

* Toolrecord library:
- move all toolrun files into a subdirectory
- fix timestamp generation in filenames

* add protecode group's URL to toolrecord data

* fix syntax error from previous commit in cmd/protecodeExecuteScan.go

* toolrecord: fix projectVersionID and generated URLs in fortifyExecuteScan.go

* cmd/fortifyExecuteScan.go: replace a hard-coded servername with
config.ServerURL

* update description

* add toolrecord file to detectExecuteScan

* toolrecord/whitesource: add project names as context

Co-authored-by: Kevin Stiehl <kevin.stiehl@numericas.de>
Co-authored-by: ffeldmann <felix@bnbit.de>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2021-07-23 08:48:48 +02:00
Anil Keshav
3c41788405
fix(mavenBuild) looses trust to existing java cacerts (#2977)
* using default java truststore

* default java keytore

* remove trust store

* working directory

* change dir java_home

* env variable

* trying to find jre home

* changing directory to jre home and java home

* trying java jre

* remving $

* trying to search the cacerts

* copying existing cacerts

* removing change directory

* searching for secrutiy folder only

* searching cacerts

* new path for cacert

* path to ca-cert

* new trust store

* changing cacert location

* only adding maven_opts env variable once

* log message

* ca cert path from user

* handelling interface modification

* enhance logs and code clean up

* code clean up

Co-authored-by: Your Name <you@example.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-07-22 11:06:46 +02:00
Mayur Belur Mohan
a763112fe6
integrationArtifactUpload Fixes (#2987)
* integrationArtifactUpload Fixes

* format fixes

* Doc fixes

* Fix unit tests

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
2021-07-16 16:09:55 +02:00
Sven Merk
ee6d46a77c
fix(protecodeExecuteScan): Fix typo in report name (#2982)
* Fix exclude and enhance docs

* Fix test

* Fix test

* Add reporting to checkmarx step

* Improve text

* Add protecode report

* Fix fmt

* Add error handling

* Fix report name
2021-07-16 09:08:28 +02:00
Christopher Fenner
f78777f784
feat(npm): allow to publish artifact to registry (#2871)
* add new paraeters

* update generated sources

* run npm publish

* add repositoryUrl parameter

* handle registry credentials

* rename parameter

* handle base64encoding

* remove vault reference

* make username secret

* add publish method

* use publish method

* use dedicated registry

* use dry run

* fix

* prepend path

* fix workdir

* move code to npm package

* do changes

* update dependencies

* correct property init

* remomve dry-run

* regenerate

* add mock

* add logging

* add debug log

* dry-run

* remove try run

* remove append

* add debug outut

* change

* add debug output

* changes

* cleanup

* use different auth property

* add credential utils

* add debug log outputs

* remove auth handling & reuse writeFile

* rename

* fix debug output

* remove comments

* update comment

* rename function

* update docs

* update generated files

* handle npm ignore

* remove commented code

* add debug output
2021-07-15 14:46:04 +02:00
ffeldmann
4922a75ac1
Improves error logging for Splunk hook (#2966)
* Improved logging of splunk connectivity errors

* Splunk logging

* Moved error logging message

* Bugfix for response body

* Moves response body check, logging of connectivity errors

* Reformatting
2021-07-14 08:48:48 +02:00
ffeldmann
e9d8175c9b
Adds retry mechanism for whitesource in case the download of the unified agent or JRE fails (#2961) 2021-07-13 20:36:36 +02:00
Sven Merk
3e7595920f
feat(protecodeExecuteScan): Add protecode report (#2981)
* Fix exclude and enhance docs

* Fix test

* Fix test

* Add reporting to checkmarx step

* Improve text

* Add protecode report

* Fix fmt

* Add error handling
2021-07-12 12:20:25 +02:00
Sven Merk
9571fd28f4
feat(checkmarxExecuteScan): Reporting for pipeline optimization (#2976)
* Fix exclude and enhance docs

* Fix test

* Fix test

* Add reporting to checkmarx step

* Improve text
2021-07-09 10:19:42 +02:00
Oliver Nocon
805a8fd88f
feat(config): read config/defaults with authentication (#2975)
* feat(config):read config/defaults with authentication

This change allows to use defaults and config files from a protected GitHub repository.

The options `--customConfig` and `--defaultConfig` already allowed to provide a link to an uprotected file.

Now, by passing a value in the form `<hostname>:<token>` to parameter `gitHubTokens` (this parameter can be passed multiple times) a token can be provided for dedicated hosts.

This makes it possible to use a link like
`https://api.github.com/repos/SAP/jenkins-library/contents/resources/my-defaults.yml?ref=master`
as reference to a default file or similarly as reference to a configuration file.

* update generation to allow protected config/defaults

* fix CodeClimate issues

* update missing generations
2021-07-08 15:26:07 +02:00
Roland Stengel
e2fa05587d
RFC Upload GO (#2903)
* RFC Upload GO
- yaml review
- add cpe
- harmonize with solman
2021-07-01 13:11:21 +02:00
Siarhei Pazdniakou
54f2a0d471
Added go-based influxWriteData step (#2890)
* Added go-based influxWriteData step

* Wrote tests & fixed issues

* Fixed issues

* Created go-based step tests. Fixed issues

* Fixed issues

* Integration test was added

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2021-06-30 11:18:49 +02:00
Thorsten Duda
7910df0e8c
new step integrationArtifactTriggerIntegrationTest (#2951)
* new step integrationArtifactTriggerIntegrationTest

* add new step into allow list

* add the new step to main command

* refer cpe

* remove unused unit tests

* Check methods and URLs of http request

* Add TriggerIntegration to mockingutils

* Format code

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
Co-authored-by: Linda Siebert <linda.siebert@sap.com>
2021-06-29 14:50:19 +02:00
Linda Siebert
78a29d782b
CPI - Introduce service key (#2901)
* Switch to service key for CPI GetMplStatus

Introduces read method for service key files, mock utils and tests.

* Use secret text instead of file

* Change serviceKey definition

* Update cpiUpload to use Service Key

retrieved the host and uaa information from service key

* Update cpiDeploy to use service key

retrieved the host and uaa information from service key

* Update cpiServiceEndpoint to use Service Key

retrieved the host and uaa information from service key

* Update cpiDownload to use Service Key

retrieved the host and uaa information from service key

* Update cpiUpdateConfig to use Service Key

retrieved the host and uaa information from service key

* Refactor serviceKey var name

* Fixed references to service key to follow the real format

they should be accessed through oauth instead of uaa because of the format of the json

* Rename ServiceKey to APIServiceKey

To support having a different service key(and for readability), we need to change the name to API.

* Add STAGES and STEPS yaml

add in to each yaml file of cpi integration

* Revert "Add STAGES and STEPS yaml"

This reverts commit aa2665d158.

* Change comments/formatting commonUtils

Make comments more understandable and follow code climate suggestions

* Change documentation files for steps

remove OAuth and host and change credentials to be servicekey

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
Co-authored-by: Thorsten Duda <thorsten.duda@sap.com>
2021-06-28 10:50:33 +02:00
Marc Bormeth
b5357f9437
feat: Add environment information to orchestrator package (#2942)
* Fix Orchestrator detection

* Add unit tests

* Add environment info to orchestrator package
2021-06-25 10:50:56 +02:00
Kevin Stiehl
a48b8afc31
fix(commonPipelineEnvironment): keep json numbers untouched (#2908)
* keep numbers untouched

* rebase master
2021-06-23 20:20:43 +02:00
larsbrueckner
61fe88e199
Add "toolrecord" files to Fortify, Checkmarx, Protecode and Whitesource results (#2929)
* Toolrecord framework -
provide a common entry point for post processing code scan results

Changes to be committed:
	new file:   pkg/toolrecord/REAMDE_toolrecord.md
	new file:   pkg/toolrecord/toolrecord_main.go
	new file:   pkg/toolrecord/toolrecord_test.go

* Add toolrecord file to Checkmarx results
modified:   cmd/checkmarxExecuteScan.go

* Add toolrecord file to Fortify results
	modified:   cmd/fortifyExecuteScan.go

* Add toolrecord file to Whitesource results
modified:   cmd/whitesourceExecuteScan.go

* unset umask (#2927)

* (feat) adds error logging output for downloading reports from whitesource (#2928)

* Add toolrecord file to Protecode results

* address code climate findings (1/2)

* address codeclimate findings (2/2)

* add comments to all methods

Co-authored-by: Kevin Stiehl <kevin.stiehl@numericas.de>
Co-authored-by: ffeldmann <felix@bnbit.de>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2021-06-23 15:05:00 +02:00
Sven Merk
ab9e154d10
Replace io.Copy (#2934)
* Replace io.Copy

* Test coverage

* Improve test

* Fix fmt

* Improve error handling in test

* Fix code

* Improve test error log

* Fix fmt

* Fix unix file handles

* Fix error message

* Resolve code climate issue
2021-06-23 14:41:52 +02:00
ffeldmann
6671afb909
(feat) adds error logging output for downloading reports from whitesource (#2928) 2021-06-21 13:36:08 +02:00
Kevin Stiehl
792d435a7f
fix(commonPipelineEnvironment) file and directory permissions (#2924) 2021-06-18 09:56:31 +02:00
Kevin Stiehl
173e887064
[CPE] Missing pipeline environment in sonar docker container (#2909)
* execute writepipeline env in sonar

* non exisitng dir is no error

* add log message
2021-06-16 11:46:55 +02:00
Oliver Nocon
0b48bfcc73
feat: retrieve metadata by stepName - corrected (#2892)
* refactored getConfig to allow stepName param for metadata fetching

* extended step generator

* go generate

* Update cmd/getConfig.go

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Update cmd/getConfig.go

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Update cmd/getConfig.go

* update generated files

* update golden files to care for generator update

* update and add tests

* update generated files

* Update cmd/getConfig.go

* Update cmd/getConfig.go

* update/fix formatting

* feat: retrieve metadata by stepName - corrected

* update generation

* update condition logic for defaults

* update generation & tests

* support multiple conditions

* update generation

* Add generated

Co-authored-by: Leander Schulz <leander.schulz01@sap.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Sven Merk <s.merk@sap.com>
2021-06-16 08:43:30 +02:00
Christopher Fenner
367ca6211a
refactor(protecode): simplify protecode calls (#2838)
* simplify protecode calls

* add todos

* reomve todo

* restore go.sum

* Update cmd/protecodeExecuteScan.go

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>

* Apply suggestions from code review

* remove productID

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2021-06-15 22:29:24 +02:00
Sven Merk
a43f46465a
feat(fortifyExecuteScan): HTML report for Fortify (#2879)
* Tune test

* Fix report implementation

* Fix tests

* Fix values

* Fix code and test

* Report writing fix

* Commit generated sources

* Update cmd/fortifyExecuteScan.go

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Externalize report generation

* Fix fmt

* Fix fmt 2

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2021-06-15 14:53:42 +02:00
Kevin Stiehl
d1c8abc6b3
refactor: move common pipeline environment handling to golang (#2823)
* make use of new read,writePipelineEnv Steps in groovy

* remove unused cat

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2021-06-15 14:34:56 +02:00
Sven Merk
7fa31ae9cc
feat(http): Resilience via retry on intermittent communication issues (#2877)
* feat(http): Resilience on connectivity issues

* Update whitesourceExecuteScan.go

* Update splunk_test.go

* Fix initialization

* Change http interaction

* Fix fmt and tests

* Final test fix
2021-06-15 11:13:24 +02:00
Marcus Holl
fe5ce61d9e
rfc upload (#2533)
* Add RFC upload command
2021-06-14 12:36:18 +02:00
Oliver Nocon
4250ca8bed
Revert "feat(getConfig): retrieve metadata by stepName (#2736)" (#2891)
This reverts commit ae4a24c594.
2021-06-14 10:57:44 +02:00
lndrschlz
ae4a24c594
feat(getConfig): retrieve metadata by stepName (#2736)
* refactored getConfig to allow stepName param for metadata fetching

* extended step generator

* go generate

* Update cmd/getConfig.go

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Update cmd/getConfig.go

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Update cmd/getConfig.go

* update generated files

* update golden files to care for generator update

* update and add tests

* update generated files

* Update cmd/getConfig.go

* Update cmd/getConfig.go

* update/fix formatting

Co-authored-by: Leander Schulz <leander.schulz01@sap.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: OliverNocon <oliver.nocon@sap.com>
2021-06-14 08:58:41 +02:00
Kevin Stiehl
6c7814e4d5
feat(cpm): Add read and write CPE Go step (#2888)
* add read write cpe go steps

* Update pkg/piperenv/CPEMap.go

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Update pkg/piperenv/CPEMap.go

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Update pkg/piperenv/environment.go

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* rename file

* add error handling

* add error handling

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2021-06-11 16:17:40 +02:00
Marc Bormeth
84c3cd399d
fix: orchestrator detection (#2886)
* Fix Orchestrator detection

* Add unit tests

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2021-06-10 23:47:38 +02:00
Marc Bormeth
911d4bc770
feat(sonar): make step orchestrator-agnostic (#2874)
* Make sonarExecuteScan orchestrator-agnostic

* Increase coverage + support empty or false env vars

* Use cleared env for unit tests

* Refactor to standalone package

* Fix review findings

* Fix review findings

* Fix unit test

* Add logging

* Refactor

* Add to codeowners 😎

* Apply suggestions from code review

* Remove unreachable code

* no message

* fix typos

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2021-06-09 09:38:52 +02:00
Christopher Fenner
824cd7d768
fix(jenkins): fix job invocation (#2868)
* update mock

* update signarture

* add test case

* use latest gojenkins

* add integration test

* update mock

* add todo

* add job wrapper

* add job mock

* add test cases

* refactor

* cleanup

* update integration test case
2021-06-02 16:45:22 +02:00
Anil Keshav
a830a35800
Extending mavenBuild step with an option to perform maven deploy with publish flag (#2833)
* modifying detect.maven.excluded.scopes from TEST to test

* new maven alt deployment flags

* changing flag names

* tlsCertificate addtion

* adding publish flags

* new flags

* publish flag

* enhance maven builds

* enhance maven builds

* creating new settings xml

* updating project settings

* changing interface for artifactPreparation that uses the same maven util niterface

* adding general scope to maven params

* global reference

* removing vault tmp

* debuging deployment user

* more debug

* maven build paras

* using smaller case

* adding incorrect error check

* adding deployment flags

* code refactor

* unit tests

* changing scope of paramter for tls certs

* new scope for tls

* remove trailing space in mavenBuild.yaml

* trailing space fix

* typo fix and jenkins secret

* including jenkins credentials for repo pass in the maven build groovy

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-06-01 09:24:36 +02:00
Christopher Fenner
e24b38da24
fix(jenkins): correct interface (#2862)
* update mock

* update signarture

* add test case
2021-05-31 08:54:04 +02:00
Christopher Fenner
62836a64e7
feat(http): expose standard client (#2860)
* expose standard client

* reset go.summ
2021-05-28 12:13:19 +02:00
Christopher Fenner
40c32598a4
feat(jenkins): update jenkins go dependency (#2848)
* update dependency

* use context

* regenerate mock

* decapitalize error messages

* use getBuildFromQueueID

* add comments about mockery

* update mocks

* update tests

* add context parameter

* pull up context

* update mocks

* update tests
2021-05-28 11:12:42 +02:00
ffeldmann
52c25a0dbb
feat(splunk): Adds head commitID of current build to commonPipelineEnvironment (#2844)
* Adds headCommitId, which stores the head commit has of the current build, includes tests and generated files

* Adds headCommitId, which stores the head commit has of the current build, includes tests and generated files

* Go fmt fix

* Fixes artifactoryPrepareVersion test

* Removes xMake CommitId
2021-05-20 13:11:57 +02:00
ravikumarsinghshankarsingh
cac7f2e904
"ignoreSourceFiles" parameter removed from general section and added … (#2814)
* "ignoreSourceFiles" parameter removed from general section and added to specific build tool section

* Update configHelper_test.go

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-05-19 14:57:49 +02:00
Sven Merk
ce06b82450
fix(protecodeExecuteScan): Handling of empty findings (#2818)
* Don't fail if components list is empty.  Resolves failures when scanning images from Crossplane.

* Update formatting with go fmt

* Update pkg/protecode/protecode.go

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Update pkg/protecode/protecode.go

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Fix change, make consistent

Co-authored-by: d.small@sap.com <d.small@sap.com>
Co-authored-by: dee0 <dsmallzero@gmail.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2021-05-17 13:38:17 +02:00
ffeldmann
b88ebdad6c
feat(splunk) Sending telemetry and logging information to Splunk (#2796)
* Adds inital splunk hook and logCollector

* Adds documentation of the Splunk hook

* Fixes markdown lint issues and removes comment from telemetry.go file

* Fixes markdown lint issues and adds missing generated file

* Markdown linting

* Changes documentation according to review, adds Splunk token automatically during init

* Adds error handling for marshalling hook config

* Markdown lint und correct Splunk token in httpclient

* Registeres Splunk token as secret and adjusts test cases

* Adds missing error handling and removes unnecessary comments

* Creates new function readPipelineEnvironment, adds tests

* Moves MonitoringData struct, edits defaults for json fields

* Adds gitRepository and gitOwner to telemetry information

* Simplifies readCommonPipelineEnvironment function, adds more descriptive errors, adds automated adding of Splunk prefix token

* Adjusts error handling

* Cleaner error logging
2021-05-17 12:14:04 +02:00
Christopher Fenner
3a14a91ae5
refactor(versioning): simplify versioning model method (#2825)
* rename artifactVersion to version

* simplify versioningModel
2021-05-14 09:35:31 +02:00
Oliver Nocon
4b666003c8
feat(reporting) update markdown reports (#2819) 2021-05-11 08:01:02 +02:00
Christopher Fenner
98fac0a455
chore(deprecation): change format of deprecation notice from config command (#2822)
* Update config.go

* Update config.go

* add library name

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-05-10 19:18:16 +02:00
Oliver Nocon
5899746767
chore: update error categories (#2827) 2021-05-10 17:44:28 +02:00
Siarhei Pazdniakou
0673d3fed6
Writing secrets to vault (#2770)
Co-authored-by: Kevin Stiehl <kevin.stiehl@numericas.de>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-05-06 10:27:23 +02:00
Christopher Fenner
2426486e7b
feat(sonar): add versioningModel and customScanVersion to sonar scan (#2787)
* add versioningModel parameter

* extract versioning model to own package

* move log message

* use versioning method

* add customScanVersion parameter

* use customScanVersion

* adjust docs on other steps

* update test case
2021-05-05 09:02:19 +02:00
pkrasnousov
e9464b2594
feat(whitesourcescan): add param workDir to whitesourcescan (#2791)
* Add dir to whitesource scan

* Add default for "dir" option

* Change param name to workDir

* Change param name WorkDir to ScanPath

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-05-03 10:53:16 +02:00
Mayur Belur Mohan
05973a8fcd
Help document fixes (#2793)
* byte code Fix

* Help Url Fixes

* Yaml Changes

* Error Message changes

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
2021-04-30 09:43:25 +02:00
Mikalai Dzemidzenka
b82ecb0ff7
convert batsExecuteTests to go implementation (#2737)
* convert batsExecuteTests to go implementation

* added additional test cases, added container definition to batsExecuteTests.yaml

* added influx, for junit added container definition

* added parameter envVars

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-04-29 16:50:23 +02:00
tiloKo
9ac29fd8e6
set User-Agent Header (#2797) 2021-04-29 13:30:25 +02:00
Sven Merk
bf428d1ef9
Fix project lookup query (#2785)
* Fix project lookup

* Added test for space

* Update pkg/fortify/fortify.go

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-04-28 13:59:59 +02:00
Oliver Nocon
b9cd425091
fix(whitesource) proper error message when tokens are not found (#2786) 2021-04-23 11:33:10 +02:00
Sven Merk
2af512e12e
Update configHelper.go (#2768) 2021-04-19 12:01:24 +02:00
Oliver Nocon
d04a7c2eb3
feat(pipelineCreateScanSummary) Create groovy wrapper (#2743)
* feat(pipelineCreateScanSummary) Create groovy wrapper

* add command to binary

* stash step reports

* update stash

* fix typo

* unstash reports first

* update reporting

* update json reporting

* update tests & enhance logging

* update md report

* update md reporting

* fix rendering

* update tests
2021-04-15 07:45:06 +02:00
lndrschlz
b8e9201df5
feat(vault): provide test credentials from vault in every piper go step (#2715)
* added support for test credentials
Co-authored-by: Kevin Stiehl <kevin.stiehl@numericas.de>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-04-09 10:04:35 +02:00
Sven Merk
ff909cb630
Update http.go (#2744) 2021-04-07 15:02:40 +02:00
Oliver Nocon
0696f64ecd
feat(whitesourceExecuteScan) allow custom maven arguments (#2735)
* feat(whitesourceExecuteScan) allow custom maven arguments

* remove comment

Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2021-04-01 14:17:15 +02:00
Sven Merk
60fe1761b6
Update http.go (#2731)
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-03-31 16:08:35 +02:00
Oliver Nocon
24cb3c92e1
fix(config): consider defaults with higher prio than env (#2725)
* fix(config): properly handle empty values from cpe

* fix(config): consider defaults with higher prio than env
2021-03-29 10:40:04 +02:00
Sven Merk
e4346460ed
Add support for yarn (#2721)
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-03-26 09:28:40 +01:00
Sven Merk
dfe9cb6149
Add capability for yarn scanning (#2716)
* Add capability for yarn scanning

* Update whitesource.yaml

* Update configHelper.go

* Add newly generated impl
2021-03-24 16:33:32 +01:00
ravikumarsinghshankarsingh
500556488d
Update configHelper.go (#2687)
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-03-19 16:17:46 +01:00
Oliver Nocon
9b6a8cb3b6
feat(whitesource): create json report (#2699)
* feat(withesource): create json report

* proper constant value

* update generation
2021-03-19 11:10:08 +01:00
tiloKo
b04e3140de
SAP BTP ABAP Environment: addon build with supported Languages (#2689)
* first try

* fix unit test

* remove obsolete attributes
2021-03-15 20:24:43 +01:00
Marcus Holl
0186989593
transport request create solman go (#2583)
* Introduce transport request create solman

* log the transport request id also in case creation failed

maybe there are some rare cases where a transport request id is returned nevertheless.

* report exit code always

* inline parameter

* Provide more parameters in log message, might help with troubleshooting

Co-authored-by: Roland Stengel <r.stengel@sap.com>
2021-03-15 16:44:18 +01:00
Christopher Fenner
0fcbfa8da7
feat(sonar): create report for issue count (#2691)
* create json report

* archive report

* add test case

* generate report

* correct report name
2021-03-12 15:05:07 +01:00
Mayur Belur Mohan
1b27805f64
IntegrationArtifact Cmd Fixes (#2628)
* IntegrationArtifact Cmd Fixes

* CodeClimate Fixes

* CodeClimate Fix

* integrationArtifactDownload Command Fix

* CodeClimate Fixes

* CodeReview Fixes

* CodeReview Fixes

* CodeReview Fixes

* CodeReview Fixes

* CodeReview Fixes

* CodeReview Fixes

* CodeReview FIxes

* CodeReview Fixes

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
2021-03-11 12:03:32 +01:00
tiloKo
b81b11ca9d
abapEnvironmentAssemblePackages new Features (#2661)
* Unit Test Assemble Package

* Remove obsolete lines

dust wiping

* climate change

* climate change #2

* climate change #3

* climate change #4

* climate change #5

* NSPC serial builds

* Actual Delivery Commit

* Download Delivery_logs.zip

* Publish Result

* Testing

* !Polling

* Provide Commit to BF only if set

* dust wiping

* More Dust to Wipe

* Publish more than on file

* Write Log for Publish

* fix unit test (now Dummy Entries)

* save one line of code for climate change

* Update cmd/abapEnvironmentAssemblePackages.go

Co-authored-by: Daniel Mieg <56156797+DanielMieg@users.noreply.github.com>

* More Detailed Log Messages

Co-authored-by: Daniel Mieg <56156797+DanielMieg@users.noreply.github.com>
2021-03-11 11:55:12 +01:00
Christopher Fenner
f999925788
fix(influx): correct data type of influx measurements (#2171)
* update data type of influx measurements

* Update checkmarx.yaml

* pick changes from #1885 for testing

* update generated code

* update to new datatype

* adjust to type changes

* change back to string type

* Update fortifyExecuteScan.go

* add typo to be backward compatible

* change type to int for files_scanned and lines_of_code_scanned

* add typo

* add measurements to whitesource

* update generated sources

* adjust test cases

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-03-10 16:00:53 +01:00
Louis Lecaroz
67ef1f21c7
nexusUpload parameters reading from commonPipelineEnvironment (#2658)
* nexusUpload parameters reading from commonPipelineEnvironment

* Cleaning code formatting

* Fixing go fmt issues

* Fixing go generate files

* Fixing codeclimate issue

* Fixing with PR feedback

* Fixing unit test failure due to changed error message

* Fixing codeclimate on punctuatin in error message

* Fixing typo error in error message

Co-authored-by: LECAROZ <louis.lecaroz@sap.com>
2021-03-10 15:06:42 +01:00
Kevin Stiehl
f2b0683880
add token revocation (#2686)
* add token revocation

* update mocks

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-03-10 08:36:50 +01:00
Sven Merk
e1ea56076f
Http improve retry on timeouts (#2681)
* Add sca cmd extensibility

* Fix formatting

* HTTP retry

* Improve handling of retry on timeout

* Go fmt

* Fix test

* Fix test

* Test stability

* Fix test

* Fix test

* Fix test

* Update fortifyExecuteScan.go
2021-03-09 13:41:07 +01:00
Oliver Nocon
d769fb5148
fix(config): properly handle empty values from cpe (#2682) 2021-03-09 09:30:18 +01:00
Oliver Nocon
c077cdd733
fix(whitesourceExecuteScan) get full list of projects (#2672)
* fix(whitesourceExecuteScan) get full list of projects

* update doc
2021-03-08 17:01:18 +01:00
Kevin Stiehl
649b6d6201
[Vault] Fix ContextParamFilters for vaultSecretFile (#2671)
* fix contextFilter

Co-authored-by: lndrschlz <leander.schulz01@sap.com>

* reduce Congitive Complexity warning

Co-authored-by: lndrschlz <leander.schulz01@sap.com>
2021-03-08 10:16:32 +01:00
Oliver Nocon
4f9eff2303
fix(whitesourceExecuteScan) npm modules only (#2670)
do not fail step in case mta contains only node modules.
No pom.xml required then.

Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2021-03-04 10:38:57 +01:00
Siarhei Pazdniakou
cc66b7cabd
Added createBOM parameter to the npmExecuteScripts step (#2642)
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-03-04 10:16:59 +01:00
Roland Stengel
cd8515acce
Go Unit Tests fail due to windows/linux file separator mismatch (#2662)
* Go Unit Tests fail due to windows/linux
file separator mismatch. See issue 2660.

* review results

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2021-03-03 22:58:29 +01:00
tiloKo
218a743fb1
abapEnvironmentAssemblePackages Unit Tests & Error Handling (#2654)
* Unit Test Assemble Package

* Remove obsolete lines

dust wiping

* climate change

* climate change #2

* climate change #3

* climate change #4

* climate change #5

Co-authored-by: Daniel Mieg <56156797+DanielMieg@users.noreply.github.com>
2021-03-01 13:51:44 +01:00
lndrschlz
4ca9186f39
fix(uiveri5ExecuteTests): add 'tests' stash to step yaml (#2641)
* add tests stash to uiveri5 step

* add stash to uiveri5

* extend step generator with input resources

* add step generator test
2021-03-01 13:03:42 +01:00
Sven Merk
d2eb2877e0
fortifyExecuteScan: Functional enhancements (#2647)
* Improvements

* Formatting

* Fix test

* Update resources/metadata/fortify.yaml

Enhance description

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

* Unify version handling with ws step

* Part 2

* go fmt

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-02-26 13:43:03 +01:00
Roland Stengel
af65dc9076
transport request: consider old label default (#2655) 2021-02-25 17:17:41 +01:00
Oliver Nocon
1581f941b1
fix(whitesource): remove parameters from scan call (#2651)
* fix(whitesource): remove parameters from scan call

parameters are forced into the config thus they don't need to be passed again.

* add proper project to config
2021-02-25 13:16:48 +01:00
Christopher Fenner
80515f3cfb
fix(sonar): remove warnings handling (#2650) 2021-02-24 19:21:40 +01:00
Christopher Fenner
cb3fa7c293
feat(sonar): add output of measurements (#2218)
* add sonarqube measurements

* fetch measurements from API

* add api for fetching issue counts

* add debug outputs

* add further severities

* log number of issues

* report failure

* expose method to send request

* Fixed what was broken.

* add debug output

* wip

* correct opaque property

* push client handling to apiClient.go

* use correct API URL

* correct log outputs

* remove logging

* remove option validation

* extend search options

* restructure

* rename api client file

* simplify client usage

* simplify issue client

* write sonar values to influx

* extract issue service

* reorder imports

* add sonar integration test

* allow unknown fields

* add test case

* add test case

* remove

* fix

* Update http.go

* Apply suggestions from code review

* Update cmd/sonarExecuteScan.go

* rework test cases

* use explicit returns

* add task service

* add waitfortask

* fix typo

* remove fixme

* expose poll interval

* rename test cases

* add test cases

* use newAPIClient method

* use waitForTask

* rename services

* finalize code

* handle error

* move defer

* move types

* add test case

* use http.status...

* add test case

* expose api endpoint names

* extract api client

* adjust test cases

* Update integration-tests-pr.yaml

* Update integration-tests.yaml

* improve require message

* Update integration-tests-pr.yaml

* Update integration-tests-pr.yaml
2021-02-24 15:44:23 +01:00
Marcus Holl
60feb3409f
solman upload go the step (#2522)
Upload content into a transport request via SOLMAN

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
2021-02-19 13:07:18 +01:00
Mayur Belur Mohan
06afce3005
IntegrationArtifactUpload Command (#2610)
* IntegrationArtifactUpload Command
2021-02-17 09:59:56 +01:00
Roland Stengel
7cb4e30497
introduce plainOpen in pkg/git (#2602)
* move transportrequest plainOpen into pkg/git
* artifactPrepareVersion: use plainOpen from git utils
2021-02-15 12:34:19 +01:00
Kevin Stiehl
ffffe8295e
(Vault) Improvements (#2439)
* vault improvements

* Update cloudFoundryDeploy.yaml

remove double PARAMETERS

* go generate

* fix type & resturcutre paragraph to a list

* remove non-existent secrets

* build trigger

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2021-02-15 09:48:51 +01:00
Oliver Nocon
2df2771734
fix(whitesourceExecuteScan) correctly handle structured projects (#2597)
* fix(whitesourceExecuteScan) correctly handle structured projects

report file names of projects which had a structure in the name
like `@test/myProject ` were incorrectly handled

This now prevents that reports are targeted to a sub directory.
Structure is now part of the filename.

* fix CodeClimate finding
2021-02-11 19:39:59 +01:00
Mayur Belur Mohan
f030bc9861
IntegrationArtifactDownload Command (#2587)
* IntegrationArtifactDownload Command

Co-authored-by: Marcus Holl <marcus.holl@sap.com>
2021-02-10 17:08:23 +01:00
Oliver Nocon
d47a17c8fc
feat(whitesource): consolidated reporting and versioning alignment (#2571)
* update reporting and add todo comments

* enhance reporting, allow directory creation for reports

* properly pass reports

* update templating and increase verbosity of errors

* add todo

* add detail table

* update sorting

* add test and improve error message

* fix error message in test

* extend tests

* enhance tests

* enhance versioning behavior accoring to #1846

* create markdown overview report

* small fix

* fix small issue

* make sure that report directory exists

* align reporting directory with default directory from UA

* add missing comments

* add policy check incl. tests

* enhance logging and tests

* update versioning to allow custom version usage properly

* fix report paths and golang image

* update styling of md

* update test
2021-02-10 16:18:00 +01:00
Christopher Fenner
5184a76453
chore(http): allow to send prepared requests (#2580)
* Update http.go

* fix c&p issue

* convert URL to string

* Apply suggestions from code review

fix code climate findings

* add test case
2021-02-08 14:26:15 +01:00
Oliver Nocon
f2f5dbecb8
feat(versioning): add re-use function for proper Scan versioning (#2544)
* feat(versioning): add re-use function for proper Scan versioning

addresses #1846

* add missing comment

Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2021-02-08 12:18:36 +01:00
Marcus Holl
0b9dd80008
Provide getters for stdout, stderr on ShellRunner, ExecRunner (#1787)
* Provide getters for stdout, stderr on ShellRunner, ExecRunner

we need that in order to set the streams back in case we have to scan the command
output ourselvs during some function calls.
2021-02-05 10:54:38 +01:00
Mayur Belur Mohan
538256774a
IntegrationArtifactGetServiceEndpoint Command (#2582)
* GetIntegrationArtifactServiceEndpoint Command

Co-authored-by: Marcus Holl <marcus.holl@sap.com>
2021-02-05 10:35:55 +01:00
Daniel Mieg
d23e26f87f
Revert "Parallel test processing for abaputils (#2570)" (#2578)
This reverts commit e6086ad4a7.
2021-02-04 15:19:42 +01:00
Christopher Fenner
9ec282fd0c
test: allow usage of httpmock in tests (#2576)
* add test case

* add flag for default transport

* Update go.mod

* Update go.sum

* fix spaces

* fix format

* Update http_test.go

* Update http_test.go

* Update http_test.go

* Update http_test.go

* Update http_test.go
2021-02-04 14:58:35 +01:00
Mayur Belur Mohan
426c106765
IntegrationArtifactGetMplStatus Command (#2558)
* IntegrationArtifactGetMplStatus Command

Co-authored-by: Marcus Holl <marcus.holl@sap.com>
2021-02-04 10:50:15 +01:00
Stephan Aßmus
d41d7bfbac
Force WS Project update for mvn plugin (#2575)
In case of policy violations, the scan is failed and default behavior is to not update the project in the WS system.
See docs at https://whitesource.atlassian.net/wiki/spaces/WD/pages/33914890/Maven+Plugin.
Since this is inconvenient and we also changed the behavior already for NPM, we force the update.
2021-02-04 09:17:48 +01:00
Daniel Mieg
e6086ad4a7
Parallel test processing for abaputils (#2570)
* add parallel test processing

* Add more t.Parallel

* add parallel to run

* Remove obsolete parallel statements
2021-02-03 17:31:49 +01:00
Oliver Nocon
a104b2a06d
feat(whitesourceExecuteScan): UA for all build tools, e.g. maven & npm (#2501)
* feat(whitesource): add config helper

this helps to ease & enforce config settings

* fix accidential change of class

* add todos wrt java download

* use existing scanOptions, add option to download jre

* update generation

* fix generation

* allow running UA via go library

* correct image, improve logging

* add removal of downloaded JVM

* update java creation and deletion

* refactor and add log output

* remove obsolete ToDo

* increase test coverage

* increase test coverage

* adding aliases and tests

* make go modules as default

* maven: update behavior of projectNaming

* add Docker capabilities

* correct parameter name

* retrieve Docker coordinates

* docker coordinates only to provide artifact

* add ToDos

* add mta capability

* add aliases, mvn arguments for settings

* clean up groovy part

* update defaults

* add container for pip

* add defaults, add maven specifics, ...

* properly download settings

* maven: check existence of excluded files

* fix reporting

* Update CommonStepsTest.groovy

* update comment

* fix CodeClimate finding

* add tests for pip & fix minor issues

* fix order of pip build descriptors

* update pip container options

* fix pip virtualEnv parameter

* update report permissions

* fix test

* update container options

* add use fileUtils to load properties file

* update parameter description

* adding Docker scanning defaults

* clean up configHelper

* consider also npm tool cache

* add todos
2021-02-03 14:52:48 +01:00
Oliver Nocon
b7754437b3
new step to create a scan summary report (#2559)
* new step to create a scan summary report

* add flag to collect only failed reports

* add stepName to report
2021-02-02 14:36:40 +01:00
Marcus Holl
f0828ad5e5
Find label in log range (#2461)
* LogRange provide git log ref1..ref2

This we need for checking inside the commit range
for transportRequestIds and changeDocumentIDs in the
body of the commit message.
2021-02-01 11:54:30 +01:00
Marcus Holl
1f34b135da
LogRange provide git log ref1..ref2 (#2440)
* LogRange provide git log ref1..ref2

This we need for checking inside the commit range
for transportRequestIds and changeDocumentIDs in the
body of the commit message.
2021-01-29 12:36:15 +01:00
Marcus Holl
c74b99d2c7
Helper for removing duplicates from a string slice (#2455) 2021-01-29 10:52:58 +01:00
Mayur Belur Mohan
0ed5cce53a
IntegrationArtifactUpdateConfiguration Command (#2542)
* IntegrationArtifactUpdateConfiguration Command

Co-authored-by: Marcus Holl <marcus.holl@sap.com>
2021-01-29 09:46:47 +01:00
lndrschlz
e60bdb94a6
feat(uiVeri5): golang implmementation for uiVeri5ExecuteTests (#2394)
* added uiVeri5ExecuteTests step files

* added confPath an regenerated step

* added test for uiVeri5ExecuteTests

* config modified

* added groovy wrapper

* ambiguous method fixed

* uiveri5 wrapper

* removed install command

* fixed defaults

* added testOptions as confPath arg

* test set env

* test npm install local

* changed env settings

* tests regenerated

* go generate

* fix code climate

* overwrite groovy step

* remove groovy wrapper go

* unstash piper bin

* test older node version

* test piperExecuteBin

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* refactored params

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* set testServerUrl as os env for uiveri5

* Update config.yml

* fix naming of testServerUrl param

* wip

* refactored setEnv and fixed tests

* wip

* step param for NPM_CONFIG_PREFIX

* fix runCommand

* refactored step param, regenerate, docu, fix tests

* fix groovy wrapper test

* cleanup

* add to CommonStepsTest field whitelist

* fixed default pipeline environment vars

* fix []string default

* fix metadata.go bug

* added test for docu metadata gen

* fix metadata_test.go in doc gen

* Update metadata_generated.go

* Update metadata_generated.go

* remove npm config prefix param; doc fix

* remove tab

* changed npm config prefix

* removed groovy wrapper test

* removed groovy step defaults

* modify path variable

* modified npm config prefix

* fix error wrapper and tests

* doc update

* add testRepository support

* wip

* fix testRepository param

* wip

* add utils

* init stash content

* wip

* wip

* wip

* add comment for deprecated parameters

* fixed commonStepTest

* fixed commonStepTest

* added error category for testOptions failure

* Update vars/uiVeri5ExecuteTests.groovy

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Update vars/uiVeri5ExecuteTests.groovy

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* fix condition

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2021-01-28 11:31:24 +01:00
Oliver Nocon
44788f1d3a
fix(config): properly read non-string parameters from environment (#2549) 2021-01-28 11:09:43 +01:00
Dominik Lendle
52722c298a
Bug Fixes for abaputils class + unit tests (#2525)
* Bug Fixes

* remove command.Command

Co-authored-by: Daniel Mieg <56156797+DanielMieg@users.noreply.github.com>
2021-01-26 20:23:59 +01:00
Marcus Holl
ee7279c8fc
Add helper for finding uninitialized config (#2534)
Add helper for finding uninitialized config
2021-01-25 12:58:43 +01:00
Oliver Nocon
7bdbed3d65
feat: provide reporting re-use package (#2536)
* initial version of reporting

* added tests

* Update reporting.go
2021-01-25 11:09:03 +01:00
Mayur Belur Mohan
fe72b295d6
IntegrationArtifactDeploy Command (#2526)
* UpdateIntegrationArtifactConfiguration Command

* Fixes for codeclimate Check

* CommonStepsTest changes

* CodeReview Changes

* Git Patch

* Git Patch undo

* Code Review Comments

* code review fixes

* improve the error handling

* codeclimate fixes

* remove json parsing

* Error handling changes

* TestCase coverage fixes

* Refactoring Commands

* IntegrationArtifactDeploy Command

* Regenerate metadata_generated

* codereview fixes

* Code Review Fixes

* Code Review Fixes

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
2021-01-22 15:55:50 +01:00
Sven Merk
3e0a2835fd
Improve HTTP test to cover more retry capabilities (#2528)
* FF disable fulltextsearch

# Conflicts:
#	pkg/fortify/fortify.go
#	pkg/fortify/fortify_test.go

* Improve test on HTTP retry capabilities

* Improve HTTP retry test
2021-01-22 09:56:56 +01:00
Sven Merk
f149292374
[fix]fortifyExecuteScan: disable fulltextsearch (#2527)
* FF disable fulltextsearch

# Conflicts:
#	pkg/fortify/fortify.go
#	pkg/fortify/fortify_test.go

* Completely avoid interacting with fulltextsearch

* Remove also from version lookup
2021-01-21 16:20:46 +01:00
Kevin Stiehl
4a92d1a96a
[Vault] Mixin Vault Parameters for default config (#2523)
* load vault filters for custom defaults

* add tests
2021-01-20 14:59:47 +01:00
Marcus Holl
9e8e5da394
[refactoring] Make room for other upload action: move package (#2518)
* [refactoring] Make room for other upload action: move package

In the near future we will have more upload actions, like
SOLMAN, RFC. Here we prepare the package structure for that.

* don't use aliasing

* rename entities (no leading CTS)
2021-01-18 13:20:30 +01:00
Sven Merk
205d59c1ed
Remove obsolete parameter (#2515)
* Remove obsolete parameter

* Update pkg/fortify/fortify_test.go

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Update fortify_test.go

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2021-01-15 13:55:13 +01:00
Oliver Feldmann
ce9b958410
Document SendRequest error case (#2507)
* Document error case

* Improve docu text
2021-01-12 15:26:45 +01:00
mayurmohan
f4469c3427
DeployIntegrationArtifact Command (#2474)
DeployIntegrationArtifact Command

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
Co-authored-by: Marcus Holl <marcus.holl@sap.com>
2021-01-12 12:17:42 +01:00
Marcus Holl
6d5a30683b
[refactor] Relocate npmExecutorMock so that it can be used from pkg (#2472)
From the current location inside "cmd" the npmExecutorMock cannot
be used from any coding inside "pkg".

When we would like to re-use the npm functionality we have also to
provide tests and this requires having a mock.

In order to be able to use the mock from pkg we move the mock from
"cmd" to "pkg" into package "npm".

With the shift from package "cmd" to "npm" a lot of fields in the mock
has been made public.
2021-01-04 12:46:14 +01:00
Marcus Holl
9172fcae04
Don't expose auth header to log (#2334)
Do not expose auth header to log
2021-01-04 10:06:28 +01:00
lndrschlz
b101106a05
chore: hide warn level if verbose not configured (#2466) 2020-12-22 14:33:11 +01:00
Fabian Reh
d8d4f83fb4
Unit test enhancement (#2471)
* Fixes parallelization in generated code

* Fixes test_code_generated.golden

* generate go code

Signed-off-by: Fabian Reh <fabian.reh@sap.com>

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-12-22 09:30:54 +01:00
Marcus Holl
cdb784aaf4
fiori cts upload (#2390)
fiori cts upload


Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
2020-12-21 09:05:49 +01:00
Marcus Holl
a84692ea97
Remove strange 'Hello', most likly troubleshoot leftover (#2475)
in case this is really important it should be logged and not
emitted to stdout.
2020-12-18 11:15:15 +01:00
Marcus Holl
f9fc642c0e
http client: tell us which resource was not found (#2410)
Up to now we get a message 'Requested resource could not be found' which is not very
helpul during troubleshooting based on the log. With this change we tell the reader
which resource could not be resolved.
2020-12-16 08:55:22 +01:00
Oliver Feldmann
4f6b99cead
Don't allow empty strings as default container parameters (#2406)
First a bug fix is addressed in which the pull policy could not be configured to false by configuring the general configuration. It could neither be configured via dockerExecute or dockerExecuteOnKubernetes, even though this parameter is docker specific. Only by configuring the specific step where one wants to set the pull policy to false can it be configured.

As the bug stems from zero values being in the context config map, which is also addressed with this PR. That is the second part: Context config parameters are only set if they have a value.

* Don't set pull image if not configured

Otherwise, if the pull policy is not set explicitly for a step, dockerPullImage is set to true. Thus, before this change, it cannot be set in the general, or in dockerExecute or in dockerExecuteOnKubernetes configuration.

* Fix unit tests

* Add pullImage parameter test

* Do not place empty default values in context config

* Use putIfNotEmpty for sidecar container options

* Export common configuration

Keys that are set by both main and sidecar container can be exported

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-12-10 11:28:39 +01:00
Daniel Mieg
a3ff90f632
Fix: Change URL creation for abapEnvironmentCloneGitRepo (#2452)
* Fix url creation

* Adapt url creation
2020-12-08 09:31:08 +01:00
tiloKo
080cb025a4
AAKaaS upload in chunks (#2422)
* Upload in Chunks

* fix unit tests for file upload in chunks

* upload in chunks review round 1

* Upload in Chunks - review round 2 - comments

Co-authored-by: Daniel Mieg <56156797+DanielMieg@users.noreply.github.com>
2020-11-26 13:45:51 +01:00
Stephan Aßmus
2f444be997
WhiteSource: Force update of project when "checkPolicies" failed (#2401)
* Force update of WS project
* Refactor file filtering to avoid duplicated code
2020-11-26 11:45:53 +01:00
Sven Merk
2511ec9cea
fix backslash mess (#2428) 2020-11-26 11:22:54 +01:00
Christopher Fenner
860b28884c
chore(jenkins): expose artifacts save method (#2426)
* fix code smell

* expose artifact.Save function

* regenerate mock file
2020-11-26 10:06:14 +01:00
Sven Merk
26af83b1fc
Add slash backslash compatibility (#2425)
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-11-26 09:18:01 +01:00
Kevin Stiehl
e2335b7181
(documentation) fix documentation render bug (#2423)
* fix documentation render bug

* fix tests

* fix complexity
2020-11-26 07:51:43 +01:00
Saschl
ee27cbba05
Removed hardcoded pom.xml and added test (#2393)
Test verifies that correct pom.xml is used as provided in the parameters

Also removed the --file pom.xml in the tests as maven should take pom.xml as default
2020-11-25 17:03:11 +01:00
Sven Merk
5d1782aa01
checkmarxExecuteScan: adapt to 9.2 api (#2363)
* Update checkmarxExecuteScan.go

* api mods

* Switch default

* Fix decode

* mod marshalling

* Fix unmarshalling

* Code fmt and small fix

* Optimize preset handling

* Integer handling

* Fix test

* cleanup

* go fmt

* Improve test
2020-11-25 13:47:26 +01:00
Marcus Holl
f3f221ee80
generator helper: support for map[string]interface{} (#2232)
Up to now the code generator is not able to handle the type
map[string]interface{} which is important for nested
configurations.

With that change we support such nested configuration.

Fo now parameters with a map type are not supported via
command line parameters. Those parameters are simply
ommitted. But with this change is it possible to read
such nested structures from the pipeline configuration
(.pipeline/config.yml).

As a next step we can discuss if we would like to support
such values also via command line parameters. One possible
approach could be

```
./piper <command> -myParam key1=val1 --myParam key2=val2
```

which gets finally collected inside our map:

```
map["key1"] = "val1"
map["key2"] = "val2"
```

This is of course hard to do for deeper nestings. In that case
providing a pointer to a file might be more suitable.

In that context we need to consider how to
  - declare the default values for map like parameters in our
    metadata files.
  - deal with the different types we have for the parameter
    itself wrt the yaml like config on the one hand and on the
    level of the command line parameters on the other hand. Maybe for
    that we have to extend the metadata format (e.g. describe an
    alternate type receiving the values from the command line, like
    []string. With that approach values for simple nested (... not deep
    nested) params can be provided like described above, it would be
    possible to represent these parameters for the command line parser
    as string slice entries like "[]string{key1=val1, key2=val2". These
    parameters needs in this case transformed "by us" into the map we
    use further down the road.

In case we agree in principle on an approach as outlined here we should
adjust the golden files reflecting this use case.
2020-11-23 07:55:20 +01:00
lndrschlz
b9bab27833
feat: expose complete step metadata through generated function (#2329)
* exposing step metadata through generator

* add metadata_generated.go

* fix step go test generation

* metadata fields added to generated files

* added generated files

* removed image placeholder from fortify step

* refactored step meta generation

* go generate

* fixed metadata generator and tests

* added output resource fields/tags to metadata generator

* fix string in metadata_generated

* go generate

* fixed generator

* go generate

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-11-20 08:13:59 +01:00
Kevin Stiehl
dfab156427
Feature/approle secret id rotation (#2311)
* add new step vault secret

* add debug log

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-11-17 13:49:31 +01:00
Christopher Fenner
81c8553d6a
refactor(hadolint): implement step in GO (#1169)
* initial commit of yaml file

* initial commit for HaDoLint in GO

* add helper function to load file from url

* load config file

* write report information to disk

* comment the code

* refactor groovy code

* remove download function from FileUtils

* use http.Downloader

* rename step files

* update generated files

* update generated files

* remove duplicate commands

* add credentials for config url

* add generated test file

* reuse piperExecuteBin functions

* correct step name

* update go step

* deactivate test

* fix import

* use differing go step name

* rename step

* correct result publishing

* correct command name

* expose tls insecure flag

* hand through error

* disable tls verification

* fix tls disabling

* use credentials

* mow

* reformat

* add qgate only if set

* correct report name

* remove old defaults

* add qgate to defaults

* handle report name

* restore default

* remove unused step config

* use piperExecuteBin

* remove obsolete type

* add test cases

* remove groovy tests

* move client parameter handling to run function

* use custom interfaces and mockery

* remove commented code

* correct struct names

* rename parameter dockerfile

* add further asserts

* cleanup

* change file permission to read/write

* remove tokenize

* add further comments

* init http client only if necessary

* add todo

* Revert "rename parameter dockerfile"

This reverts commit 2a570685b8.

* add alias for dockerfile parameter

* correct test case

* Apply suggestions from code review

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>

* add comment about mock assertions

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-11-16 15:14:54 +01:00
Kevin Stiehl
6c51b76fcf
(documentation) Vault Setup documentation (#2366)
* add vault documentation

* fix test

* fix typo

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-11-16 11:53:52 +01:00
Marcus Holl
56586cae1b
Remove assert.Error right before assert.EqualError (#2344)
Is there any benefit from having

```
assert.Error(./.)
assert.EqualError(./.)
```

?

assert.Error ensures that we have an error.
assert.EqualError ensures that we have an error and
moreover it checks for a specific error. Hence
assert.EqualError does all and more what assert.Error
does.

In case there is a benefit from that pattern this PR should not be merged.
In case there is not benefit from that pattern  we should abandong that pattern.
2020-11-11 14:14:55 +01:00
Sven Merk
3c7712f2ee
Retry capabilities for HTTP requests + enablement for Checkmarx step (#2346) 2020-11-11 13:35:53 +01:00
Oliver Nocon
a70933bbd4
fortifyExecuteScan: improve error categorization (#2295)
* fortifyExecuteScan: improve error categorization

* reset error category in success case
2020-11-11 13:04:45 +01:00
Daniel Kurzynski
9a18489cc4
Refactor maven utils and add tests for install artifacts (#2318)
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-11-10 17:14:55 +01:00
Fabian Reh
12d61bc287
Marks tests as parallizable (#2343)
* Makes tests properly parallelized

* Makes tests properly parallelized
2020-11-10 15:22:03 +01:00
Stephan Aßmus
eff38f6c9d
whitesourcExecuteScan-go: Additional fixes (#2315)
* Make sure the UA scan is known to the scan object. Fixes downloading reports later on.
* Move polling into pkg/whitesource, add test for e2e scan
* Remove conditions from stash config resource
* Don't use version stored in CPE. This will prevent the versioningModel from being applied.
2020-11-10 09:09:51 +01:00
Christopher Fenner
e859ed3375
feat(http): expose tls insecure flag (#2324)
* expose tls insecure flag

* fix tls disabling

* move debug message

* add test case
2020-11-09 11:47:03 +01:00
Kevin Stiehl
9d461c6753
consider alias during secret resolving (#2327) 2020-11-06 17:54:01 +01:00
Oliver Nocon
bdf4e8da7d
versioning: gradle (#2319)
* versioning: add gradle

* complete initGetArtifact

* correct error message

* improve init handling
2020-11-06 09:20:08 +01:00
Kevin Hudemann
a04e53df2a
npmExecuteScripts: allow passing a list of build descriptors to execute scripts for (#2312)
This change extends the npmExecuteScripts step to support execution of
npm scripts for specific modules. Previously, it was not possible to
execute npm scripts only for specific modules. Now, if the parameter
buildDesriptorList is set the scripts defined by the runScripts
parameter will be executed for the modules defined by
buildDescriptorList. Note, in this case the buildDescriptorExcludeList
will be ignored.
2020-11-04 16:20:26 +01:00
Christopher Fenner
b7afe8bbf5
feat(http): support empty passwords (#2206)
* add test case

* allow zero-length passwords

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-11-03 19:36:52 +01:00
Fabian Reh
04599e97da
feat(Gitops): Gitops update deployment with helm (#2247)
* makes containerImage not mandatory

* Adds kubectl container

* Adds log statement to debug

* adds general to container image

* removes GENERAL again
Removes condition from Kubectl container

* removes workDir

* marks logs as debug

* adds workingdir again

* Adds author to commits

* Adds commit time now

* remove deprecated and reorder

* adds deprecated again to containerRegistryUrl
Adds GENERAL scope to containerImage

* updates generated file

* Renames containerImageNameTag

* adds else case

* adds debug log

* code cleanup

* adds debug log

* revert

* adds debug logs

* revert

* makes root path not hidden

* revert

* Read container properties

* Removes debug message

* Removes debug message

* Removes general scope again

* Fixes unit test

* Adds helm capabilities to the gitopsUpdateDeployment step

* Adds helm capabilities to gitopsUpdateDeployment step

* Removes condition from input field

* Adds test for invalid deploy tool

* Fixes typo

* Adds tests for git errors and file errors
Simplifies test setup

* Adds test for error on image name extraction

* fixes URL variable name

* adds workind directory to paths

* Refactors too long method

* Reverts refactoring method

* Adds repository name as parameter

* Adds glob method

* Test glob method

* Revert "Test glob method"

This reverts commit ac11b54c14.

* Revert "Adds glob method"

This reverts commit ddf47ddebe.

* Revert "Adds repository name as parameter"

This reverts commit 8fc471c909.

* Removes getWd

* Adds stash deployDescriptor

* removes = from paramters

* Revert "removes = from paramters"

This reverts commit 3ecb3665e2.

* Adds " around parameters

* adds logging of all files

* Updates helm to version 3.3.4

* Clean up debug logs

* Raise error if no branch name provided.
Defaulting should be handled by step configuration.

* clean code

* Updates fields and adds checks for required field for certain deploy tools

* Fixes default commit message

* Update long description

* Removes default parameter

* Update resources/metadata/gitopsUpdateDeployment.yaml

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Updates yaml file

* Add error category and removes too much wrapping

* Update generated file

* Checks all parameters before returning the error

* Introduces constant

* Renames constant

* Fixes unit tests

* unexpose constants

* Makes tests thread safe and resilient to failed deletion

* Remove methods that did not work properly with hash containers rather than tags.

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-11-03 18:29:46 +01:00
Daniel Mieg
aa9dd3b199
Add step abapEnvironmentCreateSystem (#2273)
* Create initial step

* Add parameters for system creation

* Creating a tmp manifest.yml

* Add descriptions

* regenerate

* Create tests

* fix codeclimate issue

* Test

* Test2

* Test3

* Replace os.getTempDir

* Change to fileUtils

* Remove FileUtil
2020-11-03 12:02:13 +01:00
Kevin Stiehl
67727147f8
switch to debug logging for some vault log messages (#2299) 2020-11-03 08:45:12 +01:00
Christopher Fenner
8d682abc83
add package for Jenkins interactions (#2296) 2020-11-02 17:11:18 +01:00
Daniel Mieg
7639175def
Add commit to clone and pull (#2258)
* commit ID for clone

* commit ID for clone

* Remove old default values

* Add commitID to Pull

* Print http response on debug level

* Fix tests

* Adapt tests

* Shorten sleep

* Fix clone tests

* Add ignore commit option

* Adapt tests

* Adapt docu

* Implement feedback

* fix codeclimate issue
2020-11-02 14:17:13 +01:00
Oliver Nocon
edd5ef7709
fix(descriptorUtils) improve error message (#2294) 2020-11-02 13:50:04 +01:00
Oliver Nocon
9725b2a9d7
fix(permissions) allow deletion of errors file (#2293) 2020-11-02 12:34:28 +01:00
Marcus Holl
073a2a8599
generator: Don't swallow output from go fmt (#2222)
* generator: Don't swallow output from go fmt

* fix formatting

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-11-02 10:31:00 +01:00
Kevin Stiehl
ea5e91672d
mask secret also when they are url encoded (#2289)
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-11-02 10:08:34 +01:00
Oliver Nocon
26cfbf7357
fix(whitesourceExecuteScan) properly handle output resources (#2266)
* fix(whitesourceExecuteScan) properly handle output resources

* fix merge issues

* add required aliases

* update generation

* fix reading custom and container environment parameters from cpe

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-11-02 08:51:58 +01:00
Daniel Kurzynski
a2656a5f71
Install artifacts before whitesource scan (#2280) 2020-10-30 15:05:14 +01:00
Daniel Kurzynski
1015371efd
Pass settings file also to install-file (#2279) 2020-10-30 10:04:38 +01:00
Sven Merk
c204abb9cf
checkmarxExecuteScan: Relax timeout (#2265) 2020-10-29 18:03:22 +01:00
Stephan Aßmus
582419e2f5
Up for discussion: Embed best practices (?) in generated steps (#1913)
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-10-29 17:29:07 +01:00
Daniel Kurzynski
6afb0ae507
Pass project settings to flatten command (#2269) 2020-10-29 16:33:58 +01:00
Stephan Aßmus
86f335811c
whitesourceExecuteScan-go: Implement parameters "timeout", "createProductFromPipeline" (#2246) 2020-10-29 09:21:01 +01:00
Stephan Aßmus
2f83ba56da
Make sure detect.sh is removed after use (#2238)
Should avoid issues with this file being owned by root (perhaps via running in docker container), preventing the workspace from being cleaned properly.
2020-10-27 14:29:22 +01:00
Fabian Reh
9c870b2514
Gitops update deployment fixes (#2234)
* makes containerImage not mandatory

* Adds kubectl container

* Adds log statement to debug

* adds general to container image

* removes GENERAL again
Removes condition from Kubectl container

* removes workDir

* marks logs as debug

* adds workingdir again

* Adds author to commits

* Adds commit time now

* remove deprecated and reorder

* adds deprecated again to containerRegistryUrl
Adds GENERAL scope to containerImage

* updates generated file

* Renames containerImageNameTag

* adds else case

* adds debug log

* code cleanup

* adds debug log

* revert

* adds debug logs

* revert

* makes root path not hidden

* revert

* Read container properties

* Removes debug message

* Removes debug message

* Removes general scope again

* Fixes unit test

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-10-27 13:45:34 +01:00
Sven Merk
9d737575aa
fortifyExecuteScan: Fix report download (#2244)
* Fix report download

* Update fortifyExecuteScan.go

* Update fortifyExecuteScan_test.go

* Update fortify.go

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-10-27 13:12:31 +01:00
Kevin Stiehl
24aafb0b69
add vaultSecretFileReferences (#2203)
* add vaultSecretFileReferences

* fix test

* fix test

* go generate

* remove code duplication

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-10-26 14:20:04 +01:00
Marcus Holl
14e1c44aae
Avoid having the mocks in the productive binary (#2199) 2020-10-21 12:48:26 +02:00
Stephan Aßmus
260ca2c5a5
whitesourceExecuteScan: Re-organize code between step and whitesource package (#2207) 2020-10-20 09:49:26 +02:00
Fabian Reh
586044192c
feat(Gitops): new step to update deployment (#2178)
* kanikoExecute: improve user experience

* ensure proper tags

* update permissions

in case a container runs with a different user
we need to make sure that the orchestrator user
can work on the file

* update permissions

* ensure availablility of directories on Jenkins

* (fix) clean up tmp dir in test

* add resilience for incorrect step yaml

* incorporate PR feedback

* Adds piper step to update deployment configuration in external git repository.

https://github.wdf.sap.corp/ContinuousDelivery/piper-ita/issues/21

* Adds handling of branchName as an optional parameter

* Update resources/metadata/gitopsUpdateDeployment.yaml

Feedback about description

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Adapt to interface guide

* Refactors to GitopsExecRunner

* Refactors to GitopsExecRunner in test

* Removes unnecessary mocked methods

* Adds tests for git utils

* Adds new step to CommonStepsTest.groovy

* Updates description from yaml

* Restricts visibility of methods and interfaces
Adds comments where necessary

* Updates comments

* Fixes URL name

* updates description

* updates generated file

* Fixes compile issue in CommonStepsTest.groovy

* Updates long description

* Updates test to run green on all kind of OS

* Removes global variables from tests

* Default branch: master

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

* Typo in Hierarchy

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

* Refactors test to allow parallel execution

* Renames utility variable in gitopsUpdateDeployment.go

* Renames error variables in gitopsUpdateDeployment.go

* simplified parameters for kubectl

* Refactors util classes to use parameters rather than global variables

* makes username and password mandatory

* remove unnecessary mandatory flag

* remove new methods from mock that are not necessary

* replaces with EqualError

* replaces with NoError

* update generated file

* refactor tests

* refactor tests

* make tests parallel executable

* parallel execution of tests

* Refactors interfaces to stop exposing interfaces

* Feedback from PR

* Simplifies failing mocks

* Renames variables and interfaces

* Fixes error messages

* shorten variable names

* Renames unused parameters in tests

* Cleanup nil parameters

* Typo

* Wrap errors and remove unnecessary logs

* Remove containername and filePath from GENERAL scope

* correct generated file

* corrects expected error messages

Co-authored-by: OliverNocon <oliver.nocon@sap.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-10-20 09:05:17 +02:00
Christopher Fenner
0797f4049a
feat(docs): use custom defaults during docs generation (#2022)
* remove docs generator code from step-generator

* add docs generator to dedicated package

* add test cases

* add entry point for docs generation

* make output more readable

* read additional defaults

* add custo defaults parameters

* remove commented code

* adjust custom default parameter in workflow

* remove conflict leftovers

* handle custom default values

* remove comment

* extract code to function

* extract metadata reading to function

* do not print empty strings in favor of PIPER_* env vars

* extract new code to own metadata file

* only reset default on booleans

* remove obsolete test case
2020-10-16 12:50:39 +02:00
Christopher Fenner
8eef130b22
fix(docs): flag mandatory parameters with default value as non-mandatory (#2193)
* fix(docs): flag mandatory parameters with default value as non-mandatory

* Update parameters.go

* add test cases
2020-10-16 09:06:39 +02:00
Christopher Fenner
b859ab411c
fix(docs): correct md syntax for headlines (#2177)
* fix doc headlines

* correct headline level

* adjust test case

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-10-15 08:21:57 +02:00
Oliver Nocon
39089bed5d
kanikoExecute: improve user experience (#2141)
* kanikoExecute: improve user experience

* ensure proper tags

* update permissions

in case a container runs with a different user
we need to make sure that the orchestrator user
can work on the file

* update permissions

* ensure availablility of directories on Jenkins

* (fix) clean up tmp dir in test

* add resilience for incorrect step yaml

* incorporate PR feedback
2020-10-14 11:13:08 +02:00
Christopher Fenner
f3a4342261
feat(influx): handle data types of non-string fields (#2132)
* add type to sonar field

* respect type of influx fields

* update generated code

* switch type

* copy changes from #1885

* log JSON data

* read simple values from json

* Update InfluxData.groovy

* Revert "Update InfluxData.groovy"

This reverts commit c8cfdf381f.

* Revert "read simple values from json"

This reverts commit 94b69866d2.

* Revert "copy changes from #1885"

This reverts commit 2471b4475e.

* update TODO
2020-10-13 16:37:48 +02:00
Oliver Nocon
2e3cf7d97e
checkmarxExecuteScan: hide token in debug mode (#2173) 2020-10-13 15:45:30 +02:00
Kevin Stiehl
3eae0c5f68
feat(vault): fetch secrets from vault (#2032)
* cloud-foundry & sonar from vault

* add vault development hint

* don't abort on vault errors

* cloudfoundry make credentialsId only mandatory when vault is not configured

* add vault ref to step ymls

* rename vaultAddress to vaultServerUrl

* rename PIPER_vaultRole* to PIPER_vaultAppRole*

* add resourceRef for detect step

* fix error when no namespace is set

* added debug logs

* added debug logs

* fix vault resolving

* add vaultCustomBasePath

* rename vault_test.go to client_test.go

* refactored vault logging

* refactored config param lookup for vault

* added tüddelchen

* rename vaultCustomBasePath to vaultPath

* fix tests

* change lookup path for group secrets

* fix interpolation tests

* added vault resource ref to versioning

* execute go generate

* rename Approle to AppRole

* change verbose back to false

Co-authored-by: Leander Schulz <leander.schulz01@sap.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-10-13 14:14:47 +02:00
Oliver Nocon
4fe4f435d9
(fix) clean up tmp dir in test (#2169) 2020-10-13 11:19:01 +02:00