1
0
mirror of https://github.com/SAP/jenkins-library.git synced 2024-12-12 10:55:20 +02:00
Commit Graph

578 Commits

Author SHA1 Message Date
larsbrueckner
dbbbe1f0b3
Updates to toolrecord framework (#2986)
* Toolrecord framework -
provide a common entry point for post processing code scan results

Changes to be committed:
	new file:   pkg/toolrecord/REAMDE_toolrecord.md
	new file:   pkg/toolrecord/toolrecord_main.go
	new file:   pkg/toolrecord/toolrecord_test.go

* Add toolrecord file to Checkmarx results
modified:   cmd/checkmarxExecuteScan.go

* Add toolrecord file to Fortify results
	modified:   cmd/fortifyExecuteScan.go

* Add toolrecord file to Whitesource results
modified:   cmd/whitesourceExecuteScan.go

* unset umask (#2927)

* (feat) adds error logging output for downloading reports from whitesource (#2928)

* Add toolrecord file to Protecode results

* address code climate findings (1/2)

* address codeclimate findings (2/2)

* add comments to all methods

* Toolrecord library:
- move all toolrun files into a subdirectory
- fix timestamp generation in filenames

* add protecode group's URL to toolrecord data

* fix syntax error from previous commit in cmd/protecodeExecuteScan.go

* toolrecord: fix projectVersionID and generated URLs in fortifyExecuteScan.go

* cmd/fortifyExecuteScan.go: replace a hard-coded servername with
config.ServerURL

* update description

* add toolrecord file to detectExecuteScan

* toolrecord/whitesource: add project names as context

Co-authored-by: Kevin Stiehl <kevin.stiehl@numericas.de>
Co-authored-by: ffeldmann <felix@bnbit.de>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2021-07-23 08:48:48 +02:00
Anil Keshav
3c41788405
fix(mavenBuild) looses trust to existing java cacerts (#2977)
* using default java truststore

* default java keytore

* remove trust store

* working directory

* change dir java_home

* env variable

* trying to find jre home

* changing directory to jre home and java home

* trying java jre

* remving $

* trying to search the cacerts

* copying existing cacerts

* removing change directory

* searching for secrutiy folder only

* searching cacerts

* new path for cacert

* path to ca-cert

* new trust store

* changing cacert location

* only adding maven_opts env variable once

* log message

* ca cert path from user

* handelling interface modification

* enhance logs and code clean up

* code clean up

Co-authored-by: Your Name <you@example.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-07-22 11:06:46 +02:00
Oliver Nocon
d04d533604
chore(pipelineCreateScanSummary): update issue text (#2999)
* chore(pipelineCreateScanSummary): update issue text

* Update cmd/pipelineCreateScanSummary.go

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2021-07-20 16:38:11 +02:00
Mayur Belur Mohan
a763112fe6
integrationArtifactUpload Fixes (#2987)
* integrationArtifactUpload Fixes

* format fixes

* Doc fixes

* Fix unit tests

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
2021-07-16 16:09:55 +02:00
Oliver Nocon
8458dd6afa
chore: export GitHub token unmarshalling (#2988) 2021-07-16 08:59:53 +02:00
Christopher Fenner
f78777f784
feat(npm): allow to publish artifact to registry (#2871)
* add new paraeters

* update generated sources

* run npm publish

* add repositoryUrl parameter

* handle registry credentials

* rename parameter

* handle base64encoding

* remove vault reference

* make username secret

* add publish method

* use publish method

* use dedicated registry

* use dry run

* fix

* prepend path

* fix workdir

* move code to npm package

* do changes

* update dependencies

* correct property init

* remomve dry-run

* regenerate

* add mock

* add logging

* add debug log

* dry-run

* remove try run

* remove append

* add debug outut

* change

* add debug output

* changes

* cleanup

* use different auth property

* add credential utils

* add debug log outputs

* remove auth handling & reuse writeFile

* rename

* fix debug output

* remove comments

* update comment

* rename function

* update docs

* update generated files

* handle npm ignore

* remove commented code

* add debug output
2021-07-15 14:46:04 +02:00
Sven Merk
3e7595920f
feat(protecodeExecuteScan): Add protecode report (#2981)
* Fix exclude and enhance docs

* Fix test

* Fix test

* Add reporting to checkmarx step

* Improve text

* Add protecode report

* Fix fmt

* Add error handling
2021-07-12 12:20:25 +02:00
Sven Merk
9571fd28f4
feat(checkmarxExecuteScan): Reporting for pipeline optimization (#2976)
* Fix exclude and enhance docs

* Fix test

* Fix test

* Add reporting to checkmarx step

* Improve text
2021-07-09 10:19:42 +02:00
Oliver Nocon
805a8fd88f
feat(config): read config/defaults with authentication (#2975)
* feat(config):read config/defaults with authentication

This change allows to use defaults and config files from a protected GitHub repository.

The options `--customConfig` and `--defaultConfig` already allowed to provide a link to an uprotected file.

Now, by passing a value in the form `<hostname>:<token>` to parameter `gitHubTokens` (this parameter can be passed multiple times) a token can be provided for dedicated hosts.

This makes it possible to use a link like
`https://api.github.com/repos/SAP/jenkins-library/contents/resources/my-defaults.yml?ref=master`
as reference to a default file or similarly as reference to a configuration file.

* update generation to allow protected config/defaults

* fix CodeClimate issues

* update missing generations
2021-07-08 15:26:07 +02:00
Mayur Belur Mohan
72a4ef16ab
Integration Artifact command Fixes (#2974)
* Add DISCARDED state in query filter

* fix formating

* fomating fixes

* codereview fixes

* Code Review Fixes

* formatting fixes

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
2021-07-08 11:51:31 +02:00
Roland Stengel
43408d0eea
RFC Upload Documentation (#2922)
RFC Upload Doc
- add RFC upload document
2021-07-08 11:07:08 +02:00
Roland Stengel
7e55556d7e
CTS Upload Go (#2969)
CTS Upload Go
- add cpe
- adjust yaml docker, cpe, text
2021-07-08 10:09:18 +02:00
Mikalai Dzemidzenka
9b3162c74a
fix(tls): default images replaced with images with public CA (#2894)
* default images replaced with images with public CA

* go files regenerated

* node buster image changed to lts-stretch in npmExecuteScripts, hadolint image rollbackw

* go files regenerated

* node buster image changed to lts-stretch in npmExecuteLint

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2021-07-05 12:07:22 +02:00
Sven Merk
fbcdd07ffc
improve(fortifyExecuteScan): Improve src and exclude maven defaults (#2953)
* Update uiVeri5ExecuteTests.yaml

* Update uiVeri5ExecuteTests.yaml

* Update uiVeri5ExecuteTests.yaml

* Update uiVeri5ExecuteTests.yaml

* Update uiVeri5ExecuteTests.yaml

* Add generated artifact

* Update fortifyExecuteScan.go

* Fix test

* Fix test

* Fix yet another test

* Back and forth

* Fix documentation

* Property to add fortify context to maven build

* Add comment
2021-07-02 09:43:34 +02:00
Roland Stengel
e2fa05587d
RFC Upload GO (#2903)
* RFC Upload GO
- yaml review
- add cpe
- harmonize with solman
2021-07-01 13:11:21 +02:00
Oliver Feldmann
9e22251c8d
CPI - Improve serviceKey parameter name and description (#2958)
* Make credential id name more descriptive

* Improve serviceKey description

* Change descriptions to add word service

Co-authored-by: Linda Siebert <linda.siebert@sap.com>
2021-07-01 10:07:57 +02:00
Linda Siebert
7a8f4f06eb
Remove platform from triggerIntegration step (#2957) 2021-06-30 17:00:49 +02:00
Linda Siebert
b68f549923
CPI- Remove platform parameter (#2956)
* Remove platform from yaml and generated

* Remove from tests and md
2021-06-30 14:32:30 +02:00
Siarhei Pazdniakou
54f2a0d471
Added go-based influxWriteData step (#2890)
* Added go-based influxWriteData step

* Wrote tests & fixed issues

* Fixed issues

* Created go-based step tests. Fixed issues

* Fixed issues

* Integration test was added

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2021-06-30 11:18:49 +02:00
ffeldmann
3f6eb603c7
Error logging (#2947)
* (feat) adds error logging output for downloading reports from whitesource

* Extended error logging with errors.Wrapf()

* Adjusted whitesource tests
2021-06-30 11:11:41 +02:00
Thorsten Duda
7910df0e8c
new step integrationArtifactTriggerIntegrationTest (#2951)
* new step integrationArtifactTriggerIntegrationTest

* add new step into allow list

* add the new step to main command

* refer cpe

* remove unused unit tests

* Check methods and URLs of http request

* Add TriggerIntegration to mockingutils

* Format code

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
Co-authored-by: Linda Siebert <linda.siebert@sap.com>
2021-06-29 14:50:19 +02:00
Oliver Feldmann
90d0baa56f
Put share cpi parameters in general scope (#2948)
Co-authored-by: Thorsten Duda <thorsten.duda@sap.com>
2021-06-29 10:09:18 +02:00
Linda Siebert
49f4fcf43b
CPI- Change resourceRef of apiServiceKey (#2950)
* Change name and param in yaml

Change name and param in resourceRef of apiServiceKey param

* Generate go code
2021-06-28 16:57:37 +02:00
Linda Siebert
6426a2f792
CPI- Refactor Credential ID (#2946)
* Change credential name in yaml

Change cpiAPIServiceKeyCredentialId to cpiApiServiceKeyCredentialsId in cpi yaml files

* Refactor groovy files

Change cpiAPIServiceKeyCredentialId to cpiApiServiceKeyCredentialsId in cpi groovy files

* Change credential name in md

Change cpiAPIServiceKeyCredentialId to cpiApiServiceKeyCredentialsId in cpi md files
2021-06-28 14:54:03 +02:00
Sven Merk
7b553e1e9a
fix(fortifyExecuteScan): Address module interdependencies (#2938)
* Make sure artifacts go to local repo

* Just package

* Fix test

* Try out silent mode

* Try fail at end

* Bring resilience back

* Follow new strategy

* Fix test
2021-06-28 12:40:20 +02:00
Linda Siebert
78a29d782b
CPI - Introduce service key (#2901)
* Switch to service key for CPI GetMplStatus

Introduces read method for service key files, mock utils and tests.

* Use secret text instead of file

* Change serviceKey definition

* Update cpiUpload to use Service Key

retrieved the host and uaa information from service key

* Update cpiDeploy to use service key

retrieved the host and uaa information from service key

* Update cpiServiceEndpoint to use Service Key

retrieved the host and uaa information from service key

* Update cpiDownload to use Service Key

retrieved the host and uaa information from service key

* Update cpiUpdateConfig to use Service Key

retrieved the host and uaa information from service key

* Refactor serviceKey var name

* Fixed references to service key to follow the real format

they should be accessed through oauth instead of uaa because of the format of the json

* Rename ServiceKey to APIServiceKey

To support having a different service key(and for readability), we need to change the name to API.

* Add STAGES and STEPS yaml

add in to each yaml file of cpi integration

* Revert "Add STAGES and STEPS yaml"

This reverts commit aa2665d158.

* Change comments/formatting commonUtils

Make comments more understandable and follow code climate suggestions

* Change documentation files for steps

remove OAuth and host and change credentials to be servicekey

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
Co-authored-by: Thorsten Duda <thorsten.duda@sap.com>
2021-06-28 10:50:33 +02:00
Marc Bormeth
b5357f9437
feat: Add environment information to orchestrator package (#2942)
* Fix Orchestrator detection

* Add unit tests

* Add environment info to orchestrator package
2021-06-25 10:50:56 +02:00
Kevin Stiehl
a48b8afc31
fix(commonPipelineEnvironment): keep json numbers untouched (#2908)
* keep numbers untouched

* rebase master
2021-06-23 20:20:43 +02:00
Sven Merk
e94cbb0840
Revert "fix(fortifyExecuteScan): Support MTA interdepedencies (#2916)" (#2937)
This reverts commit f7bc956058.
2021-06-23 17:20:15 +02:00
larsbrueckner
61fe88e199
Add "toolrecord" files to Fortify, Checkmarx, Protecode and Whitesource results (#2929)
* Toolrecord framework -
provide a common entry point for post processing code scan results

Changes to be committed:
	new file:   pkg/toolrecord/REAMDE_toolrecord.md
	new file:   pkg/toolrecord/toolrecord_main.go
	new file:   pkg/toolrecord/toolrecord_test.go

* Add toolrecord file to Checkmarx results
modified:   cmd/checkmarxExecuteScan.go

* Add toolrecord file to Fortify results
	modified:   cmd/fortifyExecuteScan.go

* Add toolrecord file to Whitesource results
modified:   cmd/whitesourceExecuteScan.go

* unset umask (#2927)

* (feat) adds error logging output for downloading reports from whitesource (#2928)

* Add toolrecord file to Protecode results

* address code climate findings (1/2)

* address codeclimate findings (2/2)

* add comments to all methods

Co-authored-by: Kevin Stiehl <kevin.stiehl@numericas.de>
Co-authored-by: ffeldmann <felix@bnbit.de>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2021-06-23 15:05:00 +02:00
Sven Merk
f7bc956058
fix(fortifyExecuteScan): Support MTA interdepedencies (#2916)
* Make sure artifacts go to local repo

* Just package

* Fix test
2021-06-23 11:55:34 +02:00
Andre
f63ac3bba0
mavenExecuteIntegration: add maven lifecycle goal as parameter (#2930)
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2021-06-22 12:51:47 +02:00
Kevin Stiehl
c0ae0df2f0
unset umask (#2927) 2021-06-21 09:45:10 +02:00
Oliver Nocon
8883a5148c
feat(mavenBuild): accept build profiles (#2921) 2021-06-18 11:57:00 +02:00
tiloKo
9db249fe1c
Docu updates abap steps (#2910)
* docu updates

* go generate

* remove example as better explained in scenario docu

* Update abapAddonAssemblyKitCheckCVs_generated.go

* PV Step update

* md lint

* create TV

* publish TV

* lint

* register

* release

* reserve

* yaml lint

* Update abapAddonAssemblyKitRegisterPackages_generated.go

* Update resources/metadata/abapAddonAssemblyKitCreateTargetVector.yaml

Co-authored-by: Daniel Mieg <56156797+DanielMieg@users.noreply.github.com>

* Update resources/metadata/abapAddonAssemblyKitPublishTargetVector.yaml

Co-authored-by: Daniel Mieg <56156797+DanielMieg@users.noreply.github.com>

* refer

Co-authored-by: Daniel Mieg <56156797+DanielMieg@users.noreply.github.com>
2021-06-17 21:28:15 +02:00
Kevin Stiehl
29b991d6fc
feat(commonPipelineEnv): consume pipeline environment from env variable if set (#2919)
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2021-06-17 16:54:57 +02:00
Kevin Stiehl
80974ea930
update docker image (#2918)
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-06-17 16:19:39 +02:00
Johannes Schneider
3fa46f20d3
report vulnerability reports in the aggregated vulnerability report only (#2915) 2021-06-17 13:58:10 +02:00
Oliver Nocon
188e409a87
feat(protecodeExecuteScan): allow scanning a binary (#2889)
* feat(protecodeExecuteScan): allow scanning a binary

so far the step only accepts a Docker image or a FetchURL
This adds functionality to also pass a FilePath

* Update protecodeExecuteScan.go

* Update protecodeExecuteScan.go

* Update protecodeExecuteScan.go

* Update protecodeExecuteScan.go

* Update protecodeExecuteScan.go

* Update protecodeExecuteScan.go

* Fix fmt

Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Sven Merk <s.merk@sap.com>
2021-06-17 09:40:21 +02:00
Oliver Nocon
78d7c4c71a
fix(githubCreateIssue): prevent panic for Github error (#2914)
In case the Github connection leads to an error it can happen that a nil pointer dereference exception can occur.
This is to fix this.

Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2021-06-16 22:31:01 +02:00
Kevin Stiehl
ff507b959e
pin kaniko version to v1.3.0-debug (latest working version) (#2911) 2021-06-16 16:37:39 +02:00
Oliver Nocon
0b48bfcc73
feat: retrieve metadata by stepName - corrected (#2892)
* refactored getConfig to allow stepName param for metadata fetching

* extended step generator

* go generate

* Update cmd/getConfig.go

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Update cmd/getConfig.go

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Update cmd/getConfig.go

* update generated files

* update golden files to care for generator update

* update and add tests

* update generated files

* Update cmd/getConfig.go

* Update cmd/getConfig.go

* update/fix formatting

* feat: retrieve metadata by stepName - corrected

* update generation

* update condition logic for defaults

* update generation & tests

* support multiple conditions

* update generation

* Add generated

Co-authored-by: Leander Schulz <leander.schulz01@sap.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Sven Merk <s.merk@sap.com>
2021-06-16 08:43:30 +02:00
Sven Merk
07b90dc10b
fix(fortifyExecuteScan): Throw error on classpath detection issues (#2876)
* Update fortifyExecuteScan.go

* Raise error to the top level

* Update fortifyExecuteScan.go

* Update fortifyExecuteScan.go

* Fix code and test

* Add tests

* Fix test

* Last attempt
2021-06-16 08:15:41 +02:00
Christopher Fenner
367ca6211a
refactor(protecode): simplify protecode calls (#2838)
* simplify protecode calls

* add todos

* reomve todo

* restore go.sum

* Update cmd/protecodeExecuteScan.go

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>

* Apply suggestions from code review

* remove productID

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2021-06-15 22:29:24 +02:00
Sven Merk
a43f46465a
feat(fortifyExecuteScan): HTML report for Fortify (#2879)
* Tune test

* Fix report implementation

* Fix tests

* Fix values

* Fix code and test

* Report writing fix

* Commit generated sources

* Update cmd/fortifyExecuteScan.go

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Externalize report generation

* Fix fmt

* Fix fmt 2

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2021-06-15 14:53:42 +02:00
Sven Merk
7fa31ae9cc
feat(http): Resilience via retry on intermittent communication issues (#2877)
* feat(http): Resilience on connectivity issues

* Update whitesourceExecuteScan.go

* Update splunk_test.go

* Fix initialization

* Change http interaction

* Fix fmt and tests

* Final test fix
2021-06-15 11:13:24 +02:00
Tom Bendrath
3520b36558
Add mtaBuild step parameters: target, source (#2858)
* Add mtaBuild step parameters: target, source

* Add unit test for mtaBuild with custom source

* Simplify mtaBuild parameter defaults for source, target

* Fix mtaBuild default values and path in unit test

* Only append custom target and source params in mtaBuild

* Set mtaBuild source and target to ./ when not customized

* mtaBuild unit test: fix expected params sequence

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Thorsten Duda <thorsten.duda@sap.com>
2021-06-14 16:06:47 +02:00
Oliver Feldmann
f9bfb037a0
Don't configure secret with config file (#2866) 2021-06-14 15:42:23 +02:00
Marcus Holl
fe5ce61d9e
rfc upload (#2533)
* Add RFC upload command
2021-06-14 12:36:18 +02:00
Oliver Nocon
4250ca8bed
Revert "feat(getConfig): retrieve metadata by stepName (#2736)" (#2891)
This reverts commit ae4a24c594.
2021-06-14 10:57:44 +02:00