1
0
mirror of https://github.com/SAP/jenkins-library.git synced 2024-12-14 11:03:09 +02:00
Commit Graph

641 Commits

Author SHA1 Message Date
Johannes Dillmann
5f643826f5
Add source repo for helmExecute (#4082)
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>

Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
2022-11-07 15:35:00 +00:00
Vyacheslav Starostin
c802aa512d
artifactPrepareVersion: update documentation (#4107) 2022-11-07 20:55:03 +06:00
Ashly Mathew
e2c710c035
Introducing syft to generate SBOMS for kaniko builds (#4093)
* Update kanikoExecute.go
* Syft function

* Change installation directory to ease cleanup

* Add createBOM option

* Unit tests

* Refactor code

Co-authored-by: raman-susla-epam <104915202+raman-susla-epam@users.noreply.github.com>
2022-11-07 14:27:05 +01:00
Frank Cheng
7f6fbe4bac
chore: let fortify support maven self Define parameters (#4091)
* let fortify support maven self Define parameters
2022-11-07 16:47:11 +05:30
Jordan Levin
2866ef5592
feat (protecodeExecuteScan) support custom data headers in upload request (#3836)
* support custom data headers in Protecode upload API

* parse custom data properly

* better debugging

* pre-pend META- to custom data header

* prepend to key, not value

* fix debug output

* Fix debug log level

* Make custom header meta upper case key

* fix

* Update http.go

* Update protecode.go

* Update http.go

* address PR review

* Fix merge conflicts

* Update fortifyExecuteScan.yaml

* Update protecodeExecuteScan_generated.go

* Generate step

* fix generated code

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-11-03 17:53:23 +01:00
charly-lemee
885a5e73e3
fix: typo in checkmarx scan (#4072)
* fix: typo with checkmarx report
2022-11-03 13:18:17 +05:30
Christian Volk
6702e146b1
fix(githubPublishRelease): allow repo and owner to be configured globally (#4090) 2022-11-02 13:41:15 +00:00
Vyacheslav Starostin
a84b9d65ac
sonarExecuteScan: run sonar image as root (#4097) 2022-11-01 20:51:25 +06:00
Vyacheslav Starostin
e9c366392e
Update sonar image version (#4094)
Co-authored-by: Vyacheslav Starostin <vyacheslav.starostin@sap.com>
2022-10-31 16:33:24 +01:00
Johannes Dillmann
cfacde97be
Allow run-image to be specified (#4088)
Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>
2022-10-27 15:18:32 +02:00
Vyacheslav Starostin
a7dd0bac50
gradleExecuteBuild: get published artifacts names and write those to CPE (#4076)
* Add new output to CPE && go generate

* Add published artifacts names to CPE

* Add tests

* Change name

Co-authored-by: Vyacheslav Starostin <vyacheslav.starostin@sap.com>
Co-authored-by: Ashly Mathew <ashly.mathew@sap.com>
Co-authored-by: Anil Keshav <anil.keshav@sap.com>
2022-10-25 13:17:42 +02:00
Anil Keshav
f270aa4a17
including sarif files when running implicit report upload from the step itself (#4068)
Co-authored-by: anilkeshav27 <you@example.com>
2022-10-18 09:48:07 +02:00
thtri
8d1e1bac96
fix(fortify,checkmarx): enable SARIF generation as baseline (#4053)
Co-authored-by: sumeet patil <sumeet.patil@sap.com>
2022-10-17 14:09:02 +02:00
Umidjon Urunov
e9707e7638
Added user API key authentication method (#3748)
* changes to detectExec before master merge

* changes for detectExecuteScan

* self generated code added

* fix syntax errors and update docu

* added unit tests for fail and Group

* fix failOn bug

* add Groups as string array

* add Groups as string array

* tests and validation for groups, failOn

* Updated docs and added more tests

* documentation md files should not be changed

* Handle merge conflicts from PR 1845

* fix merge errors

* remove duplicate groups, merge error

* adding buildCode and buildTool as params

* switching build options

* building maven modules

* parameter correction

* parameter correction

* gnerate with new build parameter

* adding comments

* removing piper lib master and modifying goUtils to download 1.5.7 release

* first cleaning then installing

* multi module maven built

* multi module maven built removing unwanted code

* multi module maven built moving inside switch

* testing

* modifying the default use case to also call maven build

* modifying the default use case to also call maven build wih --

* corrected maven build command

* corrected maven build command with %v

* skipping test runs

* testing for MTA project with single pom

* adding absolute path to m2 path

* clean up

* adding switch for mta and maven and removing env from containers

* commiting changes for new detect step

* correting log message

* code clean up

* unit tests changes to detectExecute

* basic tests for new change

* restoring piperGoUtils to download correct piper binary

* code clean up

* code clean up

* protecodeExecuteScan -> Added authentication with user API key

* protecodeExecuteScan -> updating .yml file

* protecodeExecuteScan -> go generate fixed

* protecodeExecuteScan -> naming convention applied for UserAPIKey parameter

* protecodeExecuteScan -> extending groovy code for mapping jenkins credentials

Co-authored-by: D072410 <giridhar.shenoy@sap.com>
Co-authored-by: Keshav <anil.keshav@sap.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-10-10 10:55:21 +02:00
Christopher Fenner
07eeb2f33e
feat: update node image for various steps to new LTS (#3913)
* update node image for steps

* update defaults

* update generated sources

* update tests

Co-authored-by: Vyacheslav Starostin <vyacheslav.starostin@sap.com>
2022-09-26 18:40:55 +06:00
Giridhar Shenoy
d31c0584ff
detectExecuteScan : Add minScanInterval parameter through Piper (#4006)
* add minscaninterval parameter

* update detectExec

* removed a single trailing space which caused a lint failure

* Add test case

* Ensure unmap is false

* fix test case

* update format of param value
2022-09-26 14:08:12 +02:00
Vyacheslav Starostin
6cadb15a11
kubernetesDeploy: add additional unstashing (#4017)
Co-authored-by: Alexey Matvievsky <72048991+matvievsky@users.noreply.github.com>
2022-09-23 17:46:05 +06:00
Vyacheslav Starostin
72e257e83e
helmExecute: rename remoteHelmChartPath to helmChartUrl (#3999) 2022-09-02 15:55:30 +06:00
Daniel Bernd
721994fac5
Feature ATC Run Step - fail on severity (#3986)
* new Parameter - for ATC step "Fail on Severity"

* .yml - add description

* fail on Severity & Unit Test

* Update abapEnvironmentRunATCCheck_test.go

Unit Test correction

* Fail Message

* Fail Step - at last (after persisting files)

* Parameter description change

* yaml trailing spaces removal

* correct Parameter typo and description adjust

* Unit Test correct

* Update cmd/abapEnvironmentRunATCCheck.go

Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com>

* Update abapEnvironmentRunATCCheck.go

* Update abapEnvironmentRunATCCheck_test.go

typo

Co-authored-by: Daniel Bernd <93763187+danManSAP@users.noreply.github.com>
Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com>
Co-authored-by: Daniel Mieg <56156797+DanielMieg@users.noreply.github.com>
2022-08-31 16:57:20 +02:00
Artem Bannikov
4b257377ec
[TMS] Reimplement tmsUpload step in Go (#3399)
* Initially generated tmsUpload<...> files

* First provisioning of parameters supported by tmsUpload step

* Refer to Go step from tmsUpload.groovy

* Initial client implementation

* Reverting line delimiters in tmsUpoad.groovy back to Unix ones

* Temporarily remove when-condition for Release stage

* Define useGoStep parameter in tmsUpload.groovy

* Unstash buildResult if useGoStep is true

* No unstashing and empty credentials, when using go step

* Register TmsUploadCommand in piper.go

* Cleanup groovy-related changes - they will be temporarily implemented in a different repo

* Make getting OAuth token success

* Look through the code and cleanup it a bit

* Read service key from Jenkins credentials store

* Provide initial set of unit tests for methods in /pkg/tms/tms.go file

* Minor improvements on logging response on http call error

* Check, if positive HTTP status code is as expected

* Cleanup tms.yaml file, provide additional unit test for tms.go

* Provide unit test for the case, when request body contains spaces

* Specify nodeExtDescriptorMapping parameter as of type map in tms.yaml

* Implement client method for getting nodes

* Write tests for GetNodes method

* Add GetMtaExtDescriptor client method and cover it with unit tests

* Provide first implementation for Update- and UploadMtaExtDescriptor
client methods

* Provide first implementation for Update- and UploadMtaExtDescriptor
client methods

* Provide UploadFile and UploadFileToNode client methods

* Provide tests for Update- and UploadMtaExtDescriptor client methods

* Write tests for FileUpload and FileUploadToNode client methods

* Minor corrections

* Remove some TODO comments

* Rename some of response structures

* Revert change for line delimiters in cmd/piper.go

* Add uploadType string parameter to UploadFile and UploadRequest methods
of uploader mock to reflect the changed Uploader implementation

* Start to implement execution logic in tmsUpload.go

* Changes in tms.yaml file

- remove resources from inputs in tms.yaml and implement mtaPath
parameter settings in the yaml file the same way, as it is done in
cloudFoundryDeploy.yaml
- rename tms.yaml to tmsUpload.yaml, since some generation policy
changed meanwhile

* Rename tms.yaml to tmsUpload.yaml and do go generate

* Use provided proxy on communication with UAA and TMS

* Set proxy even before getting OAuth token

* Further implementation of tmsUpload.go

* Continuation on implementing the tmsUpload.go executor

* Get mtarFilePath and git commitId from commonPipelineEnvironment, if
they are missing in configuration file + minor changes

* Implement a happy path test for tmsUpload logic

* Cover with unit tests main happy and error paths of tmsUpload.go logic

* Extend set of unit tests for tmsUpload.go

* Eliminate some TODOs, extend unit tests for tmsUpload.go

* Delete some TODOs

* Remove a couple of more TODOs from tms_test.go file

* Provide additional unit test for error due unexpected positive http
status code on upload

* Revert back line delimiters in cmd/piper.go

* Comment out file uploading calls in tmsUpload.go

* Run go generate to update generated files

* Convert line delimiters in tmsUpload.yaml to Unix ones, as well as
provide new line character in the end of the file, plus minor fix for
logging in tmsUpload.go file (pipeline complained)

* Correct description of a parameter in tmsUpload.yaml, extend unit tests
to check for trimming a slash in the end of TMS url for client methods
that do upload

* [minor] Add a comment in the test code

* Add stashContent parameter to do unstashing in tmsUpload.groovy, remove
some of the clarified TODOs

* Uncomment uploading file calls in tmsUpload.go, declare buildResult
stash in tmsUpload.yaml

* Remove clarified TODOs from the tmsUpload.go file

* Run go fmt for jenkins-library/pkg/tms

* Do not get explicitly values from common pipeline environment - all
configurations are provided in yaml file

* Remove unused struct from tmsUpload_test.go

* Run go fmt jenkins-library\pkg\tms

* Revise descriptions of parameters provided in tmsUpload.yaml file

* Specify STAGES scope for tmsUpload parameters

* Provide STAGES scope for the tmsUpload parameters, provide default value
for stashContent parameter

* Remove trailing space from tmsUpload.yaml

* Provide unit tests for proxy-related changes in http.go file

* Improve proxy implementation in tmsUpload.go file

* Make tmsServiceKey again a mandatory parameter

* Run go generate command to make the generated files correspond the yaml
state

* Change line delimiters back to Unix ones (were switched while resolving
the conflicts)

* Remove trailing spaces from tmsUpload.yaml

* Minor change in a comment to trigger pipelines with commit

* Improve checks for zero-structs and for empty maps, as well as use
different package to read files in the tests

* Revert line endings in http.go

* Revert comments formatting changes in files that do not belong to the tmsUpload step
2022-08-30 10:16:09 +02:00
Vyacheslav Starostin
5e305eca40
kubernetesDeploy: add resourceRef (custom/localHelmChartPath) (#3977) 2022-08-25 17:12:13 +06:00
Pavel Busko
14ce92b4fb
cnbBuild: create CycloneDX SBOM file (#3959)
Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
2022-08-18 10:03:24 +02:00
Oliver Feldmann
3da753930c
Allow transport request id from config (#3969) 2022-08-16 11:33:04 +02:00
Vyacheslav Starostin
b31549cf7f
helmExecute: add remoteHelmChartPath CPE value (#3965)
* Add remoteHelmChartPath CPE value

* Fix tests

* Add empty line at the end of yaml file

* Fix yaml file
2022-08-16 01:41:24 +06:00
Daniel Bernd
11e7b5e137
Feature usage com scen901 (#3917)
* Usage of Communication Scenario 901 (instead 510)

* Stage Defaults ATC - Com.Scen 901

* added missing step in Introduction docu

* Correct Link

* Link adapt 2 - generated

* remove Push ATC system configuration again (as only substep)

* feat: allow uploading multiple boms (#3900)

* WIP: Adapt bom names

* + WIP: Adapt bom filenames

* Upgrade cyclonedx gradle plugin and use cyclonedxBom config parameters

* Fix unit tests - use correct name in bom creation

* Fix pythonBuild bom name

* introduce and use npmBomFilename const

* Introduce and use mvnBomFilename const

* Introduce and use gradleBomFilename const

* Use build-tool names for bom suffix

* + Adapt tests (build tool suffix)

* Use BOM schema version 1.2 in gradleExecuteBuild

* Pin version of cyclonedx-maven-plugin to 2.7.1

* Adapt generated files

* Fix integration tests

* Fix integration tests

* Fix gradle build integration tests

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

* fix(cnbBuild): Create separate temp folder for each creator call (#3910)

Co-authored-by: Pavel Busko <pavel.busko@sap.com>

* feat(fortifyExecuteScan): new spotcheck flags (#3923)

* feat: improve vulnerability reporting via GitHub issues (#3924)

* feat: improve vulnerability reporting via GitHub issues

* feat: update reports

* chore: add tls cert links

* only write log on error

* chore: update formatting

* chore: update handling of direct dependencies

* chore: fix linting issue

* chore: minor updates

* correct typo

Co-authored-by: Daniel Bernd <93763187+danManSAP@users.noreply.github.com>
Co-authored-by: R. Kloe <55529941+rkloe@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
Co-authored-by: Pavel Busko <pavel.busko@sap.com>
Co-authored-by: sumeet patil <sumeet.patil@sap.com>
2022-08-15 10:56:08 +02:00
Ashly Mathew
62f47e2d1f
Changes to point release commitish to head commit (#3776)
* Changes to point release commitish to head commit

Signed-off-by: Ashly Mathew <ashly.mathew@sap.com>
2022-08-15 09:48:28 +02:00
Oliver Nocon
05319abfa3
fix(kanikoExecute): enforce image name for multi image build (#3909)
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2022-08-15 09:05:20 +02:00
sumeet patil
be565f1543
feat(fortifyExecuteScan): set python version (#3960)
* set python version
2022-08-12 15:17:11 +02:00
sumeet patil
bb85aa1d7a
fix(fortify): minor fixes (#3946)
* fix(fortify): minor fixes
2022-08-09 15:26:07 +02:00
Sven Merk
b3f37650a2
SBOM creation for Mend (#3934)
* Fix docs and format

* Assessment format added

* Added sample file

* Added parsing

* Added packageurl implementation

* Slight refinement

* Refactored assessment options

* Adapted sample file

* First attempt of ws sbom gen

* Reworked SBOM generation

* Fix test code

* Add assessment handling

* Update dependencies

* Added golden test

* Small fix

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-08-09 13:56:01 +02:00
thtri
2536a9f598
feat(checkmarxExecuteScan): Support threshold for Low finding per Query name (#3938)
* feat(checkmarx): Support threshold for Low finding per Query name

Co-authored-by: sumeet patil <sumeet.patil@sap.com>
2022-08-05 00:17:07 +02:00
Mihai Herda
8061a5c0ab
Add cds generated code to Fortify scans by default. (#3940)
* Add cds generated source code to Fortify scans.

This generated source code is needed to avoid false negatives when scanning code that uses the CAP framework.

* Also change documentation.

* Forgot comma.

* Run go generate.

* Change test.

Co-authored-by: sumeet patil <sumeet.patil@sap.com>
2022-08-04 16:20:14 +02:00
Vyacheslav Starostin
a610e1df6a
Update dtzar/helm-kubectl image version for kuberntesDeploy (#3927) 2022-08-02 14:41:35 +06:00
sumeet patil
c8f069efb2
feat(fortifyExecuteScan): new spotcheck flags (#3923) 2022-08-01 23:06:05 +02:00
R. Kloe
3cad6ac2cd
feat: allow uploading multiple boms (#3900)
* WIP: Adapt bom names

* + WIP: Adapt bom filenames

* Upgrade cyclonedx gradle plugin and use cyclonedxBom config parameters

* Fix unit tests - use correct name in bom creation

* Fix pythonBuild bom name

* introduce and use npmBomFilename const

* Introduce and use mvnBomFilename const

* Introduce and use gradleBomFilename const

* Use build-tool names for bom suffix

* + Adapt tests (build tool suffix)

* Use BOM schema version 1.2 in gradleExecuteBuild

* Pin version of cyclonedx-maven-plugin to 2.7.1

* Adapt generated files

* Fix integration tests

* Fix integration tests

* Fix gradle build integration tests

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-08-01 13:38:49 +02:00
Vyacheslav Starostin
79b07e625b
Add linting capability to step ``golangBuild`` (#3903)
* add golangci-lint functionality

* fix log typos

* fix golangci-lint install dir

* log golangci-lint output report

* specify golangci-lint version, as recommended

* log spelling consistency

* clean code

* refactor golangci-lint runner

* fail build if linter found issues

* fix bug where exit status can't be derived from nil error

* refactor runGolangciLint

* refactor retrieveGolangciLint

* uncomment golang tests

* Use FileWrite method from utils

* Add tests

* Fix test

* fix typo

* alter runLinter param name, improve docs

* undo commenting RunTests...

* alter runLinter name in generated and tests too

* fix variable name (thanks code climate)

* Add usage of ‘go install’ instead of ‘curl’

* Fix tests

* Add usage of functionality of http pkg

* Update tests

* Update tests

* Add usage of piperhttp pkg && update tests

* Add DownloadFile method

* Update tests

Co-authored-by: Jordi van Liempt <35920075+jliempt@users.noreply.github.com>
2022-07-27 11:22:35 +06:00
sumeet patil
818be9d428
feat(codeql): new codeql db parameter (#3902) 2022-07-20 10:07:57 +02:00
Daniel Mieg
39a5ca04b1
Fix typo (#3901) 2022-07-19 16:04:15 +02:00
sumeet patil
bc974ffdd2
Fix documentation for SARIF (#3895) 2022-07-18 12:19:04 +02:00
Anil Keshav
72896fab70
fix (kanikoExecute) enhance existing docker config json with additional credential params : user, password and registry Url (#3892)
* passing registry username and password

* enhance the case for creating docker config json with user credentials

* refactoring code

* unit test and maintaing user provided docker config json file

* go generate

* remove addtional file addition to unit test

Co-authored-by: anilkeshav27 <you@example.com>
2022-07-15 08:40:33 +02:00
Oliver Nocon
b7c0831b7f
feat: allow OSVM scans to succeed with vulnerabilities (#3889)
For running open source vulnerability scans in de-coupled processes
it is helpful to allow that steps only create
compliance reports to inform users/teams
but not fail the pipeline.

This can now be achieved constitently with the flag:
`failOnSevereVulnerabilities`

Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2022-07-12 11:43:24 +02:00
sumeet patil
9c4446ae0a
feat(codeql) merge commit git reference (#3877)
Sets git reference and gitRemoteCommitId.
Jenkins has 2 strategies - 'Merging the pull request with the current target branch revision' and 'The current pull request revision'. When 'Merging the pull request with the current target branch revision' is run, Jenkins creates a local merge commit and runs a job for that particular merge commitId. This commitId is then used for codeql to upload sarif, on upload it throws an error as the merge commit does not exist in github. To resolve this we have introduces a new variable 'gitRemoteCommitId' in commonPipelineEnvironment which gives the remote merge commit id.
2022-07-12 10:25:17 +02:00
Giridhar Shenoy
e6115a54b2
detectExecuteScan : Bug fix : Dont consider ignored components (#3867)
* fix project version limiting issue

* add tests for detectExecute

* fix bug with vuln count

* adjust unit tests

* update documentation for detect versions
2022-07-11 10:50:31 +02:00
Pavel Busko
70d8331904 docs(cnbBuild): use correct address for the buildpacks in the example
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
Co-authored-by: Pavel Busko <pavel.busko@sap.com>
2022-07-08 09:14:28 +02:00
Mayur Belur Mohan
5931415d9c
ApiProviderList Command (#3879)
* ApiProviderList Command

* Metadata Fix

* Metadata Fix

* CodeReview Fixes

* Documentation Fixes

* unit test fix

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
2022-07-07 15:48:59 +03:00
rosemarieB
000e3ab4a9
Add abap source client to generic build step (#3834)
* enable build without values

* add sap-client as option

* use function from /net/url to add parameters

Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com>
2022-07-07 08:44:51 +02:00
Anil Keshav
8187bf2ec5
fix (shellExecute) including comma seperated strings as arguments (#3846)
* including comma seperated strings as arguments

* fix unit test

* adding unit test

* fix unit test no param case

Co-authored-by: anilkeshav27 <you@example.com>
2022-07-06 08:41:44 +02:00
Daniel Mieg
6cf4e98b12
Add alias to CreateTag (#3871) 2022-07-05 09:59:37 +02:00
Daniel Mieg
6ff4143b08
Execute checkout & pull when already cloned (#3850)
* Execute checkout & pull when already cloned

* Revert

* Disallow config overload

* Add custom error handler for clone

* Implement new pull parameters

* Add tests

* Formatting

* Rename Param

* Add comment

* Add docu

* Adapt testst to merge

* Fix Unit Test
2022-06-30 10:43:33 +02:00
Oliver Nocon
5da174aeb0
feat(kubernetesDeploy): add infrastructure extensibility (#3853)
* feat(kubernetesDeploy): add infrastructure extensibility

* update comment

* update error handling

* remove trailing spaces

* chore: refactor to use reuse capabilities

* chore: add tests

* fix: use proper download function

* fix: expose credentials via groovy step

* fix: test

* chore: remove comment

* chore: address CodeClimate findings
2022-06-29 12:00:37 +02:00