1
0
mirror of https://github.com/SAP/jenkins-library.git synced 2024-12-14 11:03:09 +02:00
Commit Graph

4307 Commits

Author SHA1 Message Date
Artem Bannikov
4b257377ec
[TMS] Reimplement tmsUpload step in Go (#3399)
* Initially generated tmsUpload<...> files

* First provisioning of parameters supported by tmsUpload step

* Refer to Go step from tmsUpload.groovy

* Initial client implementation

* Reverting line delimiters in tmsUpoad.groovy back to Unix ones

* Temporarily remove when-condition for Release stage

* Define useGoStep parameter in tmsUpload.groovy

* Unstash buildResult if useGoStep is true

* No unstashing and empty credentials, when using go step

* Register TmsUploadCommand in piper.go

* Cleanup groovy-related changes - they will be temporarily implemented in a different repo

* Make getting OAuth token success

* Look through the code and cleanup it a bit

* Read service key from Jenkins credentials store

* Provide initial set of unit tests for methods in /pkg/tms/tms.go file

* Minor improvements on logging response on http call error

* Check, if positive HTTP status code is as expected

* Cleanup tms.yaml file, provide additional unit test for tms.go

* Provide unit test for the case, when request body contains spaces

* Specify nodeExtDescriptorMapping parameter as of type map in tms.yaml

* Implement client method for getting nodes

* Write tests for GetNodes method

* Add GetMtaExtDescriptor client method and cover it with unit tests

* Provide first implementation for Update- and UploadMtaExtDescriptor
client methods

* Provide first implementation for Update- and UploadMtaExtDescriptor
client methods

* Provide UploadFile and UploadFileToNode client methods

* Provide tests for Update- and UploadMtaExtDescriptor client methods

* Write tests for FileUpload and FileUploadToNode client methods

* Minor corrections

* Remove some TODO comments

* Rename some of response structures

* Revert change for line delimiters in cmd/piper.go

* Add uploadType string parameter to UploadFile and UploadRequest methods
of uploader mock to reflect the changed Uploader implementation

* Start to implement execution logic in tmsUpload.go

* Changes in tms.yaml file

- remove resources from inputs in tms.yaml and implement mtaPath
parameter settings in the yaml file the same way, as it is done in
cloudFoundryDeploy.yaml
- rename tms.yaml to tmsUpload.yaml, since some generation policy
changed meanwhile

* Rename tms.yaml to tmsUpload.yaml and do go generate

* Use provided proxy on communication with UAA and TMS

* Set proxy even before getting OAuth token

* Further implementation of tmsUpload.go

* Continuation on implementing the tmsUpload.go executor

* Get mtarFilePath and git commitId from commonPipelineEnvironment, if
they are missing in configuration file + minor changes

* Implement a happy path test for tmsUpload logic

* Cover with unit tests main happy and error paths of tmsUpload.go logic

* Extend set of unit tests for tmsUpload.go

* Eliminate some TODOs, extend unit tests for tmsUpload.go

* Delete some TODOs

* Remove a couple of more TODOs from tms_test.go file

* Provide additional unit test for error due unexpected positive http
status code on upload

* Revert back line delimiters in cmd/piper.go

* Comment out file uploading calls in tmsUpload.go

* Run go generate to update generated files

* Convert line delimiters in tmsUpload.yaml to Unix ones, as well as
provide new line character in the end of the file, plus minor fix for
logging in tmsUpload.go file (pipeline complained)

* Correct description of a parameter in tmsUpload.yaml, extend unit tests
to check for trimming a slash in the end of TMS url for client methods
that do upload

* [minor] Add a comment in the test code

* Add stashContent parameter to do unstashing in tmsUpload.groovy, remove
some of the clarified TODOs

* Uncomment uploading file calls in tmsUpload.go, declare buildResult
stash in tmsUpload.yaml

* Remove clarified TODOs from the tmsUpload.go file

* Run go fmt for jenkins-library/pkg/tms

* Do not get explicitly values from common pipeline environment - all
configurations are provided in yaml file

* Remove unused struct from tmsUpload_test.go

* Run go fmt jenkins-library\pkg\tms

* Revise descriptions of parameters provided in tmsUpload.yaml file

* Specify STAGES scope for tmsUpload parameters

* Provide STAGES scope for the tmsUpload parameters, provide default value
for stashContent parameter

* Remove trailing space from tmsUpload.yaml

* Provide unit tests for proxy-related changes in http.go file

* Improve proxy implementation in tmsUpload.go file

* Make tmsServiceKey again a mandatory parameter

* Run go generate command to make the generated files correspond the yaml
state

* Change line delimiters back to Unix ones (were switched while resolving
the conflicts)

* Remove trailing spaces from tmsUpload.yaml

* Minor change in a comment to trigger pipelines with commit

* Improve checks for zero-structs and for empty maps, as well as use
different package to read files in the tests

* Revert line endings in http.go

* Revert comments formatting changes in files that do not belong to the tmsUpload step
2022-08-30 10:16:09 +02:00
Jordi van Liempt
402c6085c9
feat(checkIfStepActive): enhance with pipelineEnvironmentFilled condition (#3975)
* add CommonPipelineEnvironmentVariableExists to StepCondition

* add CPE variable existance check to checkIfStepActive

* refactor getCPEEntry()

* condition type -> string

* rename CPE filled condition

* rename CPE condition

Co-authored-by: I557621 <jordi.van.liempt@sap.com>
Co-authored-by: Anil Keshav <anil.keshav@sap.com>
2022-08-29 13:44:32 +02:00
Ashly Mathew
b75d6cf9ed
Enable Jenkins to use checkIfStepActive go step (#3931)
* Extend checkIfStepActive

* adapt default format in abap Tests
Co-authored-by: dominiklendle <d.lendle@sap.com>
Co-authored-by: “Raman <“raman_susla@epam.com”>
2022-08-29 11:39:08 +02:00
Vyacheslav Starostin
5e305eca40
kubernetesDeploy: add resourceRef (custom/localHelmChartPath) (#3977) 2022-08-25 17:12:13 +06:00
Anil Keshav
5b581efc31
fix (mtaBuild) keep .mtar if added as a part of mta id (#3935)
* only remove mtar when its an file extension not other

* keep .mtar if added as a part of mta id

* not adding an addional mtar

* keeping the mtaId intact if the condition

* respecting mta artifact name

* removing condition

* trim only when suffix

* chaging the suffix

* handle case when mtar is natively suffixed

Co-authored-by: anilkeshav27 <you@example.com>
Co-authored-by: Ashly Mathew <ashly.mathew@sap.com>
2022-08-24 14:19:04 +02:00
tiloKo
0a2327dda8
Abap relese confirm fault tollerant (#3916)
* no error if package release fails
2022-08-23 15:33:42 +02:00
Daniel Bernd
f089e3d89e
Optimized json handling during Patch Request build. No fix json struc… (#3976)
* Optimized json handling during Patch Request build. No fix json struct for Config Base Json

* Error handling

* Update abapEnvironmentPushATCSystemConfig.go

Error Handling 2

* Update abapEnvironmentPushATCSystemConfig.go

Error Handling 3

Co-authored-by: Daniel Bernd <93763187+danManSAP@users.noreply.github.com>
2022-08-23 09:49:03 +02:00
Pavel Busko
14ce92b4fb
cnbBuild: create CycloneDX SBOM file (#3959)
Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
2022-08-18 10:03:24 +02:00
Oliver Nocon
85b277c7c6
fix: register base64 encoded secret (#3973)
fixes #3908
2022-08-16 17:52:20 +02:00
Oliver Feldmann
3da753930c
Allow transport request id from config (#3969) 2022-08-16 11:33:04 +02:00
ffeldmann
c52e42fc84
feat(orchestrator) Adds BuildReasons to Jenkins Orchestrator. (#3961)
* Adds PullRequest and ResourceTrigger as BuildReason for Jenkins
2022-08-16 08:33:23 +02:00
Vyacheslav Starostin
b31549cf7f
helmExecute: add remoteHelmChartPath CPE value (#3965)
* Add remoteHelmChartPath CPE value

* Fix tests

* Add empty line at the end of yaml file

* Fix yaml file
2022-08-16 01:41:24 +06:00
Oliver Nocon
74cc828221
chore!: improve performance on Kubernetes (#3932)
* chore: test stashBack excludes

* improve logging

* chore!: improve performance on Kubernetes

* update stash pattern

* update test

Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2022-08-15 12:55:51 +02:00
Daniel Bernd
11e7b5e137
Feature usage com scen901 (#3917)
* Usage of Communication Scenario 901 (instead 510)

* Stage Defaults ATC - Com.Scen 901

* added missing step in Introduction docu

* Correct Link

* Link adapt 2 - generated

* remove Push ATC system configuration again (as only substep)

* feat: allow uploading multiple boms (#3900)

* WIP: Adapt bom names

* + WIP: Adapt bom filenames

* Upgrade cyclonedx gradle plugin and use cyclonedxBom config parameters

* Fix unit tests - use correct name in bom creation

* Fix pythonBuild bom name

* introduce and use npmBomFilename const

* Introduce and use mvnBomFilename const

* Introduce and use gradleBomFilename const

* Use build-tool names for bom suffix

* + Adapt tests (build tool suffix)

* Use BOM schema version 1.2 in gradleExecuteBuild

* Pin version of cyclonedx-maven-plugin to 2.7.1

* Adapt generated files

* Fix integration tests

* Fix integration tests

* Fix gradle build integration tests

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

* fix(cnbBuild): Create separate temp folder for each creator call (#3910)

Co-authored-by: Pavel Busko <pavel.busko@sap.com>

* feat(fortifyExecuteScan): new spotcheck flags (#3923)

* feat: improve vulnerability reporting via GitHub issues (#3924)

* feat: improve vulnerability reporting via GitHub issues

* feat: update reports

* chore: add tls cert links

* only write log on error

* chore: update formatting

* chore: update handling of direct dependencies

* chore: fix linting issue

* chore: minor updates

* correct typo

Co-authored-by: Daniel Bernd <93763187+danManSAP@users.noreply.github.com>
Co-authored-by: R. Kloe <55529941+rkloe@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
Co-authored-by: Pavel Busko <pavel.busko@sap.com>
Co-authored-by: sumeet patil <sumeet.patil@sap.com>
2022-08-15 10:56:08 +02:00
Ashly Mathew
62f47e2d1f
Changes to point release commitish to head commit (#3776)
* Changes to point release commitish to head commit

Signed-off-by: Ashly Mathew <ashly.mathew@sap.com>
2022-08-15 09:48:28 +02:00
Oliver Nocon
05319abfa3
fix(kanikoExecute): enforce image name for multi image build (#3909)
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2022-08-15 09:05:20 +02:00
Oliver Nocon
1e2f2d93e3
feat: add commit and pipeline to GitHub reporting (#3930)
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2022-08-15 07:48:22 +02:00
sumeet patil
91eb80fc68
feat(checkmarxExecuteScan): new json report flag (#3964) 2022-08-12 18:39:28 +02:00
Peter Persiel
5d2ab86674
Add-on Build Pipeline: cause the current build to stop depending on ATC check quality gate (#3833)
* Update extensibility.md

* Update abapEnvironmentAddons.md

* Update documentation/docs/pipelines/abapEnvironment/extensibility.md

Co-authored-by: Daniel Mieg <56156797+DanielMieg@users.noreply.github.com>

* Update extensibility.md

Co-authored-by: Daniel Mieg <56156797+DanielMieg@users.noreply.github.com>
2022-08-12 16:03:44 +02:00
sumeet patil
be565f1543
feat(fortifyExecuteScan): set python version (#3960)
* set python version
2022-08-12 15:17:11 +02:00
sumeet patil
89bc41a3d0
feat(fortifyExecuteScan): json report new flag (#3963)
- json report new flag isSpotChecksPerCategoryAudited for spotchecks

Co-authored-by: thtri <thanh.hai.trinh@sap.com>
2022-08-12 13:27:31 +02:00
Sven Merk
21416d82ed
Fix SBOM component generation (#3958)
* Further improve library types

* Fix translate

* Added debug output

* Enhance data

* Added debug output

* Fix code

* Added test

* Fix test
2022-08-12 11:59:47 +02:00
Ashly Mathew
374cdb777b
fix(checkIfStepActive): nested key logic (#3939)
* fix(checkIfStepActive): nested key logic

Co-authored-by: Raman Susla <raman_susla@epam.com>
Co-authored-by: Ashly Mathew <ashly.mathew@sap.com>
2022-08-12 09:02:15 +02:00
Sven Merk
c81e741224
Refinement of SARIF generation for BD and WS (#3942)
* Fix docs and format

* Assessment format added

* Added sample file

* Added parsing

* Added packageurl implementation

* Slight refinement

* Refactored assessment options

* Adapted sample file

* First attempt of ws sbom gen

* Reworked SBOM generation

* Fix test code

* Add assessment handling

* Update dependencies

* Added golden test

* Small fix

* feat(fortify): Added a check for fortify binary in $PATH (#3925)

* added check for fortifyupdate and sourceanalyzer bin

Co-authored-by: sumeet patil <sumeet.patil@sap.com>

* Modify SARIF

* Enhanced SARID contents

* Small refinement for hub detect

* Small adjustments

* Extend SARIF contents

* Consistency to Mend part

* Fix tests

* Fix merge

* Fix test

* Add debug log, enhance output

* Enhance meta info

* Fix libType for node

* Fix log entry

* Fix pointers and test

* Fix test

* Fix library types

* Fix test

* Extend libType mappings

Co-authored-by: Vinayak S <vinayaks439@gmail.com>
Co-authored-by: sumeet patil <sumeet.patil@sap.com>
2022-08-11 13:12:14 +02:00
sumeet patil
ed4467282f
fix(fortify): Fortify spotcheck logic consistent with checkmarxs low (#3955)
* Improve logging

* Fortify spotcheck logic consistent with checkmarx
2022-08-11 11:44:16 +02:00
Vyacheslav Starostin
65a637ca20
Update golangci-lint version (#3952)
Co-authored-by: Jordi van Liempt <35920075+jliempt@users.noreply.github.com>
2022-08-11 14:21:24 +06:00
thtri
10a7b72418
fix(checkmarx): inconsistent logic in calculating Low/Info audited numbers (#3953) 2022-08-10 18:46:22 +02:00
ffeldmann
a7d43b8571
fix: complete changeSet list (#3951)
* fixes: complete changeset list

* Adds test cases for multiple changeSets

* Adds PrNumber to ChangeSet

* Changes timestamp to Timestamp
2022-08-10 14:57:21 +02:00
Oliver Nocon
924ff6552f
fix(abapAddonAssemblyKitCheckPV): report generation (#3949) 2022-08-09 18:02:57 +02:00
Anil Keshav
cc1bc02501
addig correct ws api call (#3948)
Co-authored-by: anilkeshav27 <you@example.com>
2022-08-09 17:29:23 +02:00
sumeet patil
bb85aa1d7a
fix(fortify): minor fixes (#3946)
* fix(fortify): minor fixes
2022-08-09 15:26:07 +02:00
Sven Merk
b3f37650a2
SBOM creation for Mend (#3934)
* Fix docs and format

* Assessment format added

* Added sample file

* Added parsing

* Added packageurl implementation

* Slight refinement

* Refactored assessment options

* Adapted sample file

* First attempt of ws sbom gen

* Reworked SBOM generation

* Fix test code

* Add assessment handling

* Update dependencies

* Added golden test

* Small fix

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-08-09 13:56:01 +02:00
Oliver Nocon
a46f796bcd
chore: cleanup reporting & some incorrect file usage in tests (#3943)
* chore: cleanup reporting & some incorrect file usage in tests

* cleanup interface

* chore: remove comment

* preserve error handling

* Rename FileUtils.go to fileUtils.go

* clean up formatting

* chore: address static check findings

* fix brittle test

* chore: cleanup formatting
2022-08-09 10:57:02 +02:00
Alexey Matvievsky
9f8064d733
url log permission hotfix (#3945) 2022-08-08 12:10:35 +04:00
Oliver Nocon
43bbea477c
fix(protecodeExecuteScan): correct regex pattern for replacing spaces (#3941) 2022-08-05 16:16:36 +02:00
Alexey Matvievsky
da8cda6dbe
feat: http report creation for build steps (#3888)
* URL logging feature for execution step provided
2022-08-05 15:08:19 +04:00
thtri
2536a9f598
feat(checkmarxExecuteScan): Support threshold for Low finding per Query name (#3938)
* feat(checkmarx): Support threshold for Low finding per Query name

Co-authored-by: sumeet patil <sumeet.patil@sap.com>
2022-08-05 00:17:07 +02:00
Mihai Herda
8061a5c0ab
Add cds generated code to Fortify scans by default. (#3940)
* Add cds generated source code to Fortify scans.

This generated source code is needed to avoid false negatives when scanning code that uses the CAP framework.

* Also change documentation.

* Forgot comma.

* Run go generate.

* Change test.

Co-authored-by: sumeet patil <sumeet.patil@sap.com>
2022-08-04 16:20:14 +02:00
Vinayak S
aa41641d41
feat(fortify): Added a check for fortify binary in $PATH (#3925)
* added check for fortifyupdate and sourceanalyzer bin

Co-authored-by: sumeet patil <sumeet.patil@sap.com>
2022-08-04 14:04:54 +02:00
Oliver Nocon
73f7d61743
fix: remove side-effects of #3875 (#3928)
with #3875 temp directory was created in current workspace.
This had negative side-effects: For example npm build packaged and published temporary files

Co-authored-by: Anil Keshav <anil.keshav@sap.com>
2022-08-04 09:20:59 +02:00
Vyacheslav Starostin
a610e1df6a
Update dtzar/helm-kubectl image version for kuberntesDeploy (#3927) 2022-08-02 14:41:35 +06:00
Oliver Nocon
d640d72dc6
feat: improve vulnerability reporting via GitHub issues (#3924)
* feat: improve vulnerability reporting via GitHub issues

* feat: update reports

* chore: add tls cert links

* only write log on error

* chore: update formatting

* chore: update handling of direct dependencies

* chore: fix linting issue

* chore: minor updates
2022-08-02 08:26:26 +02:00
sumeet patil
c8f069efb2
feat(fortifyExecuteScan): new spotcheck flags (#3923) 2022-08-01 23:06:05 +02:00
Ralf Pannemans
2f1f4b18ac
fix(cnbBuild): Create separate temp folder for each creator call (#3910)
Co-authored-by: Pavel Busko <pavel.busko@sap.com>
2022-08-01 17:02:52 +02:00
R. Kloe
3cad6ac2cd
feat: allow uploading multiple boms (#3900)
* WIP: Adapt bom names

* + WIP: Adapt bom filenames

* Upgrade cyclonedx gradle plugin and use cyclonedxBom config parameters

* Fix unit tests - use correct name in bom creation

* Fix pythonBuild bom name

* introduce and use npmBomFilename const

* Introduce and use mvnBomFilename const

* Introduce and use gradleBomFilename const

* Use build-tool names for bom suffix

* + Adapt tests (build tool suffix)

* Use BOM schema version 1.2 in gradleExecuteBuild

* Pin version of cyclonedx-maven-plugin to 2.7.1

* Adapt generated files

* Fix integration tests

* Fix integration tests

* Fix gradle build integration tests

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-08-01 13:38:49 +02:00
Srinikitha Kondreddy
1103a99519
Add dist folder into build result (#3914) 2022-07-29 09:57:39 +02:00
Vyacheslav Starostin
79b07e625b
Add linting capability to step ``golangBuild`` (#3903)
* add golangci-lint functionality

* fix log typos

* fix golangci-lint install dir

* log golangci-lint output report

* specify golangci-lint version, as recommended

* log spelling consistency

* clean code

* refactor golangci-lint runner

* fail build if linter found issues

* fix bug where exit status can't be derived from nil error

* refactor runGolangciLint

* refactor retrieveGolangciLint

* uncomment golang tests

* Use FileWrite method from utils

* Add tests

* Fix test

* fix typo

* alter runLinter param name, improve docs

* undo commenting RunTests...

* alter runLinter name in generated and tests too

* fix variable name (thanks code climate)

* Add usage of ‘go install’ instead of ‘curl’

* Fix tests

* Add usage of functionality of http pkg

* Update tests

* Update tests

* Add usage of piperhttp pkg && update tests

* Add DownloadFile method

* Update tests

Co-authored-by: Jordi van Liempt <35920075+jliempt@users.noreply.github.com>
2022-07-27 11:22:35 +06:00
Vyacheslav Starostin
1f242ea139
feat(helmExecute): update value files with dynamic values (#3861)
* Add getAndRenderImageInfo func

* Add unit tests

* Add comments

* Improve value files handling

* Rename getAndRenderImageInfo to parseAndRenderCPETemplate

* Clean up

* Update logic to parse and render templates

* Update tests

* Test: use t.TempDir for creating temporary dir

* Use ParseTemplate method from piperenv pkg

* Fix err message

* Fix test
2022-07-25 14:14:30 +06:00
Jesse Awan
274c11d28f
Add transport request to GPP (#3862)
* Add TransportRequestUploadCTS step to Release

* typo comma

* test transportRequest git ID

* Update piperPipelineStageInit.groovy

* add echo

* aggressive echo

* Update piperPipelineStageInit.groovy

* remove echo + add unitTests

* fix typos and documentation syntax

* test documentation syntax

* test documentation syntax

* Switch to shell

* Documentation changes

* Add review changes

* Remove echo

* Refactor test cases

Co-authored-by: Kondreddy <srinikitha.kondreddy@sap.com>
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
Co-authored-by: Roland Stengel <r.stengel@sap.com>
2022-07-22 15:15:53 +02:00
Pavel Busko
5fb43a9ead
cnbBuild: update buildpack versions for integration tests (#3907)
Co-authored-by: Pavel Busko <pavel.busko@sap.com>
2022-07-21 15:41:37 +02:00