1
0
mirror of https://github.com/SAP/jenkins-library.git synced 2024-12-14 11:03:09 +02:00
Commit Graph

783 Commits

Author SHA1 Message Date
Eng Zer Jun
0f4e30e9db
test: use T.TempDir to create temporary test directory (#3721)
This commit replaces `ioutil.TempDir` with `t.TempDir` in tests. The
directory created by `t.TempDir` is automatically removed when the test
and all its subtests complete.

Prior to this commit, temporary directory created using `ioutil.TempDir`
needs to be removed manually by calling `os.RemoveAll`, which is omitted
in some tests. The error handling boilerplate e.g.
	defer func() {
		if err := os.RemoveAll(dir); err != nil {
			t.Fatal(err)
		}
	}
is also tedious, but `t.TempDir` handles this for us nicely.

Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-07-12 15:19:12 +02:00
Giridhar Shenoy
e6115a54b2
detectExecuteScan : Bug fix : Dont consider ignored components (#3867)
* fix project version limiting issue

* add tests for detectExecute

* fix bug with vuln count

* adjust unit tests

* update documentation for detect versions
2022-07-11 10:50:31 +02:00
rosemarieB
000e3ab4a9
Add abap source client to generic build step (#3834)
* enable build without values

* add sap-client as option

* use function from /net/url to add parameters

Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com>
2022-07-07 08:44:51 +02:00
raman-susla-epam
6bc96faba9
Update vault.go (#3875) 2022-07-05 16:20:53 +02:00
Vyacheslav Starostin
9e4b32778f
helmExecute: Improve RunHelmLint method (#3873)
* Update RunHelmLint method

* Add test

Co-authored-by: Vyacheslav Starostin <vyacheslav.starostin@sap.com>
2022-07-05 18:22:49 +06:00
Oliver Feldmann
ebedf412e4
remove Pipeline word in subject (#3874) 2022-07-05 11:55:07 +03:00
xgoffin
e4b22deec0
fix(fortifyExecuteScan): remove unneeded parameter (project), fix rule ID in rules (#3866)
* fix(fortifyExecuteScan): remove unneeded parameter (project), fix rule ID in rules

* trigger UT
2022-06-30 17:00:29 +02:00
ffeldmann
f490900f1f
feat(orchestrator package) Adds GetChangeSet functionality to Jenkins provider (#3852)
* Adds GetChangeSet functionality to Jenkins provider
2022-06-30 12:49:21 +02:00
Daniel Mieg
6ff4143b08
Execute checkout & pull when already cloned (#3850)
* Execute checkout & pull when already cloned

* Revert

* Disallow config overload

* Add custom error handler for clone

* Implement new pull parameters

* Add tests

* Formatting

* Rename Param

* Add comment

* Add docu

* Adapt testst to merge

* Fix Unit Test
2022-06-30 10:43:33 +02:00
Daniel Mieg
8923d1e583
Add Execution Log (#3863)
* Add Execution Log

* Adapt Tests

* Remove log
2022-06-29 15:50:33 +02:00
Oliver Nocon
5da174aeb0
feat(kubernetesDeploy): add infrastructure extensibility (#3853)
* feat(kubernetesDeploy): add infrastructure extensibility

* update comment

* update error handling

* remove trailing spaces

* chore: refactor to use reuse capabilities

* chore: add tests

* fix: use proper download function

* fix: expose credentials via groovy step

* fix: test

* chore: remove comment

* chore: address CodeClimate findings
2022-06-29 12:00:37 +02:00
Siarhei Pazdniakou
ac821917d1
Fix helmExecute step (#3827)
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-06-29 10:03:54 +02:00
xgoffin
51138cfb49
feat(checkmarxExecuteScan): remove some useless SARIF code, added security-severity (#3857) 2022-06-28 15:39:17 +02:00
Daniel Mieg
3184785c5e
Add parameter to pull step (#3851)
* Add parameter to pull step

* Add test for config

* Adapt docu

* Update resources/metadata/abapEnvironmentPullGitRepo.yaml

Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com>

* Regenerate

Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com>
2022-06-28 11:02:15 +02:00
Vinayak S
b25f596af5
fix(fortify): project version fetch bug (#3854)
Fixed the filter for projectversion GET call
2022-06-27 14:43:57 +02:00
Oliver Feldmann
9af9a2418d
Change pipeline prefix to pipeline (#3855) 2022-06-27 11:38:33 +02:00
Martin Zuber
1cb1a203cb
Support custom general purpose vault credentials without prefix. (#3779)
Co-authored-by: Anil Keshav <anil.keshav@sap.com>
2022-06-27 09:24:52 +02:00
sumeet patil
78cf40799e
feat(codeql): new codeql piper step (#3765)
* codeql piper step
2022-06-24 09:04:24 +02:00
Linda Siebert
acbcc5646b
[ANS] Change helper and re-generate steps (#3675)
* Add ans implementation

* Remove todo comment

* Rename test function

Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>

* Better wording

Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>

* Add reading of response body function

* Use http pkg ReadResponseBody

* Check read error

* Better test case description

* Fix formatting

* Create own package for read response body

* Omit empty nested resource struct

* Separate Resource struct from Event struct

* Merge and unmarshall instead of only unmarshalling

* Improve status code error message

* Remove unchangeable event fields

* Separate event parts

* Change log level setter function

* Restructure ans send test

* Revert exporting readResponseBody function

Instead the code is duplicated in the xsuaa and ans package

* Add check correct ans setup request

* Add set options function for mocking

* Review fixes

* Correct function name

* Use strict unmarshalling

* Validate event

* Move functions

* Add documentation comments

* improve test

* Validate event

* Add logrus hook for ans

* Set defaults on new hook creation

* Fix log level on error

* Don't alter entry log level

* Set severity fatal on 'fatal error' log message

* Ensure that log entries don't affect each other

* Remove unnecessary correlationID

* Use file path instead of event template string

* Improve warning messages

* Add empty log message check

* Allow configuration from file and string

* Add sourceEventId to tags

* Change resourceType to Pipeline

* Use structured config approach

* Use new log level set function

* Check correct setup and return error

* Mock http requests

* Only send log level warning or higher

* Use new function name

* One-liner ifs

* Improve test name

* Fix tests

* Prevent double firing

* Reduce Fire test size

* Add error message to test

* Reduce newANSHook test size

* Further check error

* Rename to defaultEvent in hook struct

* Reduce ifs further

* Fix set error category test

The ansHook Fire test cannot run in parallel, as it would affect the
other tests that use the error category.

* Change function name to SetServiceKey

* Validate event

* Rename to eventTemplate in hook struct

* Move copy to event.go

* Fix function mix

* Remove unnecessary cleanup

* Remove parallel test

The translation fails now and again when parallel is on.

* Remove prefix test

* Remove unused copyEvent function

* Fix ifs

* Add docu comment

* Register ans hook from pkg

* register hook and setup event template seperately

* Exclusively read eventTemplate from environment

* setupEventTemplate tests

* adjust hook levels test

* sync tests- wlill still fail

* migrate TestANSHook_registerANSHook test

* fixes

* Introduce necessary parameters

* Setup hook test

* Use file instead

* Adapt helper for ans

* Generate go files

* Add ans config to general config

* Change generator

* Regenerate steps

* Allow hook config from user config

Merges with hook config from defaults

* Remove ans flags from root command

* Get environment variables

* Generate files

* Add test when calling merge twice

* Update generator

* Regenerate steps

* Check two location for ans service key env var

* Re-generate

* Fix if

* Generate files with fix

* Duplicate config struct

* Add type casting test for ans config

* Fix helper

* Fix format

* Fix type casting of config

* Revert "Allow hook config from user config"

This reverts commit 4864499a4c497998c9ffc3e157ef491be955e68e.

* Revert "Add test when calling merge twice"

This reverts commit b82320fd07b82f5a597c5071049d918bcf62de00.

* Add ans config tests

* Improve helper code

* Re-generate commands

* Fix helper unit tests

* Change to only one argument

* Fix helper tests

* Re-generate

* Revert piper and config changes

* Re-generate missing step

* Generate new steps

* [ANS] Add servicekey credential to environment (#3684)

* Add ANS credential

* Switch to hooks and remove comments

* Add subsection for ans

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>

* Remove changes to piper.go

* Remove formatting

* Add test for ANS

* Define hook credential seperately from step credential

* Add test for retrieval from general section

* Add comment

* Get ans hook info from DefaultValueCache

* [ANS] Add documentation (#3704)

* Add ANS credential

* Switch to hooks and remove comments

* Add subsection for ans

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>

* Remove changes to piper.go

* Remove formatting

* Add test for ANS

* Define hook credential seperately from step credential

* Add test for retrieval from general section

* Add comment

* Add documentation

* Review changes

* Review comments

* Improve documentation further

* Add note of two event templates

* Add log level destinction

* Further improvements

* Improve text

* Remove unused things

* Add ANS credential

* Switch to hooks and remove comments

* Add subsection for ans

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>

* Remove changes to piper.go

* Remove formatting

* Add test for ANS

* Define hook credential seperately from step credential

* Add test for retrieval from general section

* Add comment

* Get ans hook info from DefaultValueCache

* Improvements

Co-authored-by: Linda Siebert <linda.siebert@sap.com>
Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>

* New lines

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
Co-authored-by: Roland Stengel <r.stengel@sap.com>
Co-authored-by: Thorsten Duda <thorsten.duda@sap.com>
2022-06-22 13:31:17 +02:00
xgoffin
c35d85fecc
feat(SARIF): ContextRegion is now a pointer, can be omitted. In checkmarxExecuteScan: threadflows now added, only first location saved (#3844) 2022-06-22 08:54:24 +02:00
Mayur Belur Mohan
cdea4b7713
ApiProxyList Command (#3794)
* ApiProxyList Command

* CodeReview Fixes

* CodeReview Fixes

* CodeReview FIxes

* CodeReview Fixes

* CodeReview FIxes

* CodeReview Fixes

* fixing unit test

* doc fixes

* Update documentation/docs/steps/apiProxyList.md

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>

* CodeReview Fixes

* CodeReview Fixes

* CodeReview Fixes

* codereview fix

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
2022-06-21 13:08:00 +02:00
Oliver Feldmann
c5b83de7e1
[ANS] logrus hook (#3671)
* Add ans implementation

* Remove todo comment

* Rename test function

Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>

* Better wording

Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>

* Add reading of response body function

* Use http pkg ReadResponseBody

* Check read error

* Better test case description

* Fix formatting

* Create own package for read response body

* Omit empty nested resource struct

* Separate Resource struct from Event struct

* Merge and unmarshall instead of only unmarshalling

* Improve status code error message

* Remove unchangeable event fields

* Separate event parts

* Change log level setter function

* Restructure ans send test

* Revert exporting readResponseBody function

Instead the code is duplicated in the xsuaa and ans package

* Add check correct ans setup request

* Add set options function for mocking

* Review fixes

* Correct function name

* Use strict unmarshalling

* Validate event

* Move functions

* Add documentation comments

* improve test

* Validate event

* Add logrus hook for ans

* Set defaults on new hook creation

* Fix log level on error

* Don't alter entry log level

* Set severity fatal on 'fatal error' log message

* Ensure that log entries don't affect each other

* Remove unnecessary correlationID

* Use file path instead of event template string

* Improve warning messages

* Add empty log message check

* Allow configuration from file and string

* Add sourceEventId to tags

* Change resourceType to Pipeline

* Use structured config approach

* Use new log level set function

* Check correct setup and return error

* Mock http requests

* Only send log level warning or higher

* Use new function name

* One-liner ifs

* Improve test name

* Fix tests

* Prevent double firing

* Reduce Fire test size

* Add error message to test

* Reduce newANSHook test size

* Further check error

* Rename to defaultEvent in hook struct

* Reduce ifs further

* Fix set error category test

The ansHook Fire test cannot run in parallel, as it would affect the
other tests that use the error category.

* Change function name to SetServiceKey

* Validate event

* Rename to eventTemplate in hook struct

* Move copy to event.go

* Fix function mix

* Remove unnecessary cleanup

* Remove parallel test

The translation fails now and again when parallel is on.

* Remove prefix test

* Remove unused copyEvent function

* Fix ifs

* Add docu comment

* Register ans hook from pkg

* register hook and setup event template seperately

* Exclusively read eventTemplate from environment

* setupEventTemplate tests

* adjust hook levels test

* sync tests- wlill still fail

* migrate TestANSHook_registerANSHook test

* fixes

* review - cleanup, reuse poke

* Apply suggestions from code review

* Change subject

* Review fixes

* Set stepName 'n/a' if not available

* Fix fire tests

Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>
Co-authored-by: Roland Stengel <r.stengel@sap.com>
2022-06-17 16:40:45 +02:00
xgoffin
0457601efd
feat(sarif): add GUID as part of properties, change ruleID (#3838) 2022-06-17 08:53:44 +02:00
xgoffin
c11110d791
feat(sarif): add a "conversion" object to SARIF files (#3837)
* feat(fortifyExecuteScan): add conversion object

* feat(checkmarxExecuteScan): add conversion object
2022-06-16 15:24:23 +02:00
Hilmar Falkenberg
e828a94c85
add FortifyReportData.URL to scanReport (#3821)
hopefully that gives users a direct link back to the original fortify project+version

Co-authored-by: xgoffin <86716549+xgoffin@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-06-15 13:45:09 +02:00
Pavel Busko
4ab32f7e7c feat(cnbBuild): generate buildSettingInfo
Co-authored-by: Sumit Kulhadia <sumit.kulhadia@sap.com>
Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
2022-06-15 12:37:21 +02:00
xgoffin
5edb0d2566
feat(fortifyExecuteScan): implement a system to limit the number of API calls upon request failures (#3818)
* feat(fortifyExecuteScan): add a max number of retries for API calls in SARIF conversion

* feat(checkmarxExecuteScan): implement max number of retries on API call for descriptions in SARIF processing

* feat(checkmarx/fortify): extra logging line when failing an API request in SARIF conversion

* fix(fortifyExecuteScan): panic if undefined projectversion in sarif

* fix(fortifyExecuteScan): logging improvement

* fix(fortifyExecuteScan): wrong if condition caused crash

* fix(fortifyExecuteScan): do not log if retries hit -1, adjust logging

* fix(SARIF): commenting API calls for Checkmarx until a solution can be found for the API issues

* feat(SARIF): add omitempty to extensions
2022-06-09 10:32:08 +02:00
Siarhei Pazdniakou
92837fde18
feat(gradleExecuteBuild, fortifyExecuteScan): gradle improvements (#3807)
* Improvements were made

* fixed tests

* fixed issues

* fix versioning

* fix Inclusive Language warnings

* gradle support to fortifyExecuteScan. Classpath resolving

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-06-07 10:24:10 +02:00
Oliver Feldmann
aecf1babd9
[ANS] Add SAP Alert Notification Service to pkg (#3654)
* Add ans implementation

* Remove todo comment

* Rename test function

Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>

* Better wording

Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>

* Add reading of response body function

* Use http pkg ReadResponseBody

* Check read error

* Better test case description

* Fix formatting

* Create own package for read response body

* Omit empty nested resource struct

* Separate Resource struct from Event struct

* Merge and unmarshall instead of only unmarshalling

* Improve status code error message

* Remove unchangeable event fields

* Separate event parts

* Change log level setter function

* Restructure ans send test

* Revert exporting readResponseBody function

Instead the code is duplicated in the xsuaa and ans package

* Add check correct ans setup request

* Add set options function for mocking

* Review fixes

* Correct function name

* Use strict unmarshalling

* Validate event

* Move functions

* Add documentation comments

* improve test

Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>
Co-authored-by: Roland Stengel <r.stengel@sap.com>
2022-06-03 10:16:14 +02:00
xgoffin
903f273012
feat(checkmarxExecuteScan): added API to get description, incorporated to SARIF file (#3814) 2022-06-01 15:48:56 +02:00
Daniel Mieg
8ce7577a34
Handle empty http response correctly (#3805) 2022-05-25 13:57:13 +02:00
xgoffin
6a43e9f455
feat(fortifyExecuteScan): further improvements to the SARIF generation (#3799)
* feat(fortfiyExecuteScan): proper XML unescaping, added rulepacks to SARIF, added kingdom/type/subtype to tags

* feat(fortifyExecuteScan): proper handling of severity, kinds, levels in SARIF

* fix(fortifyExecuteScan): edge case when handling properties taht could lead to a crash

* fix(fortifyExecuteScan): ensure SARIF processing is done after latest FPR is processed by SSC
2022-05-24 13:40:49 +02:00
Daniel Mieg
cda6e71ab9
Add ABAP step: createTag (#3633)
* Cloud Platform -> BTP

* Initial generation of new step

* add flag

* wip

* Fix warnings

* Add command

* Added Tag Decription

* Add status check

* Improve handling

* Improve handling

* Add test for happy path

* Add test reports to gitignore

* Add second test

* Improve createTag

* Add testcase

* Adaptions

* Add test

* Update cmd/abapEnvironmentCreateTag.go

Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com>

* Update cmd/abapEnvironmentCreateTag.go

Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com>

* Adapt error handling

* Improve coding

* Add info

* Disallow repositories and repositoryName at the
same time

* Regenerate

* Adapt to feedback

* Update cmd/abapEnvironmentCreateTag.go

Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com>

* Update cmd/abapEnvironmentCreateTag.go

Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com>

Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com>
2022-05-23 15:15:22 +02:00
Philipp Stehle
17ed9468cb
replace depreacted strings.Title function (#3786) 2022-05-20 18:50:03 +02:00
thtri
4c2845b79c
fix(fortify):add SSC project identifier to ouput and JSON report (#3787) 2022-05-20 10:24:16 +02:00
Daniel Mieg
a5fcc7517b
Improve logging (#3788) 2022-05-19 16:59:37 +02:00
xgoffin
1fde2ce677
feat(checkmarxExecuteScan): improvements to SARIF file generation (#3781)
* feat(checkmarxExecuteScan): respect SARIF standard more closely

* fix(checkmarxExecuteScan): edge case where message would be empty in SARIF

* fix(checkmarxExecuteScan): better message handling to ensure field is populated

* feat(checkmarxExecuteScan): SARIF file readability

* feat(checkmarxExecuteScan): include the helpURL as part of the Help object

* fix(sarif): remove wrong structure addition

* feat(checkmarxExecuteScan): safer handling of version in SARIF file

* feat(checkmarxExecuteScan): add CWE number to tags
2022-05-19 14:57:13 +02:00
Adrien
3d48364862
Fix project config reset when preset is set (#3782) 2022-05-18 17:10:00 +02:00
Oliver Nocon
6070549704
fix(helmExecute): respect version from Chart (#3780)
* fix(helmExecute): respect version from Chart

using version from CPE can create failure situations in case format is not semver.
This is the case for maven artifacts, for example.

* chore: simplify condition

* chore: cleanup

* chore: cleanup
2022-05-18 08:58:01 +02:00
Mayur Belur Mohan
ef2db4c2b0
ApiProviderUpload Command (#3663)
* ApiProviderUpload Command

* formatting fix

* formatting fix

* formatting fix

* formatting fix

* CodeReview Changes

* CodeReview Fix

* CodeReview fix

* CodeReview Fix

* CodeReview FIx

* CodeReview FIxes

* CodeReview Fixes

* CodeReview Fixes

* CodeReview Fixes

* CodeReview Fix

* CodeReview Fix

* CodeReview Fixes

* CodeReview Fix

* Doc Fixes

* CodeReview Fix

* CodeReview Fixes

* Doc Fixes

* Update cmd/apiProviderUpload_test.go

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>

* CodeReview FIxes

* Update cmd/apiProviderUpload.go

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>

* CodeReview Fixes

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
2022-05-16 11:52:04 +02:00
Anil Keshav
e6724d7f05
fix (whitesourceExecuteScan) keep tar extension for target image donwload (#3774)
* explicitly adding tar extension to project name when constructing the targetFilePath for whitesource docker image download

* comments

* correcting comment for better readability

* replace spaces in the project name with underscroe

* better comments

* passing legacy format download

* appending format to value

* keeping the download format for protecode as legacy

* improving docu

* keeping legacy format the default

* keeping tar file name same as project name to avoid duplicate names

* keeping legacy format download hard coded

Co-authored-by: anilkeshav27 <you@example.com>
2022-05-13 18:56:41 +02:00
Anil Keshav
b7cc1eb62f
parsing the entire url and not spliting it (#3775)
Co-authored-by: anilkeshav27 <you@example.com>
2022-05-13 10:02:46 +02:00
xgoffin
7d9f018529
feat(fortifyExecuteScan): SARIF generation improvements (#3769)
* feat(fpr_to_sarif & GHAS): adjustments to fit some rules

* feat(fortifyExecuteScan): fit GH ingestion rules better

* feat(fortifyExecuteScan): readability in SARIF report

* feat(fortifyExecuteScan): restore escaped chars in XML text

* feat(fortifyExecuteScan): properly replace threadflowlocations in each threadflow

* fix(fortifyExecuteScan): fixed missing threadflow in SARIF generation

* feat(fortifyExecuteScan): properly handle threadflows when a node has another node as Reason (node-in-node edge case)

* feat(fortifyExecuteScan): better sarif ruleID field

Co-authored-by: thtri <trinhthanhhai@gmail.com>
2022-05-11 17:05:51 +02:00
Ashly Mathew
823ef47db6
Golang build publish binaries and add group id (#3749)
* Add golang binaries to CPE and add groupID in coordinates

Signed-off-by: Ashly Mathew <ashly.mathew@sap.com>

* Introducing binaries as artifacts

Signed-off-by: Ashly Mathew <ashly.mathew@sap.com>
2022-05-05 12:39:59 +02:00
Giridhar Shenoy
fe9b3a796b
detectExecuteScan: fix project version limiting api call (#3754)
* fix project version limiting issue

* add tests for detectExecute
2022-05-05 09:58:43 +02:00
Anil Keshav
7ad96ee686
feat (cpe) introduce a new artifact cpe type (#3763)
* including a artifact cpe type

* removing type kind related to PR 3717

* clean up

* eliminating local path

* go formatting fix

Co-authored-by: anilkeshav27 <you@example.com>
2022-05-03 09:53:12 +02:00
Daniel Bernd
c65e569cdf
add OSL Package Tree on ObjectSet level (#3762)
Co-authored-by: Daniel Bernd <93763187+danManSAP@users.noreply.github.com>
2022-04-29 16:50:57 +02:00
Daniel Bernd
a3f1234a60
Feature: ATC & AUNIT - ObjectSetLibrary(OSL) usage (#3755)
* own OSL go File & first changes ATC step

* OSL Integration in AUnit & ATC steps 1

* OSL & Unit Tests

* Unit Tests 1

* OSL - AUnit & ATC usage

* Unittest ATC: packagetree --> package incl. subpackages

* correct spelling

* yaml & generated update

* generated Metadata

Co-authored-by: Daniel Bernd <93763187+danManSAP@users.noreply.github.com>
2022-04-27 13:30:43 +02:00
Vitalii Sidorov
9d8f3fc6ad
fix runHelmAdd (#3747)
* fix runHelmAdd

* Add tests

Co-authored-by: “Vitalii <“vitalii.sidorov@sap.com”>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-04-26 17:50:37 +02:00
tiloKo
5a252392f2
enableBuildWithoutValues (#3753) 2022-04-26 17:14:55 +02:00
xgoffin
0696db5e0d
feat(sarif): logging improvements (#3727)
* fix(fortifyExecuteScan): check audit data length in all cases

* fix(fortifyExecuteScan): check audit data length in all cases

* feat(SARIF): logging improvements in debug mode

* fix(logging): readability

Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2022-04-26 12:34:54 +02:00
Giridhar Shenoy
2ed1ed76fc
whitesourceExecuteScan : add log level trace (#3745)
* add log level trace

* add log level files trace

* add tests
2022-04-22 13:19:49 +02:00
Jordi van Liempt
eb73cae888
feat(getDefaults): add parameter for CRD-style config (#3736)
* export LoadConditionsV1() in config pkg

* getDefaults: read stage conditions as PipelineDefinitionV1

* add getDefaults useV1 tests

* add comment (pointed out by code climate analysis)

* small addition to comment (from previous commit)

Co-authored-by: I557621 <jordi.van.liempt@sap.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2022-04-19 12:34:40 +02:00
Vitalii Sidorov
63cdfc0e68
Fix helm execute command (#3701)
* Add small fix

* fix unit-tests

* Add deploymentName and packageVersion as flags

* small fix

* Change getting version of helm chart

* small fix

Co-authored-by: “Vitalii <“vitalii.sidorov@sap.com”>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-04-14 15:43:47 +02:00
Siarhei Pazdniakou
d519966fe5
feat(gradleExecuteBuild): publish (upload to staging) (#3662)
* gradle publish

* Fix gradle utils

* Added unit test

* Add descriptions for parameters

* Added unit tests

* fix publish init script template

* fix artifactId and groupId

* fix project path

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-04-14 14:59:09 +02:00
Anil Keshav
3c08a3bd7a
including container names when no conditions (#3733)
Co-authored-by: anilkeshav27 <you@example.com>
2022-04-13 17:34:42 +02:00
rosemarieB
9fdaf86856
Enable additional piecelist for assembly (#3730)
* add useClassicCTS

* add option additionalPiecelist

* Update abapEnvironmentAssemblePackages.go
2022-04-12 16:00:17 +02:00
rosemarieB
1ffe22903e
Enable classic cts (#3728)
* add useClassicCTS

* debug output and unit test

* unsaved fix

* error logging

* Update abapEnvironmentAssemblePackages.go

* Update abapEnvironmentAssemblePackages.go

* add general scope and remove logging

* Update bfw.go

* Update abapEnvironmentAssemblePackages.go

* Update abapEnvironmentAssemblePackages.go

* Update package.go

* Update abapEnvironmentAssemblePackages.go

* change scope and add certificates

Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com>
2022-04-12 14:39:42 +02:00
Michael
6676da1f9d
feat(gitopsUpdateDeployment) forcePush (#3665)
* feat(gitopsUpdateDeployment) forcePush

fix(gitopsUpdateDeployment) include registry

The push operation in this step can be forced to bypass branch-protection

Signed-off-by: Michael Sprauer <Michael.Sprauer@sap.com>

* add unit test

Signed-off-by: Michael Sprauer <Michael.Sprauer@sap.com>

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-04-07 16:33:46 +02:00
xgoffin
fb9792ad71
feat(fortifyExecuteScan): optimization of the SARIF conversion code (#3710)
* feat(fortifyExecuteScan): query SSC once for batch audit data

* fix(fortifyExecuteScan): check audit data length in all cases

* feat(fortifyExecuteScan): in fpr_to_sarif, better detection of error cases, unit tests

* fix(log): comment useless error message

* fix(fortifyExecuteScan): clarify log message

* fix(fortifyExecuteScan): adapt unit tests
2022-04-07 13:11:52 +02:00
Julian Schmitt
6151f36d88
Implement Release 3.0.0 of CM-Client (#3664)
* Remove --backend-type

* Delete CTS in isChangeDevelopment and change Dockerimage of CM-Client

* fix groovy unit tests

* another fix of groovy unit tests

* try to fix import of fork for Jenkins-Testing

* add workflow to create Go Binary for Jenkins-Server

* Change RepoOwner to test in Fork

* remove previous changes

* adjust docker image for TransportRequestCreate and Release

* Remove CTS from Documentation

Co-authored-by: Thorsten Duda <thorsten.duda@sap.com>
2022-04-05 12:33:19 +02:00
xgoffin
3c55d3c99c
feat(checkmarxExecuteScan): convert Checkmarx xml report to SARIF (#3696)
* feat(checkmarxExecuteScan): sarif conversion for Checkmarx XML reports

* feat(checkmarxExecuteScan): added taxonomies and similarityID

* fix(checkmarxExecuteScan): proper handling of ruleId and ruleIndex

* fix(sarif): mistype in checkmarx properties

* fix(checkmarxExecuteScan): fixed occasional panics when handling audit comment

* chore(sarif): proper variable naming

* chore(code): fix missing and unrecognized comments

* trigger PR

* fix(format): extra space

Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2022-04-04 16:12:35 +02:00
Oliver Nocon
62b3a9a459
fix(versioning): properly consider go.mod (#3687)
* fix(versioning): properly consider go.mod

* fix test
2022-03-31 17:07:10 +02:00
Ralf Pannemans
5b42b6af70
feat(cnbBuild): Support of username/password authorization (#3690)
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
2022-03-31 16:13:21 +02:00
Ralf Pannemans
0c28ecc4fb
debug(cnbbuild): Add some debug messages (#3688)
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>
2022-03-31 15:02:06 +02:00
xgoffin
dc91332e29
fix(fortifyExecuteScan): occasional panics when snippets are undefined or contain XML comments (#3686) 2022-03-31 12:13:17 +02:00
Pavel Busko
1f750af16d
feat(cnbBuild): cache buildpacks during multi-image build (#3635)
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
2022-03-30 13:58:16 +02:00
Anil Keshav
e8fc08e43b
fix (untar) correct condition for gunzip file (#3679)
* debug if file is gzipped

* looking 3 byte value

* debug for gzip file

* reading decimal values instead of hexadecimal

* looking for byte values

* modifying return

* simplyfying conditions

* correcting err condition

* trying with hex

* using hex

Co-authored-by: anilkeshav27 <you@example.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-03-30 12:30:57 +02:00
Vitalii Sidorov
d62c3d73a0
Add helm dependency command (#3669)
* Add helm dependency command

* Change name of flag for package command

Co-authored-by: “Vitalii <“vitalii.sidorov@sap.com”>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-03-30 08:18:51 +02:00
ffeldmann
a9329fba9e
Reuse SOURCEBRANCH for branch name (#3672) 2022-03-28 13:48:55 +02:00
ffeldmann
ccc1c976ee
fix(orchestrator) usage of correct env variables (#3650)
* Reorders getApiInformation, changes variables to get start time, adjusts and adds test cases
* Changes the way to get apiInformation and reduces number of requests
* Changes getting pipeline start time from correct env variable
* Refactors getApiInformation functionality
* Adds GetBuildReason() for Azure and Jenkins
* Updates JobURL for ADO
2022-03-28 09:52:15 +02:00
Christian Volk
22f6aa156f
feat(docker): use crane for pulling docker images (#3652) 2022-03-23 10:02:00 +01:00
Sven Merk
f06890a9b2
SARIF format and GHIssue format improvements (#3646)
* Improve reporting

* Fix location

* Align casing

* Fix severity mapping

* Fix format

* Improve title

* Title format

* Fix severity

* Align title

* Fix schema reference

* Fix schema reference

* Fix fmt

* Fix fmt2

* Fix tests

* fix(sarif): proper handling of omitempty in SnippetSarif

* fix(fortifyExecuteScan): sarif format version

* Addressing comments

* Fix SARIF

* fix(sarif): omitempty handling

* fix(fortifyExecuteScan): pointer indirection

* Added TODOs for audit data

Co-authored-by: Xavier Goffin <x.goffin@sap.com>
Co-authored-by: xgoffin <86716549+xgoffin@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-03-22 14:47:19 +01:00
Siarhei Pazdniakou
db5360fb89
feat(gradleExecuteBuild): BOM creation. Integration tests (#3603)
* Implemented bom creation

* Made small fixes. Added integration tests

* go generate

* minor fixes

* fix tests

* Added unit tests

* minor fixes

* use fileutils

* integration tests optimization

* change integraton tests timeout to 25m

* Fix Inclusive Language warnings

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-03-21 10:17:03 +01:00
Vitalii Sidorov
2a4052d13c
feat(helmExecute): run complete lint, build, publish cycle (#3546)
* Add runHelmCommand

* Add dryRun for debug

* Add default case in helmExecute

* Fix unit-tests

* small fix

* Fix RunHelmAdd and change RunHelmPublish methods

* Fix RunHelmPublish

* Fix unit-tests

* Fix unit-test

* small fix

* small fix

* small fix

* Add LintFlag PackageFlag PublishFlag flags

* Add tests for httpClient.go

* test

* test

* smal fix

* small fix

* Add getting name and version from Chart.yaml

* Add test

* Fix

* small fix

* Fix according to comments

* small fix

Co-authored-by: “Vitalii <“vitalii.sidorov@sap.com”>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Vitalii Sidorov <vitalii_sidorov@sap.com>
2022-03-17 17:13:34 +01:00
Sven Merk
c30e93bc6a
feat(detectExecuteScan): SARIF export and GH issue creation (#3637)
* Added SARIF and GH issue creation
2022-03-17 15:32:48 +01:00
xgoffin
3f6e4b9e3b
feat(fortifyExecuteScan): added parameter to generated sarif file (#3644)
* fix(sarif): change format to fit omitempty cases better

* feat(fortifyExecuteScan): include category in sarif file

* fix(fortifyExecuteScan): access to undefined pointer in some cases

Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2022-03-17 13:09:15 +01:00
Anil Keshav
818636b048
feat (checkIfStepActive) including cpe conditions to check if a step is active / not active (#3642)
* including cpe struct

* evaluating types

* adding environment root path

* unit tests

Co-authored-by: anilkeshav27 <you@example.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-03-17 12:09:16 +01:00
Christian Volk
58b30cc99b
fix(golangBuild): publishing of binaries (#3643) 2022-03-17 11:07:51 +01:00
Maurice Breit
9f2d8a70b1
(fix) match regexes in sliceContains to support vaultSecretNames (#3618)
* (fix) match regexes in sliceContains to support vaultSecretNames

* add test for regex matching in sliceContains

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-03-17 09:41:51 +01:00
Oliver Nocon
504f076613
feat: support cpe credentials for multiple repos (#3641) 2022-03-17 08:01:00 +01:00
Oliver Feldmann
4208bc050c
Add bearer token retrieval function (#3595)
* Add bearer token retrieval function

Retrieving a bearer token from the xsuaa service on BTP is always the
same. With these functions one can retrieve a bearer token and set it
to the given header as 'Authorization'.

* CodeClimate fixes

* Refactor test

* Add basic auth to token retrieve request

Co-authored-by: Thorsten Duda <thorsten.duda@sap.com>
2022-03-16 11:22:48 +01:00
Oliver Nocon
7ec512cb9f
feat(artifactPrepareVersion): helm & propagate version (#3627)
* feat(artifactPrepareVersion): helm & propagate version

* chore: small refactoring

* chore: fix linting issue

* fix version persistence
2022-03-15 09:08:24 +01:00
xgoffin
dfd2278639
feat(fortifyExecuteScan): full FPR to SARIF implementation (#3604)
* feat(FPRtoSARIF): boilerplate & comments

* Feat(Ingest): Build done, Vulnerabilities partway

* feat(Vulnerabilities): now entirely parsed

* feat(FprToSarif): integration in Piper step, full xml structure

* feat(fpr_to_sarif): base program. Need to replace names in messages

* feat(fpr_to_sarif): message substitution and custom definition integration

* fix(fpr_to_sarif): missing replacement in tools object

* fix(fpr_to_sarif): failing unit test

* Fix fortify folder creation for generating sarif

* deletion of unzip folder

* feat(fpr_to_sarif): better unit test

* fix(fpr_to_sarif): pr tests failing

* feat(fortifyExecuteScan): complete SARIF file generation

* fix(fpr_to_sarif): add extra check and test to prevent panics

* rebase onto master, fix ALL conflicts, adapt code and format

* fix missing added properties

* fix(SARIF): structure

* fix(whitesource): wrong sarif structures

* Update pkg/fortify/fpr_to_sarif.go

* Update pkg/format/sarif.go

* Update pkg/format/sarif.go

Co-authored-by: Sumeet PATIL <sumeet.patil@sap.com>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2022-03-14 11:26:05 +01:00
ffeldmann
b224f2294c
Activates debug information for environment variables (#3630)
* Activates debug information for environment variables

* Adds tests for environment variable reading

* Reduces batch size to send messages to Splunk to 5000
2022-03-14 10:17:55 +01:00
Oliver Nocon
2a56723d40
feat(kanikoExecute): Trim names for multi-image builds (#3617)
* feat(kanikoExecute): Trim names for multi-image builds

* chore: fix yaml linting issue
2022-03-11 09:47:44 +01:00
Oliver Nocon
2c837927d4
chore(whitesourceExecuteScan): Gradle config changes (#3621)
* chore(whitesourceExecuteScan): Gradle config changes

supersedes #3293
closes #3293

* update config

Co-authored-by: ffeldmann <f.feldmann@sap.com>
2022-03-11 08:18:21 +01:00
Anil Keshav
157f6d4e35
feat (checkIfStepActive) enhance to include not active conditions (#3616)
* including negative conditions

* clean up and todos

* removing debug logging

* clean up

* fix unit test name

* fixing unit tests

* negative stage test

Co-authored-by: anilkeshav27 <you@example.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-03-10 13:35:28 +01:00
Christian Volk
b9c0aa008f
feat(piperExecuteBin): support running images from private docker registries (#3622) 2022-03-09 17:35:57 +01:00
Johannes Dillmann
3708f274cc
feat(cnbBuild) read target image name from github cpe (#3620)
Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
2022-03-09 14:06:26 +01:00
maheshsrikrishnan
64a00c540a
New piper step APIProviderDownload contribution (#3349)
* added store file function in cpi common utils

Change-Id: Ia429a2792266e082d139025a71799d21c30a7df9

* Added api provider steps

Change-Id: Icd2829a91db4c53d0de2330822d2b33933973868

* Update apiProviderDownload.yaml

* fixed yaml JLINT issue

Change-Id: Iac974abc30fa00e68c0177072b93716b0af5e0c5

* Removed trailing spaces

Change-Id: I927e9314fce6e9cab68d6b97577c7c96bb2bddad

* Resolved common steps groovy script conflict

Change-Id: I3ad144b618e1c77953aaeccaa5bf7309aff77ca9

* Change for conflict resolution

Change-Id: Ic955833eca844f090b7983f99f9d3649ebb981c7

* Fixed method name and its corresponding implementation

Change-Id: I465c1f1d5306bb978386de9efca3c521e385b89c

* Moved re-usable function to commonUtils package

Change-Id: Ide06462b01caeb2bf438ad7661e01c15bf8e8e24

* Changed the implementation to use existing writeFile method

* Fixed review comments on documentation and test structuring

Change-Id: Ifebd2f4b50754b2097b2d564fb3cc37c433ef6c9

* Fixed documentation alignment issues

* Fixed spaces issue

Change-Id: I834bd94e01bce72e7f81ab49ba32671c91c66ca9

* Documentation removed extra spaces

Change-Id: I9a639d76ed9b81c870f18349504044bb70753b52

* Fixed doc build issue

Change-Id: I96c3e15e73834b64f8b8e3432ce59f6b037f93fd

* Fixed documentation build issues

Change-Id: I7fca2ba69bc7b7298ee300ccd1ae16a6238dc96b

* Re-generated code for fixing build failure

Change-Id: I22b7ee6162f643d9f3b60f6a33eb7858927182a0

* Adopted file utils & mock

Change-Id: Ic46462003527f41df64395a5a615c19bf374e8ef

* Removed ioutil call in the test & adopted error variable names

* Removed commented lines

Change-Id: I99a12e39bc04323e9c19f1409d97eeca267e6fdb

* Added test for asserting file download and adopted error variables

Change-Id: I49463a3b75987bf68f5261d45602d2d7bd960a05

* Added download path assertion positive & negative case

Change-Id: Ieee461c3973b9dfa8f395dc936e4241ff9694c7b

* Modified tests with DownloadPath variable

Change-Id: Iaf14c9ea1a8242b6c8d8e9e4fac8c23d9c1b3a74

* Added testcase to validate file content

Change-Id: I21aed481b433450c3b536dbb29d45291f61848d8

* Refactored test for file content check to avoid failures

Change-Id: I3b4fe9a0de678f437fd4cc0a8203ae9434d9fa8e

* Removed auto-generated comments

Change-Id: I86c4ac3e7e4476a75d6cbed58826ec1f3278d7d2

* Fixed documentation review comments

Change-Id: I4faf31473b53fc53a5517d418c343bf7320eec55

* Fixed documentation indentation

Change-Id: I386f046cf4e10ee6deb5a81fcfc8c430c97086c8

* Fix build issue

Change-Id: I61a829cabaf03ffd5e77cddc594486a650118fa3
2022-03-09 13:07:23 +01:00
ffeldmann
3e89fb0cd7
fixes retry when sending telemetry to Splunk (#3615)
* Do not retry sending data to Splunk HEC
* Adds error handling
2022-03-08 13:26:00 +01:00
Oliver Nocon
0998a80f07
chore: update to go 1.17 (#3557)
* chore: update to go 1.17

Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>

* update workflows

Co-authored-by: Pavel Busko <pavel.busko@sap.com>

* fixed multipart upload tests

Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>

* go fmt

Co-authored-by: Pavel Busko <pavel.busko@sap.com>

* Update to go 1.17

Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>

* Update vault to v1.9.3

Co-authored-by: Pavel Busko <pavel.busko@sap.com>

Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
Co-authored-by: Pavel Busko <pavel.busko@sap.com>
Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
Co-authored-by: ffeldmann <f.feldmann@sap.com>
2022-03-07 17:15:36 +01:00
Mayur Belur Mohan
9a3b800b9d
ApiKeyValueMapUpload Command (#3407)
* ApiKeyValueMapUpload Command

* include command in metadata

* TestCase Fixes

* CodeReview Fixes

* CodeReview Fixes

* Code Review Fixes

* CodeReview Fixes

* CodeReview Fixes

* CodeReview FIxes

* CodeReview Fixes

* Documenation change

* documentation fix

* Documentation Fix

* Documentation Fix

* documentation Fix

* CodeReview Fixes

* CodeReview Fixes

* Revert changes

* Documentation Fix

* CodeReview FIxes

* Doc Fixes

* Code Review Fixes

* Code Review Fixes

* CodeReview FIxes

* Documentation Fix

* Documentation Changes

* Documentation Fix

* codereview fix

* Documentation Fix

* CodeReview Fixes

* CodeReview Fix

* Documentation FIx

* doc fix

* Doc Fix

* Documentation Fix

* codereview fix

* revert fix

* Code Review Fix

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
2022-03-07 11:03:44 +01:00
ffeldmann
6398e61995
Splunk reporting; Sending messages in batches (#3611)
* Refactors logfile sending logic, renaming of fields, adds proper piper sourcetype
* Sets maximum retries to three and transport timeout to 10 seconds for azure and jenkins
2022-03-07 09:55:12 +01:00
Christian Volk
5821a311cc
chore(npmExecuteScripts): config@2 support (#3607) 2022-03-04 10:26:46 +01:00
Pavel Busko
0de06c6207
feat(cnbBuild): write image digests to the CPE (#3602)
Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
2022-03-02 16:26:45 +01:00
Christian Volk
e49820f5e4
chore(npmExecuteScripts): unit tests (#3597) 2022-03-02 14:06:51 +01:00
Adrien
a73951909b
checkmarxExecuteScan fixes (#3540)
* Fix FilterByTeamName and LoadExistingProject

* Fix project name loop

Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2022-02-28 14:22:47 +01:00
Eugene Kortelyov
8ced7f8184
Feature/fortify execute scan gradle (#3582)
* initial fortify gradle commit

* initial fortify gradle commit

Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-02-28 11:35:38 +01:00
ffeldmann
5f4cd838cf
Updates telemetry logging information for internal reporting (#3585)
* Add StepStartTime, Renames StepDuration, adds PiperCommithash, removes Branch, GitOwner, GitRepository from logged telemetry information

* Fixes test case for telemetry logging

* Activates step monitoring data in debug mode

* Pretty debug json printing

* Reduces log noise, setting warning to debug
2022-02-28 09:45:57 +01:00
Eugene Kortelyov
af7496d012
feat: url log parsing / reporting when executing a command (#3461)
* Initial PR for url reporting

* Rename URLReport var to URLReportFileName

* Remove URLReportFileName from piper flags

* Update pkg/command/command.go

* Update pkg/command/command.go

* Update pkg/command/command.go

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-02-25 15:35:44 +01:00
thtrinh
d86cfce6e6
Checkmarx json report (#3565)
* feat(checkmarx) : Checkmarx JSON Report

* Test cases with some fix

* Information total and audited test assertions

* feat(checkmarx): align total/audited with existing calculation

* fix(checkmarx): Reporting unit test

Co-authored-by: Sumeet PATIL <sumeet.patil@sap.com>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-02-25 14:20:36 +01:00
Ralf Pannemans
08bfe1554e
feat(cnbBuild) Introducing preserveFiles config to copy back files (#3562)
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
2022-02-23 18:54:59 +01:00
Philipp Stehle
4c18f2a128
feat: Add support for deprecating step parameters (#3554)
this was already used in fortifyExecuteScan, but had no effect.

Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>

Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
2022-02-23 15:16:05 +01:00
Christian Volk
d428cfcbd1
feat(kanikoExecute): image digests (#3566)
* feat(init stage): artifact version

* feat(kaniko): expose image digests to cpe

* fix

* handle tmp folder creation differently

* fix

* fix

* fix tests

* set ignore-path to /
2022-02-23 11:41:26 +01:00
Sven Merk
a1988f6808
feat(whitesourceExecuteScan): GitHub issue creation + SARIF (#3535)
* Add GH issue creation + SARIF

* Code cleanup

* Fix fmt, add debug

* Code enhancements

* Fix

* Added debug info

* Rework UA log scan

* Fix code

* read UA version

* Fix nil reference

* Extraction

* Credentials

* Issue creation

* Error handling

* Fix issue creation

* query escape

* Query escape 2

* Revert

* Test avoid update

* HTTP client

* Add support for custom TLS certs

* Fix code

* Fix code 2

* Fix code 3

* Disable cert check

* Fix auth

* Remove implicit trust

* Skip verification

* Fix

* Fix client

* Fix HTTP auth

* Fix trusted certs

* Trim version

* Code

* Add token

* Added token handling to client

* Fix token

* Cleanup

* Fix token

* Token rework

* Fix code

* Kick out oauth client

* Kick out oauth client

* Transport wrapping

* Token

* Simplification

* Refactor

* Variation

* Check

* Fix

* Debug

* Switch client

* Variation

* Debug

* Switch to cert check

* Add debug

* Parse self

* Cleanup

* Update resources/metadata/whitesourceExecuteScan.yaml

* Add debug

* Expose subjects

* Patch

* Debug

* Debug2

* Debug3

* Fix logging response body

* Cleanup

* Cleanup

* Fix request body logging

* Cleanup import

* Fix import cycle

* Cleanup

* Fix fmt

* Fix NopCloser reference

* Regenerate

* Reintroduce

* Fix test

* Fix tests

* Correction

* Fix error

* Code fix

* Fix tests

* Add tests

* Fix code climate issues

* Code climate

* Code climate again

* Code climate again

* Fix fmt

* Fix fmt 2

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-02-23 09:30:19 +01:00
ffeldmann
ffd4a7efb8
(fix) nil pointer dereference in case credentials are not available (#3564)
* adds return in gcs upload in case error occurs e.g. no credentials, avoid nil pointer dereference

* Adds generated files

* Updates generated files

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-02-22 18:32:09 +01:00
Oliver Nocon
6247c5dddd
feat(buildsettings): golang and future tools (#3561)
Co-authored-by: Christian Volk <christian.volk@sap.com>
2022-02-21 09:16:48 +01:00
Christian Volk
b2246a021f
chore: noise reduction (#3558)
* fix(telemitry): noise reduction

* partially revert
2022-02-18 11:54:39 +01:00
ffeldmann
d7a13aa5d0
Fixes telemetry logging not reporting an error, adds tests (#3551)
* fixes error logging in case fatalError is empty, adds inital test cases for logStepTelemetryData

* Tests logging output
2022-02-17 15:39:01 +01:00
Oliver Nocon
a4a0873081
feat(checkmarx): create GitHub issue with findings (#3543)
* feat(checkmarx): create GitHub issue with findings

* add github issue reporting
2022-02-17 15:16:55 +01:00
Christian Volk
a1cd6cdb3a
fix(kaniko): set cpe consistently (#3545) 2022-02-17 08:20:30 +01:00
Ralf Pannemans
e7db09db12
feat(cnbBuild): added dockerimage to the telemetry data (#3501)
Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>
Co-authored-by: Sumit Kulhadia <sumit.kulhadia@sap.com>
Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
2022-02-16 13:28:51 +01:00
Ralf Pannemans
4b2f61589d
feat(cnbbuild) enable multi image build (#3521)
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
Co-authored-by: Pavel Busko <pavel.busko@sap.com>
Co-authored-by: Sumit Kulhadia <sumit.kulhadia@sap.com>
2022-02-15 14:39:14 +01:00
ffeldmann
19a05a9c70
Reporting: Variable change to deprecate step Splunk Hook; logging in telemetry (#3539)
* Adds fatalError variable with helper setter and getter helper functions
* Adds logging function to telemetry package (always executed)
2022-02-14 08:56:11 +01:00
Daniel Mieg
f08ff92171
New log entities for Pull & Clone (#3517)
* WIP

* New Logs

* Improving

* Determine log output based on available entities

* Increase width

* Add line

* Adapt TestPollEntity

* Format

* Fix query

* Adapt tests

* Fix test

* Improve formatting

* Retern early in case of no logs

* Remove duplicate log
2022-02-11 10:16:40 +01:00
Christian Volk
c888e21e6c
chore(multiarch): helper to parse targetArchitectures (#3525)
* chore(docker): helper to parse targetArchitectures

* missing files
2022-02-10 16:46:00 +01:00
Vitalii Sidorov
93e3801945
feat: add step for Helm execution (#3419)
* Implement helm step

* Create kubernetes package

* Refactoring helm.go

* Add package, test commands

* Add test for helm package

* Add tests for helm.go

* Add tests for helm.go

* Add tests for utils.go

* Add tests for helmExecute.go

* small fix

* Add helm lint

* small fix

* small fix

* Fix according to comments

* Fix test

* small fix

* Add helm add function

* Changes according to new comments

* Add helm push

* Add unit tests

* Add tests for helmExecute

* Add small fix

* small fix

* small fix

* Move DeployUtilsBundle from kubernetesDeploy to kubernetes package

* small fix

* small fix

* Add unit-tests

* Fix

* Update resources/metadata/helmExecute.yaml

* Update resources/metadata/helmExecute.yaml

* Add helm chart server parameterization

* small fix

* small fix

Co-authored-by: “Vitalii <“vitalii.sidorov@sap.com”>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-02-10 10:25:03 +01:00
Christian Volk
5aa9661db6
feat(kanikoExecute): preparation for multi-arch builds (#3515)
* feat(kanikoExecute): preparation for multi-arch builds

* missing files
2022-02-09 10:17:34 +01:00
xgoffin
2cebf370c9
feat(fortifyExecuteScan): added conversion to SARIF for FPR files (#3485)
* feat(FPRtoSARIF): boilerplate & comments

* Feat(Ingest): Build done, Vulnerabilities partway

* feat(Vulnerabilities): now entirely parsed

* feat(Ingestion): handle Description object

* feat(FprToSarif): integration in Piper step, full xml structure

* feat(fpr_to_sarif): base program. Need to replace names in messages

* feat(fpr_to_sarif): message substitution and custom definition integration

* fix(fpr_to_sarif): missing replacement in tools object

* fix(fortifyExecuteScan): unit tests

* fix(fpr_to_sarif): failing unit test

* Fix fortify folder creation for generating sarif

* deletion of unzip folder

* fix(fortifyExecuteScan): change logging to info

* feat(fpr_to_sarif): better unit test

* fix(fpr_to_sarif): pr tests failing

* feat(fpr_to_sarif): add specific properties to sarif

* feat(fpr_to_sarif): severity integration

* fix(fpr_to_sarif): unit test fixed

Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Sumeet PATIL <sumeet.patil@sap.com>
2022-02-08 14:10:40 +01:00
Giridhar Shenoy
286ff1b6ef
detectExecuteScan : support for detect 7 (#3453)
* enable detect 7 script

* unit test uses detect7 as default

* add detect6 test case

* add check for OSEnv detect version

* add test for OSEnv detect version

* update customEnvironmentVariables docu

* fix linting

Co-authored-by: ffeldmann <f.feldmann@sap.com>
2022-02-08 11:55:01 +01:00
rosemarieB
d1eb87363f
Generic step input from addon descriptor (#3467)
* enhance build step to run based on addon.yml in steampunk environment

Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com>
2022-02-07 20:34:11 +01:00
Matthias Scudlik
907dcd7dc1
fix(sonarExecuteScan): error handling for report metrics (#3512) 2022-02-07 16:41:36 +01:00
ManjunathMS35
8108bb8f6f
Update npmExecuteScripts step (#3211)
* Update npmExecuteScripts step

* Fixed failing build

* Fixed path issue

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Christian Volk <christian.volk@sap.com>
2022-02-07 14:46:03 +01:00
ffeldmann
f3566ab7e0
Changes GetJobName value to repositoryname (#3509) 2022-02-07 08:52:12 +01:00
Oliver Nocon
2ae1d9dac1
feat(kanikoExecute): allow building multiple images (#3443)
* feat(kanikoExecute): allow building multiple images

* enhance tests

* chore: allow running tests in parallel

* small fixes

* fix: fix destination bug

* update formatting and defaults

* fix yml formatting

* chore: change cpe parameter names

* chore: improve variable naming
2022-02-07 07:58:41 +01:00
Christian Volk
54ca265860
fix(httpclient): folder permissions (#3506) 2022-02-04 14:40:18 +01:00
Matthias Scudlik
5eb996c43c
sonarqube coverage: additional metrics (#3465)
* sonarqube coverage: additional metrics

* sonarExecuteScan: add lines of code and language distribution to sonarscan.json

* sonarExecuteScan: consider branch in componentService requests

* SonarQube: Do not omit empty values in SonarCoverage

* sonarExecuteScan: Add integration tests for ComponentService getLinesOfCode

* fix tests

* sonarExecuteScan: use pullRequest in componentService

Co-authored-by: I550025 <r.kloe@sap.com>
Co-authored-by: Marc Bormeth <marc.bormeth@sap.com>
2022-02-04 09:52:29 +01:00
kulhadia
6feb1217aa
Feat(cnbBuild): collecting the telemetry data for cnbBuild (#3368)
Co-authored-by: Sumit Kulhadia <sumit.kulhadia@sap.com>
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>
Co-authored-by: Pavel Busko <pavel.busko@sap.com>
2022-01-31 12:27:29 +01:00
Christian Volk
880be73a4c
feat(golangBuild): support private modules (#3471)
* feat(golangBuild): support private module repositories
2022-01-31 10:45:40 +01:00
Anil Keshav
1a96d7a67e
fix (npmExecuteScripts) allow npm pack before npm publish (#3455)
* adding config to piperNpmr

* scope in cli

* adding scope to repo url and npmrc

* publish to scoped

* removing scope

* changing scope position

* adding scope to userconfig

* adding registry=

* pack and then tar

* not removing tmp folder

* adding flag

* pack before publish

* adding log

* debug

* debug with change directory

* publishing created tar ball

* debug

* üath

* adding main npmrc

* renaming old npmrc file

* error renaming old npmrc file

* renaming err

* correcting npmrc file path

* renaming file back to original

* current working directory

* renaming the npmrc file

* avoiding change directory

* with current working dir

* adding dot

* renaming npmrc and defer removal

* rename files

* Update pkg/npm/publish.go

* Update pkg/npm/publish.go

Co-authored-by: anilkeshav27 <you@example.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-01-25 09:52:22 +01:00
Eugene Kortelyov
da450c5243
feat(gradleExecuteBuild): add stage scope to path parameter, fix project dir exist issue (#3401)
* add stage scope to path parameter, fix project dir exist issue

* fix unit test for gradleExecuteBuild

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-01-24 15:08:39 +01:00
Christopher Fenner
ebcbb73b46
feat: add command to fetch default files (#3445)
* some ideas..

* Add getDefaults command (WIP) (#3444)

* add getYAML function for configs

* create getDefaults command(based on getConfig)

* add getDefaults command to CLI

* read defaults files, using github tokens as well

* write defaults to stdout as JSON object with YAMLs embedded

* catch case where no input files are given

* actually write output to file if outputFile is specified

* mark defaultsFile flag as required

* add basic tests

* add output (string)  test

* adapt generateDefaults() to return output (used for test of previous commit)

* Changes to getDefaults() JSON output (#3449)

* change JSON output to contain separate fields

* filename -> filepath

* Apply suggestions from code review

* Apply suggestions from code review

* Update pkg/config/config_test.go

Co-authored-by: Jordi van Liempt <35920075+jliempt@users.noreply.github.com>
2022-01-24 13:55:28 +01:00
Christopher Fenner
8e0209db74
feat(GitHub): fetch commit statistics (#3381)
* fetch GH statistics

* move GH and Sonar integration tests to own files

* fix imports

* add integration test case

* add result type

* Apply suggestions from code review
2022-01-24 13:43:07 +01:00
Christian Volk
b0e4599d4d
feat(malwareExecuteScan): refactoring and docker support (#3421)
* feat(malwareExecuteScan): add support for scanning docker images

* refactoring

* print out finding if available

* generate toolrecord for malware scan

* persist scan report

* docs

* fix

* fix

* rollback cmd/init_unix.go

* auhenticated pull

* fix

* fix: report shall be consistent with the api model

* gcs upload

* fix linter
2022-01-24 09:48:01 +01:00
Sven Merk
6520115950
Upload Fortify scan results to GitHub issue (#3300)
* fix(fortifyExecuteScan): Propagate translation errors

Force translation related errors to stop the execution of the step.

* Extend testcase

* Update fortifyExecuteScan.go

* Fix fmt and test

* Fix code

* feat(fortifyExecuteScan): Create GitHub issue

* Fix expectation

* Fix fmt

* Fix fmt add test

* Added tests

* Go fmt

* Add switch

* Rewrite githubCreateIssue

* Fix tests

* Added switch

* Issue only in case of violations

* Fix CPE reference

* Add  debug message to issue creation/update

* Update fortifyExecuteScan.go

* Add credential for GH to groovy wrapper

* Update fortifyExecuteScan.go
2022-01-21 10:52:17 +01:00
Raghunath Deshpande
af4bc6e989
Update maxWaitTime timeout for WhiteSource (#3284)
* Update scanPolling.go

Changing maxWaitTime from 15 to 30 to overcome WhiteSource results reflection in the backend issue.

* Update configHelper.go

* Reset configHelper changes to fix PR 3284

 Committer: raghunathd8

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: raghunathd8 <root@docker-evaluation.openstack.eu-nl-1.cloud.sap>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2022-01-20 12:14:00 +01:00
Siarhei Pazdniakou
c93f1f861d
GCS Upload fixes (#3387)
* Fixed argument type in persist function

* Fixed gcp upload to be usable in internal piper

* Fixed import of packages

* Updated logs

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-01-19 13:16:57 +01:00
Umidjon Urunov
c2ebdfd9ec
feat(protecode): add versioning model (#3373)
* changes to detectExec before master merge

* changes for detectExecuteScan

* self generated code added

* fix syntax errors and update docu

* added unit tests for fail and Group

* fix failOn bug

* add Groups as string array

* add Groups as string array

* tests and validation for groups, failOn

* Updated docs and added more tests

* documentation md files should not be changed

* Handle merge conflicts from PR 1845

* fix merge errors

* remove duplicate groups, merge error

* adding buildCode and buildTool as params

* switching build options

* building maven modules

* parameter correction

* parameter correction

* gnerate with new build parameter

* adding comments

* removing piper lib master and modifying goUtils to download 1.5.7 release

* first cleaning then installing

* multi module maven built

* multi module maven built removing unwanted code

* multi module maven built moving inside switch

* testing

* modifying the default use case to also call maven build

* modifying the default use case to also call maven build wih --

* corrected maven build command

* corrected maven build command with %v

* skipping test runs

* testing for MTA project with single pom

* adding absolute path to m2 path

* clean up

* adding switch for mta and maven and removing env from containers

* commiting changes for new detect step

* correting log message

* code clean up

* unit tests changes to detectExecute

* basic tests for new change

* restoring piperGoUtils to download correct piper binary

* code clean up

* code clean up

* protecodeExecuteScan :: versioning model draft - 1

* protecodeExecuteScan :: version model draft-2

* protecodeExecuteScan :: changing filename and version concatenation

* protecodeExecuteScan :: update documentation

* protecodeExecuteScan :: double URL encoding has been corrected & console messaging improved

* protecodeExecuteScan :: fixed Go/generate validation fail

* protecodeExecuteScan :: fixing failed unit tests

* protecodeExecuteScan :: Version field added

* protecodeExecuteScan :: Version field add => minor changes

* protecodeExecuteScan :: Version field add => fixing tests

Co-authored-by: D072410 <giridhar.shenoy@sap.com>
Co-authored-by: Keshav <anil.keshav@sap.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2022-01-19 10:30:59 +01:00
Nikolay Grechanov
17510f8996
fix: correct typo in log message (#3439)
Closes #3438
2022-01-19 08:41:11 +01:00
Ralf Pannemans
c97625e840
feat(cnbBuild) containerImageName will be defaulted if possible (#3437)
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
Co-authored-by: Sumit Kulhadia <sumit.kulhadia@sap.com>
2022-01-14 16:49:45 +01:00
Anil Keshav
04c4460c78
including vaultSecretFile to parameter generator docu (#3436)
Co-authored-by: anilkeshav27 <you@example.com>
2022-01-14 15:03:29 +01:00
Pavel Busko
1750b75cb8
feat(cnbBuild): preserve maven test results in the workspace (#3429)
Co-authored-by: Pavel Busko <pavel.busko@sap.com>
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
2022-01-14 11:05:11 +01:00
Anil Keshav
f3b65ae43b
feat (npmExecuteScripts) create seperate npmrc file for publish to private repo (#3422)
* creating new npm rc file

* publishing to registry staging

* exposing base64 version of env variables

* changing encoding param

* fixing unit test for the new path

* debugging env var

* remove debug message

* update docu

* changing new npmrc file name

* adding new npmrc to ignore

* adding new npmrc to ignore

Co-authored-by: anilkeshav27 <you@example.com>
2022-01-13 13:08:19 +01:00
Daniel Mieg
8634d8bb12
Add easy mode for AUnit & ATC (#3389)
* remove mandatory flag from config files

* Enable repo.yml as config

* Adapt to merge

* Refactoring

* Refactoring

* avoid panic

* Add comments

* Add easy mode for atc

* Add tests

* Add test

* Refactor

* Add test for MPS

* Updates

* Rename functions

* Add files to gitignore

* Rename

* Renaming

* Renaming

* Renaming

* Improve error messages

* Update documentation

* Add logging

* Rename

* Extend gitignore
2022-01-12 12:02:27 +01:00
Marc Bormeth
78c4a8fe93
Revert "feat: Add debug output to getConfig (#3341)" (#3396)
This reverts commit b760bf48e7.

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-01-11 10:47:03 +01:00
Pavel Busko
547ef5b55d
feat(cnbBuild): remove docker config after parsing (#3417)
Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>
2022-01-11 10:01:15 +01:00
Mayur Belur Mohan
f27cb4e482
ApiProxyUpload Command (#3295)
* ApiProxyUpload Command

* Code Review Fixes

* CodeReview Changes

* CodeReview Fixes

* YAML fixes

* CodeReview Fix

* Code Review Fixes

* CodeReview Fixes

* Code Climate Fixes

* Code Review Fixes

* Code Review Fixes

Co-authored-by: Roland Stengel <r.stengel@sap.com>
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
2021-12-28 09:31:50 +01:00
Pavel Busko
81fa0ee2d8
feat(cnbBuild): support for fetching binding content from url (#3388)
Co-authored-by: I546443 <sumit.kulhadia@sap.com>
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
2021-12-23 10:21:13 +01:00