1
0
mirror of https://github.com/SAP/jenkins-library.git synced 2024-12-14 11:03:09 +02:00
Commit Graph

1145 Commits

Author SHA1 Message Date
Oliver Feldmann
810d197665
[ANS] Step implementation (#3764)
* Add ans implementation

* Remove todo comment

* Rename test function

Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>

* Better wording

Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>

* Add reading of response body function

* Use http pkg ReadResponseBody

* Check read error

* Better test case description

* Fix formatting

* Create own package for read response body

* Omit empty nested resource struct

* Separate Resource struct from Event struct

* Merge and unmarshall instead of only unmarshalling

* Improve status code error message

* Remove unchangeable event fields

* Separate event parts

* Change log level setter function

* Restructure ans send test

* Revert exporting readResponseBody function

Instead the code is duplicated in the xsuaa and ans package

* Add check correct ans setup request

* Add set options function for mocking

* Review fixes

* Correct function name

* Use strict unmarshalling

* Validate event

* Move functions

* Add documentation comments

* improve test

* Validate event

* Add logrus hook for ans

* Set defaults on new hook creation

* Fix log level on error

* Don't alter entry log level

* Set severity fatal on 'fatal error' log message

* Ensure that log entries don't affect each other

* Remove unnecessary correlationID

* Use file path instead of event template string

* Improve warning messages

* Add empty log message check

* Allow configuration from file and string

* Add sourceEventId to tags

* Change resourceType to Pipeline

* Use structured config approach

* Use new log level set function

* Check correct setup and return error

* Mock http requests

* Only send log level warning or higher

* Use new function name

* One-liner ifs

* Improve test name

* Fix tests

* Prevent double firing

* Reduce Fire test size

* Add error message to test

* Reduce newANSHook test size

* Further check error

* Rename to defaultEvent in hook struct

* Reduce ifs further

* Fix set error category test

The ansHook Fire test cannot run in parallel, as it would affect the
other tests that use the error category.

* Change function name to SetServiceKey

* Validate event

* Rename to eventTemplate in hook struct

* Move copy to event.go

* Fix function mix

* Remove unnecessary cleanup

* Remove parallel test

The translation fails now and again when parallel is on.

* Remove prefix test

* Remove unused copyEvent function

* Fix ifs

* Add docu comment

* Register ans hook from pkg

* register hook and setup event template seperately

* Exclusively read eventTemplate from environment

* setupEventTemplate tests

* adjust hook levels test

* sync tests- wlill still fail

* migrate TestANSHook_registerANSHook test

* fixes

* Add ans send event step

* Fix tests

* Add groovy wrapper

* Add groovy wrapper test

* Fix function names

* Reduce ifs

* Fix docu

* We always set the timestamp

* Validate event

* Test unknown field in json

* Make test list test

* Set all event fields as separate parameters

* Generate and fix code

* Review fixes

* Format test file

* Format go code

* Fix common steps tests

* Print event to console if verbose

Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>
Co-authored-by: Roland Stengel <r.stengel@sap.com>
2022-06-21 15:45:07 +02:00
Mayur Belur Mohan
cdea4b7713
ApiProxyList Command (#3794)
* ApiProxyList Command

* CodeReview Fixes

* CodeReview Fixes

* CodeReview FIxes

* CodeReview Fixes

* CodeReview FIxes

* CodeReview Fixes

* fixing unit test

* doc fixes

* Update documentation/docs/steps/apiProxyList.md

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>

* CodeReview Fixes

* CodeReview Fixes

* CodeReview Fixes

* codereview fix

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
2022-06-21 13:08:00 +02:00
Ralf Pannemans
d28a48896c
cnbBuild: add alias for imageNames for easier valueMapping in helmDeploy (#3822)
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
Co-authored-by: Sumit Kulhadia <sumit.kulhadia@sap.com>
Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>
2022-06-15 14:17:46 +02:00
Pavel Busko
4ab32f7e7c feat(cnbBuild): generate buildSettingInfo
Co-authored-by: Sumit Kulhadia <sumit.kulhadia@sap.com>
Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
2022-06-15 12:37:21 +02:00
Julian Schmitt
dd0aae6ded
Create azureBlobUpload (#3766)
* add Step azureBlobUpload

* add azure sdk and unit tests

* add Documentation

* fix Groovy Wrapper

* adopt the requested changes from awsS3Upload

* fix lint tests

* downgrade azure sdk to go 1.17

* multiple fixes e.g. use of temporary files for tests

* fix tests

* Update cmd/azureBlobUpload.go

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>

* Update cmd/azureBlobUpload.go

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>

* Update documentation/docs/steps/azureBlobUpload.md

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>

* Update documentation/docs/steps/azureBlobUpload.md

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>

* Update documentation/docs/steps/azureBlobUpload.md

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>

* Update documentation/docs/steps/azureBlobUpload.md

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>

* requested changes

* use latest version of azure sdk after update to go 1.18

* change staticcheck from 1.1.0 to 1.2.0

* try to fix lint test by pre-compiling go 1.18

* fix caching for lint test

* improve error handling by dividing runner

* improve error handling and add validation

* multiple naming fixes

* add new test for unmarshalling JSON-Structs

* Update cmd/azureBlobUpload_test.go

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>

* Update cmd/azureBlobUpload_test.go

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>

* Update cmd/azureBlobUpload_test.go

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>

* fix JSON unmarshall test

* Update documentation/docs/steps/azureBlobUpload.md

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>

* Update cmd/azureBlobUpload_test.go

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>

* Update cmd/azureBlobUpload.go

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>

* fix uploadFunc

Co-authored-by: Thorsten Duda <thorsten.duda@sap.com>
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
2022-06-15 09:41:02 +02:00
tiloKo
114e91f328
Update abapEnvironmentAssembleConfirm.go (#3825)
* Update abapEnvironmentAssembleConfirm.go
2022-06-13 13:13:15 +02:00
Anil Keshav
011ba40f87
feat (shellExecute) extend shellExecute to accept script arguments (#3823)
* adding arguments

* splitting strings into args and checking position

* addtional check on adding arguments

* unit testing

* refactoring code

* unit test clean up

* add unit test for multiple params in multiple scripts

* unit test name

Co-authored-by: anilkeshav27 <you@example.com>
2022-06-10 16:10:46 +02:00
Siarhei Pazdniakou
92837fde18
feat(gradleExecuteBuild, fortifyExecuteScan): gradle improvements (#3807)
* Improvements were made

* fixed tests

* fixed issues

* fix versioning

* fix Inclusive Language warnings

* gradle support to fortifyExecuteScan. Classpath resolving

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-06-07 10:24:10 +02:00
xgoffin
903f273012
feat(checkmarxExecuteScan): added API to get description, incorporated to SARIF file (#3814) 2022-06-01 15:48:56 +02:00
xgoffin
6a43e9f455
feat(fortifyExecuteScan): further improvements to the SARIF generation (#3799)
* feat(fortfiyExecuteScan): proper XML unescaping, added rulepacks to SARIF, added kingdom/type/subtype to tags

* feat(fortifyExecuteScan): proper handling of severity, kinds, levels in SARIF

* fix(fortifyExecuteScan): edge case when handling properties taht could lead to a crash

* fix(fortifyExecuteScan): ensure SARIF processing is done after latest FPR is processed by SSC
2022-05-24 13:40:49 +02:00
Daniel Mieg
cda6e71ab9
Add ABAP step: createTag (#3633)
* Cloud Platform -> BTP

* Initial generation of new step

* add flag

* wip

* Fix warnings

* Add command

* Added Tag Decription

* Add status check

* Improve handling

* Improve handling

* Add test for happy path

* Add test reports to gitignore

* Add second test

* Improve createTag

* Add testcase

* Adaptions

* Add test

* Update cmd/abapEnvironmentCreateTag.go

Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com>

* Update cmd/abapEnvironmentCreateTag.go

Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com>

* Adapt error handling

* Improve coding

* Add info

* Disallow repositories and repositoryName at the
same time

* Regenerate

* Adapt to feedback

* Update cmd/abapEnvironmentCreateTag.go

Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com>

* Update cmd/abapEnvironmentCreateTag.go

Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com>

Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com>
2022-05-23 15:15:22 +02:00
rosemarieB
d30395931b
Do not merge before go live of steampunk 2205! Insert generic build step into abap environment pipeline (#3507)
* enhance build step

* Update abapEnvironmentBuild.go

* build with addonDescriptor

* use addondescriptor

* Use Addondescriptor

* add error

* improve logging

* rename intervall to interval

* update yaml

* Update abapEnvironmentBuild.go

* Update abapEnvironmentBuild.yaml

* insert generation phase

* we do not know what we have done

* Add createServiceKey to test

Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com>
Co-authored-by: Daniel Mieg <daniel.mieg@sap.com>
2022-05-23 13:47:03 +02:00
Marc Vornetran
2dbb198804
fix(malwareExecuteScan): Set ImageFormat to tarball (#3791) 2022-05-23 11:22:49 +02:00
thtri
4c2845b79c
fix(fortify):add SSC project identifier to ouput and JSON report (#3787) 2022-05-20 10:24:16 +02:00
Adrien
3d48364862
Fix project config reset when preset is set (#3782) 2022-05-18 17:10:00 +02:00
Julian Schmitt
6714794066
Create awsS3Upload (#3737)
* add Step awsS3Upload

* fix JSON Read

* fix groovy wrapper

* change credentials to secret text

* Change credentials type to token

* add cleanup for environment variables

* Add AwsS3UploadCommand to piper.go

* add documentation of awsS3Upload

* Fix JSON String Example in Documentation

* add the Upload of whole directories

* add Logging to awsS3Upload.go

* imporve Logging

* fix and improve unit tests

* fix non-existing-filepaths-Bug

* fix windows filepaths

* remove ... from logging

* change step description

* fix PR Tests

* remove redundant code

* try to run tests sequentially

* fix file.Close

* executed go mod tidy

* requested changes

* fix comments and awsCredentials

* Update documentation/docs/steps/awsS3Upload.md

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>

* use temporary files for unit tests

* executed go generate

Co-authored-by: ffeldmann <f.feldmann@sap.com>
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
Co-authored-by: Thorsten Duda <thorsten.duda@sap.com>
2022-05-18 10:37:54 +02:00
Oliver Nocon
6070549704
fix(helmExecute): respect version from Chart (#3780)
* fix(helmExecute): respect version from Chart

using version from CPE can create failure situations in case format is not semver.
This is the case for maven artifacts, for example.

* chore: simplify condition

* chore: cleanup

* chore: cleanup
2022-05-18 08:58:01 +02:00
Thorsten Duda
434e4529d5
cfcli update (#3777)
* cfcli update

* cfcli update
2022-05-17 08:53:19 +02:00
Mayur Belur Mohan
ef2db4c2b0
ApiProviderUpload Command (#3663)
* ApiProviderUpload Command

* formatting fix

* formatting fix

* formatting fix

* formatting fix

* CodeReview Changes

* CodeReview Fix

* CodeReview fix

* CodeReview Fix

* CodeReview FIx

* CodeReview FIxes

* CodeReview Fixes

* CodeReview Fixes

* CodeReview Fixes

* CodeReview Fix

* CodeReview Fix

* CodeReview Fixes

* CodeReview Fix

* Doc Fixes

* CodeReview Fix

* CodeReview Fixes

* Doc Fixes

* Update cmd/apiProviderUpload_test.go

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>

* CodeReview FIxes

* Update cmd/apiProviderUpload.go

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>

* CodeReview Fixes

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
2022-05-16 11:52:04 +02:00
Oliver Nocon
57347702d4
fix(versioning): correct propagated version (#3778) 2022-05-16 10:02:25 +02:00
Anil Keshav
e6724d7f05
fix (whitesourceExecuteScan) keep tar extension for target image donwload (#3774)
* explicitly adding tar extension to project name when constructing the targetFilePath for whitesource docker image download

* comments

* correcting comment for better readability

* replace spaces in the project name with underscroe

* better comments

* passing legacy format download

* appending format to value

* keeping the download format for protecode as legacy

* improving docu

* keeping legacy format the default

* keeping tar file name same as project name to avoid duplicate names

* keeping legacy format download hard coded

Co-authored-by: anilkeshav27 <you@example.com>
2022-05-13 18:56:41 +02:00
Ashly Mathew
3c288e81ad
fix (golang) : publish binaries to custom in cpe (#3767)
Signed-off-by: Ashly Mathew <ashly.mathew@sap.com>
2022-05-05 15:06:42 +02:00
Ashly Mathew
823ef47db6
Golang build publish binaries and add group id (#3749)
* Add golang binaries to CPE and add groupID in coordinates

Signed-off-by: Ashly Mathew <ashly.mathew@sap.com>

* Introducing binaries as artifacts

Signed-off-by: Ashly Mathew <ashly.mathew@sap.com>
2022-05-05 12:39:59 +02:00
Giridhar Shenoy
fe9b3a796b
detectExecuteScan: fix project version limiting api call (#3754)
* fix project version limiting issue

* add tests for detectExecute
2022-05-05 09:58:43 +02:00
Adrien
9d56cda0f9
Add Checkmarx failure message to Piper error log (#3716) 2022-05-03 17:34:14 +02:00
Daniel Bernd
a3f1234a60
Feature: ATC & AUNIT - ObjectSetLibrary(OSL) usage (#3755)
* own OSL go File & first changes ATC step

* OSL Integration in AUnit & ATC steps 1

* OSL & Unit Tests

* Unit Tests 1

* OSL - AUnit & ATC usage

* Unittest ATC: packagetree --> package incl. subpackages

* correct spelling

* yaml & generated update

* generated Metadata

Co-authored-by: Daniel Bernd <93763187+danManSAP@users.noreply.github.com>
2022-04-27 13:30:43 +02:00
Anil Keshav
1272b763f8
feat (pythonBuild) enable creation of a virtual environment (venv) (#3746)
* create virtual env

* adding bin bash source

* using sources from bin bash

* trying with bash

* appending filename to source

* using standard pip install

* not using root user

* adding path for pip

* using virtual env

* using virtual env name in path

* removing virtual env

* adding file path manually

* using root

* not using root and postpone removing venv

* trying to use the python from venv

* test to remove the venve

* seeing which python

* using symlink for python

* unit test

* python docu stub

* fix unit test and yaml extra line

* fixing unit test

* unit test success case fix

* unit test fix

* unit test fixes

* unit test and default publish flag

* fix integration test

Co-authored-by: anilkeshav27 <you@example.com>
2022-04-26 14:15:24 +02:00
Pavel Busko
2740f00134 added missing git credentials handling
Co-authored-by: Gareth Evans <g.evans@sap.com>
2022-04-26 09:38:49 +02:00
Gareth Evans
74b6b09609 fix(golangBuild) properly handle multi main package builds
Co-authored-by: Pavel Busko <pavel.busko@sap.com>
2022-04-26 09:38:49 +02:00
Jordi van Liempt
eb73cae888
feat(getDefaults): add parameter for CRD-style config (#3736)
* export LoadConditionsV1() in config pkg

* getDefaults: read stage conditions as PipelineDefinitionV1

* add getDefaults useV1 tests

* add comment (pointed out by code climate analysis)

* small addition to comment (from previous commit)

Co-authored-by: I557621 <jordi.van.liempt@sap.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2022-04-19 12:34:40 +02:00
Vitalii Sidorov
63cdfc0e68
Fix helm execute command (#3701)
* Add small fix

* fix unit-tests

* Add deploymentName and packageVersion as flags

* small fix

* Change getting version of helm chart

* small fix

Co-authored-by: “Vitalii <“vitalii.sidorov@sap.com”>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-04-14 15:43:47 +02:00
Siarhei Pazdniakou
d519966fe5
feat(gradleExecuteBuild): publish (upload to staging) (#3662)
* gradle publish

* Fix gradle utils

* Added unit test

* Add descriptions for parameters

* Added unit tests

* fix publish init script template

* fix artifactId and groupId

* fix project path

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-04-14 14:59:09 +02:00
Martin Zuber
7e0ec525bf
Use full file path when trying to publish MTAR in mtaBuild step. (#3735) 2022-04-13 14:50:13 +02:00
tiloKo
fa24ab9ed8
increase default timeout to 16min (#3732) 2022-04-13 09:03:22 +02:00
Ralf Pannemans
34a0d63d9f Use builder:base instead of builder:full because nc is no longer needed 2022-04-12 16:41:51 +02:00
rosemarieB
9fdaf86856
Enable additional piecelist for assembly (#3730)
* add useClassicCTS

* add option additionalPiecelist

* Update abapEnvironmentAssemblePackages.go
2022-04-12 16:00:17 +02:00
rosemarieB
1ffe22903e
Enable classic cts (#3728)
* add useClassicCTS

* debug output and unit test

* unsaved fix

* error logging

* Update abapEnvironmentAssemblePackages.go

* Update abapEnvironmentAssemblePackages.go

* add general scope and remove logging

* Update bfw.go

* Update abapEnvironmentAssemblePackages.go

* Update abapEnvironmentAssemblePackages.go

* Update package.go

* Update abapEnvironmentAssemblePackages.go

* change scope and add certificates

Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com>
2022-04-12 14:39:42 +02:00
Ralf Pannemans
f2305a6b08
feat(cnbBuild) Allow path to be glob (#3726)
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
2022-04-12 11:04:05 +02:00
Anil Keshav
6dcda42dd6
fix (mavenBuild) removing root path when creating keystore (#3714)
* remving root paths for trust store

* adding read / write permission to keystore

Co-authored-by: anilkeshav27 <you@example.com>
2022-04-08 13:19:08 +02:00
Christopher Fenner
465bab8ddf
feat(githubRelease): publish multiple assets (#3702)
* add parameter for uploading multiple assets

* use assetPathList parameter

* add test case

* fix typo

* fix test case

* generage ghClient mock

* add test files

* make function testable

* add test case

* regenerate mock

* regenerate mocks
2022-04-08 11:29:07 +02:00
Michael
6676da1f9d
feat(gitopsUpdateDeployment) forcePush (#3665)
* feat(gitopsUpdateDeployment) forcePush

fix(gitopsUpdateDeployment) include registry

The push operation in this step can be forced to bypass branch-protection

Signed-off-by: Michael Sprauer <Michael.Sprauer@sap.com>

* add unit test

Signed-off-by: Michael Sprauer <Michael.Sprauer@sap.com>

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-04-07 16:33:46 +02:00
xgoffin
fb9792ad71
feat(fortifyExecuteScan): optimization of the SARIF conversion code (#3710)
* feat(fortifyExecuteScan): query SSC once for batch audit data

* fix(fortifyExecuteScan): check audit data length in all cases

* feat(fortifyExecuteScan): in fpr_to_sarif, better detection of error cases, unit tests

* fix(log): comment useless error message

* fix(fortifyExecuteScan): clarify log message

* fix(fortifyExecuteScan): adapt unit tests
2022-04-07 13:11:52 +02:00
Anke Ravalitera
5a4b7c9925
Creating a new scenario for gcts pipeline steps (#3711)
* updated gCTS_Scenario.md gCTSExecuteABAPUnitTests.md gCTSExecuteABAPUnitTests.yaml gCTSExecuteABAPQualityChecks.yaml

* add gCTS images for scenario

* text updates to gctsRollback step

* go generate

* updated gCTS_Scenario.md

* gCTS_Scenario.md updates

* corrected text in gCTS_Scenario.md

* correct formatting in gCTS_Scenario.md

* getting used to trailing spaces

Co-authored-by: Rinita Asani <rinita.asani@sap.com>
2022-04-07 09:45:14 +02:00
Anil Keshav
f8af3c154e
changing trust store permission (#3709)
Co-authored-by: anilkeshav27 <you@example.com>
2022-04-06 16:06:04 +02:00
Julian Schmitt
6151f36d88
Implement Release 3.0.0 of CM-Client (#3664)
* Remove --backend-type

* Delete CTS in isChangeDevelopment and change Dockerimage of CM-Client

* fix groovy unit tests

* another fix of groovy unit tests

* try to fix import of fork for Jenkins-Testing

* add workflow to create Go Binary for Jenkins-Server

* Change RepoOwner to test in Fork

* remove previous changes

* adjust docker image for TransportRequestCreate and Release

* Remove CTS from Documentation

Co-authored-by: Thorsten Duda <thorsten.duda@sap.com>
2022-04-05 12:33:19 +02:00
Philipp Stehle
56981b8f53
feat(cnbBuild): exclude .git folder from cnb workspace by default (#3699)
Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
2022-04-05 10:09:56 +02:00
Anil Keshav
aa74e090d5
fix (pythonBuild) run as root and generate build settings info (#3695)
* adding PIP to BuildTool.groovy

* trying to run the container with root

* only creating sdist

* including wheel distribution

* adding settings info

Co-authored-by: anilkeshav27 <you@example.com>
2022-04-05 09:16:46 +02:00
xgoffin
3c55d3c99c
feat(checkmarxExecuteScan): convert Checkmarx xml report to SARIF (#3696)
* feat(checkmarxExecuteScan): sarif conversion for Checkmarx XML reports

* feat(checkmarxExecuteScan): added taxonomies and similarityID

* fix(checkmarxExecuteScan): proper handling of ruleId and ruleIndex

* fix(sarif): mistype in checkmarx properties

* fix(checkmarxExecuteScan): fixed occasional panics when handling audit comment

* chore(sarif): proper variable naming

* chore(code): fix missing and unrecognized comments

* trigger PR

* fix(format): extra space

Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2022-04-04 16:12:35 +02:00
Philipp Stehle
6b6208a35c
remove confusing log message (#3685) 2022-04-04 15:00:14 +02:00
Anil Keshav
2bd49fbe19
feat (shellExecute) extend sources param to download scripts. (#3674)
* first version to download script from git

* unit test adjust

* adding git token

* info messages

* removing extra info message

* changing file permission for scrtips

* modying sources to handle https download

* adding script downloads

* commenting the file permission change

* changing persmission

* adding header to download file

* adding perimssions

* adding perimssions

* not touching file permissions

* adding to pipeline

* return file name

* changing script name

* adding file permission changes

* adding file permission changes

* using current directory

* file permission

* downloading in .pipeline folder

* removing permission handeling

* improving the step docu

* improving the step docu

* unit test and code cleaning

* fix typo

* adding read execute permission

* fix unit test

* fix unit test

* removing negative test

Co-authored-by: anilkeshav27 <you@example.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-03-31 18:18:20 +02:00
Oliver Nocon
62b3a9a459
fix(versioning): properly consider go.mod (#3687)
* fix(versioning): properly consider go.mod

* fix test
2022-03-31 17:07:10 +02:00
Oliver Nocon
20c5f0a63b
fix(optimization) use proper cpe reference (#3683) 2022-03-31 10:52:54 +02:00
Pavel Busko
1f750af16d
feat(cnbBuild): cache buildpacks during multi-image build (#3635)
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
2022-03-30 13:58:16 +02:00
Philipp Stehle
f4f11dba7f
chore(kubernetesDeploy): bump docker image to helm 3.8.1 (#3470) 2022-03-30 13:23:16 +02:00
Oliver Nocon
276844e6a2
fix(optimization): expose parameters to general section (#3680) 2022-03-30 12:20:51 +02:00
Vitalii Sidorov
d62c3d73a0
Add helm dependency command (#3669)
* Add helm dependency command

* Change name of flag for package command

Co-authored-by: “Vitalii <“vitalii.sidorov@sap.com”>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-03-30 08:18:51 +02:00
Anke Ravalitera
b6a5ceaa03
Update gCTS steps (#3673)
* gCTS Scenario and two images

* Updated gCTS Scenario

* updated gCTS_Scenario.md and images

* added checkstyle stage

* upated the gCTSExecuteABAPQualityChecks.yaml file with severity information

* updates to gCTSExecuteABAPQualityChecks and gCTSExecuteABAPUnitTests steps

* updated gCTS_Scenario.md and relevant images

* Add more explanation for workspace parameter, change atc variant default value to DEFAULT

* Fix a type in description of workspace parameter

* changes to gctsExecuteABApUnitTests.md gctsExecuteABAPQualityChecks.yaml and gctsExecuteABApUnitTests.yaml

* final updates to gCTS steps and scenario

* added env. to GIT_COMMIT in gCTSExecuteABAPQualityChecks.md and gCTSExecuteABAPUnitTests.md

* remove the gScenario description from the branch

* Correct column typo in yaml file and generate files

* Add a new line in docu for gCTSExecuteABAPUnitTests

* Remove blank line from docu in gctsExecuteABAPUnitTests

Co-authored-by: Rinita Asani <rinita.asani@sap.com>
Co-authored-by: Sarat Krishnan <78093145+sarat-krk@users.noreply.github.com>
2022-03-29 21:24:10 +02:00
TheShifter
188e743f7b
PythonBuild: Implementation of pythonBuild step (#3483)
* Implementation of pythonBuild step

* minor update and refactoring

* minor update

* add integration test and test project to testdata dir

* remove generated build data dir

* Rewrite some logic. Minor fix in integration tests for python

* Add new input parameters to pythonBuild.yaml

* rewrite logic
remove some checks

* rollback

* resolve merge conflict in piper.go
Update logic in python build. Create bom now works fine

* remove duplicate line

* refactoring fix

* resolve comment. Remove install build and change build command. Change twine upload command

* add groovy wrapper for pythonBuild step

* Rewrite tests. Remove some cheks from pythonBuild.go

* add some test to pythonBuild_test.go

* Add some parameters and credentials to the pythonBuild.groovy

* fix issue in unit tests

* add pythonBuild to fieldRelatedWhitelist

* update integration test for pythonBuild

* add imports

* update integration tests and add a new one

* minor fix

* fix some issues in integration tests

* update integration tests. Make it works again

Co-authored-by: Anil Keshav <anil.keshav@sap.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-03-29 19:01:44 +02:00
ffeldmann
ccc1c976ee
fix(orchestrator) usage of correct env variables (#3650)
* Reorders getApiInformation, changes variables to get start time, adjusts and adds test cases
* Changes the way to get apiInformation and reduces number of requests
* Changes getting pipeline start time from correct env variable
* Refactors getApiInformation functionality
* Adds GetBuildReason() for Azure and Jenkins
* Updates JobURL for ADO
2022-03-28 09:52:15 +02:00
Christian Volk
26bf3808fe
chore(checkmarxExecuteScan): split and trim filterPattern (#3661) 2022-03-23 11:45:05 +01:00
Christian Volk
22f6aa156f
feat(docker): use crane for pulling docker images (#3652) 2022-03-23 10:02:00 +01:00
Ralf Pannemans
62a7f6336a
feat(cnbbuild) use commitid fom cpe as tag if artifact version is not available (#3659)
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
Co-authored-by: Sumit Kulhadia <sumit.kulhadia@sap.com>
2022-03-22 14:25:20 +01:00
Oliver Nocon
fed08c2399
fix(whitesourceExecuteScan): proper container options for golang (#3660) 2022-03-22 10:40:33 +01:00
Oliver Nocon
eef6c028aa
feat(artifactPrepareVersion): context-sensitive versioningType (#3579)
* feat(artifactPrepareVersion): context-sensitive versioningType

Add functionality to switch to `versioningType: cloud_noTag` for
* PR voting
* scheduled and optimizes pipelines

* add missing file

* update tests

* update docs

* chore: update generation

* chore: fix tests

Co-authored-by: ffeldmann <f.feldmann@sap.com>
2022-03-22 08:31:59 +01:00
Siarhei Pazdniakou
db5360fb89
feat(gradleExecuteBuild): BOM creation. Integration tests (#3603)
* Implemented bom creation

* Made small fixes. Added integration tests

* go generate

* minor fixes

* fix tests

* Added unit tests

* minor fixes

* use fileutils

* integration tests optimization

* change integraton tests timeout to 25m

* Fix Inclusive Language warnings

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-03-21 10:17:03 +01:00
Christian Volk
4b29f2e001
fix(kaniko): deactivate reproducible flag (#3653) 2022-03-21 09:14:36 +01:00
Oliver Nocon
a466b60aaf
feat(helmExecute): allow reading specific credentials from cpe (#3647) 2022-03-21 08:41:22 +01:00
Michael
f4a29554ed
(chore) bump to dtzar/helm-kubectl:3.8.0 (#3573)
Signed-off-by: Michael Sprauer <Michael.Sprauer@sap.com>

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-03-18 14:56:36 +01:00
Maximilian Braun
040a2c36ef
fix(golangBuild): order of arguments (#3645)
* fix: order of go build arguments

As per https://pkg.go.dev/cmd/go#hdr-Compile_packages_and_dependencies 
The `go build [-o output] [build flags] [packages] ` -ldflags shall be put before packages.

The build may fail this way
```
running command: go build -trimpath -o foo-linux.amd64 ./cmd/main.go -ldflags '-linkmode=external'
named files must be .go files: -ldflags

```
2022-03-18 12:03:45 +01:00
Vitalii Sidorov
2a4052d13c
feat(helmExecute): run complete lint, build, publish cycle (#3546)
* Add runHelmCommand

* Add dryRun for debug

* Add default case in helmExecute

* Fix unit-tests

* small fix

* Fix RunHelmAdd and change RunHelmPublish methods

* Fix RunHelmPublish

* Fix unit-tests

* Fix unit-test

* small fix

* small fix

* small fix

* Add LintFlag PackageFlag PublishFlag flags

* Add tests for httpClient.go

* test

* test

* smal fix

* small fix

* Add getting name and version from Chart.yaml

* Add test

* Fix

* small fix

* Fix according to comments

* small fix

Co-authored-by: “Vitalii <“vitalii.sidorov@sap.com”>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Vitalii Sidorov <vitalii_sidorov@sap.com>
2022-03-17 17:13:34 +01:00
Sven Merk
c30e93bc6a
feat(detectExecuteScan): SARIF export and GH issue creation (#3637)
* Added SARIF and GH issue creation
2022-03-17 15:32:48 +01:00
xgoffin
3f6e4b9e3b
feat(fortifyExecuteScan): added parameter to generated sarif file (#3644)
* fix(sarif): change format to fit omitempty cases better

* feat(fortifyExecuteScan): include category in sarif file

* fix(fortifyExecuteScan): access to undefined pointer in some cases

Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2022-03-17 13:09:15 +01:00
Anil Keshav
818636b048
feat (checkIfStepActive) including cpe conditions to check if a step is active / not active (#3642)
* including cpe struct

* evaluating types

* adding environment root path

* unit tests

Co-authored-by: anilkeshav27 <you@example.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-03-17 12:09:16 +01:00
Oliver Nocon
504f076613
feat: support cpe credentials for multiple repos (#3641) 2022-03-17 08:01:00 +01:00
Oliver Nocon
7ec512cb9f
feat(artifactPrepareVersion): helm & propagate version (#3627)
* feat(artifactPrepareVersion): helm & propagate version

* chore: small refactoring

* chore: fix linting issue

* fix version persistence
2022-03-15 09:08:24 +01:00
Daniel Mieg
6651eaf6c8
Cloud Platform -> BTP (#3634) 2022-03-14 15:12:35 +01:00
Oliver Nocon
1151fc4de0
chore: export config functions (#3631)
* chore: export config functions

* add docs
2022-03-14 12:33:52 +01:00
Daniel Bernd
69fc4103c2
Response logging (#3599)
* Log adaption 1

* log handling 2

* log Handling 3

* log & handle body

* Unit Test & response Nil

* Response not Nil

* no change

no change

* no change

no change

* no change

no change

Co-authored-by: Daniel Bernd <93763187+danManSAP@users.noreply.github.com>
Co-authored-by: Daniel Mieg <56156797+DanielMieg@users.noreply.github.com>
2022-03-14 09:49:51 +01:00
Anil Keshav
fcbf28e3ff
checking if the maven install/verfiy return errors (#3625)
Co-authored-by: anilkeshav27 <you@example.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-03-11 10:02:50 +01:00
Oliver Nocon
2a56723d40
feat(kanikoExecute): Trim names for multi-image builds (#3617)
* feat(kanikoExecute): Trim names for multi-image builds

* chore: fix yaml linting issue
2022-03-11 09:47:44 +01:00
Anil Keshav
809a90ee27
adding vault references for username and password (#3624)
Co-authored-by: anilkeshav27 <you@example.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-03-11 09:25:46 +01:00
Oliver Nocon
2c837927d4
chore(whitesourceExecuteScan): Gradle config changes (#3621)
* chore(whitesourceExecuteScan): Gradle config changes

supersedes #3293
closes #3293

* update config

Co-authored-by: ffeldmann <f.feldmann@sap.com>
2022-03-11 08:18:21 +01:00
Johannes Dillmann
3708f274cc
feat(cnbBuild) read target image name from github cpe (#3620)
Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
2022-03-09 14:06:26 +01:00
maheshsrikrishnan
64a00c540a
New piper step APIProviderDownload contribution (#3349)
* added store file function in cpi common utils

Change-Id: Ia429a2792266e082d139025a71799d21c30a7df9

* Added api provider steps

Change-Id: Icd2829a91db4c53d0de2330822d2b33933973868

* Update apiProviderDownload.yaml

* fixed yaml JLINT issue

Change-Id: Iac974abc30fa00e68c0177072b93716b0af5e0c5

* Removed trailing spaces

Change-Id: I927e9314fce6e9cab68d6b97577c7c96bb2bddad

* Resolved common steps groovy script conflict

Change-Id: I3ad144b618e1c77953aaeccaa5bf7309aff77ca9

* Change for conflict resolution

Change-Id: Ic955833eca844f090b7983f99f9d3649ebb981c7

* Fixed method name and its corresponding implementation

Change-Id: I465c1f1d5306bb978386de9efca3c521e385b89c

* Moved re-usable function to commonUtils package

Change-Id: Ide06462b01caeb2bf438ad7661e01c15bf8e8e24

* Changed the implementation to use existing writeFile method

* Fixed review comments on documentation and test structuring

Change-Id: Ifebd2f4b50754b2097b2d564fb3cc37c433ef6c9

* Fixed documentation alignment issues

* Fixed spaces issue

Change-Id: I834bd94e01bce72e7f81ab49ba32671c91c66ca9

* Documentation removed extra spaces

Change-Id: I9a639d76ed9b81c870f18349504044bb70753b52

* Fixed doc build issue

Change-Id: I96c3e15e73834b64f8b8e3432ce59f6b037f93fd

* Fixed documentation build issues

Change-Id: I7fca2ba69bc7b7298ee300ccd1ae16a6238dc96b

* Re-generated code for fixing build failure

Change-Id: I22b7ee6162f643d9f3b60f6a33eb7858927182a0

* Adopted file utils & mock

Change-Id: Ic46462003527f41df64395a5a615c19bf374e8ef

* Removed ioutil call in the test & adopted error variable names

* Removed commented lines

Change-Id: I99a12e39bc04323e9c19f1409d97eeca267e6fdb

* Added test for asserting file download and adopted error variables

Change-Id: I49463a3b75987bf68f5261d45602d2d7bd960a05

* Added download path assertion positive & negative case

Change-Id: Ieee461c3973b9dfa8f395dc936e4241ff9694c7b

* Modified tests with DownloadPath variable

Change-Id: Iaf14c9ea1a8242b6c8d8e9e4fac8c23d9c1b3a74

* Added testcase to validate file content

Change-Id: I21aed481b433450c3b536dbb29d45291f61848d8

* Refactored test for file content check to avoid failures

Change-Id: I3b4fe9a0de678f437fd4cc0a8203ae9434d9fa8e

* Removed auto-generated comments

Change-Id: I86c4ac3e7e4476a75d6cbed58826ec1f3278d7d2

* Fixed documentation review comments

Change-Id: I4faf31473b53fc53a5517d418c343bf7320eec55

* Fixed documentation indentation

Change-Id: I386f046cf4e10ee6deb5a81fcfc8c430c97086c8

* Fix build issue

Change-Id: I61a829cabaf03ffd5e77cddc594486a650118fa3
2022-03-09 13:07:23 +01:00
Pavel Busko
c086f84668
fix(cnbBuild): filter overall buildpacks key in the telemetry data (#3619)
Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
2022-03-09 11:48:47 +01:00
Pavel Busko
d2ef57bf5f
feat(kubernetesDeploy): support for image digests (#3613)
Co-authored-by: Sumit Kulhadia <sumit.kulhadia@sap.com>
2022-03-09 09:48:19 +01:00
Pavel Busko
e2de22f5ea
feat(kubernetesDeploy): use go templating for the deployTool: kubectl (#3600)
Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
2022-03-08 14:52:43 +01:00
Siarhei Pazdniakou
f5be111078
Fixed deriving stage name from the orchestrator via env var (#3219)
* Fixed deriving stage name from the orchestrator via env var

* make the stageName the leading parameter

* Added tests

* Update cmd/checkIfStepActive.go

* Update cmd/checkIfStepActive_test.go

* Update cmd/checkIfStepActive_test.go

* Update cmd/checkIfStepActive.go

Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-03-08 12:12:56 +01:00
Oliver Nocon
f9ad6dc048
fix(whitesourceExecuteScan): orgToken from Vault (#3614)
supersedes #3168
2022-03-08 08:59:12 +01:00
Johannes Dillmann
0cef702d9d
bug(cnbBuild) Skip restoring image content (#3612)
Co-authored-by: Pavel Busko <pavel.busko@sap.com>
Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
2022-03-07 14:16:22 +01:00
Ralf Pannemans
424d8ec3a7
fix(kanikoExecute) Apply workaround for multi stage builds (#3610) 2022-03-07 11:42:33 +01:00
Mayur Belur Mohan
9a3b800b9d
ApiKeyValueMapUpload Command (#3407)
* ApiKeyValueMapUpload Command

* include command in metadata

* TestCase Fixes

* CodeReview Fixes

* CodeReview Fixes

* Code Review Fixes

* CodeReview Fixes

* CodeReview Fixes

* CodeReview FIxes

* CodeReview Fixes

* Documenation change

* documentation fix

* Documentation Fix

* Documentation Fix

* documentation Fix

* CodeReview Fixes

* CodeReview Fixes

* Revert changes

* Documentation Fix

* CodeReview FIxes

* Doc Fixes

* Code Review Fixes

* Code Review Fixes

* CodeReview FIxes

* Documentation Fix

* Documentation Changes

* Documentation Fix

* codereview fix

* Documentation Fix

* CodeReview Fixes

* CodeReview Fix

* Documentation FIx

* doc fix

* Doc Fix

* Documentation Fix

* codereview fix

* revert fix

* Code Review Fix

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
2022-03-07 11:03:44 +01:00
Pavel Busko
0de06c6207
feat(cnbBuild): write image digests to the CPE (#3602)
Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
2022-03-02 16:26:45 +01:00
Sven Merk
c1d2e6ad16
Add toggle for GH issue creation (#3601)
* Add toggle for GH issue creation

* Fix fmt
2022-03-02 15:46:56 +01:00
Daniel Bernd
e38d5ef69f
Space removed due to incorrect Link building to Comm.Scen. in long Description (#3593)
* Space removed due to incorrect Link building to Comm.Scen. in lond Description

* generated

* Translate verbose to sonar binary verbose flag (#3567)

* Translate Piper verbose to sonar bin flag

* Add test

* feat(golangBuild): Integration tests (#3575)

* Add entryPointPath option

* Integration tests for golangBuild

* Revert "Add entryPointPath option"

This reverts commit b541e64a4f.

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

* fix(kaniko): backwards compatibility with kaniko 1.3.0 (#3583)

* fix(kaniko): jnlp issues

* fix

* fix(kanikoExecute): backwards compatibility with kaniko 1.3.0

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

* Adjust sonarExecuteScan step (#3527)

* add check for instance parameter

* remove default from yaml

* adapt changes

* Update stashSettings.yml

* Unstash

* Add echo

* Stash changes

* Remove changes

* stash everything

* Change it up

* Change go

* Switch to new solution

* Try new stash

* Remove compliance

* Move Artifacts

* Change stashing

* Match others

* Remove space

* Test if warning goes away

Co-authored-by: Jesse Awan <jesse.awan@sap.com>

* Checkmarx json report (#3565)

* feat(checkmarx) : Checkmarx JSON Report

* Test cases with some fix

* Information total and audited test assertions

* feat(checkmarx): align total/audited with existing calculation

* fix(checkmarx): Reporting unit test

Co-authored-by: Sumeet PATIL <sumeet.patil@sap.com>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

* feat: url log parsing / reporting when executing a command (#3461)

* Initial PR for url reporting

* Rename URLReport var to URLReportFileName

* Remove URLReportFileName from piper flags

* Update pkg/command/command.go

* Update pkg/command/command.go

* Update pkg/command/command.go

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

* (fix) gitopsUpdateDeployment - integration test for kustomize (#3572)

* (fix) gitopsUpdateDeployment - integration test for kustomize

Signed-off-by: Michael Sprauer <Michael.Sprauer@sap.com>

* (fix) test: dash login shell is "-l"

Signed-off-by: Michael Sprauer <Michael.Sprauer@sap.com>

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

* fix (mavenBuild) multi-module maven builds  (#3587)

* moving jacoco agent preparation to after the goal flatten

* adding jacoco plugin to end

Co-authored-by: anilkeshav27 <you@example.com>

* fix(malwareExecuteScan): dont bind docker scanning to buildtool (#3547)

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

* Updates telemetry logging information for internal reporting (#3585)

* Add StepStartTime, Renames StepDuration, adds PiperCommithash, removes Branch, GitOwner, GitRepository from logged telemetry information

* Fixes test case for telemetry logging

* Activates step monitoring data in debug mode

* Pretty debug json printing

* Reduces log noise, setting warning to debug

* feat(kubernetesDeploy): added valuesMapping config option (#3568)

* Feature/fortify execute scan gradle (#3582)

* initial fortify gradle commit

* initial fortify gradle commit

Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

* add access_token alias to artifactPrepareVersion (#3588)

Co-authored-by: Anil Keshav <anil.keshav@sap.com>

* fix(whitesourceExecuteScan): docker image download if dockerConfigJSON is set in vault (#3591)

Co-authored-by: ffeldmann <f.feldmann@sap.com>

* checkmarxExecuteScan fixes (#3540)

* Fix FilterByTeamName and LoadExistingProject

* Fix project name loop

Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>

Co-authored-by: Daniel Bernd <93763187+danManSAP@users.noreply.github.com>
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
Co-authored-by: Siarhei Pazdniakou <siarhei.pazdniakou@sap.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Christian Volk <christian.volk@sap.com>
Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>
Co-authored-by: Jesse Awan <jesse.awan@sap.com>
Co-authored-by: thtrinh <trinhthanhhai@gmail.com>
Co-authored-by: Sumeet PATIL <sumeet.patil@sap.com>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Eugene Kortelyov <eugene.kortelyov@sap.com>
Co-authored-by: Michael <MichaelSp@users.noreply.github.com>
Co-authored-by: Anil Keshav <anil.keshav@sap.com>
Co-authored-by: anilkeshav27 <you@example.com>
Co-authored-by: ffeldmann <f.feldmann@sap.com>
Co-authored-by: Pavel Busko <pavel.busko@sap.com>
Co-authored-by: Maurice Breit <github@mauricebreit.de>
Co-authored-by: Adrien <99400874+hubadr@users.noreply.github.com>
Co-authored-by: Daniel Mieg <56156797+DanielMieg@users.noreply.github.com>
2022-03-01 09:23:11 +01:00
Anil Keshav
ed0359899f
fix (documentation) : mavenBuild and npmExecuteScripts (#3589)
* build depdencies docu fix

* fix trailing spaces

Co-authored-by: anilkeshav27 <you@example.com>
2022-02-28 15:08:49 +01:00
Adrien
a73951909b
checkmarxExecuteScan fixes (#3540)
* Fix FilterByTeamName and LoadExistingProject

* Fix project name loop

Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2022-02-28 14:22:47 +01:00
Christian Volk
421168a169
fix(whitesourceExecuteScan): docker image download if dockerConfigJSON is set in vault (#3591)
Co-authored-by: ffeldmann <f.feldmann@sap.com>
2022-02-28 13:57:23 +01:00
Maurice Breit
246d37a2fb
add access_token alias to artifactPrepareVersion (#3588)
Co-authored-by: Anil Keshav <anil.keshav@sap.com>
2022-02-28 13:00:02 +01:00
Eugene Kortelyov
8ced7f8184
Feature/fortify execute scan gradle (#3582)
* initial fortify gradle commit

* initial fortify gradle commit

Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-02-28 11:35:38 +01:00
Pavel Busko
a67b4ce558
feat(kubernetesDeploy): added valuesMapping config option (#3568) 2022-02-28 10:43:55 +01:00
Christian Volk
0596f25e84
fix(malwareExecuteScan): dont bind docker scanning to buildtool (#3547)
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-02-28 08:50:58 +01:00
Anil Keshav
9b9b507607
fix (mavenBuild) multi-module maven builds (#3587)
* moving jacoco agent preparation to after the goal flatten

* adding jacoco plugin to end

Co-authored-by: anilkeshav27 <you@example.com>
2022-02-25 19:20:21 +01:00
Michael
2c04f0acad
(fix) gitopsUpdateDeployment - integration test for kustomize (#3572)
* (fix) gitopsUpdateDeployment - integration test for kustomize

Signed-off-by: Michael Sprauer <Michael.Sprauer@sap.com>

* (fix) test: dash login shell is "-l"

Signed-off-by: Michael Sprauer <Michael.Sprauer@sap.com>

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-02-25 16:38:02 +01:00
thtrinh
d86cfce6e6
Checkmarx json report (#3565)
* feat(checkmarx) : Checkmarx JSON Report

* Test cases with some fix

* Information total and audited test assertions

* feat(checkmarx): align total/audited with existing calculation

* fix(checkmarx): Reporting unit test

Co-authored-by: Sumeet PATIL <sumeet.patil@sap.com>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-02-25 14:20:36 +01:00
Linda Siebert
8eb0c6aa48
Adjust sonarExecuteScan step (#3527)
* add check for instance parameter

* remove default from yaml

* adapt changes

* Update stashSettings.yml

* Unstash

* Add echo

* Stash changes

* Remove changes

* stash everything

* Change it up

* Change go

* Switch to new solution

* Try new stash

* Remove compliance

* Move Artifacts

* Change stashing

* Match others

* Remove space

* Test if warning goes away

Co-authored-by: Jesse Awan <jesse.awan@sap.com>
2022-02-25 13:09:50 +01:00
Christian Volk
956cab5de3
fix(kaniko): backwards compatibility with kaniko 1.3.0 (#3583)
* fix(kaniko): jnlp issues

* fix

* fix(kanikoExecute): backwards compatibility with kaniko 1.3.0

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-02-24 17:38:47 +01:00
Oliver Feldmann
b75db369d0
Translate verbose to sonar binary verbose flag (#3567)
* Translate Piper verbose to sonar bin flag

* Add test
2022-02-24 16:37:07 +01:00
Philipp Stehle
402b8b28fa
feat(kubernetesDeploy): initial support for multiple images (#3548)
Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
2022-02-24 12:01:36 +01:00
Ralf Pannemans
08bfe1554e
feat(cnbBuild) Introducing preserveFiles config to copy back files (#3562)
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
2022-02-23 18:54:59 +01:00
Philipp Stehle
4c18f2a128
feat: Add support for deprecating step parameters (#3554)
this was already used in fortifyExecuteScan, but had no effect.

Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>

Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
2022-02-23 15:16:05 +01:00
Christian Volk
d428cfcbd1
feat(kanikoExecute): image digests (#3566)
* feat(init stage): artifact version

* feat(kaniko): expose image digests to cpe

* fix

* handle tmp folder creation differently

* fix

* fix

* fix tests

* set ignore-path to /
2022-02-23 11:41:26 +01:00
RobertOzga87
9fa8390ef2
--ignore-path=/ in order to fix (#3570)
```
ERROR: Process exited immediately after creation. See output below
Executing /busybox/sh script inside container container-exec of pod dynamic-agent-...
```
On jenkins

Co-authored-by: Robert Ozga <robert.ozga@sap.com>
Co-authored-by: Christian Volk <christian.volk@sap.com>
2022-02-23 11:05:59 +01:00
Sven Merk
a1988f6808
feat(whitesourceExecuteScan): GitHub issue creation + SARIF (#3535)
* Add GH issue creation + SARIF

* Code cleanup

* Fix fmt, add debug

* Code enhancements

* Fix

* Added debug info

* Rework UA log scan

* Fix code

* read UA version

* Fix nil reference

* Extraction

* Credentials

* Issue creation

* Error handling

* Fix issue creation

* query escape

* Query escape 2

* Revert

* Test avoid update

* HTTP client

* Add support for custom TLS certs

* Fix code

* Fix code 2

* Fix code 3

* Disable cert check

* Fix auth

* Remove implicit trust

* Skip verification

* Fix

* Fix client

* Fix HTTP auth

* Fix trusted certs

* Trim version

* Code

* Add token

* Added token handling to client

* Fix token

* Cleanup

* Fix token

* Token rework

* Fix code

* Kick out oauth client

* Kick out oauth client

* Transport wrapping

* Token

* Simplification

* Refactor

* Variation

* Check

* Fix

* Debug

* Switch client

* Variation

* Debug

* Switch to cert check

* Add debug

* Parse self

* Cleanup

* Update resources/metadata/whitesourceExecuteScan.yaml

* Add debug

* Expose subjects

* Patch

* Debug

* Debug2

* Debug3

* Fix logging response body

* Cleanup

* Cleanup

* Fix request body logging

* Cleanup import

* Fix import cycle

* Cleanup

* Fix fmt

* Fix NopCloser reference

* Regenerate

* Reintroduce

* Fix test

* Fix tests

* Correction

* Fix error

* Code fix

* Fix tests

* Add tests

* Fix code climate issues

* Code climate

* Code climate again

* Code climate again

* Fix fmt

* Fix fmt 2

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-02-23 09:30:19 +01:00
ffeldmann
ffd4a7efb8
(fix) nil pointer dereference in case credentials are not available (#3564)
* adds return in gcs upload in case error occurs e.g. no credentials, avoid nil pointer dereference

* Adds generated files

* Updates generated files

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-02-22 18:32:09 +01:00
Rinita Asani
cd2ae89229
Local scopeg cts execute abap unit test (#3563)
* Adjust local changed objects with commit history

* fix the righ history entry

* fix for local packages commit history

* Adapt unit tests and fix for retry after 500

* Rename gctsExecuteABAPUnitTests to gctsExecuteABAPQualityChecks

* Add unit test file for gCTSExecuteABAPUnitTest

* add a test step

* gcts Test change files

* Add unit test for gCTS test

* adapt gctsTEST

* deletegcts

* command

* maintain old step

* Adapt changes to documentation

* fix for go generate
2022-02-22 09:58:22 +01:00
Oliver Nocon
f711925bc1
chore(cloudFoundryDeploy): deprecate deep config structure (#3486)
* chore(cloudFoundryDeploy): deprecate deep config structure

* Update cloudFoundryDeploy.yaml

* update generation
2022-02-21 15:08:07 +01:00
Siarhei Pazdniakou
4a4526b976
feat(npmExecuteScripts): add gcs upload to the step (#3428)
* Add gcs upload to npmExecuteScripts step

* go generate

* removed patterns related to sapGenerateEnvironmentInfo step

* patterns was updated

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-02-21 10:01:14 +01:00
Siarhei Pazdniakou
a059a41c68
feat(whitesourceExecuteScan): add gcs upload to the step (#3427)
* Add gcs upload to whitesourceExecuteScan step

* go generate

* patterns were updated

Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-02-21 09:51:52 +01:00
Siarhei Pazdniakou
8ac861061d
feat(protecodeExecuteScan): add gcs upload to the step (#3425)
* Add gcs upload to protecodeExecuteScan step

* patterns were updated

Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-02-21 09:31:43 +01:00
Siarhei Pazdniakou
98e28befe1
feat(fortifyExecuteScan): add gcs upload to the step (#3424)
* Add gcs upload to fortifyExecuteScan step

* go generate

* Patterns were updated

Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-02-21 09:23:54 +01:00
Oliver Nocon
6247c5dddd
feat(buildsettings): golang and future tools (#3561)
Co-authored-by: Christian Volk <christian.volk@sap.com>
2022-02-21 09:16:48 +01:00
Siarhei Pazdniakou
61a6309f02
feat(checkmarxExecuteScan): add gcs upload to the step (#3386)
* cumulus upload was integrated

* Patterns was updated

* go generate

* patterns were updated

Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-02-21 09:01:05 +01:00
Siarhei Pazdniakou
9994a4db45
feat(detectExecuteScan): add gcs upload to the step (#3423)
* Add gcs upload to detectExecuteScan step

* go generate

* patterns were updated

Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-02-21 08:07:55 +01:00
Christian Volk
65ab680f53
fix(golangBuild): remove system paths from the compiled executables (#3560) 2022-02-18 14:46:21 +01:00
Daniel Bernd
c829ac466a
Feature atc config - add Info Message for PatchIfExisting=false (#3553) 2022-02-18 11:11:33 +01:00
Michael
385038e652
(feat) gitopsUpdateDeployment supports globbing 🌟 (#3533)
* (feat) support for kustomize in gitopsUpdateDeployment step

Signed-off-by: Michael Sprauer <Michael.Sprauer@sap.com>

* add missing documentation

* add another detail in the documentation

Signed-off-by: Michael Sprauer <Michael.Sprauer@sap.com>

* generate again the update doc

Signed-off-by: Michael Sprauer <Michael.Sprauer@sap.com>

* (feat) gitopsUpdateDeployment now supports globbing

Signed-off-by: Michael Sprauer <Michael.Sprauer@sap.com>

* generate and fmt

Signed-off-by: Michael Sprauer <Michael.Sprauer@sap.com>

* fix tests

Signed-off-by: Michael Sprauer <Michael.Sprauer@sap.com>

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-02-18 08:43:34 +01:00
Christian Volk
20f5e955f9
Revert "feat(kaniko): pass image digests to cpe (#3542)" (#3556)
This reverts commit 2af9d3aae5.
2022-02-18 08:25:31 +01:00
Christian Volk
25abbe42cc
feat(golangBuild): upload reports to GCS (#3550)
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-02-17 17:56:45 +01:00
Oliver Nocon
a4a0873081
feat(checkmarx): create GitHub issue with findings (#3543)
* feat(checkmarx): create GitHub issue with findings

* add github issue reporting
2022-02-17 15:16:55 +01:00
Christian Volk
2af9d3aae5
feat(kaniko): pass image digests to cpe (#3542)
* feat(kaniko): expose image digests to cpe

* fix
2022-02-17 10:06:54 +01:00
Christian Volk
a1cd6cdb3a
fix(kaniko): set cpe consistently (#3545) 2022-02-17 08:20:30 +01:00
Ralf Pannemans
e7db09db12
feat(cnbBuild): added dockerimage to the telemetry data (#3501)
Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>
Co-authored-by: Sumit Kulhadia <sumit.kulhadia@sap.com>
Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
2022-02-16 13:28:51 +01:00
Ralf Pannemans
4b2f61589d
feat(cnbbuild) enable multi image build (#3521)
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
Co-authored-by: Pavel Busko <pavel.busko@sap.com>
Co-authored-by: Sumit Kulhadia <sumit.kulhadia@sap.com>
2022-02-15 14:39:14 +01:00
Johannes Dillmann
32acc9f91f
Collect buildTool in cnbBuild metrics (#3530) 2022-02-14 15:16:45 +01:00
Christian Volk
6a466b28fd
fix(golangBuild): cyclonedx-gomod import path (#3541)
* fix(golangBuild): cyclonedx-gomod import path

* fix tests
2022-02-14 12:07:25 +01:00
Daniel Bernd
dddb5992b9
Feature atc config - Correction in Batch (#3536)
* Checkin Go File - Push ATC Config

* ATC Push Config - Testfile

* Initial generated go files for ATC System Configuration

* ApiProxyUpload Command (#3295)

* ApiProxyUpload Command

* Code Review Fixes

* CodeReview Changes

* CodeReview Fixes

* YAML fixes

* CodeReview Fix

* Code Review Fixes

* CodeReview Fixes

* Code Climate Fixes

* Code Review Fixes

* Code Review Fixes

Co-authored-by: Roland Stengel <r.stengel@sap.com>
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>

* Fetch Xcrsf-Token

* Update  abapEnvironmentPushATCSystemConfig.yaml

* docs: fix urls (#3400)

* fix stash and adapt unit tests accordingly (#3403)

* Push ATC System Configuration - File read, send request, parse response

* fetchATCXcsrfToken

* fix(whitesourceExecuteScan): added missing credential declaration for the docker config (#3404)

* insert command in piper.go

* feat(cnbBuild): remove docker config after parsing (#3417)

Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>

* more funcs

* more funcs

* Revert "feat: Add debug output to getConfig (#3341)" (#3396)

This reverts commit b760bf48e7.

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

* feat (kubernetesDeploy) allow secret creation in cluster with kubectl as deploy tool (#3398)

* defaulting deocker config json location

* function change

* harmonising secret creation via json config apply

* adding the env path to kaniko

* env var

* adding path

* adding path

* adding path

* rolling back changes for path

* adapting condition for secret creation

* conditions based on username and password only

* fix unit test as per new secret creation

* update documentation with regards to secret creation

* fixing yaml lint empty line

* fixing trailing line from kaniko yaml

* error condition when path of the docker config json file is not found

Co-authored-by: anilkeshav27 <you@example.com>

* readPipelineEnv -> Serialize Directly to os.Stdout (#3180)

* Serialize CPE Directly to os.Stdout

* Response Error Handling 1

* Add easy mode for AUnit & ATC (#3389)

* remove mandatory flag from config files

* Enable repo.yml as config

* Adapt to merge

* Refactoring

* Refactoring

* avoid panic

* Add comments

* Add easy mode for atc

* Add tests

* Add test

* Refactor

* Add test for MPS

* Updates

* Rename functions

* Add files to gitignore

* Rename

* Renaming

* Renaming

* Renaming

* Improve error messages

* Update documentation

* Add logging

* Rename

* Extend gitignore

* fix(sonar): fix file patterns for gcs upload (#3406)

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* testing piper forks  (#3420)

* developer doku update on how to test with forked repos
Co-authored-by: rosemarieB <45030247+rosemarieB@users.noreply.github.com>

* Error Messages parsing

* Update yaml parameter

* feat (npmExecuteScripts) create seperate npmrc file for publish to private repo (#3422)

* creating new npm rc file

* publishing to registry staging

* exposing base64 version of env variables

* changing encoding param

* fixing unit test for the new path

* debugging env var

* remove debug message

* update docu

* changing new npmrc file name

* adding new npmrc to ignore

* adding new npmrc to ignore

Co-authored-by: anilkeshav27 <you@example.com>

* minor

* small changes

* small changes

* UNIT Test

* Unit Test 2

* Update .gitignore

Co-authored-by: Daniel Mieg <56156797+DanielMieg@users.noreply.github.com>

* Update abapEnvironmentAddons.md: Unable to update target vectors with unchanged SWC version (#3392)

* Update abapEnvironmentAddons.md

* Update documentation/docs/scenarios/abapEnvironmentAddons.md

Co-authored-by: Daniel Mieg <56156797+DanielMieg@users.noreply.github.com>

* Update abapEnvironmentAddons.md

Co-authored-by: Daniel Mieg <56156797+DanielMieg@users.noreply.github.com>

* Returning earlier & parseOdataResp 1

* new Parameter for overwrite existing

* feat(cnbBuild): preserve maven test results in the workspace (#3429)

Co-authored-by: Pavel Busko <pavel.busko@sap.com>
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>

* Update Introduction (#3433)

* Check Existence of configuration

* Quote strings in addon.yml (#3435)

* including vaultSecretFile to parameter generator docu (#3436)

Co-authored-by: anilkeshav27 <you@example.com>

* fix: wrong command line format for multiple extension files (#3434)

* feat(cnbBuild) containerImageName will be defaulted if possible (#3437)

Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
Co-authored-by: Sumit Kulhadia <sumit.kulhadia@sap.com>

* Corr. check existence

* check existence 2

* existence handling 2

* refinement existence handling & param rename

* config yaml change - no mandatory for bool param

* PATCH for existing Configuration

* fix: correct typo in log message (#3439)

Closes #3438

* feat(protecode): add versioning model (#3373)

* changes to detectExec before master merge

* changes for detectExecuteScan

* self generated code added

* fix syntax errors and update docu

* added unit tests for fail and Group

* fix failOn bug

* add Groups as string array

* add Groups as string array

* tests and validation for groups, failOn

* Updated docs and added more tests

* documentation md files should not be changed

* Handle merge conflicts from PR 1845

* fix merge errors

* remove duplicate groups, merge error

* adding buildCode and buildTool as params

* switching build options

* building maven modules

* parameter correction

* parameter correction

* gnerate with new build parameter

* adding comments

* removing piper lib master and modifying goUtils to download 1.5.7 release

* first cleaning then installing

* multi module maven built

* multi module maven built removing unwanted code

* multi module maven built moving inside switch

* testing

* modifying the default use case to also call maven build

* modifying the default use case to also call maven build wih --

* corrected maven build command

* corrected maven build command with %v

* skipping test runs

* testing for MTA project with single pom

* adding absolute path to m2 path

* clean up

* adding switch for mta and maven and removing env from containers

* commiting changes for new detect step

* correting log message

* code clean up

* unit tests changes to detectExecute

* basic tests for new change

* restoring piperGoUtils to download correct piper binary

* code clean up

* code clean up

* protecodeExecuteScan :: versioning model draft - 1

* protecodeExecuteScan :: version model draft-2

* protecodeExecuteScan :: changing filename and version concatenation

* protecodeExecuteScan :: update documentation

* protecodeExecuteScan :: double URL encoding has been corrected & console messaging improved

* protecodeExecuteScan :: fixed Go/generate validation fail

* protecodeExecuteScan :: fixing failed unit tests

* protecodeExecuteScan :: Version field added

* protecodeExecuteScan :: Version field add => minor changes

* protecodeExecuteScan :: Version field add => fixing tests

Co-authored-by: D072410 <giridhar.shenoy@sap.com>
Co-authored-by: Keshav <anil.keshav@sap.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>

* GCS Upload fixes (#3387)

* Fixed argument type in persist function

* Fixed gcp upload to be usable in internal piper

* Fixed import of packages

* Updated logs

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

* Unit Test 2

* Unit Tests

* Add gcs upload to mavenExecuteIntegration step (#3432)

* Add gcs upload to mavenExecuteIntegration step

* go generate

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

* Check times refine

* Permwrite 0644

* disable http retry (#3447)

* Update maxWaitTime timeout for WhiteSource (#3284)

* Update scanPolling.go

Changing maxWaitTime from 15 to 30 to overcome WhiteSource results reflection in the backend issue.

* Update configHelper.go

* Reset configHelper changes to fix PR 3284

 Committer: raghunathd8

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: raghunathd8 <root@docker-evaluation.openstack.eu-nl-1.cloud.sap>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>

* removed trailing spaces

* yaml Update

* minor mock for token & Unit Tests

* rename Parameter Patch

* lil refactor & Unit Test

* fixes PATCH & more Unit Tests

* add log at certain points

* not writing configuration file back

* feat(malwareExecuteScan): refactoring and docker support (#3421)

* feat(malwareExecuteScan): add support for scanning docker images

* refactoring

* print out finding if available

* generate toolrecord for malware scan

* persist scan report

* docs

* fix

* fix

* rollback cmd/init_unix.go

* auhenticated pull

* fix

* fix: report shall be consistent with the api model

* gcs upload

* fix linter

* Fix potential nil reference (#3460)

* Add gcs upload to karmaExecuteTests step (#3431)

* Add gcs upload to karmaExecuteTests step

* go generate

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

* Add gcs upload to gaugeExecuteTests step (#3410)

* Add gcs upload to gaugeExecuteTests step

* go generate

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

* Add gcs upload to newmanExecute step (#3408)

* Add gcs upload to newmanExecute step

* go generate

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

* feat(GitHub): fetch commit statistics (#3381)

* fetch GH statistics

* move GH and Sonar integration tests to own files

* fix imports

* add integration test case

* add result type

* Apply suggestions from code review

* feat: add command to fetch default files (#3445)

* some ideas..

* Add getDefaults command (WIP) (#3444)

* add getYAML function for configs

* create getDefaults command(based on getConfig)

* add getDefaults command to CLI

* read defaults files, using github tokens as well

* write defaults to stdout as JSON object with YAMLs embedded

* catch case where no input files are given

* actually write output to file if outputFile is specified

* mark defaultsFile flag as required

* add basic tests

* add output (string)  test

* adapt generateDefaults() to return output (used for test of previous commit)

* Changes to getDefaults() JSON output (#3449)

* change JSON output to contain separate fields

* filename -> filepath

* Apply suggestions from code review

* Apply suggestions from code review

* Update pkg/config/config_test.go

Co-authored-by: Jordi van Liempt <35920075+jliempt@users.noreply.github.com>

* BATCH request preparation

* Add gcs upload to mtaBuild step (#3405)

* Add gcs upload to mtaBuild

* Patterns was updated

* go generate

* removed patterns related to sapGenerateEnvironmentInfo step

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

* Add gcs upload to mavenBuild step (#3394)

* Add gcs upload to mavenBuild step

* Patterns were updated

* go generate

* removed patterns related to sapGenerateEnvironmentInfo step

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

* feat(gradleExecuteBuild): add stage scope to path parameter, fix project dir exist issue (#3401)

* add stage scope to path parameter, fix project dir exist issue

* fix unit test for gradleExecuteBuild

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

* Add gcs upload to uiVeri5ExecuteTests step (#3409)

* Add gcs upload to uiVeri5ExecuteTests step

* go generate

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

* Address further nil references (#3462)

* Address further nil references

* Message text

* Final checks

* fix (npmExecuteScripts) allow npm pack before npm publish (#3455)

* adding config to piperNpmr

* scope in cli

* adding scope to repo url and npmrc

* publish to scoped

* removing scope

* changing scope position

* adding scope to userconfig

* adding registry=

* pack and then tar

* not removing tmp folder

* adding flag

* pack before publish

* adding log

* debug

* debug with change directory

* publishing created tar ball

* debug

* üath

* adding main npmrc

* renaming old npmrc file

* error renaming old npmrc file

* renaming err

* correcting npmrc file path

* renaming file back to original

* current working directory

* renaming the npmrc file

* avoiding change directory

* with current working dir

* adding dot

* renaming npmrc and defer removal

* rename files

* Update pkg/npm/publish.go

* Update pkg/npm/publish.go

Co-authored-by: anilkeshav27 <you@example.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

* Add testing for helm during acceptance stage (#3402)

* Add kubernetesDeploy to Acceptance

* Add more kubernetesDeploy

* Add helm tests

* Change documentation

* Fix docu

* Change generated

* Add tests

* Add groovy tests

* Fix tests

* Change tests

Co-authored-by: Thorsten Duda <thorsten.duda@sap.com>

* Batch Request for Patches

* Fix(gctsDeploy) : add client in config url, disable retry for create and pull (#3464)

* minor Unit Test correction

* lil cleanup

* lil refactoring

* removed nuneeded linebreaks

* refactoring Command on Entity in Batch

* some Unit Tests for Build Batch Request

* more Unittests - Build Batch Request

* fix(fortifyExecuteScan): User assignment based on PR ownership (#3472)

* Debug PR user details

* Check association

* Change to login

* Fix PR creator assignment

* Improve docs

* Fix test

* Update gradle version (#3476)

* Update gradle version

* Update generated file

* small correction Text & Test

* chore(kubernetesDeploy): add telemetry for deployTool (#3469)

* chore(kubernetesDeploy): add telemetry for deployTool

* Update cmd/kubernetesDeploy.go

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

* rename Parameter patchIfExisting

* fix (mtaBuild) keep mtar artifact name in synch with maven gav (#3454)

* keep mtar name in synch with maven gav

* retaining slashes

* keeping dots in artifact name

Co-authored-by: anilkeshav27 <you@example.com>

* Update image for gradle build (#3479)

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

* Last Changed not Zero check added

* Added pseudo_comment_policy in struct

* Unit Test adaption

* smaller refactoring, Unit Test, removed doubles

* fix(cnbBuild) do not add complete path to telemetry (#3487)

Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>

* Add limitation that table keys are exported generically (#3490)

* Add limitation that table keys are exported generically

* Fix formatting

* Clarify delta calculation of different package types (#3482)

Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com>

* Add troubleshooting for missing add-on registration (#3491)

Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com>

* feat (Documentation) documentation for mavenBuild and npmExecuteScripts that consume dependencies from a private repo. (#3484)

* build depdency docu for maven and npm

* removing trailing white space in yaml

* Update resources/metadata/mavenBuild.yaml

* relative url for vault and mta docu

* running go generate

* keeping vault relative path

* go generate

* reverting to global paths

* go generate

* wild card for a higher level dir

* searching on top folder only

* relative level above

* pointing to infrastructure

* correcting links

Co-authored-by: anilkeshav27 <you@example.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

* fix(kanikoExecute): tmp dir (#3478)

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

* feat(golangBuild): add support for publishing binaries (#3495)

* fix(golangBuild): keep track of the artifactversion in binary names (#3498)

* fix(golangBuild): keep track of the artifactversion in binary names

* improve the naming scheme

* fix (build dependency) typo corrections (#3494)

* typo corrections

* trailing white space

* go generate

Co-authored-by: anilkeshav27 <you@example.com>

* Revert "fix(golangBuild): keep track of the artifactversion in binary names (#3498)" (#3499)

This reverts commit ec783b0da6.

* sonarqube coverage: additional metrics (#3465)

* sonarqube coverage: additional metrics

* sonarExecuteScan: add lines of code and language distribution to sonarscan.json

* sonarExecuteScan: consider branch in componentService requests

* SonarQube: Do not omit empty values in SonarCoverage

* sonarExecuteScan: Add integration tests for ComponentService getLinesOfCode

* fix tests

* sonarExecuteScan: use pullRequest in componentService

Co-authored-by: I550025 <r.kloe@sap.com>
Co-authored-by: Marc Bormeth <marc.bormeth@sap.com>

* fix(golangBuild): targetRepositoryUrl (#3502)

* fix(golangBuild): targetRepositoryUrl

* regen

* fix(httpclient): folder permissions (#3506)

* fix(golangBuild): use PUT instead of POST (#3504)

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

* feat(golangBuild): publish to subfolder (#3503)

* feat(golangBuild): publish to subfolder

* edge case handling

* fix

* fix(golangBuild): expect status 201 instead of 200 (#3508)

* feat(kanikoExecute): allow building multiple images (#3443)

* feat(kanikoExecute): allow building multiple images

* enhance tests

* chore: allow running tests in parallel

* small fixes

* fix: fix destination bug

* update formatting and defaults

* fix yml formatting

* chore: change cpe parameter names

* chore: improve variable naming

* Changes GetJobName value to repositoryname (#3509)

* Update npmExecuteScripts step (#3211)

* Update npmExecuteScripts step

* Fixed failing build

* Fixed path issue

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Christian Volk <christian.volk@sap.com>

* Ignore violations if failOn is NONE (#3513)

* fix(sonarExecuteScan): error handling for report metrics (#3512)

* Generic step   input from addon descriptor (#3467)

* enhance build step to run based on addon.yml in steampunk environment

Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com>

* feat(golangBuild): add BOM creation (#3496)

* Added BOM creation

* Fix cyclonedx install

* undo config.yml changes

* metadata was updated

* Update golangBuild.go

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

* If statement dearrangement

* Correction in Batch - ConfUUID

* dummy

Co-authored-by: Daniel Bernd <93763187+danManSAP@users.noreply.github.com>
Co-authored-by: Mayur Belur Mohan <68264167+mayurmohan@users.noreply.github.com>
Co-authored-by: Roland Stengel <r.stengel@sap.com>
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
Co-authored-by: Jordi van Liempt <35920075+jliempt@users.noreply.github.com>
Co-authored-by: Jesse Awan <jesse.awan@sap.com>
Co-authored-by: Pavel Busko <pavel.busko@sap.com>
Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>
Co-authored-by: Marc Bormeth <marc.bormeth@sap.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Anil Keshav <anil.keshav@sap.com>
Co-authored-by: anilkeshav27 <you@example.com>
Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com>
Co-authored-by: Daniel Mieg <56156797+DanielMieg@users.noreply.github.com>
Co-authored-by: Siarhei Pazdniakou <siarhei.pazdniakou@sap.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Peter Persiel <6087940+peterpersiel@users.noreply.github.com>
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
Co-authored-by: Marcus Holl <marcus.holl@sap.com>
Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
Co-authored-by: Sumit Kulhadia <sumit.kulhadia@sap.com>
Co-authored-by: Nikolay Grechanov <nikolay.grechanov@sap.com>
Co-authored-by: Umidjon Urunov <79094563+UmidjonUrunov@users.noreply.github.com>
Co-authored-by: D072410 <giridhar.shenoy@sap.com>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Sarat Krishnan <78093145+sarat-krk@users.noreply.github.com>
Co-authored-by: Raghunath Deshpande <raghunath.deshpande@sap.com>
Co-authored-by: raghunathd8 <root@docker-evaluation.openstack.eu-nl-1.cloud.sap>
Co-authored-by: Christian Volk <christian.volk@sap.com>
Co-authored-by: Eugene Kortelyov <eugene.kortelyov@sap.com>
Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>
Co-authored-by: Thorsten Duda <thorsten.duda@sap.com>
Co-authored-by: Matthias Scudlik <matthias.scudlik@gmail.com>
Co-authored-by: I550025 <r.kloe@sap.com>
Co-authored-by: ffeldmann <f.feldmann@sap.com>
Co-authored-by: ManjunathMS35 <manjunathms35@gmail.com>
Co-authored-by: Matthias Scudlik <matthias.scudlik@sap.com>
Co-authored-by: rosemarieB <45030247+rosemarieB@users.noreply.github.com>
2022-02-14 11:04:44 +01:00
Michael
1ea965ae69
(feat) support for kustomize in gitopsUpdateDeployment step (#3524)
* (feat) support for kustomize in gitopsUpdateDeployment step

Signed-off-by: Michael Sprauer <Michael.Sprauer@sap.com>

* add missing documentation

* add another detail in the documentation

Signed-off-by: Michael Sprauer <Michael.Sprauer@sap.com>

* generate again the update doc

Signed-off-by: Michael Sprauer <Michael.Sprauer@sap.com>

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-02-14 07:45:54 +01:00
Daniel Mieg
f08ff92171
New log entities for Pull & Clone (#3517)
* WIP

* New Logs

* Improving

* Determine log output based on available entities

* Increase width

* Add line

* Adapt TestPollEntity

* Format

* Fix query

* Adapt tests

* Fix test

* Improve formatting

* Retern early in case of no logs

* Remove duplicate log
2022-02-11 10:16:40 +01:00
Marcus Holl
be10e14c88
fix replacement of credentials placeholders in extension files for cf-deploy (#3532)
fix replacement of credentials placeholders in extension files for cf-deploy
2022-02-11 09:36:19 +01:00
Christian Volk
c888e21e6c
chore(multiarch): helper to parse targetArchitectures (#3525)
* chore(docker): helper to parse targetArchitectures

* missing files
2022-02-10 16:46:00 +01:00
Vitalii Sidorov
93e3801945
feat: add step for Helm execution (#3419)
* Implement helm step

* Create kubernetes package

* Refactoring helm.go

* Add package, test commands

* Add test for helm package

* Add tests for helm.go

* Add tests for helm.go

* Add tests for utils.go

* Add tests for helmExecute.go

* small fix

* Add helm lint

* small fix

* small fix

* Fix according to comments

* Fix test

* small fix

* Add helm add function

* Changes according to new comments

* Add helm push

* Add unit tests

* Add tests for helmExecute

* Add small fix

* small fix

* small fix

* Move DeployUtilsBundle from kubernetesDeploy to kubernetes package

* small fix

* small fix

* Add unit-tests

* Fix

* Update resources/metadata/helmExecute.yaml

* Update resources/metadata/helmExecute.yaml

* Add helm chart server parameterization

* small fix

* small fix

Co-authored-by: “Vitalii <“vitalii.sidorov@sap.com”>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-02-10 10:25:03 +01:00
Oliver Nocon
0f26838f0a
fix(kanikoExecute): use repository root as context (#3522) 2022-02-09 12:36:43 +01:00
Christian Volk
5aa9661db6
feat(kanikoExecute): preparation for multi-arch builds (#3515)
* feat(kanikoExecute): preparation for multi-arch builds

* missing files
2022-02-09 10:17:34 +01:00
Marcus Holl
5ad6d0a20a
mtaExtensionCredentials: docu, logging, less strict replacements (#3489)
* Enhance doku for mtaExtensionCredentials for cloudFoundryDeploy

* Error message starting with lower case

* Log entry in case a variable is not used.

* Be less strict when replacing placeholders

* Check the variable name since we use now regex

* fix error message in assert

* simplify check for not replaced placeholders

* report unresolved

* report updated

* test updated=false

* 'do not' instead of 'does not' (test name)

* wip

* content not shared between tests

* rework parameter docu

* fix missing trailing quotation mark in test content

* Adjust quantifier when looking for not replaced placeholders

* fmt
2022-02-09 09:33:12 +01:00
Daniel Bernd
0d8bd7dc52
Add step abapEnvironmentPushATCSystemConfig (#3426)
* Checkin Go File - Push ATC Config

* ATC Push Config - Testfile

* Initial generated go files for ATC System Configuration

* ApiProxyUpload Command (#3295)

* ApiProxyUpload Command

* Code Review Fixes

* CodeReview Changes

* CodeReview Fixes

* YAML fixes

* CodeReview Fix

* Code Review Fixes

* CodeReview Fixes

* Code Climate Fixes

* Code Review Fixes

* Code Review Fixes

Co-authored-by: Roland Stengel <r.stengel@sap.com>
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>

* Fetch Xcrsf-Token

* Update  abapEnvironmentPushATCSystemConfig.yaml

* docs: fix urls (#3400)

* fix stash and adapt unit tests accordingly (#3403)

* Push ATC System Configuration - File read, send request, parse response

* fetchATCXcsrfToken

* fix(whitesourceExecuteScan): added missing credential declaration for the docker config (#3404)

* insert command in piper.go

* feat(cnbBuild): remove docker config after parsing (#3417)

Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>

* more funcs

* more funcs

* Revert "feat: Add debug output to getConfig (#3341)" (#3396)

This reverts commit b760bf48e7.

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

* feat (kubernetesDeploy) allow secret creation in cluster with kubectl as deploy tool (#3398)

* defaulting deocker config json location

* function change

* harmonising secret creation via json config apply

* adding the env path to kaniko

* env var

* adding path

* adding path

* adding path

* rolling back changes for path

* adapting condition for secret creation

* conditions based on username and password only

* fix unit test as per new secret creation

* update documentation with regards to secret creation

* fixing yaml lint empty line

* fixing trailing line from kaniko yaml

* error condition when path of the docker config json file is not found

Co-authored-by: anilkeshav27 <you@example.com>

* readPipelineEnv -> Serialize Directly to os.Stdout (#3180)

* Serialize CPE Directly to os.Stdout

* Response Error Handling 1

* Add easy mode for AUnit & ATC (#3389)

* remove mandatory flag from config files

* Enable repo.yml as config

* Adapt to merge

* Refactoring

* Refactoring

* avoid panic

* Add comments

* Add easy mode for atc

* Add tests

* Add test

* Refactor

* Add test for MPS

* Updates

* Rename functions

* Add files to gitignore

* Rename

* Renaming

* Renaming

* Renaming

* Improve error messages

* Update documentation

* Add logging

* Rename

* Extend gitignore

* fix(sonar): fix file patterns for gcs upload (#3406)

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* testing piper forks  (#3420)

* developer doku update on how to test with forked repos
Co-authored-by: rosemarieB <45030247+rosemarieB@users.noreply.github.com>

* Error Messages parsing

* Update yaml parameter

* feat (npmExecuteScripts) create seperate npmrc file for publish to private repo (#3422)

* creating new npm rc file

* publishing to registry staging

* exposing base64 version of env variables

* changing encoding param

* fixing unit test for the new path

* debugging env var

* remove debug message

* update docu

* changing new npmrc file name

* adding new npmrc to ignore

* adding new npmrc to ignore

Co-authored-by: anilkeshav27 <you@example.com>

* minor

* small changes

* small changes

* UNIT Test

* Unit Test 2

* Update .gitignore

Co-authored-by: Daniel Mieg <56156797+DanielMieg@users.noreply.github.com>

* Update abapEnvironmentAddons.md: Unable to update target vectors with unchanged SWC version (#3392)

* Update abapEnvironmentAddons.md

* Update documentation/docs/scenarios/abapEnvironmentAddons.md

Co-authored-by: Daniel Mieg <56156797+DanielMieg@users.noreply.github.com>

* Update abapEnvironmentAddons.md

Co-authored-by: Daniel Mieg <56156797+DanielMieg@users.noreply.github.com>

* Returning earlier & parseOdataResp 1

* new Parameter for overwrite existing

* feat(cnbBuild): preserve maven test results in the workspace (#3429)

Co-authored-by: Pavel Busko <pavel.busko@sap.com>
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>

* Update Introduction (#3433)

* Check Existence of configuration

* Quote strings in addon.yml (#3435)

* including vaultSecretFile to parameter generator docu (#3436)

Co-authored-by: anilkeshav27 <you@example.com>

* fix: wrong command line format for multiple extension files (#3434)

* feat(cnbBuild) containerImageName will be defaulted if possible (#3437)

Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
Co-authored-by: Sumit Kulhadia <sumit.kulhadia@sap.com>

* Corr. check existence

* check existence 2

* existence handling 2

* refinement existence handling & param rename

* config yaml change - no mandatory for bool param

* PATCH for existing Configuration

* fix: correct typo in log message (#3439)

Closes #3438

* feat(protecode): add versioning model (#3373)

* changes to detectExec before master merge

* changes for detectExecuteScan

* self generated code added

* fix syntax errors and update docu

* added unit tests for fail and Group

* fix failOn bug

* add Groups as string array

* add Groups as string array

* tests and validation for groups, failOn

* Updated docs and added more tests

* documentation md files should not be changed

* Handle merge conflicts from PR 1845

* fix merge errors

* remove duplicate groups, merge error

* adding buildCode and buildTool as params

* switching build options

* building maven modules

* parameter correction

* parameter correction

* gnerate with new build parameter

* adding comments

* removing piper lib master and modifying goUtils to download 1.5.7 release

* first cleaning then installing

* multi module maven built

* multi module maven built removing unwanted code

* multi module maven built moving inside switch

* testing

* modifying the default use case to also call maven build

* modifying the default use case to also call maven build wih --

* corrected maven build command

* corrected maven build command with %v

* skipping test runs

* testing for MTA project with single pom

* adding absolute path to m2 path

* clean up

* adding switch for mta and maven and removing env from containers

* commiting changes for new detect step

* correting log message

* code clean up

* unit tests changes to detectExecute

* basic tests for new change

* restoring piperGoUtils to download correct piper binary

* code clean up

* code clean up

* protecodeExecuteScan :: versioning model draft - 1

* protecodeExecuteScan :: version model draft-2

* protecodeExecuteScan :: changing filename and version concatenation

* protecodeExecuteScan :: update documentation

* protecodeExecuteScan :: double URL encoding has been corrected & console messaging improved

* protecodeExecuteScan :: fixed Go/generate validation fail

* protecodeExecuteScan :: fixing failed unit tests

* protecodeExecuteScan :: Version field added

* protecodeExecuteScan :: Version field add => minor changes

* protecodeExecuteScan :: Version field add => fixing tests

Co-authored-by: D072410 <giridhar.shenoy@sap.com>
Co-authored-by: Keshav <anil.keshav@sap.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>

* GCS Upload fixes (#3387)

* Fixed argument type in persist function

* Fixed gcp upload to be usable in internal piper

* Fixed import of packages

* Updated logs

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

* Unit Test 2

* Unit Tests

* Add gcs upload to mavenExecuteIntegration step (#3432)

* Add gcs upload to mavenExecuteIntegration step

* go generate

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

* Check times refine

* Permwrite 0644

* disable http retry (#3447)

* Update maxWaitTime timeout for WhiteSource (#3284)

* Update scanPolling.go

Changing maxWaitTime from 15 to 30 to overcome WhiteSource results reflection in the backend issue.

* Update configHelper.go

* Reset configHelper changes to fix PR 3284

 Committer: raghunathd8

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: raghunathd8 <root@docker-evaluation.openstack.eu-nl-1.cloud.sap>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>

* removed trailing spaces

* yaml Update

* minor mock for token & Unit Tests

* rename Parameter Patch

* lil refactor & Unit Test

* fixes PATCH & more Unit Tests

* add log at certain points

* not writing configuration file back

* feat(malwareExecuteScan): refactoring and docker support (#3421)

* feat(malwareExecuteScan): add support for scanning docker images

* refactoring

* print out finding if available

* generate toolrecord for malware scan

* persist scan report

* docs

* fix

* fix

* rollback cmd/init_unix.go

* auhenticated pull

* fix

* fix: report shall be consistent with the api model

* gcs upload

* fix linter

* Fix potential nil reference (#3460)

* Add gcs upload to karmaExecuteTests step (#3431)

* Add gcs upload to karmaExecuteTests step

* go generate

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

* Add gcs upload to gaugeExecuteTests step (#3410)

* Add gcs upload to gaugeExecuteTests step

* go generate

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

* Add gcs upload to newmanExecute step (#3408)

* Add gcs upload to newmanExecute step

* go generate

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

* feat(GitHub): fetch commit statistics (#3381)

* fetch GH statistics

* move GH and Sonar integration tests to own files

* fix imports

* add integration test case

* add result type

* Apply suggestions from code review

* feat: add command to fetch default files (#3445)

* some ideas..

* Add getDefaults command (WIP) (#3444)

* add getYAML function for configs

* create getDefaults command(based on getConfig)

* add getDefaults command to CLI

* read defaults files, using github tokens as well

* write defaults to stdout as JSON object with YAMLs embedded

* catch case where no input files are given

* actually write output to file if outputFile is specified

* mark defaultsFile flag as required

* add basic tests

* add output (string)  test

* adapt generateDefaults() to return output (used for test of previous commit)

* Changes to getDefaults() JSON output (#3449)

* change JSON output to contain separate fields

* filename -> filepath

* Apply suggestions from code review

* Apply suggestions from code review

* Update pkg/config/config_test.go

Co-authored-by: Jordi van Liempt <35920075+jliempt@users.noreply.github.com>

* BATCH request preparation

* Add gcs upload to mtaBuild step (#3405)

* Add gcs upload to mtaBuild

* Patterns was updated

* go generate

* removed patterns related to sapGenerateEnvironmentInfo step

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

* Add gcs upload to mavenBuild step (#3394)

* Add gcs upload to mavenBuild step

* Patterns were updated

* go generate

* removed patterns related to sapGenerateEnvironmentInfo step

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

* feat(gradleExecuteBuild): add stage scope to path parameter, fix project dir exist issue (#3401)

* add stage scope to path parameter, fix project dir exist issue

* fix unit test for gradleExecuteBuild

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

* Add gcs upload to uiVeri5ExecuteTests step (#3409)

* Add gcs upload to uiVeri5ExecuteTests step

* go generate

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

* Address further nil references (#3462)

* Address further nil references

* Message text

* Final checks

* fix (npmExecuteScripts) allow npm pack before npm publish (#3455)

* adding config to piperNpmr

* scope in cli

* adding scope to repo url and npmrc

* publish to scoped

* removing scope

* changing scope position

* adding scope to userconfig

* adding registry=

* pack and then tar

* not removing tmp folder

* adding flag

* pack before publish

* adding log

* debug

* debug with change directory

* publishing created tar ball

* debug

* üath

* adding main npmrc

* renaming old npmrc file

* error renaming old npmrc file

* renaming err

* correcting npmrc file path

* renaming file back to original

* current working directory

* renaming the npmrc file

* avoiding change directory

* with current working dir

* adding dot

* renaming npmrc and defer removal

* rename files

* Update pkg/npm/publish.go

* Update pkg/npm/publish.go

Co-authored-by: anilkeshav27 <you@example.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

* Add testing for helm during acceptance stage (#3402)

* Add kubernetesDeploy to Acceptance

* Add more kubernetesDeploy

* Add helm tests

* Change documentation

* Fix docu

* Change generated

* Add tests

* Add groovy tests

* Fix tests

* Change tests

Co-authored-by: Thorsten Duda <thorsten.duda@sap.com>

* Batch Request for Patches

* Fix(gctsDeploy) : add client in config url, disable retry for create and pull (#3464)

* minor Unit Test correction

* lil cleanup

* lil refactoring

* removed nuneeded linebreaks

* refactoring Command on Entity in Batch

* some Unit Tests for Build Batch Request

* more Unittests - Build Batch Request

* fix(fortifyExecuteScan): User assignment based on PR ownership (#3472)

* Debug PR user details

* Check association

* Change to login

* Fix PR creator assignment

* Improve docs

* Fix test

* Update gradle version (#3476)

* Update gradle version

* Update generated file

* small correction Text & Test

* chore(kubernetesDeploy): add telemetry for deployTool (#3469)

* chore(kubernetesDeploy): add telemetry for deployTool

* Update cmd/kubernetesDeploy.go

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

* rename Parameter patchIfExisting

* fix (mtaBuild) keep mtar artifact name in synch with maven gav (#3454)

* keep mtar name in synch with maven gav

* retaining slashes

* keeping dots in artifact name

Co-authored-by: anilkeshav27 <you@example.com>

* Update image for gradle build (#3479)

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

* Last Changed not Zero check added

* Added pseudo_comment_policy in struct

* Unit Test adaption

* smaller refactoring, Unit Test, removed doubles

* fix(cnbBuild) do not add complete path to telemetry (#3487)

Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>

* Add limitation that table keys are exported generically (#3490)

* Add limitation that table keys are exported generically

* Fix formatting

* Clarify delta calculation of different package types (#3482)

Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com>

* Add troubleshooting for missing add-on registration (#3491)

Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com>

* feat (Documentation) documentation for mavenBuild and npmExecuteScripts that consume dependencies from a private repo. (#3484)

* build depdency docu for maven and npm

* removing trailing white space in yaml

* Update resources/metadata/mavenBuild.yaml

* relative url for vault and mta docu

* running go generate

* keeping vault relative path

* go generate

* reverting to global paths

* go generate

* wild card for a higher level dir

* searching on top folder only

* relative level above

* pointing to infrastructure

* correcting links

Co-authored-by: anilkeshav27 <you@example.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

* fix(kanikoExecute): tmp dir (#3478)

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

* feat(golangBuild): add support for publishing binaries (#3495)

* fix(golangBuild): keep track of the artifactversion in binary names (#3498)

* fix(golangBuild): keep track of the artifactversion in binary names

* improve the naming scheme

* fix (build dependency) typo corrections (#3494)

* typo corrections

* trailing white space

* go generate

Co-authored-by: anilkeshav27 <you@example.com>

* Revert "fix(golangBuild): keep track of the artifactversion in binary names (#3498)" (#3499)

This reverts commit ec783b0da6.

* sonarqube coverage: additional metrics (#3465)

* sonarqube coverage: additional metrics

* sonarExecuteScan: add lines of code and language distribution to sonarscan.json

* sonarExecuteScan: consider branch in componentService requests

* SonarQube: Do not omit empty values in SonarCoverage

* sonarExecuteScan: Add integration tests for ComponentService getLinesOfCode

* fix tests

* sonarExecuteScan: use pullRequest in componentService

Co-authored-by: I550025 <r.kloe@sap.com>
Co-authored-by: Marc Bormeth <marc.bormeth@sap.com>

* fix(golangBuild): targetRepositoryUrl (#3502)

* fix(golangBuild): targetRepositoryUrl

* regen

* fix(httpclient): folder permissions (#3506)

* fix(golangBuild): use PUT instead of POST (#3504)

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

* feat(golangBuild): publish to subfolder (#3503)

* feat(golangBuild): publish to subfolder

* edge case handling

* fix

* fix(golangBuild): expect status 201 instead of 200 (#3508)

* feat(kanikoExecute): allow building multiple images (#3443)

* feat(kanikoExecute): allow building multiple images

* enhance tests

* chore: allow running tests in parallel

* small fixes

* fix: fix destination bug

* update formatting and defaults

* fix yml formatting

* chore: change cpe parameter names

* chore: improve variable naming

* Changes GetJobName value to repositoryname (#3509)

* Update npmExecuteScripts step (#3211)

* Update npmExecuteScripts step

* Fixed failing build

* Fixed path issue

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Christian Volk <christian.volk@sap.com>

* Ignore violations if failOn is NONE (#3513)

* fix(sonarExecuteScan): error handling for report metrics (#3512)

* Generic step   input from addon descriptor (#3467)

* enhance build step to run based on addon.yml in steampunk environment

Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com>

* feat(golangBuild): add BOM creation (#3496)

* Added BOM creation

* Fix cyclonedx install

* undo config.yml changes

* metadata was updated

* Update golangBuild.go

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

* If statement dearrangement

Co-authored-by: Daniel Bernd <93763187+danManSAP@users.noreply.github.com>
Co-authored-by: Mayur Belur Mohan <68264167+mayurmohan@users.noreply.github.com>
Co-authored-by: Roland Stengel <r.stengel@sap.com>
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
Co-authored-by: Jordi van Liempt <35920075+jliempt@users.noreply.github.com>
Co-authored-by: Jesse Awan <jesse.awan@sap.com>
Co-authored-by: Pavel Busko <pavel.busko@sap.com>
Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>
Co-authored-by: Marc Bormeth <marc.bormeth@sap.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Anil Keshav <anil.keshav@sap.com>
Co-authored-by: anilkeshav27 <you@example.com>
Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com>
Co-authored-by: Daniel Mieg <56156797+DanielMieg@users.noreply.github.com>
Co-authored-by: Siarhei Pazdniakou <siarhei.pazdniakou@sap.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Peter Persiel <6087940+peterpersiel@users.noreply.github.com>
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
Co-authored-by: Marcus Holl <marcus.holl@sap.com>
Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
Co-authored-by: Sumit Kulhadia <sumit.kulhadia@sap.com>
Co-authored-by: Nikolay Grechanov <nikolay.grechanov@sap.com>
Co-authored-by: Umidjon Urunov <79094563+UmidjonUrunov@users.noreply.github.com>
Co-authored-by: D072410 <giridhar.shenoy@sap.com>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Sarat Krishnan <78093145+sarat-krk@users.noreply.github.com>
Co-authored-by: Raghunath Deshpande <raghunath.deshpande@sap.com>
Co-authored-by: raghunathd8 <root@docker-evaluation.openstack.eu-nl-1.cloud.sap>
Co-authored-by: Christian Volk <christian.volk@sap.com>
Co-authored-by: Eugene Kortelyov <eugene.kortelyov@sap.com>
Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>
Co-authored-by: Thorsten Duda <thorsten.duda@sap.com>
Co-authored-by: Matthias Scudlik <matthias.scudlik@gmail.com>
Co-authored-by: I550025 <r.kloe@sap.com>
Co-authored-by: ffeldmann <f.feldmann@sap.com>
Co-authored-by: ManjunathMS35 <manjunathms35@gmail.com>
Co-authored-by: Matthias Scudlik <matthias.scudlik@sap.com>
Co-authored-by: rosemarieB <45030247+rosemarieB@users.noreply.github.com>
2022-02-08 16:14:29 +01:00
xgoffin
2cebf370c9
feat(fortifyExecuteScan): added conversion to SARIF for FPR files (#3485)
* feat(FPRtoSARIF): boilerplate & comments

* Feat(Ingest): Build done, Vulnerabilities partway

* feat(Vulnerabilities): now entirely parsed

* feat(Ingestion): handle Description object

* feat(FprToSarif): integration in Piper step, full xml structure

* feat(fpr_to_sarif): base program. Need to replace names in messages

* feat(fpr_to_sarif): message substitution and custom definition integration

* fix(fpr_to_sarif): missing replacement in tools object

* fix(fortifyExecuteScan): unit tests

* fix(fpr_to_sarif): failing unit test

* Fix fortify folder creation for generating sarif

* deletion of unzip folder

* fix(fortifyExecuteScan): change logging to info

* feat(fpr_to_sarif): better unit test

* fix(fpr_to_sarif): pr tests failing

* feat(fpr_to_sarif): add specific properties to sarif

* feat(fpr_to_sarif): severity integration

* fix(fpr_to_sarif): unit test fixed

Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Sumeet PATIL <sumeet.patil@sap.com>
2022-02-08 14:10:40 +01:00
Giridhar Shenoy
286ff1b6ef
detectExecuteScan : support for detect 7 (#3453)
* enable detect 7 script

* unit test uses detect7 as default

* add detect6 test case

* add check for OSEnv detect version

* add test for OSEnv detect version

* update customEnvironmentVariables docu

* fix linting

Co-authored-by: ffeldmann <f.feldmann@sap.com>
2022-02-08 11:55:01 +01:00
Sven Merk
07e5c66b5f
Fix test code (#3514)
* Fix test code

* Fix

* Fix
2022-02-08 11:21:25 +01:00
Siarhei Pazdniakou
4660f3a311
feat(golangBuild): add BOM creation (#3496)
* Added BOM creation

* Fix cyclonedx install

* undo config.yml changes

* metadata was updated

* Update golangBuild.go

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-02-08 09:23:33 +01:00
rosemarieB
d1eb87363f
Generic step input from addon descriptor (#3467)
* enhance build step to run based on addon.yml in steampunk environment

Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com>
2022-02-07 20:34:11 +01:00
Matthias Scudlik
907dcd7dc1
fix(sonarExecuteScan): error handling for report metrics (#3512) 2022-02-07 16:41:36 +01:00
ffeldmann
831c5b0061
Ignore violations if failOn is NONE (#3513) 2022-02-07 15:29:32 +01:00
ManjunathMS35
8108bb8f6f
Update npmExecuteScripts step (#3211)
* Update npmExecuteScripts step

* Fixed failing build

* Fixed path issue

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Christian Volk <christian.volk@sap.com>
2022-02-07 14:46:03 +01:00
Oliver Nocon
2ae1d9dac1
feat(kanikoExecute): allow building multiple images (#3443)
* feat(kanikoExecute): allow building multiple images

* enhance tests

* chore: allow running tests in parallel

* small fixes

* fix: fix destination bug

* update formatting and defaults

* fix yml formatting

* chore: change cpe parameter names

* chore: improve variable naming
2022-02-07 07:58:41 +01:00
Christian Volk
56726d96f1
fix(golangBuild): expect status 201 instead of 200 (#3508) 2022-02-04 15:30:42 +01:00
Christian Volk
ed7323175d
feat(golangBuild): publish to subfolder (#3503)
* feat(golangBuild): publish to subfolder

* edge case handling

* fix
2022-02-04 15:22:57 +01:00
Christian Volk
a90cf78565
fix(golangBuild): use PUT instead of POST (#3504)
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-02-04 14:46:52 +01:00
Christian Volk
f432d8455c
fix(golangBuild): targetRepositoryUrl (#3502)
* fix(golangBuild): targetRepositoryUrl

* regen
2022-02-04 12:18:35 +01:00
Matthias Scudlik
5eb996c43c
sonarqube coverage: additional metrics (#3465)
* sonarqube coverage: additional metrics

* sonarExecuteScan: add lines of code and language distribution to sonarscan.json

* sonarExecuteScan: consider branch in componentService requests

* SonarQube: Do not omit empty values in SonarCoverage

* sonarExecuteScan: Add integration tests for ComponentService getLinesOfCode

* fix tests

* sonarExecuteScan: use pullRequest in componentService

Co-authored-by: I550025 <r.kloe@sap.com>
Co-authored-by: Marc Bormeth <marc.bormeth@sap.com>
2022-02-04 09:52:29 +01:00
Oliver Nocon
42b9c2ce48
Revert "fix(golangBuild): keep track of the artifactversion in binary names (#3498)" (#3499)
This reverts commit ec783b0da6.
2022-02-03 11:23:34 +01:00
Anil Keshav
3f3f24cde6
fix (build dependency) typo corrections (#3494)
* typo corrections

* trailing white space

* go generate

Co-authored-by: anilkeshav27 <you@example.com>
2022-02-02 22:37:48 +01:00
Christian Volk
ec783b0da6
fix(golangBuild): keep track of the artifactversion in binary names (#3498)
* fix(golangBuild): keep track of the artifactversion in binary names

* improve the naming scheme
2022-02-02 17:30:56 +01:00
Christian Volk
78dac5973b
feat(golangBuild): add support for publishing binaries (#3495) 2022-02-02 15:44:51 +01:00
Christian Volk
eb0b5c1ffe
fix(kanikoExecute): tmp dir (#3478)
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-02-02 08:04:07 +01:00
Anil Keshav
1546ca293a
feat (Documentation) documentation for mavenBuild and npmExecuteScripts that consume dependencies from a private repo. (#3484)
* build depdency docu for maven and npm

* removing trailing white space in yaml

* Update resources/metadata/mavenBuild.yaml

* relative url for vault and mta docu

* running go generate

* keeping vault relative path

* go generate

* reverting to global paths

* go generate

* wild card for a higher level dir

* searching on top folder only

* relative level above

* pointing to infrastructure

* correcting links

Co-authored-by: anilkeshav27 <you@example.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-02-01 20:32:05 +01:00
Ralf Pannemans
e11369dd3f
fix(cnbBuild) do not add complete path to telemetry (#3487)
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>
2022-02-01 08:23:53 +01:00
kulhadia
6feb1217aa
Feat(cnbBuild): collecting the telemetry data for cnbBuild (#3368)
Co-authored-by: Sumit Kulhadia <sumit.kulhadia@sap.com>
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>
Co-authored-by: Pavel Busko <pavel.busko@sap.com>
2022-01-31 12:27:29 +01:00
Christian Volk
880be73a4c
feat(golangBuild): support private modules (#3471)
* feat(golangBuild): support private module repositories
2022-01-31 10:45:40 +01:00
Eugene Kortelyov
91420e02ff
Update image for gradle build (#3479)
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-01-28 15:58:01 +01:00
Anil Keshav
3cda84e214
fix (mtaBuild) keep mtar artifact name in synch with maven gav (#3454)
* keep mtar name in synch with maven gav

* retaining slashes

* keeping dots in artifact name

Co-authored-by: anilkeshav27 <you@example.com>
2022-01-28 14:41:09 +01:00
Philipp Stehle
6958e028d1
chore(kubernetesDeploy): add telemetry for deployTool (#3469)
* chore(kubernetesDeploy): add telemetry for deployTool

* Update cmd/kubernetesDeploy.go

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-01-28 08:31:34 +01:00
Eugene Kortelyov
f4db4e7e07
Update gradle version (#3476)
* Update gradle version

* Update generated file
2022-01-27 16:11:51 +01:00
Sven Merk
01c6f1a66c
fix(fortifyExecuteScan): User assignment based on PR ownership (#3472)
* Debug PR user details

* Check association

* Change to login

* Fix PR creator assignment

* Improve docs

* Fix test
2022-01-27 10:45:45 +01:00
Sarat Krishnan
6e139aee96
Fix(gctsDeploy) : add client in config url, disable retry for create and pull (#3464) 2022-01-25 17:18:05 +01:00
Linda Siebert
480d1c260f
Add testing for helm during acceptance stage (#3402)
* Add kubernetesDeploy to Acceptance

* Add more kubernetesDeploy

* Add helm tests

* Change documentation

* Fix docu

* Change generated

* Add tests

* Add groovy tests

* Fix tests

* Change tests

Co-authored-by: Thorsten Duda <thorsten.duda@sap.com>
2022-01-25 10:42:42 +01:00
Anil Keshav
1a96d7a67e
fix (npmExecuteScripts) allow npm pack before npm publish (#3455)
* adding config to piperNpmr

* scope in cli

* adding scope to repo url and npmrc

* publish to scoped

* removing scope

* changing scope position

* adding scope to userconfig

* adding registry=

* pack and then tar

* not removing tmp folder

* adding flag

* pack before publish

* adding log

* debug

* debug with change directory

* publishing created tar ball

* debug

* üath

* adding main npmrc

* renaming old npmrc file

* error renaming old npmrc file

* renaming err

* correcting npmrc file path

* renaming file back to original

* current working directory

* renaming the npmrc file

* avoiding change directory

* with current working dir

* adding dot

* renaming npmrc and defer removal

* rename files

* Update pkg/npm/publish.go

* Update pkg/npm/publish.go

Co-authored-by: anilkeshav27 <you@example.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-01-25 09:52:22 +01:00
Sven Merk
4e0684cf78
Address further nil references (#3462)
* Address further nil references

* Message text

* Final checks
2022-01-24 17:09:49 +01:00
Siarhei Pazdniakou
db7d85f3f2
Add gcs upload to uiVeri5ExecuteTests step (#3409)
* Add gcs upload to uiVeri5ExecuteTests step

* go generate

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-01-24 15:18:27 +01:00
Eugene Kortelyov
da450c5243
feat(gradleExecuteBuild): add stage scope to path parameter, fix project dir exist issue (#3401)
* add stage scope to path parameter, fix project dir exist issue

* fix unit test for gradleExecuteBuild

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-01-24 15:08:39 +01:00
Siarhei Pazdniakou
a8ef69bc0f
Add gcs upload to mavenBuild step (#3394)
* Add gcs upload to mavenBuild step

* Patterns were updated

* go generate

* removed patterns related to sapGenerateEnvironmentInfo step

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-01-24 15:00:30 +01:00
Siarhei Pazdniakou
07ae563928
Add gcs upload to mtaBuild step (#3405)
* Add gcs upload to mtaBuild

* Patterns was updated

* go generate

* removed patterns related to sapGenerateEnvironmentInfo step

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-01-24 14:50:52 +01:00
Christopher Fenner
ebcbb73b46
feat: add command to fetch default files (#3445)
* some ideas..

* Add getDefaults command (WIP) (#3444)

* add getYAML function for configs

* create getDefaults command(based on getConfig)

* add getDefaults command to CLI

* read defaults files, using github tokens as well

* write defaults to stdout as JSON object with YAMLs embedded

* catch case where no input files are given

* actually write output to file if outputFile is specified

* mark defaultsFile flag as required

* add basic tests

* add output (string)  test

* adapt generateDefaults() to return output (used for test of previous commit)

* Changes to getDefaults() JSON output (#3449)

* change JSON output to contain separate fields

* filename -> filepath

* Apply suggestions from code review

* Apply suggestions from code review

* Update pkg/config/config_test.go

Co-authored-by: Jordi van Liempt <35920075+jliempt@users.noreply.github.com>
2022-01-24 13:55:28 +01:00
Siarhei Pazdniakou
8171730b60
Add gcs upload to newmanExecute step (#3408)
* Add gcs upload to newmanExecute step

* go generate

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-01-24 12:21:41 +01:00
Siarhei Pazdniakou
c19a94032c
Add gcs upload to gaugeExecuteTests step (#3410)
* Add gcs upload to gaugeExecuteTests step

* go generate

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-01-24 12:13:33 +01:00
Siarhei Pazdniakou
ab9848d49e
Add gcs upload to karmaExecuteTests step (#3431)
* Add gcs upload to karmaExecuteTests step

* go generate

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-01-24 12:06:35 +01:00
Sven Merk
ffa82c383e
Fix potential nil reference (#3460) 2022-01-24 11:59:33 +01:00
Christian Volk
b0e4599d4d
feat(malwareExecuteScan): refactoring and docker support (#3421)
* feat(malwareExecuteScan): add support for scanning docker images

* refactoring

* print out finding if available

* generate toolrecord for malware scan

* persist scan report

* docs

* fix

* fix

* rollback cmd/init_unix.go

* auhenticated pull

* fix

* fix: report shall be consistent with the api model

* gcs upload

* fix linter
2022-01-24 09:48:01 +01:00
Sven Merk
6520115950
Upload Fortify scan results to GitHub issue (#3300)
* fix(fortifyExecuteScan): Propagate translation errors

Force translation related errors to stop the execution of the step.

* Extend testcase

* Update fortifyExecuteScan.go

* Fix fmt and test

* Fix code

* feat(fortifyExecuteScan): Create GitHub issue

* Fix expectation

* Fix fmt

* Fix fmt add test

* Added tests

* Go fmt

* Add switch

* Rewrite githubCreateIssue

* Fix tests

* Added switch

* Issue only in case of violations

* Fix CPE reference

* Add  debug message to issue creation/update

* Update fortifyExecuteScan.go

* Add credential for GH to groovy wrapper

* Update fortifyExecuteScan.go
2022-01-21 10:52:17 +01:00
Sarat Krishnan
f88a979896
disable http retry (#3447) 2022-01-20 10:46:24 +01:00
Siarhei Pazdniakou
198fbbeb8d
Add gcs upload to mavenExecuteIntegration step (#3432)
* Add gcs upload to mavenExecuteIntegration step

* go generate

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-01-20 08:18:57 +01:00
Siarhei Pazdniakou
c93f1f861d
GCS Upload fixes (#3387)
* Fixed argument type in persist function

* Fixed gcp upload to be usable in internal piper

* Fixed import of packages

* Updated logs

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-01-19 13:16:57 +01:00
Umidjon Urunov
c2ebdfd9ec
feat(protecode): add versioning model (#3373)
* changes to detectExec before master merge

* changes for detectExecuteScan

* self generated code added

* fix syntax errors and update docu

* added unit tests for fail and Group

* fix failOn bug

* add Groups as string array

* add Groups as string array

* tests and validation for groups, failOn

* Updated docs and added more tests

* documentation md files should not be changed

* Handle merge conflicts from PR 1845

* fix merge errors

* remove duplicate groups, merge error

* adding buildCode and buildTool as params

* switching build options

* building maven modules

* parameter correction

* parameter correction

* gnerate with new build parameter

* adding comments

* removing piper lib master and modifying goUtils to download 1.5.7 release

* first cleaning then installing

* multi module maven built

* multi module maven built removing unwanted code

* multi module maven built moving inside switch

* testing

* modifying the default use case to also call maven build

* modifying the default use case to also call maven build wih --

* corrected maven build command

* corrected maven build command with %v

* skipping test runs

* testing for MTA project with single pom

* adding absolute path to m2 path

* clean up

* adding switch for mta and maven and removing env from containers

* commiting changes for new detect step

* correting log message

* code clean up

* unit tests changes to detectExecute

* basic tests for new change

* restoring piperGoUtils to download correct piper binary

* code clean up

* code clean up

* protecodeExecuteScan :: versioning model draft - 1

* protecodeExecuteScan :: version model draft-2

* protecodeExecuteScan :: changing filename and version concatenation

* protecodeExecuteScan :: update documentation

* protecodeExecuteScan :: double URL encoding has been corrected & console messaging improved

* protecodeExecuteScan :: fixed Go/generate validation fail

* protecodeExecuteScan :: fixing failed unit tests

* protecodeExecuteScan :: Version field added

* protecodeExecuteScan :: Version field add => minor changes

* protecodeExecuteScan :: Version field add => fixing tests

Co-authored-by: D072410 <giridhar.shenoy@sap.com>
Co-authored-by: Keshav <anil.keshav@sap.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2022-01-19 10:30:59 +01:00
Ralf Pannemans
c97625e840
feat(cnbBuild) containerImageName will be defaulted if possible (#3437)
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
Co-authored-by: Sumit Kulhadia <sumit.kulhadia@sap.com>
2022-01-14 16:49:45 +01:00
Marcus Holl
4acee9fc85
fix: wrong command line format for multiple extension files (#3434) 2022-01-14 16:13:24 +01:00
Pavel Busko
1750b75cb8
feat(cnbBuild): preserve maven test results in the workspace (#3429)
Co-authored-by: Pavel Busko <pavel.busko@sap.com>
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
2022-01-14 11:05:11 +01:00
Siarhei Pazdniakou
30aa6eea7d
fix(sonar): fix file patterns for gcs upload (#3406)
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2022-01-12 13:06:14 +01:00
Daniel Mieg
8634d8bb12
Add easy mode for AUnit & ATC (#3389)
* remove mandatory flag from config files

* Enable repo.yml as config

* Adapt to merge

* Refactoring

* Refactoring

* avoid panic

* Add comments

* Add easy mode for atc

* Add tests

* Add test

* Refactor

* Add test for MPS

* Updates

* Rename functions

* Add files to gitignore

* Rename

* Renaming

* Renaming

* Renaming

* Improve error messages

* Update documentation

* Add logging

* Rename

* Extend gitignore
2022-01-12 12:02:27 +01:00
tiloKo
24a2340921
readPipelineEnv -> Serialize Directly to os.Stdout (#3180)
* Serialize CPE Directly to os.Stdout
2022-01-12 09:17:03 +01:00
Anil Keshav
af6a1801ab
feat (kubernetesDeploy) allow secret creation in cluster with kubectl as deploy tool (#3398)
* defaulting deocker config json location

* function change

* harmonising secret creation via json config apply

* adding the env path to kaniko

* env var

* adding path

* adding path

* adding path

* rolling back changes for path

* adapting condition for secret creation

* conditions based on username and password only

* fix unit test as per new secret creation

* update documentation with regards to secret creation

* fixing yaml lint empty line

* fixing trailing line from kaniko yaml

* error condition when path of the docker config json file is not found

Co-authored-by: anilkeshav27 <you@example.com>
2022-01-11 11:30:40 +01:00
Marc Bormeth
78c4a8fe93
Revert "feat: Add debug output to getConfig (#3341)" (#3396)
This reverts commit b760bf48e7.

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-01-11 10:47:03 +01:00
Pavel Busko
547ef5b55d
feat(cnbBuild): remove docker config after parsing (#3417)
Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>
2022-01-11 10:01:15 +01:00
Mayur Belur Mohan
f27cb4e482
ApiProxyUpload Command (#3295)
* ApiProxyUpload Command

* Code Review Fixes

* CodeReview Changes

* CodeReview Fixes

* YAML fixes

* CodeReview Fix

* Code Review Fixes

* CodeReview Fixes

* Code Climate Fixes

* Code Review Fixes

* Code Review Fixes

Co-authored-by: Roland Stengel <r.stengel@sap.com>
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
2021-12-28 09:31:50 +01:00
Daniel Mieg
b5da011200
Consistent config for AUnit (#3395)
* Unify singular/plural in config files

* singular -> plural

* adapt

* Change
2021-12-23 13:33:01 +01:00
Pavel Busko
81fa0ee2d8
feat(cnbBuild): support for fetching binding content from url (#3388)
Co-authored-by: I546443 <sumit.kulhadia@sap.com>
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
2021-12-23 10:21:13 +01:00
Eugene Kortelyov
4d787ded6d
gradleExecuteBuild initial commit (#3337)
* gradleExecuteBuild initial commit

* fmt fixes for metadata_generated

* flow updates, some tests were added

* add gradle path alias, add stage scope to task parameter

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-12-22 15:34:36 +01:00
Dominik Lendle
5372074d7f
Change default ATC CheckVariant (#3384)
* change return after files are persisted & Change logging

* Change ATC CheckVariant Default

* change default checkvariant tests

* add docu hint
2021-12-22 12:34:42 +01:00
Dominik Lendle
1930e086cc
A unit (#3385)
* change return after files are persisted & Change logging

* reduce object set to single object

* adapt tests

* adapt docu

* adapt stage docu

* adapt tests
2021-12-22 11:04:22 +01:00
Dominik Lendle
9917b08ff8
A unit object set enhancement (#3273)
* change return after files are persisted & Change logging

* object set enhancement

* enhance object set

* Object set enhancement

* object set enhancement

* reduce object set

* adapt documentation

* adapt docu

* remove comment blocks

* remove comment blocks

* adapt object set usage

* adapt documentation to new object set usage

* adapt documentation to new object set usage & remove trailing spaces

* Adapt Unit tests

* Adapt Unit tests

* Adapt Unit tests

* cleanup code

* cleanup code

* refactor object set

* remove parallel tests

* reduce set basis

* adapt tests

* dummy commit

* adapt docu

* Update documentation/docs/steps/abapEnvironmentRunAUnitTest.md

* Update documentation/docs/steps/abapEnvironmentRunAUnitTest.md

* Update documentation/docs/steps/abapEnvironmentRunAUnitTest.md

Co-authored-by: Daniel Mieg <56156797+DanielMieg@users.noreply.github.com>
2021-12-21 15:24:54 +01:00
Christopher Fenner
51feacadbf
chore: align Vault naming (#3372)
* Align Vault naming

* remove duplicate debug  output

* correct message

* align vault naming

* Update vault.md
2021-12-21 11:52:10 +01:00
Daniel Mieg
2bb6c21337
ABAP environment: add tag support (#3376) 2021-12-20 17:58:58 +01:00
Christopher Fenner
9180f54f86
feat(sonar): reuse existing TLS truststore (#3312)
* do not load sap certificates with groovy

* add toggle

* add keytool package

* copy existing default truststore

* ignore import failure

* fix typo

* rename

* extract maven opts

* add todo

* add tests

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-12-16 12:49:15 +01:00
Eugene Kortelyov
f16a6beb19
fix(vault): update terraform secretName parameter to follow naming convention (#3365)
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-12-15 20:12:57 +01:00
Siarhei Pazdniakou
91ddbf9152
fix(sonar): disable gcp upload when not configured (#3369) 2021-12-15 16:25:46 +01:00
Siarhei Pazdniakou
cd243ee542
feat(gcs): allow upload to gcs from steps (#3034)
* Upload reports to Google Cloud Storage bucket

* Added tests. Made fixes

* Update step generation. GCS client was moved to GeneralConfig

* Code was refactored

* Fixed issues

* Fixed issues

* Code correction due to PR comments

* Improved gcs client and integration tests

* Integrated gcp config. Updated step metadata

* Fixed issues. Added tests

* Added cpe, vault, aliases resolving for reporting parameters

* Added tests

* Uncommented DeferExitHandler. Removed useless comments

* fixed cloning of config

* Added comments for exported functions. Removed unused mock

* minor fix

* Implemented setting of report name via paramRef

* some refactoring. Writing tests

* Update pkg/config/reporting.go

* Update cmd/sonarExecuteScan_generated.go

* Apply suggestions from code review

* Update pkg/config/reporting.go

* Update pkg/config/reporting.go

* fixed removing valut secret files

* Update pkg/config/reporting.go

* restore order

* restore order

* Apply suggestions from code review

* go generate

* fixed tests

* Update resources/metadata/sonarExecuteScan.yaml

* Update resources.go

* Fixed tests. Code was regenerated

* changed somewhere gcp to gcs. Fixed one test

* move gcsSubFolder to input parameters

* fixed removing valut secret files

* minor fix in integration tests

* fix integration tests

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2021-12-15 15:07:47 +01:00
Christopher Fenner
b7e1d28675
fix: prevent endless loop in exit handler (#3363)
* do not fatal in exit handler

* update generated sources

* update golden sources

* update generated sources
2021-12-15 14:26:23 +01:00
Rinita Asani
278c47b808
Refactor of gctsExecuteABAPUnitTests (#3338)
* Update gctsExecuteABAPUnitTests.yaml

* Update gctsExecuteABAPUnitTests.go

* Update gctsExecuteABAPUnitTests.go

* Update gctsExecuteABAPUnitTests.yaml

* Add commit parameter

* Update gctsExecuteABAPUnitTests.yaml

* Update gctsExecuteABAPUnitTests.go

* Update gctsExecuteABAPUnitTests.go

* Update gctsExecuteABAPUnitTests.yaml

* Update gctsExecuteABAPUnitTests.go

* Change the commit parameter in Yaml file

* Adding scope paramter to gCTSExecuteABAPUnitTest

* Adding logs

* Adapting checkstyle for success case

* write ATC file

* add log

* time duration

* add log

* Improve log

* Improve unit test

* severity error

* Fix the xml file for Unit Tests

* Editing UnitTest Results for better parsing

* Change file name

* Add folder

* Add path to file

* Add src

* Add src and object type

* Add filename

* Add path

* Handling success cases

* complete refactoring

* fix for file path

* filname lower case

* Add log entry

* Delete ATC Results

* Change ATC name

* Functiongroups

* When ATC errors Pipeline fails

* Reading a file into chuncks

* escape string

* Fix for a bug

* Unesscape URL

* Change from fatal error to error

* When unit test fails pipeline fails too

* Severity errors

* Severity for Unit Test

* Fix for successful unit test

* Code review

* Telemetry was missing

* yaml file

* generate yaml file

* add logs

* fix location

* do not close body

* checkstyle severity

* checkstyle

* checkstyle2

* atc method

* line calculation

* global file

* method

* line

* fix

* add logs

* add log

* add logg

* unit test error

* Code checked and finalized

* change commit paramter to optional

* Add logs for current local and remote commit

* Add logs for object type and name

* parse ATC checks error

* add log of http response

* add log

* remove end of function findline

* log cleaning

* log extra cleaning

* cleaning log for unit test

* more cleaning

* double cleanser for your code

* more cleaning

* docu checks

* file name fix for function group

* handle programs-reports

* fix a bug

* fix regex

* fix regex for test class

* fix regexos

* fix dots

* fix a error message in unit tests

* calculate dynamically file extension

* add log for path

* add method name

* add method name-

* fix repo scope

* include table type

* Change paramter commitId to commit only

* change log output

* give more detailed examples

* Table Name in get file name

* change logs

* improve log entries

* improve log entries again

* final improvements

* Delete UnitTestResults

* Review

* Change text and comments

* fix a type in commit paramter

* change repository method name

* Fix typos in yaml file

* Remove empty lines in yaml file

* Remove blank lines in the end

* Add new line

* go generate

* Remove tabs

Co-authored-by: Martin Bredy <58846699+martin-bredy@users.noreply.github.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2021-12-15 10:55:53 +01:00
Siarhei Pazdniakou
0879fa591a
fix(influx): fix generated output parameters (#3362)
* fixed generated output parameters for influx

* change name to lower case

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2021-12-15 09:40:50 +01:00
Christopher Fenner
279ad5800c
fix(sonar): correct VaultSecretName (#3358)
* fix(sonar): correct VaultSecretName

* update generated sources
2021-12-14 18:28:20 +01:00
Roland Stengel
47d8d2c357
GPP Init Stage needs to consider mta yaml path (#3305)
mtaBuild - support config.source to yaml path
2021-12-14 14:43:02 +01:00
Christopher Fenner
6f06ce0f56
feat(hadolint): allow config secret from vault (#3353)
* allow config secret from vault

* add credential alias

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-12-14 13:42:06 +01:00
Dominik Lendle
f188979412
Add html processing to AUnit stage/step (#3302)
* change return after files are persisted & Change logging

* html processing

* Extend step parameters

* Add documentation

* html processing

* add generateHTML flag to general

* test adaptions

* step metadata corrections

* adapt formatting

* Update documentation/docs/pipelines/abapEnvironment/extensibility.md

Co-authored-by: Daniel Mieg <56156797+DanielMieg@users.noreply.github.com>

* Update documentation/docs/pipelines/abapEnvironment/extensibility.md

Co-authored-by: Daniel Mieg <56156797+DanielMieg@users.noreply.github.com>

* adapt unit tests

Co-authored-by: Daniel Mieg <56156797+DanielMieg@users.noreply.github.com>
2021-12-13 16:39:32 +01:00
Giridhar Shenoy
ad0a3bda2b
Whitesource Docker scan : use the project name as the tar file path (#3256)
* add file path as project name

* fix test for containerSaveImage

Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2021-12-13 16:14:32 +01:00
Christopher Fenner
38e4e0a133
fix(hadolint): change docker image (#3351)
* use hadolint alpine image

* update generated files
2021-12-13 14:21:09 +01:00
Eugene Kortelyov
e727601ea4
feat(shellExecute): cleanup, Jenkins step, docs … (#3313)
* remove functionality with script downloading (security issue), clean up code

* remove vault client creating, remove vault params

* fix go generate issue

* error handling and test updated

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-12-13 12:31:31 +01:00
Christian Volk
ff6a26dd42
feat(malwareExecuteScan): support for vault (#3346)
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-12-13 11:58:33 +01:00
Marc Bormeth
61d1358ca5
hotfix: Don't write SonarQube coverage metrics to InfluxDB (#3347) 2021-12-10 09:13:25 +01:00
rosemarieB
e6c5c8a72f
Abap refactoring (#3340)
* Update abapAddonAssemblyKitReleasePackages.go

* add error handling for initAAKaaS

* runtime to config, url.QueryEscape, check return body

* Update abapAddonAssemblyKitCheckCVs_test.go

* add log entrys for testing

* correct yaml

* change unmarshal logic

* Update abapAddonAssemblyKitCheckPV_test.go

* adding utils functionality

* Update bfw_test.go

* reset CheckPV

* reset CheckCVs

* reset ReserveNextPackages

* set back CheckPV and CheckCVs

* moving mock

* renaming

* renaming

* moving mocking to checkCVs test

* fixing unittests

* trying mock with comment

* referencing unittest to mockfile

* Update abapAddonAssemblyKitCheckCVs.go

* Update cmd/abapAddonAssemblyKitCheckPV.go

Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com>

* Update cmd/abapAddonAssemblyKitReleasePackages.go

Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com>

* Update cmd/abapAddonAssemblyKitPublishTargetVector.go

Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com>

* Update cmd/abapAddonAssemblyKitReserveNextPackages.go

Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com>

* fixing unittests

* fixing mock comment

Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com>
2021-12-09 12:54:18 +01:00
tiloKo
c1dc725fcc
Correct Typo in Log Message (#3333) 2021-12-09 10:37:53 +01:00
Marc Bormeth
b760bf48e7
feat: Add debug output to getConfig (#3341)
* Fix Orchestrator detection

* Add unit tests

* Add debug output to getConfig

* Update getConfig.go

* fix whitespaces

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2021-12-08 10:16:54 +01:00
Marc Bormeth
b213af1089
Add Coverage Info to sonarscan.json (#3262)
* Add coverage metrics to report + influx

* Write unit tests

* Add integration test for Sonar Measures Component Service
2021-12-08 09:02:12 +01:00
Christopher Fenner
d0762f5db9
fix: use correct metadata resolver (#3342)
* fix something

* add default metadata resolver

Co-authored-by: ffeldmann <f.feldmann@sap.com>
2021-12-07 15:54:59 +01:00
Pavel Busko
79f7ccd0f5
feat(cnbBuild): increased platform api compatibility (#3330)
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
Co-authored-by: Benjamin Haegenlaeuer <benjamin.haegenlaeuer@sap.com>
2021-12-07 14:06:35 +01:00
Christopher Fenner
b9eb48c149
fix: add kaniko env var for container detection (#3157)
* add kaniko env var

* Rename kaniko.yaml to kanikoExecute.yaml

* update generated files
2021-12-07 08:38:53 +01:00
Oliver Nocon
9a78fabc89
feat(golangBuild): add new step for building go (#3178)
* feat(golangBuild): add new step for building go

* chore(golangBuild): increase test coverage

* remove indirect dependencies

* cleanup go.sum

* chore: remove trailing spaces

* chore(golangBuild): cleanup params, add groovy wrapper

* fix: update docker options

* update docs

* update installation according to https://golang.org/doc/go-get-install-deprecation

* fix: update installation

* update groovy test exclusion

* Update vars/golangBuild.groovy

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* update branch

* address PR feedback

* fix compilation error

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2021-12-06 16:17:59 +01:00
rosemarieB
e90856d5bf
Generic build step (#3323)
* new step abapEnvironmentBuild

* Update piper.go

* Update abapEnvironmentBuild.go

* update yaml file

* Logging for debugging

* Update abaputils.go

* Update connector.go

* assigning connector

* delete debugging logging

* Update abapEnvironmentBuild.go

* certificate to yaml

* Update abapEnvironmentBuild.go

* add scope

* Update abapEnvironmentBuild.go

* Update abapEnvironmentBuild.yaml

* change certificate name in yaml

* test my new gitscript

* logging for debugging

* debugging...

* adding options to client.

* skip verification

* debugging

* debugging...

* switch of transportskipverification

* changing connector return

* deleting additional set options

* fixed timeout error

* adding certificate

* testing without certificate set

* testing with certificate set

* download, publish and value logic

* write values to cpe

* logging

* adding condition on string length

* change publishmethod and some logging

* change download method -> using references

* evaluation of parameter for download

* add case for empty string

* adding unittests

* Update mockClient.go

* make abapEnvironmentBuildUtilsBundle powerful

* refactor abapEnvironmentBuild into pieces

* check error message

* check error message 2

* check error message 3

* check error message 4

* remove check error message

* cleanup

* adding unittests

* unittests and docu

* docu

* docu

* Update abapEnvironmentBuild.md

* removing trailing spaces and adding empty lines in docu

* Update abapEnvironmentBuild.md

* fixing unittest and PR recommen

* Update abapEnvironmentPipelineStageBuild.groovy

* Update abapEnvironmentPipelineStageBuild.groovy

* Update abapEnvironmentPipelineStageBuild.groovy

* Update abapEnvironmentPipelineStageBuild.groovy

* changes derived from pull request

Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com>
2021-12-06 14:43:37 +01:00
Sven Merk
ae9cb1e6f2
Fix alias definition for agentUrl (#3328)
* Fix alias definition for agentUrl

* Update generated file
2021-12-03 12:16:31 +01:00
Oliver Nocon
a76b9d563d
feat(kubernetesDeploy): consume credentials from environment (#3129)
* feat(kubernetesDeploy): consume credentials from environment

* update vault

* update handling of docker config.json

* chore: remove comments
2021-12-02 12:18:21 +01:00
kingvvgo
b29a2f2264
Add Changes for value of docker image (#3303)
* Add Changes for value of docker image

* Get docker image value

* Fix

* Fix unit

* Add chnages for kaniko and mta builds

* Fix

* Test changes

* Test

* Move func ResolveMetadata to stepmeta.go

* Fix

* Change getConfig.go

* Fix getting docker value for mta, npm and kaniko

* Fix according to suggestions

* Add func to get only value of docker image

* Test empty value of docker image

* Fix for getDockerImageValue

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-12-01 07:46:18 +01:00
kulhadia
347ae68ea9
added the beta notification for cnbBuild step (#3316) 2021-11-30 11:51:00 +01:00
Pavel Busko
ac09e6e4aa
feat(cnbBuild): refactored buildEnvVars input property, added resourceRef to the buildpacks (#3310)
Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
2021-11-29 11:32:32 +01:00
Pavel Busko
74c0fe5e28
feat(protecodeExecuteScan): use dockerConfigJSON from the commonPipel… (#3248)
* feat(protecodeExecuteScan): use dockerConfigJSON from the commonPipelineEnvironment.custom.dockerConfigJSON

Co-authored-by: Pavel Busko <pavel.busko@sap.com>
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
2021-11-25 13:12:19 +01:00
Roland Stengel
6320275f47
Step Parameter Alias deprecate key is ignored (#3158)
* Step Parameter Alias deprecate key is ignored
2021-11-23 15:37:28 +01:00
tiloKo
081f01f93b
add scope (#3291)
Co-authored-by: Daniel Mieg <56156797+DanielMieg@users.noreply.github.com>
Co-authored-by: Peter Persiel <6087940+peterpersiel@users.noreply.github.com>
2021-11-23 08:50:46 +01:00
Eugene Kortelyov
d395b362ed
feat: step to execute shell scripts (#3196)
* shell executor initial commit

* functionality updates

* changes in logging implementation (using internal logging), changes in execution

* remove unused field

* remove duplicate from code

* update vault flow and remove unnecessary params

* update generated step file

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-11-19 10:05:39 +01:00
ffeldmann
42b92d1bfe
Changes for Pipeline Reporting (#3213)
* Adds GetLog() function to orchestrator

* Fixes BUILD_NUMBER env variable

* Fixes correct env var for JENKINS_HOME

* Adds getEnv to read env variables with default value, adds test for jenkins GetLog() implementation

* Adds possibility to read errorJsons; updates splunk package for log files (WIP)

* Uncommenting dev code

* Adds GetLog() function to orchestrator

* Fixes BUILD_NUMBER env variable

* Fixes correct env var for JENKINS_HOME

* Adds getEnv to read env variables with default value, adds test for jenkins GetLog() implementation

* Adds possibility to read errorJsons; updates splunk package for log files (WIP)

* Uncommenting dev code

* Adds GetRequest function which holds the response in memory (not saved to disk)

* Implements GetLog() function for ADO, adds function to read PipelineRuntime

* PAT has been revoked

* Changes http package, s.t. if password only is required basic auth works too

* Adds env variable for azure token, error handling in case of unauthenticated/nil response

* Adds logging output in case env variable can not be read and fallback variable needs to be used

* Adds usage of environment variables for auth, uses jenkins api

* Adds init functionality for orchestrators, updates GetLog() and GetPipelineStartTime() function

* Adds initaliziation function for orchestrator authetnication

* Adds settings struct for orchestrator authentication

* Adds function to whole logfile to Splunk

* Struct for pipeline related telemetry information

* Increase messagebatch size to 10k

* Changes splunk package to a pointer based implementation, updates generated files and corresponding template and tests for splunk

* Changes telemetry package to pointer based implementation to have multiple telemetry objects, adjusted tests and splunk implementation

* Changes content type to txt

* Send telemetry independent of logfiles, increases amount of messages per file

* Adds JobURL for orchestrators and UnknownOrchestrator as fallback

* telemetry makes use of orchestrator specific information

* Adds orchestrator independent correlationID

* Adds custom fields for pipeline status

* go fmt

* Removes env var test - no env variables are read anymore

* Use UnknownOrchestratorConfigProvider in case the orchestrator can not be initalized

* Removes Custom fields from telemetry as these can not be reflected in SWA

* Adds custom telemetry information (piperHash,..) to each step telemetry information

* Removes falltrough in case no orchestrator has been found

* Updates tests for orchestrator package

* Adds orchestrator import in generated files

* Updates generator files for internal library

* Adds orchestrator telemetry information to steps

* Updates generated files, fatalHook writes to cpe

* Go generate from master, go fmt

* Adds Custom Data field LastErrorCode

* Removes GetLog() test

* Update init_unix.go

* Update docker_integration_test_executor.go

* Update integration_api_cli_test.go

* Reverts go1.17 fmt formatting

* Reverts go1.17 fmt formatting

* Reverts go1.17 fmt formatting

* Renames customTelemetryData to stepTelemetryData

* Adjustments to orchestrator-package, cleanup, adds JobName

* Adjusts commonPipelineEnvironment path

* Adds pipelineTelemetry struct to telemetry package, removes pipeline telemetry structs from splunk package

* Go fmt

* Changes path for errorDetails, adds debug information

* Removes custom fields from step, adds orchestrator, commithash to baseMetadata

* Adjusts tests for telemetry package

* Adds tests for orchestrator

* Updates generated files, initalization of splunk client only if its available in the config

* Fixes typo in helper go

* Update pkg/http/downloader.go

* Update pkg/http/downloader.go

* Update pkg/log/fatalHook.go

* Update fatalHook.go

* Update pkg/splunk/splunk.go

* Update pkg/telemetry/data.go

* Adds GetBuildStatus() and GetAPIInformation() to orchestrators

* error formatting

* Bugfix: dont send telemetry data if disabled, adjusts test

* go fmt

* Use correct error handling

* Update pkg/telemetry/telemetry.go

* Fixes telemetry disabled in the tests

* Fixes http tests

* Log fatal errors to logFile

* Adds CustomReportingConfig to hooks

* Cleanup comments in splunk package

* Adds possibility to send telemetry to custom endpoint

* Adds debug output for the payload

* Debug output for the payload as a string

* Adds test cases for changes in telemetry package

* go fmt

* Adds generated files for new step

* Reverts changes for http tests, causing problems with go1.15, changes need to be applied for newer go version >=1.17

* Adjusts test for sonarExecuteScan

* Adjusts test for sonarExecuteScan

* Adds explanation for customreportingConfig

* Makes disableing of customSend more obvious

* Adds custom step reporting to each step, updates generated files, adjusts helper testdata

* fixes unit test wrong usage of logging

* Send pipeline data altough there has been no error, adjust test cases

* Reverts changes for customReporting

* Updates generated files, removes customReporting

* Removes writing errorDetails to CPE

* Reverts usage of customreporting

* go fmt

* reverts changes in http_test

* reverts changes in http_test

* Skips integration cnb test

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-11-18 17:50:03 +01:00
Pavel Busko
56f58ebd8b
fix(cnbBuild): exclude registry domain from the imageNameTag CPE field (#3296) 2021-11-18 17:16:29 +01:00
Mayur Belur Mohan
3a4f756573
IntegrationArtifact Fixes for issue #3121 (#3286)
Co-authored-by: Roland Stengel <r.stengel@sap.com>
2021-11-18 09:08:03 +01:00
Oliver Nocon
6c5434f957
feat(checkIfStepActive): support new CRD style conditions (#3254)
* feat: first parts of new run struct

* add parts for new stage condition handling

* update conditions

* feat: finalize conditions and tests

* feat(checkIfStepActive): support new CRD style conditions

* feat(docs): allow generating stage docs

* chore(docs): make step directory configurable

* fix: tests

* add option to output file

* Update checkIfStepActive_test.go
2021-11-18 07:24:00 +01:00
tiloKo
359cf9eeb3
add missing spaces in error message (#3288)
* add missing spaces

* typo: on -> one
2021-11-17 15:06:31 +01:00