1
0
mirror of https://github.com/SAP/jenkins-library.git synced 2024-12-12 10:55:20 +02:00
Commit Graph

829 Commits

Author SHA1 Message Date
Vyacheslav Starostin
a50fad3b1a
feat(Splunk): Use repository and organization from pipelineEnv variable git/* (#4619) 2023-10-10 11:44:24 +02:00
Anil Keshav
40e13f1635
feat (npmExecuteScripts) enhance multi package publish from npm builds (#4579)
* enabling publish to only publish sub packages

* changing directory and then coming back to original after the publish runs

* searching the glob tar and npmrc in the current directory

* excluding build descriptor check and addtional target tool check

* changing the npm pack before publish to run only in sub packages

* removing commented code clean up

* adding the correct npm pack

* improve logging

* fix error handling and a bit style fix

* fix unit tests

* remove commented lines

* respecting build descriptor list when provided

* improve docu for the step param

* fixing linting issues

* improve docu

---------

Co-authored-by: Gulom Alimov <gulomjon.alimov@sap.com>
Co-authored-by: Jordi van Liempt <35920075+jliempt@users.noreply.github.com>
2023-10-09 09:21:09 +02:00
Akramdzhon Azamov
9e64744339
added logic of fetching golang private packages for whitesource step (#4595)
* added logic of fetching golang private packages for whitesource step and detectExecuteScan step

* changed logic of checking by config.PrivateModulesGitToken

* moved func prepareGolangPrivatePackages to golangBuild.go

* fix (gitOpsUpdateDeployment) add CA bundle options to plain clone and commit to trust enterprise github instances (#4602)

* downloading ca cert bundle when added as config

* adding logging statements

* allowing bats test to handle ca cert

* adding info message

* hard coding file names

* including correct http client util bundle

* removing logging message not needed

* adding cert bundle to commit and push

* improving the condition to add ca cert in commit and push

* fixing unit test

* fixing unit test

* fixing unit test

* fixing unit test

* fixing unit test

* feat(kanikoExecute): add dockerfilePath param to multipleImages (#4569)

* add containerDockerfilePath param to multipleImages

* rename ContainerDockerfilePath param to DockerfilePath

* Fix trailing spaces

---------

Co-authored-by: Egor Balakin <egor.balakin@sap.com>
Co-authored-by: Vyacheslav Starostin <32613074+vstarostin@users.noreply.github.com>

* fix(helm): forward sourceRepositoryCredentialsId from groovy to go layer (#4604)

forward sourceRepositoryCredentialsId from groovy to go layer in the same way how this is done for the targetRepositoryCredentialsId

* feat(config): exporting generateConfig function and applying minor changes (#4605)

* exporting generateConfig function and applying minor changes

* Added setConfigOptions to set configOptions variable.
Added possibility to set format output, json or yaml for now.

* Correcting mistake on cmd/getDefaults.go

Co-authored-by: Jordi van Liempt <35920075+jliempt@users.noreply.github.com>

---------

Co-authored-by: Jordi van Liempt <35920075+jliempt@users.noreply.github.com>

* moved func prepareGolangPrivatePackages to pkg/golang

---------

Co-authored-by: Akramdzhon Azamov <MY_NAME@example.com>
Co-authored-by: Andrei Kireev <andrei.kireev@sap.com>
Co-authored-by: Anil Keshav <anil.keshav@sap.com>
Co-authored-by: Egor Balakin <14162703+m1ron0xFF@users.noreply.github.com>
Co-authored-by: Egor Balakin <egor.balakin@sap.com>
Co-authored-by: Vyacheslav Starostin <32613074+vstarostin@users.noreply.github.com>
Co-authored-by: Marcus Holl <marcus.holl@sap.com>
Co-authored-by: Jk1484 <35270240+Jk1484@users.noreply.github.com>
Co-authored-by: Jordi van Liempt <35920075+jliempt@users.noreply.github.com>
2023-10-06 16:22:26 +02:00
Jordi van Liempt
5a56726bde
Revert "chore(vault): custom retry check function (#4475)" (#4616)
This reverts commit 2ab1e2a1bc.
2023-10-06 10:22:45 +02:00
Anil Keshav
b34ea9e335
fix (gitOpsUpdateDeployment) add CA bundle options to plain clone and commit to trust enterprise github instances (#4602)
* downloading ca cert bundle when added as config

* adding logging statements

* allowing bats test to handle ca cert

* adding info message

* hard coding file names

* including correct http client util bundle

* removing logging message not needed

* adding cert bundle to commit and push

* improving the condition to add ca cert in commit and push

* fixing unit test

* fixing unit test

* fixing unit test

* fixing unit test

* fixing unit test
2023-09-28 11:31:51 +02:00
Jk1484
2ab1e2a1bc
chore(vault): custom retry check function (#4475)
* vault retry check function

---------

Co-authored-by: I557621 <jordi.van.liempt@sap.com>
2023-09-26 11:57:36 +05:00
Googlom
3744787348
chore(refactor): Switch GitHub actions provider to use github sdk (#4563)
* refactor github package and use builder pattern for client

* switch to github package

* some renamings

* fix panic on uninitialized provider

* fix according to review comments

---------

Co-authored-by: Gulom Alimov <gulomjon.alimov@sap.com>
Co-authored-by: Jordi van Liempt <35920075+jliempt@users.noreply.github.com>
2023-09-20 09:38:45 +00:00
larsbrueckner
ef11a8a726
toolrecord files: remove the timestamp from the filename (#4540)
Co-authored-by: sumeet patil <sumeet.patil@sap.com>
2023-09-12 22:03:49 +05:30
sumeet patil
4de0e3e0f3
fix(cxone): fix test cases (#4564)
Co-authored-by: thtri <thanh.hai.trinh@sap.com>
2023-09-12 19:33:44 +05:30
michaelkubiaczyk
bc8d5efe46
Cxone release supporting applications (#4548)
* Initial in progress

* compiling but not yet functional

* Missed file

* updated checkmarxone step

* Working up to fetching a project then breaks

* Missed file

* Breaks when retrieving projects+proxy set

* Create project & run scan working, now polling

* Fixed polling

* added back the zipfile remove command

* Fixed polling again

* Generates and downloads PDF report

* Updated and working, prep for refactor

* Added compliance steps

* Cleanup, reporting, added groovy connector

* fixed groovy file

* checkmarxone to checkmarxOne

* checkmarxone to checkmarxOne

* split credentials (id+secret, apikey), renamed pullrequestname to branch, groovy fix

* Fixed filenames & yaml

* missed the metadata_generated.go

* added json to sarif conversion

* fix:type in new checkmarxone package

* fix:type in new checkmarxone package

* removed test logs, added temp error log for creds

* extra debugging to fix crash

* improved auth logging, fixed query parse issue

* fixed bug with group fetch when using oauth user

* CWE can be -1 if not defined, can't be uint

* Query also had CweID

* Disabled predicates-fetch in sarif generation

* Removing leftover info log message

* Better error handling

* fixed default preset configuration

* removing .bat files - sorry

* Cleanup per initial review

* refactoring per Gist, fixed project find, add apps

* small fix - sorry for commit noise while testing

* Fixing issues with incremental scans.

* removing maxretries

* Updated per PR feedback, further changes todo toda

* JSON Report changes and reporting cleanup

* removing .bat (again?)

* adding docs, groovy unit test, linter fixes

* Started adding tests maybe 15% covered

* fix(checkmarxOne): test cases for pkg and reporting

* fix(checkmarxOne):fix formatting

* feat(checkmarxone): update interface with missing method

* feat(checkmarxone):change runStep signature to be able to inject dependency

* feat(checkmarxone): add tests for step (wip)

* Adding a bit more coverage

* feat(checkmarxOne): fix code review

* feat(checkmarxOne): fix code review

* feat(checkmarxOne): fix code review

* feat(checkmarxOne): fix integration test PR

* adding scan-summary bug workaround, reportgen fail

* enforceThresholds fix when no results passed in

* fixed gap when preset empty in yaml & project conf

* fixed another gap in preset selection

* fix 0-result panic

* fail when no preset is set anywhere

* removed comment

* initial project-under-app support

* fixing sarif reportgen

* some cleanup of error messages

* post-merge test fixes

* revert previous upstream merge

* fix:formatting

* fix(checkmarxOne):yamllint too many blank lines

* fix(checkmarxOne):unit test

* fix(checkmarxOne):generated code

---------

Co-authored-by: thtri <trinhthanhhai@gmail.com>
Co-authored-by: Thanh-Hai Trinh <thanh.hai.trinh@sap.com>
2023-09-05 21:49:27 +02:00
sumeet patil
f6e6d04408
feat(fortifyExecuteScan): Fortify proxy parameter (#4543) 2023-08-31 17:18:18 +05:30
Googlom
e805beda70
feat(orchestrator): add implementation for GitHub (#4525)
* add comments with examples to methods

* a bit refactoring and cleanup

* actionsURL

* GetBuildStatus

* GetBuildID, GetChangeSet, GetPipelineStartTime

* GetStageName and GetBuildReason

* refactor fetching jobs

* GetJobName and GetJobURL

* chnage GetBuildURL

* refactor actionsURL

* fix guessCurrentJob bug

* unit tests for all

* refactor GetLog

* refactor and fix tests

* change GetBuildURL to use env vars

* fix issues

* leftover

* add comment

* fix according to review comments

---------

Co-authored-by: Gulom Alimov <gulomjon.alimov@sap.com>
Co-authored-by: Jordi van Liempt <35920075+jliempt@users.noreply.github.com>
2023-08-29 09:32:35 +02:00
Christopher Fenner
0c1a52acd8
fix(malwareExecuteScan): add missing error handling (#4536) 2023-08-28 15:06:35 +02:00
Jordi van Liempt
0ba4c2206c
chore(deps): Replace io/ioutil package (#4494)
* update all deprecated ioutil usages

* forgotten changes

* add missing imports

* undo changing comment

* add missing 'os' import

* fix integration test

---------

Co-authored-by: I557621 <jordi.van.liempt@sap.com>
Co-authored-by: Gulom Alimov <gulomjon.alimov@sap.com>
2023-08-16 12:57:04 +02:00
Andrei Kireev
69347fe2af
feat(whitesourceExecuteStep) Unified audit format of vulnerabilities in SARIF file for whitesource (#4465)
* Unified audit state for whitesource step

* reverted unrelated to pr changes

* go fmt

* Fixed tests and formating

* fixed format issue in whitesource/reporting.go

---------

Co-authored-by: sumeet patil <sumeet.patil@sap.com>
2023-08-15 08:26:57 +02:00
Googlom
8c863e457f
sapCumulusUpload step deactivation if its the only active step in stage (#4476)
* implement deactivation logic

* add step condition field

* add unit test and fix evaluateConditions

* add unit test for v1 and fix evaluateConditionsV1

* rollback old evaluator

* rollback v1 evaluator

* move into notActiveCondition and fix unit tests

* add a comment about sapCumulusUpload step

* optimize evaluateConditionsV1 parameters and map memory allocation

* refactor unit tests and add more test cases

* evaluateConditionsV1 refactored

---------

Co-authored-by: Gulom Alimov <gulomjon.alimov@sap.com>
Co-authored-by: Jordi van Liempt <35920075+jliempt@users.noreply.github.com>
2023-08-10 16:11:33 +05:00
Ralf Pannemans
97edad076a
feat(cnbbuild): add build summary (#4506)
Co-authored-by: Pavel Busko <pavel.busko@sap.com>
2023-08-09 09:18:48 +02:00
Johannes Dillmann
b474eb2de7
Refactor buildpacks telemetry (#4467) 2023-08-04 13:31:33 +02:00
Egor Balakin
ff5eddfb7c
orchestrator: fix TestGitHubActions flaky test (#4495)
* fix data race

* fix test

---------

Co-authored-by: Egor Balakin <egor.balakin@sap.com>
2023-08-03 14:25:15 +02:00
Daria Kuznetsova
84dead704b
added querySuite to codeql audit report (#4485) 2023-07-25 15:50:26 +03:00
thtri
c339f32d97
fix(checkmarxOne): changed json report (#4478)
* fix(checkmarxOne): changed json report
2023-07-21 10:16:25 +05:30
Daria Kuznetsova
e117067a66
fix(codeqlExecuteScan): changed audit report format (#4474)
Co-authored-by: sumeet patil <sumeet.patil@sap.com>
2023-07-19 19:16:05 +05:30
Daniel Mieg
994e87479d
ABAP: Add details for EOF errors (#4442)
* Add details for EOF errors

* Add testcase

* remove test

* Add unit test
2023-07-18 09:05:53 +02:00
triThirty
d0e587729d
fix(helmExecute): only expand environment variables start with PIPER_VAULTCREDENTIAL_ (#4468)
* only expand environment variables start with PIPER_VAULTCREDENTIAL_

* use VaultCredentialEnvPrefixDefault instead of hard coding

* go fmt

---------

Co-authored-by: Vyacheslav Starostin <vyacheslav.starostin@sap.com>
2023-07-17 14:16:17 +06:00
Vyacheslav Starostin
3d63ce235c
Add the possibility to push data to Splunk Prod (#4459)
* Add Splunk Prod

* Update test data

* Update naming

* Unit-test: update naming

* Fix

* Fix typo

* Fix test

* go generate
2023-07-14 19:19:57 +06:00
Christopher Fenner
2be4461cde
Update vault.go (#4464)
Co-authored-by: Anil Keshav <anil.keshav@sap.com>
2023-07-14 09:50:20 +02:00
Ralf Pannemans
146c77df52
fix(helmExecute) allowWhitespaces in AdditionalParams again (#4466) 2023-07-13 16:40:57 +02:00
Dmitrii Pavlukhin
ef98a4351f
Unification of audit status in blackduck sarif (#4450)
* added unified status value

* added-unified-status

* added-unified-status

* added-unified-status
2023-07-12 14:22:19 +02:00
triThirty
c105b2129b
helmExecute: Grab values from environment variables (#4374)
* Grab values from environment variables

* use os.ExpandEnv in RunHelmUpgrade function

* use os.ExpandEnv in RunHelmUpgrade function

* use os.ExpandEnv in RunHelmUpgrade function

* Extract new logic to separate func && update tests

---------

Co-authored-by: Mao <bruce.mao@sap.com>
Co-authored-by: Vyacheslav Starostin <32613074+vstarostin@users.noreply.github.com>
2023-07-12 13:14:03 +06:00
Ashly Mathew
34202c708f
fix(npm): Introduce installation for cyclone-node-npm in another folder and fallback to cyclonedx/bom to help users generate BOM (#4390)
* Test

* Try omit

* Introduce global installation and fallback

* Extract to a separate function

* Fix unit tests

* Add root permissions for docker image for Azure

* Install in another folder

* fix unit tests

* Cleanup

* introduce back --no-save,change directory name, fix tests

* add tmp folder to npmignore

* change docker image for guage

* Revert "change docker image for guage"

This reverts commit 45ac7ca9bc.
2023-07-11 16:18:20 +02:00
Pavel Busko
610e212306
feat(cnbBuild) Add support for pre and post-buildpacks (#4448)
* Add pre and post buildpacks

Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
Co-authored-by: Pavel Busko <pavel.busko@sap.com>

* fix integration tests

Co-authored-by: Pavel Busko <pavel.busko@sap.com>
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>

* simplify if clauses

Co-authored-by: Pavel Busko <pavel.busko@sap.com>

---------

Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
2023-07-06 11:34:05 +02:00
Pavel Busko
13f1e94ade
fix(cnbBuild): read dockerConfigJSON from CPE and merge it with user-provided (#4444)
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
2023-07-04 14:19:02 +02:00
Pavel Busko
1befaa80a2
fix(cnbBuild): correctly construct docker config using credentials (#4441)
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
2023-07-03 08:55:06 +00:00
Anil Keshav
a9bab48557
feat (url-logger) Implement logic for a selection classifier (#4411)
* forcing the urls finder to relaxed

* adding a classifier map

* passing the stepName to the kaniko command executor bundle

* pass stepName to maven utils for mavenBuild

* improve enabling of Maven access log generation

* Revert "improve enabling of Maven access log generation"

This reverts commit 80b77223cd.

* Revert "pass stepName to maven utils for mavenBuild"

This reverts commit a4f99ae160.

* use reflection to update command stepName for mavenBuild

* Revert "use reflection to update command stepName for mavenBuild"

This reverts commit ef85c78669.

---------

Co-authored-by: I557621 <jordi.van.liempt@sap.com>
Co-authored-by: Jordi van Liempt <35920075+jliempt@users.noreply.github.com>
2023-06-26 08:47:11 +02:00
Daria Kuznetsova
6eb4c2e72d
fix(codeqlExecuteScan): added waiting for the SARIF file upload (#4409)
* added waiting for the sarif file uploaded & tests

* increased polling time, added timeout for waiting response from server & tests

* fixed handling error while waiting sarif uploaded

* added params for checking sarif uploaded & refactor

* added test logs

* fixed logs and test

* added returning missed error

* changed params descriptions and server response error processing processing

* fixed retrying logic

* increased polling timeout params & refactored
2023-06-20 15:50:28 +03:00
sumeet patil
9b60fcf506
fix(codeqlExecuteScan): fixed logic for getting code-scanning alerts (#4393) 2023-06-14 16:59:01 +05:30
Daria Kuznetsova
cd71282f00
fix(codeqlExecuteScan): pagination call for getting codescanning results (#4370)
pagination call for getting code scanning results

---------

Co-authored-by: sumeet patil <sumeet.patil@sap.com>
2023-05-31 14:07:09 +05:30
Leonard Heilos
c15448b4e0
feat(whitesourceExecuteScan): allow to specify InstallCommand (#4376)
* feat(whitesourceExecuteScan) allow to specify InstallCommand

* reorder imports

---------

Co-authored-by: sumeet patil <sumeet.patil@sap.com>
Co-authored-by: Andrei Kireev <andrei.kireev@sap.com>
2023-05-30 16:00:02 +02:00
sumeet patil
1c018dbff7
feat(codeqlExecuteScan) : auto fill api url (#4369) 2023-05-22 19:59:43 +05:30
Jk1484
27c3c3c4c7
feat(vault): support for multiple general purpose credential paths (#4360)
* created wrapper

* tests added

* update documentation

* tests data race fix

---------

Co-authored-by: Jordi van Liempt <35920075+jliempt@users.noreply.github.com>
2023-05-22 10:49:28 +02:00
larsbrueckner
1e4b88a6f8
detectExecuteScan: fix toolrun data (#4366) 2023-05-17 17:21:03 +05:30
Ashly Mathew
00f376d757
fix(npm) use npm install for cyclonedx (#4357) 2023-05-11 14:32:39 +02:00
Ashly Mathew
f476e8ddce
fix(npm): Update npm cyclonedx/bom to cyclonedx-npm (#4342)
* fix(npm): Update npm cycloneDx to cyclonedx-npm

* Remove --no-validate and fix ut

* remove global

* Change to npm

* Apply suggestions from code review

---------

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2023-05-11 09:55:54 +02:00
Ralf Pannemans
019ef17fd7
feat(helmExecute): Allow custom delimiter (#4312)
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
Co-authored-by: Jan von Loewenstein <jan.von.loewenstein@sap.com>
2023-05-08 16:24:24 +02:00
Pavel Busko
35a55044b4
Add renderSubchartNotes property to helmExecute and kubernetesDeploy (#4238)
Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
2023-05-05 14:23:11 +02:00
michaelkubiaczyk
d12f01d90f
Adding support for CheckmarxOne platform (#4317)
* Initial in progress

* compiling but not yet functional

* Missed file

* updated checkmarxone step

* Working up to fetching a project then breaks

* Missed file

* Breaks when retrieving projects+proxy set

* Create project & run scan working, now polling

* Fixed polling

* added back the zipfile remove command

* Fixed polling again

* Generates and downloads PDF report

* Updated and working, prep for refactor

* Added compliance steps

* Cleanup, reporting, added groovy connector

* fixed groovy file

* checkmarxone to checkmarxOne

* checkmarxone to checkmarxOne

* split credentials (id+secret, apikey), renamed pullrequestname to branch, groovy fix

* Fixed filenames & yaml

* missed the metadata_generated.go

* added json to sarif conversion

* fix:type in new checkmarxone package

* fix:type in new checkmarxone package

* removed test logs, added temp error log for creds

* extra debugging to fix crash

* improved auth logging, fixed query parse issue

* fixed bug with group fetch when using oauth user

* CWE can be -1 if not defined, can't be uint

* Query also had CweID

* Disabled predicates-fetch in sarif generation

* Removing leftover info log message

* Better error handling

* fixed default preset configuration

* removing .bat files - sorry

* Cleanup per initial review

* refactoring per Gist, fixed project find, add apps

* small fix - sorry for commit noise while testing

* Fixing issues with incremental scans.

* removing maxretries

* Updated per PR feedback, further changes todo toda

* JSON Report changes and reporting cleanup

* removing .bat (again?)

* adding docs, groovy unit test, linter fixes

* Started adding tests maybe 15% covered

* fix(checkmarxOne): test cases for pkg and reporting

* fix(checkmarxOne):fix formatting

* feat(checkmarxone): update interface with missing method

* feat(checkmarxone):change runStep signature to be able to inject dependency

* feat(checkmarxone): add tests for step (wip)

* Adding a bit more coverage

* feat(checkmarxOne): fix code review

* feat(checkmarxOne): fix code review

* feat(checkmarxOne): fix code review

* feat(checkmarxOne): fix integration test PR

---------

Co-authored-by: thtri <trinhthanhhai@gmail.com>
Co-authored-by: Thanh-Hai Trinh <thanh.hai.trinh@sap.com>
2023-05-05 14:05:58 +02:00
Jk1484
ffc931aad1
feat(golangBuild): use 'unit' build tag to include tests during test execution (#4345)
* Added unit tag as argument. Added description to runTests command. Changed code generator to have unit build tag in generated unit test files.

* Added unit build tag to all unit test files.

* added to new unit test unit build tag

* Update verify-go.yml

* small fix

---------

Co-authored-by: Muhammadali Nazarov <Muhammadali.Nazarov@acronis.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2023-05-03 21:02:11 +05:00
SiarheiKrystseu
fbb27b2b6d
DESS-1710: modified WaitForBuildToFinish function (#4270)
* DESS-1710: modified WaitForBuildToFinish function - added exception handling

* DESS-1710: fixed parameters in WaitForBuildToFinish

* DESS-1710: added error handling in WaitForBuildToFinish

* DESS-1710: formatted build.go

* DESS-1710: fixed error logging

* DESS-1710: fixed Poll retrying

* DESS-1710: renamed WaitForBuildToFinish

* DESS-1710: refactored WaitForBuildToFinish

* DESS-1710: changed maxRetries to 4 attempt

* DESS-1710: fixed error handling

---------

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Ashly Mathew <ashly.mathew@sap.com>
2023-05-02 09:58:41 +02:00
sumeet patil
6dad124367
feat(codeqlExecuteScan): CodeQL compliance report and check (#4335)
* CodeQL compliance report and check

* fix test cases

---------

Co-authored-by: Daria Kuznetsova <d.kuznetsova@sap.com>
2023-04-28 15:47:05 +02:00
Ashly Mathew
01cfb07d15
feat(gradle): Support for more than one task/flags (#4329)
* feat(gradle) support task list

* Change parameter name to buildFlags to align with other piper steps'
2023-04-24 09:09:31 +02:00