1
0
mirror of https://github.com/SAP/jenkins-library.git synced 2024-12-12 10:55:20 +02:00
Commit Graph

840 Commits

Author SHA1 Message Date
Pavel Busko
26bfec19b3
feat(cnbBuild): support builders with different CNB user ids (#4625)
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
2023-11-02 16:03:11 +01:00
Daria Kuznetsova
df0c9c7b3f
fix(codeqlExecuteScan): exclude codeqlDB from uploaded sources to github (#4652)
* fixed unzipping db sources without db dir

* fixed tests
2023-10-31 14:03:57 +03:00
Artem Bannikov
e11896c5b2
[tmsUpload, tmsExport] Minor: remove confusing comments (#4640)
* Remove confusing comments in tmsClient.go
2023-10-24 11:43:37 +02:00
thtri
f0bd3950c8
fix(cxone):update Name field in reporting for policy as code (#4637) 2023-10-19 19:02:12 +00:00
Jordi van Liempt
d05b53cccd
fix(Vault): retry Vault request (EOF) errors (#4620)
* retry Vault request errors and add extensive logging

* fix sigsegv and change logging

* refining logs

* changing debug to info

---------

Co-authored-by: jliempt <>
Co-authored-by: Muhammadali Nazarov <muhammadalinazarov@gmail.com>
2023-10-19 14:34:03 +02:00
Daria Kuznetsova
6331d1b839
feat(codeqlExecuteScan): cloning project from non-github scm to github #4630
Co-authored-by: sumeet patil <sumeet.patil@sap.com>
2023-10-18 16:50:15 +05:30
Andrei Kireev
49f4c81344
Add new unified fields to Mend and Blackduck SARIF (#4611)
* Add new unified fields to Mend and Blackduck SARIF

* fmt project

---------

Co-authored-by: Dmitrii Pavlukhin <dmitrii.pavlukhin@sap.com>
2023-10-17 11:48:52 +02:00
thtri
646d17bcfe
CheckmarxOne: Fix 1st scan failed due to 403 forbiden (#4633)
* fix(cxone): 1st scan failed due to 403 forbiden

* fix(cxone): 1st scan failed due to 403 forbiden

* fix(cxone): 1st scan failed due to 403 forbiden
2023-10-16 13:06:30 +02:00
Artem Bannikov
09940e8f0a
[tmsUpload, tmsExport] Provide additional log message on successful upload and export to node (#4624)
* Provide additional log message on successful upload and export to node

---------

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
2023-10-13 15:51:21 +02:00
Christopher Fenner
20e1fb53aa
chore(vault): structure vault logs (#4627)
* structure vault logs

* Update client.go
2023-10-11 10:14:31 +02:00
Vyacheslav Starostin
14c7feba1c
feat(artifactPrepareVersion): Support custom certificates (#4617)
* Add CertificateDownload func to certutils package

* Add customTlsCertificateLinks param for artifactPrepareVersion

* Add the possibility to provide custom certs for artifactPrepareVersion

* Update tests

* Return back build flags

* Return back build flags

* Update pkg/certutils/certutils.go

Apply suggestion from code review

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

---------

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2023-10-11 13:58:38 +06:00
Vyacheslav Starostin
a50fad3b1a
feat(Splunk): Use repository and organization from pipelineEnv variable git/* (#4619) 2023-10-10 11:44:24 +02:00
Anil Keshav
40e13f1635
feat (npmExecuteScripts) enhance multi package publish from npm builds (#4579)
* enabling publish to only publish sub packages

* changing directory and then coming back to original after the publish runs

* searching the glob tar and npmrc in the current directory

* excluding build descriptor check and addtional target tool check

* changing the npm pack before publish to run only in sub packages

* removing commented code clean up

* adding the correct npm pack

* improve logging

* fix error handling and a bit style fix

* fix unit tests

* remove commented lines

* respecting build descriptor list when provided

* improve docu for the step param

* fixing linting issues

* improve docu

---------

Co-authored-by: Gulom Alimov <gulomjon.alimov@sap.com>
Co-authored-by: Jordi van Liempt <35920075+jliempt@users.noreply.github.com>
2023-10-09 09:21:09 +02:00
Akramdzhon Azamov
9e64744339
added logic of fetching golang private packages for whitesource step (#4595)
* added logic of fetching golang private packages for whitesource step and detectExecuteScan step

* changed logic of checking by config.PrivateModulesGitToken

* moved func prepareGolangPrivatePackages to golangBuild.go

* fix (gitOpsUpdateDeployment) add CA bundle options to plain clone and commit to trust enterprise github instances (#4602)

* downloading ca cert bundle when added as config

* adding logging statements

* allowing bats test to handle ca cert

* adding info message

* hard coding file names

* including correct http client util bundle

* removing logging message not needed

* adding cert bundle to commit and push

* improving the condition to add ca cert in commit and push

* fixing unit test

* fixing unit test

* fixing unit test

* fixing unit test

* fixing unit test

* feat(kanikoExecute): add dockerfilePath param to multipleImages (#4569)

* add containerDockerfilePath param to multipleImages

* rename ContainerDockerfilePath param to DockerfilePath

* Fix trailing spaces

---------

Co-authored-by: Egor Balakin <egor.balakin@sap.com>
Co-authored-by: Vyacheslav Starostin <32613074+vstarostin@users.noreply.github.com>

* fix(helm): forward sourceRepositoryCredentialsId from groovy to go layer (#4604)

forward sourceRepositoryCredentialsId from groovy to go layer in the same way how this is done for the targetRepositoryCredentialsId

* feat(config): exporting generateConfig function and applying minor changes (#4605)

* exporting generateConfig function and applying minor changes

* Added setConfigOptions to set configOptions variable.
Added possibility to set format output, json or yaml for now.

* Correcting mistake on cmd/getDefaults.go

Co-authored-by: Jordi van Liempt <35920075+jliempt@users.noreply.github.com>

---------

Co-authored-by: Jordi van Liempt <35920075+jliempt@users.noreply.github.com>

* moved func prepareGolangPrivatePackages to pkg/golang

---------

Co-authored-by: Akramdzhon Azamov <MY_NAME@example.com>
Co-authored-by: Andrei Kireev <andrei.kireev@sap.com>
Co-authored-by: Anil Keshav <anil.keshav@sap.com>
Co-authored-by: Egor Balakin <14162703+m1ron0xFF@users.noreply.github.com>
Co-authored-by: Egor Balakin <egor.balakin@sap.com>
Co-authored-by: Vyacheslav Starostin <32613074+vstarostin@users.noreply.github.com>
Co-authored-by: Marcus Holl <marcus.holl@sap.com>
Co-authored-by: Jk1484 <35270240+Jk1484@users.noreply.github.com>
Co-authored-by: Jordi van Liempt <35920075+jliempt@users.noreply.github.com>
2023-10-06 16:22:26 +02:00
Jordi van Liempt
5a56726bde
Revert "chore(vault): custom retry check function (#4475)" (#4616)
This reverts commit 2ab1e2a1bc.
2023-10-06 10:22:45 +02:00
Anil Keshav
b34ea9e335
fix (gitOpsUpdateDeployment) add CA bundle options to plain clone and commit to trust enterprise github instances (#4602)
* downloading ca cert bundle when added as config

* adding logging statements

* allowing bats test to handle ca cert

* adding info message

* hard coding file names

* including correct http client util bundle

* removing logging message not needed

* adding cert bundle to commit and push

* improving the condition to add ca cert in commit and push

* fixing unit test

* fixing unit test

* fixing unit test

* fixing unit test

* fixing unit test
2023-09-28 11:31:51 +02:00
Jk1484
2ab1e2a1bc
chore(vault): custom retry check function (#4475)
* vault retry check function

---------

Co-authored-by: I557621 <jordi.van.liempt@sap.com>
2023-09-26 11:57:36 +05:00
Googlom
3744787348
chore(refactor): Switch GitHub actions provider to use github sdk (#4563)
* refactor github package and use builder pattern for client

* switch to github package

* some renamings

* fix panic on uninitialized provider

* fix according to review comments

---------

Co-authored-by: Gulom Alimov <gulomjon.alimov@sap.com>
Co-authored-by: Jordi van Liempt <35920075+jliempt@users.noreply.github.com>
2023-09-20 09:38:45 +00:00
larsbrueckner
ef11a8a726
toolrecord files: remove the timestamp from the filename (#4540)
Co-authored-by: sumeet patil <sumeet.patil@sap.com>
2023-09-12 22:03:49 +05:30
sumeet patil
4de0e3e0f3
fix(cxone): fix test cases (#4564)
Co-authored-by: thtri <thanh.hai.trinh@sap.com>
2023-09-12 19:33:44 +05:30
michaelkubiaczyk
bc8d5efe46
Cxone release supporting applications (#4548)
* Initial in progress

* compiling but not yet functional

* Missed file

* updated checkmarxone step

* Working up to fetching a project then breaks

* Missed file

* Breaks when retrieving projects+proxy set

* Create project & run scan working, now polling

* Fixed polling

* added back the zipfile remove command

* Fixed polling again

* Generates and downloads PDF report

* Updated and working, prep for refactor

* Added compliance steps

* Cleanup, reporting, added groovy connector

* fixed groovy file

* checkmarxone to checkmarxOne

* checkmarxone to checkmarxOne

* split credentials (id+secret, apikey), renamed pullrequestname to branch, groovy fix

* Fixed filenames & yaml

* missed the metadata_generated.go

* added json to sarif conversion

* fix:type in new checkmarxone package

* fix:type in new checkmarxone package

* removed test logs, added temp error log for creds

* extra debugging to fix crash

* improved auth logging, fixed query parse issue

* fixed bug with group fetch when using oauth user

* CWE can be -1 if not defined, can't be uint

* Query also had CweID

* Disabled predicates-fetch in sarif generation

* Removing leftover info log message

* Better error handling

* fixed default preset configuration

* removing .bat files - sorry

* Cleanup per initial review

* refactoring per Gist, fixed project find, add apps

* small fix - sorry for commit noise while testing

* Fixing issues with incremental scans.

* removing maxretries

* Updated per PR feedback, further changes todo toda

* JSON Report changes and reporting cleanup

* removing .bat (again?)

* adding docs, groovy unit test, linter fixes

* Started adding tests maybe 15% covered

* fix(checkmarxOne): test cases for pkg and reporting

* fix(checkmarxOne):fix formatting

* feat(checkmarxone): update interface with missing method

* feat(checkmarxone):change runStep signature to be able to inject dependency

* feat(checkmarxone): add tests for step (wip)

* Adding a bit more coverage

* feat(checkmarxOne): fix code review

* feat(checkmarxOne): fix code review

* feat(checkmarxOne): fix code review

* feat(checkmarxOne): fix integration test PR

* adding scan-summary bug workaround, reportgen fail

* enforceThresholds fix when no results passed in

* fixed gap when preset empty in yaml & project conf

* fixed another gap in preset selection

* fix 0-result panic

* fail when no preset is set anywhere

* removed comment

* initial project-under-app support

* fixing sarif reportgen

* some cleanup of error messages

* post-merge test fixes

* revert previous upstream merge

* fix:formatting

* fix(checkmarxOne):yamllint too many blank lines

* fix(checkmarxOne):unit test

* fix(checkmarxOne):generated code

---------

Co-authored-by: thtri <trinhthanhhai@gmail.com>
Co-authored-by: Thanh-Hai Trinh <thanh.hai.trinh@sap.com>
2023-09-05 21:49:27 +02:00
sumeet patil
f6e6d04408
feat(fortifyExecuteScan): Fortify proxy parameter (#4543) 2023-08-31 17:18:18 +05:30
Googlom
e805beda70
feat(orchestrator): add implementation for GitHub (#4525)
* add comments with examples to methods

* a bit refactoring and cleanup

* actionsURL

* GetBuildStatus

* GetBuildID, GetChangeSet, GetPipelineStartTime

* GetStageName and GetBuildReason

* refactor fetching jobs

* GetJobName and GetJobURL

* chnage GetBuildURL

* refactor actionsURL

* fix guessCurrentJob bug

* unit tests for all

* refactor GetLog

* refactor and fix tests

* change GetBuildURL to use env vars

* fix issues

* leftover

* add comment

* fix according to review comments

---------

Co-authored-by: Gulom Alimov <gulomjon.alimov@sap.com>
Co-authored-by: Jordi van Liempt <35920075+jliempt@users.noreply.github.com>
2023-08-29 09:32:35 +02:00
Christopher Fenner
0c1a52acd8
fix(malwareExecuteScan): add missing error handling (#4536) 2023-08-28 15:06:35 +02:00
Jordi van Liempt
0ba4c2206c
chore(deps): Replace io/ioutil package (#4494)
* update all deprecated ioutil usages

* forgotten changes

* add missing imports

* undo changing comment

* add missing 'os' import

* fix integration test

---------

Co-authored-by: I557621 <jordi.van.liempt@sap.com>
Co-authored-by: Gulom Alimov <gulomjon.alimov@sap.com>
2023-08-16 12:57:04 +02:00
Andrei Kireev
69347fe2af
feat(whitesourceExecuteStep) Unified audit format of vulnerabilities in SARIF file for whitesource (#4465)
* Unified audit state for whitesource step

* reverted unrelated to pr changes

* go fmt

* Fixed tests and formating

* fixed format issue in whitesource/reporting.go

---------

Co-authored-by: sumeet patil <sumeet.patil@sap.com>
2023-08-15 08:26:57 +02:00
Googlom
8c863e457f
sapCumulusUpload step deactivation if its the only active step in stage (#4476)
* implement deactivation logic

* add step condition field

* add unit test and fix evaluateConditions

* add unit test for v1 and fix evaluateConditionsV1

* rollback old evaluator

* rollback v1 evaluator

* move into notActiveCondition and fix unit tests

* add a comment about sapCumulusUpload step

* optimize evaluateConditionsV1 parameters and map memory allocation

* refactor unit tests and add more test cases

* evaluateConditionsV1 refactored

---------

Co-authored-by: Gulom Alimov <gulomjon.alimov@sap.com>
Co-authored-by: Jordi van Liempt <35920075+jliempt@users.noreply.github.com>
2023-08-10 16:11:33 +05:00
Ralf Pannemans
97edad076a
feat(cnbbuild): add build summary (#4506)
Co-authored-by: Pavel Busko <pavel.busko@sap.com>
2023-08-09 09:18:48 +02:00
Johannes Dillmann
b474eb2de7
Refactor buildpacks telemetry (#4467) 2023-08-04 13:31:33 +02:00
Egor Balakin
ff5eddfb7c
orchestrator: fix TestGitHubActions flaky test (#4495)
* fix data race

* fix test

---------

Co-authored-by: Egor Balakin <egor.balakin@sap.com>
2023-08-03 14:25:15 +02:00
Daria Kuznetsova
84dead704b
added querySuite to codeql audit report (#4485) 2023-07-25 15:50:26 +03:00
thtri
c339f32d97
fix(checkmarxOne): changed json report (#4478)
* fix(checkmarxOne): changed json report
2023-07-21 10:16:25 +05:30
Daria Kuznetsova
e117067a66
fix(codeqlExecuteScan): changed audit report format (#4474)
Co-authored-by: sumeet patil <sumeet.patil@sap.com>
2023-07-19 19:16:05 +05:30
Daniel Mieg
994e87479d
ABAP: Add details for EOF errors (#4442)
* Add details for EOF errors

* Add testcase

* remove test

* Add unit test
2023-07-18 09:05:53 +02:00
triThirty
d0e587729d
fix(helmExecute): only expand environment variables start with PIPER_VAULTCREDENTIAL_ (#4468)
* only expand environment variables start with PIPER_VAULTCREDENTIAL_

* use VaultCredentialEnvPrefixDefault instead of hard coding

* go fmt

---------

Co-authored-by: Vyacheslav Starostin <vyacheslav.starostin@sap.com>
2023-07-17 14:16:17 +06:00
Vyacheslav Starostin
3d63ce235c
Add the possibility to push data to Splunk Prod (#4459)
* Add Splunk Prod

* Update test data

* Update naming

* Unit-test: update naming

* Fix

* Fix typo

* Fix test

* go generate
2023-07-14 19:19:57 +06:00
Christopher Fenner
2be4461cde
Update vault.go (#4464)
Co-authored-by: Anil Keshav <anil.keshav@sap.com>
2023-07-14 09:50:20 +02:00
Ralf Pannemans
146c77df52
fix(helmExecute) allowWhitespaces in AdditionalParams again (#4466) 2023-07-13 16:40:57 +02:00
Dmitrii Pavlukhin
ef98a4351f
Unification of audit status in blackduck sarif (#4450)
* added unified status value

* added-unified-status

* added-unified-status

* added-unified-status
2023-07-12 14:22:19 +02:00
triThirty
c105b2129b
helmExecute: Grab values from environment variables (#4374)
* Grab values from environment variables

* use os.ExpandEnv in RunHelmUpgrade function

* use os.ExpandEnv in RunHelmUpgrade function

* use os.ExpandEnv in RunHelmUpgrade function

* Extract new logic to separate func && update tests

---------

Co-authored-by: Mao <bruce.mao@sap.com>
Co-authored-by: Vyacheslav Starostin <32613074+vstarostin@users.noreply.github.com>
2023-07-12 13:14:03 +06:00
Ashly Mathew
34202c708f
fix(npm): Introduce installation for cyclone-node-npm in another folder and fallback to cyclonedx/bom to help users generate BOM (#4390)
* Test

* Try omit

* Introduce global installation and fallback

* Extract to a separate function

* Fix unit tests

* Add root permissions for docker image for Azure

* Install in another folder

* fix unit tests

* Cleanup

* introduce back --no-save,change directory name, fix tests

* add tmp folder to npmignore

* change docker image for guage

* Revert "change docker image for guage"

This reverts commit 45ac7ca9bc.
2023-07-11 16:18:20 +02:00
Pavel Busko
610e212306
feat(cnbBuild) Add support for pre and post-buildpacks (#4448)
* Add pre and post buildpacks

Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
Co-authored-by: Pavel Busko <pavel.busko@sap.com>

* fix integration tests

Co-authored-by: Pavel Busko <pavel.busko@sap.com>
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>

* simplify if clauses

Co-authored-by: Pavel Busko <pavel.busko@sap.com>

---------

Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
2023-07-06 11:34:05 +02:00
Pavel Busko
13f1e94ade
fix(cnbBuild): read dockerConfigJSON from CPE and merge it with user-provided (#4444)
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
2023-07-04 14:19:02 +02:00
Pavel Busko
1befaa80a2
fix(cnbBuild): correctly construct docker config using credentials (#4441)
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
2023-07-03 08:55:06 +00:00
Anil Keshav
a9bab48557
feat (url-logger) Implement logic for a selection classifier (#4411)
* forcing the urls finder to relaxed

* adding a classifier map

* passing the stepName to the kaniko command executor bundle

* pass stepName to maven utils for mavenBuild

* improve enabling of Maven access log generation

* Revert "improve enabling of Maven access log generation"

This reverts commit 80b77223cd.

* Revert "pass stepName to maven utils for mavenBuild"

This reverts commit a4f99ae160.

* use reflection to update command stepName for mavenBuild

* Revert "use reflection to update command stepName for mavenBuild"

This reverts commit ef85c78669.

---------

Co-authored-by: I557621 <jordi.van.liempt@sap.com>
Co-authored-by: Jordi van Liempt <35920075+jliempt@users.noreply.github.com>
2023-06-26 08:47:11 +02:00
Daria Kuznetsova
6eb4c2e72d
fix(codeqlExecuteScan): added waiting for the SARIF file upload (#4409)
* added waiting for the sarif file uploaded & tests

* increased polling time, added timeout for waiting response from server & tests

* fixed handling error while waiting sarif uploaded

* added params for checking sarif uploaded & refactor

* added test logs

* fixed logs and test

* added returning missed error

* changed params descriptions and server response error processing processing

* fixed retrying logic

* increased polling timeout params & refactored
2023-06-20 15:50:28 +03:00
sumeet patil
9b60fcf506
fix(codeqlExecuteScan): fixed logic for getting code-scanning alerts (#4393) 2023-06-14 16:59:01 +05:30
Daria Kuznetsova
cd71282f00
fix(codeqlExecuteScan): pagination call for getting codescanning results (#4370)
pagination call for getting code scanning results

---------

Co-authored-by: sumeet patil <sumeet.patil@sap.com>
2023-05-31 14:07:09 +05:30
Leonard Heilos
c15448b4e0
feat(whitesourceExecuteScan): allow to specify InstallCommand (#4376)
* feat(whitesourceExecuteScan) allow to specify InstallCommand

* reorder imports

---------

Co-authored-by: sumeet patil <sumeet.patil@sap.com>
Co-authored-by: Andrei Kireev <andrei.kireev@sap.com>
2023-05-30 16:00:02 +02:00
sumeet patil
1c018dbff7
feat(codeqlExecuteScan) : auto fill api url (#4369) 2023-05-22 19:59:43 +05:30