1
0
mirror of https://github.com/SAP/jenkins-library.git synced 2024-12-14 11:03:09 +02:00
Commit Graph

806 Commits

Author SHA1 Message Date
triThirty
d0e587729d
fix(helmExecute): only expand environment variables start with PIPER_VAULTCREDENTIAL_ (#4468)
* only expand environment variables start with PIPER_VAULTCREDENTIAL_

* use VaultCredentialEnvPrefixDefault instead of hard coding

* go fmt

---------

Co-authored-by: Vyacheslav Starostin <vyacheslav.starostin@sap.com>
2023-07-17 14:16:17 +06:00
Vyacheslav Starostin
3d63ce235c
Add the possibility to push data to Splunk Prod (#4459)
* Add Splunk Prod

* Update test data

* Update naming

* Unit-test: update naming

* Fix

* Fix typo

* Fix test

* go generate
2023-07-14 19:19:57 +06:00
Christopher Fenner
2be4461cde
Update vault.go (#4464)
Co-authored-by: Anil Keshav <anil.keshav@sap.com>
2023-07-14 09:50:20 +02:00
Ralf Pannemans
146c77df52
fix(helmExecute) allowWhitespaces in AdditionalParams again (#4466) 2023-07-13 16:40:57 +02:00
Dmitrii Pavlukhin
ef98a4351f
Unification of audit status in blackduck sarif (#4450)
* added unified status value

* added-unified-status

* added-unified-status

* added-unified-status
2023-07-12 14:22:19 +02:00
triThirty
c105b2129b
helmExecute: Grab values from environment variables (#4374)
* Grab values from environment variables

* use os.ExpandEnv in RunHelmUpgrade function

* use os.ExpandEnv in RunHelmUpgrade function

* use os.ExpandEnv in RunHelmUpgrade function

* Extract new logic to separate func && update tests

---------

Co-authored-by: Mao <bruce.mao@sap.com>
Co-authored-by: Vyacheslav Starostin <32613074+vstarostin@users.noreply.github.com>
2023-07-12 13:14:03 +06:00
Ashly Mathew
34202c708f
fix(npm): Introduce installation for cyclone-node-npm in another folder and fallback to cyclonedx/bom to help users generate BOM (#4390)
* Test

* Try omit

* Introduce global installation and fallback

* Extract to a separate function

* Fix unit tests

* Add root permissions for docker image for Azure

* Install in another folder

* fix unit tests

* Cleanup

* introduce back --no-save,change directory name, fix tests

* add tmp folder to npmignore

* change docker image for guage

* Revert "change docker image for guage"

This reverts commit 45ac7ca9bc.
2023-07-11 16:18:20 +02:00
Pavel Busko
610e212306
feat(cnbBuild) Add support for pre and post-buildpacks (#4448)
* Add pre and post buildpacks

Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
Co-authored-by: Pavel Busko <pavel.busko@sap.com>

* fix integration tests

Co-authored-by: Pavel Busko <pavel.busko@sap.com>
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>

* simplify if clauses

Co-authored-by: Pavel Busko <pavel.busko@sap.com>

---------

Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
2023-07-06 11:34:05 +02:00
Pavel Busko
13f1e94ade
fix(cnbBuild): read dockerConfigJSON from CPE and merge it with user-provided (#4444)
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
2023-07-04 14:19:02 +02:00
Pavel Busko
1befaa80a2
fix(cnbBuild): correctly construct docker config using credentials (#4441)
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
2023-07-03 08:55:06 +00:00
Anil Keshav
a9bab48557
feat (url-logger) Implement logic for a selection classifier (#4411)
* forcing the urls finder to relaxed

* adding a classifier map

* passing the stepName to the kaniko command executor bundle

* pass stepName to maven utils for mavenBuild

* improve enabling of Maven access log generation

* Revert "improve enabling of Maven access log generation"

This reverts commit 80b77223cd.

* Revert "pass stepName to maven utils for mavenBuild"

This reverts commit a4f99ae160.

* use reflection to update command stepName for mavenBuild

* Revert "use reflection to update command stepName for mavenBuild"

This reverts commit ef85c78669.

---------

Co-authored-by: I557621 <jordi.van.liempt@sap.com>
Co-authored-by: Jordi van Liempt <35920075+jliempt@users.noreply.github.com>
2023-06-26 08:47:11 +02:00
Daria Kuznetsova
6eb4c2e72d
fix(codeqlExecuteScan): added waiting for the SARIF file upload (#4409)
* added waiting for the sarif file uploaded & tests

* increased polling time, added timeout for waiting response from server & tests

* fixed handling error while waiting sarif uploaded

* added params for checking sarif uploaded & refactor

* added test logs

* fixed logs and test

* added returning missed error

* changed params descriptions and server response error processing processing

* fixed retrying logic

* increased polling timeout params & refactored
2023-06-20 15:50:28 +03:00
sumeet patil
9b60fcf506
fix(codeqlExecuteScan): fixed logic for getting code-scanning alerts (#4393) 2023-06-14 16:59:01 +05:30
Daria Kuznetsova
cd71282f00
fix(codeqlExecuteScan): pagination call for getting codescanning results (#4370)
pagination call for getting code scanning results

---------

Co-authored-by: sumeet patil <sumeet.patil@sap.com>
2023-05-31 14:07:09 +05:30
Leonard Heilos
c15448b4e0
feat(whitesourceExecuteScan): allow to specify InstallCommand (#4376)
* feat(whitesourceExecuteScan) allow to specify InstallCommand

* reorder imports

---------

Co-authored-by: sumeet patil <sumeet.patil@sap.com>
Co-authored-by: Andrei Kireev <andrei.kireev@sap.com>
2023-05-30 16:00:02 +02:00
sumeet patil
1c018dbff7
feat(codeqlExecuteScan) : auto fill api url (#4369) 2023-05-22 19:59:43 +05:30
Jk1484
27c3c3c4c7
feat(vault): support for multiple general purpose credential paths (#4360)
* created wrapper

* tests added

* update documentation

* tests data race fix

---------

Co-authored-by: Jordi van Liempt <35920075+jliempt@users.noreply.github.com>
2023-05-22 10:49:28 +02:00
larsbrueckner
1e4b88a6f8
detectExecuteScan: fix toolrun data (#4366) 2023-05-17 17:21:03 +05:30
Ashly Mathew
00f376d757
fix(npm) use npm install for cyclonedx (#4357) 2023-05-11 14:32:39 +02:00
Ashly Mathew
f476e8ddce
fix(npm): Update npm cyclonedx/bom to cyclonedx-npm (#4342)
* fix(npm): Update npm cycloneDx to cyclonedx-npm

* Remove --no-validate and fix ut

* remove global

* Change to npm

* Apply suggestions from code review

---------

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2023-05-11 09:55:54 +02:00
Ralf Pannemans
019ef17fd7
feat(helmExecute): Allow custom delimiter (#4312)
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
Co-authored-by: Jan von Loewenstein <jan.von.loewenstein@sap.com>
2023-05-08 16:24:24 +02:00
Pavel Busko
35a55044b4
Add renderSubchartNotes property to helmExecute and kubernetesDeploy (#4238)
Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
2023-05-05 14:23:11 +02:00
michaelkubiaczyk
d12f01d90f
Adding support for CheckmarxOne platform (#4317)
* Initial in progress

* compiling but not yet functional

* Missed file

* updated checkmarxone step

* Working up to fetching a project then breaks

* Missed file

* Breaks when retrieving projects+proxy set

* Create project & run scan working, now polling

* Fixed polling

* added back the zipfile remove command

* Fixed polling again

* Generates and downloads PDF report

* Updated and working, prep for refactor

* Added compliance steps

* Cleanup, reporting, added groovy connector

* fixed groovy file

* checkmarxone to checkmarxOne

* checkmarxone to checkmarxOne

* split credentials (id+secret, apikey), renamed pullrequestname to branch, groovy fix

* Fixed filenames & yaml

* missed the metadata_generated.go

* added json to sarif conversion

* fix:type in new checkmarxone package

* fix:type in new checkmarxone package

* removed test logs, added temp error log for creds

* extra debugging to fix crash

* improved auth logging, fixed query parse issue

* fixed bug with group fetch when using oauth user

* CWE can be -1 if not defined, can't be uint

* Query also had CweID

* Disabled predicates-fetch in sarif generation

* Removing leftover info log message

* Better error handling

* fixed default preset configuration

* removing .bat files - sorry

* Cleanup per initial review

* refactoring per Gist, fixed project find, add apps

* small fix - sorry for commit noise while testing

* Fixing issues with incremental scans.

* removing maxretries

* Updated per PR feedback, further changes todo toda

* JSON Report changes and reporting cleanup

* removing .bat (again?)

* adding docs, groovy unit test, linter fixes

* Started adding tests maybe 15% covered

* fix(checkmarxOne): test cases for pkg and reporting

* fix(checkmarxOne):fix formatting

* feat(checkmarxone): update interface with missing method

* feat(checkmarxone):change runStep signature to be able to inject dependency

* feat(checkmarxone): add tests for step (wip)

* Adding a bit more coverage

* feat(checkmarxOne): fix code review

* feat(checkmarxOne): fix code review

* feat(checkmarxOne): fix code review

* feat(checkmarxOne): fix integration test PR

---------

Co-authored-by: thtri <trinhthanhhai@gmail.com>
Co-authored-by: Thanh-Hai Trinh <thanh.hai.trinh@sap.com>
2023-05-05 14:05:58 +02:00
Jk1484
ffc931aad1
feat(golangBuild): use 'unit' build tag to include tests during test execution (#4345)
* Added unit tag as argument. Added description to runTests command. Changed code generator to have unit build tag in generated unit test files.

* Added unit build tag to all unit test files.

* added to new unit test unit build tag

* Update verify-go.yml

* small fix

---------

Co-authored-by: Muhammadali Nazarov <Muhammadali.Nazarov@acronis.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2023-05-03 21:02:11 +05:00
SiarheiKrystseu
fbb27b2b6d
DESS-1710: modified WaitForBuildToFinish function (#4270)
* DESS-1710: modified WaitForBuildToFinish function - added exception handling

* DESS-1710: fixed parameters in WaitForBuildToFinish

* DESS-1710: added error handling in WaitForBuildToFinish

* DESS-1710: formatted build.go

* DESS-1710: fixed error logging

* DESS-1710: fixed Poll retrying

* DESS-1710: renamed WaitForBuildToFinish

* DESS-1710: refactored WaitForBuildToFinish

* DESS-1710: changed maxRetries to 4 attempt

* DESS-1710: fixed error handling

---------

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Ashly Mathew <ashly.mathew@sap.com>
2023-05-02 09:58:41 +02:00
sumeet patil
6dad124367
feat(codeqlExecuteScan): CodeQL compliance report and check (#4335)
* CodeQL compliance report and check

* fix test cases

---------

Co-authored-by: Daria Kuznetsova <d.kuznetsova@sap.com>
2023-04-28 15:47:05 +02:00
Ashly Mathew
01cfb07d15
feat(gradle): Support for more than one task/flags (#4329)
* feat(gradle) support task list

* Change parameter name to buildFlags to align with other piper steps'
2023-04-24 09:09:31 +02:00
Pavel Busko
5d8e89b08a
feat(cnbBuild): use SHA256 hashed values for redacted telemetry properties (#4328)
* feat(cnbBuild): use SHA256 hashed values for redacted telemetry properties

* update unit tests
2023-04-18 09:10:38 +02:00
Maurice Breit
4f4e667797
add ascAppUpload step (#4286)
* add ascAppUpload step

add step that uploads an app binary to Application Support Center (ASC)

* re-run go generate

* fix typo in CODEOWNERS

* Update CODEOWNERS
2023-04-18 08:56:32 +02:00
Jordi van Liempt
e3935ca088
feat(vault): Vault secret rotation for GH Actions (#4280)
* rotate Vault secret on GH Actions

* test alternative sodium package

* try doing it without libsodium

* disable validity check for testing purposes

* basic unit test

* re-enable secret validity check

* tidy

* tidy parameters

* forgot to update param names in code

* apply review feedback

* improve error logging

* update step metadata

* apply metadata suggestion from review

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* align githubToken param

* Fix secretStore

* Add alias for githubToken

* Move logic to separate file

---------

Co-authored-by: I557621 <jordi.van.liempt@sap.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Vyacheslav Starostin <vyacheslav.starostin@sap.com>
2023-04-17 08:35:13 +02:00
Christopher Fenner
99603cdee9
chore(tests): correct test assertions (#4276) 2023-04-11 23:15:00 +02:00
tiloKo
126fbbcc69
reduce Log Output (Info -> Debug) (#4322)
Co-authored-by: ffeldmann <f.feldmann@sap.com>
2023-04-06 08:31:22 +02:00
Daniel Mieg
7147209e3e
Improve logging for ABAP steps (#4316)
* Fix logs

* Increase number of entries
2023-04-04 12:46:03 +02:00
Daniel Mieg
549b32c675
Adapt to backend API changes (#4309)
* Remove legacy logging

* Implement new requests

* Improve Tests

* Adapt tests

* Refactor

* Fix tests
2023-03-31 15:26:38 +02:00
Pavel Busko
81d9a0ac8c
feat(cnbBuild): support Vault general purpose secrets as a binding content source (#4281)
* feat(cnbBuild): support Vault general purpose secrets as a binding content source

Co-authored-by: Pavel Busko <pavel.busko@sap.com>

* fix npm project integration test

Co-authored-by: Pavel Busko <pavel.busko@sap.com>

---------

Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
Co-authored-by: Sumit Kulhadia <sumit.kulhadia@sap.com>
2023-03-31 09:36:59 +02:00
gerstneralex
f5c33d51bb
Tms export (#4160)
* Change parameter type of nodeExtDescriptorMapping

(cherry picked from commit ca7ce0485a)

* Remove usage of the depricated ioutil package

(cherry picked from commit 9821915b33)

* Fix cmd failure if neither git/commitId nor customDescription are
provided

(cherry picked from commit c362681e45)

* Fix unit test

(cherry picked from commit 53a90aabb5)

* Step metadata, step code generation

* change type of nodeExtDescriptorMapping for export

* Refactoring and export implementation

* integration test

* Add export step

* Integration test

* format

* discard piper.go

* Review related changes

* restore piper.go

* remove unused method

* Extend documentation

* Add parameter useGoStep to tmsUpload.groovy

* Regenerate steps

* Rename function

* refactor constants

* Add error path tests

* Move some code to tms package

* Move more code to tms

* Combine tmsUpload, tmsUtils

* Add groovy wrapper

* add parameters to groovy step

* add import

* jenkinsUtils instance

* comment namedUser logic in groovy

* namedUser param

* remove logic for namedUser param

* Remove TMS integration tests

* discard changes in tmsUpload.groovy

* Remove parameters

* Restore parameters

* Change type of NodeExtDescriptorMapping to map[string]interface{}

* tmsUpload: Change type of NodeExtDescriptorMapping to map

* Resolve ioutil deprecation

* Review related changes

* Formatting

* Review related improvements

* Add tmsUtils test

* Formatting tmsUtils_test

* Remove parameters from groovy wrapper

* Remove tmsUtils_test

* Add TMS steps to fieldRelatedWhitelist

* Add integration test

* Add test to github_actions_integration_test_list.yml

* Move test helper method

* Step documentation placeholder

* Remove parameter StashContent

* Restore cmd/integrationArtifactTransport.go

---------

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
2023-03-27 16:55:29 +02:00
Oliver Feldmann
92a782a6c3
[Integration Suite] Store response in CPE (#4277)
* Store response body in CPE

* Also store response headers in CPE
2023-03-27 14:34:59 +00:00
Oliver Nocon
7c4262b0c6
chore (containerExecuteStructureTests): switch default container (#4288) 2023-03-21 08:06:22 +01:00
gerstneralex
cfe21ebb7e
it test (#4134)
* it test

* transfer credentials

* Change parameter type of nodeExtDescriptorMapping

* Extend test

* Fix unit test

* Remove usage of the depricated ioutil package

* Fix cmd failure if neither git/commitId nor customDescription are
provided

* Extend test

* Add TMS test to the job matrix

* Map env. variable

* Remove usage of the env. TMS_UPLOAD_IT_KEY

* remove os

* update test

* use os.Gerenv

* test fix

* Update integration-tests.yml

* env mapping in it pr workflow

* print tmsServiceKey

* read env with upper case

* Update integration-tests.yml

* Update integration-tests.yml

* Update integration-tests-pr.yml

* Delete cover.out

* Remove TMS service key from environment in integration test workflow job

* Extend integration tests

* Revert change parameter type of nodeExtDescriptorMapping

* Extend tests

* Extend tests

* Remove unused method

* Change default TR description

* Add check for custom description

* Remove personal data from MTARs

* Register client secret to log as secret

* Move RegisterSecret to earlier point in runtime

* RegisterSecret for encodedUsernameColonPassword

* Update integration/integration_tms_upload_test.go

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>

* Use one test data directory

* Add a negative test

* fix config file name

---------

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
2023-03-07 17:49:28 +01:00
gerstneralex
34fc844ac0
Tms upload fixes (#4204)
* Fix proxy usage in tmsUpload

* Fix no default description if custom description is not provided
2023-03-03 18:23:40 +01:00
Akramdzhon Azamov
f4fbf0f1ed
feat(detectExecuteScan) execution of rapid scans (#4211)
Co-authored-by: akram8008 <900658008.akram@email.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Andrei Kireev <andrei.kireev@sap.com>
Co-authored-by: ffeldmann <f.feldmann@sap.com>
Co-authored-by: sumeet patil <sumeet.patil@sap.com>
2023-03-02 11:04:21 +01:00
Vyacheslav Starostin
9aac0ae06c
GHActions: Add the possibility to fetch logs (#4250)
* Initial commit

* getStageId added

* setting token for orchestrator

* Add possibility to fetch logs (GHActions)

* Update

* Clean up: delete test log file

* go mod tidy

* Rename token

* Rename token

* Apply suggestion from code review

* Update

* Correct err msg

* Add new line

* Update pkg/orchestrator/gitHubActions.go

Co-authored-by: ffeldmann <f.feldmann@sap.com>

* Update pkg/orchestrator/gitHubActions.go

Co-authored-by: ffeldmann <f.feldmann@sap.com>

* Update pkg/orchestrator/gitHubActions.go

Co-authored-by: ffeldmann <f.feldmann@sap.com>

* Add httpHeader variable && little updates

* Clean up

* Make structs unexported

---------

Co-authored-by: Aibyn Sadu <aibaend@yandex.ru>
Co-authored-by: ffeldmann <f.feldmann@sap.com>
2023-03-01 10:38:40 +06:00
Vyacheslav Starostin
1e242a7fd6
Orchestrator/GHActions: fix pr-number determination (#4237)
* Orchestrator/GHActions: fix pr-number determination

* Apply suggestions from code review

* go fmt

---------

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2023-02-22 15:38:33 +06:00
Ralf Pannemans
906512a162
feat(cnbbuild): allow bindings to have multiple keys (#4231)
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
Co-authored-by: Pavel Busko <pavel.busko@sap.com>
2023-02-16 15:29:08 +01:00
tiloKo
9e9b402b49
Variable dotted version string fix #2 (#4233) 2023-02-16 12:47:02 +01:00
Andrei Kireev
4098e70742
Added audit information in Sarif file in detectExecuteStep (#4198)
* Added audit information in sarif file in blackduck step
---------

Co-authored-by: sumeet patil <sumeet.patil@sap.com>
2023-02-07 17:02:28 +01:00
tiloKo
501f7d214d
Variable Dotted Version String fixes (#4220) 2023-02-07 10:02:40 +01:00
Vyacheslav Starostin
5201733150
fix(orchestrator): determine if it is a pull request (GH Actions) (#4218) 2023-02-06 17:47:59 +06:00
Anil Keshav
f3c1bf6edc
fix (npmExecuteScripts) fix for npmrc _auth (#4212)
* fix for npmrc _auth

* correct prefix removal

* fix unit test

* bringing back unit test

* adapt unit test

* typo fix
2023-02-03 12:50:53 +01:00
Andrei Kireev
d069acab59
Fixed issue with not allowing to update parameter resolveAllDependencies (#4202)
Co-authored-by: sumeet patil <sumeet.patil@sap.com>
2023-02-01 22:19:01 +01:00