1
0
mirror of https://github.com/SAP/jenkins-library.git synced 2024-12-12 10:55:20 +02:00
Commit Graph

4091 Commits

Author SHA1 Message Date
Anke Ravalitera
b6a5ceaa03
Update gCTS steps (#3673)
* gCTS Scenario and two images

* Updated gCTS Scenario

* updated gCTS_Scenario.md and images

* added checkstyle stage

* upated the gCTSExecuteABAPQualityChecks.yaml file with severity information

* updates to gCTSExecuteABAPQualityChecks and gCTSExecuteABAPUnitTests steps

* updated gCTS_Scenario.md and relevant images

* Add more explanation for workspace parameter, change atc variant default value to DEFAULT

* Fix a type in description of workspace parameter

* changes to gctsExecuteABApUnitTests.md gctsExecuteABAPQualityChecks.yaml and gctsExecuteABApUnitTests.yaml

* final updates to gCTS steps and scenario

* added env. to GIT_COMMIT in gCTSExecuteABAPQualityChecks.md and gCTSExecuteABAPUnitTests.md

* remove the gScenario description from the branch

* Correct column typo in yaml file and generate files

* Add a new line in docu for gCTSExecuteABAPUnitTests

* Remove blank line from docu in gctsExecuteABAPUnitTests

Co-authored-by: Rinita Asani <rinita.asani@sap.com>
Co-authored-by: Sarat Krishnan <78093145+sarat-krk@users.noreply.github.com>
2022-03-29 21:24:10 +02:00
TheShifter
188e743f7b
PythonBuild: Implementation of pythonBuild step (#3483)
* Implementation of pythonBuild step

* minor update and refactoring

* minor update

* add integration test and test project to testdata dir

* remove generated build data dir

* Rewrite some logic. Minor fix in integration tests for python

* Add new input parameters to pythonBuild.yaml

* rewrite logic
remove some checks

* rollback

* resolve merge conflict in piper.go
Update logic in python build. Create bom now works fine

* remove duplicate line

* refactoring fix

* resolve comment. Remove install build and change build command. Change twine upload command

* add groovy wrapper for pythonBuild step

* Rewrite tests. Remove some cheks from pythonBuild.go

* add some test to pythonBuild_test.go

* Add some parameters and credentials to the pythonBuild.groovy

* fix issue in unit tests

* add pythonBuild to fieldRelatedWhitelist

* update integration test for pythonBuild

* add imports

* update integration tests and add a new one

* minor fix

* fix some issues in integration tests

* update integration tests. Make it works again

Co-authored-by: Anil Keshav <anil.keshav@sap.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-03-29 19:01:44 +02:00
Anil Keshav
9ab7894c1a
adding parallel execution for integration test of golang gradle and maven (#3677)
Co-authored-by: anilkeshav27 <you@example.com>
2022-03-29 07:38:11 +02:00
ffeldmann
a9329fba9e
Reuse SOURCEBRANCH for branch name (#3672) 2022-03-28 13:48:55 +02:00
ffeldmann
ccc1c976ee
fix(orchestrator) usage of correct env variables (#3650)
* Reorders getApiInformation, changes variables to get start time, adjusts and adds test cases
* Changes the way to get apiInformation and reduces number of requests
* Changes getting pipeline start time from correct env variable
* Refactors getApiInformation functionality
* Adds GetBuildReason() for Azure and Jenkins
* Updates JobURL for ADO
2022-03-28 09:52:15 +02:00
Vitalii Sidorov
5926aa7f77
Add helmExecute.groovy (#3658)
* Add helmExecute.groovy

* Add helmExecute in fieldRelatedWhitelist

* Fix helmExecute.groovy

Co-authored-by: “Vitalii <“vitalii.sidorov@sap.com”>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-03-25 09:45:52 +01:00
Johannes Dillmann
08f26e9605
chore(cnbBuild) unify integration test args (#3666)
Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
2022-03-25 08:01:32 +01:00
Christian Volk
26bf3808fe
chore(checkmarxExecuteScan): split and trim filterPattern (#3661) 2022-03-23 11:45:05 +01:00
Christian Volk
22f6aa156f
feat(docker): use crane for pulling docker images (#3652) 2022-03-23 10:02:00 +01:00
Sven Merk
f06890a9b2
SARIF format and GHIssue format improvements (#3646)
* Improve reporting

* Fix location

* Align casing

* Fix severity mapping

* Fix format

* Improve title

* Title format

* Fix severity

* Align title

* Fix schema reference

* Fix schema reference

* Fix fmt

* Fix fmt2

* Fix tests

* fix(sarif): proper handling of omitempty in SnippetSarif

* fix(fortifyExecuteScan): sarif format version

* Addressing comments

* Fix SARIF

* fix(sarif): omitempty handling

* fix(fortifyExecuteScan): pointer indirection

* Added TODOs for audit data

Co-authored-by: Xavier Goffin <x.goffin@sap.com>
Co-authored-by: xgoffin <86716549+xgoffin@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-03-22 14:47:19 +01:00
Ralf Pannemans
62a7f6336a
feat(cnbbuild) use commitid fom cpe as tag if artifact version is not available (#3659)
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
Co-authored-by: Sumit Kulhadia <sumit.kulhadia@sap.com>
2022-03-22 14:25:20 +01:00
Oliver Nocon
fed08c2399
fix(whitesourceExecuteScan): proper container options for golang (#3660) 2022-03-22 10:40:33 +01:00
Oliver Nocon
eef6c028aa
feat(artifactPrepareVersion): context-sensitive versioningType (#3579)
* feat(artifactPrepareVersion): context-sensitive versioningType

Add functionality to switch to `versioningType: cloud_noTag` for
* PR voting
* scheduled and optimizes pipelines

* add missing file

* update tests

* update docs

* chore: update generation

* chore: fix tests

Co-authored-by: ffeldmann <f.feldmann@sap.com>
2022-03-22 08:31:59 +01:00
Siarhei Pazdniakou
db5360fb89
feat(gradleExecuteBuild): BOM creation. Integration tests (#3603)
* Implemented bom creation

* Made small fixes. Added integration tests

* go generate

* minor fixes

* fix tests

* Added unit tests

* minor fixes

* use fileutils

* integration tests optimization

* change integraton tests timeout to 25m

* Fix Inclusive Language warnings

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-03-21 10:17:03 +01:00
Christian Volk
4b29f2e001
fix(kaniko): deactivate reproducible flag (#3653) 2022-03-21 09:14:36 +01:00
Oliver Nocon
a466b60aaf
feat(helmExecute): allow reading specific credentials from cpe (#3647) 2022-03-21 08:41:22 +01:00
Siarhei Pazdniakou
a1e2d8e8f3
Increased timeout for integration tests (#3648) 2022-03-18 16:30:41 +01:00
Michael
f4a29554ed
(chore) bump to dtzar/helm-kubectl:3.8.0 (#3573)
Signed-off-by: Michael Sprauer <Michael.Sprauer@sap.com>

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-03-18 14:56:36 +01:00
Maximilian Braun
040a2c36ef
fix(golangBuild): order of arguments (#3645)
* fix: order of go build arguments

As per https://pkg.go.dev/cmd/go#hdr-Compile_packages_and_dependencies 
The `go build [-o output] [build flags] [packages] ` -ldflags shall be put before packages.

The build may fail this way
```
running command: go build -trimpath -o foo-linux.amd64 ./cmd/main.go -ldflags '-linkmode=external'
named files must be .go files: -ldflags

```
2022-03-18 12:03:45 +01:00
Vitalii Sidorov
2a4052d13c
feat(helmExecute): run complete lint, build, publish cycle (#3546)
* Add runHelmCommand

* Add dryRun for debug

* Add default case in helmExecute

* Fix unit-tests

* small fix

* Fix RunHelmAdd and change RunHelmPublish methods

* Fix RunHelmPublish

* Fix unit-tests

* Fix unit-test

* small fix

* small fix

* small fix

* Add LintFlag PackageFlag PublishFlag flags

* Add tests for httpClient.go

* test

* test

* smal fix

* small fix

* Add getting name and version from Chart.yaml

* Add test

* Fix

* small fix

* Fix according to comments

* small fix

Co-authored-by: “Vitalii <“vitalii.sidorov@sap.com”>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Vitalii Sidorov <vitalii_sidorov@sap.com>
2022-03-17 17:13:34 +01:00
Sven Merk
c30e93bc6a
feat(detectExecuteScan): SARIF export and GH issue creation (#3637)
* Added SARIF and GH issue creation
2022-03-17 15:32:48 +01:00
xgoffin
3f6e4b9e3b
feat(fortifyExecuteScan): added parameter to generated sarif file (#3644)
* fix(sarif): change format to fit omitempty cases better

* feat(fortifyExecuteScan): include category in sarif file

* fix(fortifyExecuteScan): access to undefined pointer in some cases

Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2022-03-17 13:09:15 +01:00
Anil Keshav
818636b048
feat (checkIfStepActive) including cpe conditions to check if a step is active / not active (#3642)
* including cpe struct

* evaluating types

* adding environment root path

* unit tests

Co-authored-by: anilkeshav27 <you@example.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-03-17 12:09:16 +01:00
Christian Volk
58b30cc99b
fix(golangBuild): publishing of binaries (#3643) 2022-03-17 11:07:51 +01:00
Maurice Breit
9f2d8a70b1
(fix) match regexes in sliceContains to support vaultSecretNames (#3618)
* (fix) match regexes in sliceContains to support vaultSecretNames

* add test for regex matching in sliceContains

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-03-17 09:41:51 +01:00
Oliver Nocon
504f076613
feat: support cpe credentials for multiple repos (#3641) 2022-03-17 08:01:00 +01:00
Oliver Feldmann
4208bc050c
Add bearer token retrieval function (#3595)
* Add bearer token retrieval function

Retrieving a bearer token from the xsuaa service on BTP is always the
same. With these functions one can retrieve a bearer token and set it
to the given header as 'Authorization'.

* CodeClimate fixes

* Refactor test

* Add basic auth to token retrieve request

Co-authored-by: Thorsten Duda <thorsten.duda@sap.com>
2022-03-16 11:22:48 +01:00
Oliver Nocon
7ec512cb9f
feat(artifactPrepareVersion): helm & propagate version (#3627)
* feat(artifactPrepareVersion): helm & propagate version

* chore: small refactoring

* chore: fix linting issue

* fix version persistence
2022-03-15 09:08:24 +01:00
Daniel Mieg
6651eaf6c8
Cloud Platform -> BTP (#3634) 2022-03-14 15:12:35 +01:00
Oliver Nocon
1151fc4de0
chore: export config functions (#3631)
* chore: export config functions

* add docs
2022-03-14 12:33:52 +01:00
xgoffin
dfd2278639
feat(fortifyExecuteScan): full FPR to SARIF implementation (#3604)
* feat(FPRtoSARIF): boilerplate & comments

* Feat(Ingest): Build done, Vulnerabilities partway

* feat(Vulnerabilities): now entirely parsed

* feat(FprToSarif): integration in Piper step, full xml structure

* feat(fpr_to_sarif): base program. Need to replace names in messages

* feat(fpr_to_sarif): message substitution and custom definition integration

* fix(fpr_to_sarif): missing replacement in tools object

* fix(fpr_to_sarif): failing unit test

* Fix fortify folder creation for generating sarif

* deletion of unzip folder

* feat(fpr_to_sarif): better unit test

* fix(fpr_to_sarif): pr tests failing

* feat(fortifyExecuteScan): complete SARIF file generation

* fix(fpr_to_sarif): add extra check and test to prevent panics

* rebase onto master, fix ALL conflicts, adapt code and format

* fix missing added properties

* fix(SARIF): structure

* fix(whitesource): wrong sarif structures

* Update pkg/fortify/fpr_to_sarif.go

* Update pkg/format/sarif.go

* Update pkg/format/sarif.go

Co-authored-by: Sumeet PATIL <sumeet.patil@sap.com>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2022-03-14 11:26:05 +01:00
ffeldmann
b224f2294c
Activates debug information for environment variables (#3630)
* Activates debug information for environment variables

* Adds tests for environment variable reading

* Reduces batch size to send messages to Splunk to 5000
2022-03-14 10:17:55 +01:00
Daniel Bernd
69fc4103c2
Response logging (#3599)
* Log adaption 1

* log handling 2

* log Handling 3

* log & handle body

* Unit Test & response Nil

* Response not Nil

* no change

no change

* no change

no change

* no change

no change

Co-authored-by: Daniel Bernd <93763187+danManSAP@users.noreply.github.com>
Co-authored-by: Daniel Mieg <56156797+DanielMieg@users.noreply.github.com>
2022-03-14 09:49:51 +01:00
Anil Keshav
fcbf28e3ff
checking if the maven install/verfiy return errors (#3625)
Co-authored-by: anilkeshav27 <you@example.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-03-11 10:02:50 +01:00
Oliver Nocon
2a56723d40
feat(kanikoExecute): Trim names for multi-image builds (#3617)
* feat(kanikoExecute): Trim names for multi-image builds

* chore: fix yaml linting issue
2022-03-11 09:47:44 +01:00
Anil Keshav
809a90ee27
adding vault references for username and password (#3624)
Co-authored-by: anilkeshav27 <you@example.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-03-11 09:25:46 +01:00
Thorsten Duda
23fde71bf8
Removing deprecated cx-server from doc (#3626)
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-03-11 08:56:13 +01:00
Oliver Nocon
2c837927d4
chore(whitesourceExecuteScan): Gradle config changes (#3621)
* chore(whitesourceExecuteScan): Gradle config changes

supersedes #3293
closes #3293

* update config

Co-authored-by: ffeldmann <f.feldmann@sap.com>
2022-03-11 08:18:21 +01:00
Anil Keshav
157f6d4e35
feat (checkIfStepActive) enhance to include not active conditions (#3616)
* including negative conditions

* clean up and todos

* removing debug logging

* clean up

* fix unit test name

* fixing unit tests

* negative stage test

Co-authored-by: anilkeshav27 <you@example.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-03-10 13:35:28 +01:00
Christian Volk
b9c0aa008f
feat(piperExecuteBin): support running images from private docker registries (#3622) 2022-03-09 17:35:57 +01:00
Johannes Dillmann
3708f274cc
feat(cnbBuild) read target image name from github cpe (#3620)
Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
2022-03-09 14:06:26 +01:00
maheshsrikrishnan
64a00c540a
New piper step APIProviderDownload contribution (#3349)
* added store file function in cpi common utils

Change-Id: Ia429a2792266e082d139025a71799d21c30a7df9

* Added api provider steps

Change-Id: Icd2829a91db4c53d0de2330822d2b33933973868

* Update apiProviderDownload.yaml

* fixed yaml JLINT issue

Change-Id: Iac974abc30fa00e68c0177072b93716b0af5e0c5

* Removed trailing spaces

Change-Id: I927e9314fce6e9cab68d6b97577c7c96bb2bddad

* Resolved common steps groovy script conflict

Change-Id: I3ad144b618e1c77953aaeccaa5bf7309aff77ca9

* Change for conflict resolution

Change-Id: Ic955833eca844f090b7983f99f9d3649ebb981c7

* Fixed method name and its corresponding implementation

Change-Id: I465c1f1d5306bb978386de9efca3c521e385b89c

* Moved re-usable function to commonUtils package

Change-Id: Ide06462b01caeb2bf438ad7661e01c15bf8e8e24

* Changed the implementation to use existing writeFile method

* Fixed review comments on documentation and test structuring

Change-Id: Ifebd2f4b50754b2097b2d564fb3cc37c433ef6c9

* Fixed documentation alignment issues

* Fixed spaces issue

Change-Id: I834bd94e01bce72e7f81ab49ba32671c91c66ca9

* Documentation removed extra spaces

Change-Id: I9a639d76ed9b81c870f18349504044bb70753b52

* Fixed doc build issue

Change-Id: I96c3e15e73834b64f8b8e3432ce59f6b037f93fd

* Fixed documentation build issues

Change-Id: I7fca2ba69bc7b7298ee300ccd1ae16a6238dc96b

* Re-generated code for fixing build failure

Change-Id: I22b7ee6162f643d9f3b60f6a33eb7858927182a0

* Adopted file utils & mock

Change-Id: Ic46462003527f41df64395a5a615c19bf374e8ef

* Removed ioutil call in the test & adopted error variable names

* Removed commented lines

Change-Id: I99a12e39bc04323e9c19f1409d97eeca267e6fdb

* Added test for asserting file download and adopted error variables

Change-Id: I49463a3b75987bf68f5261d45602d2d7bd960a05

* Added download path assertion positive & negative case

Change-Id: Ieee461c3973b9dfa8f395dc936e4241ff9694c7b

* Modified tests with DownloadPath variable

Change-Id: Iaf14c9ea1a8242b6c8d8e9e4fac8c23d9c1b3a74

* Added testcase to validate file content

Change-Id: I21aed481b433450c3b536dbb29d45291f61848d8

* Refactored test for file content check to avoid failures

Change-Id: I3b4fe9a0de678f437fd4cc0a8203ae9434d9fa8e

* Removed auto-generated comments

Change-Id: I86c4ac3e7e4476a75d6cbed58826ec1f3278d7d2

* Fixed documentation review comments

Change-Id: I4faf31473b53fc53a5517d418c343bf7320eec55

* Fixed documentation indentation

Change-Id: I386f046cf4e10ee6deb5a81fcfc8c430c97086c8

* Fix build issue

Change-Id: I61a829cabaf03ffd5e77cddc594486a650118fa3
2022-03-09 13:07:23 +01:00
Pavel Busko
c086f84668
fix(cnbBuild): filter overall buildpacks key in the telemetry data (#3619)
Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
2022-03-09 11:48:47 +01:00
Pavel Busko
d2ef57bf5f
feat(kubernetesDeploy): support for image digests (#3613)
Co-authored-by: Sumit Kulhadia <sumit.kulhadia@sap.com>
2022-03-09 09:48:19 +01:00
Pavel Busko
e2de22f5ea
feat(kubernetesDeploy): use go templating for the deployTool: kubectl (#3600)
Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
2022-03-08 14:52:43 +01:00
ffeldmann
3e89fb0cd7
fixes retry when sending telemetry to Splunk (#3615)
* Do not retry sending data to Splunk HEC
* Adds error handling
2022-03-08 13:26:00 +01:00
Marc Vornetran
b0b6be8d29
docs: Add underscore suffix to custom vault prefix (#3488)
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-03-08 12:50:51 +01:00
Siarhei Pazdniakou
f5be111078
Fixed deriving stage name from the orchestrator via env var (#3219)
* Fixed deriving stage name from the orchestrator via env var

* make the stageName the leading parameter

* Added tests

* Update cmd/checkIfStepActive.go

* Update cmd/checkIfStepActive_test.go

* Update cmd/checkIfStepActive_test.go

* Update cmd/checkIfStepActive.go

Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-03-08 12:12:56 +01:00
Oliver Nocon
f9ad6dc048
fix(whitesourceExecuteScan): orgToken from Vault (#3614)
supersedes #3168
2022-03-08 08:59:12 +01:00
Oliver Nocon
0998a80f07
chore: update to go 1.17 (#3557)
* chore: update to go 1.17

Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>

* update workflows

Co-authored-by: Pavel Busko <pavel.busko@sap.com>

* fixed multipart upload tests

Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>

* go fmt

Co-authored-by: Pavel Busko <pavel.busko@sap.com>

* Update to go 1.17

Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>

* Update vault to v1.9.3

Co-authored-by: Pavel Busko <pavel.busko@sap.com>

Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
Co-authored-by: Pavel Busko <pavel.busko@sap.com>
Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
Co-authored-by: ffeldmann <f.feldmann@sap.com>
2022-03-07 17:15:36 +01:00