Oliver Feldmann
4208bc050c
Add bearer token retrieval function ( #3595 )
...
* Add bearer token retrieval function
Retrieving a bearer token from the xsuaa service on BTP is always the
same. With these functions one can retrieve a bearer token and set it
to the given header as 'Authorization'.
* CodeClimate fixes
* Refactor test
* Add basic auth to token retrieve request
Co-authored-by: Thorsten Duda <thorsten.duda@sap.com>
2022-03-16 11:22:48 +01:00
Oliver Nocon
7ec512cb9f
feat(artifactPrepareVersion): helm & propagate version ( #3627 )
...
* feat(artifactPrepareVersion): helm & propagate version
* chore: small refactoring
* chore: fix linting issue
* fix version persistence
2022-03-15 09:08:24 +01:00
xgoffin
dfd2278639
feat(fortifyExecuteScan): full FPR to SARIF implementation ( #3604 )
...
* feat(FPRtoSARIF): boilerplate & comments
* Feat(Ingest): Build done, Vulnerabilities partway
* feat(Vulnerabilities): now entirely parsed
* feat(FprToSarif): integration in Piper step, full xml structure
* feat(fpr_to_sarif): base program. Need to replace names in messages
* feat(fpr_to_sarif): message substitution and custom definition integration
* fix(fpr_to_sarif): missing replacement in tools object
* fix(fpr_to_sarif): failing unit test
* Fix fortify folder creation for generating sarif
* deletion of unzip folder
* feat(fpr_to_sarif): better unit test
* fix(fpr_to_sarif): pr tests failing
* feat(fortifyExecuteScan): complete SARIF file generation
* fix(fpr_to_sarif): add extra check and test to prevent panics
* rebase onto master, fix ALL conflicts, adapt code and format
* fix missing added properties
* fix(SARIF): structure
* fix(whitesource): wrong sarif structures
* Update pkg/fortify/fpr_to_sarif.go
* Update pkg/format/sarif.go
* Update pkg/format/sarif.go
Co-authored-by: Sumeet PATIL <sumeet.patil@sap.com>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2022-03-14 11:26:05 +01:00
ffeldmann
b224f2294c
Activates debug information for environment variables ( #3630 )
...
* Activates debug information for environment variables
* Adds tests for environment variable reading
* Reduces batch size to send messages to Splunk to 5000
2022-03-14 10:17:55 +01:00
Oliver Nocon
2a56723d40
feat(kanikoExecute): Trim names for multi-image builds ( #3617 )
...
* feat(kanikoExecute): Trim names for multi-image builds
* chore: fix yaml linting issue
2022-03-11 09:47:44 +01:00
Oliver Nocon
2c837927d4
chore(whitesourceExecuteScan): Gradle config changes ( #3621 )
...
* chore(whitesourceExecuteScan): Gradle config changes
supersedes #3293
closes #3293
* update config
Co-authored-by: ffeldmann <f.feldmann@sap.com>
2022-03-11 08:18:21 +01:00
Anil Keshav
157f6d4e35
feat (checkIfStepActive) enhance to include not active conditions ( #3616 )
...
* including negative conditions
* clean up and todos
* removing debug logging
* clean up
* fix unit test name
* fixing unit tests
* negative stage test
Co-authored-by: anilkeshav27 <you@example.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-03-10 13:35:28 +01:00
Christian Volk
b9c0aa008f
feat(piperExecuteBin): support running images from private docker registries ( #3622 )
2022-03-09 17:35:57 +01:00
Johannes Dillmann
3708f274cc
feat(cnbBuild) read target image name from github cpe ( #3620 )
...
Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
2022-03-09 14:06:26 +01:00
maheshsrikrishnan
64a00c540a
New piper step APIProviderDownload contribution ( #3349 )
...
* added store file function in cpi common utils
Change-Id: Ia429a2792266e082d139025a71799d21c30a7df9
* Added api provider steps
Change-Id: Icd2829a91db4c53d0de2330822d2b33933973868
* Update apiProviderDownload.yaml
* fixed yaml JLINT issue
Change-Id: Iac974abc30fa00e68c0177072b93716b0af5e0c5
* Removed trailing spaces
Change-Id: I927e9314fce6e9cab68d6b97577c7c96bb2bddad
* Resolved common steps groovy script conflict
Change-Id: I3ad144b618e1c77953aaeccaa5bf7309aff77ca9
* Change for conflict resolution
Change-Id: Ic955833eca844f090b7983f99f9d3649ebb981c7
* Fixed method name and its corresponding implementation
Change-Id: I465c1f1d5306bb978386de9efca3c521e385b89c
* Moved re-usable function to commonUtils package
Change-Id: Ide06462b01caeb2bf438ad7661e01c15bf8e8e24
* Changed the implementation to use existing writeFile method
* Fixed review comments on documentation and test structuring
Change-Id: Ifebd2f4b50754b2097b2d564fb3cc37c433ef6c9
* Fixed documentation alignment issues
* Fixed spaces issue
Change-Id: I834bd94e01bce72e7f81ab49ba32671c91c66ca9
* Documentation removed extra spaces
Change-Id: I9a639d76ed9b81c870f18349504044bb70753b52
* Fixed doc build issue
Change-Id: I96c3e15e73834b64f8b8e3432ce59f6b037f93fd
* Fixed documentation build issues
Change-Id: I7fca2ba69bc7b7298ee300ccd1ae16a6238dc96b
* Re-generated code for fixing build failure
Change-Id: I22b7ee6162f643d9f3b60f6a33eb7858927182a0
* Adopted file utils & mock
Change-Id: Ic46462003527f41df64395a5a615c19bf374e8ef
* Removed ioutil call in the test & adopted error variable names
* Removed commented lines
Change-Id: I99a12e39bc04323e9c19f1409d97eeca267e6fdb
* Added test for asserting file download and adopted error variables
Change-Id: I49463a3b75987bf68f5261d45602d2d7bd960a05
* Added download path assertion positive & negative case
Change-Id: Ieee461c3973b9dfa8f395dc936e4241ff9694c7b
* Modified tests with DownloadPath variable
Change-Id: Iaf14c9ea1a8242b6c8d8e9e4fac8c23d9c1b3a74
* Added testcase to validate file content
Change-Id: I21aed481b433450c3b536dbb29d45291f61848d8
* Refactored test for file content check to avoid failures
Change-Id: I3b4fe9a0de678f437fd4cc0a8203ae9434d9fa8e
* Removed auto-generated comments
Change-Id: I86c4ac3e7e4476a75d6cbed58826ec1f3278d7d2
* Fixed documentation review comments
Change-Id: I4faf31473b53fc53a5517d418c343bf7320eec55
* Fixed documentation indentation
Change-Id: I386f046cf4e10ee6deb5a81fcfc8c430c97086c8
* Fix build issue
Change-Id: I61a829cabaf03ffd5e77cddc594486a650118fa3
2022-03-09 13:07:23 +01:00
ffeldmann
3e89fb0cd7
fixes retry when sending telemetry to Splunk ( #3615 )
...
* Do not retry sending data to Splunk HEC
* Adds error handling
2022-03-08 13:26:00 +01:00
Oliver Nocon
0998a80f07
chore: update to go 1.17 ( #3557 )
...
* chore: update to go 1.17
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
* update workflows
Co-authored-by: Pavel Busko <pavel.busko@sap.com>
* fixed multipart upload tests
Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
* go fmt
Co-authored-by: Pavel Busko <pavel.busko@sap.com>
* Update to go 1.17
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
* Update vault to v1.9.3
Co-authored-by: Pavel Busko <pavel.busko@sap.com>
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
Co-authored-by: Pavel Busko <pavel.busko@sap.com>
Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
Co-authored-by: ffeldmann <f.feldmann@sap.com>
2022-03-07 17:15:36 +01:00
Mayur Belur Mohan
9a3b800b9d
ApiKeyValueMapUpload Command ( #3407 )
...
* ApiKeyValueMapUpload Command
* include command in metadata
* TestCase Fixes
* CodeReview Fixes
* CodeReview Fixes
* Code Review Fixes
* CodeReview Fixes
* CodeReview Fixes
* CodeReview FIxes
* CodeReview Fixes
* Documenation change
* documentation fix
* Documentation Fix
* Documentation Fix
* documentation Fix
* CodeReview Fixes
* CodeReview Fixes
* Revert changes
* Documentation Fix
* CodeReview FIxes
* Doc Fixes
* Code Review Fixes
* Code Review Fixes
* CodeReview FIxes
* Documentation Fix
* Documentation Changes
* Documentation Fix
* codereview fix
* Documentation Fix
* CodeReview Fixes
* CodeReview Fix
* Documentation FIx
* doc fix
* Doc Fix
* Documentation Fix
* codereview fix
* revert fix
* Code Review Fix
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
2022-03-07 11:03:44 +01:00
ffeldmann
6398e61995
Splunk reporting; Sending messages in batches ( #3611 )
...
* Refactors logfile sending logic, renaming of fields, adds proper piper sourcetype
* Sets maximum retries to three and transport timeout to 10 seconds for azure and jenkins
2022-03-07 09:55:12 +01:00
Christian Volk
5821a311cc
chore(npmExecuteScripts): config@2 support ( #3607 )
2022-03-04 10:26:46 +01:00
Pavel Busko
0de06c6207
feat(cnbBuild): write image digests to the CPE ( #3602 )
...
Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
2022-03-02 16:26:45 +01:00
Christian Volk
e49820f5e4
chore(npmExecuteScripts): unit tests ( #3597 )
2022-03-02 14:06:51 +01:00
Adrien
a73951909b
checkmarxExecuteScan fixes ( #3540 )
...
* Fix FilterByTeamName and LoadExistingProject
* Fix project name loop
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2022-02-28 14:22:47 +01:00
Eugene Kortelyov
8ced7f8184
Feature/fortify execute scan gradle ( #3582 )
...
* initial fortify gradle commit
* initial fortify gradle commit
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-02-28 11:35:38 +01:00
ffeldmann
5f4cd838cf
Updates telemetry logging information for internal reporting ( #3585 )
...
* Add StepStartTime, Renames StepDuration, adds PiperCommithash, removes Branch, GitOwner, GitRepository from logged telemetry information
* Fixes test case for telemetry logging
* Activates step monitoring data in debug mode
* Pretty debug json printing
* Reduces log noise, setting warning to debug
2022-02-28 09:45:57 +01:00
Eugene Kortelyov
af7496d012
feat: url log parsing / reporting when executing a command ( #3461 )
...
* Initial PR for url reporting
* Rename URLReport var to URLReportFileName
* Remove URLReportFileName from piper flags
* Update pkg/command/command.go
* Update pkg/command/command.go
* Update pkg/command/command.go
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-02-25 15:35:44 +01:00
thtrinh
d86cfce6e6
Checkmarx json report ( #3565 )
...
* feat(checkmarx) : Checkmarx JSON Report
* Test cases with some fix
* Information total and audited test assertions
* feat(checkmarx): align total/audited with existing calculation
* fix(checkmarx): Reporting unit test
Co-authored-by: Sumeet PATIL <sumeet.patil@sap.com>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-02-25 14:20:36 +01:00
Ralf Pannemans
08bfe1554e
feat(cnbBuild) Introducing preserveFiles config to copy back files ( #3562 )
...
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
2022-02-23 18:54:59 +01:00
Philipp Stehle
4c18f2a128
feat: Add support for deprecating step parameters ( #3554 )
...
this was already used in fortifyExecuteScan, but had no effect.
Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
2022-02-23 15:16:05 +01:00
Christian Volk
d428cfcbd1
feat(kanikoExecute): image digests ( #3566 )
...
* feat(init stage): artifact version
* feat(kaniko): expose image digests to cpe
* fix
* handle tmp folder creation differently
* fix
* fix
* fix tests
* set ignore-path to /
2022-02-23 11:41:26 +01:00
Sven Merk
a1988f6808
feat(whitesourceExecuteScan): GitHub issue creation + SARIF ( #3535 )
...
* Add GH issue creation + SARIF
* Code cleanup
* Fix fmt, add debug
* Code enhancements
* Fix
* Added debug info
* Rework UA log scan
* Fix code
* read UA version
* Fix nil reference
* Extraction
* Credentials
* Issue creation
* Error handling
* Fix issue creation
* query escape
* Query escape 2
* Revert
* Test avoid update
* HTTP client
* Add support for custom TLS certs
* Fix code
* Fix code 2
* Fix code 3
* Disable cert check
* Fix auth
* Remove implicit trust
* Skip verification
* Fix
* Fix client
* Fix HTTP auth
* Fix trusted certs
* Trim version
* Code
* Add token
* Added token handling to client
* Fix token
* Cleanup
* Fix token
* Token rework
* Fix code
* Kick out oauth client
* Kick out oauth client
* Transport wrapping
* Token
* Simplification
* Refactor
* Variation
* Check
* Fix
* Debug
* Switch client
* Variation
* Debug
* Switch to cert check
* Add debug
* Parse self
* Cleanup
* Update resources/metadata/whitesourceExecuteScan.yaml
* Add debug
* Expose subjects
* Patch
* Debug
* Debug2
* Debug3
* Fix logging response body
* Cleanup
* Cleanup
* Fix request body logging
* Cleanup import
* Fix import cycle
* Cleanup
* Fix fmt
* Fix NopCloser reference
* Regenerate
* Reintroduce
* Fix test
* Fix tests
* Correction
* Fix error
* Code fix
* Fix tests
* Add tests
* Fix code climate issues
* Code climate
* Code climate again
* Code climate again
* Fix fmt
* Fix fmt 2
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-02-23 09:30:19 +01:00
ffeldmann
ffd4a7efb8
(fix) nil pointer dereference in case credentials are not available ( #3564 )
...
* adds return in gcs upload in case error occurs e.g. no credentials, avoid nil pointer dereference
* Adds generated files
* Updates generated files
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-02-22 18:32:09 +01:00
Oliver Nocon
6247c5dddd
feat(buildsettings): golang and future tools ( #3561 )
...
Co-authored-by: Christian Volk <christian.volk@sap.com>
2022-02-21 09:16:48 +01:00
Christian Volk
b2246a021f
chore: noise reduction ( #3558 )
...
* fix(telemitry): noise reduction
* partially revert
2022-02-18 11:54:39 +01:00
ffeldmann
d7a13aa5d0
Fixes telemetry logging not reporting an error, adds tests ( #3551 )
...
* fixes error logging in case fatalError is empty, adds inital test cases for logStepTelemetryData
* Tests logging output
2022-02-17 15:39:01 +01:00
Oliver Nocon
a4a0873081
feat(checkmarx): create GitHub issue with findings ( #3543 )
...
* feat(checkmarx): create GitHub issue with findings
* add github issue reporting
2022-02-17 15:16:55 +01:00
Christian Volk
a1cd6cdb3a
fix(kaniko): set cpe consistently ( #3545 )
2022-02-17 08:20:30 +01:00
Ralf Pannemans
e7db09db12
feat(cnbBuild): added dockerimage to the telemetry data ( #3501 )
...
Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>
Co-authored-by: Sumit Kulhadia <sumit.kulhadia@sap.com>
Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
2022-02-16 13:28:51 +01:00
Ralf Pannemans
4b2f61589d
feat(cnbbuild) enable multi image build ( #3521 )
...
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
Co-authored-by: Pavel Busko <pavel.busko@sap.com>
Co-authored-by: Sumit Kulhadia <sumit.kulhadia@sap.com>
2022-02-15 14:39:14 +01:00
ffeldmann
19a05a9c70
Reporting: Variable change to deprecate step Splunk Hook; logging in telemetry ( #3539 )
...
* Adds fatalError variable with helper setter and getter helper functions
* Adds logging function to telemetry package (always executed)
2022-02-14 08:56:11 +01:00
Daniel Mieg
f08ff92171
New log entities for Pull & Clone ( #3517 )
...
* WIP
* New Logs
* Improving
* Determine log output based on available entities
* Increase width
* Add line
* Adapt TestPollEntity
* Format
* Fix query
* Adapt tests
* Fix test
* Improve formatting
* Retern early in case of no logs
* Remove duplicate log
2022-02-11 10:16:40 +01:00
Christian Volk
c888e21e6c
chore(multiarch): helper to parse targetArchitectures ( #3525 )
...
* chore(docker): helper to parse targetArchitectures
* missing files
2022-02-10 16:46:00 +01:00
Vitalii Sidorov
93e3801945
feat: add step for Helm execution ( #3419 )
...
* Implement helm step
* Create kubernetes package
* Refactoring helm.go
* Add package, test commands
* Add test for helm package
* Add tests for helm.go
* Add tests for helm.go
* Add tests for utils.go
* Add tests for helmExecute.go
* small fix
* Add helm lint
* small fix
* small fix
* Fix according to comments
* Fix test
* small fix
* Add helm add function
* Changes according to new comments
* Add helm push
* Add unit tests
* Add tests for helmExecute
* Add small fix
* small fix
* small fix
* Move DeployUtilsBundle from kubernetesDeploy to kubernetes package
* small fix
* small fix
* Add unit-tests
* Fix
* Update resources/metadata/helmExecute.yaml
* Update resources/metadata/helmExecute.yaml
* Add helm chart server parameterization
* small fix
* small fix
Co-authored-by: “Vitalii <“vitalii.sidorov@sap.com”>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-02-10 10:25:03 +01:00
Christian Volk
5aa9661db6
feat(kanikoExecute): preparation for multi-arch builds ( #3515 )
...
* feat(kanikoExecute): preparation for multi-arch builds
* missing files
2022-02-09 10:17:34 +01:00
xgoffin
2cebf370c9
feat(fortifyExecuteScan): added conversion to SARIF for FPR files ( #3485 )
...
* feat(FPRtoSARIF): boilerplate & comments
* Feat(Ingest): Build done, Vulnerabilities partway
* feat(Vulnerabilities): now entirely parsed
* feat(Ingestion): handle Description object
* feat(FprToSarif): integration in Piper step, full xml structure
* feat(fpr_to_sarif): base program. Need to replace names in messages
* feat(fpr_to_sarif): message substitution and custom definition integration
* fix(fpr_to_sarif): missing replacement in tools object
* fix(fortifyExecuteScan): unit tests
* fix(fpr_to_sarif): failing unit test
* Fix fortify folder creation for generating sarif
* deletion of unzip folder
* fix(fortifyExecuteScan): change logging to info
* feat(fpr_to_sarif): better unit test
* fix(fpr_to_sarif): pr tests failing
* feat(fpr_to_sarif): add specific properties to sarif
* feat(fpr_to_sarif): severity integration
* fix(fpr_to_sarif): unit test fixed
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Sumeet PATIL <sumeet.patil@sap.com>
2022-02-08 14:10:40 +01:00
Giridhar Shenoy
286ff1b6ef
detectExecuteScan : support for detect 7 ( #3453 )
...
* enable detect 7 script
* unit test uses detect7 as default
* add detect6 test case
* add check for OSEnv detect version
* add test for OSEnv detect version
* update customEnvironmentVariables docu
* fix linting
Co-authored-by: ffeldmann <f.feldmann@sap.com>
2022-02-08 11:55:01 +01:00
rosemarieB
d1eb87363f
Generic step input from addon descriptor ( #3467 )
...
* enhance build step to run based on addon.yml in steampunk environment
Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com>
2022-02-07 20:34:11 +01:00
Matthias Scudlik
907dcd7dc1
fix(sonarExecuteScan): error handling for report metrics ( #3512 )
2022-02-07 16:41:36 +01:00
ManjunathMS35
8108bb8f6f
Update npmExecuteScripts step ( #3211 )
...
* Update npmExecuteScripts step
* Fixed failing build
* Fixed path issue
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Christian Volk <christian.volk@sap.com>
2022-02-07 14:46:03 +01:00
ffeldmann
f3566ab7e0
Changes GetJobName value to repositoryname ( #3509 )
2022-02-07 08:52:12 +01:00
Oliver Nocon
2ae1d9dac1
feat(kanikoExecute): allow building multiple images ( #3443 )
...
* feat(kanikoExecute): allow building multiple images
* enhance tests
* chore: allow running tests in parallel
* small fixes
* fix: fix destination bug
* update formatting and defaults
* fix yml formatting
* chore: change cpe parameter names
* chore: improve variable naming
2022-02-07 07:58:41 +01:00
Christian Volk
54ca265860
fix(httpclient): folder permissions ( #3506 )
2022-02-04 14:40:18 +01:00
Matthias Scudlik
5eb996c43c
sonarqube coverage: additional metrics ( #3465 )
...
* sonarqube coverage: additional metrics
* sonarExecuteScan: add lines of code and language distribution to sonarscan.json
* sonarExecuteScan: consider branch in componentService requests
* SonarQube: Do not omit empty values in SonarCoverage
* sonarExecuteScan: Add integration tests for ComponentService getLinesOfCode
* fix tests
* sonarExecuteScan: use pullRequest in componentService
Co-authored-by: I550025 <r.kloe@sap.com>
Co-authored-by: Marc Bormeth <marc.bormeth@sap.com>
2022-02-04 09:52:29 +01:00
kulhadia
6feb1217aa
Feat(cnbBuild): collecting the telemetry data for cnbBuild ( #3368 )
...
Co-authored-by: Sumit Kulhadia <sumit.kulhadia@sap.com>
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>
Co-authored-by: Pavel Busko <pavel.busko@sap.com>
2022-01-31 12:27:29 +01:00
Christian Volk
880be73a4c
feat(golangBuild): support private modules ( #3471 )
...
* feat(golangBuild): support private module repositories
2022-01-31 10:45:40 +01:00
