1
0
mirror of https://github.com/SAP/jenkins-library.git synced 2024-12-16 11:09:33 +02:00
Commit Graph

578 Commits

Author SHA1 Message Date
Christian Volk
d428cfcbd1
feat(kanikoExecute): image digests (#3566)
* feat(init stage): artifact version

* feat(kaniko): expose image digests to cpe

* fix

* handle tmp folder creation differently

* fix

* fix

* fix tests

* set ignore-path to /
2022-02-23 11:41:26 +01:00
Sven Merk
a1988f6808
feat(whitesourceExecuteScan): GitHub issue creation + SARIF (#3535)
* Add GH issue creation + SARIF

* Code cleanup

* Fix fmt, add debug

* Code enhancements

* Fix

* Added debug info

* Rework UA log scan

* Fix code

* read UA version

* Fix nil reference

* Extraction

* Credentials

* Issue creation

* Error handling

* Fix issue creation

* query escape

* Query escape 2

* Revert

* Test avoid update

* HTTP client

* Add support for custom TLS certs

* Fix code

* Fix code 2

* Fix code 3

* Disable cert check

* Fix auth

* Remove implicit trust

* Skip verification

* Fix

* Fix client

* Fix HTTP auth

* Fix trusted certs

* Trim version

* Code

* Add token

* Added token handling to client

* Fix token

* Cleanup

* Fix token

* Token rework

* Fix code

* Kick out oauth client

* Kick out oauth client

* Transport wrapping

* Token

* Simplification

* Refactor

* Variation

* Check

* Fix

* Debug

* Switch client

* Variation

* Debug

* Switch to cert check

* Add debug

* Parse self

* Cleanup

* Update resources/metadata/whitesourceExecuteScan.yaml

* Add debug

* Expose subjects

* Patch

* Debug

* Debug2

* Debug3

* Fix logging response body

* Cleanup

* Cleanup

* Fix request body logging

* Cleanup import

* Fix import cycle

* Cleanup

* Fix fmt

* Fix NopCloser reference

* Regenerate

* Reintroduce

* Fix test

* Fix tests

* Correction

* Fix error

* Code fix

* Fix tests

* Add tests

* Fix code climate issues

* Code climate

* Code climate again

* Code climate again

* Fix fmt

* Fix fmt 2

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-02-23 09:30:19 +01:00
ffeldmann
ffd4a7efb8
(fix) nil pointer dereference in case credentials are not available (#3564)
* adds return in gcs upload in case error occurs e.g. no credentials, avoid nil pointer dereference

* Adds generated files

* Updates generated files

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-02-22 18:32:09 +01:00
Oliver Nocon
6247c5dddd
feat(buildsettings): golang and future tools (#3561)
Co-authored-by: Christian Volk <christian.volk@sap.com>
2022-02-21 09:16:48 +01:00
Christian Volk
b2246a021f
chore: noise reduction (#3558)
* fix(telemitry): noise reduction

* partially revert
2022-02-18 11:54:39 +01:00
ffeldmann
d7a13aa5d0
Fixes telemetry logging not reporting an error, adds tests (#3551)
* fixes error logging in case fatalError is empty, adds inital test cases for logStepTelemetryData

* Tests logging output
2022-02-17 15:39:01 +01:00
Oliver Nocon
a4a0873081
feat(checkmarx): create GitHub issue with findings (#3543)
* feat(checkmarx): create GitHub issue with findings

* add github issue reporting
2022-02-17 15:16:55 +01:00
Christian Volk
a1cd6cdb3a
fix(kaniko): set cpe consistently (#3545) 2022-02-17 08:20:30 +01:00
Ralf Pannemans
e7db09db12
feat(cnbBuild): added dockerimage to the telemetry data (#3501)
Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>
Co-authored-by: Sumit Kulhadia <sumit.kulhadia@sap.com>
Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
2022-02-16 13:28:51 +01:00
Ralf Pannemans
4b2f61589d
feat(cnbbuild) enable multi image build (#3521)
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
Co-authored-by: Pavel Busko <pavel.busko@sap.com>
Co-authored-by: Sumit Kulhadia <sumit.kulhadia@sap.com>
2022-02-15 14:39:14 +01:00
ffeldmann
19a05a9c70
Reporting: Variable change to deprecate step Splunk Hook; logging in telemetry (#3539)
* Adds fatalError variable with helper setter and getter helper functions
* Adds logging function to telemetry package (always executed)
2022-02-14 08:56:11 +01:00
Daniel Mieg
f08ff92171
New log entities for Pull & Clone (#3517)
* WIP

* New Logs

* Improving

* Determine log output based on available entities

* Increase width

* Add line

* Adapt TestPollEntity

* Format

* Fix query

* Adapt tests

* Fix test

* Improve formatting

* Retern early in case of no logs

* Remove duplicate log
2022-02-11 10:16:40 +01:00
Christian Volk
c888e21e6c
chore(multiarch): helper to parse targetArchitectures (#3525)
* chore(docker): helper to parse targetArchitectures

* missing files
2022-02-10 16:46:00 +01:00
Vitalii Sidorov
93e3801945
feat: add step for Helm execution (#3419)
* Implement helm step

* Create kubernetes package

* Refactoring helm.go

* Add package, test commands

* Add test for helm package

* Add tests for helm.go

* Add tests for helm.go

* Add tests for utils.go

* Add tests for helmExecute.go

* small fix

* Add helm lint

* small fix

* small fix

* Fix according to comments

* Fix test

* small fix

* Add helm add function

* Changes according to new comments

* Add helm push

* Add unit tests

* Add tests for helmExecute

* Add small fix

* small fix

* small fix

* Move DeployUtilsBundle from kubernetesDeploy to kubernetes package

* small fix

* small fix

* Add unit-tests

* Fix

* Update resources/metadata/helmExecute.yaml

* Update resources/metadata/helmExecute.yaml

* Add helm chart server parameterization

* small fix

* small fix

Co-authored-by: “Vitalii <“vitalii.sidorov@sap.com”>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-02-10 10:25:03 +01:00
Christian Volk
5aa9661db6
feat(kanikoExecute): preparation for multi-arch builds (#3515)
* feat(kanikoExecute): preparation for multi-arch builds

* missing files
2022-02-09 10:17:34 +01:00
xgoffin
2cebf370c9
feat(fortifyExecuteScan): added conversion to SARIF for FPR files (#3485)
* feat(FPRtoSARIF): boilerplate & comments

* Feat(Ingest): Build done, Vulnerabilities partway

* feat(Vulnerabilities): now entirely parsed

* feat(Ingestion): handle Description object

* feat(FprToSarif): integration in Piper step, full xml structure

* feat(fpr_to_sarif): base program. Need to replace names in messages

* feat(fpr_to_sarif): message substitution and custom definition integration

* fix(fpr_to_sarif): missing replacement in tools object

* fix(fortifyExecuteScan): unit tests

* fix(fpr_to_sarif): failing unit test

* Fix fortify folder creation for generating sarif

* deletion of unzip folder

* fix(fortifyExecuteScan): change logging to info

* feat(fpr_to_sarif): better unit test

* fix(fpr_to_sarif): pr tests failing

* feat(fpr_to_sarif): add specific properties to sarif

* feat(fpr_to_sarif): severity integration

* fix(fpr_to_sarif): unit test fixed

Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Sumeet PATIL <sumeet.patil@sap.com>
2022-02-08 14:10:40 +01:00
Giridhar Shenoy
286ff1b6ef
detectExecuteScan : support for detect 7 (#3453)
* enable detect 7 script

* unit test uses detect7 as default

* add detect6 test case

* add check for OSEnv detect version

* add test for OSEnv detect version

* update customEnvironmentVariables docu

* fix linting

Co-authored-by: ffeldmann <f.feldmann@sap.com>
2022-02-08 11:55:01 +01:00
rosemarieB
d1eb87363f
Generic step input from addon descriptor (#3467)
* enhance build step to run based on addon.yml in steampunk environment

Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com>
2022-02-07 20:34:11 +01:00
Matthias Scudlik
907dcd7dc1
fix(sonarExecuteScan): error handling for report metrics (#3512) 2022-02-07 16:41:36 +01:00
ManjunathMS35
8108bb8f6f
Update npmExecuteScripts step (#3211)
* Update npmExecuteScripts step

* Fixed failing build

* Fixed path issue

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Christian Volk <christian.volk@sap.com>
2022-02-07 14:46:03 +01:00
ffeldmann
f3566ab7e0
Changes GetJobName value to repositoryname (#3509) 2022-02-07 08:52:12 +01:00
Oliver Nocon
2ae1d9dac1
feat(kanikoExecute): allow building multiple images (#3443)
* feat(kanikoExecute): allow building multiple images

* enhance tests

* chore: allow running tests in parallel

* small fixes

* fix: fix destination bug

* update formatting and defaults

* fix yml formatting

* chore: change cpe parameter names

* chore: improve variable naming
2022-02-07 07:58:41 +01:00
Christian Volk
54ca265860
fix(httpclient): folder permissions (#3506) 2022-02-04 14:40:18 +01:00
Matthias Scudlik
5eb996c43c
sonarqube coverage: additional metrics (#3465)
* sonarqube coverage: additional metrics

* sonarExecuteScan: add lines of code and language distribution to sonarscan.json

* sonarExecuteScan: consider branch in componentService requests

* SonarQube: Do not omit empty values in SonarCoverage

* sonarExecuteScan: Add integration tests for ComponentService getLinesOfCode

* fix tests

* sonarExecuteScan: use pullRequest in componentService

Co-authored-by: I550025 <r.kloe@sap.com>
Co-authored-by: Marc Bormeth <marc.bormeth@sap.com>
2022-02-04 09:52:29 +01:00
kulhadia
6feb1217aa
Feat(cnbBuild): collecting the telemetry data for cnbBuild (#3368)
Co-authored-by: Sumit Kulhadia <sumit.kulhadia@sap.com>
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>
Co-authored-by: Pavel Busko <pavel.busko@sap.com>
2022-01-31 12:27:29 +01:00
Christian Volk
880be73a4c
feat(golangBuild): support private modules (#3471)
* feat(golangBuild): support private module repositories
2022-01-31 10:45:40 +01:00
Anil Keshav
1a96d7a67e
fix (npmExecuteScripts) allow npm pack before npm publish (#3455)
* adding config to piperNpmr

* scope in cli

* adding scope to repo url and npmrc

* publish to scoped

* removing scope

* changing scope position

* adding scope to userconfig

* adding registry=

* pack and then tar

* not removing tmp folder

* adding flag

* pack before publish

* adding log

* debug

* debug with change directory

* publishing created tar ball

* debug

* üath

* adding main npmrc

* renaming old npmrc file

* error renaming old npmrc file

* renaming err

* correcting npmrc file path

* renaming file back to original

* current working directory

* renaming the npmrc file

* avoiding change directory

* with current working dir

* adding dot

* renaming npmrc and defer removal

* rename files

* Update pkg/npm/publish.go

* Update pkg/npm/publish.go

Co-authored-by: anilkeshav27 <you@example.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-01-25 09:52:22 +01:00
Eugene Kortelyov
da450c5243
feat(gradleExecuteBuild): add stage scope to path parameter, fix project dir exist issue (#3401)
* add stage scope to path parameter, fix project dir exist issue

* fix unit test for gradleExecuteBuild

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-01-24 15:08:39 +01:00
Christopher Fenner
ebcbb73b46
feat: add command to fetch default files (#3445)
* some ideas..

* Add getDefaults command (WIP) (#3444)

* add getYAML function for configs

* create getDefaults command(based on getConfig)

* add getDefaults command to CLI

* read defaults files, using github tokens as well

* write defaults to stdout as JSON object with YAMLs embedded

* catch case where no input files are given

* actually write output to file if outputFile is specified

* mark defaultsFile flag as required

* add basic tests

* add output (string)  test

* adapt generateDefaults() to return output (used for test of previous commit)

* Changes to getDefaults() JSON output (#3449)

* change JSON output to contain separate fields

* filename -> filepath

* Apply suggestions from code review

* Apply suggestions from code review

* Update pkg/config/config_test.go

Co-authored-by: Jordi van Liempt <35920075+jliempt@users.noreply.github.com>
2022-01-24 13:55:28 +01:00
Christopher Fenner
8e0209db74
feat(GitHub): fetch commit statistics (#3381)
* fetch GH statistics

* move GH and Sonar integration tests to own files

* fix imports

* add integration test case

* add result type

* Apply suggestions from code review
2022-01-24 13:43:07 +01:00
Christian Volk
b0e4599d4d
feat(malwareExecuteScan): refactoring and docker support (#3421)
* feat(malwareExecuteScan): add support for scanning docker images

* refactoring

* print out finding if available

* generate toolrecord for malware scan

* persist scan report

* docs

* fix

* fix

* rollback cmd/init_unix.go

* auhenticated pull

* fix

* fix: report shall be consistent with the api model

* gcs upload

* fix linter
2022-01-24 09:48:01 +01:00
Sven Merk
6520115950
Upload Fortify scan results to GitHub issue (#3300)
* fix(fortifyExecuteScan): Propagate translation errors

Force translation related errors to stop the execution of the step.

* Extend testcase

* Update fortifyExecuteScan.go

* Fix fmt and test

* Fix code

* feat(fortifyExecuteScan): Create GitHub issue

* Fix expectation

* Fix fmt

* Fix fmt add test

* Added tests

* Go fmt

* Add switch

* Rewrite githubCreateIssue

* Fix tests

* Added switch

* Issue only in case of violations

* Fix CPE reference

* Add  debug message to issue creation/update

* Update fortifyExecuteScan.go

* Add credential for GH to groovy wrapper

* Update fortifyExecuteScan.go
2022-01-21 10:52:17 +01:00
Raghunath Deshpande
af4bc6e989
Update maxWaitTime timeout for WhiteSource (#3284)
* Update scanPolling.go

Changing maxWaitTime from 15 to 30 to overcome WhiteSource results reflection in the backend issue.

* Update configHelper.go

* Reset configHelper changes to fix PR 3284

 Committer: raghunathd8

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: raghunathd8 <root@docker-evaluation.openstack.eu-nl-1.cloud.sap>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2022-01-20 12:14:00 +01:00
Siarhei Pazdniakou
c93f1f861d
GCS Upload fixes (#3387)
* Fixed argument type in persist function

* Fixed gcp upload to be usable in internal piper

* Fixed import of packages

* Updated logs

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-01-19 13:16:57 +01:00
Umidjon Urunov
c2ebdfd9ec
feat(protecode): add versioning model (#3373)
* changes to detectExec before master merge

* changes for detectExecuteScan

* self generated code added

* fix syntax errors and update docu

* added unit tests for fail and Group

* fix failOn bug

* add Groups as string array

* add Groups as string array

* tests and validation for groups, failOn

* Updated docs and added more tests

* documentation md files should not be changed

* Handle merge conflicts from PR 1845

* fix merge errors

* remove duplicate groups, merge error

* adding buildCode and buildTool as params

* switching build options

* building maven modules

* parameter correction

* parameter correction

* gnerate with new build parameter

* adding comments

* removing piper lib master and modifying goUtils to download 1.5.7 release

* first cleaning then installing

* multi module maven built

* multi module maven built removing unwanted code

* multi module maven built moving inside switch

* testing

* modifying the default use case to also call maven build

* modifying the default use case to also call maven build wih --

* corrected maven build command

* corrected maven build command with %v

* skipping test runs

* testing for MTA project with single pom

* adding absolute path to m2 path

* clean up

* adding switch for mta and maven and removing env from containers

* commiting changes for new detect step

* correting log message

* code clean up

* unit tests changes to detectExecute

* basic tests for new change

* restoring piperGoUtils to download correct piper binary

* code clean up

* code clean up

* protecodeExecuteScan :: versioning model draft - 1

* protecodeExecuteScan :: version model draft-2

* protecodeExecuteScan :: changing filename and version concatenation

* protecodeExecuteScan :: update documentation

* protecodeExecuteScan :: double URL encoding has been corrected & console messaging improved

* protecodeExecuteScan :: fixed Go/generate validation fail

* protecodeExecuteScan :: fixing failed unit tests

* protecodeExecuteScan :: Version field added

* protecodeExecuteScan :: Version field add => minor changes

* protecodeExecuteScan :: Version field add => fixing tests

Co-authored-by: D072410 <giridhar.shenoy@sap.com>
Co-authored-by: Keshav <anil.keshav@sap.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2022-01-19 10:30:59 +01:00
Nikolay Grechanov
17510f8996
fix: correct typo in log message (#3439)
Closes #3438
2022-01-19 08:41:11 +01:00
Ralf Pannemans
c97625e840
feat(cnbBuild) containerImageName will be defaulted if possible (#3437)
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
Co-authored-by: Sumit Kulhadia <sumit.kulhadia@sap.com>
2022-01-14 16:49:45 +01:00
Anil Keshav
04c4460c78
including vaultSecretFile to parameter generator docu (#3436)
Co-authored-by: anilkeshav27 <you@example.com>
2022-01-14 15:03:29 +01:00
Pavel Busko
1750b75cb8
feat(cnbBuild): preserve maven test results in the workspace (#3429)
Co-authored-by: Pavel Busko <pavel.busko@sap.com>
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
2022-01-14 11:05:11 +01:00
Anil Keshav
f3b65ae43b
feat (npmExecuteScripts) create seperate npmrc file for publish to private repo (#3422)
* creating new npm rc file

* publishing to registry staging

* exposing base64 version of env variables

* changing encoding param

* fixing unit test for the new path

* debugging env var

* remove debug message

* update docu

* changing new npmrc file name

* adding new npmrc to ignore

* adding new npmrc to ignore

Co-authored-by: anilkeshav27 <you@example.com>
2022-01-13 13:08:19 +01:00
Daniel Mieg
8634d8bb12
Add easy mode for AUnit & ATC (#3389)
* remove mandatory flag from config files

* Enable repo.yml as config

* Adapt to merge

* Refactoring

* Refactoring

* avoid panic

* Add comments

* Add easy mode for atc

* Add tests

* Add test

* Refactor

* Add test for MPS

* Updates

* Rename functions

* Add files to gitignore

* Rename

* Renaming

* Renaming

* Renaming

* Improve error messages

* Update documentation

* Add logging

* Rename

* Extend gitignore
2022-01-12 12:02:27 +01:00
Marc Bormeth
78c4a8fe93
Revert "feat: Add debug output to getConfig (#3341)" (#3396)
This reverts commit b760bf48e7.

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-01-11 10:47:03 +01:00
Pavel Busko
547ef5b55d
feat(cnbBuild): remove docker config after parsing (#3417)
Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>
2022-01-11 10:01:15 +01:00
Mayur Belur Mohan
f27cb4e482
ApiProxyUpload Command (#3295)
* ApiProxyUpload Command

* Code Review Fixes

* CodeReview Changes

* CodeReview Fixes

* YAML fixes

* CodeReview Fix

* Code Review Fixes

* CodeReview Fixes

* Code Climate Fixes

* Code Review Fixes

* Code Review Fixes

Co-authored-by: Roland Stengel <r.stengel@sap.com>
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
2021-12-28 09:31:50 +01:00
Pavel Busko
81fa0ee2d8
feat(cnbBuild): support for fetching binding content from url (#3388)
Co-authored-by: I546443 <sumit.kulhadia@sap.com>
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
2021-12-23 10:21:13 +01:00
Eugene Kortelyov
4d787ded6d
gradleExecuteBuild initial commit (#3337)
* gradleExecuteBuild initial commit

* fmt fixes for metadata_generated

* flow updates, some tests were added

* add gradle path alias, add stage scope to task parameter

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-12-22 15:34:36 +01:00
Christopher Fenner
51feacadbf
chore: align Vault naming (#3372)
* Align Vault naming

* remove duplicate debug  output

* correct message

* align vault naming

* Update vault.md
2021-12-21 11:52:10 +01:00
Anil Keshav
3a0e7e3e86
fetch general purpose credentials from vault (#3380)
Co-authored-by: anilkeshav27 <you@example.com>
2021-12-21 11:00:13 +01:00
Anil Keshav
c34b511fdf
correct directory permission (#3378)
Co-authored-by: Your Name <you@example.com>
2021-12-21 07:48:36 +01:00
Daniel Mieg
2bb6c21337
ABAP environment: add tag support (#3376) 2021-12-20 17:58:58 +01:00