1
0
mirror of https://github.com/SAP/jenkins-library.git synced 2024-12-14 11:03:09 +02:00
Commit Graph

67 Commits

Author SHA1 Message Date
Akramdzhon Azamov
f9617f5315
feat(blackduck): Npm extra parameters (#4327)
* added two new parameters for npm

---------

Co-authored-by: Andrei Kireev <andrei.kireev@sap.com>
2023-04-13 12:10:26 +02:00
Andrei Kireev
e55c2f857c
feat(detectExecuteScan) enabling possibility to scan MTA projects (#4300)
* feat(detectExecuteScan) enabling possibility to scan MTA projects
2023-03-27 10:42:39 +02:00
Andrei Kireev
22f61be2c4
Removed downloading of detect script version 6 (#4261)
Co-authored-by: ffeldmann <f.feldmann@sap.com>
2023-03-07 14:43:14 +01:00
Sven Merk
a055b905f9
Fix cumulus uploads for WhiteSource and BlackDuck (#4128)
* Refurbish upload to compliance store

* Fix BD IP json path
2022-11-17 11:05:27 +01:00
Matthias Kuhr
4d45724a26
fix(detectExecuteScan): Update links to Synopsys docs (#4038)
* Update links to Synopsys docs

Co-authored-by: sumeet patil <sumeet.patil@sap.com>
2022-11-15 12:45:11 +05:30
Giridhar Shenoy
d31c0584ff
detectExecuteScan : Add minScanInterval parameter through Piper (#4006)
* add minscaninterval parameter

* update detectExec

* removed a single trailing space which caused a lint failure

* Add test case

* Ensure unmap is false

* fix test case

* update format of param value
2022-09-26 14:08:12 +02:00
Sven Merk
b3f37650a2
SBOM creation for Mend (#3934)
* Fix docs and format

* Assessment format added

* Added sample file

* Added parsing

* Added packageurl implementation

* Slight refinement

* Refactored assessment options

* Adapted sample file

* First attempt of ws sbom gen

* Reworked SBOM generation

* Fix test code

* Add assessment handling

* Update dependencies

* Added golden test

* Small fix

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-08-09 13:56:01 +02:00
Oliver Nocon
b7c0831b7f
feat: allow OSVM scans to succeed with vulnerabilities (#3889)
For running open source vulnerability scans in de-coupled processes
it is helpful to allow that steps only create
compliance reports to inform users/teams
but not fail the pipeline.

This can now be achieved constitently with the flag:
`failOnSevereVulnerabilities`

Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2022-07-12 11:43:24 +02:00
Giridhar Shenoy
e6115a54b2
detectExecuteScan : Bug fix : Dont consider ignored components (#3867)
* fix project version limiting issue

* add tests for detectExecute

* fix bug with vuln count

* adjust unit tests

* update documentation for detect versions
2022-07-11 10:50:31 +02:00
Linda Siebert
acbcc5646b
[ANS] Change helper and re-generate steps (#3675)
* Add ans implementation

* Remove todo comment

* Rename test function

Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>

* Better wording

Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>

* Add reading of response body function

* Use http pkg ReadResponseBody

* Check read error

* Better test case description

* Fix formatting

* Create own package for read response body

* Omit empty nested resource struct

* Separate Resource struct from Event struct

* Merge and unmarshall instead of only unmarshalling

* Improve status code error message

* Remove unchangeable event fields

* Separate event parts

* Change log level setter function

* Restructure ans send test

* Revert exporting readResponseBody function

Instead the code is duplicated in the xsuaa and ans package

* Add check correct ans setup request

* Add set options function for mocking

* Review fixes

* Correct function name

* Use strict unmarshalling

* Validate event

* Move functions

* Add documentation comments

* improve test

* Validate event

* Add logrus hook for ans

* Set defaults on new hook creation

* Fix log level on error

* Don't alter entry log level

* Set severity fatal on 'fatal error' log message

* Ensure that log entries don't affect each other

* Remove unnecessary correlationID

* Use file path instead of event template string

* Improve warning messages

* Add empty log message check

* Allow configuration from file and string

* Add sourceEventId to tags

* Change resourceType to Pipeline

* Use structured config approach

* Use new log level set function

* Check correct setup and return error

* Mock http requests

* Only send log level warning or higher

* Use new function name

* One-liner ifs

* Improve test name

* Fix tests

* Prevent double firing

* Reduce Fire test size

* Add error message to test

* Reduce newANSHook test size

* Further check error

* Rename to defaultEvent in hook struct

* Reduce ifs further

* Fix set error category test

The ansHook Fire test cannot run in parallel, as it would affect the
other tests that use the error category.

* Change function name to SetServiceKey

* Validate event

* Rename to eventTemplate in hook struct

* Move copy to event.go

* Fix function mix

* Remove unnecessary cleanup

* Remove parallel test

The translation fails now and again when parallel is on.

* Remove prefix test

* Remove unused copyEvent function

* Fix ifs

* Add docu comment

* Register ans hook from pkg

* register hook and setup event template seperately

* Exclusively read eventTemplate from environment

* setupEventTemplate tests

* adjust hook levels test

* sync tests- wlill still fail

* migrate TestANSHook_registerANSHook test

* fixes

* Introduce necessary parameters

* Setup hook test

* Use file instead

* Adapt helper for ans

* Generate go files

* Add ans config to general config

* Change generator

* Regenerate steps

* Allow hook config from user config

Merges with hook config from defaults

* Remove ans flags from root command

* Get environment variables

* Generate files

* Add test when calling merge twice

* Update generator

* Regenerate steps

* Check two location for ans service key env var

* Re-generate

* Fix if

* Generate files with fix

* Duplicate config struct

* Add type casting test for ans config

* Fix helper

* Fix format

* Fix type casting of config

* Revert "Allow hook config from user config"

This reverts commit 4864499a4c497998c9ffc3e157ef491be955e68e.

* Revert "Add test when calling merge twice"

This reverts commit b82320fd07b82f5a597c5071049d918bcf62de00.

* Add ans config tests

* Improve helper code

* Re-generate commands

* Fix helper unit tests

* Change to only one argument

* Fix helper tests

* Re-generate

* Revert piper and config changes

* Re-generate missing step

* Generate new steps

* [ANS] Add servicekey credential to environment (#3684)

* Add ANS credential

* Switch to hooks and remove comments

* Add subsection for ans

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>

* Remove changes to piper.go

* Remove formatting

* Add test for ANS

* Define hook credential seperately from step credential

* Add test for retrieval from general section

* Add comment

* Get ans hook info from DefaultValueCache

* [ANS] Add documentation (#3704)

* Add ANS credential

* Switch to hooks and remove comments

* Add subsection for ans

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>

* Remove changes to piper.go

* Remove formatting

* Add test for ANS

* Define hook credential seperately from step credential

* Add test for retrieval from general section

* Add comment

* Add documentation

* Review changes

* Review comments

* Improve documentation further

* Add note of two event templates

* Add log level destinction

* Further improvements

* Improve text

* Remove unused things

* Add ANS credential

* Switch to hooks and remove comments

* Add subsection for ans

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>

* Remove changes to piper.go

* Remove formatting

* Add test for ANS

* Define hook credential seperately from step credential

* Add test for retrieval from general section

* Add comment

* Get ans hook info from DefaultValueCache

* Improvements

Co-authored-by: Linda Siebert <linda.siebert@sap.com>
Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>

* New lines

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
Co-authored-by: Roland Stengel <r.stengel@sap.com>
Co-authored-by: Thorsten Duda <thorsten.duda@sap.com>
2022-06-22 13:31:17 +02:00
Oliver Nocon
20c5f0a63b
fix(optimization) use proper cpe reference (#3683) 2022-03-31 10:52:54 +02:00
Oliver Nocon
276844e6a2
fix(optimization): expose parameters to general section (#3680) 2022-03-30 12:20:51 +02:00
Sven Merk
c30e93bc6a
feat(detectExecuteScan): SARIF export and GH issue creation (#3637)
* Added SARIF and GH issue creation
2022-03-17 15:32:48 +01:00
ffeldmann
ffd4a7efb8
(fix) nil pointer dereference in case credentials are not available (#3564)
* adds return in gcs upload in case error occurs e.g. no credentials, avoid nil pointer dereference

* Adds generated files

* Updates generated files

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-02-22 18:32:09 +01:00
Siarhei Pazdniakou
9994a4db45
feat(detectExecuteScan): add gcs upload to the step (#3423)
* Add gcs upload to detectExecuteScan step

* go generate

* patterns were updated

Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-02-21 08:07:55 +01:00
Giridhar Shenoy
286ff1b6ef
detectExecuteScan : support for detect 7 (#3453)
* enable detect 7 script

* unit test uses detect7 as default

* add detect6 test case

* add check for OSEnv detect version

* add test for OSEnv detect version

* update customEnvironmentVariables docu

* fix linting

Co-authored-by: ffeldmann <f.feldmann@sap.com>
2022-02-08 11:55:01 +01:00
Siarhei Pazdniakou
cd243ee542
feat(gcs): allow upload to gcs from steps (#3034)
* Upload reports to Google Cloud Storage bucket

* Added tests. Made fixes

* Update step generation. GCS client was moved to GeneralConfig

* Code was refactored

* Fixed issues

* Fixed issues

* Code correction due to PR comments

* Improved gcs client and integration tests

* Integrated gcp config. Updated step metadata

* Fixed issues. Added tests

* Added cpe, vault, aliases resolving for reporting parameters

* Added tests

* Uncommented DeferExitHandler. Removed useless comments

* fixed cloning of config

* Added comments for exported functions. Removed unused mock

* minor fix

* Implemented setting of report name via paramRef

* some refactoring. Writing tests

* Update pkg/config/reporting.go

* Update cmd/sonarExecuteScan_generated.go

* Apply suggestions from code review

* Update pkg/config/reporting.go

* Update pkg/config/reporting.go

* fixed removing valut secret files

* Update pkg/config/reporting.go

* restore order

* restore order

* Apply suggestions from code review

* go generate

* fixed tests

* Update resources/metadata/sonarExecuteScan.yaml

* Update resources.go

* Fixed tests. Code was regenerated

* changed somewhere gcp to gcs. Fixed one test

* move gcsSubFolder to input parameters

* fixed removing valut secret files

* minor fix in integration tests

* fix integration tests

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2021-12-15 15:07:47 +01:00
Christopher Fenner
b7e1d28675
fix: prevent endless loop in exit handler (#3363)
* do not fatal in exit handler

* update generated sources

* update golden sources

* update generated sources
2021-12-15 14:26:23 +01:00
Siarhei Pazdniakou
0879fa591a
fix(influx): fix generated output parameters (#3362)
* fixed generated output parameters for influx

* change name to lower case

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2021-12-15 09:40:50 +01:00
Roland Stengel
6320275f47
Step Parameter Alias deprecate key is ignored (#3158)
* Step Parameter Alias deprecate key is ignored
2021-11-23 15:37:28 +01:00
ffeldmann
42b92d1bfe
Changes for Pipeline Reporting (#3213)
* Adds GetLog() function to orchestrator

* Fixes BUILD_NUMBER env variable

* Fixes correct env var for JENKINS_HOME

* Adds getEnv to read env variables with default value, adds test for jenkins GetLog() implementation

* Adds possibility to read errorJsons; updates splunk package for log files (WIP)

* Uncommenting dev code

* Adds GetLog() function to orchestrator

* Fixes BUILD_NUMBER env variable

* Fixes correct env var for JENKINS_HOME

* Adds getEnv to read env variables with default value, adds test for jenkins GetLog() implementation

* Adds possibility to read errorJsons; updates splunk package for log files (WIP)

* Uncommenting dev code

* Adds GetRequest function which holds the response in memory (not saved to disk)

* Implements GetLog() function for ADO, adds function to read PipelineRuntime

* PAT has been revoked

* Changes http package, s.t. if password only is required basic auth works too

* Adds env variable for azure token, error handling in case of unauthenticated/nil response

* Adds logging output in case env variable can not be read and fallback variable needs to be used

* Adds usage of environment variables for auth, uses jenkins api

* Adds init functionality for orchestrators, updates GetLog() and GetPipelineStartTime() function

* Adds initaliziation function for orchestrator authetnication

* Adds settings struct for orchestrator authentication

* Adds function to whole logfile to Splunk

* Struct for pipeline related telemetry information

* Increase messagebatch size to 10k

* Changes splunk package to a pointer based implementation, updates generated files and corresponding template and tests for splunk

* Changes telemetry package to pointer based implementation to have multiple telemetry objects, adjusted tests and splunk implementation

* Changes content type to txt

* Send telemetry independent of logfiles, increases amount of messages per file

* Adds JobURL for orchestrators and UnknownOrchestrator as fallback

* telemetry makes use of orchestrator specific information

* Adds orchestrator independent correlationID

* Adds custom fields for pipeline status

* go fmt

* Removes env var test - no env variables are read anymore

* Use UnknownOrchestratorConfigProvider in case the orchestrator can not be initalized

* Removes Custom fields from telemetry as these can not be reflected in SWA

* Adds custom telemetry information (piperHash,..) to each step telemetry information

* Removes falltrough in case no orchestrator has been found

* Updates tests for orchestrator package

* Adds orchestrator import in generated files

* Updates generator files for internal library

* Adds orchestrator telemetry information to steps

* Updates generated files, fatalHook writes to cpe

* Go generate from master, go fmt

* Adds Custom Data field LastErrorCode

* Removes GetLog() test

* Update init_unix.go

* Update docker_integration_test_executor.go

* Update integration_api_cli_test.go

* Reverts go1.17 fmt formatting

* Reverts go1.17 fmt formatting

* Reverts go1.17 fmt formatting

* Renames customTelemetryData to stepTelemetryData

* Adjustments to orchestrator-package, cleanup, adds JobName

* Adjusts commonPipelineEnvironment path

* Adds pipelineTelemetry struct to telemetry package, removes pipeline telemetry structs from splunk package

* Go fmt

* Changes path for errorDetails, adds debug information

* Removes custom fields from step, adds orchestrator, commithash to baseMetadata

* Adjusts tests for telemetry package

* Adds tests for orchestrator

* Updates generated files, initalization of splunk client only if its available in the config

* Fixes typo in helper go

* Update pkg/http/downloader.go

* Update pkg/http/downloader.go

* Update pkg/log/fatalHook.go

* Update fatalHook.go

* Update pkg/splunk/splunk.go

* Update pkg/telemetry/data.go

* Adds GetBuildStatus() and GetAPIInformation() to orchestrators

* error formatting

* Bugfix: dont send telemetry data if disabled, adjusts test

* go fmt

* Use correct error handling

* Update pkg/telemetry/telemetry.go

* Fixes telemetry disabled in the tests

* Fixes http tests

* Log fatal errors to logFile

* Adds CustomReportingConfig to hooks

* Cleanup comments in splunk package

* Adds possibility to send telemetry to custom endpoint

* Adds debug output for the payload

* Debug output for the payload as a string

* Adds test cases for changes in telemetry package

* go fmt

* Adds generated files for new step

* Reverts changes for http tests, causing problems with go1.15, changes need to be applied for newer go version >=1.17

* Adjusts test for sonarExecuteScan

* Adjusts test for sonarExecuteScan

* Adds explanation for customreportingConfig

* Makes disableing of customSend more obvious

* Adds custom step reporting to each step, updates generated files, adjusts helper testdata

* fixes unit test wrong usage of logging

* Send pipeline data altough there has been no error, adjust test cases

* Reverts changes for customReporting

* Updates generated files, removes customReporting

* Removes writing errorDetails to CPE

* Reverts usage of customreporting

* go fmt

* reverts changes in http_test

* reverts changes in http_test

* Skips integration cnb test

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-11-18 17:50:03 +01:00
Siarhei Pazdniakou
e97242b7e7
Fixed validation for possibleValues option (#3228)
* Fixed validation for possibleValues option

* Change oneof-custom to possible-values

* go generate

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-11-15 12:06:48 +01:00
Oliver Nocon
817901c2e3
fix(detectEcecuteScan) remove possible values for multi-value fields (#3144)
Looks like validation introduced with PR #3125 has issues with
validation of possible values for multi-value parameters.

This is a workaround to remove the list of possible values for some parameters
and prevent the issue from happening
2021-10-04 08:22:46 +02:00
Siarhei Pazdniakou
46bafc40a3
Improve validation of configuration (#3125)
* Implemented validation for the option possibleValues

* Has been added the option mandatoryIf to config with validation

* Fixed issues found during code review

* improved golang template

* Fixed tests. Added validation for mandatoryIf option

* Fix typo

* Fixed tests

* Validation was refactored. Added options

* Added default value for parameters with possibleValues option

* Validation was moved after the configuration resolution

* Canceled some default values

Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-10-01 12:49:05 +02:00
Eugene Kortelyov
56be54c504
Feature/vault refactoring (#3113)
* refactor vault code

* adjust generator

* wip: fix tests

* regenerate influxdb

* fix test

* add another test

* fix test & docs

* fix formatting

* Minorupdate and fixes

Co-authored-by: Kevin Stiehl <kevin.stiehl@numericas.de>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-09-21 13:06:32 +02:00
Giridhar Shenoy
045c72cd3e
detect : Create html and json report upon scan completion (#3042)
* changes to detectExec before master merge

* changes for detectExecuteScan

* self generated code added

* fix syntax errors and update docu

* added unit tests for fail and Group

* fix failOn bug

* add Groups as string array

* add Groups as string array

* tests and validation for groups, failOn

* Updated docs and added more tests

* documentation md files should not be changed

* Handle merge conflicts from PR 1845

* fix merge errors

* remove duplicate groups, merge error

* adding buildCode and buildTool as params

* switching build options

* building maven modules

* parameter correction

* parameter correction

* gnerate with new build parameter

* adding comments

* removing piper lib master and modifying goUtils to download 1.5.7 release

* first cleaning then installing

* multi module maven built

* multi module maven built removing unwanted code

* multi module maven built moving inside switch

* testing

* modifying the default use case to also call maven build

* modifying the default use case to also call maven build wih --

* corrected maven build command

* corrected maven build command with %v

* skipping test runs

* testing for MTA project with single pom

* adding absolute path to m2 path

* clean up

* adding switch for mta and maven and removing env from containers

* commiting changes for new detect step

* correting log message

* code clean up

* unit tests changes to detectExecute

* basic tests for new change

* restoring piperGoUtils to download correct piper binary

* code clean up

* code clean up

* add basic reporting

* write html and json reports

* fix syntax errors and tests

* sort values in report by vuln

* add more unit tests

Co-authored-by: Keshav <anil.keshav@sap.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2021-08-12 15:58:33 +02:00
Oliver Nocon
805a8fd88f
feat(config): read config/defaults with authentication (#2975)
* feat(config):read config/defaults with authentication

This change allows to use defaults and config files from a protected GitHub repository.

The options `--customConfig` and `--defaultConfig` already allowed to provide a link to an uprotected file.

Now, by passing a value in the form `<hostname>:<token>` to parameter `gitHubTokens` (this parameter can be passed multiple times) a token can be provided for dedicated hosts.

This makes it possible to use a link like
`https://api.github.com/repos/SAP/jenkins-library/contents/resources/my-defaults.yml?ref=master`
as reference to a default file or similarly as reference to a configuration file.

* update generation to allow protected config/defaults

* fix CodeClimate issues

* update missing generations
2021-07-08 15:26:07 +02:00
Oliver Nocon
0b48bfcc73
feat: retrieve metadata by stepName - corrected (#2892)
* refactored getConfig to allow stepName param for metadata fetching

* extended step generator

* go generate

* Update cmd/getConfig.go

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Update cmd/getConfig.go

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Update cmd/getConfig.go

* update generated files

* update golden files to care for generator update

* update and add tests

* update generated files

* Update cmd/getConfig.go

* Update cmd/getConfig.go

* update/fix formatting

* feat: retrieve metadata by stepName - corrected

* update generation

* update condition logic for defaults

* update generation & tests

* support multiple conditions

* update generation

* Add generated

Co-authored-by: Leander Schulz <leander.schulz01@sap.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Sven Merk <s.merk@sap.com>
2021-06-16 08:43:30 +02:00
Oliver Nocon
4250ca8bed
Revert "feat(getConfig): retrieve metadata by stepName (#2736)" (#2891)
This reverts commit ae4a24c594.
2021-06-14 10:57:44 +02:00
lndrschlz
ae4a24c594
feat(getConfig): retrieve metadata by stepName (#2736)
* refactored getConfig to allow stepName param for metadata fetching

* extended step generator

* go generate

* Update cmd/getConfig.go

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Update cmd/getConfig.go

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Update cmd/getConfig.go

* update generated files

* update golden files to care for generator update

* update and add tests

* update generated files

* Update cmd/getConfig.go

* Update cmd/getConfig.go

* update/fix formatting

Co-authored-by: Leander Schulz <leander.schulz01@sap.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: OliverNocon <oliver.nocon@sap.com>
2021-06-14 08:58:41 +02:00
ffeldmann
b88ebdad6c
feat(splunk) Sending telemetry and logging information to Splunk (#2796)
* Adds inital splunk hook and logCollector

* Adds documentation of the Splunk hook

* Fixes markdown lint issues and removes comment from telemetry.go file

* Fixes markdown lint issues and adds missing generated file

* Markdown linting

* Changes documentation according to review, adds Splunk token automatically during init

* Adds error handling for marshalling hook config

* Markdown lint und correct Splunk token in httpclient

* Registeres Splunk token as secret and adjusts test cases

* Adds missing error handling and removes unnecessary comments

* Creates new function readPipelineEnvironment, adds tests

* Moves MonitoringData struct, edits defaults for json fields

* Adds gitRepository and gitOwner to telemetry information

* Simplifies readCommonPipelineEnvironment function, adds more descriptive errors, adds automated adding of Splunk prefix token

* Adjusts error handling

* Cleaner error logging
2021-05-17 12:14:04 +02:00
Christopher Fenner
804e66d4cd
feat(detect): add customScanVersion to detect scan (#2790)
* add versioningModel parameter

* extract versioning model to own package

* move log message

* use versioning method

* add customScanVersion parameter

* use customScanVersion

* adjust docs on other steps

* add customScanVersion parameter

* use customScanVersion

* adjust docs on other steps

* change log message

* update test case

* fix typo

* correct variable name
2021-05-05 10:24:05 +02:00
Giridhar Shenoy
eec0ebb235
fix(detect): bug fix for codelocation not picking the right location (#2745)
* changes to detectExec before master merge

* changes for detectExecuteScan

* self generated code added

* fix syntax errors and update docu

* added unit tests for fail and Group

* fix failOn bug

* add Groups as string array

* add Groups as string array

* tests and validation for groups, failOn

* Updated docs and added more tests

* documentation md files should not be changed

* Handle merge conflicts from PR 1845

* fix merge errors

* remove duplicate groups, merge error

* adding buildCode and buildTool as params

* switching build options

* building maven modules

* parameter correction

* parameter correction

* gnerate with new build parameter

* adding comments

* removing piper lib master and modifying goUtils to download 1.5.7 release

* first cleaning then installing

* multi module maven built

* multi module maven built removing unwanted code

* multi module maven built moving inside switch

* testing

* modifying the default use case to also call maven build

* modifying the default use case to also call maven build wih --

* corrected maven build command

* corrected maven build command with %v

* skipping test runs

* testing for MTA project with single pom

* adding absolute path to m2 path

* clean up

* adding switch for mta and maven and removing env from containers

* commiting changes for new detect step

* correting log message

* code clean up

* unit tests changes to detectExecute

* basic tests for new change

* restoring piperGoUtils to download correct piper binary

* code clean up

* code clean up

* clean detect code version

* Changes for detect codelocation

* remove unmap from scanProperties

* fix/add unit tests for remove unmap

* handle spaces in scanProperties

* update default scanproperties to remove deprecations

* Set default scanonchange to true

* handle multiple unmap true params

* add custom env variables

* fix codeclimate issues

* Update resources/metadata/detect.yaml

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Update resources/metadata/detect.yaml

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* update generated files from yaml

* bug fix - revert code location changes

Co-authored-by: Keshav <anil.keshav@sap.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2021-04-08 11:04:49 +02:00
Giridhar Shenoy
97b3a23336
HotFix detectExecuteScan: Use environment variables to get supported version of detect (#2738)
* changes to detectExec before master merge

* changes for detectExecuteScan

* self generated code added

* fix syntax errors and update docu

* added unit tests for fail and Group

* fix failOn bug

* add Groups as string array

* add Groups as string array

* tests and validation for groups, failOn

* Updated docs and added more tests

* documentation md files should not be changed

* Handle merge conflicts from PR 1845

* fix merge errors

* remove duplicate groups, merge error

* adding buildCode and buildTool as params

* switching build options

* building maven modules

* parameter correction

* parameter correction

* gnerate with new build parameter

* adding comments

* removing piper lib master and modifying goUtils to download 1.5.7 release

* first cleaning then installing

* multi module maven built

* multi module maven built removing unwanted code

* multi module maven built moving inside switch

* testing

* modifying the default use case to also call maven build

* modifying the default use case to also call maven build wih --

* corrected maven build command

* corrected maven build command with %v

* skipping test runs

* testing for MTA project with single pom

* adding absolute path to m2 path

* clean up

* adding switch for mta and maven and removing env from containers

* commiting changes for new detect step

* correting log message

* code clean up

* unit tests changes to detectExecute

* basic tests for new change

* restoring piperGoUtils to download correct piper binary

* code clean up

* code clean up

* clean detect code version

* Changes for detect codelocation

* remove unmap from scanProperties

* fix/add unit tests for remove unmap

* handle spaces in scanProperties

* update default scanproperties to remove deprecations

* Set default scanonchange to true

* handle multiple unmap true params

* add custom env variables

* fix codeclimate issues

* Update resources/metadata/detect.yaml

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Update resources/metadata/detect.yaml

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* update generated files from yaml

Co-authored-by: Keshav <anil.keshav@sap.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2021-04-01 11:24:25 +02:00
Giridhar Shenoy
6805654cdb
detectExecuteScan : Default to scanOnChanges to true, reduce load on the server (#2733)
* changes to detectExec before master merge

* changes for detectExecuteScan

* self generated code added

* fix syntax errors and update docu

* added unit tests for fail and Group

* fix failOn bug

* add Groups as string array

* add Groups as string array

* tests and validation for groups, failOn

* Updated docs and added more tests

* documentation md files should not be changed

* Handle merge conflicts from PR 1845

* fix merge errors

* remove duplicate groups, merge error

* adding buildCode and buildTool as params

* switching build options

* building maven modules

* parameter correction

* parameter correction

* gnerate with new build parameter

* adding comments

* removing piper lib master and modifying goUtils to download 1.5.7 release

* first cleaning then installing

* multi module maven built

* multi module maven built removing unwanted code

* multi module maven built moving inside switch

* testing

* modifying the default use case to also call maven build

* modifying the default use case to also call maven build wih --

* corrected maven build command

* corrected maven build command with %v

* skipping test runs

* testing for MTA project with single pom

* adding absolute path to m2 path

* clean up

* adding switch for mta and maven and removing env from containers

* commiting changes for new detect step

* correting log message

* code clean up

* unit tests changes to detectExecute

* basic tests for new change

* restoring piperGoUtils to download correct piper binary

* code clean up

* code clean up

* clean detect code version

* Changes for detect codelocation

* remove unmap from scanProperties

* fix/add unit tests for remove unmap

* handle spaces in scanProperties

* update default scanproperties to remove deprecations

* Set default scanonchange to true

* handle multiple unmap true params

Co-authored-by: Keshav <anil.keshav@sap.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-03-31 10:53:49 +02:00
Giridhar Shenoy
03f46ef90c
DetectExecuteScan : Codelocations autodetermined by detect script (#2704)
* changes to detectExec before master merge

* changes for detectExecuteScan

* self generated code added

* fix syntax errors and update docu

* added unit tests for fail and Group

* fix failOn bug

* add Groups as string array

* add Groups as string array

* tests and validation for groups, failOn

* Updated docs and added more tests

* documentation md files should not be changed

* Handle merge conflicts from PR 1845

* fix merge errors

* remove duplicate groups, merge error

* adding buildCode and buildTool as params

* switching build options

* building maven modules

* parameter correction

* parameter correction

* gnerate with new build parameter

* adding comments

* removing piper lib master and modifying goUtils to download 1.5.7 release

* first cleaning then installing

* multi module maven built

* multi module maven built removing unwanted code

* multi module maven built moving inside switch

* testing

* modifying the default use case to also call maven build

* modifying the default use case to also call maven build wih --

* corrected maven build command

* corrected maven build command with %v

* skipping test runs

* testing for MTA project with single pom

* adding absolute path to m2 path

* clean up

* adding switch for mta and maven and removing env from containers

* commiting changes for new detect step

* correting log message

* code clean up

* unit tests changes to detectExecute

* basic tests for new change

* restoring piperGoUtils to download correct piper binary

* code clean up

* code clean up

* clean detect code version

* Changes for detect codelocation

* remove unmap from scanProperties

* fix/add unit tests for remove unmap

* handle spaces in scanProperties

* update default scanproperties to remove deprecations

Co-authored-by: Keshav <anil.keshav@sap.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-03-26 13:06:13 +01:00
lndrschlz
4ca9186f39
fix(uiveri5ExecuteTests): add 'tests' stash to step yaml (#2641)
* add tests stash to uiveri5 step

* add stash to uiveri5

* extend step generator with input resources

* add step generator test
2021-03-01 13:03:42 +01:00
Giridhar Shenoy
d3f31acc9f
feat(detect): Scanonchanges : Parameter to reduce scan running time and load on BlackDuck server (#2538)
* changes to detectExec before master merge

* changes for detectExecuteScan

* self generated code added

* fix syntax errors and update docu

* added unit tests for fail and Group

* fix failOn bug

* add Groups as string array

* add Groups as string array

* tests and validation for groups, failOn

* Updated docs and added more tests

* documentation md files should not be changed

* Handle merge conflicts from PR 1845

* fix merge errors

* remove duplicate groups, merge error

* adding buildCode and buildTool as params

* switching build options

* building maven modules

* parameter correction

* parameter correction

* gnerate with new build parameter

* adding comments

* removing piper lib master and modifying goUtils to download 1.5.7 release

* first cleaning then installing

* multi module maven built

* multi module maven built removing unwanted code

* multi module maven built moving inside switch

* testing

* modifying the default use case to also call maven build

* modifying the default use case to also call maven build wih --

* corrected maven build command

* corrected maven build command with %v

* skipping test runs

* testing for MTA project with single pom

* adding absolute path to m2 path

* clean up

* adding switch for mta and maven and removing env from containers

* commiting changes for new detect step

* correting log message

* code clean up

* unit tests changes to detectExecute

* basic tests for new change

* restoring piperGoUtils to download correct piper binary

* code clean up

* code clean up

* clean detect code

* add the the scanOnChanges parameter to detect

* fix codeclimate issue

* updated detect.yaml

Co-authored-by: Keshav <anil.keshav@sap.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2021-01-29 10:17:02 +01:00
Giridhar Shenoy
205bbc1bed
fix(detect): scan paths to better handle signature scan and dependency scan paths (#2508)
* changes to detectExec before master merge

* changes for detectExecuteScan

* self generated code added

* fix syntax errors and update docu

* added unit tests for fail and Group

* fix failOn bug

* add Groups as string array

* add Groups as string array

* tests and validation for groups, failOn

* Updated docs and added more tests

* documentation md files should not be changed

* Handle merge conflicts from PR 1845

* fix merge errors

* remove duplicate groups, merge error

* adding buildCode and buildTool as params

* switching build options

* building maven modules

* parameter correction

* parameter correction

* gnerate with new build parameter

* adding comments

* removing piper lib master and modifying goUtils to download 1.5.7 release

* first cleaning then installing

* multi module maven built

* multi module maven built removing unwanted code

* multi module maven built moving inside switch

* testing

* modifying the default use case to also call maven build

* modifying the default use case to also call maven build wih --

* corrected maven build command

* corrected maven build command with %v

* skipping test runs

* testing for MTA project with single pom

* adding absolute path to m2 path

* clean up

* adding switch for mta and maven and removing env from containers

* commiting changes for new detect step

* correting log message

* code clean up

* unit tests changes to detectExecute

* basic tests for new change

* restoring piperGoUtils to download correct piper binary

* code clean up

* code clean up

* revert to clean version

* add unmap parameter to detect

* Added Additional parameters for BlackDuck scan

* Added detect tools paramater

* fix detect.yaml to accept correct data type

* fix codeclimate issue in detect.yaml

Co-authored-by: Keshav <anil.keshav@sap.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2021-01-21 14:57:00 +01:00
lndrschlz
b9bab27833
feat: expose complete step metadata through generated function (#2329)
* exposing step metadata through generator

* add metadata_generated.go

* fix step go test generation

* metadata fields added to generated files

* added generated files

* removed image placeholder from fortify step

* refactored step meta generation

* go generate

* fixed metadata generator and tests

* added output resource fields/tags to metadata generator

* fix string in metadata_generated

* go generate

* fixed generator

* go generate

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-11-20 08:13:59 +01:00
Daniel Kurzynski
7946265e21
Install maven artifacts before running Detect (#2292) 2020-11-03 11:08:23 +01:00
Kevin Stiehl
24aafb0b69
add vaultSecretFileReferences (#2203)
* add vaultSecretFileReferences

* fix test

* fix test

* go generate

* remove code duplication

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-10-26 14:20:04 +01:00
Kevin Stiehl
3eae0c5f68
feat(vault): fetch secrets from vault (#2032)
* cloud-foundry & sonar from vault

* add vault development hint

* don't abort on vault errors

* cloudfoundry make credentialsId only mandatory when vault is not configured

* add vault ref to step ymls

* rename vaultAddress to vaultServerUrl

* rename PIPER_vaultRole* to PIPER_vaultAppRole*

* add resourceRef for detect step

* fix error when no namespace is set

* added debug logs

* added debug logs

* fix vault resolving

* add vaultCustomBasePath

* rename vault_test.go to client_test.go

* refactored vault logging

* refactored config param lookup for vault

* added tüddelchen

* rename vaultCustomBasePath to vaultPath

* fix tests

* change lookup path for group secrets

* fix interpolation tests

* added vault resource ref to versioning

* execute go generate

* rename Approle to AppRole

* change verbose back to false

Co-authored-by: Leander Schulz <leander.schulz01@sap.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-10-13 14:14:47 +02:00
Anil Keshav
627769f9c1
modifying detect.maven.excluded.scopes from TEST to test (#2129)
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-10-06 13:32:52 +02:00
Oliver Nocon
b506235398
detect: add parameter aliases (#2099)
* align parameters for Detect

* include feedback

* update generation and formatting

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Daniel Kurzynski <daniel.kurzynski@sap.com>
2020-10-01 13:34:51 +02:00
Oliver Nocon
19c1732826
Telemetry: report error category (#2085) 2020-09-29 13:49:40 +02:00
Anil Keshav
69dfdff003
adding --detect.maven.excluded.scopes=TEST parameter as a default to detectExecuteScan step (#2038)
* adding defult parameter to exclude test to detectExecute yml

* generating the modified cmd/detectExecuteScan_generated.go

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-09-18 16:05:52 +02:00
Kevin Stiehl
d589038206
Vault AppRole login (#1971)
* added interpolation package in config

* vault allow paths to hold config references

* allow referencing properties in vaultPaths

* fix small typo

* add approleAuth

* register resolved secrets to logger

* generate steps

* clean up

* add integration test

* add vault to context filter

* reduce Cognitive Complexity & added tests

* Update pkg/config/stepmeta_test.go

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

* go generate

* go generate after merge

* rename VaultAppRole* to VaultRole*

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-09-16 14:50:09 +02:00
Daniel Kurzynski
8ee0d358b9
Support maven params in detect scan (#1855)
Co-authored-by: Florian Wilhelm <florian.wilhelm02@sap.com>
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-07-30 10:35:46 +02:00
Stephan Aßmus
9009c831fb
Declare non-optional server URLs mandatory (#1866)
* Declare non-optional server URLs mandatory
2020-07-30 09:13:46 +02:00