* changes to detectExec before master merge
* changes for detectExecuteScan
* self generated code added
* fix syntax errors and update docu
* added unit tests for fail and Group
* fix failOn bug
* add Groups as string array
* add Groups as string array
* tests and validation for groups, failOn
* Updated docs and added more tests
* documentation md files should not be changed
* Handle merge conflicts from PR 1845
* fix merge errors
* remove duplicate groups, merge error
* adding buildCode and buildTool as params
* switching build options
* building maven modules
* parameter correction
* parameter correction
* gnerate with new build parameter
* adding comments
* removing piper lib master and modifying goUtils to download 1.5.7 release
* first cleaning then installing
* multi module maven built
* multi module maven built removing unwanted code
* multi module maven built moving inside switch
* testing
* modifying the default use case to also call maven build
* modifying the default use case to also call maven build wih --
* corrected maven build command
* corrected maven build command with %v
* skipping test runs
* testing for MTA project with single pom
* adding absolute path to m2 path
* clean up
* adding switch for mta and maven and removing env from containers
* commiting changes for new detect step
* correting log message
* code clean up
* unit tests changes to detectExecute
* basic tests for new change
* restoring piperGoUtils to download correct piper binary
* code clean up
* code clean up
* protecodeExecuteScan :: fixing file upload for binaries
* protecodeExecuteScan :: fixing protecode generate file
* Fix upload test
* protecodeExecuteScan -> fixing tests
Co-authored-by: D072410 <giridhar.shenoy@sap.com>
Co-authored-by: Keshav <anil.keshav@sap.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
This is because the cnb builder images usually don't run as root user.
As a workaround we:
- Copied the system truststore to a tmp-file
- Added the certificates to the tmp-file
- Set the `SSL_CERT_FILE` environment variable
Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>
Co-authored-by: Sumit Kulhadia <sumit.kulhadia@sap.com>
Looks like validation introduced with PR #3125 has issues with
validation of possible values for multi-value parameters.
This is a workaround to remove the list of possible values for some parameters
and prevent the issue from happening
* Implemented validation for the option possibleValues
* Has been added the option mandatoryIf to config with validation
* Fixed issues found during code review
* improved golang template
* Fixed tests. Added validation for mandatoryIf option
* Fix typo
* Fixed tests
* Validation was refactored. Added options
* Added default value for parameters with possibleValues option
* Validation was moved after the configuration resolution
* Canceled some default values
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* using curl instead of http put
* avoiding multi-form upload
* clean up
Co-authored-by: Your Name <you@example.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* extend orchestator to provide stage name
* use orchestrator specific stage name
* fix test case
* remove comment
* fix test case
* prettify
* change something..
* do not exit
* Update pkg/orchestrator/azureDevOps.go
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* fix(kubernetesDeploy): created secret type incorrectly set because of double quotes
* fix(test): update tests
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* mta build config
* http request to upload mtar
* adding basic auth
* using put
* test file name
* hard coding the mta org and artifact is
* new version
* new version
* mtar group
* errors
* better error message
* log info
* log info
* correct mtar artifact name
* adding teh correct name
* test
* name changes
* clean up
* changing mtarVersion to version
* changing artifact name
* forcing release build
* forcing release build
* force profile
* force profile
* force profile
* force profile
* force profile
* force profile
* force profile
* force profile update
* force profile update
* profile update
* debug
* debug
* debug
* debug
* rewrite xml update
* rewrite xml update
* unmarshal solution
* unmarshal solution
* unmarshal solution
* unmarshal solution
* unmarshal solution
* unmarshal solution
* unmarshal solution
* unmarshal solution
* unmarshal solution
* outputin publish repo url
* removing fetch coordinates condition
* checking settings xml
* fixing artifact id cpe
* release artifact, package and group to cpe
* including versioning type as a cpe
* creating new settings xml file
* creating parent folder
* creating parent folder
* creating parent folder
* creating parent folder
* creating parent folder
* creating parent folder
* creating parent folder
* creating parent folder
* creating parent folder
* creating parent folder
* changing to project settings
* function name change
* using glbl settings xml
* modiying the npm settings
* modiying the npm settings
* modiying the npm settings
* modiying the npm settings
* using file path join for m2 settings file
* generator
* unit tests
* hardening error message
* removing versioningType
* removing versioningType
* new vault profile paths
* error message improvement
* unit test fixes
Co-authored-by: Your Name <you@example.com>
* release artifact, package and group to cpe
* including versioning type as a cpe
* unit test
* eliminating versioningType
Co-authored-by: Your Name <you@example.com>
* Docs for user provided builder images
Co-authored-by: Benjamin Haegenlaeuer <benjamin.haegenlaeuer@sap.com>
* check if provided dockerImage is a valid builder
Co-authored-by: Pavel Busko <pavel.busko@sap.com>
Co-authored-by: Benjamin Haegenlaeuer <benjamin.haegenlaeuer@sap.com>
* feat(detectExecuteScan): generate ip result json
json will currently only be created in success cases.
No information about policy violation details available in the step yet.
* update report name
* Update cmd/detectExecuteScan.go
Co-authored-by: Giridhar Shenoy <giridhar.shenoy@sap.com>
* Update cmd/detectExecuteScan.go
Co-authored-by: Giridhar Shenoy <giridhar.shenoy@sap.com>
* Update cmd/detectExecuteScan_test.go
Co-authored-by: Giridhar Shenoy <giridhar.shenoy@sap.com>
* Update cmd/detectExecuteScan_test.go
Co-authored-by: Giridhar Shenoy <giridhar.shenoy@sap.com>
* move blackduck api package
* detectExecuteStep :: Adding error code mapping
* detectExecuteScan :: fixing generate/format check failure
* detectExecuteScan :: fixing typo in test
* detectExecuteStep :: Fixing unit tests and formating issue
* detectExecuteStep :: fixing test case -> TestExitCodeMapping
Co-authored-by: OliverNocon <oliver.nocon@sap.com>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Giridhar Shenoy <giridhar.shenoy@sap.com>
* add policy status reports
* add policy status and cumulus json
* update projectver link + test
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
* Fix broken link
* Add config info into documentation
* Add line
* Add lines
* Link documentation to protocodeExecuteScan
* Remove space
* Change wording
Co-authored-by: Linda Siebert <linda.siebert@sap.com>
Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>
* Improve documentation for protecodeExecuteScan
protecodeExecuteScan expects that the docker config file is named
'config.json'.
The dockerConfigJSON parameter is used by protecodeExecuteScan to
retrieve the directory only (for environment variable DOCKER_CONFIG).
* Also improve documentation that is generated
* No relative references in generated documentation.
* Less complex documentation
* Remove irrelevant empty line
* Improvements after review
* Add blank line after code block
* Handle registry URL example as code
Markdown format checker does not like raw URLs.
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
* Broken single containerImage argument has been removed
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
* revert containerRegistryUrl param name
Co-authored-by: Pavel Busko <pavel.busko@sap.com>
* mark containerImageName, containerImageTag and containerRegistryUrl as mandatory arguments
Co-authored-by: Pavel Busko <pavel.busko@sap.com>
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
Co-authored-by: Benjamin Haegenlaeuer <benjamin.haegenlaeuer@sap.com>
* modify wrapper http to hold ca rot
* modifying the http client for maven build
* adding checks
* eliminating seperate jks
* test tls true
* insecure flag remove
* error debug
* storing cert
* sap root cert
* error
* only child cert
* test
* maven test
* moving outside loop
* changing pointer
* dont download existing certs
* typo fix
* removing mavenBuild test
* code clean up
* making hadolint using always cert true
* custom tls link for hadolint trust
* error handel
* extended condition for modifying custom tls
* unit test case
* checing when to add the customLinks
* not breaking existing hadolint client config
Co-authored-by: Your Name <you@example.com>
* Implement cnbBuild step
Co-authored-by: Benjamin Haegenlaeuer <benjamin.haegenlaeuer@sap.com>
* Add cnbBuild groovy test
Co-authored-by: Benjamin Haegenlaeuer <benjamin.haegenlaeuer@sap.com>
* Add basic documentation template
Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>
* Support specifiying name, tag and registry
Co-authored-by: Pavel Busko <pbusko@users.noreply.github.com>
Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>
Co-authored-by: Pavel Busko <pbusko@users.noreply.github.com>
* changes to detectExec before master merge
* changes for detectExecuteScan
* self generated code added
* fix syntax errors and update docu
* added unit tests for fail and Group
* fix failOn bug
* add Groups as string array
* add Groups as string array
* tests and validation for groups, failOn
* Updated docs and added more tests
* documentation md files should not be changed
* Handle merge conflicts from PR 1845
* fix merge errors
* remove duplicate groups, merge error
* adding buildCode and buildTool as params
* switching build options
* building maven modules
* parameter correction
* parameter correction
* gnerate with new build parameter
* adding comments
* removing piper lib master and modifying goUtils to download 1.5.7 release
* first cleaning then installing
* multi module maven built
* multi module maven built removing unwanted code
* multi module maven built moving inside switch
* testing
* modifying the default use case to also call maven build
* modifying the default use case to also call maven build wih --
* corrected maven build command
* corrected maven build command with %v
* skipping test runs
* testing for MTA project with single pom
* adding absolute path to m2 path
* clean up
* adding switch for mta and maven and removing env from containers
* commiting changes for new detect step
* correting log message
* code clean up
* unit tests changes to detectExecute
* basic tests for new change
* restoring piperGoUtils to download correct piper binary
* code clean up
* code clean up
* add basic reporting
* write html and json reports
* fix syntax errors and tests
* sort values in report by vuln
* add more unit tests
Co-authored-by: Keshav <anil.keshav@sap.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
it is possible to overwrite the entrypoint for docker execution:
https://docs.docker.com/engine/reference/run/#entrypoint-default-command-to-execute-at-runtime
This is ideally done by passing `entrypoint=''` and not pass two options to the call.
This also helps with escaping issues of the empty value on other systems.
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
* Added checkIfStepActive step
* Implemented npmScripts condition. Code was refactored
* Added some unit tests
* Fixed go modules
* Fixed go modules
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* feat(detectExecuteScan): generate ip result json
json will currently only be created in success cases.
No information about policy violation details available in the step yet.
* update report name
* Update cmd/detectExecuteScan.go
Co-authored-by: Giridhar Shenoy <giridhar.shenoy@sap.com>
* Update cmd/detectExecuteScan.go
Co-authored-by: Giridhar Shenoy <giridhar.shenoy@sap.com>
* Update cmd/detectExecuteScan_test.go
Co-authored-by: Giridhar Shenoy <giridhar.shenoy@sap.com>
* Update cmd/detectExecuteScan_test.go
Co-authored-by: Giridhar Shenoy <giridhar.shenoy@sap.com>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Giridhar Shenoy <giridhar.shenoy@sap.com>
* Toolrecord framework -
provide a common entry point for post processing code scan results
Changes to be committed:
new file: pkg/toolrecord/REAMDE_toolrecord.md
new file: pkg/toolrecord/toolrecord_main.go
new file: pkg/toolrecord/toolrecord_test.go
* Add toolrecord file to Checkmarx results
modified: cmd/checkmarxExecuteScan.go
* Add toolrecord file to Fortify results
modified: cmd/fortifyExecuteScan.go
* Add toolrecord file to Whitesource results
modified: cmd/whitesourceExecuteScan.go
* unset umask (#2927)
* (feat) adds error logging output for downloading reports from whitesource (#2928)
* Add toolrecord file to Protecode results
* address code climate findings (1/2)
* address codeclimate findings (2/2)
* add comments to all methods
* Toolrecord library:
- move all toolrun files into a subdirectory
- fix timestamp generation in filenames
* add protecode group's URL to toolrecord data
* fix syntax error from previous commit in cmd/protecodeExecuteScan.go
* toolrecord: fix projectVersionID and generated URLs in fortifyExecuteScan.go
* cmd/fortifyExecuteScan.go: replace a hard-coded servername with
config.ServerURL
* update description
* add toolrecord file to detectExecuteScan
* toolrecord/whitesource: add project names as context
Co-authored-by: Kevin Stiehl <kevin.stiehl@numericas.de>
Co-authored-by: ffeldmann <felix@bnbit.de>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
* using default java truststore
* default java keytore
* remove trust store
* working directory
* change dir java_home
* env variable
* trying to find jre home
* changing directory to jre home and java home
* trying java jre
* remving $
* trying to search the cacerts
* copying existing cacerts
* removing change directory
* searching for secrutiy folder only
* searching cacerts
* new path for cacert
* path to ca-cert
* new trust store
* changing cacert location
* only adding maven_opts env variable once
* log message
* ca cert path from user
* handelling interface modification
* enhance logs and code clean up
* code clean up
Co-authored-by: Your Name <you@example.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* feat(config):read config/defaults with authentication
This change allows to use defaults and config files from a protected GitHub repository.
The options `--customConfig` and `--defaultConfig` already allowed to provide a link to an uprotected file.
Now, by passing a value in the form `<hostname>:<token>` to parameter `gitHubTokens` (this parameter can be passed multiple times) a token can be provided for dedicated hosts.
This makes it possible to use a link like
`https://api.github.com/repos/SAP/jenkins-library/contents/resources/my-defaults.yml?ref=master`
as reference to a default file or similarly as reference to a configuration file.
* update generation to allow protected config/defaults
* fix CodeClimate issues
* update missing generations
* default images replaced with images with public CA
* go files regenerated
* node buster image changed to lts-stretch in npmExecuteScripts, hadolint image rollbackw
* go files regenerated
* node buster image changed to lts-stretch in npmExecuteLint
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
* Make credential id name more descriptive
* Improve serviceKey description
* Change descriptions to add word service
Co-authored-by: Linda Siebert <linda.siebert@sap.com>
* new step integrationArtifactTriggerIntegrationTest
* add new step into allow list
* add the new step to main command
* refer cpe
* remove unused unit tests
* Check methods and URLs of http request
* Add TriggerIntegration to mockingutils
* Format code
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
Co-authored-by: Linda Siebert <linda.siebert@sap.com>
* Change credential name in yaml
Change cpiAPIServiceKeyCredentialId to cpiApiServiceKeyCredentialsId in cpi yaml files
* Refactor groovy files
Change cpiAPIServiceKeyCredentialId to cpiApiServiceKeyCredentialsId in cpi groovy files
* Change credential name in md
Change cpiAPIServiceKeyCredentialId to cpiApiServiceKeyCredentialsId in cpi md files
* Make sure artifacts go to local repo
* Just package
* Fix test
* Try out silent mode
* Try fail at end
* Bring resilience back
* Follow new strategy
* Fix test
* Switch to service key for CPI GetMplStatus
Introduces read method for service key files, mock utils and tests.
* Use secret text instead of file
* Change serviceKey definition
* Update cpiUpload to use Service Key
retrieved the host and uaa information from service key
* Update cpiDeploy to use service key
retrieved the host and uaa information from service key
* Update cpiServiceEndpoint to use Service Key
retrieved the host and uaa information from service key
* Update cpiDownload to use Service Key
retrieved the host and uaa information from service key
* Update cpiUpdateConfig to use Service Key
retrieved the host and uaa information from service key
* Refactor serviceKey var name
* Fixed references to service key to follow the real format
they should be accessed through oauth instead of uaa because of the format of the json
* Rename ServiceKey to APIServiceKey
To support having a different service key(and for readability), we need to change the name to API.
* Add STAGES and STEPS yaml
add in to each yaml file of cpi integration
* Revert "Add STAGES and STEPS yaml"
This reverts commit aa2665d158.
* Change comments/formatting commonUtils
Make comments more understandable and follow code climate suggestions
* Change documentation files for steps
remove OAuth and host and change credentials to be servicekey
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
Co-authored-by: Thorsten Duda <thorsten.duda@sap.com>
* feat(protecodeExecuteScan): allow scanning a binary
so far the step only accepts a Docker image or a FetchURL
This adds functionality to also pass a FilePath
* Update protecodeExecuteScan.go
* Update protecodeExecuteScan.go
* Update protecodeExecuteScan.go
* Update protecodeExecuteScan.go
* Update protecodeExecuteScan.go
* Update protecodeExecuteScan.go
* Fix fmt
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Sven Merk <s.merk@sap.com>
In case the Github connection leads to an error it can happen that a nil pointer dereference exception can occur.
This is to fix this.
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
* Update fortifyExecuteScan.go
* Raise error to the top level
* Update fortifyExecuteScan.go
* Update fortifyExecuteScan.go
* Fix code and test
* Add tests
* Fix test
* Last attempt
* Add mtaBuild step parameters: target, source
* Add unit test for mtaBuild with custom source
* Simplify mtaBuild parameter defaults for source, target
* Fix mtaBuild default values and path in unit test
* Only append custom target and source params in mtaBuild
* Set mtaBuild source and target to ./ when not customized
* mtaBuild unit test: fix expected params sequence
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Thorsten Duda <thorsten.duda@sap.com>
* Introduce docker config.json parameter for kubectl
* Use docker config.json parameter for helm
* Export definition of kube secret parameters
* fix username password existence check
* choose more fitting name for secret spec
* Adopt review suggestions
Co-authored-by: Roland Stengel <r.stengel@sap.com>
* Make sonarExecuteScan orchestrator-agnostic
* Increase coverage + support empty or false env vars
* Use cleared env for unit tests
* Refactor to standalone package
* Fix review findings
* Fix review findings
* Fix unit test
* Add logging
* Refactor
* Add to codeowners 😎
* Apply suggestions from code review
* Remove unreachable code
* no message
* fix typos
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
* modifying detect.maven.excluded.scopes from TEST to test
* new maven alt deployment flags
* changing flag names
* tlsCertificate addtion
* adding publish flags
* new flags
* publish flag
* enhance maven builds
* enhance maven builds
* creating new settings xml
* updating project settings
* changing interface for artifactPreparation that uses the same maven util niterface
* adding general scope to maven params
* global reference
* removing vault tmp
* debuging deployment user
* more debug
* maven build paras
* using smaller case
* adding incorrect error check
* adding deployment flags
* code refactor
* unit tests
* changing scope of paramter for tls certs
* new scope for tls
* remove trailing space in mavenBuild.yaml
* trailing space fix
* typo fix and jenkins secret
* including jenkins credentials for repo pass in the maven build groovy
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* Adds headCommitId, which stores the head commit has of the current build, includes tests and generated files
* Adds headCommitId, which stores the head commit has of the current build, includes tests and generated files
* Go fmt fix
* Fixes artifactoryPrepareVersion test
* Removes xMake CommitId
* get rid of classic mta builder
* fix yaml lint error
* adding new line in yaml
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
