* changes to detectExec before master merge
* changes for detectExecuteScan
* self generated code added
* fix syntax errors and update docu
* added unit tests for fail and Group
* fix failOn bug
* add Groups as string array
* add Groups as string array
* tests and validation for groups, failOn
* Updated docs and added more tests
* documentation md files should not be changed
* Handle merge conflicts from PR 1845
* fix merge errors
* remove duplicate groups, merge error
* adding buildCode and buildTool as params
* switching build options
* building maven modules
* parameter correction
* parameter correction
* gnerate with new build parameter
* adding comments
* removing piper lib master and modifying goUtils to download 1.5.7 release
* first cleaning then installing
* multi module maven built
* multi module maven built removing unwanted code
* multi module maven built moving inside switch
* testing
* modifying the default use case to also call maven build
* modifying the default use case to also call maven build wih --
* corrected maven build command
* corrected maven build command with %v
* skipping test runs
* testing for MTA project with single pom
* adding absolute path to m2 path
* clean up
* adding switch for mta and maven and removing env from containers
* commiting changes for new detect step
* correting log message
* code clean up
* unit tests changes to detectExecute
* basic tests for new change
* restoring piperGoUtils to download correct piper binary
* code clean up
* code clean up
* protecodeExecuteScan -> Added authentication with user API key
* protecodeExecuteScan -> updating .yml file
* protecodeExecuteScan -> go generate fixed
* protecodeExecuteScan -> naming convention applied for UserAPIKey parameter
* protecodeExecuteScan -> extending groovy code for mapping jenkins credentials
Co-authored-by: D072410 <giridhar.shenoy@sap.com>
Co-authored-by: Keshav <anil.keshav@sap.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* Replace '+' to '_' in appVersion
* Add test
* Fix test
* Update test name
Co-authored-by: Vyacheslav Starostin <vyacheslav.starostin@sap.com>
Co-authored-by: Anil Keshav <anil.keshav@sap.com>
* Initially generated tmsUpload<...> files
* First provisioning of parameters supported by tmsUpload step
* Refer to Go step from tmsUpload.groovy
* Initial client implementation
* Reverting line delimiters in tmsUpoad.groovy back to Unix ones
* Temporarily remove when-condition for Release stage
* Define useGoStep parameter in tmsUpload.groovy
* Unstash buildResult if useGoStep is true
* No unstashing and empty credentials, when using go step
* Register TmsUploadCommand in piper.go
* Cleanup groovy-related changes - they will be temporarily implemented in a different repo
* Make getting OAuth token success
* Look through the code and cleanup it a bit
* Read service key from Jenkins credentials store
* Provide initial set of unit tests for methods in /pkg/tms/tms.go file
* Minor improvements on logging response on http call error
* Check, if positive HTTP status code is as expected
* Cleanup tms.yaml file, provide additional unit test for tms.go
* Provide unit test for the case, when request body contains spaces
* Specify nodeExtDescriptorMapping parameter as of type map in tms.yaml
* Implement client method for getting nodes
* Write tests for GetNodes method
* Add GetMtaExtDescriptor client method and cover it with unit tests
* Provide first implementation for Update- and UploadMtaExtDescriptor
client methods
* Provide first implementation for Update- and UploadMtaExtDescriptor
client methods
* Provide UploadFile and UploadFileToNode client methods
* Provide tests for Update- and UploadMtaExtDescriptor client methods
* Write tests for FileUpload and FileUploadToNode client methods
* Minor corrections
* Remove some TODO comments
* Rename some of response structures
* Revert change for line delimiters in cmd/piper.go
* Add uploadType string parameter to UploadFile and UploadRequest methods
of uploader mock to reflect the changed Uploader implementation
* Start to implement execution logic in tmsUpload.go
* Changes in tms.yaml file
- remove resources from inputs in tms.yaml and implement mtaPath
parameter settings in the yaml file the same way, as it is done in
cloudFoundryDeploy.yaml
- rename tms.yaml to tmsUpload.yaml, since some generation policy
changed meanwhile
* Rename tms.yaml to tmsUpload.yaml and do go generate
* Use provided proxy on communication with UAA and TMS
* Set proxy even before getting OAuth token
* Further implementation of tmsUpload.go
* Continuation on implementing the tmsUpload.go executor
* Get mtarFilePath and git commitId from commonPipelineEnvironment, if
they are missing in configuration file + minor changes
* Implement a happy path test for tmsUpload logic
* Cover with unit tests main happy and error paths of tmsUpload.go logic
* Extend set of unit tests for tmsUpload.go
* Eliminate some TODOs, extend unit tests for tmsUpload.go
* Delete some TODOs
* Remove a couple of more TODOs from tms_test.go file
* Provide additional unit test for error due unexpected positive http
status code on upload
* Revert back line delimiters in cmd/piper.go
* Comment out file uploading calls in tmsUpload.go
* Run go generate to update generated files
* Convert line delimiters in tmsUpload.yaml to Unix ones, as well as
provide new line character in the end of the file, plus minor fix for
logging in tmsUpload.go file (pipeline complained)
* Correct description of a parameter in tmsUpload.yaml, extend unit tests
to check for trimming a slash in the end of TMS url for client methods
that do upload
* [minor] Add a comment in the test code
* Add stashContent parameter to do unstashing in tmsUpload.groovy, remove
some of the clarified TODOs
* Uncomment uploading file calls in tmsUpload.go, declare buildResult
stash in tmsUpload.yaml
* Remove clarified TODOs from the tmsUpload.go file
* Run go fmt for jenkins-library/pkg/tms
* Do not get explicitly values from common pipeline environment - all
configurations are provided in yaml file
* Remove unused struct from tmsUpload_test.go
* Run go fmt jenkins-library\pkg\tms
* Revise descriptions of parameters provided in tmsUpload.yaml file
* Specify STAGES scope for tmsUpload parameters
* Provide STAGES scope for the tmsUpload parameters, provide default value
for stashContent parameter
* Remove trailing space from tmsUpload.yaml
* Provide unit tests for proxy-related changes in http.go file
* Improve proxy implementation in tmsUpload.go file
* Make tmsServiceKey again a mandatory parameter
* Run go generate command to make the generated files correspond the yaml
state
* Change line delimiters back to Unix ones (were switched while resolving
the conflicts)
* Remove trailing spaces from tmsUpload.yaml
* Minor change in a comment to trigger pipelines with commit
* Improve checks for zero-structs and for empty maps, as well as use
different package to read files in the tests
* Revert line endings in http.go
* Revert comments formatting changes in files that do not belong to the tmsUpload step
with #3875 temp directory was created in current workspace.
This had negative side-effects: For example npm build packaged and published temporary files
Co-authored-by: Anil Keshav <anil.keshav@sap.com>
* WIP: Adapt bom names
* + WIP: Adapt bom filenames
* Upgrade cyclonedx gradle plugin and use cyclonedxBom config parameters
* Fix unit tests - use correct name in bom creation
* Fix pythonBuild bom name
* introduce and use npmBomFilename const
* Introduce and use mvnBomFilename const
* Introduce and use gradleBomFilename const
* Use build-tool names for bom suffix
* + Adapt tests (build tool suffix)
* Use BOM schema version 1.2 in gradleExecuteBuild
* Pin version of cyclonedx-maven-plugin to 2.7.1
* Adapt generated files
* Fix integration tests
* Fix integration tests
* Fix gradle build integration tests
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* Cleanup of SBOM generation parameters
Adding `false` does not what is intended. If the parameters are added to the call, license texts and dev dependencies are included
* Fixed unit test
* fix(fortify): suppressed issues got "Unknown" category and state
* fix (fortify-sarif): classify findings into audit group
* fix(fortify-checkmarx-sarif): common properties bag for Fortify and Checkmarx (accepting the risk of empty value)
* fix (checkmarx-sarif): classify findings into audit group
* fix (sarif): formatting
* feat(cpe): provide go templating functions
* change type
* fix: type in test
* chore: add comment for exported function
* fix: ensure that custom returns string properly
* fix types and add tests
Co-authored-by: Anil Keshav <anil.keshav@sap.com>
* Update scanPolling.go
Changing maxWaitTime from 15 to 30 to overcome WhiteSource results reflection in the backend issue.
* Update configHelper.go
* Reset configHelper changes to fix PR 3284
Committer: raghunathd8
* ignoreSourceFiles to fileSystemScan
* Added ignoreSourceFiles param also
* minor adjustment
* minor adjustment again
* updated unit test
* tests fixed
* fmt-ed
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: raghunathd8 <root@docker-evaluation.openstack.eu-nl-1.cloud.sap>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: ffeldmann <f.feldmann@sap.com>
This commit replaces `ioutil.TempDir` with `t.TempDir` in tests. The
directory created by `t.TempDir` is automatically removed when the test
and all its subtests complete.
Prior to this commit, temporary directory created using `ioutil.TempDir`
needs to be removed manually by calling `os.RemoveAll`, which is omitted
in some tests. The error handling boilerplate e.g.
defer func() {
if err := os.RemoveAll(dir); err != nil {
t.Fatal(err)
}
}
is also tedious, but `t.TempDir` handles this for us nicely.
Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* enable build without values
* add sap-client as option
* use function from /net/url to add parameters
Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com>
* Add ans implementation
* Remove todo comment
* Rename test function
Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>
* Better wording
Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>
* Add reading of response body function
* Use http pkg ReadResponseBody
* Check read error
* Better test case description
* Fix formatting
* Create own package for read response body
* Omit empty nested resource struct
* Separate Resource struct from Event struct
* Merge and unmarshall instead of only unmarshalling
* Improve status code error message
* Remove unchangeable event fields
* Separate event parts
* Change log level setter function
* Restructure ans send test
* Revert exporting readResponseBody function
Instead the code is duplicated in the xsuaa and ans package
* Add check correct ans setup request
* Add set options function for mocking
* Review fixes
* Correct function name
* Use strict unmarshalling
* Validate event
* Move functions
* Add documentation comments
* improve test
* Validate event
* Add logrus hook for ans
* Set defaults on new hook creation
* Fix log level on error
* Don't alter entry log level
* Set severity fatal on 'fatal error' log message
* Ensure that log entries don't affect each other
* Remove unnecessary correlationID
* Use file path instead of event template string
* Improve warning messages
* Add empty log message check
* Allow configuration from file and string
* Add sourceEventId to tags
* Change resourceType to Pipeline
* Use structured config approach
* Use new log level set function
* Check correct setup and return error
* Mock http requests
* Only send log level warning or higher
* Use new function name
* One-liner ifs
* Improve test name
* Fix tests
* Prevent double firing
* Reduce Fire test size
* Add error message to test
* Reduce newANSHook test size
* Further check error
* Rename to defaultEvent in hook struct
* Reduce ifs further
* Fix set error category test
The ansHook Fire test cannot run in parallel, as it would affect the
other tests that use the error category.
* Change function name to SetServiceKey
* Validate event
* Rename to eventTemplate in hook struct
* Move copy to event.go
* Fix function mix
* Remove unnecessary cleanup
* Remove parallel test
The translation fails now and again when parallel is on.
* Remove prefix test
* Remove unused copyEvent function
* Fix ifs
* Add docu comment
* Register ans hook from pkg
* register hook and setup event template seperately
* Exclusively read eventTemplate from environment
* setupEventTemplate tests
* adjust hook levels test
* sync tests- wlill still fail
* migrate TestANSHook_registerANSHook test
* fixes
* Introduce necessary parameters
* Setup hook test
* Use file instead
* Adapt helper for ans
* Generate go files
* Add ans config to general config
* Change generator
* Regenerate steps
* Allow hook config from user config
Merges with hook config from defaults
* Remove ans flags from root command
* Get environment variables
* Generate files
* Add test when calling merge twice
* Update generator
* Regenerate steps
* Check two location for ans service key env var
* Re-generate
* Fix if
* Generate files with fix
* Duplicate config struct
* Add type casting test for ans config
* Fix helper
* Fix format
* Fix type casting of config
* Revert "Allow hook config from user config"
This reverts commit 4864499a4c497998c9ffc3e157ef491be955e68e.
* Revert "Add test when calling merge twice"
This reverts commit b82320fd07b82f5a597c5071049d918bcf62de00.
* Add ans config tests
* Improve helper code
* Re-generate commands
* Fix helper unit tests
* Change to only one argument
* Fix helper tests
* Re-generate
* Revert piper and config changes
* Re-generate missing step
* Generate new steps
* [ANS] Add servicekey credential to environment (#3684)
* Add ANS credential
* Switch to hooks and remove comments
* Add subsection for ans
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
* Remove changes to piper.go
* Remove formatting
* Add test for ANS
* Define hook credential seperately from step credential
* Add test for retrieval from general section
* Add comment
* Get ans hook info from DefaultValueCache
* [ANS] Add documentation (#3704)
* Add ANS credential
* Switch to hooks and remove comments
* Add subsection for ans
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
* Remove changes to piper.go
* Remove formatting
* Add test for ANS
* Define hook credential seperately from step credential
* Add test for retrieval from general section
* Add comment
* Add documentation
* Review changes
* Review comments
* Improve documentation further
* Add note of two event templates
* Add log level destinction
* Further improvements
* Improve text
* Remove unused things
* Add ANS credential
* Switch to hooks and remove comments
* Add subsection for ans
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
* Remove changes to piper.go
* Remove formatting
* Add test for ANS
* Define hook credential seperately from step credential
* Add test for retrieval from general section
* Add comment
* Get ans hook info from DefaultValueCache
* Improvements
Co-authored-by: Linda Siebert <linda.siebert@sap.com>
Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
* New lines
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
Co-authored-by: Roland Stengel <r.stengel@sap.com>
Co-authored-by: Thorsten Duda <thorsten.duda@sap.com>
* Add ans implementation
* Remove todo comment
* Rename test function
Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>
* Better wording
Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>
* Add reading of response body function
* Use http pkg ReadResponseBody
* Check read error
* Better test case description
* Fix formatting
* Create own package for read response body
* Omit empty nested resource struct
* Separate Resource struct from Event struct
* Merge and unmarshall instead of only unmarshalling
* Improve status code error message
* Remove unchangeable event fields
* Separate event parts
* Change log level setter function
* Restructure ans send test
* Revert exporting readResponseBody function
Instead the code is duplicated in the xsuaa and ans package
* Add check correct ans setup request
* Add set options function for mocking
* Review fixes
* Correct function name
* Use strict unmarshalling
* Validate event
* Move functions
* Add documentation comments
* improve test
* Validate event
* Add logrus hook for ans
* Set defaults on new hook creation
* Fix log level on error
* Don't alter entry log level
* Set severity fatal on 'fatal error' log message
* Ensure that log entries don't affect each other
* Remove unnecessary correlationID
* Use file path instead of event template string
* Improve warning messages
* Add empty log message check
* Allow configuration from file and string
* Add sourceEventId to tags
* Change resourceType to Pipeline
* Use structured config approach
* Use new log level set function
* Check correct setup and return error
* Mock http requests
* Only send log level warning or higher
* Use new function name
* One-liner ifs
* Improve test name
* Fix tests
* Prevent double firing
* Reduce Fire test size
* Add error message to test
* Reduce newANSHook test size
* Further check error
* Rename to defaultEvent in hook struct
* Reduce ifs further
* Fix set error category test
The ansHook Fire test cannot run in parallel, as it would affect the
other tests that use the error category.
* Change function name to SetServiceKey
* Validate event
* Rename to eventTemplate in hook struct
* Move copy to event.go
* Fix function mix
* Remove unnecessary cleanup
* Remove parallel test
The translation fails now and again when parallel is on.
* Remove prefix test
* Remove unused copyEvent function
* Fix ifs
* Add docu comment
* Register ans hook from pkg
* register hook and setup event template seperately
* Exclusively read eventTemplate from environment
* setupEventTemplate tests
* adjust hook levels test
* sync tests- wlill still fail
* migrate TestANSHook_registerANSHook test
* fixes
* review - cleanup, reuse poke
* Apply suggestions from code review
* Change subject
* Review fixes
* Set stepName 'n/a' if not available
* Fix fire tests
Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>
Co-authored-by: Roland Stengel <r.stengel@sap.com>
hopefully that gives users a direct link back to the original fortify project+version
Co-authored-by: xgoffin <86716549+xgoffin@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* feat(fortifyExecuteScan): add a max number of retries for API calls in SARIF conversion
* feat(checkmarxExecuteScan): implement max number of retries on API call for descriptions in SARIF processing
* feat(checkmarx/fortify): extra logging line when failing an API request in SARIF conversion
* fix(fortifyExecuteScan): panic if undefined projectversion in sarif
* fix(fortifyExecuteScan): logging improvement
* fix(fortifyExecuteScan): wrong if condition caused crash
* fix(fortifyExecuteScan): do not log if retries hit -1, adjust logging
* fix(SARIF): commenting API calls for Checkmarx until a solution can be found for the API issues
* feat(SARIF): add omitempty to extensions
* Improvements were made
* fixed tests
* fixed issues
* fix versioning
* fix Inclusive Language warnings
* gradle support to fortifyExecuteScan. Classpath resolving
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* Add ans implementation
* Remove todo comment
* Rename test function
Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>
* Better wording
Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>
* Add reading of response body function
* Use http pkg ReadResponseBody
* Check read error
* Better test case description
* Fix formatting
* Create own package for read response body
* Omit empty nested resource struct
* Separate Resource struct from Event struct
* Merge and unmarshall instead of only unmarshalling
* Improve status code error message
* Remove unchangeable event fields
* Separate event parts
* Change log level setter function
* Restructure ans send test
* Revert exporting readResponseBody function
Instead the code is duplicated in the xsuaa and ans package
* Add check correct ans setup request
* Add set options function for mocking
* Review fixes
* Correct function name
* Use strict unmarshalling
* Validate event
* Move functions
* Add documentation comments
* improve test
Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>
Co-authored-by: Roland Stengel <r.stengel@sap.com>
* feat(fortfiyExecuteScan): proper XML unescaping, added rulepacks to SARIF, added kingdom/type/subtype to tags
* feat(fortifyExecuteScan): proper handling of severity, kinds, levels in SARIF
* fix(fortifyExecuteScan): edge case when handling properties taht could lead to a crash
* fix(fortifyExecuteScan): ensure SARIF processing is done after latest FPR is processed by SSC
* feat(checkmarxExecuteScan): respect SARIF standard more closely
* fix(checkmarxExecuteScan): edge case where message would be empty in SARIF
* fix(checkmarxExecuteScan): better message handling to ensure field is populated
* feat(checkmarxExecuteScan): SARIF file readability
* feat(checkmarxExecuteScan): include the helpURL as part of the Help object
* fix(sarif): remove wrong structure addition
* feat(checkmarxExecuteScan): safer handling of version in SARIF file
* feat(checkmarxExecuteScan): add CWE number to tags
* fix(helmExecute): respect version from Chart
using version from CPE can create failure situations in case format is not semver.
This is the case for maven artifacts, for example.
* chore: simplify condition
* chore: cleanup
* chore: cleanup
* explicitly adding tar extension to project name when constructing the targetFilePath for whitesource docker image download
* comments
* correcting comment for better readability
* replace spaces in the project name with underscroe
* better comments
* passing legacy format download
* appending format to value
* keeping the download format for protecode as legacy
* improving docu
* keeping legacy format the default
* keeping tar file name same as project name to avoid duplicate names
* keeping legacy format download hard coded
Co-authored-by: anilkeshav27 <you@example.com>
* feat(fpr_to_sarif & GHAS): adjustments to fit some rules
* feat(fortifyExecuteScan): fit GH ingestion rules better
* feat(fortifyExecuteScan): readability in SARIF report
* feat(fortifyExecuteScan): restore escaped chars in XML text
* feat(fortifyExecuteScan): properly replace threadflowlocations in each threadflow
* fix(fortifyExecuteScan): fixed missing threadflow in SARIF generation
* feat(fortifyExecuteScan): properly handle threadflows when a node has another node as Reason (node-in-node edge case)
* feat(fortifyExecuteScan): better sarif ruleID field
Co-authored-by: thtri <trinhthanhhai@gmail.com>
* including a artifact cpe type
* removing type kind related to PR 3717
* clean up
* eliminating local path
* go formatting fix
Co-authored-by: anilkeshav27 <you@example.com>
