Sven Merk
1ddd966249
Enforce non-incremental scans when optimized and scheduled ( #3039 )
...
* Enfore non-incremental scans when optimized
* Update resources/metadata/checkmarx.yaml
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* Update generated file
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-08-10 11:27:28 +02:00
Oliver Nocon
805a8fd88f
feat(config): read config/defaults with authentication ( #2975 )
...
* feat(config):read config/defaults with authentication
This change allows to use defaults and config files from a protected GitHub repository.
The options `--customConfig` and `--defaultConfig` already allowed to provide a link to an uprotected file.
Now, by passing a value in the form `<hostname>:<token>` to parameter `gitHubTokens` (this parameter can be passed multiple times) a token can be provided for dedicated hosts.
This makes it possible to use a link like
`https://api.github.com/repos/SAP/jenkins-library/contents/resources/my-defaults.yml?ref=master `
as reference to a default file or similarly as reference to a configuration file.
* update generation to allow protected config/defaults
* fix CodeClimate issues
* update missing generations
2021-07-08 15:26:07 +02:00
Oliver Nocon
0b48bfcc73
feat: retrieve metadata by stepName - corrected ( #2892 )
...
* refactored getConfig to allow stepName param for metadata fetching
* extended step generator
* go generate
* Update cmd/getConfig.go
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
* Update cmd/getConfig.go
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
* Update cmd/getConfig.go
* update generated files
* update golden files to care for generator update
* update and add tests
* update generated files
* Update cmd/getConfig.go
* Update cmd/getConfig.go
* update/fix formatting
* feat: retrieve metadata by stepName - corrected
* update generation
* update condition logic for defaults
* update generation & tests
* support multiple conditions
* update generation
* Add generated
Co-authored-by: Leander Schulz <leander.schulz01@sap.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Sven Merk <s.merk@sap.com>
2021-06-16 08:43:30 +02:00
Oliver Nocon
4250ca8bed
Revert "feat(getConfig): retrieve metadata by stepName ( #2736 )" ( #2891 )
...
This reverts commit ae4a24c594
2021-06-14 10:57:44 +02:00
lndrschlz
ae4a24c594
feat(getConfig): retrieve metadata by stepName ( #2736 )
...
* refactored getConfig to allow stepName param for metadata fetching
* extended step generator
* go generate
* Update cmd/getConfig.go
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
* Update cmd/getConfig.go
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
* Update cmd/getConfig.go
* update generated files
* update golden files to care for generator update
* update and add tests
* update generated files
* Update cmd/getConfig.go
* Update cmd/getConfig.go
* update/fix formatting
Co-authored-by: Leander Schulz <leander.schulz01@sap.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: OliverNocon <oliver.nocon@sap.com>
2021-06-14 08:58:41 +02:00
ffeldmann
b88ebdad6c
feat(splunk) Sending telemetry and logging information to Splunk ( #2796 )
...
* Adds inital splunk hook and logCollector
* Adds documentation of the Splunk hook
* Fixes markdown lint issues and removes comment from telemetry.go file
* Fixes markdown lint issues and adds missing generated file
* Markdown linting
* Changes documentation according to review, adds Splunk token automatically during init
* Adds error handling for marshalling hook config
* Markdown lint und correct Splunk token in httpclient
* Registeres Splunk token as secret and adjusts test cases
* Adds missing error handling and removes unnecessary comments
* Creates new function readPipelineEnvironment, adds tests
* Moves MonitoringData struct, edits defaults for json fields
* Adds gitRepository and gitOwner to telemetry information
* Simplifies readCommonPipelineEnvironment function, adds more descriptive errors, adds automated adding of Splunk prefix token
* Adjusts error handling
* Cleaner error logging
2021-05-17 12:14:04 +02:00
Sven Merk
d52a1a3619
Influx step execution reporting ( #2700 )
...
* Influx step execution reporting
* influx for newmanExecute added
Co-authored-by: lndrschlz <leander.schulz01@sap.com>
2021-03-18 10:32:03 +01:00
Christopher Fenner
f999925788
fix(influx): correct data type of influx measurements ( #2171 )
...
* update data type of influx measurements
* Update checkmarx.yaml
* pick changes from #1885 for testing
* update generated code
* update to new datatype
* adjust to type changes
* change back to string type
* Update fortifyExecuteScan.go
* add typo to be backward compatible
* change type to int for files_scanned and lines_of_code_scanned
* add typo
* add measurements to whitesource
* update generated sources
* adjust test cases
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-03-10 16:00:53 +01:00
lndrschlz
4ca9186f39
fix(uiveri5ExecuteTests): add 'tests' stash to step yaml ( #2641 )
...
* add tests stash to uiveri5 step
* add stash to uiveri5
* extend step generator with input resources
* add step generator test
2021-03-01 13:03:42 +01:00
Sven Merk
5d1782aa01
checkmarxExecuteScan: adapt to 9.2 api ( #2363 )
...
* Update checkmarxExecuteScan.go
* api mods
* Switch default
* Fix decode
* mod marshalling
* Fix unmarshalling
* Code fmt and small fix
* Optimize preset handling
* Integer handling
* Fix test
* cleanup
* go fmt
* Improve test
2020-11-25 13:47:26 +01:00
lndrschlz
b9bab27833
feat: expose complete step metadata through generated function ( #2329 )
...
* exposing step metadata through generator
* add metadata_generated.go
* fix step go test generation
* metadata fields added to generated files
* added generated files
* removed image placeholder from fortify step
* refactored step meta generation
* go generate
* fixed metadata generator and tests
* added output resource fields/tags to metadata generator
* fix string in metadata_generated
* go generate
* fixed generator
* go generate
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-11-20 08:13:59 +01:00
Kevin Stiehl
9cb605ba60
[Documentation ]fix checkmarx docs ( #2372 )
...
* fix checkmarx docs
* go generate
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-11-16 13:03:31 +01:00
Sven Merk
3c7712f2ee
Retry capabilities for HTTP requests + enablement for Checkmarx step ( #2346 )
2020-11-11 13:35:53 +01:00
Kevin Stiehl
24aafb0b69
add vaultSecretFileReferences ( #2203 )
...
* add vaultSecretFileReferences
* fix test
* fix test
* go generate
* remove code duplication
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-10-26 14:20:04 +01:00
Christopher Fenner
b8d3a7d1a9
fix(influx): correct project_name field name ( #2195 )
...
* Update checkmarx.yaml
* regenerate
* adjust code
2020-10-19 13:09:17 +02:00
Kevin Stiehl
3eae0c5f68
feat(vault): fetch secrets from vault ( #2032 )
...
* cloud-foundry & sonar from vault
* add vault development hint
* don't abort on vault errors
* cloudfoundry make credentialsId only mandatory when vault is not configured
* add vault ref to step ymls
* rename vaultAddress to vaultServerUrl
* rename PIPER_vaultRole* to PIPER_vaultAppRole*
* add resourceRef for detect step
* fix error when no namespace is set
* added debug logs
* added debug logs
* fix vault resolving
* add vaultCustomBasePath
* rename vault_test.go to client_test.go
* refactored vault logging
* refactored config param lookup for vault
* added tüddelchen
* rename vaultCustomBasePath to vaultPath
* fix tests
* change lookup path for group secrets
* fix interpolation tests
* added vault resource ref to versioning
* execute go generate
* rename Approle to AppRole
* change verbose back to false
Co-authored-by: Leander Schulz <leander.schulz01@sap.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-10-13 14:14:47 +02:00
Christopher Fenner
be90876b7c
feat(output): handle non-string output values ( #2113 )
...
* handle non-string values as JSON
* change value type to interface in resources
* regenerate code
* add test cases
* handle reading of json files
* write json data to json files
* fix assignment
* use GetResourceParameter
* add test case
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-10-05 15:33:28 +02:00
Oliver Nocon
19c1732826
Telemetry: report error category ( #2085 )
2020-09-29 13:49:40 +02:00
Christopher Fenner
b219fb6514
fix(typo): found by misspell ( #2064 )
...
* fix typos in step yamls
* fix typos in go files
* regenerate step code
* fix typos in md files
* fix typos in groovy files
* fix further typos
2020-09-24 07:41:06 +02:00
Sven Merk
612d3a645b
Support verify only mode for SAST tools ( #2018 )
...
* Support verify only mode for SAST
* Include feedback
* Add tests
* Fix imports
2020-09-18 08:19:34 +02:00
Kevin Stiehl
d589038206
Vault AppRole login ( #1971 )
...
* added interpolation package in config
* vault allow paths to hold config references
* allow referencing properties in vaultPaths
* fix small typo
* add approleAuth
* register resolved secrets to logger
* generate steps
* clean up
* add integration test
* add vault to context filter
* reduce Cognitive Complexity & added tests
* Update pkg/config/stepmeta_test.go
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* go generate
* go generate after merge
* rename VaultAppRole* to VaultRole*
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-09-16 14:50:09 +02:00
Stephan Aßmus
c54a55287a
checkmarxExecuteScan: Add two (deprecated) aliases ( #1883 )
2020-08-05 14:15:24 +02:00
lndrschlz
42f91f6ff1
checkmarxExecuteScan: added resourceRef for secrets in step yaml ( #1767 )
...
* checkmarxExecuteScan: added resourceRef for secrets in step yaml
* regenerate stepmeta
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-07-20 15:44:24 +02:00
Oliver Nocon
eafe383d54
Add error category parsing to cmd execution ( #1703 )
...
* Add error category parsing to cmd execution
It is now possible to define `ErrorCategoryMapping` as a `map[string][]string` on a `Command`.
The format contains the category as key which has a list of error patterns assigned.
Example:
```
cmd := Command{
ErrorCategoryMapping: map[string][]string
"build": {"build failed"},
"compliance": {"vulnerabilities found", "outdated components found"},
"test": {"some tests failed"},
},
}
```
Setting this map triggers console log parsing when executing a command.
If a match is found the error category is stored and
it will automatically be added to the `errorDetails.json`.
* clean up go.mod
* fix test
* fix test
* Update DEVELOPMENT.md
* fix tests
* address long console content without line breaks
* scan condition update
* fix test
* add missing comment for exported function
* Update pkg/command/command.go
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-06-24 10:04:05 +02:00
Christopher Fenner
c42553593e
fix: remove unused variables from generated step coding ( #1698 )
...
* remove unused parameters from generated code
* update steps
* update steps
* correct golden files
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-06-23 18:05:21 +02:00
Sven Merk
af2a01c064
Fortify implementation in golang ( #1428 )
2020-05-25 19:48:59 +02:00
Oliver Nocon
7f7d0bce89
Piper steps: Issue SUCCESS message ( #1478 )
...
* Piper steps: Issue SUCCESS message
Log `SUCCESS` message in all steps by default if no error occured
* Update additional generated steps
* Update generated step
2020-05-18 21:52:54 +02:00
lndrschlz
39e30ff1b4
Added hook configuration; added sentry hook and tests ( #1497 )
...
* Added hook configuration; added sentry hook and tests
* steps generated by step framework
* added comments and renamed correlationID
* formatted comment on NewSentryHook method
* fix generator step tests
2020-05-05 08:36:24 +02:00
Oliver Nocon
43947e6ef5
Pass golang error details to Jenkins pipeline ( #1443 )
...
Do not exit with os.Exit(1) but using log.Entry().Fatal() instead
* Golang: forward error details
* extend groovy wrapper to provide proper error message
* create closure for error handling
2020-04-28 07:42:02 +02:00
Oliver Nocon
75a10742d3
Step generator: correct error handling ( #1429 )
...
Do not exit with os.Exit(1) but using log.Entry().Fatal() instead
2020-04-20 16:52:57 +02:00
Daniel Kurzynski
ce4be51362
Mask secrets ( #1382 )
2020-04-16 14:37:45 +02:00
Oliver Nocon
8fbeddb26c
Golang step metadata: Config aliases for steps ( #1293 )
...
* Golang step metadata: Config aliases for steps
This will ease following scenarios:
* config migration due to step name changes
* re-use of more general config, e.g. `mavenExecute` in `mavenBuild`
* fix CodeClimate finding
* Fix panic if original stage config does not exist yet
2020-03-19 17:24:35 +01:00
Florian Geckeler
48fa0a7521
Add 'Code generated by' header to stepGoTemplate
2020-02-21 12:56:09 +01:00
Oliver Nocon
9c1bd04752
Streamline step generation ( #1142 )
...
* Streamline step generation
* Include PR feedback, update DEVELOPMENT.md
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2020-02-04 10:46:43 +01:00
Oliver Nocon
b542d15998
Enhance telemetry reporting ( #1133 )
...
* Enhance telemetry reporting
* Use central telemetry data object
* Add duration
Co-Authored-By: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-01-31 14:06:08 +01:00
Christopher Fenner
aa3fb8adb4
feat(go): add telemetry reporting ( #1100 )
...
* Add telemetry support
* First round telemetry
* Add telemetry flag
* fix: move files to avoid import cycles
* add noTelemetry as global config option
* Respect telemetry configuration for reporting
* add site id, swa endpoint
* correct logger initialization
* add http logic
* rename init method
* rename consts & types
* convert struct to payload
* convert data to payload string
* move activation flag out of data structure
* extract types to own file
* build query using net/url
* correct field mapping
* extract notify coding to own file
* cleanup parameter mapping
* preare base data
* fix codeclimate issue
* correct test case
* fill values from env
* test all fields
* untrack notify.go
* ignore empty custom values
* cleanup data.go
* add test cases
* cleanup
* add usage reporting to karma step
* add usage reporting to step generator
* externalise siteID
* correct custom field names
* test env handling
* simplify method signature
* revert parameter negation
* correct import
* adjust golden file
* inclease log level
* ignore test case
* Revert "inclease log level"
This reverts commit 70cae0e029
2020-01-29 13:17:54 +01:00
Sven Merk
cbe368fe36
Checkmarx as golang ( #1075 )
...
* Added base functionality for checkmarx interaction
* Extend http client with file upload capabilities
* Latest changes
* Add debug logging
* Introduce Uploader interface
* Add tests for checkmarx client
* Hook new checkmarx command
* Improve coverage
* Add tests
* Improved test coverage and fixed code
* Add influx reporting
* Add alternation capabilities
* Add groovy step
* Try fix cmd
* Enhancements
* Fix report generation
* Final performance improvements
* Fix code
* Structure code, cleanup
* Improvements
* Fix codeclimate issue
* Update groovy
* Adapt latest changes to http
* Fix test
* Fix http tests
* Fix test
* Fix test
* Fix test 2
* Fix code
* Fix code 2
* Fix code
* Code
* Fix
* Fix
* Add report and link handling
* Fix returns, add groovy test
* Review comments
* Added doc template
* Docs update
* Remove SAP internals
* Better status display
* Add name to link
* Fix test
* Fix
* Fix verbose handling
* Fix verbose handling 2
* Fix verbose handling 3
* Fix
* Tiny improvements
* Regenerate
* Fix test
* Fix test code
* Fix verbosity issue
* Fix test
* Fix test
* Fix test
2020-01-27 23:40:53 +01:00
