* WIP: Adapt bom names
* + WIP: Adapt bom filenames
* Upgrade cyclonedx gradle plugin and use cyclonedxBom config parameters
* Fix unit tests - use correct name in bom creation
* Fix pythonBuild bom name
* introduce and use npmBomFilename const
* Introduce and use mvnBomFilename const
* Introduce and use gradleBomFilename const
* Use build-tool names for bom suffix
* + Adapt tests (build tool suffix)
* Use BOM schema version 1.2 in gradleExecuteBuild
* Pin version of cyclonedx-maven-plugin to 2.7.1
* Adapt generated files
* Fix integration tests
* Fix integration tests
* Fix gradle build integration tests
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* Add ans implementation
* Remove todo comment
* Rename test function
Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>
* Better wording
Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>
* Add reading of response body function
* Use http pkg ReadResponseBody
* Check read error
* Better test case description
* Fix formatting
* Create own package for read response body
* Omit empty nested resource struct
* Separate Resource struct from Event struct
* Merge and unmarshall instead of only unmarshalling
* Improve status code error message
* Remove unchangeable event fields
* Separate event parts
* Change log level setter function
* Restructure ans send test
* Revert exporting readResponseBody function
Instead the code is duplicated in the xsuaa and ans package
* Add check correct ans setup request
* Add set options function for mocking
* Review fixes
* Correct function name
* Use strict unmarshalling
* Validate event
* Move functions
* Add documentation comments
* improve test
* Validate event
* Add logrus hook for ans
* Set defaults on new hook creation
* Fix log level on error
* Don't alter entry log level
* Set severity fatal on 'fatal error' log message
* Ensure that log entries don't affect each other
* Remove unnecessary correlationID
* Use file path instead of event template string
* Improve warning messages
* Add empty log message check
* Allow configuration from file and string
* Add sourceEventId to tags
* Change resourceType to Pipeline
* Use structured config approach
* Use new log level set function
* Check correct setup and return error
* Mock http requests
* Only send log level warning or higher
* Use new function name
* One-liner ifs
* Improve test name
* Fix tests
* Prevent double firing
* Reduce Fire test size
* Add error message to test
* Reduce newANSHook test size
* Further check error
* Rename to defaultEvent in hook struct
* Reduce ifs further
* Fix set error category test
The ansHook Fire test cannot run in parallel, as it would affect the
other tests that use the error category.
* Change function name to SetServiceKey
* Validate event
* Rename to eventTemplate in hook struct
* Move copy to event.go
* Fix function mix
* Remove unnecessary cleanup
* Remove parallel test
The translation fails now and again when parallel is on.
* Remove prefix test
* Remove unused copyEvent function
* Fix ifs
* Add docu comment
* Register ans hook from pkg
* register hook and setup event template seperately
* Exclusively read eventTemplate from environment
* setupEventTemplate tests
* adjust hook levels test
* sync tests- wlill still fail
* migrate TestANSHook_registerANSHook test
* fixes
* Introduce necessary parameters
* Setup hook test
* Use file instead
* Adapt helper for ans
* Generate go files
* Add ans config to general config
* Change generator
* Regenerate steps
* Allow hook config from user config
Merges with hook config from defaults
* Remove ans flags from root command
* Get environment variables
* Generate files
* Add test when calling merge twice
* Update generator
* Regenerate steps
* Check two location for ans service key env var
* Re-generate
* Fix if
* Generate files with fix
* Duplicate config struct
* Add type casting test for ans config
* Fix helper
* Fix format
* Fix type casting of config
* Revert "Allow hook config from user config"
This reverts commit 4864499a4c497998c9ffc3e157ef491be955e68e.
* Revert "Add test when calling merge twice"
This reverts commit b82320fd07b82f5a597c5071049d918bcf62de00.
* Add ans config tests
* Improve helper code
* Re-generate commands
* Fix helper unit tests
* Change to only one argument
* Fix helper tests
* Re-generate
* Revert piper and config changes
* Re-generate missing step
* Generate new steps
* [ANS] Add servicekey credential to environment (#3684)
* Add ANS credential
* Switch to hooks and remove comments
* Add subsection for ans
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
* Remove changes to piper.go
* Remove formatting
* Add test for ANS
* Define hook credential seperately from step credential
* Add test for retrieval from general section
* Add comment
* Get ans hook info from DefaultValueCache
* [ANS] Add documentation (#3704)
* Add ANS credential
* Switch to hooks and remove comments
* Add subsection for ans
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
* Remove changes to piper.go
* Remove formatting
* Add test for ANS
* Define hook credential seperately from step credential
* Add test for retrieval from general section
* Add comment
* Add documentation
* Review changes
* Review comments
* Improve documentation further
* Add note of two event templates
* Add log level destinction
* Further improvements
* Improve text
* Remove unused things
* Add ANS credential
* Switch to hooks and remove comments
* Add subsection for ans
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
* Remove changes to piper.go
* Remove formatting
* Add test for ANS
* Define hook credential seperately from step credential
* Add test for retrieval from general section
* Add comment
* Get ans hook info from DefaultValueCache
* Improvements
Co-authored-by: Linda Siebert <linda.siebert@sap.com>
Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
* New lines
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
Co-authored-by: Roland Stengel <r.stengel@sap.com>
Co-authored-by: Thorsten Duda <thorsten.duda@sap.com>
* adds return in gcs upload in case error occurs e.g. no credentials, avoid nil pointer dereference
* Adds generated files
* Updates generated files
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* build depdency docu for maven and npm
* removing trailing white space in yaml
* Update resources/metadata/mavenBuild.yaml
* relative url for vault and mta docu
* running go generate
* keeping vault relative path
* go generate
* reverting to global paths
* go generate
* wild card for a higher level dir
* searching on top folder only
* relative level above
* pointing to infrastructure
* correcting links
Co-authored-by: anilkeshav27 <you@example.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* Add gcs upload to mavenBuild step
* Patterns were updated
* go generate
* removed patterns related to sapGenerateEnvironmentInfo step
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* fixed generated output parameters for influx
* change name to lower case
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
* Adds GetLog() function to orchestrator
* Fixes BUILD_NUMBER env variable
* Fixes correct env var for JENKINS_HOME
* Adds getEnv to read env variables with default value, adds test for jenkins GetLog() implementation
* Adds possibility to read errorJsons; updates splunk package for log files (WIP)
* Uncommenting dev code
* Adds GetLog() function to orchestrator
* Fixes BUILD_NUMBER env variable
* Fixes correct env var for JENKINS_HOME
* Adds getEnv to read env variables with default value, adds test for jenkins GetLog() implementation
* Adds possibility to read errorJsons; updates splunk package for log files (WIP)
* Uncommenting dev code
* Adds GetRequest function which holds the response in memory (not saved to disk)
* Implements GetLog() function for ADO, adds function to read PipelineRuntime
* PAT has been revoked
* Changes http package, s.t. if password only is required basic auth works too
* Adds env variable for azure token, error handling in case of unauthenticated/nil response
* Adds logging output in case env variable can not be read and fallback variable needs to be used
* Adds usage of environment variables for auth, uses jenkins api
* Adds init functionality for orchestrators, updates GetLog() and GetPipelineStartTime() function
* Adds initaliziation function for orchestrator authetnication
* Adds settings struct for orchestrator authentication
* Adds function to whole logfile to Splunk
* Struct for pipeline related telemetry information
* Increase messagebatch size to 10k
* Changes splunk package to a pointer based implementation, updates generated files and corresponding template and tests for splunk
* Changes telemetry package to pointer based implementation to have multiple telemetry objects, adjusted tests and splunk implementation
* Changes content type to txt
* Send telemetry independent of logfiles, increases amount of messages per file
* Adds JobURL for orchestrators and UnknownOrchestrator as fallback
* telemetry makes use of orchestrator specific information
* Adds orchestrator independent correlationID
* Adds custom fields for pipeline status
* go fmt
* Removes env var test - no env variables are read anymore
* Use UnknownOrchestratorConfigProvider in case the orchestrator can not be initalized
* Removes Custom fields from telemetry as these can not be reflected in SWA
* Adds custom telemetry information (piperHash,..) to each step telemetry information
* Removes falltrough in case no orchestrator has been found
* Updates tests for orchestrator package
* Adds orchestrator import in generated files
* Updates generator files for internal library
* Adds orchestrator telemetry information to steps
* Updates generated files, fatalHook writes to cpe
* Go generate from master, go fmt
* Adds Custom Data field LastErrorCode
* Removes GetLog() test
* Update init_unix.go
* Update docker_integration_test_executor.go
* Update integration_api_cli_test.go
* Reverts go1.17 fmt formatting
* Reverts go1.17 fmt formatting
* Reverts go1.17 fmt formatting
* Renames customTelemetryData to stepTelemetryData
* Adjustments to orchestrator-package, cleanup, adds JobName
* Adjusts commonPipelineEnvironment path
* Adds pipelineTelemetry struct to telemetry package, removes pipeline telemetry structs from splunk package
* Go fmt
* Changes path for errorDetails, adds debug information
* Removes custom fields from step, adds orchestrator, commithash to baseMetadata
* Adjusts tests for telemetry package
* Adds tests for orchestrator
* Updates generated files, initalization of splunk client only if its available in the config
* Fixes typo in helper go
* Update pkg/http/downloader.go
* Update pkg/http/downloader.go
* Update pkg/log/fatalHook.go
* Update fatalHook.go
* Update pkg/splunk/splunk.go
* Update pkg/telemetry/data.go
* Adds GetBuildStatus() and GetAPIInformation() to orchestrators
* error formatting
* Bugfix: dont send telemetry data if disabled, adjusts test
* go fmt
* Use correct error handling
* Update pkg/telemetry/telemetry.go
* Fixes telemetry disabled in the tests
* Fixes http tests
* Log fatal errors to logFile
* Adds CustomReportingConfig to hooks
* Cleanup comments in splunk package
* Adds possibility to send telemetry to custom endpoint
* Adds debug output for the payload
* Debug output for the payload as a string
* Adds test cases for changes in telemetry package
* go fmt
* Adds generated files for new step
* Reverts changes for http tests, causing problems with go1.15, changes need to be applied for newer go version >=1.17
* Adjusts test for sonarExecuteScan
* Adjusts test for sonarExecuteScan
* Adds explanation for customreportingConfig
* Makes disableing of customSend more obvious
* Adds custom step reporting to each step, updates generated files, adjusts helper testdata
* fixes unit test wrong usage of logging
* Send pipeline data altough there has been no error, adjust test cases
* Reverts changes for customReporting
* Updates generated files, removes customReporting
* Removes writing errorDetails to CPE
* Reverts usage of customreporting
* go fmt
* reverts changes in http_test
* reverts changes in http_test
* Skips integration cnb test
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* create build settings for maven
* cases for when mavenBuild may be present
* fixing unit test for mavenBuild to include cpe
* changing position of buildSettngsJson to be called atfter build runs
* package
* extending the struct for other build types
* adding values for mta build settings
* changing config data type
* adding npm build settings
* unit tests
* fix trailing space
* typo correction in yaml
* Vitalii/build settings info (#3277)
* Add buildsettings package
* Improve buildSetting package for mta, npm
* Add unit-test
* Fix
* Fix
Co-authored-by: Vitalii Sidorov <vitalii.sidorov@sap.com>
* review changes
* removing buildTool param
* changing npm script name
* fix npmExecute tests
* including build settings info in npm struct
Co-authored-by: Your Name <you@example.com>
Co-authored-by: kingvvgo <56587879+kingvvgo@users.noreply.github.com>
Co-authored-by: Vitalii Sidorov <vitalii.sidorov@sap.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* Implemented validation for the option possibleValues
* Has been added the option mandatoryIf to config with validation
* Fixed issues found during code review
* improved golang template
* Fixed tests. Added validation for mandatoryIf option
* Fix typo
* Fixed tests
* Validation was refactored. Added options
* Added default value for parameters with possibleValues option
* Validation was moved after the configuration resolution
* Canceled some default values
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* using default java truststore
* default java keytore
* remove trust store
* working directory
* change dir java_home
* env variable
* trying to find jre home
* changing directory to jre home and java home
* trying java jre
* remving $
* trying to search the cacerts
* copying existing cacerts
* removing change directory
* searching for secrutiy folder only
* searching cacerts
* new path for cacert
* path to ca-cert
* new trust store
* changing cacert location
* only adding maven_opts env variable once
* log message
* ca cert path from user
* handelling interface modification
* enhance logs and code clean up
* code clean up
Co-authored-by: Your Name <you@example.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* feat(config):read config/defaults with authentication
This change allows to use defaults and config files from a protected GitHub repository.
The options `--customConfig` and `--defaultConfig` already allowed to provide a link to an uprotected file.
Now, by passing a value in the form `<hostname>:<token>` to parameter `gitHubTokens` (this parameter can be passed multiple times) a token can be provided for dedicated hosts.
This makes it possible to use a link like
`https://api.github.com/repos/SAP/jenkins-library/contents/resources/my-defaults.yml?ref=master`
as reference to a default file or similarly as reference to a configuration file.
* update generation to allow protected config/defaults
* fix CodeClimate issues
* update missing generations
* modifying detect.maven.excluded.scopes from TEST to test
* new maven alt deployment flags
* changing flag names
* tlsCertificate addtion
* adding publish flags
* new flags
* publish flag
* enhance maven builds
* enhance maven builds
* creating new settings xml
* updating project settings
* changing interface for artifactPreparation that uses the same maven util niterface
* adding general scope to maven params
* global reference
* removing vault tmp
* debuging deployment user
* more debug
* maven build paras
* using smaller case
* adding incorrect error check
* adding deployment flags
* code refactor
* unit tests
* changing scope of paramter for tls certs
* new scope for tls
* remove trailing space in mavenBuild.yaml
* trailing space fix
* typo fix and jenkins secret
* including jenkins credentials for repo pass in the maven build groovy
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* add vaultSecretFileReferences
* fix test
* fix test
* go generate
* remove code duplication
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
* Add error category parsing to cmd execution
It is now possible to define `ErrorCategoryMapping` as a `map[string][]string` on a `Command`.
The format contains the category as key which has a list of error patterns assigned.
Example:
```
cmd := Command{
ErrorCategoryMapping: map[string][]string
"build": {"build failed"},
"compliance": {"vulnerabilities found", "outdated components found"},
"test": {"some tests failed"},
},
}
```
Setting this map triggers console log parsing when executing a command.
If a match is found the error category is stored and
it will automatically be added to the `errorDetails.json`.
* clean up go.mod
* fix test
* fix test
* Update DEVELOPMENT.md
* fix tests
* address long console content without line breaks
* scan condition update
* fix test
* add missing comment for exported function
* Update pkg/command/command.go
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Do not exit with os.Exit(1) but using log.Entry().Fatal() instead
* Golang: forward error details
* extend groovy wrapper to provide proper error message
* create closure for error handling
* Golang step metadata: Config aliases for steps
This will ease following scenarios:
* config migration due to step name changes
* re-use of more general config, e.g. `mavenExecute` in `mavenBuild`
* fix CodeClimate finding
* Fix panic if original stage config does not exist yet
