1
0
mirror of https://github.com/SAP/jenkins-library.git synced 2024-12-14 11:03:09 +02:00
Commit Graph

15 Commits

Author SHA1 Message Date
ffeldmann
42b92d1bfe
Changes for Pipeline Reporting (#3213)
* Adds GetLog() function to orchestrator

* Fixes BUILD_NUMBER env variable

* Fixes correct env var for JENKINS_HOME

* Adds getEnv to read env variables with default value, adds test for jenkins GetLog() implementation

* Adds possibility to read errorJsons; updates splunk package for log files (WIP)

* Uncommenting dev code

* Adds GetLog() function to orchestrator

* Fixes BUILD_NUMBER env variable

* Fixes correct env var for JENKINS_HOME

* Adds getEnv to read env variables with default value, adds test for jenkins GetLog() implementation

* Adds possibility to read errorJsons; updates splunk package for log files (WIP)

* Uncommenting dev code

* Adds GetRequest function which holds the response in memory (not saved to disk)

* Implements GetLog() function for ADO, adds function to read PipelineRuntime

* PAT has been revoked

* Changes http package, s.t. if password only is required basic auth works too

* Adds env variable for azure token, error handling in case of unauthenticated/nil response

* Adds logging output in case env variable can not be read and fallback variable needs to be used

* Adds usage of environment variables for auth, uses jenkins api

* Adds init functionality for orchestrators, updates GetLog() and GetPipelineStartTime() function

* Adds initaliziation function for orchestrator authetnication

* Adds settings struct for orchestrator authentication

* Adds function to whole logfile to Splunk

* Struct for pipeline related telemetry information

* Increase messagebatch size to 10k

* Changes splunk package to a pointer based implementation, updates generated files and corresponding template and tests for splunk

* Changes telemetry package to pointer based implementation to have multiple telemetry objects, adjusted tests and splunk implementation

* Changes content type to txt

* Send telemetry independent of logfiles, increases amount of messages per file

* Adds JobURL for orchestrators and UnknownOrchestrator as fallback

* telemetry makes use of orchestrator specific information

* Adds orchestrator independent correlationID

* Adds custom fields for pipeline status

* go fmt

* Removes env var test - no env variables are read anymore

* Use UnknownOrchestratorConfigProvider in case the orchestrator can not be initalized

* Removes Custom fields from telemetry as these can not be reflected in SWA

* Adds custom telemetry information (piperHash,..) to each step telemetry information

* Removes falltrough in case no orchestrator has been found

* Updates tests for orchestrator package

* Adds orchestrator import in generated files

* Updates generator files for internal library

* Adds orchestrator telemetry information to steps

* Updates generated files, fatalHook writes to cpe

* Go generate from master, go fmt

* Adds Custom Data field LastErrorCode

* Removes GetLog() test

* Update init_unix.go

* Update docker_integration_test_executor.go

* Update integration_api_cli_test.go

* Reverts go1.17 fmt formatting

* Reverts go1.17 fmt formatting

* Reverts go1.17 fmt formatting

* Renames customTelemetryData to stepTelemetryData

* Adjustments to orchestrator-package, cleanup, adds JobName

* Adjusts commonPipelineEnvironment path

* Adds pipelineTelemetry struct to telemetry package, removes pipeline telemetry structs from splunk package

* Go fmt

* Changes path for errorDetails, adds debug information

* Removes custom fields from step, adds orchestrator, commithash to baseMetadata

* Adjusts tests for telemetry package

* Adds tests for orchestrator

* Updates generated files, initalization of splunk client only if its available in the config

* Fixes typo in helper go

* Update pkg/http/downloader.go

* Update pkg/http/downloader.go

* Update pkg/log/fatalHook.go

* Update fatalHook.go

* Update pkg/splunk/splunk.go

* Update pkg/telemetry/data.go

* Adds GetBuildStatus() and GetAPIInformation() to orchestrators

* error formatting

* Bugfix: dont send telemetry data if disabled, adjusts test

* go fmt

* Use correct error handling

* Update pkg/telemetry/telemetry.go

* Fixes telemetry disabled in the tests

* Fixes http tests

* Log fatal errors to logFile

* Adds CustomReportingConfig to hooks

* Cleanup comments in splunk package

* Adds possibility to send telemetry to custom endpoint

* Adds debug output for the payload

* Debug output for the payload as a string

* Adds test cases for changes in telemetry package

* go fmt

* Adds generated files for new step

* Reverts changes for http tests, causing problems with go1.15, changes need to be applied for newer go version >=1.17

* Adjusts test for sonarExecuteScan

* Adjusts test for sonarExecuteScan

* Adds explanation for customreportingConfig

* Makes disableing of customSend more obvious

* Adds custom step reporting to each step, updates generated files, adjusts helper testdata

* fixes unit test wrong usage of logging

* Send pipeline data altough there has been no error, adjust test cases

* Reverts changes for customReporting

* Updates generated files, removes customReporting

* Removes writing errorDetails to CPE

* Reverts usage of customreporting

* go fmt

* reverts changes in http_test

* reverts changes in http_test

* Skips integration cnb test

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-11-18 17:50:03 +01:00
Siarhei Pazdniakou
46bafc40a3
Improve validation of configuration (#3125)
* Implemented validation for the option possibleValues

* Has been added the option mandatoryIf to config with validation

* Fixed issues found during code review

* improved golang template

* Fixed tests. Added validation for mandatoryIf option

* Fix typo

* Fixed tests

* Validation was refactored. Added options

* Added default value for parameters with possibleValues option

* Validation was moved after the configuration resolution

* Canceled some default values

Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-10-01 12:49:05 +02:00
Oliver Nocon
805a8fd88f
feat(config): read config/defaults with authentication (#2975)
* feat(config):read config/defaults with authentication

This change allows to use defaults and config files from a protected GitHub repository.

The options `--customConfig` and `--defaultConfig` already allowed to provide a link to an uprotected file.

Now, by passing a value in the form `<hostname>:<token>` to parameter `gitHubTokens` (this parameter can be passed multiple times) a token can be provided for dedicated hosts.

This makes it possible to use a link like
`https://api.github.com/repos/SAP/jenkins-library/contents/resources/my-defaults.yml?ref=master`
as reference to a default file or similarly as reference to a configuration file.

* update generation to allow protected config/defaults

* fix CodeClimate issues

* update missing generations
2021-07-08 15:26:07 +02:00
tiloKo
9db249fe1c
Docu updates abap steps (#2910)
* docu updates

* go generate

* remove example as better explained in scenario docu

* Update abapAddonAssemblyKitCheckCVs_generated.go

* PV Step update

* md lint

* create TV

* publish TV

* lint

* register

* release

* reserve

* yaml lint

* Update abapAddonAssemblyKitRegisterPackages_generated.go

* Update resources/metadata/abapAddonAssemblyKitCreateTargetVector.yaml

Co-authored-by: Daniel Mieg <56156797+DanielMieg@users.noreply.github.com>

* Update resources/metadata/abapAddonAssemblyKitPublishTargetVector.yaml

Co-authored-by: Daniel Mieg <56156797+DanielMieg@users.noreply.github.com>

* refer

Co-authored-by: Daniel Mieg <56156797+DanielMieg@users.noreply.github.com>
2021-06-17 21:28:15 +02:00
Oliver Nocon
0b48bfcc73
feat: retrieve metadata by stepName - corrected (#2892)
* refactored getConfig to allow stepName param for metadata fetching

* extended step generator

* go generate

* Update cmd/getConfig.go

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Update cmd/getConfig.go

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Update cmd/getConfig.go

* update generated files

* update golden files to care for generator update

* update and add tests

* update generated files

* Update cmd/getConfig.go

* Update cmd/getConfig.go

* update/fix formatting

* feat: retrieve metadata by stepName - corrected

* update generation

* update condition logic for defaults

* update generation & tests

* support multiple conditions

* update generation

* Add generated

Co-authored-by: Leander Schulz <leander.schulz01@sap.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Sven Merk <s.merk@sap.com>
2021-06-16 08:43:30 +02:00
Oliver Nocon
4250ca8bed
Revert "feat(getConfig): retrieve metadata by stepName (#2736)" (#2891)
This reverts commit ae4a24c594.
2021-06-14 10:57:44 +02:00
lndrschlz
ae4a24c594
feat(getConfig): retrieve metadata by stepName (#2736)
* refactored getConfig to allow stepName param for metadata fetching

* extended step generator

* go generate

* Update cmd/getConfig.go

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Update cmd/getConfig.go

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Update cmd/getConfig.go

* update generated files

* update golden files to care for generator update

* update and add tests

* update generated files

* Update cmd/getConfig.go

* Update cmd/getConfig.go

* update/fix formatting

Co-authored-by: Leander Schulz <leander.schulz01@sap.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: OliverNocon <oliver.nocon@sap.com>
2021-06-14 08:58:41 +02:00
ffeldmann
b88ebdad6c
feat(splunk) Sending telemetry and logging information to Splunk (#2796)
* Adds inital splunk hook and logCollector

* Adds documentation of the Splunk hook

* Fixes markdown lint issues and removes comment from telemetry.go file

* Fixes markdown lint issues and adds missing generated file

* Markdown linting

* Changes documentation according to review, adds Splunk token automatically during init

* Adds error handling for marshalling hook config

* Markdown lint und correct Splunk token in httpclient

* Registeres Splunk token as secret and adjusts test cases

* Adds missing error handling and removes unnecessary comments

* Creates new function readPipelineEnvironment, adds tests

* Moves MonitoringData struct, edits defaults for json fields

* Adds gitRepository and gitOwner to telemetry information

* Simplifies readCommonPipelineEnvironment function, adds more descriptive errors, adds automated adding of Splunk prefix token

* Adjusts error handling

* Cleaner error logging
2021-05-17 12:14:04 +02:00
lndrschlz
b9bab27833
feat: expose complete step metadata through generated function (#2329)
* exposing step metadata through generator

* add metadata_generated.go

* fix step go test generation

* metadata fields added to generated files

* added generated files

* removed image placeholder from fortify step

* refactored step meta generation

* go generate

* fixed metadata generator and tests

* added output resource fields/tags to metadata generator

* fix string in metadata_generated

* go generate

* fixed generator

* go generate

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-11-20 08:13:59 +01:00
tiloKo
2a5514c116
AAK endpoint default (#2356)
Co-authored-by: Daniel Mieg <56156797+DanielMieg@users.noreply.github.com>
2020-11-16 13:40:57 +01:00
Kevin Stiehl
24aafb0b69
add vaultSecretFileReferences (#2203)
* add vaultSecretFileReferences

* fix test

* fix test

* go generate

* remove code duplication

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-10-26 14:20:04 +01:00
Christopher Fenner
be90876b7c
feat(output): handle non-string output values (#2113)
* handle non-string values as JSON

* change value type to interface in resources

* regenerate code

* add test cases

* handle reading of json files

* write json data to json files

* fix assignment

* use GetResourceParameter

* add test case

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-10-05 15:33:28 +02:00
Oliver Nocon
19c1732826
Telemetry: report error category (#2085) 2020-09-29 13:49:40 +02:00
tiloKo
427f78b3ed
AAKaaS: 2 Check Product Version (#2037)
* adding my steps

* messy step

* Update abapEnvironmentAssembly.go

* clean up

* change yaml

* corrections

* Update cloudFoundryDeploy.go

* update

* delete simulation step

* remove simulate

* Update PiperGoUtils.groovy

* Update PiperGoUtils.groovy

* Update CommonStepsTest.groovy

* add docu

* Update abapEnvironmentAssembly.md

* changes due to PR

* Update .gitignore

* b

* CV list

* Update abapEnvironmentAssembly.go

* testing with simulation

* Update abapEnvironmentAssembly.go

* remove simulation

* renaming

* Update mkdocs.yml

* moving service key to yaml and fixing code climate

* Update abapEnvironmentAssemblePackages.go

* Update abapEnvironmentAssemblePackages.go

* Update abapEnvironmentAssemblePackages.go

* Update abapEnvironmentAssemblePackages.go

* change input

* Update abapEnvironmentAssemblePackages.go

* change json tag

* fixed error handling

* documentation

* Update abapEnvironmentAssemblePackages.md

* Update abapEnvironmentAssemblePackages.md

* fixing code climate issues

* fixing code climate issues

* Update abapEnvironmentAssemblePackages.yaml

* fixing code climate issues

* Update abapEnvironmentAssemblePackages.yaml

* adding unittests

* adding unittests and improved logging

* yaml -> json

* change scope of cfServiceKeyName

* correct indentation

* Update CommonStepsTest.groovy

* maintain correct step order

* AAKaaS CheckPV step

* AAKaaS CheckPV step #2

* AAKaaS CheckPV step #3

* AAKaaS CheckPV step #4

* AAKaaS CheckPV step #5

* Update resources/metadata/abapAddonAssemblyKitCheckCVs.yaml

Co-authored-by: Daniel Mieg <56156797+DanielMieg@users.noreply.github.com>

* Update resources/metadata/abapAddonAssemblyKitCheckPV.yaml

Co-authored-by: Daniel Mieg <56156797+DanielMieg@users.noreply.github.com>

* Update resources/metadata/abapAddonAssemblyKitCheckPV.yaml

Co-authored-by: Daniel Mieg <56156797+DanielMieg@users.noreply.github.com>

* Update resources/metadata/abapAddonAssemblyKitCheckCVs.yaml

Co-authored-by: Daniel Mieg <56156797+DanielMieg@users.noreply.github.com>

* AAKaaS CheckPV step #6

Co-authored-by: rosemarieB <45030247+rosemarieB@users.noreply.github.com>
Co-authored-by: Daniel Mieg <56156797+DanielMieg@users.noreply.github.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-09-17 15:56:14 +02:00
tiloKo
2a776ba7eb
AAKaaS:1 check Component Versions (#2023)
* adding my steps

* messy step

* Update abapEnvironmentAssembly.go

* clean up

* change yaml

* corrections

* Update cloudFoundryDeploy.go

* update

* delete simulation step

* remove simulate

* Update PiperGoUtils.groovy

* Update PiperGoUtils.groovy

* Update CommonStepsTest.groovy

* add docu

* Update abapEnvironmentAssembly.md

* changes due to PR

* Update .gitignore

* b

* CV list

* Update abapEnvironmentAssembly.go

* testing with simulation

* Update abapEnvironmentAssembly.go

* remove simulation

* renaming

* Update mkdocs.yml

* moving service key to yaml and fixing code climate

* Update abapEnvironmentAssemblePackages.go

* Update abapEnvironmentAssemblePackages.go

* Update abapEnvironmentAssemblePackages.go

* Update abapEnvironmentAssemblePackages.go

* change input

* Update abapEnvironmentAssemblePackages.go

* change json tag

* fixed error handling

* documentation

* Update abapEnvironmentAssemblePackages.md

* Update abapEnvironmentAssemblePackages.md

* fixing code climate issues

* fixing code climate issues

* Update abapEnvironmentAssemblePackages.yaml

* fixing code climate issues

* Update abapEnvironmentAssemblePackages.yaml

* adding unittests

* adding unittests and improved logging

* yaml -> json

* change scope of cfServiceKeyName

* correct indentation

* Update CommonStepsTest.groovy

* maintain correct step order

* AAKaaS CheckCV step

* AAKaaS CheckCV step #2

* AAKaaS CheckCV step #3

* AAKaaS CheckCV step #4

* AAKaaS CheckCV step #5

* AAKaaS CheckCV step #6

* AAKaaS CheckCV step #7

* AAKaaS CheckCV step #8

* AAKaaS CheckCV step #9

* AAKaaS CheckCV step #10

* AAKaaS CheckCV step #11

* AAKaaS CheckCV step #12

* AAKaaS CheckCV step #13

Co-authored-by: rosemarieB <45030247+rosemarieB@users.noreply.github.com>
Co-authored-by: Daniel Mieg <56156797+DanielMieg@users.noreply.github.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-09-17 11:01:19 +02:00