1
0
mirror of https://github.com/SAP/jenkins-library.git synced 2024-12-12 10:55:20 +02:00
Commit Graph

12 Commits

Author SHA1 Message Date
Vyacheslav Starostin
b644bf7e15
Use Pendo as analytics tool (#4828)
* Try pendo

* Try pendo

* Fix typo

* Optimize data struct

* Adjust kaniko and kubernetes to try pendo

* Include golangBuild

* Update generator helper

* go generate

* Delete token param from some steps && go generate

* Add checking on ExportPrefix

* Update telemetry struct

* Try pipelineID for artifactPrepVersion

* Try pipelineID as a CPE value

* Update

* Try CF

* Read pipelineID from CPE

* Enable telemetry

* Fix name

* Do not show token in header

* Read pipelineID from CPE

* Clean up

* Clean up

* Update template

* go generate

* Update template

* Try Cribl

* HTTPS

* Update

* Clean-up

* Fix tests

* Update
2024-02-20 17:17:20 +06:00
Vyacheslav Starostin
59c2f75ce1
Splunk: Initialize Splunk client if dsn or prodCriblEndpoint provided (#4653)
* Initialize Splunk client if dsn or prodCriblEndpoint provided

* Fix tests

* Apply suggestions from code review

* go generate

---------

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2023-11-06 14:25:13 +06:00
Vyacheslav Starostin
3d63ce235c
Add the possibility to push data to Splunk Prod (#4459)
* Add Splunk Prod

* Update test data

* Update naming

* Unit-test: update naming

* Fix

* Fix typo

* Fix test

* go generate
2023-07-14 19:19:57 +06:00
Linda Siebert
acbcc5646b
[ANS] Change helper and re-generate steps (#3675)
* Add ans implementation

* Remove todo comment

* Rename test function

Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>

* Better wording

Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>

* Add reading of response body function

* Use http pkg ReadResponseBody

* Check read error

* Better test case description

* Fix formatting

* Create own package for read response body

* Omit empty nested resource struct

* Separate Resource struct from Event struct

* Merge and unmarshall instead of only unmarshalling

* Improve status code error message

* Remove unchangeable event fields

* Separate event parts

* Change log level setter function

* Restructure ans send test

* Revert exporting readResponseBody function

Instead the code is duplicated in the xsuaa and ans package

* Add check correct ans setup request

* Add set options function for mocking

* Review fixes

* Correct function name

* Use strict unmarshalling

* Validate event

* Move functions

* Add documentation comments

* improve test

* Validate event

* Add logrus hook for ans

* Set defaults on new hook creation

* Fix log level on error

* Don't alter entry log level

* Set severity fatal on 'fatal error' log message

* Ensure that log entries don't affect each other

* Remove unnecessary correlationID

* Use file path instead of event template string

* Improve warning messages

* Add empty log message check

* Allow configuration from file and string

* Add sourceEventId to tags

* Change resourceType to Pipeline

* Use structured config approach

* Use new log level set function

* Check correct setup and return error

* Mock http requests

* Only send log level warning or higher

* Use new function name

* One-liner ifs

* Improve test name

* Fix tests

* Prevent double firing

* Reduce Fire test size

* Add error message to test

* Reduce newANSHook test size

* Further check error

* Rename to defaultEvent in hook struct

* Reduce ifs further

* Fix set error category test

The ansHook Fire test cannot run in parallel, as it would affect the
other tests that use the error category.

* Change function name to SetServiceKey

* Validate event

* Rename to eventTemplate in hook struct

* Move copy to event.go

* Fix function mix

* Remove unnecessary cleanup

* Remove parallel test

The translation fails now and again when parallel is on.

* Remove prefix test

* Remove unused copyEvent function

* Fix ifs

* Add docu comment

* Register ans hook from pkg

* register hook and setup event template seperately

* Exclusively read eventTemplate from environment

* setupEventTemplate tests

* adjust hook levels test

* sync tests- wlill still fail

* migrate TestANSHook_registerANSHook test

* fixes

* Introduce necessary parameters

* Setup hook test

* Use file instead

* Adapt helper for ans

* Generate go files

* Add ans config to general config

* Change generator

* Regenerate steps

* Allow hook config from user config

Merges with hook config from defaults

* Remove ans flags from root command

* Get environment variables

* Generate files

* Add test when calling merge twice

* Update generator

* Regenerate steps

* Check two location for ans service key env var

* Re-generate

* Fix if

* Generate files with fix

* Duplicate config struct

* Add type casting test for ans config

* Fix helper

* Fix format

* Fix type casting of config

* Revert "Allow hook config from user config"

This reverts commit 4864499a4c497998c9ffc3e157ef491be955e68e.

* Revert "Add test when calling merge twice"

This reverts commit b82320fd07b82f5a597c5071049d918bcf62de00.

* Add ans config tests

* Improve helper code

* Re-generate commands

* Fix helper unit tests

* Change to only one argument

* Fix helper tests

* Re-generate

* Revert piper and config changes

* Re-generate missing step

* Generate new steps

* [ANS] Add servicekey credential to environment (#3684)

* Add ANS credential

* Switch to hooks and remove comments

* Add subsection for ans

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>

* Remove changes to piper.go

* Remove formatting

* Add test for ANS

* Define hook credential seperately from step credential

* Add test for retrieval from general section

* Add comment

* Get ans hook info from DefaultValueCache

* [ANS] Add documentation (#3704)

* Add ANS credential

* Switch to hooks and remove comments

* Add subsection for ans

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>

* Remove changes to piper.go

* Remove formatting

* Add test for ANS

* Define hook credential seperately from step credential

* Add test for retrieval from general section

* Add comment

* Add documentation

* Review changes

* Review comments

* Improve documentation further

* Add note of two event templates

* Add log level destinction

* Further improvements

* Improve text

* Remove unused things

* Add ANS credential

* Switch to hooks and remove comments

* Add subsection for ans

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>

* Remove changes to piper.go

* Remove formatting

* Add test for ANS

* Define hook credential seperately from step credential

* Add test for retrieval from general section

* Add comment

* Get ans hook info from DefaultValueCache

* Improvements

Co-authored-by: Linda Siebert <linda.siebert@sap.com>
Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>

* New lines

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
Co-authored-by: Roland Stengel <r.stengel@sap.com>
Co-authored-by: Thorsten Duda <thorsten.duda@sap.com>
2022-06-22 13:31:17 +02:00
Julian Schmitt
6151f36d88
Implement Release 3.0.0 of CM-Client (#3664)
* Remove --backend-type

* Delete CTS in isChangeDevelopment and change Dockerimage of CM-Client

* fix groovy unit tests

* another fix of groovy unit tests

* try to fix import of fork for Jenkins-Testing

* add workflow to create Go Binary for Jenkins-Server

* Change RepoOwner to test in Fork

* remove previous changes

* adjust docker image for TransportRequestCreate and Release

* Remove CTS from Documentation

Co-authored-by: Thorsten Duda <thorsten.duda@sap.com>
2022-04-05 12:33:19 +02:00
Siarhei Pazdniakou
cd243ee542
feat(gcs): allow upload to gcs from steps (#3034)
* Upload reports to Google Cloud Storage bucket

* Added tests. Made fixes

* Update step generation. GCS client was moved to GeneralConfig

* Code was refactored

* Fixed issues

* Fixed issues

* Code correction due to PR comments

* Improved gcs client and integration tests

* Integrated gcp config. Updated step metadata

* Fixed issues. Added tests

* Added cpe, vault, aliases resolving for reporting parameters

* Added tests

* Uncommented DeferExitHandler. Removed useless comments

* fixed cloning of config

* Added comments for exported functions. Removed unused mock

* minor fix

* Implemented setting of report name via paramRef

* some refactoring. Writing tests

* Update pkg/config/reporting.go

* Update cmd/sonarExecuteScan_generated.go

* Apply suggestions from code review

* Update pkg/config/reporting.go

* Update pkg/config/reporting.go

* fixed removing valut secret files

* Update pkg/config/reporting.go

* restore order

* restore order

* Apply suggestions from code review

* go generate

* fixed tests

* Update resources/metadata/sonarExecuteScan.yaml

* Update resources.go

* Fixed tests. Code was regenerated

* changed somewhere gcp to gcs. Fixed one test

* move gcsSubFolder to input parameters

* fixed removing valut secret files

* minor fix in integration tests

* fix integration tests

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2021-12-15 15:07:47 +01:00
Christopher Fenner
b7e1d28675
fix: prevent endless loop in exit handler (#3363)
* do not fatal in exit handler

* update generated sources

* update golden sources

* update generated sources
2021-12-15 14:26:23 +01:00
Siarhei Pazdniakou
0879fa591a
fix(influx): fix generated output parameters (#3362)
* fixed generated output parameters for influx

* change name to lower case

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2021-12-15 09:40:50 +01:00
ffeldmann
42b92d1bfe
Changes for Pipeline Reporting (#3213)
* Adds GetLog() function to orchestrator

* Fixes BUILD_NUMBER env variable

* Fixes correct env var for JENKINS_HOME

* Adds getEnv to read env variables with default value, adds test for jenkins GetLog() implementation

* Adds possibility to read errorJsons; updates splunk package for log files (WIP)

* Uncommenting dev code

* Adds GetLog() function to orchestrator

* Fixes BUILD_NUMBER env variable

* Fixes correct env var for JENKINS_HOME

* Adds getEnv to read env variables with default value, adds test for jenkins GetLog() implementation

* Adds possibility to read errorJsons; updates splunk package for log files (WIP)

* Uncommenting dev code

* Adds GetRequest function which holds the response in memory (not saved to disk)

* Implements GetLog() function for ADO, adds function to read PipelineRuntime

* PAT has been revoked

* Changes http package, s.t. if password only is required basic auth works too

* Adds env variable for azure token, error handling in case of unauthenticated/nil response

* Adds logging output in case env variable can not be read and fallback variable needs to be used

* Adds usage of environment variables for auth, uses jenkins api

* Adds init functionality for orchestrators, updates GetLog() and GetPipelineStartTime() function

* Adds initaliziation function for orchestrator authetnication

* Adds settings struct for orchestrator authentication

* Adds function to whole logfile to Splunk

* Struct for pipeline related telemetry information

* Increase messagebatch size to 10k

* Changes splunk package to a pointer based implementation, updates generated files and corresponding template and tests for splunk

* Changes telemetry package to pointer based implementation to have multiple telemetry objects, adjusted tests and splunk implementation

* Changes content type to txt

* Send telemetry independent of logfiles, increases amount of messages per file

* Adds JobURL for orchestrators and UnknownOrchestrator as fallback

* telemetry makes use of orchestrator specific information

* Adds orchestrator independent correlationID

* Adds custom fields for pipeline status

* go fmt

* Removes env var test - no env variables are read anymore

* Use UnknownOrchestratorConfigProvider in case the orchestrator can not be initalized

* Removes Custom fields from telemetry as these can not be reflected in SWA

* Adds custom telemetry information (piperHash,..) to each step telemetry information

* Removes falltrough in case no orchestrator has been found

* Updates tests for orchestrator package

* Adds orchestrator import in generated files

* Updates generator files for internal library

* Adds orchestrator telemetry information to steps

* Updates generated files, fatalHook writes to cpe

* Go generate from master, go fmt

* Adds Custom Data field LastErrorCode

* Removes GetLog() test

* Update init_unix.go

* Update docker_integration_test_executor.go

* Update integration_api_cli_test.go

* Reverts go1.17 fmt formatting

* Reverts go1.17 fmt formatting

* Reverts go1.17 fmt formatting

* Renames customTelemetryData to stepTelemetryData

* Adjustments to orchestrator-package, cleanup, adds JobName

* Adjusts commonPipelineEnvironment path

* Adds pipelineTelemetry struct to telemetry package, removes pipeline telemetry structs from splunk package

* Go fmt

* Changes path for errorDetails, adds debug information

* Removes custom fields from step, adds orchestrator, commithash to baseMetadata

* Adjusts tests for telemetry package

* Adds tests for orchestrator

* Updates generated files, initalization of splunk client only if its available in the config

* Fixes typo in helper go

* Update pkg/http/downloader.go

* Update pkg/http/downloader.go

* Update pkg/log/fatalHook.go

* Update fatalHook.go

* Update pkg/splunk/splunk.go

* Update pkg/telemetry/data.go

* Adds GetBuildStatus() and GetAPIInformation() to orchestrators

* error formatting

* Bugfix: dont send telemetry data if disabled, adjusts test

* go fmt

* Use correct error handling

* Update pkg/telemetry/telemetry.go

* Fixes telemetry disabled in the tests

* Fixes http tests

* Log fatal errors to logFile

* Adds CustomReportingConfig to hooks

* Cleanup comments in splunk package

* Adds possibility to send telemetry to custom endpoint

* Adds debug output for the payload

* Debug output for the payload as a string

* Adds test cases for changes in telemetry package

* go fmt

* Adds generated files for new step

* Reverts changes for http tests, causing problems with go1.15, changes need to be applied for newer go version >=1.17

* Adjusts test for sonarExecuteScan

* Adjusts test for sonarExecuteScan

* Adds explanation for customreportingConfig

* Makes disableing of customSend more obvious

* Adds custom step reporting to each step, updates generated files, adjusts helper testdata

* fixes unit test wrong usage of logging

* Send pipeline data altough there has been no error, adjust test cases

* Reverts changes for customReporting

* Updates generated files, removes customReporting

* Removes writing errorDetails to CPE

* Reverts usage of customreporting

* go fmt

* reverts changes in http_test

* reverts changes in http_test

* Skips integration cnb test

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-11-18 17:50:03 +01:00
Roland Stengel
a0f9c3669f
isChangeInDevelopment documentation (#3140)
* checkChangeInDevelopment to GO
- add groovy step isChangeInDevelopment
- direct to new Groovy
- direct to GO
2021-10-18 15:16:22 +02:00
Roland Stengel
41ed285afa
isChangeInDevelopment creation (#3139)
* checkChangeInDevelopment to GO
- add groovy step isChangeInDevelopment
- direct to new Groovy
- direct to GO
2021-10-04 14:35:38 +02:00
Roland Stengel
7612272992
checkChangeInDevelopment (#3115)
- refactor GO implementation
2021-10-04 13:08:34 +02:00