Commit Graph
20 Commits
Author SHA1 Message Date
ffeldmann d0e205d018 chore: Disables webanalytics telemetry reporting by default (#4788)
* Disables telemetry reporting by default

* Update cmd/piper.go
2024-01-22 14:12:44 +01:00
ffeldmann a7d1ca2cb9 Increases retry mechanism to 1 and timeout for transport to 10 seconds (#4201) 2023-01-24 14:07:16 +01:00
ffeldmannandVyacheslav Starostin 4ae97a8a73 (fix) change in protecode for cvss from float to string (#4167)
* fixes change in protecode for cvss from float to string

* Fixes protecode json files with new string format for cvss

Co-authored-by: Vyacheslav Starostin <vyacheslav.starostin@sap.com>
2022-12-19 18:49:59 +01:00
ffeldmann c52e42fc84 feat(orchestrator) Adds BuildReasons to Jenkins Orchestrator. (#3961)
* Adds PullRequest and ResourceTrigger as BuildReason for Jenkins
2022-08-16 08:33:23 +02:00
ffeldmann a7d43b8571 fix: complete changeSet list (#3951)
* fixes: complete changeset list

* Adds test cases for multiple changeSets

* Adds PrNumber to ChangeSet

* Changes timestamp to Timestamp
2022-08-10 14:57:21 +02:00
ffeldmann f490900f1f feat(orchestrator package) Adds GetChangeSet functionality to Jenkins provider (#3852)
* Adds GetChangeSet functionality to Jenkins provider
2022-06-30 12:49:21 +02:00
ffeldmann a9329fba9e Reuse SOURCEBRANCH for branch name (#3672) 2022-03-28 13:48:55 +02:00
ffeldmann ccc1c976ee fix(orchestrator) usage of correct env variables (#3650)
* Reorders getApiInformation, changes variables to get start time, adjusts and adds test cases
* Changes the way to get apiInformation and reduces number of requests
* Changes getting pipeline start time from correct env variable
* Refactors getApiInformation functionality
* Adds GetBuildReason() for Azure and Jenkins
* Updates JobURL for ADO
2022-03-28 09:52:15 +02:00
ffeldmann b224f2294c Activates debug information for environment variables (#3630)
* Activates debug information for environment variables

* Adds tests for environment variable reading

* Reduces batch size to send messages to Splunk to 5000
2022-03-14 10:17:55 +01:00
ffeldmann 3e89fb0cd7 fixes retry when sending telemetry to Splunk (#3615)
* Do not retry sending data to Splunk HEC
* Adds error handling
2022-03-08 13:26:00 +01:00
ffeldmann 6398e61995 Splunk reporting; Sending messages in batches (#3611)
* Refactors logfile sending logic, renaming of fields, adds proper piper sourcetype
* Sets maximum retries to three and transport timeout to 10 seconds for azure and jenkins
2022-03-07 09:55:12 +01:00
ffeldmann 5f4cd838cf Updates telemetry logging information for internal reporting (#3585)
* Add StepStartTime, Renames StepDuration, adds PiperCommithash, removes Branch, GitOwner, GitRepository from logged telemetry information

* Fixes test case for telemetry logging

* Activates step monitoring data in debug mode

* Pretty debug json printing

* Reduces log noise, setting warning to debug
2022-02-28 09:45:57 +01:00
ffeldmannandOliver Nocon ffd4a7efb8 (fix) nil pointer dereference in case credentials are not available (#3564)
* adds return in gcs upload in case error occurs e.g. no credentials, avoid nil pointer dereference

* Adds generated files

* Updates generated files

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-02-22 18:32:09 +01:00
ffeldmann d7a13aa5d0 Fixes telemetry logging not reporting an error, adds tests (#3551)
* fixes error logging in case fatalError is empty, adds inital test cases for logStepTelemetryData

* Tests logging output
2022-02-17 15:39:01 +01:00
ffeldmann 19a05a9c70 Reporting: Variable change to deprecate step Splunk Hook; logging in telemetry (#3539)
* Adds fatalError variable with helper setter and getter helper functions
* Adds logging function to telemetry package (always executed)
2022-02-14 08:56:11 +01:00
ffeldmann 831c5b0061 Ignore violations if failOn is NONE (#3513) 2022-02-07 15:29:32 +01:00
ffeldmann f3566ab7e0 Changes GetJobName value to repositoryname (#3509) 2022-02-07 08:52:12 +01:00
ffeldmannandOliver Nocon d9e30418bc feat: pipeline reporting, adds logging information to fatal hook and splunk hook (#3308)
* Removes unecessary fields from telemetry, restructuring splunk pkg

* Removes t.skip() and reactivates integration test

* Adjusts tests for fatalHook and helper functions, including log test

* Moves pipelinetelemetry to inner source, removes pipelineTelemetry from telemetry pkg, using generic map[string]interface for splunk

* Removes Read JSON from fatalHook -> moves to inner source

* Removed log output test

* go fmt

* log step telemetry data send to splunk

* Adjusts error logging

* Adds log information in case api information could not be retrieved

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-11-26 16:11:44 +01:00
ffeldmann f16b11b804 Removes t.Skip() and re-activates integration test (#3299) 2021-11-19 11:28:15 +01:00
ffeldmannandOliver Nocon 42b92d1bfe Changes for Pipeline Reporting (#3213)
* Adds GetLog() function to orchestrator

* Fixes BUILD_NUMBER env variable

* Fixes correct env var for JENKINS_HOME

* Adds getEnv to read env variables with default value, adds test for jenkins GetLog() implementation

* Adds possibility to read errorJsons; updates splunk package for log files (WIP)

* Uncommenting dev code

* Adds GetLog() function to orchestrator

* Fixes BUILD_NUMBER env variable

* Fixes correct env var for JENKINS_HOME

* Adds getEnv to read env variables with default value, adds test for jenkins GetLog() implementation

* Adds possibility to read errorJsons; updates splunk package for log files (WIP)

* Uncommenting dev code

* Adds GetRequest function which holds the response in memory (not saved to disk)

* Implements GetLog() function for ADO, adds function to read PipelineRuntime

* PAT has been revoked

* Changes http package, s.t. if password only is required basic auth works too

* Adds env variable for azure token, error handling in case of unauthenticated/nil response

* Adds logging output in case env variable can not be read and fallback variable needs to be used

* Adds usage of environment variables for auth, uses jenkins api

* Adds init functionality for orchestrators, updates GetLog() and GetPipelineStartTime() function

* Adds initaliziation function for orchestrator authetnication

* Adds settings struct for orchestrator authentication

* Adds function to whole logfile to Splunk

* Struct for pipeline related telemetry information

* Increase messagebatch size to 10k

* Changes splunk package to a pointer based implementation, updates generated files and corresponding template and tests for splunk

* Changes telemetry package to pointer based implementation to have multiple telemetry objects, adjusted tests and splunk implementation

* Changes content type to txt

* Send telemetry independent of logfiles, increases amount of messages per file

* Adds JobURL for orchestrators and UnknownOrchestrator as fallback

* telemetry makes use of orchestrator specific information

* Adds orchestrator independent correlationID

* Adds custom fields for pipeline status

* go fmt

* Removes env var test - no env variables are read anymore

* Use UnknownOrchestratorConfigProvider in case the orchestrator can not be initalized

* Removes Custom fields from telemetry as these can not be reflected in SWA

* Adds custom telemetry information (piperHash,..) to each step telemetry information

* Removes falltrough in case no orchestrator has been found

* Updates tests for orchestrator package

* Adds orchestrator import in generated files

* Updates generator files for internal library

* Adds orchestrator telemetry information to steps

* Updates generated files, fatalHook writes to cpe

* Go generate from master, go fmt

* Adds Custom Data field LastErrorCode

* Removes GetLog() test

* Update init_unix.go

* Update docker_integration_test_executor.go

* Update integration_api_cli_test.go

* Reverts go1.17 fmt formatting

* Reverts go1.17 fmt formatting

* Reverts go1.17 fmt formatting

* Renames customTelemetryData to stepTelemetryData

* Adjustments to orchestrator-package, cleanup, adds JobName

* Adjusts commonPipelineEnvironment path

* Adds pipelineTelemetry struct to telemetry package, removes pipeline telemetry structs from splunk package

* Go fmt

* Changes path for errorDetails, adds debug information

* Removes custom fields from step, adds orchestrator, commithash to baseMetadata

* Adjusts tests for telemetry package

* Adds tests for orchestrator

* Updates generated files, initalization of splunk client only if its available in the config

* Fixes typo in helper go

* Update pkg/http/downloader.go

* Update pkg/http/downloader.go

* Update pkg/log/fatalHook.go

* Update fatalHook.go

* Update pkg/splunk/splunk.go

* Update pkg/telemetry/data.go

* Adds GetBuildStatus() and GetAPIInformation() to orchestrators

* error formatting

* Bugfix: dont send telemetry data if disabled, adjusts test

* go fmt

* Use correct error handling

* Update pkg/telemetry/telemetry.go

* Fixes telemetry disabled in the tests

* Fixes http tests

* Log fatal errors to logFile

* Adds CustomReportingConfig to hooks

* Cleanup comments in splunk package

* Adds possibility to send telemetry to custom endpoint

* Adds debug output for the payload

* Debug output for the payload as a string

* Adds test cases for changes in telemetry package

* go fmt

* Adds generated files for new step

* Reverts changes for http tests, causing problems with go1.15, changes need to be applied for newer go version >=1.17

* Adjusts test for sonarExecuteScan

* Adjusts test for sonarExecuteScan

* Adds explanation for customreportingConfig

* Makes disableing of customSend more obvious

* Adds custom step reporting to each step, updates generated files, adjusts helper testdata

* fixes unit test wrong usage of logging

* Send pipeline data altough there has been no error, adjust test cases

* Reverts changes for customReporting

* Updates generated files, removes customReporting

* Removes writing errorDetails to CPE

* Reverts usage of customreporting

* go fmt

* reverts changes in http_test

* reverts changes in http_test

* Skips integration cnb test

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-11-18 17:50:03 +01:00