1
0
mirror of https://github.com/SAP/jenkins-library.git synced 2024-12-14 11:03:09 +02:00
sap-jenkins-library/cmd/newmanExecute_generated.go

329 lines
13 KiB
Go
Raw Normal View History

feat(newmanExecute): golang implmementation for newmanExecute (#2513) * Automates first parts of newmanExecute.groovy Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds newman installation Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Removes warning Signed-off-by: Fabian Reh <fabian.reh@sap.com> * makes tests robust for later shell calls Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds version logging Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds tests for version logging Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds newman shell execution Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Prepare cloud foundry apps with secrets handling Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds further process to CF Utils Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Fixes unit test Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds error category Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Add fix to execute step locally Currently only tested on windows machine locally in powershell. Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adapt unit test to fix of runCommand Signed-off-by: Fabian Reh <fabian.reh@sap.com> * refactored golang step to newmanExecute * wip * added test config * refactored newmanExecute groovy wrapper step * exclude newmanExecute from common step test * cleaups * add credential support * fix groovy credential providing * add import * add stageName * define script * remove unused vars * add import * fix iterator ref * golang secret handling and cleanups * wip * wip * wip * update go step * implement cf credential proposal * testRepository functionality implemented * register secrets to logger * add missing dependecies * test xsuaa credential handling * wip * wip * cleanups * add import * remove mandatory params * add container definition * test runCommand * test runCommand * fix npm path * fix npm path * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * added newmanEnvironment to templating * wip * use env and globals params in runCommand when no templating * fix condition * wip * reverted config edit * updated documentation * install with shell * wip * wip * fix tests * refactor tests * wip * remove old test * wip * escape go tmpl * escape go tmpl * fix defaults * add doc comment * remove test case * refactored newman commands * add cli reporter * refactor options * mock os getenv and fix all tests * refactoring and doc update * go generate * small refactor * spelling * fix newman doc * remove MaskPasswords wrapper; fix stash bug; * docu fix Co-authored-by: Fabian Reh <fabian.reh@sap.com>
2021-03-17 09:08:33 +02:00
// Code generated by piper's step-generator. DO NOT EDIT.
package cmd
import (
"fmt"
"os"
"path/filepath"
"reflect"
"strings"
feat(newmanExecute): golang implmementation for newmanExecute (#2513) * Automates first parts of newmanExecute.groovy Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds newman installation Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Removes warning Signed-off-by: Fabian Reh <fabian.reh@sap.com> * makes tests robust for later shell calls Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds version logging Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds tests for version logging Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds newman shell execution Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Prepare cloud foundry apps with secrets handling Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds further process to CF Utils Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Fixes unit test Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds error category Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Add fix to execute step locally Currently only tested on windows machine locally in powershell. Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adapt unit test to fix of runCommand Signed-off-by: Fabian Reh <fabian.reh@sap.com> * refactored golang step to newmanExecute * wip * added test config * refactored newmanExecute groovy wrapper step * exclude newmanExecute from common step test * cleaups * add credential support * fix groovy credential providing * add import * add stageName * define script * remove unused vars * add import * fix iterator ref * golang secret handling and cleanups * wip * wip * wip * update go step * implement cf credential proposal * testRepository functionality implemented * register secrets to logger * add missing dependecies * test xsuaa credential handling * wip * wip * cleanups * add import * remove mandatory params * add container definition * test runCommand * test runCommand * fix npm path * fix npm path * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * added newmanEnvironment to templating * wip * use env and globals params in runCommand when no templating * fix condition * wip * reverted config edit * updated documentation * install with shell * wip * wip * fix tests * refactor tests * wip * remove old test * wip * escape go tmpl * escape go tmpl * fix defaults * add doc comment * remove test case * refactored newman commands * add cli reporter * refactor options * mock os getenv and fix all tests * refactoring and doc update * go generate * small refactor * spelling * fix newman doc * remove MaskPasswords wrapper; fix stash bug; * docu fix Co-authored-by: Fabian Reh <fabian.reh@sap.com>
2021-03-17 09:08:33 +02:00
"time"
"github.com/SAP/jenkins-library/pkg/config"
"github.com/SAP/jenkins-library/pkg/gcs"
feat(newmanExecute): golang implmementation for newmanExecute (#2513) * Automates first parts of newmanExecute.groovy Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds newman installation Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Removes warning Signed-off-by: Fabian Reh <fabian.reh@sap.com> * makes tests robust for later shell calls Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds version logging Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds tests for version logging Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds newman shell execution Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Prepare cloud foundry apps with secrets handling Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds further process to CF Utils Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Fixes unit test Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds error category Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Add fix to execute step locally Currently only tested on windows machine locally in powershell. Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adapt unit test to fix of runCommand Signed-off-by: Fabian Reh <fabian.reh@sap.com> * refactored golang step to newmanExecute * wip * added test config * refactored newmanExecute groovy wrapper step * exclude newmanExecute from common step test * cleaups * add credential support * fix groovy credential providing * add import * add stageName * define script * remove unused vars * add import * fix iterator ref * golang secret handling and cleanups * wip * wip * wip * update go step * implement cf credential proposal * testRepository functionality implemented * register secrets to logger * add missing dependecies * test xsuaa credential handling * wip * wip * cleanups * add import * remove mandatory params * add container definition * test runCommand * test runCommand * fix npm path * fix npm path * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * added newmanEnvironment to templating * wip * use env and globals params in runCommand when no templating * fix condition * wip * reverted config edit * updated documentation * install with shell * wip * wip * fix tests * refactor tests * wip * remove old test * wip * escape go tmpl * escape go tmpl * fix defaults * add doc comment * remove test case * refactored newman commands * add cli reporter * refactor options * mock os getenv and fix all tests * refactoring and doc update * go generate * small refactor * spelling * fix newman doc * remove MaskPasswords wrapper; fix stash bug; * docu fix Co-authored-by: Fabian Reh <fabian.reh@sap.com>
2021-03-17 09:08:33 +02:00
"github.com/SAP/jenkins-library/pkg/log"
"github.com/SAP/jenkins-library/pkg/piperenv"
"github.com/SAP/jenkins-library/pkg/splunk"
feat(newmanExecute): golang implmementation for newmanExecute (#2513) * Automates first parts of newmanExecute.groovy Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds newman installation Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Removes warning Signed-off-by: Fabian Reh <fabian.reh@sap.com> * makes tests robust for later shell calls Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds version logging Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds tests for version logging Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds newman shell execution Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Prepare cloud foundry apps with secrets handling Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds further process to CF Utils Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Fixes unit test Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds error category Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Add fix to execute step locally Currently only tested on windows machine locally in powershell. Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adapt unit test to fix of runCommand Signed-off-by: Fabian Reh <fabian.reh@sap.com> * refactored golang step to newmanExecute * wip * added test config * refactored newmanExecute groovy wrapper step * exclude newmanExecute from common step test * cleaups * add credential support * fix groovy credential providing * add import * add stageName * define script * remove unused vars * add import * fix iterator ref * golang secret handling and cleanups * wip * wip * wip * update go step * implement cf credential proposal * testRepository functionality implemented * register secrets to logger * add missing dependecies * test xsuaa credential handling * wip * wip * cleanups * add import * remove mandatory params * add container definition * test runCommand * test runCommand * fix npm path * fix npm path * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * added newmanEnvironment to templating * wip * use env and globals params in runCommand when no templating * fix condition * wip * reverted config edit * updated documentation * install with shell * wip * wip * fix tests * refactor tests * wip * remove old test * wip * escape go tmpl * escape go tmpl * fix defaults * add doc comment * remove test case * refactored newman commands * add cli reporter * refactor options * mock os getenv and fix all tests * refactoring and doc update * go generate * small refactor * spelling * fix newman doc * remove MaskPasswords wrapper; fix stash bug; * docu fix Co-authored-by: Fabian Reh <fabian.reh@sap.com>
2021-03-17 09:08:33 +02:00
"github.com/SAP/jenkins-library/pkg/telemetry"
"github.com/SAP/jenkins-library/pkg/validation"
"github.com/bmatcuk/doublestar"
feat(newmanExecute): golang implmementation for newmanExecute (#2513) * Automates first parts of newmanExecute.groovy Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds newman installation Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Removes warning Signed-off-by: Fabian Reh <fabian.reh@sap.com> * makes tests robust for later shell calls Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds version logging Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds tests for version logging Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds newman shell execution Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Prepare cloud foundry apps with secrets handling Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds further process to CF Utils Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Fixes unit test Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds error category Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Add fix to execute step locally Currently only tested on windows machine locally in powershell. Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adapt unit test to fix of runCommand Signed-off-by: Fabian Reh <fabian.reh@sap.com> * refactored golang step to newmanExecute * wip * added test config * refactored newmanExecute groovy wrapper step * exclude newmanExecute from common step test * cleaups * add credential support * fix groovy credential providing * add import * add stageName * define script * remove unused vars * add import * fix iterator ref * golang secret handling and cleanups * wip * wip * wip * update go step * implement cf credential proposal * testRepository functionality implemented * register secrets to logger * add missing dependecies * test xsuaa credential handling * wip * wip * cleanups * add import * remove mandatory params * add container definition * test runCommand * test runCommand * fix npm path * fix npm path * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * added newmanEnvironment to templating * wip * use env and globals params in runCommand when no templating * fix condition * wip * reverted config edit * updated documentation * install with shell * wip * wip * fix tests * refactor tests * wip * remove old test * wip * escape go tmpl * escape go tmpl * fix defaults * add doc comment * remove test case * refactored newman commands * add cli reporter * refactor options * mock os getenv and fix all tests * refactoring and doc update * go generate * small refactor * spelling * fix newman doc * remove MaskPasswords wrapper; fix stash bug; * docu fix Co-authored-by: Fabian Reh <fabian.reh@sap.com>
2021-03-17 09:08:33 +02:00
"github.com/spf13/cobra"
)
type newmanExecuteOptions struct {
NewmanCollection string `json:"newmanCollection,omitempty"`
NewmanRunCommand string `json:"newmanRunCommand,omitempty"`
RunOptions []string `json:"runOptions,omitempty"`
NewmanInstallCommand string `json:"newmanInstallCommand,omitempty"`
NewmanEnvironment string `json:"newmanEnvironment,omitempty"`
NewmanGlobals string `json:"newmanGlobals,omitempty"`
FailOnError bool `json:"failOnError,omitempty"`
CfAppsWithSecrets []string `json:"cfAppsWithSecrets,omitempty"`
}
type newmanExecuteInflux struct {
step_data struct {
fields struct {
newman bool
}
tags struct {
}
}
}
func (i *newmanExecuteInflux) persist(path, resourceName string) {
measurementContent := []struct {
measurement string
valType string
name string
value interface{}
}{
{valType: config.InfluxField, measurement: "step_data", name: "newman", value: i.step_data.fields.newman},
}
errCount := 0
for _, metric := range measurementContent {
err := piperenv.SetResourceParameter(path, resourceName, filepath.Join(metric.measurement, fmt.Sprintf("%vs", metric.valType), metric.name), metric.value)
if err != nil {
log.Entry().WithError(err).Error("Error persisting influx environment.")
errCount++
}
}
if errCount > 0 {
log.Entry().Error("failed to persist Influx environment")
}
}
type newmanExecuteReports struct {
}
func (p *newmanExecuteReports) persist(stepConfig newmanExecuteOptions, gcpJsonKeyFilePath string, gcsBucketId string, gcsFolderPath string, gcsSubFolder string) {
if gcsBucketId == "" {
log.Entry().Info("persisting reports to GCS is disabled, because gcsBucketId is empty")
return
}
log.Entry().Info("Uploading reports to Google Cloud Storage...")
content := []gcs.ReportOutputParam{
{FilePattern: "**/TEST-*.xml", ParamRef: "", StepResultType: "acceptance-test"},
{FilePattern: "**/requirement.mapping", ParamRef: "", StepResultType: "requirement-mapping"},
{FilePattern: "**/delivery.mapping", ParamRef: "", StepResultType: "delivery-mapping"},
}
envVars := []gcs.EnvVar{
{Name: "GOOGLE_APPLICATION_CREDENTIALS", Value: gcpJsonKeyFilePath, Modified: false},
}
gcsClient, err := gcs.NewClient(gcs.WithEnvVars(envVars))
if err != nil {
log.Entry().Errorf("creation of GCS client failed: %v", err)
return
}
defer gcsClient.Close()
structVal := reflect.ValueOf(&stepConfig).Elem()
inputParameters := map[string]string{}
for i := 0; i < structVal.NumField(); i++ {
field := structVal.Type().Field(i)
if field.Type.String() == "string" {
paramName := strings.Split(field.Tag.Get("json"), ",")
paramValue, _ := structVal.Field(i).Interface().(string)
inputParameters[paramName[0]] = paramValue
}
}
if err := gcs.PersistReportsToGCS(gcsClient, content, inputParameters, gcsFolderPath, gcsBucketId, gcsSubFolder, doublestar.Glob, os.Stat); err != nil {
log.Entry().Errorf("failed to persist reports: %v", err)
}
}
feat(newmanExecute): golang implmementation for newmanExecute (#2513) * Automates first parts of newmanExecute.groovy Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds newman installation Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Removes warning Signed-off-by: Fabian Reh <fabian.reh@sap.com> * makes tests robust for later shell calls Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds version logging Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds tests for version logging Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds newman shell execution Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Prepare cloud foundry apps with secrets handling Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds further process to CF Utils Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Fixes unit test Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds error category Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Add fix to execute step locally Currently only tested on windows machine locally in powershell. Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adapt unit test to fix of runCommand Signed-off-by: Fabian Reh <fabian.reh@sap.com> * refactored golang step to newmanExecute * wip * added test config * refactored newmanExecute groovy wrapper step * exclude newmanExecute from common step test * cleaups * add credential support * fix groovy credential providing * add import * add stageName * define script * remove unused vars * add import * fix iterator ref * golang secret handling and cleanups * wip * wip * wip * update go step * implement cf credential proposal * testRepository functionality implemented * register secrets to logger * add missing dependecies * test xsuaa credential handling * wip * wip * cleanups * add import * remove mandatory params * add container definition * test runCommand * test runCommand * fix npm path * fix npm path * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * added newmanEnvironment to templating * wip * use env and globals params in runCommand when no templating * fix condition * wip * reverted config edit * updated documentation * install with shell * wip * wip * fix tests * refactor tests * wip * remove old test * wip * escape go tmpl * escape go tmpl * fix defaults * add doc comment * remove test case * refactored newman commands * add cli reporter * refactor options * mock os getenv and fix all tests * refactoring and doc update * go generate * small refactor * spelling * fix newman doc * remove MaskPasswords wrapper; fix stash bug; * docu fix Co-authored-by: Fabian Reh <fabian.reh@sap.com>
2021-03-17 09:08:33 +02:00
// NewmanExecuteCommand Installs newman and executes specified newman collections.
func NewmanExecuteCommand() *cobra.Command {
const STEP_NAME = "newmanExecute"
metadata := newmanExecuteMetadata()
var stepConfig newmanExecuteOptions
var startTime time.Time
var influx newmanExecuteInflux
var reports newmanExecuteReports
var logCollector *log.CollectorHook
Changes for Pipeline Reporting (#3213) * Adds GetLog() function to orchestrator * Fixes BUILD_NUMBER env variable * Fixes correct env var for JENKINS_HOME * Adds getEnv to read env variables with default value, adds test for jenkins GetLog() implementation * Adds possibility to read errorJsons; updates splunk package for log files (WIP) * Uncommenting dev code * Adds GetLog() function to orchestrator * Fixes BUILD_NUMBER env variable * Fixes correct env var for JENKINS_HOME * Adds getEnv to read env variables with default value, adds test for jenkins GetLog() implementation * Adds possibility to read errorJsons; updates splunk package for log files (WIP) * Uncommenting dev code * Adds GetRequest function which holds the response in memory (not saved to disk) * Implements GetLog() function for ADO, adds function to read PipelineRuntime * PAT has been revoked * Changes http package, s.t. if password only is required basic auth works too * Adds env variable for azure token, error handling in case of unauthenticated/nil response * Adds logging output in case env variable can not be read and fallback variable needs to be used * Adds usage of environment variables for auth, uses jenkins api * Adds init functionality for orchestrators, updates GetLog() and GetPipelineStartTime() function * Adds initaliziation function for orchestrator authetnication * Adds settings struct for orchestrator authentication * Adds function to whole logfile to Splunk * Struct for pipeline related telemetry information * Increase messagebatch size to 10k * Changes splunk package to a pointer based implementation, updates generated files and corresponding template and tests for splunk * Changes telemetry package to pointer based implementation to have multiple telemetry objects, adjusted tests and splunk implementation * Changes content type to txt * Send telemetry independent of logfiles, increases amount of messages per file * Adds JobURL for orchestrators and UnknownOrchestrator as fallback * telemetry makes use of orchestrator specific information * Adds orchestrator independent correlationID * Adds custom fields for pipeline status * go fmt * Removes env var test - no env variables are read anymore * Use UnknownOrchestratorConfigProvider in case the orchestrator can not be initalized * Removes Custom fields from telemetry as these can not be reflected in SWA * Adds custom telemetry information (piperHash,..) to each step telemetry information * Removes falltrough in case no orchestrator has been found * Updates tests for orchestrator package * Adds orchestrator import in generated files * Updates generator files for internal library * Adds orchestrator telemetry information to steps * Updates generated files, fatalHook writes to cpe * Go generate from master, go fmt * Adds Custom Data field LastErrorCode * Removes GetLog() test * Update init_unix.go * Update docker_integration_test_executor.go * Update integration_api_cli_test.go * Reverts go1.17 fmt formatting * Reverts go1.17 fmt formatting * Reverts go1.17 fmt formatting * Renames customTelemetryData to stepTelemetryData * Adjustments to orchestrator-package, cleanup, adds JobName * Adjusts commonPipelineEnvironment path * Adds pipelineTelemetry struct to telemetry package, removes pipeline telemetry structs from splunk package * Go fmt * Changes path for errorDetails, adds debug information * Removes custom fields from step, adds orchestrator, commithash to baseMetadata * Adjusts tests for telemetry package * Adds tests for orchestrator * Updates generated files, initalization of splunk client only if its available in the config * Fixes typo in helper go * Update pkg/http/downloader.go * Update pkg/http/downloader.go * Update pkg/log/fatalHook.go * Update fatalHook.go * Update pkg/splunk/splunk.go * Update pkg/telemetry/data.go * Adds GetBuildStatus() and GetAPIInformation() to orchestrators * error formatting * Bugfix: dont send telemetry data if disabled, adjusts test * go fmt * Use correct error handling * Update pkg/telemetry/telemetry.go * Fixes telemetry disabled in the tests * Fixes http tests * Log fatal errors to logFile * Adds CustomReportingConfig to hooks * Cleanup comments in splunk package * Adds possibility to send telemetry to custom endpoint * Adds debug output for the payload * Debug output for the payload as a string * Adds test cases for changes in telemetry package * go fmt * Adds generated files for new step * Reverts changes for http tests, causing problems with go1.15, changes need to be applied for newer go version >=1.17 * Adjusts test for sonarExecuteScan * Adjusts test for sonarExecuteScan * Adds explanation for customreportingConfig * Makes disableing of customSend more obvious * Adds custom step reporting to each step, updates generated files, adjusts helper testdata * fixes unit test wrong usage of logging * Send pipeline data altough there has been no error, adjust test cases * Reverts changes for customReporting * Updates generated files, removes customReporting * Removes writing errorDetails to CPE * Reverts usage of customreporting * go fmt * reverts changes in http_test * reverts changes in http_test * Skips integration cnb test Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-11-18 18:50:03 +02:00
var splunkClient *splunk.Splunk
telemetryClient := &telemetry.Telemetry{}
feat(newmanExecute): golang implmementation for newmanExecute (#2513) * Automates first parts of newmanExecute.groovy Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds newman installation Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Removes warning Signed-off-by: Fabian Reh <fabian.reh@sap.com> * makes tests robust for later shell calls Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds version logging Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds tests for version logging Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds newman shell execution Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Prepare cloud foundry apps with secrets handling Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds further process to CF Utils Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Fixes unit test Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds error category Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Add fix to execute step locally Currently only tested on windows machine locally in powershell. Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adapt unit test to fix of runCommand Signed-off-by: Fabian Reh <fabian.reh@sap.com> * refactored golang step to newmanExecute * wip * added test config * refactored newmanExecute groovy wrapper step * exclude newmanExecute from common step test * cleaups * add credential support * fix groovy credential providing * add import * add stageName * define script * remove unused vars * add import * fix iterator ref * golang secret handling and cleanups * wip * wip * wip * update go step * implement cf credential proposal * testRepository functionality implemented * register secrets to logger * add missing dependecies * test xsuaa credential handling * wip * wip * cleanups * add import * remove mandatory params * add container definition * test runCommand * test runCommand * fix npm path * fix npm path * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * added newmanEnvironment to templating * wip * use env and globals params in runCommand when no templating * fix condition * wip * reverted config edit * updated documentation * install with shell * wip * wip * fix tests * refactor tests * wip * remove old test * wip * escape go tmpl * escape go tmpl * fix defaults * add doc comment * remove test case * refactored newman commands * add cli reporter * refactor options * mock os getenv and fix all tests * refactoring and doc update * go generate * small refactor * spelling * fix newman doc * remove MaskPasswords wrapper; fix stash bug; * docu fix Co-authored-by: Fabian Reh <fabian.reh@sap.com>
2021-03-17 09:08:33 +02:00
var createNewmanExecuteCmd = &cobra.Command{
Use: STEP_NAME,
Short: "Installs newman and executes specified newman collections.",
Long: `This script executes [Postman](https://www.getpostman.com) tests from a collection via the [Newman](https://www.getpostman.com/docs/v6/postman/collection_runs/command_line_integration_with_newman) command line tool.`,
PreRunE: func(cmd *cobra.Command, _ []string) error {
startTime = time.Now()
log.SetStepName(STEP_NAME)
log.SetVerbose(GeneralConfig.Verbose)
GeneralConfig.GitHubAccessTokens = ResolveAccessTokens(GeneralConfig.GitHubTokens)
feat(newmanExecute): golang implmementation for newmanExecute (#2513) * Automates first parts of newmanExecute.groovy Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds newman installation Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Removes warning Signed-off-by: Fabian Reh <fabian.reh@sap.com> * makes tests robust for later shell calls Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds version logging Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds tests for version logging Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds newman shell execution Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Prepare cloud foundry apps with secrets handling Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds further process to CF Utils Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Fixes unit test Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds error category Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Add fix to execute step locally Currently only tested on windows machine locally in powershell. Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adapt unit test to fix of runCommand Signed-off-by: Fabian Reh <fabian.reh@sap.com> * refactored golang step to newmanExecute * wip * added test config * refactored newmanExecute groovy wrapper step * exclude newmanExecute from common step test * cleaups * add credential support * fix groovy credential providing * add import * add stageName * define script * remove unused vars * add import * fix iterator ref * golang secret handling and cleanups * wip * wip * wip * update go step * implement cf credential proposal * testRepository functionality implemented * register secrets to logger * add missing dependecies * test xsuaa credential handling * wip * wip * cleanups * add import * remove mandatory params * add container definition * test runCommand * test runCommand * fix npm path * fix npm path * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * added newmanEnvironment to templating * wip * use env and globals params in runCommand when no templating * fix condition * wip * reverted config edit * updated documentation * install with shell * wip * wip * fix tests * refactor tests * wip * remove old test * wip * escape go tmpl * escape go tmpl * fix defaults * add doc comment * remove test case * refactored newman commands * add cli reporter * refactor options * mock os getenv and fix all tests * refactoring and doc update * go generate * small refactor * spelling * fix newman doc * remove MaskPasswords wrapper; fix stash bug; * docu fix Co-authored-by: Fabian Reh <fabian.reh@sap.com>
2021-03-17 09:08:33 +02:00
path, _ := os.Getwd()
fatalHook := &log.FatalHook{CorrelationID: GeneralConfig.CorrelationID, Path: path}
log.RegisterHook(fatalHook)
err := PrepareConfig(cmd, &metadata, STEP_NAME, &stepConfig, config.OpenPiperFile)
if err != nil {
log.SetErrorCategory(log.ErrorConfiguration)
return err
}
if len(GeneralConfig.HookConfig.SentryConfig.Dsn) > 0 {
sentryHook := log.NewSentryHook(GeneralConfig.HookConfig.SentryConfig.Dsn, GeneralConfig.CorrelationID)
log.RegisterHook(&sentryHook)
}
if len(GeneralConfig.HookConfig.SplunkConfig.Dsn) > 0 {
Changes for Pipeline Reporting (#3213) * Adds GetLog() function to orchestrator * Fixes BUILD_NUMBER env variable * Fixes correct env var for JENKINS_HOME * Adds getEnv to read env variables with default value, adds test for jenkins GetLog() implementation * Adds possibility to read errorJsons; updates splunk package for log files (WIP) * Uncommenting dev code * Adds GetLog() function to orchestrator * Fixes BUILD_NUMBER env variable * Fixes correct env var for JENKINS_HOME * Adds getEnv to read env variables with default value, adds test for jenkins GetLog() implementation * Adds possibility to read errorJsons; updates splunk package for log files (WIP) * Uncommenting dev code * Adds GetRequest function which holds the response in memory (not saved to disk) * Implements GetLog() function for ADO, adds function to read PipelineRuntime * PAT has been revoked * Changes http package, s.t. if password only is required basic auth works too * Adds env variable for azure token, error handling in case of unauthenticated/nil response * Adds logging output in case env variable can not be read and fallback variable needs to be used * Adds usage of environment variables for auth, uses jenkins api * Adds init functionality for orchestrators, updates GetLog() and GetPipelineStartTime() function * Adds initaliziation function for orchestrator authetnication * Adds settings struct for orchestrator authentication * Adds function to whole logfile to Splunk * Struct for pipeline related telemetry information * Increase messagebatch size to 10k * Changes splunk package to a pointer based implementation, updates generated files and corresponding template and tests for splunk * Changes telemetry package to pointer based implementation to have multiple telemetry objects, adjusted tests and splunk implementation * Changes content type to txt * Send telemetry independent of logfiles, increases amount of messages per file * Adds JobURL for orchestrators and UnknownOrchestrator as fallback * telemetry makes use of orchestrator specific information * Adds orchestrator independent correlationID * Adds custom fields for pipeline status * go fmt * Removes env var test - no env variables are read anymore * Use UnknownOrchestratorConfigProvider in case the orchestrator can not be initalized * Removes Custom fields from telemetry as these can not be reflected in SWA * Adds custom telemetry information (piperHash,..) to each step telemetry information * Removes falltrough in case no orchestrator has been found * Updates tests for orchestrator package * Adds orchestrator import in generated files * Updates generator files for internal library * Adds orchestrator telemetry information to steps * Updates generated files, fatalHook writes to cpe * Go generate from master, go fmt * Adds Custom Data field LastErrorCode * Removes GetLog() test * Update init_unix.go * Update docker_integration_test_executor.go * Update integration_api_cli_test.go * Reverts go1.17 fmt formatting * Reverts go1.17 fmt formatting * Reverts go1.17 fmt formatting * Renames customTelemetryData to stepTelemetryData * Adjustments to orchestrator-package, cleanup, adds JobName * Adjusts commonPipelineEnvironment path * Adds pipelineTelemetry struct to telemetry package, removes pipeline telemetry structs from splunk package * Go fmt * Changes path for errorDetails, adds debug information * Removes custom fields from step, adds orchestrator, commithash to baseMetadata * Adjusts tests for telemetry package * Adds tests for orchestrator * Updates generated files, initalization of splunk client only if its available in the config * Fixes typo in helper go * Update pkg/http/downloader.go * Update pkg/http/downloader.go * Update pkg/log/fatalHook.go * Update fatalHook.go * Update pkg/splunk/splunk.go * Update pkg/telemetry/data.go * Adds GetBuildStatus() and GetAPIInformation() to orchestrators * error formatting * Bugfix: dont send telemetry data if disabled, adjusts test * go fmt * Use correct error handling * Update pkg/telemetry/telemetry.go * Fixes telemetry disabled in the tests * Fixes http tests * Log fatal errors to logFile * Adds CustomReportingConfig to hooks * Cleanup comments in splunk package * Adds possibility to send telemetry to custom endpoint * Adds debug output for the payload * Debug output for the payload as a string * Adds test cases for changes in telemetry package * go fmt * Adds generated files for new step * Reverts changes for http tests, causing problems with go1.15, changes need to be applied for newer go version >=1.17 * Adjusts test for sonarExecuteScan * Adjusts test for sonarExecuteScan * Adds explanation for customreportingConfig * Makes disableing of customSend more obvious * Adds custom step reporting to each step, updates generated files, adjusts helper testdata * fixes unit test wrong usage of logging * Send pipeline data altough there has been no error, adjust test cases * Reverts changes for customReporting * Updates generated files, removes customReporting * Removes writing errorDetails to CPE * Reverts usage of customreporting * go fmt * reverts changes in http_test * reverts changes in http_test * Skips integration cnb test Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-11-18 18:50:03 +02:00
splunkClient = &splunk.Splunk{}
logCollector = &log.CollectorHook{CorrelationID: GeneralConfig.CorrelationID}
log.RegisterHook(logCollector)
}
[ANS] Change helper and re-generate steps (#3675) * Add ans implementation * Remove todo comment * Rename test function Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com> * Better wording Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com> * Add reading of response body function * Use http pkg ReadResponseBody * Check read error * Better test case description * Fix formatting * Create own package for read response body * Omit empty nested resource struct * Separate Resource struct from Event struct * Merge and unmarshall instead of only unmarshalling * Improve status code error message * Remove unchangeable event fields * Separate event parts * Change log level setter function * Restructure ans send test * Revert exporting readResponseBody function Instead the code is duplicated in the xsuaa and ans package * Add check correct ans setup request * Add set options function for mocking * Review fixes * Correct function name * Use strict unmarshalling * Validate event * Move functions * Add documentation comments * improve test * Validate event * Add logrus hook for ans * Set defaults on new hook creation * Fix log level on error * Don't alter entry log level * Set severity fatal on 'fatal error' log message * Ensure that log entries don't affect each other * Remove unnecessary correlationID * Use file path instead of event template string * Improve warning messages * Add empty log message check * Allow configuration from file and string * Add sourceEventId to tags * Change resourceType to Pipeline * Use structured config approach * Use new log level set function * Check correct setup and return error * Mock http requests * Only send log level warning or higher * Use new function name * One-liner ifs * Improve test name * Fix tests * Prevent double firing * Reduce Fire test size * Add error message to test * Reduce newANSHook test size * Further check error * Rename to defaultEvent in hook struct * Reduce ifs further * Fix set error category test The ansHook Fire test cannot run in parallel, as it would affect the other tests that use the error category. * Change function name to SetServiceKey * Validate event * Rename to eventTemplate in hook struct * Move copy to event.go * Fix function mix * Remove unnecessary cleanup * Remove parallel test The translation fails now and again when parallel is on. * Remove prefix test * Remove unused copyEvent function * Fix ifs * Add docu comment * Register ans hook from pkg * register hook and setup event template seperately * Exclusively read eventTemplate from environment * setupEventTemplate tests * adjust hook levels test * sync tests- wlill still fail * migrate TestANSHook_registerANSHook test * fixes * Introduce necessary parameters * Setup hook test * Use file instead * Adapt helper for ans * Generate go files * Add ans config to general config * Change generator * Regenerate steps * Allow hook config from user config Merges with hook config from defaults * Remove ans flags from root command * Get environment variables * Generate files * Add test when calling merge twice * Update generator * Regenerate steps * Check two location for ans service key env var * Re-generate * Fix if * Generate files with fix * Duplicate config struct * Add type casting test for ans config * Fix helper * Fix format * Fix type casting of config * Revert "Allow hook config from user config" This reverts commit 4864499a4c497998c9ffc3e157ef491be955e68e. * Revert "Add test when calling merge twice" This reverts commit b82320fd07b82f5a597c5071049d918bcf62de00. * Add ans config tests * Improve helper code * Re-generate commands * Fix helper unit tests * Change to only one argument * Fix helper tests * Re-generate * Revert piper and config changes * Re-generate missing step * Generate new steps * [ANS] Add servicekey credential to environment (#3684) * Add ANS credential * Switch to hooks and remove comments * Add subsection for ans Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com> * Remove changes to piper.go * Remove formatting * Add test for ANS * Define hook credential seperately from step credential * Add test for retrieval from general section * Add comment * Get ans hook info from DefaultValueCache * [ANS] Add documentation (#3704) * Add ANS credential * Switch to hooks and remove comments * Add subsection for ans Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com> * Remove changes to piper.go * Remove formatting * Add test for ANS * Define hook credential seperately from step credential * Add test for retrieval from general section * Add comment * Add documentation * Review changes * Review comments * Improve documentation further * Add note of two event templates * Add log level destinction * Further improvements * Improve text * Remove unused things * Add ANS credential * Switch to hooks and remove comments * Add subsection for ans Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com> * Remove changes to piper.go * Remove formatting * Add test for ANS * Define hook credential seperately from step credential * Add test for retrieval from general section * Add comment * Get ans hook info from DefaultValueCache * Improvements Co-authored-by: Linda Siebert <linda.siebert@sap.com> Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com> Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com> * New lines Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com> Co-authored-by: Roland Stengel <r.stengel@sap.com> Co-authored-by: Thorsten Duda <thorsten.duda@sap.com>
2022-06-22 13:31:17 +02:00
if err = log.RegisterANSHookIfConfigured(GeneralConfig.CorrelationID); err != nil {
log.Entry().WithError(err).Warn("failed to set up SAP Alert Notification Service log hook")
}
validation, err := validation.New(validation.WithJSONNamesForStructFields(), validation.WithPredefinedErrorMessages())
if err != nil {
return err
}
if err = validation.ValidateStruct(stepConfig); err != nil {
log.SetErrorCategory(log.ErrorConfiguration)
return err
}
feat(newmanExecute): golang implmementation for newmanExecute (#2513) * Automates first parts of newmanExecute.groovy Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds newman installation Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Removes warning Signed-off-by: Fabian Reh <fabian.reh@sap.com> * makes tests robust for later shell calls Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds version logging Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds tests for version logging Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds newman shell execution Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Prepare cloud foundry apps with secrets handling Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds further process to CF Utils Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Fixes unit test Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds error category Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Add fix to execute step locally Currently only tested on windows machine locally in powershell. Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adapt unit test to fix of runCommand Signed-off-by: Fabian Reh <fabian.reh@sap.com> * refactored golang step to newmanExecute * wip * added test config * refactored newmanExecute groovy wrapper step * exclude newmanExecute from common step test * cleaups * add credential support * fix groovy credential providing * add import * add stageName * define script * remove unused vars * add import * fix iterator ref * golang secret handling and cleanups * wip * wip * wip * update go step * implement cf credential proposal * testRepository functionality implemented * register secrets to logger * add missing dependecies * test xsuaa credential handling * wip * wip * cleanups * add import * remove mandatory params * add container definition * test runCommand * test runCommand * fix npm path * fix npm path * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * added newmanEnvironment to templating * wip * use env and globals params in runCommand when no templating * fix condition * wip * reverted config edit * updated documentation * install with shell * wip * wip * fix tests * refactor tests * wip * remove old test * wip * escape go tmpl * escape go tmpl * fix defaults * add doc comment * remove test case * refactored newman commands * add cli reporter * refactor options * mock os getenv and fix all tests * refactoring and doc update * go generate * small refactor * spelling * fix newman doc * remove MaskPasswords wrapper; fix stash bug; * docu fix Co-authored-by: Fabian Reh <fabian.reh@sap.com>
2021-03-17 09:08:33 +02:00
return nil
},
Run: func(_ *cobra.Command, _ []string) {
Changes for Pipeline Reporting (#3213) * Adds GetLog() function to orchestrator * Fixes BUILD_NUMBER env variable * Fixes correct env var for JENKINS_HOME * Adds getEnv to read env variables with default value, adds test for jenkins GetLog() implementation * Adds possibility to read errorJsons; updates splunk package for log files (WIP) * Uncommenting dev code * Adds GetLog() function to orchestrator * Fixes BUILD_NUMBER env variable * Fixes correct env var for JENKINS_HOME * Adds getEnv to read env variables with default value, adds test for jenkins GetLog() implementation * Adds possibility to read errorJsons; updates splunk package for log files (WIP) * Uncommenting dev code * Adds GetRequest function which holds the response in memory (not saved to disk) * Implements GetLog() function for ADO, adds function to read PipelineRuntime * PAT has been revoked * Changes http package, s.t. if password only is required basic auth works too * Adds env variable for azure token, error handling in case of unauthenticated/nil response * Adds logging output in case env variable can not be read and fallback variable needs to be used * Adds usage of environment variables for auth, uses jenkins api * Adds init functionality for orchestrators, updates GetLog() and GetPipelineStartTime() function * Adds initaliziation function for orchestrator authetnication * Adds settings struct for orchestrator authentication * Adds function to whole logfile to Splunk * Struct for pipeline related telemetry information * Increase messagebatch size to 10k * Changes splunk package to a pointer based implementation, updates generated files and corresponding template and tests for splunk * Changes telemetry package to pointer based implementation to have multiple telemetry objects, adjusted tests and splunk implementation * Changes content type to txt * Send telemetry independent of logfiles, increases amount of messages per file * Adds JobURL for orchestrators and UnknownOrchestrator as fallback * telemetry makes use of orchestrator specific information * Adds orchestrator independent correlationID * Adds custom fields for pipeline status * go fmt * Removes env var test - no env variables are read anymore * Use UnknownOrchestratorConfigProvider in case the orchestrator can not be initalized * Removes Custom fields from telemetry as these can not be reflected in SWA * Adds custom telemetry information (piperHash,..) to each step telemetry information * Removes falltrough in case no orchestrator has been found * Updates tests for orchestrator package * Adds orchestrator import in generated files * Updates generator files for internal library * Adds orchestrator telemetry information to steps * Updates generated files, fatalHook writes to cpe * Go generate from master, go fmt * Adds Custom Data field LastErrorCode * Removes GetLog() test * Update init_unix.go * Update docker_integration_test_executor.go * Update integration_api_cli_test.go * Reverts go1.17 fmt formatting * Reverts go1.17 fmt formatting * Reverts go1.17 fmt formatting * Renames customTelemetryData to stepTelemetryData * Adjustments to orchestrator-package, cleanup, adds JobName * Adjusts commonPipelineEnvironment path * Adds pipelineTelemetry struct to telemetry package, removes pipeline telemetry structs from splunk package * Go fmt * Changes path for errorDetails, adds debug information * Removes custom fields from step, adds orchestrator, commithash to baseMetadata * Adjusts tests for telemetry package * Adds tests for orchestrator * Updates generated files, initalization of splunk client only if its available in the config * Fixes typo in helper go * Update pkg/http/downloader.go * Update pkg/http/downloader.go * Update pkg/log/fatalHook.go * Update fatalHook.go * Update pkg/splunk/splunk.go * Update pkg/telemetry/data.go * Adds GetBuildStatus() and GetAPIInformation() to orchestrators * error formatting * Bugfix: dont send telemetry data if disabled, adjusts test * go fmt * Use correct error handling * Update pkg/telemetry/telemetry.go * Fixes telemetry disabled in the tests * Fixes http tests * Log fatal errors to logFile * Adds CustomReportingConfig to hooks * Cleanup comments in splunk package * Adds possibility to send telemetry to custom endpoint * Adds debug output for the payload * Debug output for the payload as a string * Adds test cases for changes in telemetry package * go fmt * Adds generated files for new step * Reverts changes for http tests, causing problems with go1.15, changes need to be applied for newer go version >=1.17 * Adjusts test for sonarExecuteScan * Adjusts test for sonarExecuteScan * Adds explanation for customreportingConfig * Makes disableing of customSend more obvious * Adds custom step reporting to each step, updates generated files, adjusts helper testdata * fixes unit test wrong usage of logging * Send pipeline data altough there has been no error, adjust test cases * Reverts changes for customReporting * Updates generated files, removes customReporting * Removes writing errorDetails to CPE * Reverts usage of customreporting * go fmt * reverts changes in http_test * reverts changes in http_test * Skips integration cnb test Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-11-18 18:50:03 +02:00
stepTelemetryData := telemetry.CustomData{}
stepTelemetryData.ErrorCode = "1"
feat(newmanExecute): golang implmementation for newmanExecute (#2513) * Automates first parts of newmanExecute.groovy Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds newman installation Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Removes warning Signed-off-by: Fabian Reh <fabian.reh@sap.com> * makes tests robust for later shell calls Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds version logging Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds tests for version logging Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds newman shell execution Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Prepare cloud foundry apps with secrets handling Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds further process to CF Utils Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Fixes unit test Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds error category Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Add fix to execute step locally Currently only tested on windows machine locally in powershell. Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adapt unit test to fix of runCommand Signed-off-by: Fabian Reh <fabian.reh@sap.com> * refactored golang step to newmanExecute * wip * added test config * refactored newmanExecute groovy wrapper step * exclude newmanExecute from common step test * cleaups * add credential support * fix groovy credential providing * add import * add stageName * define script * remove unused vars * add import * fix iterator ref * golang secret handling and cleanups * wip * wip * wip * update go step * implement cf credential proposal * testRepository functionality implemented * register secrets to logger * add missing dependecies * test xsuaa credential handling * wip * wip * cleanups * add import * remove mandatory params * add container definition * test runCommand * test runCommand * fix npm path * fix npm path * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * added newmanEnvironment to templating * wip * use env and globals params in runCommand when no templating * fix condition * wip * reverted config edit * updated documentation * install with shell * wip * wip * fix tests * refactor tests * wip * remove old test * wip * escape go tmpl * escape go tmpl * fix defaults * add doc comment * remove test case * refactored newman commands * add cli reporter * refactor options * mock os getenv and fix all tests * refactoring and doc update * go generate * small refactor * spelling * fix newman doc * remove MaskPasswords wrapper; fix stash bug; * docu fix Co-authored-by: Fabian Reh <fabian.reh@sap.com>
2021-03-17 09:08:33 +02:00
handler := func() {
influx.persist(GeneralConfig.EnvRootPath, "influx")
reports.persist(stepConfig, GeneralConfig.GCPJsonKeyFilePath, GeneralConfig.GCSBucketId, GeneralConfig.GCSFolderPath, GeneralConfig.GCSSubFolder)
feat(gcs): allow upload to gcs from steps (#3034) * Upload reports to Google Cloud Storage bucket * Added tests. Made fixes * Update step generation. GCS client was moved to GeneralConfig * Code was refactored * Fixed issues * Fixed issues * Code correction due to PR comments * Improved gcs client and integration tests * Integrated gcp config. Updated step metadata * Fixed issues. Added tests * Added cpe, vault, aliases resolving for reporting parameters * Added tests * Uncommented DeferExitHandler. Removed useless comments * fixed cloning of config * Added comments for exported functions. Removed unused mock * minor fix * Implemented setting of report name via paramRef * some refactoring. Writing tests * Update pkg/config/reporting.go * Update cmd/sonarExecuteScan_generated.go * Apply suggestions from code review * Update pkg/config/reporting.go * Update pkg/config/reporting.go * fixed removing valut secret files * Update pkg/config/reporting.go * restore order * restore order * Apply suggestions from code review * go generate * fixed tests * Update resources/metadata/sonarExecuteScan.yaml * Update resources.go * Fixed tests. Code was regenerated * changed somewhere gcp to gcs. Fixed one test * move gcsSubFolder to input parameters * fixed removing valut secret files * minor fix in integration tests * fix integration tests Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com> Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com> Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2021-12-15 16:07:47 +02:00
config.RemoveVaultSecretFiles()
Changes for Pipeline Reporting (#3213) * Adds GetLog() function to orchestrator * Fixes BUILD_NUMBER env variable * Fixes correct env var for JENKINS_HOME * Adds getEnv to read env variables with default value, adds test for jenkins GetLog() implementation * Adds possibility to read errorJsons; updates splunk package for log files (WIP) * Uncommenting dev code * Adds GetLog() function to orchestrator * Fixes BUILD_NUMBER env variable * Fixes correct env var for JENKINS_HOME * Adds getEnv to read env variables with default value, adds test for jenkins GetLog() implementation * Adds possibility to read errorJsons; updates splunk package for log files (WIP) * Uncommenting dev code * Adds GetRequest function which holds the response in memory (not saved to disk) * Implements GetLog() function for ADO, adds function to read PipelineRuntime * PAT has been revoked * Changes http package, s.t. if password only is required basic auth works too * Adds env variable for azure token, error handling in case of unauthenticated/nil response * Adds logging output in case env variable can not be read and fallback variable needs to be used * Adds usage of environment variables for auth, uses jenkins api * Adds init functionality for orchestrators, updates GetLog() and GetPipelineStartTime() function * Adds initaliziation function for orchestrator authetnication * Adds settings struct for orchestrator authentication * Adds function to whole logfile to Splunk * Struct for pipeline related telemetry information * Increase messagebatch size to 10k * Changes splunk package to a pointer based implementation, updates generated files and corresponding template and tests for splunk * Changes telemetry package to pointer based implementation to have multiple telemetry objects, adjusted tests and splunk implementation * Changes content type to txt * Send telemetry independent of logfiles, increases amount of messages per file * Adds JobURL for orchestrators and UnknownOrchestrator as fallback * telemetry makes use of orchestrator specific information * Adds orchestrator independent correlationID * Adds custom fields for pipeline status * go fmt * Removes env var test - no env variables are read anymore * Use UnknownOrchestratorConfigProvider in case the orchestrator can not be initalized * Removes Custom fields from telemetry as these can not be reflected in SWA * Adds custom telemetry information (piperHash,..) to each step telemetry information * Removes falltrough in case no orchestrator has been found * Updates tests for orchestrator package * Adds orchestrator import in generated files * Updates generator files for internal library * Adds orchestrator telemetry information to steps * Updates generated files, fatalHook writes to cpe * Go generate from master, go fmt * Adds Custom Data field LastErrorCode * Removes GetLog() test * Update init_unix.go * Update docker_integration_test_executor.go * Update integration_api_cli_test.go * Reverts go1.17 fmt formatting * Reverts go1.17 fmt formatting * Reverts go1.17 fmt formatting * Renames customTelemetryData to stepTelemetryData * Adjustments to orchestrator-package, cleanup, adds JobName * Adjusts commonPipelineEnvironment path * Adds pipelineTelemetry struct to telemetry package, removes pipeline telemetry structs from splunk package * Go fmt * Changes path for errorDetails, adds debug information * Removes custom fields from step, adds orchestrator, commithash to baseMetadata * Adjusts tests for telemetry package * Adds tests for orchestrator * Updates generated files, initalization of splunk client only if its available in the config * Fixes typo in helper go * Update pkg/http/downloader.go * Update pkg/http/downloader.go * Update pkg/log/fatalHook.go * Update fatalHook.go * Update pkg/splunk/splunk.go * Update pkg/telemetry/data.go * Adds GetBuildStatus() and GetAPIInformation() to orchestrators * error formatting * Bugfix: dont send telemetry data if disabled, adjusts test * go fmt * Use correct error handling * Update pkg/telemetry/telemetry.go * Fixes telemetry disabled in the tests * Fixes http tests * Log fatal errors to logFile * Adds CustomReportingConfig to hooks * Cleanup comments in splunk package * Adds possibility to send telemetry to custom endpoint * Adds debug output for the payload * Debug output for the payload as a string * Adds test cases for changes in telemetry package * go fmt * Adds generated files for new step * Reverts changes for http tests, causing problems with go1.15, changes need to be applied for newer go version >=1.17 * Adjusts test for sonarExecuteScan * Adjusts test for sonarExecuteScan * Adds explanation for customreportingConfig * Makes disableing of customSend more obvious * Adds custom step reporting to each step, updates generated files, adjusts helper testdata * fixes unit test wrong usage of logging * Send pipeline data altough there has been no error, adjust test cases * Reverts changes for customReporting * Updates generated files, removes customReporting * Removes writing errorDetails to CPE * Reverts usage of customreporting * go fmt * reverts changes in http_test * reverts changes in http_test * Skips integration cnb test Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-11-18 18:50:03 +02:00
stepTelemetryData.Duration = fmt.Sprintf("%v", time.Since(startTime).Milliseconds())
stepTelemetryData.ErrorCategory = log.GetErrorCategory().String()
stepTelemetryData.PiperCommitHash = GitCommit
telemetryClient.SetData(&stepTelemetryData)
telemetryClient.Send()
if len(GeneralConfig.HookConfig.SplunkConfig.Dsn) > 0 {
Changes for Pipeline Reporting (#3213) * Adds GetLog() function to orchestrator * Fixes BUILD_NUMBER env variable * Fixes correct env var for JENKINS_HOME * Adds getEnv to read env variables with default value, adds test for jenkins GetLog() implementation * Adds possibility to read errorJsons; updates splunk package for log files (WIP) * Uncommenting dev code * Adds GetLog() function to orchestrator * Fixes BUILD_NUMBER env variable * Fixes correct env var for JENKINS_HOME * Adds getEnv to read env variables with default value, adds test for jenkins GetLog() implementation * Adds possibility to read errorJsons; updates splunk package for log files (WIP) * Uncommenting dev code * Adds GetRequest function which holds the response in memory (not saved to disk) * Implements GetLog() function for ADO, adds function to read PipelineRuntime * PAT has been revoked * Changes http package, s.t. if password only is required basic auth works too * Adds env variable for azure token, error handling in case of unauthenticated/nil response * Adds logging output in case env variable can not be read and fallback variable needs to be used * Adds usage of environment variables for auth, uses jenkins api * Adds init functionality for orchestrators, updates GetLog() and GetPipelineStartTime() function * Adds initaliziation function for orchestrator authetnication * Adds settings struct for orchestrator authentication * Adds function to whole logfile to Splunk * Struct for pipeline related telemetry information * Increase messagebatch size to 10k * Changes splunk package to a pointer based implementation, updates generated files and corresponding template and tests for splunk * Changes telemetry package to pointer based implementation to have multiple telemetry objects, adjusted tests and splunk implementation * Changes content type to txt * Send telemetry independent of logfiles, increases amount of messages per file * Adds JobURL for orchestrators and UnknownOrchestrator as fallback * telemetry makes use of orchestrator specific information * Adds orchestrator independent correlationID * Adds custom fields for pipeline status * go fmt * Removes env var test - no env variables are read anymore * Use UnknownOrchestratorConfigProvider in case the orchestrator can not be initalized * Removes Custom fields from telemetry as these can not be reflected in SWA * Adds custom telemetry information (piperHash,..) to each step telemetry information * Removes falltrough in case no orchestrator has been found * Updates tests for orchestrator package * Adds orchestrator import in generated files * Updates generator files for internal library * Adds orchestrator telemetry information to steps * Updates generated files, fatalHook writes to cpe * Go generate from master, go fmt * Adds Custom Data field LastErrorCode * Removes GetLog() test * Update init_unix.go * Update docker_integration_test_executor.go * Update integration_api_cli_test.go * Reverts go1.17 fmt formatting * Reverts go1.17 fmt formatting * Reverts go1.17 fmt formatting * Renames customTelemetryData to stepTelemetryData * Adjustments to orchestrator-package, cleanup, adds JobName * Adjusts commonPipelineEnvironment path * Adds pipelineTelemetry struct to telemetry package, removes pipeline telemetry structs from splunk package * Go fmt * Changes path for errorDetails, adds debug information * Removes custom fields from step, adds orchestrator, commithash to baseMetadata * Adjusts tests for telemetry package * Adds tests for orchestrator * Updates generated files, initalization of splunk client only if its available in the config * Fixes typo in helper go * Update pkg/http/downloader.go * Update pkg/http/downloader.go * Update pkg/log/fatalHook.go * Update fatalHook.go * Update pkg/splunk/splunk.go * Update pkg/telemetry/data.go * Adds GetBuildStatus() and GetAPIInformation() to orchestrators * error formatting * Bugfix: dont send telemetry data if disabled, adjusts test * go fmt * Use correct error handling * Update pkg/telemetry/telemetry.go * Fixes telemetry disabled in the tests * Fixes http tests * Log fatal errors to logFile * Adds CustomReportingConfig to hooks * Cleanup comments in splunk package * Adds possibility to send telemetry to custom endpoint * Adds debug output for the payload * Debug output for the payload as a string * Adds test cases for changes in telemetry package * go fmt * Adds generated files for new step * Reverts changes for http tests, causing problems with go1.15, changes need to be applied for newer go version >=1.17 * Adjusts test for sonarExecuteScan * Adjusts test for sonarExecuteScan * Adds explanation for customreportingConfig * Makes disableing of customSend more obvious * Adds custom step reporting to each step, updates generated files, adjusts helper testdata * fixes unit test wrong usage of logging * Send pipeline data altough there has been no error, adjust test cases * Reverts changes for customReporting * Updates generated files, removes customReporting * Removes writing errorDetails to CPE * Reverts usage of customreporting * go fmt * reverts changes in http_test * reverts changes in http_test * Skips integration cnb test Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-11-18 18:50:03 +02:00
splunkClient.Send(telemetryClient.GetData(), logCollector)
}
feat(newmanExecute): golang implmementation for newmanExecute (#2513) * Automates first parts of newmanExecute.groovy Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds newman installation Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Removes warning Signed-off-by: Fabian Reh <fabian.reh@sap.com> * makes tests robust for later shell calls Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds version logging Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds tests for version logging Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds newman shell execution Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Prepare cloud foundry apps with secrets handling Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds further process to CF Utils Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Fixes unit test Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds error category Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Add fix to execute step locally Currently only tested on windows machine locally in powershell. Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adapt unit test to fix of runCommand Signed-off-by: Fabian Reh <fabian.reh@sap.com> * refactored golang step to newmanExecute * wip * added test config * refactored newmanExecute groovy wrapper step * exclude newmanExecute from common step test * cleaups * add credential support * fix groovy credential providing * add import * add stageName * define script * remove unused vars * add import * fix iterator ref * golang secret handling and cleanups * wip * wip * wip * update go step * implement cf credential proposal * testRepository functionality implemented * register secrets to logger * add missing dependecies * test xsuaa credential handling * wip * wip * cleanups * add import * remove mandatory params * add container definition * test runCommand * test runCommand * fix npm path * fix npm path * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * added newmanEnvironment to templating * wip * use env and globals params in runCommand when no templating * fix condition * wip * reverted config edit * updated documentation * install with shell * wip * wip * fix tests * refactor tests * wip * remove old test * wip * escape go tmpl * escape go tmpl * fix defaults * add doc comment * remove test case * refactored newman commands * add cli reporter * refactor options * mock os getenv and fix all tests * refactoring and doc update * go generate * small refactor * spelling * fix newman doc * remove MaskPasswords wrapper; fix stash bug; * docu fix Co-authored-by: Fabian Reh <fabian.reh@sap.com>
2021-03-17 09:08:33 +02:00
}
log.DeferExitHandler(handler)
defer handler()
Changes for Pipeline Reporting (#3213) * Adds GetLog() function to orchestrator * Fixes BUILD_NUMBER env variable * Fixes correct env var for JENKINS_HOME * Adds getEnv to read env variables with default value, adds test for jenkins GetLog() implementation * Adds possibility to read errorJsons; updates splunk package for log files (WIP) * Uncommenting dev code * Adds GetLog() function to orchestrator * Fixes BUILD_NUMBER env variable * Fixes correct env var for JENKINS_HOME * Adds getEnv to read env variables with default value, adds test for jenkins GetLog() implementation * Adds possibility to read errorJsons; updates splunk package for log files (WIP) * Uncommenting dev code * Adds GetRequest function which holds the response in memory (not saved to disk) * Implements GetLog() function for ADO, adds function to read PipelineRuntime * PAT has been revoked * Changes http package, s.t. if password only is required basic auth works too * Adds env variable for azure token, error handling in case of unauthenticated/nil response * Adds logging output in case env variable can not be read and fallback variable needs to be used * Adds usage of environment variables for auth, uses jenkins api * Adds init functionality for orchestrators, updates GetLog() and GetPipelineStartTime() function * Adds initaliziation function for orchestrator authetnication * Adds settings struct for orchestrator authentication * Adds function to whole logfile to Splunk * Struct for pipeline related telemetry information * Increase messagebatch size to 10k * Changes splunk package to a pointer based implementation, updates generated files and corresponding template and tests for splunk * Changes telemetry package to pointer based implementation to have multiple telemetry objects, adjusted tests and splunk implementation * Changes content type to txt * Send telemetry independent of logfiles, increases amount of messages per file * Adds JobURL for orchestrators and UnknownOrchestrator as fallback * telemetry makes use of orchestrator specific information * Adds orchestrator independent correlationID * Adds custom fields for pipeline status * go fmt * Removes env var test - no env variables are read anymore * Use UnknownOrchestratorConfigProvider in case the orchestrator can not be initalized * Removes Custom fields from telemetry as these can not be reflected in SWA * Adds custom telemetry information (piperHash,..) to each step telemetry information * Removes falltrough in case no orchestrator has been found * Updates tests for orchestrator package * Adds orchestrator import in generated files * Updates generator files for internal library * Adds orchestrator telemetry information to steps * Updates generated files, fatalHook writes to cpe * Go generate from master, go fmt * Adds Custom Data field LastErrorCode * Removes GetLog() test * Update init_unix.go * Update docker_integration_test_executor.go * Update integration_api_cli_test.go * Reverts go1.17 fmt formatting * Reverts go1.17 fmt formatting * Reverts go1.17 fmt formatting * Renames customTelemetryData to stepTelemetryData * Adjustments to orchestrator-package, cleanup, adds JobName * Adjusts commonPipelineEnvironment path * Adds pipelineTelemetry struct to telemetry package, removes pipeline telemetry structs from splunk package * Go fmt * Changes path for errorDetails, adds debug information * Removes custom fields from step, adds orchestrator, commithash to baseMetadata * Adjusts tests for telemetry package * Adds tests for orchestrator * Updates generated files, initalization of splunk client only if its available in the config * Fixes typo in helper go * Update pkg/http/downloader.go * Update pkg/http/downloader.go * Update pkg/log/fatalHook.go * Update fatalHook.go * Update pkg/splunk/splunk.go * Update pkg/telemetry/data.go * Adds GetBuildStatus() and GetAPIInformation() to orchestrators * error formatting * Bugfix: dont send telemetry data if disabled, adjusts test * go fmt * Use correct error handling * Update pkg/telemetry/telemetry.go * Fixes telemetry disabled in the tests * Fixes http tests * Log fatal errors to logFile * Adds CustomReportingConfig to hooks * Cleanup comments in splunk package * Adds possibility to send telemetry to custom endpoint * Adds debug output for the payload * Debug output for the payload as a string * Adds test cases for changes in telemetry package * go fmt * Adds generated files for new step * Reverts changes for http tests, causing problems with go1.15, changes need to be applied for newer go version >=1.17 * Adjusts test for sonarExecuteScan * Adjusts test for sonarExecuteScan * Adds explanation for customreportingConfig * Makes disableing of customSend more obvious * Adds custom step reporting to each step, updates generated files, adjusts helper testdata * fixes unit test wrong usage of logging * Send pipeline data altough there has been no error, adjust test cases * Reverts changes for customReporting * Updates generated files, removes customReporting * Removes writing errorDetails to CPE * Reverts usage of customreporting * go fmt * reverts changes in http_test * reverts changes in http_test * Skips integration cnb test Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-11-18 18:50:03 +02:00
telemetryClient.Initialize(GeneralConfig.NoTelemetry, STEP_NAME)
if len(GeneralConfig.HookConfig.SplunkConfig.Dsn) > 0 {
Changes for Pipeline Reporting (#3213) * Adds GetLog() function to orchestrator * Fixes BUILD_NUMBER env variable * Fixes correct env var for JENKINS_HOME * Adds getEnv to read env variables with default value, adds test for jenkins GetLog() implementation * Adds possibility to read errorJsons; updates splunk package for log files (WIP) * Uncommenting dev code * Adds GetLog() function to orchestrator * Fixes BUILD_NUMBER env variable * Fixes correct env var for JENKINS_HOME * Adds getEnv to read env variables with default value, adds test for jenkins GetLog() implementation * Adds possibility to read errorJsons; updates splunk package for log files (WIP) * Uncommenting dev code * Adds GetRequest function which holds the response in memory (not saved to disk) * Implements GetLog() function for ADO, adds function to read PipelineRuntime * PAT has been revoked * Changes http package, s.t. if password only is required basic auth works too * Adds env variable for azure token, error handling in case of unauthenticated/nil response * Adds logging output in case env variable can not be read and fallback variable needs to be used * Adds usage of environment variables for auth, uses jenkins api * Adds init functionality for orchestrators, updates GetLog() and GetPipelineStartTime() function * Adds initaliziation function for orchestrator authetnication * Adds settings struct for orchestrator authentication * Adds function to whole logfile to Splunk * Struct for pipeline related telemetry information * Increase messagebatch size to 10k * Changes splunk package to a pointer based implementation, updates generated files and corresponding template and tests for splunk * Changes telemetry package to pointer based implementation to have multiple telemetry objects, adjusted tests and splunk implementation * Changes content type to txt * Send telemetry independent of logfiles, increases amount of messages per file * Adds JobURL for orchestrators and UnknownOrchestrator as fallback * telemetry makes use of orchestrator specific information * Adds orchestrator independent correlationID * Adds custom fields for pipeline status * go fmt * Removes env var test - no env variables are read anymore * Use UnknownOrchestratorConfigProvider in case the orchestrator can not be initalized * Removes Custom fields from telemetry as these can not be reflected in SWA * Adds custom telemetry information (piperHash,..) to each step telemetry information * Removes falltrough in case no orchestrator has been found * Updates tests for orchestrator package * Adds orchestrator import in generated files * Updates generator files for internal library * Adds orchestrator telemetry information to steps * Updates generated files, fatalHook writes to cpe * Go generate from master, go fmt * Adds Custom Data field LastErrorCode * Removes GetLog() test * Update init_unix.go * Update docker_integration_test_executor.go * Update integration_api_cli_test.go * Reverts go1.17 fmt formatting * Reverts go1.17 fmt formatting * Reverts go1.17 fmt formatting * Renames customTelemetryData to stepTelemetryData * Adjustments to orchestrator-package, cleanup, adds JobName * Adjusts commonPipelineEnvironment path * Adds pipelineTelemetry struct to telemetry package, removes pipeline telemetry structs from splunk package * Go fmt * Changes path for errorDetails, adds debug information * Removes custom fields from step, adds orchestrator, commithash to baseMetadata * Adjusts tests for telemetry package * Adds tests for orchestrator * Updates generated files, initalization of splunk client only if its available in the config * Fixes typo in helper go * Update pkg/http/downloader.go * Update pkg/http/downloader.go * Update pkg/log/fatalHook.go * Update fatalHook.go * Update pkg/splunk/splunk.go * Update pkg/telemetry/data.go * Adds GetBuildStatus() and GetAPIInformation() to orchestrators * error formatting * Bugfix: dont send telemetry data if disabled, adjusts test * go fmt * Use correct error handling * Update pkg/telemetry/telemetry.go * Fixes telemetry disabled in the tests * Fixes http tests * Log fatal errors to logFile * Adds CustomReportingConfig to hooks * Cleanup comments in splunk package * Adds possibility to send telemetry to custom endpoint * Adds debug output for the payload * Debug output for the payload as a string * Adds test cases for changes in telemetry package * go fmt * Adds generated files for new step * Reverts changes for http tests, causing problems with go1.15, changes need to be applied for newer go version >=1.17 * Adjusts test for sonarExecuteScan * Adjusts test for sonarExecuteScan * Adds explanation for customreportingConfig * Makes disableing of customSend more obvious * Adds custom step reporting to each step, updates generated files, adjusts helper testdata * fixes unit test wrong usage of logging * Send pipeline data altough there has been no error, adjust test cases * Reverts changes for customReporting * Updates generated files, removes customReporting * Removes writing errorDetails to CPE * Reverts usage of customreporting * go fmt * reverts changes in http_test * reverts changes in http_test * Skips integration cnb test Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-11-18 18:50:03 +02:00
splunkClient.Initialize(GeneralConfig.CorrelationID,
GeneralConfig.HookConfig.SplunkConfig.Dsn,
GeneralConfig.HookConfig.SplunkConfig.Token,
GeneralConfig.HookConfig.SplunkConfig.Index,
GeneralConfig.HookConfig.SplunkConfig.SendLogs)
}
Changes for Pipeline Reporting (#3213) * Adds GetLog() function to orchestrator * Fixes BUILD_NUMBER env variable * Fixes correct env var for JENKINS_HOME * Adds getEnv to read env variables with default value, adds test for jenkins GetLog() implementation * Adds possibility to read errorJsons; updates splunk package for log files (WIP) * Uncommenting dev code * Adds GetLog() function to orchestrator * Fixes BUILD_NUMBER env variable * Fixes correct env var for JENKINS_HOME * Adds getEnv to read env variables with default value, adds test for jenkins GetLog() implementation * Adds possibility to read errorJsons; updates splunk package for log files (WIP) * Uncommenting dev code * Adds GetRequest function which holds the response in memory (not saved to disk) * Implements GetLog() function for ADO, adds function to read PipelineRuntime * PAT has been revoked * Changes http package, s.t. if password only is required basic auth works too * Adds env variable for azure token, error handling in case of unauthenticated/nil response * Adds logging output in case env variable can not be read and fallback variable needs to be used * Adds usage of environment variables for auth, uses jenkins api * Adds init functionality for orchestrators, updates GetLog() and GetPipelineStartTime() function * Adds initaliziation function for orchestrator authetnication * Adds settings struct for orchestrator authentication * Adds function to whole logfile to Splunk * Struct for pipeline related telemetry information * Increase messagebatch size to 10k * Changes splunk package to a pointer based implementation, updates generated files and corresponding template and tests for splunk * Changes telemetry package to pointer based implementation to have multiple telemetry objects, adjusted tests and splunk implementation * Changes content type to txt * Send telemetry independent of logfiles, increases amount of messages per file * Adds JobURL for orchestrators and UnknownOrchestrator as fallback * telemetry makes use of orchestrator specific information * Adds orchestrator independent correlationID * Adds custom fields for pipeline status * go fmt * Removes env var test - no env variables are read anymore * Use UnknownOrchestratorConfigProvider in case the orchestrator can not be initalized * Removes Custom fields from telemetry as these can not be reflected in SWA * Adds custom telemetry information (piperHash,..) to each step telemetry information * Removes falltrough in case no orchestrator has been found * Updates tests for orchestrator package * Adds orchestrator import in generated files * Updates generator files for internal library * Adds orchestrator telemetry information to steps * Updates generated files, fatalHook writes to cpe * Go generate from master, go fmt * Adds Custom Data field LastErrorCode * Removes GetLog() test * Update init_unix.go * Update docker_integration_test_executor.go * Update integration_api_cli_test.go * Reverts go1.17 fmt formatting * Reverts go1.17 fmt formatting * Reverts go1.17 fmt formatting * Renames customTelemetryData to stepTelemetryData * Adjustments to orchestrator-package, cleanup, adds JobName * Adjusts commonPipelineEnvironment path * Adds pipelineTelemetry struct to telemetry package, removes pipeline telemetry structs from splunk package * Go fmt * Changes path for errorDetails, adds debug information * Removes custom fields from step, adds orchestrator, commithash to baseMetadata * Adjusts tests for telemetry package * Adds tests for orchestrator * Updates generated files, initalization of splunk client only if its available in the config * Fixes typo in helper go * Update pkg/http/downloader.go * Update pkg/http/downloader.go * Update pkg/log/fatalHook.go * Update fatalHook.go * Update pkg/splunk/splunk.go * Update pkg/telemetry/data.go * Adds GetBuildStatus() and GetAPIInformation() to orchestrators * error formatting * Bugfix: dont send telemetry data if disabled, adjusts test * go fmt * Use correct error handling * Update pkg/telemetry/telemetry.go * Fixes telemetry disabled in the tests * Fixes http tests * Log fatal errors to logFile * Adds CustomReportingConfig to hooks * Cleanup comments in splunk package * Adds possibility to send telemetry to custom endpoint * Adds debug output for the payload * Debug output for the payload as a string * Adds test cases for changes in telemetry package * go fmt * Adds generated files for new step * Reverts changes for http tests, causing problems with go1.15, changes need to be applied for newer go version >=1.17 * Adjusts test for sonarExecuteScan * Adjusts test for sonarExecuteScan * Adds explanation for customreportingConfig * Makes disableing of customSend more obvious * Adds custom step reporting to each step, updates generated files, adjusts helper testdata * fixes unit test wrong usage of logging * Send pipeline data altough there has been no error, adjust test cases * Reverts changes for customReporting * Updates generated files, removes customReporting * Removes writing errorDetails to CPE * Reverts usage of customreporting * go fmt * reverts changes in http_test * reverts changes in http_test * Skips integration cnb test Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-11-18 18:50:03 +02:00
newmanExecute(stepConfig, &stepTelemetryData, &influx)
stepTelemetryData.ErrorCode = "0"
feat(newmanExecute): golang implmementation for newmanExecute (#2513) * Automates first parts of newmanExecute.groovy Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds newman installation Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Removes warning Signed-off-by: Fabian Reh <fabian.reh@sap.com> * makes tests robust for later shell calls Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds version logging Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds tests for version logging Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds newman shell execution Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Prepare cloud foundry apps with secrets handling Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds further process to CF Utils Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Fixes unit test Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds error category Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Add fix to execute step locally Currently only tested on windows machine locally in powershell. Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adapt unit test to fix of runCommand Signed-off-by: Fabian Reh <fabian.reh@sap.com> * refactored golang step to newmanExecute * wip * added test config * refactored newmanExecute groovy wrapper step * exclude newmanExecute from common step test * cleaups * add credential support * fix groovy credential providing * add import * add stageName * define script * remove unused vars * add import * fix iterator ref * golang secret handling and cleanups * wip * wip * wip * update go step * implement cf credential proposal * testRepository functionality implemented * register secrets to logger * add missing dependecies * test xsuaa credential handling * wip * wip * cleanups * add import * remove mandatory params * add container definition * test runCommand * test runCommand * fix npm path * fix npm path * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * added newmanEnvironment to templating * wip * use env and globals params in runCommand when no templating * fix condition * wip * reverted config edit * updated documentation * install with shell * wip * wip * fix tests * refactor tests * wip * remove old test * wip * escape go tmpl * escape go tmpl * fix defaults * add doc comment * remove test case * refactored newman commands * add cli reporter * refactor options * mock os getenv and fix all tests * refactoring and doc update * go generate * small refactor * spelling * fix newman doc * remove MaskPasswords wrapper; fix stash bug; * docu fix Co-authored-by: Fabian Reh <fabian.reh@sap.com>
2021-03-17 09:08:33 +02:00
log.Entry().Info("SUCCESS")
},
}
addNewmanExecuteFlags(createNewmanExecuteCmd, &stepConfig)
return createNewmanExecuteCmd
}
func addNewmanExecuteFlags(cmd *cobra.Command, stepConfig *newmanExecuteOptions) {
cmd.Flags().StringVar(&stepConfig.NewmanCollection, "newmanCollection", `**/*.postman_collection.json`, "The test collection that should be executed. This could also be a file pattern.")
cmd.Flags().StringVar(&stepConfig.NewmanRunCommand, "newmanRunCommand", os.Getenv("PIPER_newmanRunCommand"), "+++ Deprecated +++ Please use list parameter `runOptions` instead.")
cmd.Flags().StringSliceVar(&stepConfig.RunOptions, "runOptions", []string{`run`, `{{.NewmanCollection}}`, `--reporters`, `cli,junit,html`, `--reporter-junit-export`, `target/newman/TEST-{{.CollectionDisplayName}}.xml`, `--reporter-html-export`, `target/newman/TEST-{{.CollectionDisplayName}}.html`}, "The newman command that will be executed inside the docker container. Env vars can be passed via template as in \"{{getenv MY_ENV_VAR}}\".")
feat(newmanExecute): golang implmementation for newmanExecute (#2513) * Automates first parts of newmanExecute.groovy Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds newman installation Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Removes warning Signed-off-by: Fabian Reh <fabian.reh@sap.com> * makes tests robust for later shell calls Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds version logging Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds tests for version logging Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds newman shell execution Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Prepare cloud foundry apps with secrets handling Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds further process to CF Utils Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Fixes unit test Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds error category Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Add fix to execute step locally Currently only tested on windows machine locally in powershell. Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adapt unit test to fix of runCommand Signed-off-by: Fabian Reh <fabian.reh@sap.com> * refactored golang step to newmanExecute * wip * added test config * refactored newmanExecute groovy wrapper step * exclude newmanExecute from common step test * cleaups * add credential support * fix groovy credential providing * add import * add stageName * define script * remove unused vars * add import * fix iterator ref * golang secret handling and cleanups * wip * wip * wip * update go step * implement cf credential proposal * testRepository functionality implemented * register secrets to logger * add missing dependecies * test xsuaa credential handling * wip * wip * cleanups * add import * remove mandatory params * add container definition * test runCommand * test runCommand * fix npm path * fix npm path * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * added newmanEnvironment to templating * wip * use env and globals params in runCommand when no templating * fix condition * wip * reverted config edit * updated documentation * install with shell * wip * wip * fix tests * refactor tests * wip * remove old test * wip * escape go tmpl * escape go tmpl * fix defaults * add doc comment * remove test case * refactored newman commands * add cli reporter * refactor options * mock os getenv and fix all tests * refactoring and doc update * go generate * small refactor * spelling * fix newman doc * remove MaskPasswords wrapper; fix stash bug; * docu fix Co-authored-by: Fabian Reh <fabian.reh@sap.com>
2021-03-17 09:08:33 +02:00
cmd.Flags().StringVar(&stepConfig.NewmanInstallCommand, "newmanInstallCommand", `npm install newman newman-reporter-html --global --quiet`, "The shell command that will be executed inside the docker container to install Newman.")
cmd.Flags().StringVar(&stepConfig.NewmanEnvironment, "newmanEnvironment", os.Getenv("PIPER_newmanEnvironment"), "Specify an environment file path or URL. Environments provide a set of variables that one can use within collections.")
cmd.Flags().StringVar(&stepConfig.NewmanGlobals, "newmanGlobals", os.Getenv("PIPER_newmanGlobals"), "Specify the file path or URL for global variables. Global variables are similar to environment variables but have a lower precedence and can be overridden by environment variables having the same name.")
cmd.Flags().BoolVar(&stepConfig.FailOnError, "failOnError", true, "Defines the behavior, in case tests fail.")
cmd.Flags().StringSliceVar(&stepConfig.CfAppsWithSecrets, "cfAppsWithSecrets", []string{}, "List of CloudFoundry apps with secrets")
}
// retrieve step metadata
func newmanExecuteMetadata() config.StepData {
var theMetaData = config.StepData{
Metadata: config.StepMetadata{
Name: "newmanExecute",
Aliases: []config.Alias{},
Description: "Installs newman and executes specified newman collections.",
},
Spec: config.StepSpec{
Inputs: config.StepInputs{
Resources: []config.StepResources{
{Name: "tests", Type: "stash"},
},
Parameters: []config.StepParameters{
{
Name: "newmanCollection",
ResourceRef: []config.ResourceReference{},
Scope: []string{"PARAMETERS", "STAGES", "STEPS"},
Type: "string",
Mandatory: false,
Aliases: []config.Alias{},
Default: `**/*.postman_collection.json`,
feat(newmanExecute): golang implmementation for newmanExecute (#2513) * Automates first parts of newmanExecute.groovy Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds newman installation Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Removes warning Signed-off-by: Fabian Reh <fabian.reh@sap.com> * makes tests robust for later shell calls Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds version logging Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds tests for version logging Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds newman shell execution Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Prepare cloud foundry apps with secrets handling Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds further process to CF Utils Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Fixes unit test Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds error category Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Add fix to execute step locally Currently only tested on windows machine locally in powershell. Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adapt unit test to fix of runCommand Signed-off-by: Fabian Reh <fabian.reh@sap.com> * refactored golang step to newmanExecute * wip * added test config * refactored newmanExecute groovy wrapper step * exclude newmanExecute from common step test * cleaups * add credential support * fix groovy credential providing * add import * add stageName * define script * remove unused vars * add import * fix iterator ref * golang secret handling and cleanups * wip * wip * wip * update go step * implement cf credential proposal * testRepository functionality implemented * register secrets to logger * add missing dependecies * test xsuaa credential handling * wip * wip * cleanups * add import * remove mandatory params * add container definition * test runCommand * test runCommand * fix npm path * fix npm path * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * added newmanEnvironment to templating * wip * use env and globals params in runCommand when no templating * fix condition * wip * reverted config edit * updated documentation * install with shell * wip * wip * fix tests * refactor tests * wip * remove old test * wip * escape go tmpl * escape go tmpl * fix defaults * add doc comment * remove test case * refactored newman commands * add cli reporter * refactor options * mock os getenv and fix all tests * refactoring and doc update * go generate * small refactor * spelling * fix newman doc * remove MaskPasswords wrapper; fix stash bug; * docu fix Co-authored-by: Fabian Reh <fabian.reh@sap.com>
2021-03-17 09:08:33 +02:00
},
{
Name: "newmanRunCommand",
ResourceRef: []config.ResourceReference{},
Scope: []string{"PARAMETERS", "STAGES", "STEPS"},
Type: "string",
Mandatory: false,
Aliases: []config.Alias{},
Default: os.Getenv("PIPER_newmanRunCommand"),
feat(newmanExecute): golang implmementation for newmanExecute (#2513) * Automates first parts of newmanExecute.groovy Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds newman installation Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Removes warning Signed-off-by: Fabian Reh <fabian.reh@sap.com> * makes tests robust for later shell calls Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds version logging Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds tests for version logging Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds newman shell execution Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Prepare cloud foundry apps with secrets handling Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds further process to CF Utils Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Fixes unit test Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds error category Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Add fix to execute step locally Currently only tested on windows machine locally in powershell. Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adapt unit test to fix of runCommand Signed-off-by: Fabian Reh <fabian.reh@sap.com> * refactored golang step to newmanExecute * wip * added test config * refactored newmanExecute groovy wrapper step * exclude newmanExecute from common step test * cleaups * add credential support * fix groovy credential providing * add import * add stageName * define script * remove unused vars * add import * fix iterator ref * golang secret handling and cleanups * wip * wip * wip * update go step * implement cf credential proposal * testRepository functionality implemented * register secrets to logger * add missing dependecies * test xsuaa credential handling * wip * wip * cleanups * add import * remove mandatory params * add container definition * test runCommand * test runCommand * fix npm path * fix npm path * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * added newmanEnvironment to templating * wip * use env and globals params in runCommand when no templating * fix condition * wip * reverted config edit * updated documentation * install with shell * wip * wip * fix tests * refactor tests * wip * remove old test * wip * escape go tmpl * escape go tmpl * fix defaults * add doc comment * remove test case * refactored newman commands * add cli reporter * refactor options * mock os getenv and fix all tests * refactoring and doc update * go generate * small refactor * spelling * fix newman doc * remove MaskPasswords wrapper; fix stash bug; * docu fix Co-authored-by: Fabian Reh <fabian.reh@sap.com>
2021-03-17 09:08:33 +02:00
},
{
Name: "runOptions",
ResourceRef: []config.ResourceReference{},
Scope: []string{"PARAMETERS", "STAGES", "STEPS"},
Type: "[]string",
Mandatory: false,
Aliases: []config.Alias{},
Default: []string{`run`, `{{.NewmanCollection}}`, `--reporters`, `cli,junit,html`, `--reporter-junit-export`, `target/newman/TEST-{{.CollectionDisplayName}}.xml`, `--reporter-html-export`, `target/newman/TEST-{{.CollectionDisplayName}}.html`},
feat(newmanExecute): golang implmementation for newmanExecute (#2513) * Automates first parts of newmanExecute.groovy Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds newman installation Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Removes warning Signed-off-by: Fabian Reh <fabian.reh@sap.com> * makes tests robust for later shell calls Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds version logging Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds tests for version logging Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds newman shell execution Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Prepare cloud foundry apps with secrets handling Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds further process to CF Utils Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Fixes unit test Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds error category Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Add fix to execute step locally Currently only tested on windows machine locally in powershell. Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adapt unit test to fix of runCommand Signed-off-by: Fabian Reh <fabian.reh@sap.com> * refactored golang step to newmanExecute * wip * added test config * refactored newmanExecute groovy wrapper step * exclude newmanExecute from common step test * cleaups * add credential support * fix groovy credential providing * add import * add stageName * define script * remove unused vars * add import * fix iterator ref * golang secret handling and cleanups * wip * wip * wip * update go step * implement cf credential proposal * testRepository functionality implemented * register secrets to logger * add missing dependecies * test xsuaa credential handling * wip * wip * cleanups * add import * remove mandatory params * add container definition * test runCommand * test runCommand * fix npm path * fix npm path * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * added newmanEnvironment to templating * wip * use env and globals params in runCommand when no templating * fix condition * wip * reverted config edit * updated documentation * install with shell * wip * wip * fix tests * refactor tests * wip * remove old test * wip * escape go tmpl * escape go tmpl * fix defaults * add doc comment * remove test case * refactored newman commands * add cli reporter * refactor options * mock os getenv and fix all tests * refactoring and doc update * go generate * small refactor * spelling * fix newman doc * remove MaskPasswords wrapper; fix stash bug; * docu fix Co-authored-by: Fabian Reh <fabian.reh@sap.com>
2021-03-17 09:08:33 +02:00
},
{
Name: "newmanInstallCommand",
ResourceRef: []config.ResourceReference{},
Scope: []string{"PARAMETERS", "STAGES", "STEPS"},
Type: "string",
Mandatory: false,
Aliases: []config.Alias{},
Default: `npm install newman newman-reporter-html --global --quiet`,
feat(newmanExecute): golang implmementation for newmanExecute (#2513) * Automates first parts of newmanExecute.groovy Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds newman installation Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Removes warning Signed-off-by: Fabian Reh <fabian.reh@sap.com> * makes tests robust for later shell calls Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds version logging Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds tests for version logging Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds newman shell execution Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Prepare cloud foundry apps with secrets handling Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds further process to CF Utils Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Fixes unit test Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds error category Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Add fix to execute step locally Currently only tested on windows machine locally in powershell. Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adapt unit test to fix of runCommand Signed-off-by: Fabian Reh <fabian.reh@sap.com> * refactored golang step to newmanExecute * wip * added test config * refactored newmanExecute groovy wrapper step * exclude newmanExecute from common step test * cleaups * add credential support * fix groovy credential providing * add import * add stageName * define script * remove unused vars * add import * fix iterator ref * golang secret handling and cleanups * wip * wip * wip * update go step * implement cf credential proposal * testRepository functionality implemented * register secrets to logger * add missing dependecies * test xsuaa credential handling * wip * wip * cleanups * add import * remove mandatory params * add container definition * test runCommand * test runCommand * fix npm path * fix npm path * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * added newmanEnvironment to templating * wip * use env and globals params in runCommand when no templating * fix condition * wip * reverted config edit * updated documentation * install with shell * wip * wip * fix tests * refactor tests * wip * remove old test * wip * escape go tmpl * escape go tmpl * fix defaults * add doc comment * remove test case * refactored newman commands * add cli reporter * refactor options * mock os getenv and fix all tests * refactoring and doc update * go generate * small refactor * spelling * fix newman doc * remove MaskPasswords wrapper; fix stash bug; * docu fix Co-authored-by: Fabian Reh <fabian.reh@sap.com>
2021-03-17 09:08:33 +02:00
},
{
Name: "newmanEnvironment",
ResourceRef: []config.ResourceReference{},
Scope: []string{"PARAMETERS", "STAGES", "STEPS"},
Type: "string",
Mandatory: false,
Aliases: []config.Alias{},
Default: os.Getenv("PIPER_newmanEnvironment"),
feat(newmanExecute): golang implmementation for newmanExecute (#2513) * Automates first parts of newmanExecute.groovy Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds newman installation Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Removes warning Signed-off-by: Fabian Reh <fabian.reh@sap.com> * makes tests robust for later shell calls Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds version logging Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds tests for version logging Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds newman shell execution Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Prepare cloud foundry apps with secrets handling Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds further process to CF Utils Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Fixes unit test Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds error category Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Add fix to execute step locally Currently only tested on windows machine locally in powershell. Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adapt unit test to fix of runCommand Signed-off-by: Fabian Reh <fabian.reh@sap.com> * refactored golang step to newmanExecute * wip * added test config * refactored newmanExecute groovy wrapper step * exclude newmanExecute from common step test * cleaups * add credential support * fix groovy credential providing * add import * add stageName * define script * remove unused vars * add import * fix iterator ref * golang secret handling and cleanups * wip * wip * wip * update go step * implement cf credential proposal * testRepository functionality implemented * register secrets to logger * add missing dependecies * test xsuaa credential handling * wip * wip * cleanups * add import * remove mandatory params * add container definition * test runCommand * test runCommand * fix npm path * fix npm path * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * added newmanEnvironment to templating * wip * use env and globals params in runCommand when no templating * fix condition * wip * reverted config edit * updated documentation * install with shell * wip * wip * fix tests * refactor tests * wip * remove old test * wip * escape go tmpl * escape go tmpl * fix defaults * add doc comment * remove test case * refactored newman commands * add cli reporter * refactor options * mock os getenv and fix all tests * refactoring and doc update * go generate * small refactor * spelling * fix newman doc * remove MaskPasswords wrapper; fix stash bug; * docu fix Co-authored-by: Fabian Reh <fabian.reh@sap.com>
2021-03-17 09:08:33 +02:00
},
{
Name: "newmanGlobals",
ResourceRef: []config.ResourceReference{},
Scope: []string{"PARAMETERS", "STAGES", "STEPS"},
Type: "string",
Mandatory: false,
Aliases: []config.Alias{},
Default: os.Getenv("PIPER_newmanGlobals"),
feat(newmanExecute): golang implmementation for newmanExecute (#2513) * Automates first parts of newmanExecute.groovy Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds newman installation Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Removes warning Signed-off-by: Fabian Reh <fabian.reh@sap.com> * makes tests robust for later shell calls Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds version logging Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds tests for version logging Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds newman shell execution Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Prepare cloud foundry apps with secrets handling Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds further process to CF Utils Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Fixes unit test Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds error category Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Add fix to execute step locally Currently only tested on windows machine locally in powershell. Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adapt unit test to fix of runCommand Signed-off-by: Fabian Reh <fabian.reh@sap.com> * refactored golang step to newmanExecute * wip * added test config * refactored newmanExecute groovy wrapper step * exclude newmanExecute from common step test * cleaups * add credential support * fix groovy credential providing * add import * add stageName * define script * remove unused vars * add import * fix iterator ref * golang secret handling and cleanups * wip * wip * wip * update go step * implement cf credential proposal * testRepository functionality implemented * register secrets to logger * add missing dependecies * test xsuaa credential handling * wip * wip * cleanups * add import * remove mandatory params * add container definition * test runCommand * test runCommand * fix npm path * fix npm path * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * added newmanEnvironment to templating * wip * use env and globals params in runCommand when no templating * fix condition * wip * reverted config edit * updated documentation * install with shell * wip * wip * fix tests * refactor tests * wip * remove old test * wip * escape go tmpl * escape go tmpl * fix defaults * add doc comment * remove test case * refactored newman commands * add cli reporter * refactor options * mock os getenv and fix all tests * refactoring and doc update * go generate * small refactor * spelling * fix newman doc * remove MaskPasswords wrapper; fix stash bug; * docu fix Co-authored-by: Fabian Reh <fabian.reh@sap.com>
2021-03-17 09:08:33 +02:00
},
{
Name: "failOnError",
ResourceRef: []config.ResourceReference{},
Scope: []string{"PARAMETERS", "STAGES", "STEPS"},
Type: "bool",
Mandatory: false,
Aliases: []config.Alias{},
Default: true,
feat(newmanExecute): golang implmementation for newmanExecute (#2513) * Automates first parts of newmanExecute.groovy Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds newman installation Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Removes warning Signed-off-by: Fabian Reh <fabian.reh@sap.com> * makes tests robust for later shell calls Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds version logging Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds tests for version logging Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds newman shell execution Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Prepare cloud foundry apps with secrets handling Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds further process to CF Utils Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Fixes unit test Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds error category Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Add fix to execute step locally Currently only tested on windows machine locally in powershell. Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adapt unit test to fix of runCommand Signed-off-by: Fabian Reh <fabian.reh@sap.com> * refactored golang step to newmanExecute * wip * added test config * refactored newmanExecute groovy wrapper step * exclude newmanExecute from common step test * cleaups * add credential support * fix groovy credential providing * add import * add stageName * define script * remove unused vars * add import * fix iterator ref * golang secret handling and cleanups * wip * wip * wip * update go step * implement cf credential proposal * testRepository functionality implemented * register secrets to logger * add missing dependecies * test xsuaa credential handling * wip * wip * cleanups * add import * remove mandatory params * add container definition * test runCommand * test runCommand * fix npm path * fix npm path * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * added newmanEnvironment to templating * wip * use env and globals params in runCommand when no templating * fix condition * wip * reverted config edit * updated documentation * install with shell * wip * wip * fix tests * refactor tests * wip * remove old test * wip * escape go tmpl * escape go tmpl * fix defaults * add doc comment * remove test case * refactored newman commands * add cli reporter * refactor options * mock os getenv and fix all tests * refactoring and doc update * go generate * small refactor * spelling * fix newman doc * remove MaskPasswords wrapper; fix stash bug; * docu fix Co-authored-by: Fabian Reh <fabian.reh@sap.com>
2021-03-17 09:08:33 +02:00
},
{
Name: "cfAppsWithSecrets",
ResourceRef: []config.ResourceReference{},
Scope: []string{"PARAMETERS", "STAGES", "STEPS"},
Type: "[]string",
Mandatory: false,
Aliases: []config.Alias{},
Default: []string{},
feat(newmanExecute): golang implmementation for newmanExecute (#2513) * Automates first parts of newmanExecute.groovy Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds newman installation Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Removes warning Signed-off-by: Fabian Reh <fabian.reh@sap.com> * makes tests robust for later shell calls Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds version logging Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds tests for version logging Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds newman shell execution Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Prepare cloud foundry apps with secrets handling Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds further process to CF Utils Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Fixes unit test Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds error category Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Add fix to execute step locally Currently only tested on windows machine locally in powershell. Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adapt unit test to fix of runCommand Signed-off-by: Fabian Reh <fabian.reh@sap.com> * refactored golang step to newmanExecute * wip * added test config * refactored newmanExecute groovy wrapper step * exclude newmanExecute from common step test * cleaups * add credential support * fix groovy credential providing * add import * add stageName * define script * remove unused vars * add import * fix iterator ref * golang secret handling and cleanups * wip * wip * wip * update go step * implement cf credential proposal * testRepository functionality implemented * register secrets to logger * add missing dependecies * test xsuaa credential handling * wip * wip * cleanups * add import * remove mandatory params * add container definition * test runCommand * test runCommand * fix npm path * fix npm path * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * added newmanEnvironment to templating * wip * use env and globals params in runCommand when no templating * fix condition * wip * reverted config edit * updated documentation * install with shell * wip * wip * fix tests * refactor tests * wip * remove old test * wip * escape go tmpl * escape go tmpl * fix defaults * add doc comment * remove test case * refactored newman commands * add cli reporter * refactor options * mock os getenv and fix all tests * refactoring and doc update * go generate * small refactor * spelling * fix newman doc * remove MaskPasswords wrapper; fix stash bug; * docu fix Co-authored-by: Fabian Reh <fabian.reh@sap.com>
2021-03-17 09:08:33 +02:00
},
},
},
Containers: []config.Container{
{Name: "newman", Image: "node:lts-buster", WorkingDir: "/home/node"},
feat(newmanExecute): golang implmementation for newmanExecute (#2513) * Automates first parts of newmanExecute.groovy Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds newman installation Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Removes warning Signed-off-by: Fabian Reh <fabian.reh@sap.com> * makes tests robust for later shell calls Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds version logging Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds tests for version logging Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds newman shell execution Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Prepare cloud foundry apps with secrets handling Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds further process to CF Utils Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Fixes unit test Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds error category Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Add fix to execute step locally Currently only tested on windows machine locally in powershell. Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adapt unit test to fix of runCommand Signed-off-by: Fabian Reh <fabian.reh@sap.com> * refactored golang step to newmanExecute * wip * added test config * refactored newmanExecute groovy wrapper step * exclude newmanExecute from common step test * cleaups * add credential support * fix groovy credential providing * add import * add stageName * define script * remove unused vars * add import * fix iterator ref * golang secret handling and cleanups * wip * wip * wip * update go step * implement cf credential proposal * testRepository functionality implemented * register secrets to logger * add missing dependecies * test xsuaa credential handling * wip * wip * cleanups * add import * remove mandatory params * add container definition * test runCommand * test runCommand * fix npm path * fix npm path * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * added newmanEnvironment to templating * wip * use env and globals params in runCommand when no templating * fix condition * wip * reverted config edit * updated documentation * install with shell * wip * wip * fix tests * refactor tests * wip * remove old test * wip * escape go tmpl * escape go tmpl * fix defaults * add doc comment * remove test case * refactored newman commands * add cli reporter * refactor options * mock os getenv and fix all tests * refactoring and doc update * go generate * small refactor * spelling * fix newman doc * remove MaskPasswords wrapper; fix stash bug; * docu fix Co-authored-by: Fabian Reh <fabian.reh@sap.com>
2021-03-17 09:08:33 +02:00
},
Outputs: config.StepOutputs{
Resources: []config.StepResources{
{
Name: "influx",
Type: "influx",
Parameters: []map[string]interface{}{
{"name": "step_data", "fields": []map[string]string{{"name": "newman"}}},
},
},
{
Name: "reports",
Type: "reports",
Parameters: []map[string]interface{}{
{"filePattern": "**/TEST-*.xml", "type": "acceptance-test"},
{"filePattern": "**/requirement.mapping", "type": "requirement-mapping"},
{"filePattern": "**/delivery.mapping", "type": "delivery-mapping"},
},
},
},
},
feat(newmanExecute): golang implmementation for newmanExecute (#2513) * Automates first parts of newmanExecute.groovy Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds newman installation Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Removes warning Signed-off-by: Fabian Reh <fabian.reh@sap.com> * makes tests robust for later shell calls Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds version logging Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds tests for version logging Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds newman shell execution Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Prepare cloud foundry apps with secrets handling Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds further process to CF Utils Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Fixes unit test Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adds error category Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Add fix to execute step locally Currently only tested on windows machine locally in powershell. Signed-off-by: Fabian Reh <fabian.reh@sap.com> * Adapt unit test to fix of runCommand Signed-off-by: Fabian Reh <fabian.reh@sap.com> * refactored golang step to newmanExecute * wip * added test config * refactored newmanExecute groovy wrapper step * exclude newmanExecute from common step test * cleaups * add credential support * fix groovy credential providing * add import * add stageName * define script * remove unused vars * add import * fix iterator ref * golang secret handling and cleanups * wip * wip * wip * update go step * implement cf credential proposal * testRepository functionality implemented * register secrets to logger * add missing dependecies * test xsuaa credential handling * wip * wip * cleanups * add import * remove mandatory params * add container definition * test runCommand * test runCommand * fix npm path * fix npm path * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * added newmanEnvironment to templating * wip * use env and globals params in runCommand when no templating * fix condition * wip * reverted config edit * updated documentation * install with shell * wip * wip * fix tests * refactor tests * wip * remove old test * wip * escape go tmpl * escape go tmpl * fix defaults * add doc comment * remove test case * refactored newman commands * add cli reporter * refactor options * mock os getenv and fix all tests * refactoring and doc update * go generate * small refactor * spelling * fix newman doc * remove MaskPasswords wrapper; fix stash bug; * docu fix Co-authored-by: Fabian Reh <fabian.reh@sap.com>
2021-03-17 09:08:33 +02:00
},
}
return theMetaData
}