mirror of
https://github.com/SAP/jenkins-library.git
synced 2025-04-23 12:19:04 +02:00
Provide first parts for golang implementation (#905)
* Provide first parts for golang implementation
This commit is contained in:
parent
8e987c46e1
commit
c1eb9f5c70
@ -18,6 +18,12 @@ plugins:
|
|||||||
strings:
|
strings:
|
||||||
- TODO
|
- TODO
|
||||||
- FIXME
|
- FIXME
|
||||||
|
gofmt:
|
||||||
|
enabled: true
|
||||||
|
golint:
|
||||||
|
enabled: true
|
||||||
|
govet:
|
||||||
|
enabled: true
|
||||||
markdownlint:
|
markdownlint:
|
||||||
enabled: true
|
enabled: true
|
||||||
checks:
|
checks:
|
||||||
|
@ -22,3 +22,6 @@ indent_size = none
|
|||||||
[cfg/id_rsa.enc]
|
[cfg/id_rsa.enc]
|
||||||
indent_style = none
|
indent_style = none
|
||||||
indent_size = none
|
indent_size = none
|
||||||
|
[{go.mod,go.sum,*.go}]
|
||||||
|
indent_style = tab
|
||||||
|
indent_size = 8
|
||||||
|
2
.gitignore
vendored
2
.gitignore
vendored
@ -17,3 +17,5 @@ targets/
|
|||||||
documentation/docs-gen
|
documentation/docs-gen
|
||||||
|
|
||||||
consumer-test/**/workspace
|
consumer-test/**/workspace
|
||||||
|
|
||||||
|
*.code-workspace
|
||||||
|
@ -27,7 +27,9 @@ jobs:
|
|||||||
- curl -L --output cc-test-reporter https://codeclimate.com/downloads/test-reporter/test-reporter-latest-linux-amd64
|
- curl -L --output cc-test-reporter https://codeclimate.com/downloads/test-reporter/test-reporter-latest-linux-amd64
|
||||||
- chmod +x ./cc-test-reporter
|
- chmod +x ./cc-test-reporter
|
||||||
- ./cc-test-reporter before-build
|
- ./cc-test-reporter before-build
|
||||||
script: mvn package --batch-mode
|
script:
|
||||||
|
- docker build -t piper:latest .
|
||||||
|
- mvn package --batch-mode
|
||||||
after_script:
|
after_script:
|
||||||
- JACOCO_SOURCE_PATH="src vars test" ./cc-test-reporter format-coverage target/site/jacoco/jacoco.xml --input-type jacoco
|
- JACOCO_SOURCE_PATH="src vars test" ./cc-test-reporter format-coverage target/site/jacoco/jacoco.xml --input-type jacoco
|
||||||
- ./cc-test-reporter upload-coverage
|
- ./cc-test-reporter upload-coverage
|
||||||
|
131
DEVELOPMENT.md
Normal file
131
DEVELOPMENT.md
Normal file
@ -0,0 +1,131 @@
|
|||||||
|
# Development
|
||||||
|
|
||||||
|
**Table of contents:**
|
||||||
|
|
||||||
|
1. [Getting started](#getting-started)
|
||||||
|
1. [Build the project](#build-the-project_)
|
||||||
|
1. [Logging](#logging)
|
||||||
|
1. [Error handling](#error-handling)
|
||||||
|
|
||||||
|
## Getting started
|
||||||
|
|
||||||
|
1. [Ramp up your development environment](#ramp-up)
|
||||||
|
1. [Get familiar with Go language](#go-basics)
|
||||||
|
1. Create [a GitHub account](https://github.com/join)
|
||||||
|
1. Setup [GitHub access via SSH](https://help.github.com/articles/connecting-to-github-with-ssh/)
|
||||||
|
1. [Create and checkout a repo fork](#checkout-your-fork)
|
||||||
|
1. Optional: [Get Jenkins related environment](#jenkins-environment)
|
||||||
|
1. Optional: [Get familiar with Jenkins Pipelines as Code](#jenkins-pipelines)
|
||||||
|
|
||||||
|
### Ramp up
|
||||||
|
|
||||||
|
First you need to set up an appropriate development environment:
|
||||||
|
|
||||||
|
Install Go, see [GO Getting Started](https://golang.org/doc/install)
|
||||||
|
|
||||||
|
Install an IDE with Go plugins, see for example [Go in Visual Studio Code](https://code.visualstudio.com/docs/languages/go)
|
||||||
|
|
||||||
|
### Go basics
|
||||||
|
|
||||||
|
In order to get yourself started, there is a lot of useful information out there.
|
||||||
|
|
||||||
|
As a first step to take we highly recommend the [Golang documentation](https://golang.org/doc/) especially, [A Tour of Go](https://tour.golang.org/welcome/1)
|
||||||
|
|
||||||
|
We have a strong focus on high quality software and contributions without adequate tests will not be accepted.
|
||||||
|
There is an excellent resource which teaches Go using a test-driven approach: [Learn Go with Tests](https://github.com/quii/learn-go-with-tests)
|
||||||
|
|
||||||
|
### Checkout your fork
|
||||||
|
|
||||||
|
The project uses [Go modules](https://blog.golang.org/using-go-modules). Thus please make sure to **NOT** checkout the project into your [`GOPATH`](https://github.com/golang/go/wiki/SettingGOPATH).
|
||||||
|
|
||||||
|
To check out this repository:
|
||||||
|
|
||||||
|
1. Create your own
|
||||||
|
[fork of this repo](https://help.github.com/articles/fork-a-repo/)
|
||||||
|
1. Clone it to your machine, for example like:
|
||||||
|
|
||||||
|
```shell
|
||||||
|
mkdir -p ${HOME}/projects/jenkins-library
|
||||||
|
cd ${HOME}/projects
|
||||||
|
git clone git@github.com:${YOUR_GITHUB_USERNAME}/jenkins-library.git
|
||||||
|
cd jenkins-library
|
||||||
|
git remote add upstream git@github.com:sap/jenkins-library.git
|
||||||
|
git remote set-url --push upstream no_push
|
||||||
|
```
|
||||||
|
|
||||||
|
### Jenkins environment
|
||||||
|
|
||||||
|
If you want to contribute also to the Jenkins-specific parts like
|
||||||
|
|
||||||
|
* Jenkins library step
|
||||||
|
* Jenkins pipeline integration
|
||||||
|
|
||||||
|
you need to do the following in addition:
|
||||||
|
|
||||||
|
* [Install Groovy](https://groovy-lang.org/install.html)
|
||||||
|
* [Install Maven](https://maven.apache.org/install.html)
|
||||||
|
* Get a local Jenkins installed: Use for example [cx-server](toDo: add link)
|
||||||
|
|
||||||
|
### Jenkins pipelines
|
||||||
|
|
||||||
|
The Jenkins related parts depend on
|
||||||
|
|
||||||
|
* [Jenkins Pipelines as Code](https://jenkins.io/doc/book/pipeline-as-code/)
|
||||||
|
* [Jenkins Shared Libraries](https://jenkins.io/doc/book/pipeline/shared-libraries/)
|
||||||
|
|
||||||
|
You should get familiar with these concepts for contributing to the Jenkins-specific parts.
|
||||||
|
|
||||||
|
## Build the project
|
||||||
|
|
||||||
|
### Build the executable suitable for the CI/CD Linux target environments
|
||||||
|
|
||||||
|
Use Docker:
|
||||||
|
|
||||||
|
`docker build -t piper:latest .`
|
||||||
|
|
||||||
|
You can extract the binary using Docker means to your local filesystem:
|
||||||
|
|
||||||
|
```
|
||||||
|
docker create --name piper piper:latest
|
||||||
|
docker cp piper:/piper .
|
||||||
|
docker rm piper
|
||||||
|
```
|
||||||
|
|
||||||
|
## Generating step framework
|
||||||
|
|
||||||
|
The steps are generated based on the yaml files in `resources/metadata/` with the following command
|
||||||
|
`go run pkg/generator/step-metadata.go`.
|
||||||
|
|
||||||
|
The yaml format is kept pretty close to Tekton's [task format](https://github.com/tektoncd/pipeline/blob/master/docs/tasks.md).
|
||||||
|
Where the Tekton format was not sufficient some extenstions have been made.
|
||||||
|
|
||||||
|
Examples are:
|
||||||
|
|
||||||
|
* matadata - longDescription
|
||||||
|
* spec - inputs - secrets
|
||||||
|
* spec - containers
|
||||||
|
* spec - sidecars
|
||||||
|
|
||||||
|
## Logging
|
||||||
|
|
||||||
|
to be added
|
||||||
|
|
||||||
|
## Error handling
|
||||||
|
|
||||||
|
In order to better understand the root cause of errors that occur we wrap errors like
|
||||||
|
|
||||||
|
```golang
|
||||||
|
f, err := os.Open(path)
|
||||||
|
if err != nil {
|
||||||
|
return errors.Wrapf(err, "open failed for %v", path)
|
||||||
|
}
|
||||||
|
defer f.Close()
|
||||||
|
```
|
||||||
|
|
||||||
|
We use [github.com/pkg/errors](https://github.com/pkg/errors) for that.
|
||||||
|
|
||||||
|
## Testing
|
||||||
|
|
||||||
|
Unit tests are done using basic `golang` means.
|
||||||
|
|
||||||
|
Additionally we encourage you to use [github.com/stretchr/testify/assert](https://github.com/stretchr/testify/assert) in order to have slimmer assertions if you like.
|
14
Dockerfile
Normal file
14
Dockerfile
Normal file
@ -0,0 +1,14 @@
|
|||||||
|
FROM golang:1.13 AS build-env
|
||||||
|
COPY . /build
|
||||||
|
WORKDIR /build
|
||||||
|
|
||||||
|
# execute tests
|
||||||
|
RUN go test ./... -cover
|
||||||
|
|
||||||
|
## ONLY tests so far, building to be added later
|
||||||
|
# execute build
|
||||||
|
# RUN go build -o piper
|
||||||
|
|
||||||
|
# FROM gcr.io/distroless/base:latest
|
||||||
|
# COPY --from=build-env /build/piper /piper
|
||||||
|
# ENTRYPOINT ["/piper"]
|
11
go.mod
Normal file
11
go.mod
Normal file
@ -0,0 +1,11 @@
|
|||||||
|
module github.com/SAP/jenkins-library
|
||||||
|
|
||||||
|
go 1.13
|
||||||
|
|
||||||
|
require (
|
||||||
|
github.com/ghodss/yaml v1.0.0
|
||||||
|
github.com/pkg/errors v0.8.1
|
||||||
|
github.com/spf13/cobra v0.0.5
|
||||||
|
github.com/spf13/pflag v1.0.5
|
||||||
|
github.com/stretchr/testify v1.2.2
|
||||||
|
)
|
42
go.sum
Normal file
42
go.sum
Normal file
@ -0,0 +1,42 @@
|
|||||||
|
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
|
||||||
|
github.com/armon/consul-api v0.0.0-20180202201655-eb2c6b5be1b6/go.mod h1:grANhF5doyWs3UAsr3K4I6qtAmlQcZDesFNEHPZAzj8=
|
||||||
|
github.com/coreos/etcd v3.3.10+incompatible/go.mod h1:uF7uidLiAD3TWHmW31ZFd/JWoc32PjwdhPthX9715RE=
|
||||||
|
github.com/coreos/go-etcd v2.0.0+incompatible/go.mod h1:Jez6KQU2B/sWsbdaef3ED8NzMklzPG4d5KIOhIy30Tk=
|
||||||
|
github.com/coreos/go-semver v0.2.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk=
|
||||||
|
github.com/cpuguy83/go-md2man v1.0.10/go.mod h1:SmD6nW6nTyfqj6ABTjUi3V3JVMnlJmwcJI5acqYI6dE=
|
||||||
|
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
|
||||||
|
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||||
|
github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
|
||||||
|
github.com/ghodss/yaml v1.0.0 h1:wQHKEahhL6wmXdzwWG11gIVCkOv05bNOh+Rxn0yngAk=
|
||||||
|
github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
|
||||||
|
github.com/hashicorp/hcl v1.0.0/go.mod h1:E5yfLk+7swimpb2L/Alb/PJmXilQ/rhwaUYs4T20WEQ=
|
||||||
|
github.com/inconshreveable/mousetrap v1.0.0 h1:Z8tu5sraLXCXIcARxBp/8cbvlwVa7Z1NHg9XEKhtSvM=
|
||||||
|
github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8=
|
||||||
|
github.com/magiconair/properties v1.8.0/go.mod h1:PppfXfuXeibc/6YijjN8zIbojt8czPbwD3XqdrwzmxQ=
|
||||||
|
github.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0=
|
||||||
|
github.com/mitchellh/mapstructure v1.1.2/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y=
|
||||||
|
github.com/pelletier/go-toml v1.2.0/go.mod h1:5z9KED0ma1S8pY6P1sdut58dfprrGBbd/94hg7ilaic=
|
||||||
|
github.com/pkg/errors v0.8.1 h1:iURUrRGxPUNPdy5/HRSm+Yj6okJ6UtLINN0Q9M4+h3I=
|
||||||
|
github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
|
||||||
|
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
|
||||||
|
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
||||||
|
github.com/russross/blackfriday v1.5.2/go.mod h1:JO/DiYxRf+HjHt06OyowR9PTA263kcR/rfWxYHBV53g=
|
||||||
|
github.com/spf13/afero v1.1.2/go.mod h1:j4pytiNVoe2o6bmDsKpLACNPDBIoEAkihy7loJ1B0CQ=
|
||||||
|
github.com/spf13/cast v1.3.0/go.mod h1:Qx5cxh0v+4UWYiBimWS+eyWzqEqokIECu5etghLkUJE=
|
||||||
|
github.com/spf13/cobra v0.0.5 h1:f0B+LkLX6DtmRH1isoNA9VTtNUK9K8xYd28JNNfOv/s=
|
||||||
|
github.com/spf13/cobra v0.0.5/go.mod h1:3K3wKZymM7VvHMDS9+Akkh4K60UwM26emMESw8tLCHU=
|
||||||
|
github.com/spf13/jwalterweatherman v1.0.0/go.mod h1:cQK4TGJAtQXfYWX+Ddv3mKDzgVb68N+wFjFa4jdeBTo=
|
||||||
|
github.com/spf13/pflag v1.0.3/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4=
|
||||||
|
github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA=
|
||||||
|
github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
|
||||||
|
github.com/spf13/viper v1.3.2/go.mod h1:ZiWeW+zYFKm7srdB9IoDzzZXaJaI5eL9QjNiN/DMA2s=
|
||||||
|
github.com/stretchr/testify v1.2.2 h1:bSDNvY7ZPG5RlJ8otE/7V6gMiyenm9RtJ7IUVIAoJ1w=
|
||||||
|
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
|
||||||
|
github.com/ugorji/go/codec v0.0.0-20181204163529-d75b2dcb6bc8/go.mod h1:VFNgLljTbGfSG7qAOspJ7OScBnGdDN/yBr0sguwnwf0=
|
||||||
|
github.com/xordataexchange/crypt v0.0.3-0.20170626215501-b2862e3d0a77/go.mod h1:aYKd//L2LvnjZzWKhF00oedf4jCCReLcmhLdhm1A27Q=
|
||||||
|
golang.org/x/crypto v0.0.0-20181203042331-505ab145d0a9/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
|
||||||
|
golang.org/x/sys v0.0.0-20181205085412-a5c9d58dba9a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
|
||||||
|
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
|
||||||
|
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
||||||
|
gopkg.in/yaml.v2 v2.2.2 h1:ZCJp+EgiOT7lHqUV2J862kp8Qj64Jo6az82+3Td9dZw=
|
||||||
|
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
|
189
pkg/config/config.go
Normal file
189
pkg/config/config.go
Normal file
@ -0,0 +1,189 @@
|
|||||||
|
package config
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"io/ioutil"
|
||||||
|
"os"
|
||||||
|
|
||||||
|
"github.com/ghodss/yaml"
|
||||||
|
"github.com/pkg/errors"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Config defines the structure of the config files
|
||||||
|
type Config struct {
|
||||||
|
General map[string]interface{} `json:"general"`
|
||||||
|
Stages map[string]map[string]interface{} `json:"stages"`
|
||||||
|
Steps map[string]map[string]interface{} `json:"steps"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// StepConfig defines the structure for merged step configuration
|
||||||
|
type StepConfig struct {
|
||||||
|
Config map[string]interface{}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ReadConfig loads config and returns its content
|
||||||
|
func (c *Config) ReadConfig(configuration io.ReadCloser) error {
|
||||||
|
defer configuration.Close()
|
||||||
|
|
||||||
|
content, err := ioutil.ReadAll(configuration)
|
||||||
|
if err != nil {
|
||||||
|
return errors.Wrapf(err, "error reading %v", configuration)
|
||||||
|
}
|
||||||
|
|
||||||
|
err = yaml.Unmarshal(content, &c)
|
||||||
|
if err != nil {
|
||||||
|
return NewParseError(fmt.Sprintf("error unmarshalling %q: %v", content, err))
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// GetStepConfig provides merged step configuration using defaults, config, if available
|
||||||
|
func (c *Config) GetStepConfig(flagValues map[string]interface{}, paramJSON string, configuration io.ReadCloser, defaults []io.ReadCloser, filters StepFilters, stageName, stepName string) (StepConfig, error) {
|
||||||
|
var stepConfig StepConfig
|
||||||
|
var d PipelineDefaults
|
||||||
|
|
||||||
|
if err := c.ReadConfig(configuration); err != nil {
|
||||||
|
switch err.(type) {
|
||||||
|
case *ParseError:
|
||||||
|
return StepConfig{}, errors.Wrap(err, "failed to parse custom pipeline configuration")
|
||||||
|
default:
|
||||||
|
//ignoring unavailability of config file since considered optional
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := d.ReadPipelineDefaults(defaults); err != nil {
|
||||||
|
switch err.(type) {
|
||||||
|
case *ParseError:
|
||||||
|
return StepConfig{}, errors.Wrap(err, "failed to parse pipeline default configuration")
|
||||||
|
default:
|
||||||
|
//ignoring unavailability of defaults since considered optional
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// first: read defaults & merge general -> steps (-> general -> steps ...)
|
||||||
|
for _, def := range d.Defaults {
|
||||||
|
stepConfig.mixIn(def.General, filters.General)
|
||||||
|
stepConfig.mixIn(def.Steps[stepName], filters.Steps)
|
||||||
|
}
|
||||||
|
|
||||||
|
// second: read config & merge - general -> steps -> stages
|
||||||
|
stepConfig.mixIn(c.General, filters.General)
|
||||||
|
stepConfig.mixIn(c.Steps[stepName], filters.Steps)
|
||||||
|
stepConfig.mixIn(c.Stages[stageName], filters.Stages)
|
||||||
|
|
||||||
|
// third: merge parameters provided via env vars
|
||||||
|
stepConfig.mixIn(envValues(filters.All), filters.All)
|
||||||
|
|
||||||
|
// fourth: if parameters are provided in JSON format merge them
|
||||||
|
if len(paramJSON) != 0 {
|
||||||
|
var params map[string]interface{}
|
||||||
|
json.Unmarshal([]byte(paramJSON), ¶ms)
|
||||||
|
stepConfig.mixIn(params, filters.Parameters)
|
||||||
|
}
|
||||||
|
|
||||||
|
// fifth: merge command line flags
|
||||||
|
if flagValues != nil {
|
||||||
|
stepConfig.mixIn(flagValues, filters.Parameters)
|
||||||
|
}
|
||||||
|
|
||||||
|
return stepConfig, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// GetStepConfigWithJSON provides merged step configuration using a provided stepConfigJSON with additional flags provided
|
||||||
|
func GetStepConfigWithJSON(flagValues map[string]interface{}, stepConfigJSON string, filters StepFilters) StepConfig {
|
||||||
|
var stepConfig StepConfig
|
||||||
|
|
||||||
|
stepConfigMap := map[string]interface{}{}
|
||||||
|
|
||||||
|
json.Unmarshal([]byte(stepConfigJSON), &stepConfigMap)
|
||||||
|
|
||||||
|
stepConfig.mixIn(stepConfigMap, filters.All)
|
||||||
|
|
||||||
|
// ToDo: mix in parametersJSON
|
||||||
|
|
||||||
|
if flagValues != nil {
|
||||||
|
stepConfig.mixIn(flagValues, filters.Parameters)
|
||||||
|
}
|
||||||
|
return stepConfig
|
||||||
|
}
|
||||||
|
|
||||||
|
// GetJSON returns JSON representation of an object
|
||||||
|
func GetJSON(data interface{}) (string, error) {
|
||||||
|
|
||||||
|
result, err := json.Marshal(data)
|
||||||
|
if err != nil {
|
||||||
|
return "", errors.Wrapf(err, "error marshalling json: %v", err)
|
||||||
|
}
|
||||||
|
return string(result), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func envValues(filter []string) map[string]interface{} {
|
||||||
|
vals := map[string]interface{}{}
|
||||||
|
for _, param := range filter {
|
||||||
|
if envVal := os.Getenv("PIPER_" + param); len(envVal) != 0 {
|
||||||
|
vals[param] = os.Getenv("PIPER_" + param)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return vals
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *StepConfig) mixIn(mergeData map[string]interface{}, filter []string) {
|
||||||
|
|
||||||
|
if s.Config == nil {
|
||||||
|
s.Config = map[string]interface{}{}
|
||||||
|
}
|
||||||
|
|
||||||
|
s.Config = filterMap(merge(s.Config, mergeData), filter)
|
||||||
|
}
|
||||||
|
|
||||||
|
func filterMap(data map[string]interface{}, filter []string) map[string]interface{} {
|
||||||
|
result := map[string]interface{}{}
|
||||||
|
|
||||||
|
if data == nil {
|
||||||
|
data = map[string]interface{}{}
|
||||||
|
}
|
||||||
|
|
||||||
|
for key, value := range data {
|
||||||
|
if len(filter) == 0 || sliceContains(filter, key) {
|
||||||
|
result[key] = value
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return result
|
||||||
|
}
|
||||||
|
|
||||||
|
func merge(base, overlay map[string]interface{}) map[string]interface{} {
|
||||||
|
|
||||||
|
result := map[string]interface{}{}
|
||||||
|
|
||||||
|
if base == nil {
|
||||||
|
base = map[string]interface{}{}
|
||||||
|
}
|
||||||
|
|
||||||
|
for key, value := range base {
|
||||||
|
result[key] = value
|
||||||
|
}
|
||||||
|
|
||||||
|
for key, value := range overlay {
|
||||||
|
if val, ok := value.(map[string]interface{}); ok {
|
||||||
|
if valBaseKey, ok := base[key].(map[string]interface{}); !ok {
|
||||||
|
result[key] = merge(map[string]interface{}{}, val)
|
||||||
|
} else {
|
||||||
|
result[key] = merge(valBaseKey, val)
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
result[key] = value
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return result
|
||||||
|
}
|
||||||
|
|
||||||
|
func sliceContains(slice []string, find string) bool {
|
||||||
|
for _, elem := range slice {
|
||||||
|
if elem == find {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
253
pkg/config/config_test.go
Normal file
253
pkg/config/config_test.go
Normal file
@ -0,0 +1,253 @@
|
|||||||
|
package config
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"io/ioutil"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/stretchr/testify/assert"
|
||||||
|
)
|
||||||
|
|
||||||
|
type errReadCloser int
|
||||||
|
|
||||||
|
func (errReadCloser) Read(p []byte) (n int, err error) {
|
||||||
|
return 0, errors.New("read error")
|
||||||
|
}
|
||||||
|
|
||||||
|
func (errReadCloser) Close() error {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestReadConfig(t *testing.T) {
|
||||||
|
|
||||||
|
var c Config
|
||||||
|
|
||||||
|
t.Run("Success case", func(t *testing.T) {
|
||||||
|
|
||||||
|
myConfig := strings.NewReader("general:\n generalTestKey: generalTestValue\nsteps:\n testStep:\n testStepKey: testStepValue")
|
||||||
|
|
||||||
|
err := c.ReadConfig(ioutil.NopCloser(myConfig)) // NopCloser "no-ops" the closing interface since strings do not need to be closed
|
||||||
|
if err != nil {
|
||||||
|
t.Errorf("Got error although no error expected: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if c.General["generalTestKey"] != "generalTestValue" {
|
||||||
|
t.Errorf("General config- got: %v, expected: %v", c.General["generalTestKey"], "generalTestValue")
|
||||||
|
}
|
||||||
|
|
||||||
|
if c.Steps["testStep"]["testStepKey"] != "testStepValue" {
|
||||||
|
t.Errorf("Step config - got: %v, expected: %v", c.Steps["testStep"]["testStepKey"], "testStepValue")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("Read failure", func(t *testing.T) {
|
||||||
|
var rc errReadCloser
|
||||||
|
err := c.ReadConfig(rc)
|
||||||
|
if err == nil {
|
||||||
|
t.Errorf("Got no error although error expected.")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("Unmarshalling failure", func(t *testing.T) {
|
||||||
|
myConfig := strings.NewReader("general:\n generalTestKey: generalTestValue\nsteps:\n testStep:\n\ttestStepKey: testStepValue")
|
||||||
|
err := c.ReadConfig(ioutil.NopCloser(myConfig))
|
||||||
|
if err == nil {
|
||||||
|
t.Errorf("Got no error although error expected.")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestGetStepConfig(t *testing.T) {
|
||||||
|
|
||||||
|
t.Run("Success case", func(t *testing.T) {
|
||||||
|
|
||||||
|
testConfig := `general:
|
||||||
|
p3: p3_general
|
||||||
|
px3: px3_general
|
||||||
|
p4: p4_general
|
||||||
|
steps:
|
||||||
|
step1:
|
||||||
|
p4: p4_step
|
||||||
|
px4: px4_step
|
||||||
|
p5: p5_step
|
||||||
|
stages:
|
||||||
|
stage1:
|
||||||
|
p5: p5_stage
|
||||||
|
px5: px5_stage
|
||||||
|
p6: p6_stage
|
||||||
|
`
|
||||||
|
filters := StepFilters{
|
||||||
|
General: []string{"p0", "p1", "p2", "p3", "p4"},
|
||||||
|
Steps: []string{"p0", "p1", "p2", "p3", "p4", "p5"},
|
||||||
|
Stages: []string{"p0", "p1", "p2", "p3", "p4", "p5", "p6"},
|
||||||
|
Parameters: []string{"p0", "p1", "p2", "p3", "p4", "p5", "p6", "p7"},
|
||||||
|
Env: []string{"p0", "p1", "p2", "p3", "p4", "p5"},
|
||||||
|
}
|
||||||
|
|
||||||
|
defaults1 := `general:
|
||||||
|
p0: p0_general_default
|
||||||
|
px0: px0_general_default
|
||||||
|
p1: p1_general_default
|
||||||
|
steps:
|
||||||
|
step1:
|
||||||
|
p1: p1_step_default
|
||||||
|
px1: px1_step_default
|
||||||
|
p2: p2_step_default
|
||||||
|
`
|
||||||
|
|
||||||
|
defaults2 := `general:
|
||||||
|
p2: p2_general_default
|
||||||
|
px2: px2_general_default
|
||||||
|
p3: p3_general_default
|
||||||
|
`
|
||||||
|
paramJSON := `{"p6":"p6_param","p7":"p7_param"}`
|
||||||
|
|
||||||
|
flags := map[string]interface{}{"p7": "p7_flag"}
|
||||||
|
|
||||||
|
var c Config
|
||||||
|
defaults := []io.ReadCloser{ioutil.NopCloser(strings.NewReader(defaults1)), ioutil.NopCloser(strings.NewReader(defaults2))}
|
||||||
|
|
||||||
|
myConfig := ioutil.NopCloser(strings.NewReader(testConfig))
|
||||||
|
stepConfig, err := c.GetStepConfig(flags, paramJSON, myConfig, defaults, filters, "stage1", "step1")
|
||||||
|
|
||||||
|
assert.Equal(t, nil, err, "error occured but none expected")
|
||||||
|
|
||||||
|
t.Run("Config", func(t *testing.T) {
|
||||||
|
expected := map[string]string{
|
||||||
|
"p0": "p0_general_default",
|
||||||
|
"p1": "p1_step_default",
|
||||||
|
"p2": "p2_general_default",
|
||||||
|
"p3": "p3_general",
|
||||||
|
"p4": "p4_step",
|
||||||
|
"p5": "p5_stage",
|
||||||
|
"p6": "p6_param",
|
||||||
|
"p7": "p7_flag",
|
||||||
|
}
|
||||||
|
for k, v := range expected {
|
||||||
|
t.Run(k, func(t *testing.T) {
|
||||||
|
if stepConfig.Config[k] != v {
|
||||||
|
t.Errorf("got: %v, expected: %v", stepConfig.Config[k], v)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("Config not expected", func(t *testing.T) {
|
||||||
|
notExpectedKeys := []string{"px0", "px1", "px2", "px3", "px4", "px5"}
|
||||||
|
for _, p := range notExpectedKeys {
|
||||||
|
t.Run(p, func(t *testing.T) {
|
||||||
|
if stepConfig.Config[p] != nil {
|
||||||
|
t.Errorf("unexpected: %v", p)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
})
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("Failure case config", func(t *testing.T) {
|
||||||
|
var c Config
|
||||||
|
myConfig := ioutil.NopCloser(strings.NewReader("invalid config"))
|
||||||
|
_, err := c.GetStepConfig(nil, "", myConfig, nil, StepFilters{}, "stage1", "step1")
|
||||||
|
assert.EqualError(t, err, "failed to parse custom pipeline configuration: error unmarshalling \"invalid config\": error unmarshaling JSON: json: cannot unmarshal string into Go value of type config.Config", "default error expected")
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("Failure case defaults", func(t *testing.T) {
|
||||||
|
var c Config
|
||||||
|
myConfig := ioutil.NopCloser(strings.NewReader(""))
|
||||||
|
myDefaults := []io.ReadCloser{ioutil.NopCloser(strings.NewReader("invalid defaults"))}
|
||||||
|
_, err := c.GetStepConfig(nil, "", myConfig, myDefaults, StepFilters{}, "stage1", "step1")
|
||||||
|
assert.EqualError(t, err, "failed to parse pipeline default configuration: error unmarshalling \"invalid defaults\": error unmarshaling JSON: json: cannot unmarshal string into Go value of type config.Config", "default error expected")
|
||||||
|
})
|
||||||
|
|
||||||
|
//ToDo: test merging of env and parameters/flags
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestGetStepConfigWithJSON(t *testing.T) {
|
||||||
|
|
||||||
|
filters := StepFilters{All: []string{"key1"}}
|
||||||
|
|
||||||
|
t.Run("Without flags", func(t *testing.T) {
|
||||||
|
sc := GetStepConfigWithJSON(nil, `"key1":"value1","key2":"value2"`, filters)
|
||||||
|
|
||||||
|
if sc.Config["key1"] != "value1" && sc.Config["key2"] == "value2" {
|
||||||
|
t.Errorf("got: %v, expected: %v", sc.Config, StepConfig{Config: map[string]interface{}{"key1": "value1"}})
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("With flags", func(t *testing.T) {
|
||||||
|
flags := map[string]interface{}{"key1": "flagVal1"}
|
||||||
|
sc := GetStepConfigWithJSON(flags, `"key1":"value1","key2":"value2"`, filters)
|
||||||
|
if sc.Config["key1"] != "flagVal1" {
|
||||||
|
t.Errorf("got: %v, expected: %v", sc.Config["key1"], "flagVal1")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestGetJSON(t *testing.T) {
|
||||||
|
|
||||||
|
t.Run("Success case", func(t *testing.T) {
|
||||||
|
custom := map[string]interface{}{"key1": "value1"}
|
||||||
|
json, err := GetJSON(custom)
|
||||||
|
if err != nil {
|
||||||
|
t.Errorf("Got error although no error expected: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if json != `{"key1":"value1"}` {
|
||||||
|
t.Errorf("got: %v, expected: %v", json, `{"key1":"value1"}`)
|
||||||
|
}
|
||||||
|
|
||||||
|
})
|
||||||
|
t.Run("Marshalling failure", func(t *testing.T) {
|
||||||
|
_, err := GetJSON(make(chan int))
|
||||||
|
if err == nil {
|
||||||
|
t.Errorf("Got no error although error expected")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestMerge(t *testing.T) {
|
||||||
|
|
||||||
|
testTable := []struct {
|
||||||
|
Source map[string]interface{}
|
||||||
|
Filter []string
|
||||||
|
MergeData map[string]interface{}
|
||||||
|
ExpectedOutput map[string]interface{}
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
Source: map[string]interface{}{"key1": "baseValue"},
|
||||||
|
Filter: []string{},
|
||||||
|
MergeData: map[string]interface{}{"key1": "overwrittenValue"},
|
||||||
|
ExpectedOutput: map[string]interface{}{"key1": "overwrittenValue"},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
Source: map[string]interface{}{"key1": "value1"},
|
||||||
|
Filter: []string{},
|
||||||
|
MergeData: map[string]interface{}{"key2": "value2"},
|
||||||
|
ExpectedOutput: map[string]interface{}{"key1": "value1", "key2": "value2"},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
Source: map[string]interface{}{"key1": "value1"},
|
||||||
|
Filter: []string{"key1"},
|
||||||
|
MergeData: map[string]interface{}{"key2": "value2"},
|
||||||
|
ExpectedOutput: map[string]interface{}{"key1": "value1"},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
Source: map[string]interface{}{"key1": map[string]interface{}{"key1_1": "value1"}},
|
||||||
|
Filter: []string{},
|
||||||
|
MergeData: map[string]interface{}{"key1": map[string]interface{}{"key1_2": "value2"}},
|
||||||
|
ExpectedOutput: map[string]interface{}{"key1": map[string]interface{}{"key1_1": "value1", "key1_2": "value2"}},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, row := range testTable {
|
||||||
|
t.Run(fmt.Sprintf("Merging %v into %v", row.MergeData, row.Source), func(t *testing.T) {
|
||||||
|
stepConfig := StepConfig{Config: row.Source}
|
||||||
|
stepConfig.mixIn(row.MergeData, row.Filter)
|
||||||
|
assert.Equal(t, row.ExpectedOutput, stepConfig.Config, "Mixin was incorrect")
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
40
pkg/config/defaults.go
Normal file
40
pkg/config/defaults.go
Normal file
@ -0,0 +1,40 @@
|
|||||||
|
package config
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"io/ioutil"
|
||||||
|
|
||||||
|
"github.com/ghodss/yaml"
|
||||||
|
"github.com/pkg/errors"
|
||||||
|
)
|
||||||
|
|
||||||
|
// PipelineDefaults defines the structure of the pipeline defaults
|
||||||
|
type PipelineDefaults struct {
|
||||||
|
Defaults []Config `json:"defaults"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// ReadPipelineDefaults loads defaults and returns its content
|
||||||
|
func (d *PipelineDefaults) ReadPipelineDefaults(defaultSources []io.ReadCloser) error {
|
||||||
|
|
||||||
|
for _, def := range defaultSources {
|
||||||
|
|
||||||
|
defer def.Close()
|
||||||
|
|
||||||
|
var c Config
|
||||||
|
var err error
|
||||||
|
|
||||||
|
content, err := ioutil.ReadAll(def)
|
||||||
|
if err != nil {
|
||||||
|
return errors.Wrapf(err, "error reading %v", def)
|
||||||
|
}
|
||||||
|
|
||||||
|
err = yaml.Unmarshal(content, &c)
|
||||||
|
if err != nil {
|
||||||
|
return NewParseError(fmt.Sprintf("error unmarshalling %q: %v", content, err))
|
||||||
|
}
|
||||||
|
|
||||||
|
d.Defaults = append(d.Defaults, c)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
53
pkg/config/defaults_test.go
Normal file
53
pkg/config/defaults_test.go
Normal file
@ -0,0 +1,53 @@
|
|||||||
|
package config
|
||||||
|
|
||||||
|
import (
|
||||||
|
"io"
|
||||||
|
"io/ioutil"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestReadPipelineDefaults(t *testing.T) {
|
||||||
|
|
||||||
|
var d PipelineDefaults
|
||||||
|
|
||||||
|
t.Run("Success case", func(t *testing.T) {
|
||||||
|
d0 := strings.NewReader("general:\n testStepKey1: testStepValue1")
|
||||||
|
d1 := strings.NewReader("general:\n testStepKey2: testStepValue2")
|
||||||
|
err := d.ReadPipelineDefaults([]io.ReadCloser{ioutil.NopCloser(d0), ioutil.NopCloser(d1)})
|
||||||
|
|
||||||
|
if err != nil {
|
||||||
|
t.Errorf("Got error although no error expected: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
t.Run("Defaults 0", func(t *testing.T) {
|
||||||
|
expected := "testStepValue1"
|
||||||
|
if d.Defaults[0].General["testStepKey1"] != expected {
|
||||||
|
t.Errorf("got: %v, expected: %v", d.Defaults[0].General["testStepKey1"], expected)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("Defaults 1", func(t *testing.T) {
|
||||||
|
expected := "testStepValue2"
|
||||||
|
if d.Defaults[1].General["testStepKey2"] != expected {
|
||||||
|
t.Errorf("got: %v, expected: %v", d.Defaults[1].General["testStepKey2"], expected)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("Read failure", func(t *testing.T) {
|
||||||
|
var rc errReadCloser
|
||||||
|
err := d.ReadPipelineDefaults([]io.ReadCloser{rc})
|
||||||
|
if err == nil {
|
||||||
|
t.Errorf("Got no error although error expected.")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("Unmarshalling failure", func(t *testing.T) {
|
||||||
|
myConfig := strings.NewReader("general:\n\ttestStepKey: testStepValue")
|
||||||
|
err := d.ReadPipelineDefaults([]io.ReadCloser{ioutil.NopCloser(myConfig)})
|
||||||
|
if err == nil {
|
||||||
|
t.Errorf("Got no error although error expected.")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
18
pkg/config/errors.go
Normal file
18
pkg/config/errors.go
Normal file
@ -0,0 +1,18 @@
|
|||||||
|
package config
|
||||||
|
|
||||||
|
// ParseError defines an error type for configuration parsing errors
|
||||||
|
type ParseError struct {
|
||||||
|
message string
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewParseError creates a new ParseError
|
||||||
|
func NewParseError(message string) *ParseError {
|
||||||
|
return &ParseError{
|
||||||
|
message: message,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Error returns the message of the ParseError
|
||||||
|
func (e *ParseError) Error() string {
|
||||||
|
return e.message
|
||||||
|
}
|
13
pkg/config/errors_test.go
Normal file
13
pkg/config/errors_test.go
Normal file
@ -0,0 +1,13 @@
|
|||||||
|
package config
|
||||||
|
|
||||||
|
import (
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/stretchr/testify/assert"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestParseError(t *testing.T) {
|
||||||
|
err := NewParseError("Parsing failed")
|
||||||
|
|
||||||
|
assert.Equal(t, "Parsing failed", err.Error())
|
||||||
|
}
|
43
pkg/config/flags.go
Normal file
43
pkg/config/flags.go
Normal file
@ -0,0 +1,43 @@
|
|||||||
|
package config
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
|
||||||
|
"github.com/spf13/cobra"
|
||||||
|
flag "github.com/spf13/pflag"
|
||||||
|
)
|
||||||
|
|
||||||
|
// AvailableFlagValues returns all flags incl. values which are available to the command.
|
||||||
|
func AvailableFlagValues(cmd *cobra.Command, filters *StepFilters) map[string]interface{} {
|
||||||
|
flagValues := map[string]interface{}{}
|
||||||
|
flags := cmd.Flags()
|
||||||
|
//only check flags where value has been set
|
||||||
|
flags.Visit(func(pflag *flag.Flag) {
|
||||||
|
|
||||||
|
switch pflag.Value.Type() {
|
||||||
|
case "string":
|
||||||
|
flagValues[pflag.Name] = pflag.Value.String()
|
||||||
|
case "stringSlice":
|
||||||
|
flagValues[pflag.Name], _ = flags.GetStringSlice(pflag.Name)
|
||||||
|
case "bool":
|
||||||
|
flagValues[pflag.Name], _ = flags.GetBool(pflag.Name)
|
||||||
|
default:
|
||||||
|
fmt.Printf("Meta data type not set or not known: '%v'\n", pflag.Value.Type())
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
|
filters.Parameters = append(filters.Parameters, pflag.Name)
|
||||||
|
})
|
||||||
|
return flagValues
|
||||||
|
}
|
||||||
|
|
||||||
|
// MarkFlagsWithValue marks a flag as changed if value is available for the flag through the step configuration.
|
||||||
|
func MarkFlagsWithValue(cmd *cobra.Command, stepConfig StepConfig) {
|
||||||
|
flags := cmd.Flags()
|
||||||
|
flags.VisitAll(func(pflag *flag.Flag) {
|
||||||
|
//mark as available in case default is available or config is available
|
||||||
|
if len(pflag.Value.String()) > 0 || stepConfig.Config[pflag.Name] != nil {
|
||||||
|
pflag.Changed = true
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
67
pkg/config/flags_test.go
Normal file
67
pkg/config/flags_test.go
Normal file
@ -0,0 +1,67 @@
|
|||||||
|
package config
|
||||||
|
|
||||||
|
import (
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/spf13/cobra"
|
||||||
|
"github.com/stretchr/testify/assert"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestAvailableFlagValues(t *testing.T) {
|
||||||
|
var f StepFilters
|
||||||
|
|
||||||
|
var test0 string
|
||||||
|
var test1 string
|
||||||
|
var test2 []string
|
||||||
|
var test3 bool
|
||||||
|
|
||||||
|
var c = &cobra.Command{
|
||||||
|
Use: "test",
|
||||||
|
Short: "..",
|
||||||
|
}
|
||||||
|
|
||||||
|
c.Flags().StringVar(&test0, "test0", "val0", "Test 0")
|
||||||
|
c.Flags().StringVar(&test1, "test1", "", "Test 1")
|
||||||
|
c.Flags().StringSliceVar(&test2, "test2", []string{}, "Test 2")
|
||||||
|
c.Flags().BoolVar(&test3, "test3", false, "Test 3")
|
||||||
|
|
||||||
|
c.Flags().Set("test1", "val1")
|
||||||
|
c.Flags().Set("test2", "val3_1")
|
||||||
|
c.Flags().Set("test3", "true")
|
||||||
|
|
||||||
|
v := AvailableFlagValues(c, &f)
|
||||||
|
|
||||||
|
if v["test0"] != nil {
|
||||||
|
t.Errorf("expected: 'test0' to be empty but was %v", v["test0"])
|
||||||
|
}
|
||||||
|
|
||||||
|
assert.Equal(t, "val1", v["test1"])
|
||||||
|
assert.Equal(t, []string{"val3_1"}, v["test2"])
|
||||||
|
assert.Equal(t, true, v["test3"])
|
||||||
|
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestMarkFlagsWithValue(t *testing.T) {
|
||||||
|
var test0 string
|
||||||
|
var test1 string
|
||||||
|
var test2 string
|
||||||
|
var c = &cobra.Command{
|
||||||
|
Use: "test",
|
||||||
|
Short: "..",
|
||||||
|
}
|
||||||
|
c.Flags().StringVar(&test0, "test0", "val0", "Test 0")
|
||||||
|
c.Flags().StringVar(&test1, "test1", "", "Test 1")
|
||||||
|
c.Flags().StringVar(&test2, "test2", "", "Test 2")
|
||||||
|
|
||||||
|
s := StepConfig{
|
||||||
|
Config: map[string]interface{}{
|
||||||
|
"test2": "val2",
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
MarkFlagsWithValue(c, s)
|
||||||
|
|
||||||
|
assert.Equal(t, true, c.Flags().Changed("test0"), "default not considered")
|
||||||
|
assert.Equal(t, false, c.Flags().Changed("test1"), "no value: considered as set")
|
||||||
|
assert.Equal(t, true, c.Flags().Changed("test2"), "config not considered")
|
||||||
|
}
|
139
pkg/config/stepmeta.go
Normal file
139
pkg/config/stepmeta.go
Normal file
@ -0,0 +1,139 @@
|
|||||||
|
package config
|
||||||
|
|
||||||
|
import (
|
||||||
|
"io"
|
||||||
|
"io/ioutil"
|
||||||
|
|
||||||
|
"github.com/ghodss/yaml"
|
||||||
|
"github.com/pkg/errors"
|
||||||
|
)
|
||||||
|
|
||||||
|
// StepData defines the metadata for a step, like step descriptions, parameters, ...
|
||||||
|
type StepData struct {
|
||||||
|
Metadata StepMetadata `json:"metadata"`
|
||||||
|
Spec StepSpec `json:"spec"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// StepMetadata defines the metadata for a step, like step descriptions, parameters, ...
|
||||||
|
type StepMetadata struct {
|
||||||
|
Name string `json:"name"`
|
||||||
|
Description string `json:"description"`
|
||||||
|
LongDescription string `json:"longDescription,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// StepSpec defines the spec details for a step, like step inputs, containers, sidecars, ...
|
||||||
|
type StepSpec struct {
|
||||||
|
Inputs StepInputs `json:"inputs"`
|
||||||
|
// Outputs string `json:"description,omitempty"`
|
||||||
|
Containers []StepContainers `json:"containers,omitempty"`
|
||||||
|
Sidecars []StepSidecars `json:"sidecars,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// StepInputs defines the spec details for a step, like step inputs, containers, sidecars, ...
|
||||||
|
type StepInputs struct {
|
||||||
|
Parameters []StepParameters `json:"params"`
|
||||||
|
Resources []StepResources `json:"resources,omitempty"`
|
||||||
|
Secrets []StepSecrets `json:"secrets,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// StepParameters defines the parameters for a step
|
||||||
|
type StepParameters struct {
|
||||||
|
Name string `json:"name"`
|
||||||
|
Description string `json:"description"`
|
||||||
|
LongDescription string `json:"longDescription,omitempty"`
|
||||||
|
Scope []string `json:"scope"`
|
||||||
|
Type string `json:"type"`
|
||||||
|
Mandatory bool `json:"mandatory,omitempty"`
|
||||||
|
Default interface{} `json:"default,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// StepResources defines the resources to be provided by the step context, e.g. Jenkins pipeline
|
||||||
|
type StepResources struct {
|
||||||
|
Name string `json:"name"`
|
||||||
|
Description string `json:"description,omitempty"`
|
||||||
|
Type string `json:"type,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// StepSecrets defines the secrets to be provided by the step context, e.g. Jenkins pipeline
|
||||||
|
type StepSecrets struct {
|
||||||
|
Name string `json:"name"`
|
||||||
|
Description string `json:"description,omitempty"`
|
||||||
|
Type string `json:"type,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// StepOutputs defines the outputs of a step
|
||||||
|
//type StepOutputs struct {
|
||||||
|
// Name string `json:"name"`
|
||||||
|
//}
|
||||||
|
|
||||||
|
// StepContainers defines the containers required for a step
|
||||||
|
type StepContainers struct {
|
||||||
|
Containers map[string]interface{} `json:"containers"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// StepSidecars defines any sidears required for a step
|
||||||
|
type StepSidecars struct {
|
||||||
|
Sidecars map[string]interface{} `json:"sidecars"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// StepFilters defines the filter parameters for the different sections
|
||||||
|
type StepFilters struct {
|
||||||
|
All []string
|
||||||
|
General []string
|
||||||
|
Stages []string
|
||||||
|
Steps []string
|
||||||
|
Parameters []string
|
||||||
|
Env []string
|
||||||
|
}
|
||||||
|
|
||||||
|
// ReadPipelineStepData loads step definition in yaml format
|
||||||
|
func (m *StepData) ReadPipelineStepData(metadata io.ReadCloser) error {
|
||||||
|
defer metadata.Close()
|
||||||
|
content, err := ioutil.ReadAll(metadata)
|
||||||
|
if err != nil {
|
||||||
|
return errors.Wrapf(err, "error reading %v", metadata)
|
||||||
|
}
|
||||||
|
|
||||||
|
err = yaml.Unmarshal(content, &m)
|
||||||
|
if err != nil {
|
||||||
|
return errors.Wrapf(err, "error unmarshalling: %v", err)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// GetParameterFilters retrieves all scope dependent parameter filters
|
||||||
|
func (m *StepData) GetParameterFilters() StepFilters {
|
||||||
|
var filters StepFilters
|
||||||
|
for _, param := range m.Spec.Inputs.Parameters {
|
||||||
|
filters.All = append(filters.All, param.Name)
|
||||||
|
for _, scope := range param.Scope {
|
||||||
|
switch scope {
|
||||||
|
case "GENERAL":
|
||||||
|
filters.General = append(filters.General, param.Name)
|
||||||
|
case "STEPS":
|
||||||
|
filters.Steps = append(filters.Steps, param.Name)
|
||||||
|
case "STAGES":
|
||||||
|
filters.Stages = append(filters.Stages, param.Name)
|
||||||
|
case "PARAMETERS":
|
||||||
|
filters.Parameters = append(filters.Parameters, param.Name)
|
||||||
|
case "ENV":
|
||||||
|
filters.Env = append(filters.Env, param.Name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return filters
|
||||||
|
}
|
||||||
|
|
||||||
|
// GetContextParameterFilters retrieves all scope dependent parameter filters
|
||||||
|
func (m *StepData) GetContextParameterFilters() StepFilters {
|
||||||
|
var filters StepFilters
|
||||||
|
for _, secret := range m.Spec.Inputs.Secrets {
|
||||||
|
filters.All = append(filters.All, secret.Name)
|
||||||
|
filters.General = append(filters.General, secret.Name)
|
||||||
|
filters.Steps = append(filters.Steps, secret.Name)
|
||||||
|
filters.Stages = append(filters.Stages, secret.Name)
|
||||||
|
filters.Parameters = append(filters.Parameters, secret.Name)
|
||||||
|
filters.Env = append(filters.Env, secret.Name)
|
||||||
|
}
|
||||||
|
return filters
|
||||||
|
}
|
266
pkg/config/stepmeta_test.go
Normal file
266
pkg/config/stepmeta_test.go
Normal file
@ -0,0 +1,266 @@
|
|||||||
|
package config
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"io/ioutil"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestReadPipelineStepData(t *testing.T) {
|
||||||
|
var s StepData
|
||||||
|
|
||||||
|
t.Run("Success case", func(t *testing.T) {
|
||||||
|
myMeta := strings.NewReader("metadata:\n name: testIt\nspec:\n inputs:\n params:\n - name: testParamName\n secrets:\n - name: testSecret")
|
||||||
|
err := s.ReadPipelineStepData(ioutil.NopCloser(myMeta)) // NopCloser "no-ops" the closing interface since strings do not need to be closed
|
||||||
|
|
||||||
|
if err != nil {
|
||||||
|
t.Errorf("Got error although no error expected: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
t.Run("step name", func(t *testing.T) {
|
||||||
|
if s.Metadata.Name != "testIt" {
|
||||||
|
t.Errorf("Meta name - got: %v, expected: %v", s.Metadata.Name, "testIt")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("param name", func(t *testing.T) {
|
||||||
|
if s.Spec.Inputs.Parameters[0].Name != "testParamName" {
|
||||||
|
t.Errorf("Step name - got: %v, expected: %v", s.Spec.Inputs.Parameters[0].Name, "testParamName")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("secret name", func(t *testing.T) {
|
||||||
|
if s.Spec.Inputs.Secrets[0].Name != "testSecret" {
|
||||||
|
t.Errorf("Step name - got: %v, expected: %v", s.Spec.Inputs.Secrets[0].Name, "testSecret")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("Read failure", func(t *testing.T) {
|
||||||
|
var rc errReadCloser
|
||||||
|
err := s.ReadPipelineStepData(rc)
|
||||||
|
if err == nil {
|
||||||
|
t.Errorf("Got no error although error expected.")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("Unmarshalling failure", func(t *testing.T) {
|
||||||
|
myMeta := strings.NewReader("metadata:\n\tname: testIt")
|
||||||
|
err := s.ReadPipelineStepData(ioutil.NopCloser(myMeta))
|
||||||
|
if err == nil {
|
||||||
|
t.Errorf("Got no error although error expected.")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestGetParameterFilters(t *testing.T) {
|
||||||
|
metadata1 := StepData{
|
||||||
|
Spec: StepSpec{
|
||||||
|
Inputs: StepInputs{
|
||||||
|
Parameters: []StepParameters{
|
||||||
|
{Name: "paramOne", Scope: []string{"GENERAL", "STEPS", "STAGES", "PARAMETERS", "ENV"}},
|
||||||
|
{Name: "paramTwo", Scope: []string{"STEPS", "STAGES", "PARAMETERS", "ENV"}},
|
||||||
|
{Name: "paramThree", Scope: []string{"STAGES", "PARAMETERS", "ENV"}},
|
||||||
|
{Name: "paramFour", Scope: []string{"PARAMETERS", "ENV"}},
|
||||||
|
{Name: "paramFive", Scope: []string{"ENV"}},
|
||||||
|
{Name: "paramSix"},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
metadata2 := StepData{
|
||||||
|
Spec: StepSpec{
|
||||||
|
Inputs: StepInputs{
|
||||||
|
Parameters: []StepParameters{
|
||||||
|
{Name: "paramOne", Scope: []string{"GENERAL"}},
|
||||||
|
{Name: "paramTwo", Scope: []string{"STEPS"}},
|
||||||
|
{Name: "paramThree", Scope: []string{"STAGES"}},
|
||||||
|
{Name: "paramFour", Scope: []string{"PARAMETERS"}},
|
||||||
|
{Name: "paramFive", Scope: []string{"ENV"}},
|
||||||
|
{Name: "paramSix"},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
metadata3 := StepData{
|
||||||
|
Spec: StepSpec{
|
||||||
|
Inputs: StepInputs{
|
||||||
|
Parameters: []StepParameters{},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
testTable := []struct {
|
||||||
|
Metadata StepData
|
||||||
|
ExpectedAll []string
|
||||||
|
ExpectedGeneral []string
|
||||||
|
ExpectedStages []string
|
||||||
|
ExpectedSteps []string
|
||||||
|
ExpectedParameters []string
|
||||||
|
ExpectedEnv []string
|
||||||
|
NotExpectedAll []string
|
||||||
|
NotExpectedGeneral []string
|
||||||
|
NotExpectedStages []string
|
||||||
|
NotExpectedSteps []string
|
||||||
|
NotExpectedParameters []string
|
||||||
|
NotExpectedEnv []string
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
Metadata: metadata1,
|
||||||
|
ExpectedGeneral: []string{"paramOne"},
|
||||||
|
ExpectedSteps: []string{"paramOne", "paramTwo"},
|
||||||
|
ExpectedStages: []string{"paramOne", "paramTwo", "paramThree"},
|
||||||
|
ExpectedParameters: []string{"paramOne", "paramTwo", "paramThree", "paramFour"},
|
||||||
|
ExpectedEnv: []string{"paramOne", "paramTwo", "paramThree", "paramFour", "paramFive"},
|
||||||
|
ExpectedAll: []string{"paramOne", "paramTwo", "paramThree", "paramFour", "paramFive", "paramSix"},
|
||||||
|
NotExpectedGeneral: []string{"paramTwo", "paramThree", "paramFour", "paramFive", "paramSix"},
|
||||||
|
NotExpectedSteps: []string{"paramThree", "paramFour", "paramFive", "paramSix"},
|
||||||
|
NotExpectedStages: []string{"paramFour", "paramFive", "paramSix"},
|
||||||
|
NotExpectedParameters: []string{"paramFive", "paramSix"},
|
||||||
|
NotExpectedEnv: []string{"paramSix"},
|
||||||
|
NotExpectedAll: []string{},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
Metadata: metadata2,
|
||||||
|
ExpectedGeneral: []string{"paramOne"},
|
||||||
|
ExpectedSteps: []string{"paramTwo"},
|
||||||
|
ExpectedStages: []string{"paramThree"},
|
||||||
|
ExpectedParameters: []string{"paramFour"},
|
||||||
|
ExpectedEnv: []string{"paramFive"},
|
||||||
|
ExpectedAll: []string{"paramOne", "paramTwo", "paramThree", "paramFour", "paramFive", "paramSix"},
|
||||||
|
NotExpectedGeneral: []string{"paramTwo", "paramThree", "paramFour", "paramFive", "paramSix"},
|
||||||
|
NotExpectedSteps: []string{"paramOne", "paramThree", "paramFour", "paramFive", "paramSix"},
|
||||||
|
NotExpectedStages: []string{"paramOne", "paramTwo", "paramFour", "paramFive", "paramSix"},
|
||||||
|
NotExpectedParameters: []string{"paramOne", "paramTwo", "paramThree", "paramFive", "paramSix"},
|
||||||
|
NotExpectedEnv: []string{"paramOne", "paramTwo", "paramThree", "paramFour", "paramSix"},
|
||||||
|
NotExpectedAll: []string{},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
Metadata: metadata3,
|
||||||
|
ExpectedGeneral: []string{},
|
||||||
|
ExpectedStages: []string{},
|
||||||
|
ExpectedSteps: []string{},
|
||||||
|
ExpectedParameters: []string{},
|
||||||
|
ExpectedEnv: []string{},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
for key, row := range testTable {
|
||||||
|
t.Run(fmt.Sprintf("Metadata%v", key), func(t *testing.T) {
|
||||||
|
filters := row.Metadata.GetParameterFilters()
|
||||||
|
t.Run("General", func(t *testing.T) {
|
||||||
|
for _, val := range filters.General {
|
||||||
|
if !sliceContains(row.ExpectedGeneral, val) {
|
||||||
|
t.Errorf("Creation of parameter filter failed, expected: %v to be contained in %v", val, filters.General)
|
||||||
|
}
|
||||||
|
if sliceContains(row.NotExpectedGeneral, val) {
|
||||||
|
t.Errorf("Creation of parameter filter failed, expected: %v NOT to be contained in %v", val, filters.General)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
})
|
||||||
|
t.Run("Steps", func(t *testing.T) {
|
||||||
|
for _, val := range filters.Steps {
|
||||||
|
if !sliceContains(row.ExpectedSteps, val) {
|
||||||
|
t.Errorf("Creation of parameter filter failed, expected: %v to be contained in %v", val, filters.Steps)
|
||||||
|
}
|
||||||
|
if sliceContains(row.NotExpectedSteps, val) {
|
||||||
|
t.Errorf("Creation of parameter filter failed, expected: %v NOT to be contained in %v", val, filters.Steps)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
})
|
||||||
|
t.Run("Stages", func(t *testing.T) {
|
||||||
|
for _, val := range filters.Stages {
|
||||||
|
if !sliceContains(row.ExpectedStages, val) {
|
||||||
|
t.Errorf("Creation of parameter filter failed, expected: %v to be contained in %v", val, filters.Stages)
|
||||||
|
}
|
||||||
|
if sliceContains(row.NotExpectedStages, val) {
|
||||||
|
t.Errorf("Creation of parameter filter failed, expected: %v NOT to be contained in %v", val, filters.Stages)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
})
|
||||||
|
t.Run("Parameters", func(t *testing.T) {
|
||||||
|
for _, val := range filters.Parameters {
|
||||||
|
if !sliceContains(row.ExpectedParameters, val) {
|
||||||
|
t.Errorf("Creation of parameter filter failed, expected: %v to be contained in %v", val, filters.Parameters)
|
||||||
|
}
|
||||||
|
if sliceContains(row.NotExpectedParameters, val) {
|
||||||
|
t.Errorf("Creation of parameter filter failed, expected: %v NOT to be contained in %v", val, filters.Parameters)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
})
|
||||||
|
t.Run("Env", func(t *testing.T) {
|
||||||
|
for _, val := range filters.Env {
|
||||||
|
if !sliceContains(row.ExpectedEnv, val) {
|
||||||
|
t.Errorf("Creation of parameter filter failed, expected: %v to be contained in %v", val, filters.Env)
|
||||||
|
}
|
||||||
|
if sliceContains(row.NotExpectedEnv, val) {
|
||||||
|
t.Errorf("Creation of parameter filter failed, expected: %v NOT to be contained in %v", val, filters.Env)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
})
|
||||||
|
t.Run("All", func(t *testing.T) {
|
||||||
|
for _, val := range filters.All {
|
||||||
|
if !sliceContains(row.ExpectedAll, val) {
|
||||||
|
t.Errorf("Creation of parameter filter failed, expected: %v to be contained in %v", val, filters.All)
|
||||||
|
}
|
||||||
|
if sliceContains(row.NotExpectedAll, val) {
|
||||||
|
t.Errorf("Creation of parameter filter failed, expected: %v NOT to be contained in %v", val, filters.All)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
})
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestGetContextParameterFilters(t *testing.T) {
|
||||||
|
metadata1 := StepData{
|
||||||
|
Spec: StepSpec{
|
||||||
|
Inputs: StepInputs{
|
||||||
|
Secrets: []StepSecrets{
|
||||||
|
{Name: "testSecret1", Type: "jenkins"},
|
||||||
|
{Name: "testSecret2", Type: "jenkins"},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
filters := metadata1.GetContextParameterFilters()
|
||||||
|
|
||||||
|
t.Run("Secrets", func(t *testing.T) {
|
||||||
|
for _, s := range metadata1.Spec.Inputs.Secrets {
|
||||||
|
t.Run("All", func(t *testing.T) {
|
||||||
|
if !sliceContains(filters.All, s.Name) {
|
||||||
|
t.Errorf("Creation of context filter failed, expected: %v to be contained", s.Name)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
t.Run("General", func(t *testing.T) {
|
||||||
|
if !sliceContains(filters.General, s.Name) {
|
||||||
|
t.Errorf("Creation of context filter failed, expected: %v to be contained", s.Name)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
t.Run("Step", func(t *testing.T) {
|
||||||
|
if !sliceContains(filters.Steps, s.Name) {
|
||||||
|
t.Errorf("Creation of context filter failed, expected: %v to be contained", s.Name)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
t.Run("Stages", func(t *testing.T) {
|
||||||
|
if !sliceContains(filters.Steps, s.Name) {
|
||||||
|
t.Errorf("Creation of context filter failed, expected: %v to be contained", s.Name)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
t.Run("Parameters", func(t *testing.T) {
|
||||||
|
if !sliceContains(filters.Parameters, s.Name) {
|
||||||
|
t.Errorf("Creation of context filter failed, expected: %v to be contained", s.Name)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
t.Run("Env", func(t *testing.T) {
|
||||||
|
if !sliceContains(filters.Env, s.Name) {
|
||||||
|
t.Errorf("Creation of context filter failed, expected: %v to be contained", s.Name)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
Loading…
x
Reference in New Issue
Block a user