ffeldmann
b88ebdad6c
feat(splunk) Sending telemetry and logging information to Splunk ( #2796 )
...
* Adds inital splunk hook and logCollector
* Adds documentation of the Splunk hook
* Fixes markdown lint issues and removes comment from telemetry.go file
* Fixes markdown lint issues and adds missing generated file
* Markdown linting
* Changes documentation according to review, adds Splunk token automatically during init
* Adds error handling for marshalling hook config
* Markdown lint und correct Splunk token in httpclient
* Registeres Splunk token as secret and adjusts test cases
* Adds missing error handling and removes unnecessary comments
* Creates new function readPipelineEnvironment, adds tests
* Moves MonitoringData struct, edits defaults for json fields
* Adds gitRepository and gitOwner to telemetry information
* Simplifies readCommonPipelineEnvironment function, adds more descriptive errors, adds automated adding of Splunk prefix token
* Adjusts error handling
* Cleaner error logging
2021-05-17 12:14:04 +02:00
Christopher Fenner
98fac0a455
chore(deprecation): change format of deprecation notice from config command ( #2822 )
...
* Update config.go
* Update config.go
* add library name
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-05-10 19:18:16 +02:00
Siarhei Pazdniakou
0673d3fed6
Writing secrets to vault ( #2770 )
...
Co-authored-by: Kevin Stiehl <kevin.stiehl@numericas.de>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-05-06 10:27:23 +02:00
lndrschlz
b8e9201df5
feat(vault): provide test credentials from vault in every piper go step ( #2715 )
...
* added support for test credentials
Co-authored-by: Kevin Stiehl <kevin.stiehl@numericas.de>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-04-09 10:04:35 +02:00
Oliver Nocon
24cb3c92e1
fix(config): consider defaults with higher prio than env ( #2725 )
...
* fix(config): properly handle empty values from cpe
* fix(config): consider defaults with higher prio than env
2021-03-29 10:40:04 +02:00
Kevin Stiehl
f2b0683880
add token revocation ( #2686 )
...
* add token revocation
* update mocks
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-03-10 08:36:50 +01:00
Oliver Nocon
d769fb5148
fix(config): properly handle empty values from cpe ( #2682 )
2021-03-09 09:30:18 +01:00
Kevin Stiehl
649b6d6201
[Vault] Fix ContextParamFilters for vaultSecretFile ( #2671 )
...
* fix contextFilter
Co-authored-by: lndrschlz <leander.schulz01@sap.com>
* reduce Congitive Complexity warning
Co-authored-by: lndrschlz <leander.schulz01@sap.com>
2021-03-08 10:16:32 +01:00
Kevin Stiehl
ffffe8295e
(Vault) Improvements ( #2439 )
...
* vault improvements
* Update cloudFoundryDeploy.yaml
remove double PARAMETERS
* go generate
* fix type & resturcutre paragraph to a list
* remove non-existent secrets
* build trigger
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2021-02-15 09:48:51 +01:00
Oliver Nocon
44788f1d3a
fix(config): properly read non-string parameters from environment ( #2549 )
2021-01-28 11:09:43 +01:00
Marcus Holl
ee7279c8fc
Add helper for finding uninitialized config ( #2534 )
...
Add helper for finding uninitialized config
2021-01-25 12:58:43 +01:00
Kevin Stiehl
4a92d1a96a
[Vault] Mixin Vault Parameters for default config ( #2523 )
...
* load vault filters for custom defaults
* add tests
2021-01-20 14:59:47 +01:00
lndrschlz
b101106a05
chore: hide warn level if verbose not configured ( #2466 )
2020-12-22 14:33:11 +01:00
Oliver Feldmann
4f6b99cead
Don't allow empty strings as default container parameters ( #2406 )
...
First a bug fix is addressed in which the pull policy could not be configured to false by configuring the general configuration. It could neither be configured via dockerExecute or dockerExecuteOnKubernetes, even though this parameter is docker specific. Only by configuring the specific step where one wants to set the pull policy to false can it be configured.
As the bug stems from zero values being in the context config map, which is also addressed with this PR. That is the second part: Context config parameters are only set if they have a value.
* Don't set pull image if not configured
Otherwise, if the pull policy is not set explicitly for a step, dockerPullImage is set to true. Thus, before this change, it cannot be set in the general, or in dockerExecute or in dockerExecuteOnKubernetes configuration.
* Fix unit tests
* Add pullImage parameter test
* Do not place empty default values in context config
* Use putIfNotEmpty for sidecar container options
* Export common configuration
Keys that are set by both main and sidecar container can be exported
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-12-10 11:28:39 +01:00
Kevin Stiehl
dfab156427
Feature/approle secret id rotation ( #2311 )
...
* add new step vault secret
* add debug log
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-11-17 13:49:31 +01:00
Kevin Stiehl
9d461c6753
consider alias during secret resolving ( #2327 )
2020-11-06 17:54:01 +01:00
Kevin Stiehl
67727147f8
switch to debug logging for some vault log messages ( #2299 )
2020-11-03 08:45:12 +01:00
Kevin Stiehl
24aafb0b69
add vaultSecretFileReferences ( #2203 )
...
* add vaultSecretFileReferences
* fix test
* fix test
* go generate
* remove code duplication
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-10-26 14:20:04 +01:00
Marcus Holl
14e1c44aae
Avoid having the mocks in the productive binary ( #2199 )
2020-10-21 12:48:26 +02:00
Kevin Stiehl
3eae0c5f68
feat(vault): fetch secrets from vault ( #2032 )
...
* cloud-foundry & sonar from vault
* add vault development hint
* don't abort on vault errors
* cloudfoundry make credentialsId only mandatory when vault is not configured
* add vault ref to step ymls
* rename vaultAddress to vaultServerUrl
* rename PIPER_vaultRole* to PIPER_vaultAppRole*
* add resourceRef for detect step
* fix error when no namespace is set
* added debug logs
* added debug logs
* fix vault resolving
* add vaultCustomBasePath
* rename vault_test.go to client_test.go
* refactored vault logging
* refactored config param lookup for vault
* added tüddelchen
* rename vaultCustomBasePath to vaultPath
* fix tests
* change lookup path for group secrets
* fix interpolation tests
* added vault resource ref to versioning
* execute go generate
* rename Approle to AppRole
* change verbose back to false
Co-authored-by: Leander Schulz <leander.schulz01@sap.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-10-13 14:14:47 +02:00
Christopher Fenner
be90876b7c
feat(output): handle non-string output values ( #2113 )
...
* handle non-string values as JSON
* change value type to interface in resources
* regenerate code
* add test cases
* handle reading of json files
* write json data to json files
* fix assignment
* use GetResourceParameter
* add test case
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-10-05 15:33:28 +02:00
Christopher Fenner
6999380ee3
chore(go): simplify code using gofmt -s ( #2065 )
2020-09-24 08:58:53 +02:00
Christopher Fenner
b219fb6514
fix(typo): found by misspell ( #2064 )
...
* fix typos in step yamls
* fix typos in go files
* regenerate step code
* fix typos in md files
* fix typos in groovy files
* fix further typos
2020-09-24 07:41:06 +02:00
Kevin Stiehl
d589038206
Vault AppRole login ( #1971 )
...
* added interpolation package in config
* vault allow paths to hold config references
* allow referencing properties in vaultPaths
* fix small typo
* add approleAuth
* register resolved secrets to logger
* generate steps
* clean up
* add integration test
* add vault to context filter
* reduce Cognitive Complexity & added tests
* Update pkg/config/stepmeta_test.go
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* go generate
* go generate after merge
* rename VaultAppRole* to VaultRole*
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-09-16 14:50:09 +02:00
Oliver Nocon
d691750931
Fix reading of hook configuration ( #1854 )
...
* Update piper.go
* Update piper_test.go
* Update config.go
* Update sentryHook.go
* Update piper.go
* go fmt
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-07-28 17:19:33 +02:00
Kevin Stiehl
116a91ce28
Fetch config parameters from vault ( #1782 )
...
* added secret fetching from vault
* added rootPath and vault namespaces
* write tests for vault config loading
* fix smaller bugs
* rename rootPath to basePath
* fix integration test
* make basePath optional
* write more tests
* remove unnecessary code
* make basePath configurable
* remove mixin for vault
* rewrite tests
* fix tests for searching over multiple paths
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-07-22 11:15:48 +02:00
Daniel Mieg
90571754bf
Add test cases for stepmeta.go ( #1801 )
...
* Add tests
* Remove obsolete comment
2020-07-16 11:09:49 +02:00
Oliver Nocon
43f51ba90a
Revamp documentation generation for golang steps ( #1781 )
...
* Revamp documentation generation for golang steps
* Add and update tests
* Add tests, add step outputs
* Add tests, add step outputs
* Update rendering
* fix merge conflict
* update generated files
* Update pkg/config/stepmeta.go
* Update pkg/config/stepmeta.go
* Update pkg/generator/helper/docuHelper.go
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
* Include PR feedback
* update tests
* Fix Jenkins dependecy indication
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-07-16 09:10:15 +02:00
Daniel Kurzynski
61a669778a
Apply alias to secrets ( #1802 )
2020-07-15 21:05:22 +02:00
Daniel Mieg
5dbb7be6b6
Fix commonPipelineEnvironment issue ( #1798 )
...
* Fix write to disk
* Change condition
* Test log output
* Fix
* Remove test log
2020-07-15 15:45:36 +02:00
Daniel Mieg
7f69f4eb16
Support lists in commonPipelineEnvironment ( #1797 )
...
* Enable write Lists
* Enable read
* Add tests
* Fix unit test
* CodeClimate Refactor
* Add comment
2020-07-15 10:09:42 +02:00
lndrschlz
db13b2f9ce
moved container context method to config package and expose for reuse ( #1770 )
2020-07-10 10:32:26 +02:00
Daniel Kurzynski
fb3c6b5978
Ignore wrong type of alias ( #1778 )
2020-07-09 14:13:06 +02:00
Oliver Nocon
49b7abb4ca
Fix stashing issue with binrary execution ( #1749 )
...
Co-authored-by: Daniel Kurzynski <daniel.kurzynski@sap.com>
2020-07-02 12:08:56 +02:00
Oliver Nocon
bb8cd30c4c
Extend resource references to cater for secrets ( #1718 )
...
* Add secret references to step metadata
* Update references
* update generated files
2020-06-25 16:33:38 +02:00
Daniel Kurzynski
9a30758115
Consider stage config from custom defaults ( #1670 )
2020-06-16 09:06:37 +02:00
Oliver Nocon
6116fe5528
Decouple config reading and retrieval ( #1668 )
...
There are use cases where we need to run `GetStepConfig()` multiple times.
In such cases it is more efficient to load the files once and then resolve the
respective step configuration.
2020-06-15 14:17:59 +02:00
Oliver Nocon
50b2a2d6fc
Update deprecation message ( #1656 )
2020-06-12 15:19:13 +02:00
Oliver Nocon
7e2ab744a5
mtaBuild - change default for docker image ( #1574 )
...
* mtaBuild - change default for docker image
* fix resolution of container default from step definition
* Properly fix default for container conditions
2020-05-22 09:48:11 +02:00
Oliver Nocon
a0ded365c5
Fix container condition handling ( #1562 )
...
* Add additional tests
* Update handling of containers
* Properly consider existing config at root level
* Fix parameter filtering
* Fix tests for adapted filter handling
2020-05-20 10:50:35 +02:00
Stephan Aßmus
5ab4dedcb2
Optionally ignore custom defaults ( #1538 )
...
* Add parameter "--ignoreCustomDefaults"
* Pass to piper customDefaults from config also via --defaultConfig
... and add "--ignoreCustomDefaults".
* Log output when ignoring customDefaults
Co-authored-by: Daniel Kurzynski <daniel.kurzynski@sap.com>
2020-05-14 10:50:58 +02:00
Stephan Aßmus
80d53dae79
Reading defaults in go: Make sure all io.ReadClosers are closed ( #1529 )
2020-05-12 15:46:47 +02:00
lndrschlz
39e30ff1b4
Added hook configuration; added sentry hook and tests ( #1497 )
...
* Added hook configuration; added sentry hook and tests
* steps generated by step framework
* added comments and renamed correlationID
* formatted comment on NewSentryHook method
* fix generator step tests
2020-05-05 08:36:24 +02:00
Christopher Fenner
96439972e2
docs: show possible values in step documentation ( #1453 )
...
* add possibleValues to model
* include possible values into docs
2020-04-24 14:13:02 +02:00
Daniel Kurzynski
ce4be51362
Mask secrets ( #1382 )
2020-04-16 14:37:45 +02:00
Christopher Fenner
b0144614e5
improve error message when parsing config ( #1387 )
...
* improve error message when parsing config
* adjust test case
Co-authored-by: Florian Wilhelm <florian.wilhelm02@sap.com>
2020-04-09 13:40:04 +02:00
Stephan Aßmus
da0f25fd7b
Piper-Go: Allow aliases also for inputs of type "secret" ( #1355 )
...
* Fix typos
* Support aliases also for secrets
* Adapt & extend Unit Tests
* Output deprecation warning for param/secret aliases
... if the alias is marked as 'deprecated'.
2020-04-01 20:46:33 +02:00
Oliver Nocon
91b6b0900f
Config and Defaults via http(s) ( #1302 )
...
* Config and Defaults via http(s)
Add the possibility to use http(s) urls as path to
* config.yml
* defaults
* Remove comment
* Addressing PR feedback
* Fix error handling
2020-03-31 08:47:09 +02:00
Oliver Nocon
8fbeddb26c
Golang step metadata: Config aliases for steps ( #1293 )
...
* Golang step metadata: Config aliases for steps
This will ease following scenarios:
* config migration due to step name changes
* re-use of more general config, e.g. `mavenExecute` in `mavenBuild`
* fix CodeClimate finding
* Fix panic if original stage config does not exist yet
2020-03-19 17:24:35 +01:00
Sven Merk
cbe368fe36
Checkmarx as golang ( #1075 )
...
* Added base functionality for checkmarx interaction
* Extend http client with file upload capabilities
* Latest changes
* Add debug logging
* Introduce Uploader interface
* Add tests for checkmarx client
* Hook new checkmarx command
* Improve coverage
* Add tests
* Improved test coverage and fixed code
* Add influx reporting
* Add alternation capabilities
* Add groovy step
* Try fix cmd
* Enhancements
* Fix report generation
* Final performance improvements
* Fix code
* Structure code, cleanup
* Improvements
* Fix codeclimate issue
* Update groovy
* Adapt latest changes to http
* Fix test
* Fix http tests
* Fix test
* Fix test
* Fix test 2
* Fix code
* Fix code 2
* Fix code
* Code
* Fix
* Fix
* Add report and link handling
* Fix returns, add groovy test
* Review comments
* Added doc template
* Docs update
* Remove SAP internals
* Better status display
* Add name to link
* Fix test
* Fix
* Fix verbose handling
* Fix verbose handling 2
* Fix verbose handling 3
* Fix
* Tiny improvements
* Regenerate
* Fix test
* Fix test code
* Fix verbosity issue
* Fix test
* Fix test
* Fix test
2020-01-27 23:40:53 +01:00
