Maurice Breit
9f2d8a70b1
(fix) match regexes in sliceContains to support vaultSecretNames ( #3618 )
...
* (fix) match regexes in sliceContains to support vaultSecretNames
* add test for regex matching in sliceContains
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-03-17 09:41:51 +01:00
Ralf Pannemans
e7db09db12
feat(cnbBuild): added dockerimage to the telemetry data ( #3501 )
...
Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>
Co-authored-by: Sumit Kulhadia <sumit.kulhadia@sap.com>
Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
2022-02-16 13:28:51 +01:00
Oliver Nocon
2ae1d9dac1
feat(kanikoExecute): allow building multiple images ( #3443 )
...
* feat(kanikoExecute): allow building multiple images
* enhance tests
* chore: allow running tests in parallel
* small fixes
* fix: fix destination bug
* update formatting and defaults
* fix yml formatting
* chore: change cpe parameter names
* chore: improve variable naming
2022-02-07 07:58:41 +01:00
Christopher Fenner
ebcbb73b46
feat: add command to fetch default files ( #3445 )
...
* some ideas..
* Add getDefaults command (WIP) (#3444 )
* add getYAML function for configs
* create getDefaults command(based on getConfig)
* add getDefaults command to CLI
* read defaults files, using github tokens as well
* write defaults to stdout as JSON object with YAMLs embedded
* catch case where no input files are given
* actually write output to file if outputFile is specified
* mark defaultsFile flag as required
* add basic tests
* add output (string) test
* adapt generateDefaults() to return output (used for test of previous commit)
* Changes to getDefaults() JSON output (#3449 )
* change JSON output to contain separate fields
* filename -> filepath
* Apply suggestions from code review
* Apply suggestions from code review
* Update pkg/config/config_test.go
Co-authored-by: Jordi van Liempt <35920075+jliempt@users.noreply.github.com>
2022-01-24 13:55:28 +01:00
Anil Keshav
3a0e7e3e86
fetch general purpose credentials from vault ( #3380 )
...
Co-authored-by: anilkeshav27 <you@example.com>
2021-12-21 11:00:13 +01:00
Siarhei Pazdniakou
cd243ee542
feat(gcs): allow upload to gcs from steps ( #3034 )
...
* Upload reports to Google Cloud Storage bucket
* Added tests. Made fixes
* Update step generation. GCS client was moved to GeneralConfig
* Code was refactored
* Fixed issues
* Fixed issues
* Code correction due to PR comments
* Improved gcs client and integration tests
* Integrated gcp config. Updated step metadata
* Fixed issues. Added tests
* Added cpe, vault, aliases resolving for reporting parameters
* Added tests
* Uncommented DeferExitHandler. Removed useless comments
* fixed cloning of config
* Added comments for exported functions. Removed unused mock
* minor fix
* Implemented setting of report name via paramRef
* some refactoring. Writing tests
* Update pkg/config/reporting.go
* Update cmd/sonarExecuteScan_generated.go
* Apply suggestions from code review
* Update pkg/config/reporting.go
* Update pkg/config/reporting.go
* fixed removing valut secret files
* Update pkg/config/reporting.go
* restore order
* restore order
* Apply suggestions from code review
* go generate
* fixed tests
* Update resources/metadata/sonarExecuteScan.yaml
* Update resources.go
* Fixed tests. Code was regenerated
* changed somewhere gcp to gcs. Fixed one test
* move gcsSubFolder to input parameters
* fixed removing valut secret files
* minor fix in integration tests
* fix integration tests
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2021-12-15 15:07:47 +01:00
Eugene Kortelyov
56be54c504
Feature/vault refactoring ( #3113 )
...
* refactor vault code
* adjust generator
* wip: fix tests
* regenerate influxdb
* fix test
* add another test
* fix test & docs
* fix formatting
* Minorupdate and fixes
Co-authored-by: Kevin Stiehl <kevin.stiehl@numericas.de>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-09-21 13:06:32 +02:00
Oliver Nocon
f308fbfc0d
feat(getConfig) allow reading stage configuration ( #3026 )
...
* feat(getConfig) allow reading stage configuration
* update test
* use fix version of Selenium Image for Karma tests
2021-08-04 10:09:02 +02:00
Oliver Nocon
805a8fd88f
feat(config): read config/defaults with authentication ( #2975 )
...
* feat(config):read config/defaults with authentication
This change allows to use defaults and config files from a protected GitHub repository.
The options `--customConfig` and `--defaultConfig` already allowed to provide a link to an uprotected file.
Now, by passing a value in the form `<hostname>:<token>` to parameter `gitHubTokens` (this parameter can be passed multiple times) a token can be provided for dedicated hosts.
This makes it possible to use a link like
`https://api.github.com/repos/SAP/jenkins-library/contents/resources/my-defaults.yml?ref=master `
as reference to a default file or similarly as reference to a configuration file.
* update generation to allow protected config/defaults
* fix CodeClimate issues
* update missing generations
2021-07-08 15:26:07 +02:00
Oliver Nocon
0b48bfcc73
feat: retrieve metadata by stepName - corrected ( #2892 )
...
* refactored getConfig to allow stepName param for metadata fetching
* extended step generator
* go generate
* Update cmd/getConfig.go
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
* Update cmd/getConfig.go
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
* Update cmd/getConfig.go
* update generated files
* update golden files to care for generator update
* update and add tests
* update generated files
* Update cmd/getConfig.go
* Update cmd/getConfig.go
* update/fix formatting
* feat: retrieve metadata by stepName - corrected
* update generation
* update condition logic for defaults
* update generation & tests
* support multiple conditions
* update generation
* Add generated
Co-authored-by: Leander Schulz <leander.schulz01@sap.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Sven Merk <s.merk@sap.com>
2021-06-16 08:43:30 +02:00
ffeldmann
b88ebdad6c
feat(splunk) Sending telemetry and logging information to Splunk ( #2796 )
...
* Adds inital splunk hook and logCollector
* Adds documentation of the Splunk hook
* Fixes markdown lint issues and removes comment from telemetry.go file
* Fixes markdown lint issues and adds missing generated file
* Markdown linting
* Changes documentation according to review, adds Splunk token automatically during init
* Adds error handling for marshalling hook config
* Markdown lint und correct Splunk token in httpclient
* Registeres Splunk token as secret and adjusts test cases
* Adds missing error handling and removes unnecessary comments
* Creates new function readPipelineEnvironment, adds tests
* Moves MonitoringData struct, edits defaults for json fields
* Adds gitRepository and gitOwner to telemetry information
* Simplifies readCommonPipelineEnvironment function, adds more descriptive errors, adds automated adding of Splunk prefix token
* Adjusts error handling
* Cleaner error logging
2021-05-17 12:14:04 +02:00
Christopher Fenner
98fac0a455
chore(deprecation): change format of deprecation notice from config command ( #2822 )
...
* Update config.go
* Update config.go
* add library name
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-05-10 19:18:16 +02:00
lndrschlz
b8e9201df5
feat(vault): provide test credentials from vault in every piper go step ( #2715 )
...
* added support for test credentials
Co-authored-by: Kevin Stiehl <kevin.stiehl@numericas.de>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-04-09 10:04:35 +02:00
Oliver Nocon
24cb3c92e1
fix(config): consider defaults with higher prio than env ( #2725 )
...
* fix(config): properly handle empty values from cpe
* fix(config): consider defaults with higher prio than env
2021-03-29 10:40:04 +02:00
Kevin Stiehl
f2b0683880
add token revocation ( #2686 )
...
* add token revocation
* update mocks
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-03-10 08:36:50 +01:00
Oliver Nocon
d769fb5148
fix(config): properly handle empty values from cpe ( #2682 )
2021-03-09 09:30:18 +01:00
Kevin Stiehl
ffffe8295e
(Vault) Improvements ( #2439 )
...
* vault improvements
* Update cloudFoundryDeploy.yaml
remove double PARAMETERS
* go generate
* fix type & resturcutre paragraph to a list
* remove non-existent secrets
* build trigger
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2021-02-15 09:48:51 +01:00
Kevin Stiehl
4a92d1a96a
[Vault] Mixin Vault Parameters for default config ( #2523 )
...
* load vault filters for custom defaults
* add tests
2021-01-20 14:59:47 +01:00
lndrschlz
b101106a05
chore: hide warn level if verbose not configured ( #2466 )
2020-12-22 14:33:11 +01:00
Kevin Stiehl
dfab156427
Feature/approle secret id rotation ( #2311 )
...
* add new step vault secret
* add debug log
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-11-17 13:49:31 +01:00
Kevin Stiehl
24aafb0b69
add vaultSecretFileReferences ( #2203 )
...
* add vaultSecretFileReferences
* fix test
* fix test
* go generate
* remove code duplication
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-10-26 14:20:04 +01:00
Kevin Stiehl
3eae0c5f68
feat(vault): fetch secrets from vault ( #2032 )
...
* cloud-foundry & sonar from vault
* add vault development hint
* don't abort on vault errors
* cloudfoundry make credentialsId only mandatory when vault is not configured
* add vault ref to step ymls
* rename vaultAddress to vaultServerUrl
* rename PIPER_vaultRole* to PIPER_vaultAppRole*
* add resourceRef for detect step
* fix error when no namespace is set
* added debug logs
* added debug logs
* fix vault resolving
* add vaultCustomBasePath
* rename vault_test.go to client_test.go
* refactored vault logging
* refactored config param lookup for vault
* added tüddelchen
* rename vaultCustomBasePath to vaultPath
* fix tests
* change lookup path for group secrets
* fix interpolation tests
* added vault resource ref to versioning
* execute go generate
* rename Approle to AppRole
* change verbose back to false
Co-authored-by: Leander Schulz <leander.schulz01@sap.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-10-13 14:14:47 +02:00
Kevin Stiehl
d589038206
Vault AppRole login ( #1971 )
...
* added interpolation package in config
* vault allow paths to hold config references
* allow referencing properties in vaultPaths
* fix small typo
* add approleAuth
* register resolved secrets to logger
* generate steps
* clean up
* add integration test
* add vault to context filter
* reduce Cognitive Complexity & added tests
* Update pkg/config/stepmeta_test.go
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* go generate
* go generate after merge
* rename VaultAppRole* to VaultRole*
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-09-16 14:50:09 +02:00
Oliver Nocon
d691750931
Fix reading of hook configuration ( #1854 )
...
* Update piper.go
* Update piper_test.go
* Update config.go
* Update sentryHook.go
* Update piper.go
* go fmt
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-07-28 17:19:33 +02:00
Kevin Stiehl
116a91ce28
Fetch config parameters from vault ( #1782 )
...
* added secret fetching from vault
* added rootPath and vault namespaces
* write tests for vault config loading
* fix smaller bugs
* rename rootPath to basePath
* fix integration test
* make basePath optional
* write more tests
* remove unnecessary code
* make basePath configurable
* remove mixin for vault
* rewrite tests
* fix tests for searching over multiple paths
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-07-22 11:15:48 +02:00
Daniel Kurzynski
61a669778a
Apply alias to secrets ( #1802 )
2020-07-15 21:05:22 +02:00
lndrschlz
db13b2f9ce
moved container context method to config package and expose for reuse ( #1770 )
2020-07-10 10:32:26 +02:00
Daniel Kurzynski
fb3c6b5978
Ignore wrong type of alias ( #1778 )
2020-07-09 14:13:06 +02:00
Daniel Kurzynski
9a30758115
Consider stage config from custom defaults ( #1670 )
2020-06-16 09:06:37 +02:00
Oliver Nocon
6116fe5528
Decouple config reading and retrieval ( #1668 )
...
There are use cases where we need to run `GetStepConfig()` multiple times.
In such cases it is more efficient to load the files once and then resolve the
respective step configuration.
2020-06-15 14:17:59 +02:00
Oliver Nocon
50b2a2d6fc
Update deprecation message ( #1656 )
2020-06-12 15:19:13 +02:00
Stephan Aßmus
5ab4dedcb2
Optionally ignore custom defaults ( #1538 )
...
* Add parameter "--ignoreCustomDefaults"
* Pass to piper customDefaults from config also via --defaultConfig
... and add "--ignoreCustomDefaults".
* Log output when ignoring customDefaults
Co-authored-by: Daniel Kurzynski <daniel.kurzynski@sap.com>
2020-05-14 10:50:58 +02:00
lndrschlz
39e30ff1b4
Added hook configuration; added sentry hook and tests ( #1497 )
...
* Added hook configuration; added sentry hook and tests
* steps generated by step framework
* added comments and renamed correlationID
* formatted comment on NewSentryHook method
* fix generator step tests
2020-05-05 08:36:24 +02:00
Christopher Fenner
b0144614e5
improve error message when parsing config ( #1387 )
...
* improve error message when parsing config
* adjust test case
Co-authored-by: Florian Wilhelm <florian.wilhelm02@sap.com>
2020-04-09 13:40:04 +02:00
Stephan Aßmus
da0f25fd7b
Piper-Go: Allow aliases also for inputs of type "secret" ( #1355 )
...
* Fix typos
* Support aliases also for secrets
* Adapt & extend Unit Tests
* Output deprecation warning for param/secret aliases
... if the alias is marked as 'deprecated'.
2020-04-01 20:46:33 +02:00
Oliver Nocon
91b6b0900f
Config and Defaults via http(s) ( #1302 )
...
* Config and Defaults via http(s)
Add the possibility to use http(s) urls as path to
* config.yml
* defaults
* Remove comment
* Addressing PR feedback
* Fix error handling
2020-03-31 08:47:09 +02:00
Oliver Nocon
8fbeddb26c
Golang step metadata: Config aliases for steps ( #1293 )
...
* Golang step metadata: Config aliases for steps
This will ease following scenarios:
* config migration due to step name changes
* re-use of more general config, e.g. `mavenExecute` in `mavenBuild`
* fix CodeClimate finding
* Fix panic if original stage config does not exist yet
2020-03-19 17:24:35 +01:00
Oliver Nocon
a46b57e6b4
Persisted pipeline environment for golang library ( #1091 )
...
* Use commonPipelineEnvironment in go binary
* Update groovy part incl. tests
* Rework structure and naming
* Support influx resources in steps
* Update tests and some cleanups
* Add correct defer handling
* Address PR feedback
* Fix test
* Update resources.go
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2020-01-15 12:16:25 +01:00
Marcus Holl
9cd4950437
Don't swallow error raised from file exists helper method ( #1019 )
...
* Don't swallow error raised from file exists helper method
* streamline
2019-12-11 10:13:23 +01:00
Marcus Holl
9ee2926018
xs deploy go ( #931 )
...
introduce xs deploy
2019-12-05 14:22:38 +01:00
Oliver Nocon
69bb2c9e60
Fix issues in config resolution ( #1000 )
...
* Respect defaults from step definition everywhere
* prevent empty config to overwrite default in aliasing
Address PR feedback
2019-11-22 10:30:44 +01:00
Oliver Nocon
0f33f7a2ef
Consider custom defaults in config.yml ( #995 )
...
* Consider custom defaults in config.yml
2019-11-21 16:09:57 +01:00
Sven Merk
6721c59a30
Merge branch 'master' into cfg-hierarchical-defaults
2019-11-06 12:13:15 +01:00
Marcus Holl
a456282d6a
fix: getting config should also work in case there is no project config ( #951 )
2019-11-06 10:28:15 +01:00
Sven Merk
376419e0db
Add mixin of dependent defaults
2019-11-05 16:30:41 +01:00
Oliver Nocon
e01b3327fd
Go configuration - add aliasing functionality ( #925 )
...
* Go configuration - add aliasing functionality
This assists in backward compatibility cases.
It is now possible to define per parameter aliases.
2019-10-29 10:58:24 +01:00
Oliver Nocon
c1eb9f5c70
Provide first parts for golang implementation ( #905 )
...
* Provide first parts for golang implementation
2019-10-22 15:41:27 +02:00