1
0
mirror of https://github.com/SAP/jenkins-library.git synced 2024-12-16 11:09:33 +02:00
Commit Graph

47 Commits

Author SHA1 Message Date
Maurice Breit
9f2d8a70b1
(fix) match regexes in sliceContains to support vaultSecretNames (#3618)
* (fix) match regexes in sliceContains to support vaultSecretNames

* add test for regex matching in sliceContains

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-03-17 09:41:51 +01:00
Ralf Pannemans
e7db09db12
feat(cnbBuild): added dockerimage to the telemetry data (#3501)
Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>
Co-authored-by: Sumit Kulhadia <sumit.kulhadia@sap.com>
Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
2022-02-16 13:28:51 +01:00
Oliver Nocon
2ae1d9dac1
feat(kanikoExecute): allow building multiple images (#3443)
* feat(kanikoExecute): allow building multiple images

* enhance tests

* chore: allow running tests in parallel

* small fixes

* fix: fix destination bug

* update formatting and defaults

* fix yml formatting

* chore: change cpe parameter names

* chore: improve variable naming
2022-02-07 07:58:41 +01:00
Christopher Fenner
ebcbb73b46
feat: add command to fetch default files (#3445)
* some ideas..

* Add getDefaults command (WIP) (#3444)

* add getYAML function for configs

* create getDefaults command(based on getConfig)

* add getDefaults command to CLI

* read defaults files, using github tokens as well

* write defaults to stdout as JSON object with YAMLs embedded

* catch case where no input files are given

* actually write output to file if outputFile is specified

* mark defaultsFile flag as required

* add basic tests

* add output (string)  test

* adapt generateDefaults() to return output (used for test of previous commit)

* Changes to getDefaults() JSON output (#3449)

* change JSON output to contain separate fields

* filename -> filepath

* Apply suggestions from code review

* Apply suggestions from code review

* Update pkg/config/config_test.go

Co-authored-by: Jordi van Liempt <35920075+jliempt@users.noreply.github.com>
2022-01-24 13:55:28 +01:00
Anil Keshav
3a0e7e3e86
fetch general purpose credentials from vault (#3380)
Co-authored-by: anilkeshav27 <you@example.com>
2021-12-21 11:00:13 +01:00
Siarhei Pazdniakou
cd243ee542
feat(gcs): allow upload to gcs from steps (#3034)
* Upload reports to Google Cloud Storage bucket

* Added tests. Made fixes

* Update step generation. GCS client was moved to GeneralConfig

* Code was refactored

* Fixed issues

* Fixed issues

* Code correction due to PR comments

* Improved gcs client and integration tests

* Integrated gcp config. Updated step metadata

* Fixed issues. Added tests

* Added cpe, vault, aliases resolving for reporting parameters

* Added tests

* Uncommented DeferExitHandler. Removed useless comments

* fixed cloning of config

* Added comments for exported functions. Removed unused mock

* minor fix

* Implemented setting of report name via paramRef

* some refactoring. Writing tests

* Update pkg/config/reporting.go

* Update cmd/sonarExecuteScan_generated.go

* Apply suggestions from code review

* Update pkg/config/reporting.go

* Update pkg/config/reporting.go

* fixed removing valut secret files

* Update pkg/config/reporting.go

* restore order

* restore order

* Apply suggestions from code review

* go generate

* fixed tests

* Update resources/metadata/sonarExecuteScan.yaml

* Update resources.go

* Fixed tests. Code was regenerated

* changed somewhere gcp to gcs. Fixed one test

* move gcsSubFolder to input parameters

* fixed removing valut secret files

* minor fix in integration tests

* fix integration tests

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2021-12-15 15:07:47 +01:00
Eugene Kortelyov
56be54c504
Feature/vault refactoring (#3113)
* refactor vault code

* adjust generator

* wip: fix tests

* regenerate influxdb

* fix test

* add another test

* fix test & docs

* fix formatting

* Minorupdate and fixes

Co-authored-by: Kevin Stiehl <kevin.stiehl@numericas.de>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-09-21 13:06:32 +02:00
Oliver Nocon
f308fbfc0d
feat(getConfig) allow reading stage configuration (#3026)
* feat(getConfig) allow reading stage configuration

* update test

* use fix version of Selenium Image for Karma tests
2021-08-04 10:09:02 +02:00
Oliver Nocon
805a8fd88f
feat(config): read config/defaults with authentication (#2975)
* feat(config):read config/defaults with authentication

This change allows to use defaults and config files from a protected GitHub repository.

The options `--customConfig` and `--defaultConfig` already allowed to provide a link to an uprotected file.

Now, by passing a value in the form `<hostname>:<token>` to parameter `gitHubTokens` (this parameter can be passed multiple times) a token can be provided for dedicated hosts.

This makes it possible to use a link like
`https://api.github.com/repos/SAP/jenkins-library/contents/resources/my-defaults.yml?ref=master`
as reference to a default file or similarly as reference to a configuration file.

* update generation to allow protected config/defaults

* fix CodeClimate issues

* update missing generations
2021-07-08 15:26:07 +02:00
Oliver Nocon
0b48bfcc73
feat: retrieve metadata by stepName - corrected (#2892)
* refactored getConfig to allow stepName param for metadata fetching

* extended step generator

* go generate

* Update cmd/getConfig.go

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Update cmd/getConfig.go

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Update cmd/getConfig.go

* update generated files

* update golden files to care for generator update

* update and add tests

* update generated files

* Update cmd/getConfig.go

* Update cmd/getConfig.go

* update/fix formatting

* feat: retrieve metadata by stepName - corrected

* update generation

* update condition logic for defaults

* update generation & tests

* support multiple conditions

* update generation

* Add generated

Co-authored-by: Leander Schulz <leander.schulz01@sap.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Sven Merk <s.merk@sap.com>
2021-06-16 08:43:30 +02:00
ffeldmann
b88ebdad6c
feat(splunk) Sending telemetry and logging information to Splunk (#2796)
* Adds inital splunk hook and logCollector

* Adds documentation of the Splunk hook

* Fixes markdown lint issues and removes comment from telemetry.go file

* Fixes markdown lint issues and adds missing generated file

* Markdown linting

* Changes documentation according to review, adds Splunk token automatically during init

* Adds error handling for marshalling hook config

* Markdown lint und correct Splunk token in httpclient

* Registeres Splunk token as secret and adjusts test cases

* Adds missing error handling and removes unnecessary comments

* Creates new function readPipelineEnvironment, adds tests

* Moves MonitoringData struct, edits defaults for json fields

* Adds gitRepository and gitOwner to telemetry information

* Simplifies readCommonPipelineEnvironment function, adds more descriptive errors, adds automated adding of Splunk prefix token

* Adjusts error handling

* Cleaner error logging
2021-05-17 12:14:04 +02:00
Christopher Fenner
98fac0a455
chore(deprecation): change format of deprecation notice from config command (#2822)
* Update config.go

* Update config.go

* add library name

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-05-10 19:18:16 +02:00
lndrschlz
b8e9201df5
feat(vault): provide test credentials from vault in every piper go step (#2715)
* added support for test credentials
Co-authored-by: Kevin Stiehl <kevin.stiehl@numericas.de>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-04-09 10:04:35 +02:00
Oliver Nocon
24cb3c92e1
fix(config): consider defaults with higher prio than env (#2725)
* fix(config): properly handle empty values from cpe

* fix(config): consider defaults with higher prio than env
2021-03-29 10:40:04 +02:00
Kevin Stiehl
f2b0683880
add token revocation (#2686)
* add token revocation

* update mocks

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-03-10 08:36:50 +01:00
Oliver Nocon
d769fb5148
fix(config): properly handle empty values from cpe (#2682) 2021-03-09 09:30:18 +01:00
Kevin Stiehl
ffffe8295e
(Vault) Improvements (#2439)
* vault improvements

* Update cloudFoundryDeploy.yaml

remove double PARAMETERS

* go generate

* fix type & resturcutre paragraph to a list

* remove non-existent secrets

* build trigger

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2021-02-15 09:48:51 +01:00
Kevin Stiehl
4a92d1a96a
[Vault] Mixin Vault Parameters for default config (#2523)
* load vault filters for custom defaults

* add tests
2021-01-20 14:59:47 +01:00
lndrschlz
b101106a05
chore: hide warn level if verbose not configured (#2466) 2020-12-22 14:33:11 +01:00
Kevin Stiehl
dfab156427
Feature/approle secret id rotation (#2311)
* add new step vault secret

* add debug log

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-11-17 13:49:31 +01:00
Kevin Stiehl
24aafb0b69
add vaultSecretFileReferences (#2203)
* add vaultSecretFileReferences

* fix test

* fix test

* go generate

* remove code duplication

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-10-26 14:20:04 +01:00
Kevin Stiehl
3eae0c5f68
feat(vault): fetch secrets from vault (#2032)
* cloud-foundry & sonar from vault

* add vault development hint

* don't abort on vault errors

* cloudfoundry make credentialsId only mandatory when vault is not configured

* add vault ref to step ymls

* rename vaultAddress to vaultServerUrl

* rename PIPER_vaultRole* to PIPER_vaultAppRole*

* add resourceRef for detect step

* fix error when no namespace is set

* added debug logs

* added debug logs

* fix vault resolving

* add vaultCustomBasePath

* rename vault_test.go to client_test.go

* refactored vault logging

* refactored config param lookup for vault

* added tüddelchen

* rename vaultCustomBasePath to vaultPath

* fix tests

* change lookup path for group secrets

* fix interpolation tests

* added vault resource ref to versioning

* execute go generate

* rename Approle to AppRole

* change verbose back to false

Co-authored-by: Leander Schulz <leander.schulz01@sap.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-10-13 14:14:47 +02:00
Kevin Stiehl
d589038206
Vault AppRole login (#1971)
* added interpolation package in config

* vault allow paths to hold config references

* allow referencing properties in vaultPaths

* fix small typo

* add approleAuth

* register resolved secrets to logger

* generate steps

* clean up

* add integration test

* add vault to context filter

* reduce Cognitive Complexity & added tests

* Update pkg/config/stepmeta_test.go

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

* go generate

* go generate after merge

* rename VaultAppRole* to VaultRole*

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-09-16 14:50:09 +02:00
Oliver Nocon
d691750931
Fix reading of hook configuration (#1854)
* Update piper.go

* Update piper_test.go

* Update config.go

* Update sentryHook.go

* Update piper.go

* go fmt

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-07-28 17:19:33 +02:00
Kevin Stiehl
116a91ce28
Fetch config parameters from vault (#1782)
* added secret fetching from vault

* added rootPath and vault namespaces

* write tests for vault config loading
	* fix smaller bugs

* rename rootPath to basePath

* fix integration test

* make basePath optional

* write more tests

* remove unnecessary code

* make basePath configurable

* remove mixin for vault

* rewrite tests
* fix tests for searching over multiple paths

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-07-22 11:15:48 +02:00
Daniel Kurzynski
61a669778a
Apply alias to secrets (#1802) 2020-07-15 21:05:22 +02:00
lndrschlz
db13b2f9ce
moved container context method to config package and expose for reuse (#1770) 2020-07-10 10:32:26 +02:00
Daniel Kurzynski
fb3c6b5978
Ignore wrong type of alias (#1778) 2020-07-09 14:13:06 +02:00
Daniel Kurzynski
9a30758115
Consider stage config from custom defaults (#1670) 2020-06-16 09:06:37 +02:00
Oliver Nocon
6116fe5528
Decouple config reading and retrieval (#1668)
There are use cases where we need to run `GetStepConfig()` multiple times.
In such cases it is more efficient to load the files once and then resolve the
respective step configuration.
2020-06-15 14:17:59 +02:00
Oliver Nocon
50b2a2d6fc
Update deprecation message (#1656) 2020-06-12 15:19:13 +02:00
Stephan Aßmus
5ab4dedcb2
Optionally ignore custom defaults (#1538)
* Add parameter "--ignoreCustomDefaults"
* Pass to piper customDefaults from config also via --defaultConfig
... and add "--ignoreCustomDefaults".
* Log output when ignoring customDefaults

Co-authored-by: Daniel Kurzynski <daniel.kurzynski@sap.com>
2020-05-14 10:50:58 +02:00
lndrschlz
39e30ff1b4
Added hook configuration; added sentry hook and tests (#1497)
* Added hook configuration; added sentry hook and tests
* steps generated by step framework
* added comments and renamed correlationID
* formatted comment on NewSentryHook method
* fix generator step tests
2020-05-05 08:36:24 +02:00
Christopher Fenner
b0144614e5
improve error message when parsing config (#1387)
* improve error message when parsing config

* adjust test case

Co-authored-by: Florian Wilhelm <florian.wilhelm02@sap.com>
2020-04-09 13:40:04 +02:00
Stephan Aßmus
da0f25fd7b
Piper-Go: Allow aliases also for inputs of type "secret" (#1355)
* Fix typos

* Support aliases also for secrets

* Adapt & extend Unit Tests

* Output deprecation warning for param/secret aliases

... if the alias is marked as 'deprecated'.
2020-04-01 20:46:33 +02:00
Oliver Nocon
91b6b0900f
Config and Defaults via http(s) (#1302)
* Config and Defaults via http(s)

Add the possibility to use http(s) urls as path to
* config.yml
* defaults

* Remove comment

* Addressing PR feedback

* Fix error handling
2020-03-31 08:47:09 +02:00
Oliver Nocon
8fbeddb26c
Golang step metadata: Config aliases for steps (#1293)
* Golang step metadata: Config aliases for steps

This will ease following scenarios:
* config migration due to step name changes
* re-use of more general config, e.g. `mavenExecute` in `mavenBuild`

* fix CodeClimate finding

* Fix panic if original stage config does not exist yet
2020-03-19 17:24:35 +01:00
Oliver Nocon
a46b57e6b4
Persisted pipeline environment for golang library (#1091)
* Use commonPipelineEnvironment in go binary

* Update groovy part incl. tests

* Rework structure and naming

* Support influx resources in steps

* Update tests and some cleanups

* Add correct defer handling

* Address PR feedback

* Fix test

* Update resources.go

Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2020-01-15 12:16:25 +01:00
Marcus Holl
9cd4950437
Don't swallow error raised from file exists helper method (#1019)
* Don't swallow error raised from file exists helper method

* streamline
2019-12-11 10:13:23 +01:00
Marcus Holl
9ee2926018
xs deploy go (#931)
introduce xs deploy
2019-12-05 14:22:38 +01:00
Oliver Nocon
69bb2c9e60
Fix issues in config resolution (#1000)
* Respect defaults from step definition everywhere
* prevent empty config to overwrite default in aliasing

Address PR feedback
2019-11-22 10:30:44 +01:00
Oliver Nocon
0f33f7a2ef
Consider custom defaults in config.yml (#995)
* Consider custom defaults in config.yml
2019-11-21 16:09:57 +01:00
Sven Merk
6721c59a30
Merge branch 'master' into cfg-hierarchical-defaults 2019-11-06 12:13:15 +01:00
Marcus Holl
a456282d6a
fix: getting config should also work in case there is no project config (#951) 2019-11-06 10:28:15 +01:00
Sven Merk
376419e0db Add mixin of dependent defaults 2019-11-05 16:30:41 +01:00
Oliver Nocon
e01b3327fd
Go configuration - add aliasing functionality (#925)
* Go configuration - add aliasing functionality

This assists in backward compatibility cases.
It is now possible to define per parameter aliases.
2019-10-29 10:58:24 +01:00
Oliver Nocon
c1eb9f5c70
Provide first parts for golang implementation (#905)
* Provide first parts for golang implementation
2019-10-22 15:41:27 +02:00