1
0
mirror of https://github.com/SAP/jenkins-library.git synced 2024-12-16 11:09:33 +02:00
Commit Graph

88 Commits

Author SHA1 Message Date
Nikolay Grechanov
17510f8996
fix: correct typo in log message (#3439)
Closes #3438
2022-01-19 08:41:11 +01:00
Anil Keshav
f3b65ae43b
feat (npmExecuteScripts) create seperate npmrc file for publish to private repo (#3422)
* creating new npm rc file

* publishing to registry staging

* exposing base64 version of env variables

* changing encoding param

* fixing unit test for the new path

* debugging env var

* remove debug message

* update docu

* changing new npmrc file name

* adding new npmrc to ignore

* adding new npmrc to ignore

Co-authored-by: anilkeshav27 <you@example.com>
2022-01-13 13:08:19 +01:00
Marc Bormeth
78c4a8fe93
Revert "feat: Add debug output to getConfig (#3341)" (#3396)
This reverts commit b760bf48e7.

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-01-11 10:47:03 +01:00
Christopher Fenner
51feacadbf
chore: align Vault naming (#3372)
* Align Vault naming

* remove duplicate debug  output

* correct message

* align vault naming

* Update vault.md
2021-12-21 11:52:10 +01:00
Anil Keshav
3a0e7e3e86
fetch general purpose credentials from vault (#3380)
Co-authored-by: anilkeshav27 <you@example.com>
2021-12-21 11:00:13 +01:00
Siarhei Pazdniakou
cd243ee542
feat(gcs): allow upload to gcs from steps (#3034)
* Upload reports to Google Cloud Storage bucket

* Added tests. Made fixes

* Update step generation. GCS client was moved to GeneralConfig

* Code was refactored

* Fixed issues

* Fixed issues

* Code correction due to PR comments

* Improved gcs client and integration tests

* Integrated gcp config. Updated step metadata

* Fixed issues. Added tests

* Added cpe, vault, aliases resolving for reporting parameters

* Added tests

* Uncommented DeferExitHandler. Removed useless comments

* fixed cloning of config

* Added comments for exported functions. Removed unused mock

* minor fix

* Implemented setting of report name via paramRef

* some refactoring. Writing tests

* Update pkg/config/reporting.go

* Update cmd/sonarExecuteScan_generated.go

* Apply suggestions from code review

* Update pkg/config/reporting.go

* Update pkg/config/reporting.go

* fixed removing valut secret files

* Update pkg/config/reporting.go

* restore order

* restore order

* Apply suggestions from code review

* go generate

* fixed tests

* Update resources/metadata/sonarExecuteScan.yaml

* Update resources.go

* Fixed tests. Code was regenerated

* changed somewhere gcp to gcs. Fixed one test

* move gcsSubFolder to input parameters

* fixed removing valut secret files

* minor fix in integration tests

* fix integration tests

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2021-12-15 15:07:47 +01:00
Marc Bormeth
b760bf48e7
feat: Add debug output to getConfig (#3341)
* Fix Orchestrator detection

* Add unit tests

* Add debug output to getConfig

* Update getConfig.go

* fix whitespaces

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2021-12-08 10:16:54 +01:00
kingvvgo
b29a2f2264
Add Changes for value of docker image (#3303)
* Add Changes for value of docker image

* Get docker image value

* Fix

* Fix unit

* Add chnages for kaniko and mta builds

* Fix

* Test changes

* Test

* Move func ResolveMetadata to stepmeta.go

* Fix

* Change getConfig.go

* Fix getting docker value for mta, npm and kaniko

* Fix according to suggestions

* Add func to get only value of docker image

* Test empty value of docker image

* Fix for getDockerImageValue

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-12-01 07:46:18 +01:00
Eugene Kortelyov
d395b362ed
feat: step to execute shell scripts (#3196)
* shell executor initial commit

* functionality updates

* changes in logging implementation (using internal logging), changes in execution

* remove unused field

* remove duplicate from code

* update vault flow and remove unnecessary params

* update generated step file

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-11-19 10:05:39 +01:00
Oliver Nocon
6c5434f957
feat(checkIfStepActive): support new CRD style conditions (#3254)
* feat: first parts of new run struct

* add parts for new stage condition handling

* update conditions

* feat: finalize conditions and tests

* feat(checkIfStepActive): support new CRD style conditions

* feat(docs): allow generating stage docs

* chore(docs): make step directory configurable

* fix: tests

* add option to output file

* Update checkIfStepActive_test.go
2021-11-18 07:24:00 +01:00
Siarhei Pazdniakou
46bafc40a3
Improve validation of configuration (#3125)
* Implemented validation for the option possibleValues

* Has been added the option mandatoryIf to config with validation

* Fixed issues found during code review

* improved golang template

* Fixed tests. Added validation for mandatoryIf option

* Fix typo

* Fixed tests

* Validation was refactored. Added options

* Added default value for parameters with possibleValues option

* Validation was moved after the configuration resolution

* Canceled some default values

Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-10-01 12:49:05 +02:00
Eugene Kortelyov
56be54c504
Feature/vault refactoring (#3113)
* refactor vault code

* adjust generator

* wip: fix tests

* regenerate influxdb

* fix test

* add another test

* fix test & docs

* fix formatting

* Minorupdate and fixes

Co-authored-by: Kevin Stiehl <kevin.stiehl@numericas.de>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-09-21 13:06:32 +02:00
Oliver Nocon
5096b3bfe1
fix(checkIfStepActive): respect explicit (de-)activation (#3055) 2021-08-17 11:20:04 +02:00
Oliver Nocon
97b84429f1
feat(Vault): custom prefix for test credentials (#3043)
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2021-08-11 16:20:08 +02:00
Oliver Nocon
f308fbfc0d
feat(getConfig) allow reading stage configuration (#3026)
* feat(getConfig) allow reading stage configuration

* update test

* use fix version of Selenium Image for Karma tests
2021-08-04 10:09:02 +02:00
Oliver Nocon
6f32b437be
fix(dockerOptions) allow proper usage of empty values (#3025)
This follows up on #3024
Setting emptyValue to s.th. like `--entrypoint=''` will break in case the argument is properly escaped.

Docker will return with
`container process caused: exec: "''": executable file not found in $PATH`
2021-08-02 14:57:37 +02:00
Oliver Nocon
f455a8a4a0
fix(dockerOptions) properly pass empty values (#3024)
it is possible to overwrite the entrypoint for docker execution:
https://docs.docker.com/engine/reference/run/#entrypoint-default-command-to-execute-at-runtime

This is ideally done by passing `entrypoint=''` and not pass two options to the call.
This also helps with escaping issues of the empty value on other systems.

Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2021-08-02 10:21:14 +02:00
Siarhei Pazdniakou
7a325e6fc8
Convert step/stage condition logic to golang (#2993)
* Added checkIfStepActive step

* Implemented npmScripts condition. Code was refactored

* Added some unit tests

* Fixed go modules

* Fixed go modules

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-07-26 07:47:43 +02:00
Oliver Nocon
805a8fd88f
feat(config): read config/defaults with authentication (#2975)
* feat(config):read config/defaults with authentication

This change allows to use defaults and config files from a protected GitHub repository.

The options `--customConfig` and `--defaultConfig` already allowed to provide a link to an uprotected file.

Now, by passing a value in the form `<hostname>:<token>` to parameter `gitHubTokens` (this parameter can be passed multiple times) a token can be provided for dedicated hosts.

This makes it possible to use a link like
`https://api.github.com/repos/SAP/jenkins-library/contents/resources/my-defaults.yml?ref=master`
as reference to a default file or similarly as reference to a configuration file.

* update generation to allow protected config/defaults

* fix CodeClimate issues

* update missing generations
2021-07-08 15:26:07 +02:00
Oliver Nocon
0b48bfcc73
feat: retrieve metadata by stepName - corrected (#2892)
* refactored getConfig to allow stepName param for metadata fetching

* extended step generator

* go generate

* Update cmd/getConfig.go

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Update cmd/getConfig.go

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Update cmd/getConfig.go

* update generated files

* update golden files to care for generator update

* update and add tests

* update generated files

* Update cmd/getConfig.go

* Update cmd/getConfig.go

* update/fix formatting

* feat: retrieve metadata by stepName - corrected

* update generation

* update condition logic for defaults

* update generation & tests

* support multiple conditions

* update generation

* Add generated

Co-authored-by: Leander Schulz <leander.schulz01@sap.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Sven Merk <s.merk@sap.com>
2021-06-16 08:43:30 +02:00
ffeldmann
b88ebdad6c
feat(splunk) Sending telemetry and logging information to Splunk (#2796)
* Adds inital splunk hook and logCollector

* Adds documentation of the Splunk hook

* Fixes markdown lint issues and removes comment from telemetry.go file

* Fixes markdown lint issues and adds missing generated file

* Markdown linting

* Changes documentation according to review, adds Splunk token automatically during init

* Adds error handling for marshalling hook config

* Markdown lint und correct Splunk token in httpclient

* Registeres Splunk token as secret and adjusts test cases

* Adds missing error handling and removes unnecessary comments

* Creates new function readPipelineEnvironment, adds tests

* Moves MonitoringData struct, edits defaults for json fields

* Adds gitRepository and gitOwner to telemetry information

* Simplifies readCommonPipelineEnvironment function, adds more descriptive errors, adds automated adding of Splunk prefix token

* Adjusts error handling

* Cleaner error logging
2021-05-17 12:14:04 +02:00
Christopher Fenner
98fac0a455
chore(deprecation): change format of deprecation notice from config command (#2822)
* Update config.go

* Update config.go

* add library name

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-05-10 19:18:16 +02:00
Siarhei Pazdniakou
0673d3fed6
Writing secrets to vault (#2770)
Co-authored-by: Kevin Stiehl <kevin.stiehl@numericas.de>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-05-06 10:27:23 +02:00
lndrschlz
b8e9201df5
feat(vault): provide test credentials from vault in every piper go step (#2715)
* added support for test credentials
Co-authored-by: Kevin Stiehl <kevin.stiehl@numericas.de>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-04-09 10:04:35 +02:00
Oliver Nocon
24cb3c92e1
fix(config): consider defaults with higher prio than env (#2725)
* fix(config): properly handle empty values from cpe

* fix(config): consider defaults with higher prio than env
2021-03-29 10:40:04 +02:00
Kevin Stiehl
f2b0683880
add token revocation (#2686)
* add token revocation

* update mocks

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-03-10 08:36:50 +01:00
Oliver Nocon
d769fb5148
fix(config): properly handle empty values from cpe (#2682) 2021-03-09 09:30:18 +01:00
Kevin Stiehl
649b6d6201
[Vault] Fix ContextParamFilters for vaultSecretFile (#2671)
* fix contextFilter

Co-authored-by: lndrschlz <leander.schulz01@sap.com>

* reduce Congitive Complexity warning

Co-authored-by: lndrschlz <leander.schulz01@sap.com>
2021-03-08 10:16:32 +01:00
Kevin Stiehl
ffffe8295e
(Vault) Improvements (#2439)
* vault improvements

* Update cloudFoundryDeploy.yaml

remove double PARAMETERS

* go generate

* fix type & resturcutre paragraph to a list

* remove non-existent secrets

* build trigger

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2021-02-15 09:48:51 +01:00
Oliver Nocon
44788f1d3a
fix(config): properly read non-string parameters from environment (#2549) 2021-01-28 11:09:43 +01:00
Marcus Holl
ee7279c8fc
Add helper for finding uninitialized config (#2534)
Add helper for finding uninitialized config
2021-01-25 12:58:43 +01:00
Kevin Stiehl
4a92d1a96a
[Vault] Mixin Vault Parameters for default config (#2523)
* load vault filters for custom defaults

* add tests
2021-01-20 14:59:47 +01:00
lndrschlz
b101106a05
chore: hide warn level if verbose not configured (#2466) 2020-12-22 14:33:11 +01:00
Oliver Feldmann
4f6b99cead
Don't allow empty strings as default container parameters (#2406)
First a bug fix is addressed in which the pull policy could not be configured to false by configuring the general configuration. It could neither be configured via dockerExecute or dockerExecuteOnKubernetes, even though this parameter is docker specific. Only by configuring the specific step where one wants to set the pull policy to false can it be configured.

As the bug stems from zero values being in the context config map, which is also addressed with this PR. That is the second part: Context config parameters are only set if they have a value.

* Don't set pull image if not configured

Otherwise, if the pull policy is not set explicitly for a step, dockerPullImage is set to true. Thus, before this change, it cannot be set in the general, or in dockerExecute or in dockerExecuteOnKubernetes configuration.

* Fix unit tests

* Add pullImage parameter test

* Do not place empty default values in context config

* Use putIfNotEmpty for sidecar container options

* Export common configuration

Keys that are set by both main and sidecar container can be exported

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-12-10 11:28:39 +01:00
Kevin Stiehl
dfab156427
Feature/approle secret id rotation (#2311)
* add new step vault secret

* add debug log

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-11-17 13:49:31 +01:00
Kevin Stiehl
9d461c6753
consider alias during secret resolving (#2327) 2020-11-06 17:54:01 +01:00
Kevin Stiehl
67727147f8
switch to debug logging for some vault log messages (#2299) 2020-11-03 08:45:12 +01:00
Kevin Stiehl
24aafb0b69
add vaultSecretFileReferences (#2203)
* add vaultSecretFileReferences

* fix test

* fix test

* go generate

* remove code duplication

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-10-26 14:20:04 +01:00
Marcus Holl
14e1c44aae
Avoid having the mocks in the productive binary (#2199) 2020-10-21 12:48:26 +02:00
Kevin Stiehl
3eae0c5f68
feat(vault): fetch secrets from vault (#2032)
* cloud-foundry & sonar from vault

* add vault development hint

* don't abort on vault errors

* cloudfoundry make credentialsId only mandatory when vault is not configured

* add vault ref to step ymls

* rename vaultAddress to vaultServerUrl

* rename PIPER_vaultRole* to PIPER_vaultAppRole*

* add resourceRef for detect step

* fix error when no namespace is set

* added debug logs

* added debug logs

* fix vault resolving

* add vaultCustomBasePath

* rename vault_test.go to client_test.go

* refactored vault logging

* refactored config param lookup for vault

* added tüddelchen

* rename vaultCustomBasePath to vaultPath

* fix tests

* change lookup path for group secrets

* fix interpolation tests

* added vault resource ref to versioning

* execute go generate

* rename Approle to AppRole

* change verbose back to false

Co-authored-by: Leander Schulz <leander.schulz01@sap.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-10-13 14:14:47 +02:00
Christopher Fenner
be90876b7c
feat(output): handle non-string output values (#2113)
* handle non-string values as JSON

* change value type to interface in resources

* regenerate code

* add test cases

* handle reading of json files

* write json data to json files

* fix assignment

* use GetResourceParameter

* add test case

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-10-05 15:33:28 +02:00
Christopher Fenner
6999380ee3
chore(go): simplify code using gofmt -s (#2065) 2020-09-24 08:58:53 +02:00
Christopher Fenner
b219fb6514
fix(typo): found by misspell (#2064)
* fix typos in step yamls

* fix typos in go files

* regenerate step code

* fix typos in md files

* fix typos in groovy files

* fix further typos
2020-09-24 07:41:06 +02:00
Kevin Stiehl
d589038206
Vault AppRole login (#1971)
* added interpolation package in config

* vault allow paths to hold config references

* allow referencing properties in vaultPaths

* fix small typo

* add approleAuth

* register resolved secrets to logger

* generate steps

* clean up

* add integration test

* add vault to context filter

* reduce Cognitive Complexity & added tests

* Update pkg/config/stepmeta_test.go

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

* go generate

* go generate after merge

* rename VaultAppRole* to VaultRole*

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-09-16 14:50:09 +02:00
Oliver Nocon
d691750931
Fix reading of hook configuration (#1854)
* Update piper.go

* Update piper_test.go

* Update config.go

* Update sentryHook.go

* Update piper.go

* go fmt

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-07-28 17:19:33 +02:00
Kevin Stiehl
116a91ce28
Fetch config parameters from vault (#1782)
* added secret fetching from vault

* added rootPath and vault namespaces

* write tests for vault config loading
	* fix smaller bugs

* rename rootPath to basePath

* fix integration test

* make basePath optional

* write more tests

* remove unnecessary code

* make basePath configurable

* remove mixin for vault

* rewrite tests
* fix tests for searching over multiple paths

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-07-22 11:15:48 +02:00
Daniel Mieg
90571754bf
Add test cases for stepmeta.go (#1801)
* Add tests

* Remove obsolete comment
2020-07-16 11:09:49 +02:00
Oliver Nocon
43f51ba90a
Revamp documentation generation for golang steps (#1781)
* Revamp documentation generation for golang steps

* Add and update tests

* Add tests, add step outputs

* Add tests, add step outputs

* Update rendering

* fix merge conflict

* update generated files

* Update pkg/config/stepmeta.go

* Update pkg/config/stepmeta.go

* Update pkg/generator/helper/docuHelper.go

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Include PR feedback

* update tests

* Fix Jenkins dependecy indication

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-07-16 09:10:15 +02:00
Daniel Kurzynski
61a669778a
Apply alias to secrets (#1802) 2020-07-15 21:05:22 +02:00
Daniel Mieg
5dbb7be6b6
Fix commonPipelineEnvironment issue (#1798)
* Fix write to disk

* Change condition

* Test log output

* Fix

* Remove test log
2020-07-15 15:45:36 +02:00