1
0
mirror of https://github.com/SAP/jenkins-library.git synced 2024-12-12 10:55:20 +02:00
Commit Graph

115 Commits

Author SHA1 Message Date
Christopher Fenner
20e1fb53aa
chore(vault): structure vault logs (#4627)
* structure vault logs

* Update client.go
2023-10-11 10:14:31 +02:00
Jordi van Liempt
0ba4c2206c
chore(deps): Replace io/ioutil package (#4494)
* update all deprecated ioutil usages

* forgotten changes

* add missing imports

* undo changing comment

* add missing 'os' import

* fix integration test

---------

Co-authored-by: I557621 <jordi.van.liempt@sap.com>
Co-authored-by: Gulom Alimov <gulomjon.alimov@sap.com>
2023-08-16 12:57:04 +02:00
Googlom
8c863e457f
sapCumulusUpload step deactivation if its the only active step in stage (#4476)
* implement deactivation logic

* add step condition field

* add unit test and fix evaluateConditions

* add unit test for v1 and fix evaluateConditionsV1

* rollback old evaluator

* rollback v1 evaluator

* move into notActiveCondition and fix unit tests

* add a comment about sapCumulusUpload step

* optimize evaluateConditionsV1 parameters and map memory allocation

* refactor unit tests and add more test cases

* evaluateConditionsV1 refactored

---------

Co-authored-by: Gulom Alimov <gulomjon.alimov@sap.com>
Co-authored-by: Jordi van Liempt <35920075+jliempt@users.noreply.github.com>
2023-08-10 16:11:33 +05:00
Christopher Fenner
2be4461cde
Update vault.go (#4464)
Co-authored-by: Anil Keshav <anil.keshav@sap.com>
2023-07-14 09:50:20 +02:00
Jk1484
27c3c3c4c7
feat(vault): support for multiple general purpose credential paths (#4360)
* created wrapper

* tests added

* update documentation

* tests data race fix

---------

Co-authored-by: Jordi van Liempt <35920075+jliempt@users.noreply.github.com>
2023-05-22 10:49:28 +02:00
Jk1484
ffc931aad1
feat(golangBuild): use 'unit' build tag to include tests during test execution (#4345)
* Added unit tag as argument. Added description to runTests command. Changed code generator to have unit build tag in generated unit test files.

* Added unit build tag to all unit test files.

* added to new unit test unit build tag

* Update verify-go.yml

* small fix

---------

Co-authored-by: Muhammadali Nazarov <Muhammadali.Nazarov@acronis.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2023-05-03 21:02:11 +05:00
Christopher Fenner
99603cdee9
chore(tests): correct test assertions (#4276) 2023-04-11 23:15:00 +02:00
tiloKo
126fbbcc69
reduce Log Output (Info -> Debug) (#4322)
Co-authored-by: ffeldmann <f.feldmann@sap.com>
2023-04-06 08:31:22 +02:00
Pavel Busko
81d9a0ac8c
feat(cnbBuild): support Vault general purpose secrets as a binding content source (#4281)
* feat(cnbBuild): support Vault general purpose secrets as a binding content source

Co-authored-by: Pavel Busko <pavel.busko@sap.com>

* fix npm project integration test

Co-authored-by: Pavel Busko <pavel.busko@sap.com>

---------

Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
Co-authored-by: Sumit Kulhadia <sumit.kulhadia@sap.com>
2023-03-31 09:36:59 +02:00
Jordi van Liempt
402c6085c9
feat(checkIfStepActive): enhance with pipelineEnvironmentFilled condition (#3975)
* add CommonPipelineEnvironmentVariableExists to StepCondition

* add CPE variable existance check to checkIfStepActive

* refactor getCPEEntry()

* condition type -> string

* rename CPE filled condition

* rename CPE condition

Co-authored-by: I557621 <jordi.van.liempt@sap.com>
Co-authored-by: Anil Keshav <anil.keshav@sap.com>
2022-08-29 13:44:32 +02:00
Oliver Nocon
85b277c7c6
fix: register base64 encoded secret (#3973)
fixes #3908
2022-08-16 17:52:20 +02:00
Ashly Mathew
374cdb777b
fix(checkIfStepActive): nested key logic (#3939)
* fix(checkIfStepActive): nested key logic

Co-authored-by: Raman Susla <raman_susla@epam.com>
Co-authored-by: Ashly Mathew <ashly.mathew@sap.com>
2022-08-12 09:02:15 +02:00
Oliver Nocon
73f7d61743
fix: remove side-effects of #3875 (#3928)
with #3875 temp directory was created in current workspace.
This had negative side-effects: For example npm build packaged and published temporary files

Co-authored-by: Anil Keshav <anil.keshav@sap.com>
2022-08-04 09:20:59 +02:00
Eng Zer Jun
0f4e30e9db
test: use T.TempDir to create temporary test directory (#3721)
This commit replaces `ioutil.TempDir` with `t.TempDir` in tests. The
directory created by `t.TempDir` is automatically removed when the test
and all its subtests complete.

Prior to this commit, temporary directory created using `ioutil.TempDir`
needs to be removed manually by calling `os.RemoveAll`, which is omitted
in some tests. The error handling boilerplate e.g.
	defer func() {
		if err := os.RemoveAll(dir); err != nil {
			t.Fatal(err)
		}
	}
is also tedious, but `t.TempDir` handles this for us nicely.

Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-07-12 15:19:12 +02:00
raman-susla-epam
6bc96faba9
Update vault.go (#3875) 2022-07-05 16:20:53 +02:00
Martin Zuber
1cb1a203cb
Support custom general purpose vault credentials without prefix. (#3779)
Co-authored-by: Anil Keshav <anil.keshav@sap.com>
2022-06-27 09:24:52 +02:00
Jordi van Liempt
eb73cae888
feat(getDefaults): add parameter for CRD-style config (#3736)
* export LoadConditionsV1() in config pkg

* getDefaults: read stage conditions as PipelineDefinitionV1

* add getDefaults useV1 tests

* add comment (pointed out by code climate analysis)

* small addition to comment (from previous commit)

Co-authored-by: I557621 <jordi.van.liempt@sap.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2022-04-19 12:34:40 +02:00
Anil Keshav
818636b048
feat (checkIfStepActive) including cpe conditions to check if a step is active / not active (#3642)
* including cpe struct

* evaluating types

* adding environment root path

* unit tests

Co-authored-by: anilkeshav27 <you@example.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-03-17 12:09:16 +01:00
Christian Volk
58b30cc99b
fix(golangBuild): publishing of binaries (#3643) 2022-03-17 11:07:51 +01:00
Maurice Breit
9f2d8a70b1
(fix) match regexes in sliceContains to support vaultSecretNames (#3618)
* (fix) match regexes in sliceContains to support vaultSecretNames

* add test for regex matching in sliceContains

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-03-17 09:41:51 +01:00
Oliver Nocon
504f076613
feat: support cpe credentials for multiple repos (#3641) 2022-03-17 08:01:00 +01:00
Anil Keshav
157f6d4e35
feat (checkIfStepActive) enhance to include not active conditions (#3616)
* including negative conditions

* clean up and todos

* removing debug logging

* clean up

* fix unit test name

* fixing unit tests

* negative stage test

Co-authored-by: anilkeshav27 <you@example.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-03-10 13:35:28 +01:00
Christian Volk
b9c0aa008f
feat(piperExecuteBin): support running images from private docker registries (#3622) 2022-03-09 17:35:57 +01:00
Philipp Stehle
4c18f2a128
feat: Add support for deprecating step parameters (#3554)
this was already used in fortifyExecuteScan, but had no effect.

Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>

Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
2022-02-23 15:16:05 +01:00
Ralf Pannemans
e7db09db12
feat(cnbBuild): added dockerimage to the telemetry data (#3501)
Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>
Co-authored-by: Sumit Kulhadia <sumit.kulhadia@sap.com>
Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
2022-02-16 13:28:51 +01:00
Oliver Nocon
2ae1d9dac1
feat(kanikoExecute): allow building multiple images (#3443)
* feat(kanikoExecute): allow building multiple images

* enhance tests

* chore: allow running tests in parallel

* small fixes

* fix: fix destination bug

* update formatting and defaults

* fix yml formatting

* chore: change cpe parameter names

* chore: improve variable naming
2022-02-07 07:58:41 +01:00
Christopher Fenner
ebcbb73b46
feat: add command to fetch default files (#3445)
* some ideas..

* Add getDefaults command (WIP) (#3444)

* add getYAML function for configs

* create getDefaults command(based on getConfig)

* add getDefaults command to CLI

* read defaults files, using github tokens as well

* write defaults to stdout as JSON object with YAMLs embedded

* catch case where no input files are given

* actually write output to file if outputFile is specified

* mark defaultsFile flag as required

* add basic tests

* add output (string)  test

* adapt generateDefaults() to return output (used for test of previous commit)

* Changes to getDefaults() JSON output (#3449)

* change JSON output to contain separate fields

* filename -> filepath

* Apply suggestions from code review

* Apply suggestions from code review

* Update pkg/config/config_test.go

Co-authored-by: Jordi van Liempt <35920075+jliempt@users.noreply.github.com>
2022-01-24 13:55:28 +01:00
Nikolay Grechanov
17510f8996
fix: correct typo in log message (#3439)
Closes #3438
2022-01-19 08:41:11 +01:00
Anil Keshav
f3b65ae43b
feat (npmExecuteScripts) create seperate npmrc file for publish to private repo (#3422)
* creating new npm rc file

* publishing to registry staging

* exposing base64 version of env variables

* changing encoding param

* fixing unit test for the new path

* debugging env var

* remove debug message

* update docu

* changing new npmrc file name

* adding new npmrc to ignore

* adding new npmrc to ignore

Co-authored-by: anilkeshav27 <you@example.com>
2022-01-13 13:08:19 +01:00
Marc Bormeth
78c4a8fe93
Revert "feat: Add debug output to getConfig (#3341)" (#3396)
This reverts commit b760bf48e7.

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2022-01-11 10:47:03 +01:00
Christopher Fenner
51feacadbf
chore: align Vault naming (#3372)
* Align Vault naming

* remove duplicate debug  output

* correct message

* align vault naming

* Update vault.md
2021-12-21 11:52:10 +01:00
Anil Keshav
3a0e7e3e86
fetch general purpose credentials from vault (#3380)
Co-authored-by: anilkeshav27 <you@example.com>
2021-12-21 11:00:13 +01:00
Siarhei Pazdniakou
cd243ee542
feat(gcs): allow upload to gcs from steps (#3034)
* Upload reports to Google Cloud Storage bucket

* Added tests. Made fixes

* Update step generation. GCS client was moved to GeneralConfig

* Code was refactored

* Fixed issues

* Fixed issues

* Code correction due to PR comments

* Improved gcs client and integration tests

* Integrated gcp config. Updated step metadata

* Fixed issues. Added tests

* Added cpe, vault, aliases resolving for reporting parameters

* Added tests

* Uncommented DeferExitHandler. Removed useless comments

* fixed cloning of config

* Added comments for exported functions. Removed unused mock

* minor fix

* Implemented setting of report name via paramRef

* some refactoring. Writing tests

* Update pkg/config/reporting.go

* Update cmd/sonarExecuteScan_generated.go

* Apply suggestions from code review

* Update pkg/config/reporting.go

* Update pkg/config/reporting.go

* fixed removing valut secret files

* Update pkg/config/reporting.go

* restore order

* restore order

* Apply suggestions from code review

* go generate

* fixed tests

* Update resources/metadata/sonarExecuteScan.yaml

* Update resources.go

* Fixed tests. Code was regenerated

* changed somewhere gcp to gcs. Fixed one test

* move gcsSubFolder to input parameters

* fixed removing valut secret files

* minor fix in integration tests

* fix integration tests

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2021-12-15 15:07:47 +01:00
Marc Bormeth
b760bf48e7
feat: Add debug output to getConfig (#3341)
* Fix Orchestrator detection

* Add unit tests

* Add debug output to getConfig

* Update getConfig.go

* fix whitespaces

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2021-12-08 10:16:54 +01:00
kingvvgo
b29a2f2264
Add Changes for value of docker image (#3303)
* Add Changes for value of docker image

* Get docker image value

* Fix

* Fix unit

* Add chnages for kaniko and mta builds

* Fix

* Test changes

* Test

* Move func ResolveMetadata to stepmeta.go

* Fix

* Change getConfig.go

* Fix getting docker value for mta, npm and kaniko

* Fix according to suggestions

* Add func to get only value of docker image

* Test empty value of docker image

* Fix for getDockerImageValue

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-12-01 07:46:18 +01:00
Eugene Kortelyov
d395b362ed
feat: step to execute shell scripts (#3196)
* shell executor initial commit

* functionality updates

* changes in logging implementation (using internal logging), changes in execution

* remove unused field

* remove duplicate from code

* update vault flow and remove unnecessary params

* update generated step file

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-11-19 10:05:39 +01:00
Oliver Nocon
6c5434f957
feat(checkIfStepActive): support new CRD style conditions (#3254)
* feat: first parts of new run struct

* add parts for new stage condition handling

* update conditions

* feat: finalize conditions and tests

* feat(checkIfStepActive): support new CRD style conditions

* feat(docs): allow generating stage docs

* chore(docs): make step directory configurable

* fix: tests

* add option to output file

* Update checkIfStepActive_test.go
2021-11-18 07:24:00 +01:00
Siarhei Pazdniakou
46bafc40a3
Improve validation of configuration (#3125)
* Implemented validation for the option possibleValues

* Has been added the option mandatoryIf to config with validation

* Fixed issues found during code review

* improved golang template

* Fixed tests. Added validation for mandatoryIf option

* Fix typo

* Fixed tests

* Validation was refactored. Added options

* Added default value for parameters with possibleValues option

* Validation was moved after the configuration resolution

* Canceled some default values

Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-10-01 12:49:05 +02:00
Eugene Kortelyov
56be54c504
Feature/vault refactoring (#3113)
* refactor vault code

* adjust generator

* wip: fix tests

* regenerate influxdb

* fix test

* add another test

* fix test & docs

* fix formatting

* Minorupdate and fixes

Co-authored-by: Kevin Stiehl <kevin.stiehl@numericas.de>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-09-21 13:06:32 +02:00
Oliver Nocon
5096b3bfe1
fix(checkIfStepActive): respect explicit (de-)activation (#3055) 2021-08-17 11:20:04 +02:00
Oliver Nocon
97b84429f1
feat(Vault): custom prefix for test credentials (#3043)
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2021-08-11 16:20:08 +02:00
Oliver Nocon
f308fbfc0d
feat(getConfig) allow reading stage configuration (#3026)
* feat(getConfig) allow reading stage configuration

* update test

* use fix version of Selenium Image for Karma tests
2021-08-04 10:09:02 +02:00
Oliver Nocon
6f32b437be
fix(dockerOptions) allow proper usage of empty values (#3025)
This follows up on #3024
Setting emptyValue to s.th. like `--entrypoint=''` will break in case the argument is properly escaped.

Docker will return with
`container process caused: exec: "''": executable file not found in $PATH`
2021-08-02 14:57:37 +02:00
Oliver Nocon
f455a8a4a0
fix(dockerOptions) properly pass empty values (#3024)
it is possible to overwrite the entrypoint for docker execution:
https://docs.docker.com/engine/reference/run/#entrypoint-default-command-to-execute-at-runtime

This is ideally done by passing `entrypoint=''` and not pass two options to the call.
This also helps with escaping issues of the empty value on other systems.

Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2021-08-02 10:21:14 +02:00
Siarhei Pazdniakou
7a325e6fc8
Convert step/stage condition logic to golang (#2993)
* Added checkIfStepActive step

* Implemented npmScripts condition. Code was refactored

* Added some unit tests

* Fixed go modules

* Fixed go modules

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-07-26 07:47:43 +02:00
Oliver Nocon
805a8fd88f
feat(config): read config/defaults with authentication (#2975)
* feat(config):read config/defaults with authentication

This change allows to use defaults and config files from a protected GitHub repository.

The options `--customConfig` and `--defaultConfig` already allowed to provide a link to an uprotected file.

Now, by passing a value in the form `<hostname>:<token>` to parameter `gitHubTokens` (this parameter can be passed multiple times) a token can be provided for dedicated hosts.

This makes it possible to use a link like
`https://api.github.com/repos/SAP/jenkins-library/contents/resources/my-defaults.yml?ref=master`
as reference to a default file or similarly as reference to a configuration file.

* update generation to allow protected config/defaults

* fix CodeClimate issues

* update missing generations
2021-07-08 15:26:07 +02:00
Oliver Nocon
0b48bfcc73
feat: retrieve metadata by stepName - corrected (#2892)
* refactored getConfig to allow stepName param for metadata fetching

* extended step generator

* go generate

* Update cmd/getConfig.go

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Update cmd/getConfig.go

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Update cmd/getConfig.go

* update generated files

* update golden files to care for generator update

* update and add tests

* update generated files

* Update cmd/getConfig.go

* Update cmd/getConfig.go

* update/fix formatting

* feat: retrieve metadata by stepName - corrected

* update generation

* update condition logic for defaults

* update generation & tests

* support multiple conditions

* update generation

* Add generated

Co-authored-by: Leander Schulz <leander.schulz01@sap.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Sven Merk <s.merk@sap.com>
2021-06-16 08:43:30 +02:00
ffeldmann
b88ebdad6c
feat(splunk) Sending telemetry and logging information to Splunk (#2796)
* Adds inital splunk hook and logCollector

* Adds documentation of the Splunk hook

* Fixes markdown lint issues and removes comment from telemetry.go file

* Fixes markdown lint issues and adds missing generated file

* Markdown linting

* Changes documentation according to review, adds Splunk token automatically during init

* Adds error handling for marshalling hook config

* Markdown lint und correct Splunk token in httpclient

* Registeres Splunk token as secret and adjusts test cases

* Adds missing error handling and removes unnecessary comments

* Creates new function readPipelineEnvironment, adds tests

* Moves MonitoringData struct, edits defaults for json fields

* Adds gitRepository and gitOwner to telemetry information

* Simplifies readCommonPipelineEnvironment function, adds more descriptive errors, adds automated adding of Splunk prefix token

* Adjusts error handling

* Cleaner error logging
2021-05-17 12:14:04 +02:00
Christopher Fenner
98fac0a455
chore(deprecation): change format of deprecation notice from config command (#2822)
* Update config.go

* Update config.go

* add library name

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-05-10 19:18:16 +02:00
Siarhei Pazdniakou
0673d3fed6
Writing secrets to vault (#2770)
Co-authored-by: Kevin Stiehl <kevin.stiehl@numericas.de>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2021-05-06 10:27:23 +02:00