1
0
mirror of https://github.com/SAP/jenkins-library.git synced 2024-12-14 11:03:09 +02:00
Commit Graph

35 Commits

Author SHA1 Message Date
Oliver Nocon
805a8fd88f
feat(config): read config/defaults with authentication (#2975)
* feat(config):read config/defaults with authentication

This change allows to use defaults and config files from a protected GitHub repository.

The options `--customConfig` and `--defaultConfig` already allowed to provide a link to an uprotected file.

Now, by passing a value in the form `<hostname>:<token>` to parameter `gitHubTokens` (this parameter can be passed multiple times) a token can be provided for dedicated hosts.

This makes it possible to use a link like
`https://api.github.com/repos/SAP/jenkins-library/contents/resources/my-defaults.yml?ref=master`
as reference to a default file or similarly as reference to a configuration file.

* update generation to allow protected config/defaults

* fix CodeClimate issues

* update missing generations
2021-07-08 15:26:07 +02:00
Oliver Nocon
0b48bfcc73
feat: retrieve metadata by stepName - corrected (#2892)
* refactored getConfig to allow stepName param for metadata fetching

* extended step generator

* go generate

* Update cmd/getConfig.go

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Update cmd/getConfig.go

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Update cmd/getConfig.go

* update generated files

* update golden files to care for generator update

* update and add tests

* update generated files

* Update cmd/getConfig.go

* Update cmd/getConfig.go

* update/fix formatting

* feat: retrieve metadata by stepName - corrected

* update generation

* update condition logic for defaults

* update generation & tests

* support multiple conditions

* update generation

* Add generated

Co-authored-by: Leander Schulz <leander.schulz01@sap.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Sven Merk <s.merk@sap.com>
2021-06-16 08:43:30 +02:00
Oliver Nocon
4250ca8bed
Revert "feat(getConfig): retrieve metadata by stepName (#2736)" (#2891)
This reverts commit ae4a24c594.
2021-06-14 10:57:44 +02:00
lndrschlz
ae4a24c594
feat(getConfig): retrieve metadata by stepName (#2736)
* refactored getConfig to allow stepName param for metadata fetching

* extended step generator

* go generate

* Update cmd/getConfig.go

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Update cmd/getConfig.go

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Update cmd/getConfig.go

* update generated files

* update golden files to care for generator update

* update and add tests

* update generated files

* Update cmd/getConfig.go

* Update cmd/getConfig.go

* update/fix formatting

Co-authored-by: Leander Schulz <leander.schulz01@sap.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: OliverNocon <oliver.nocon@sap.com>
2021-06-14 08:58:41 +02:00
Christopher Fenner
8c5a7f5db5
feat(docs): add possibleValues for artifactPrepareVersion (#2865)
* correct docs

* update generated sources
2021-06-04 12:20:24 +02:00
Oliver Nocon
d8a8a73184
feat(commonPipelineEnvironment): add artifact coordinates (#2867)
* add coordinates to CPE

* revert go.mod changes
2021-06-02 15:41:05 +02:00
ffeldmann
52c25a0dbb
feat(splunk): Adds head commitID of current build to commonPipelineEnvironment (#2844)
* Adds headCommitId, which stores the head commit has of the current build, includes tests and generated files

* Adds headCommitId, which stores the head commit has of the current build, includes tests and generated files

* Go fmt fix

* Fixes artifactoryPrepareVersion test

* Removes xMake CommitId
2021-05-20 13:11:57 +02:00
ffeldmann
b88ebdad6c
feat(splunk) Sending telemetry and logging information to Splunk (#2796)
* Adds inital splunk hook and logCollector

* Adds documentation of the Splunk hook

* Fixes markdown lint issues and removes comment from telemetry.go file

* Fixes markdown lint issues and adds missing generated file

* Markdown linting

* Changes documentation according to review, adds Splunk token automatically during init

* Adds error handling for marshalling hook config

* Markdown lint und correct Splunk token in httpclient

* Registeres Splunk token as secret and adjusts test cases

* Adds missing error handling and removes unnecessary comments

* Creates new function readPipelineEnvironment, adds tests

* Moves MonitoringData struct, edits defaults for json fields

* Adds gitRepository and gitOwner to telemetry information

* Simplifies readCommonPipelineEnvironment function, adds more descriptive errors, adds automated adding of Splunk prefix token

* Adjusts error handling

* Cleaner error logging
2021-05-17 12:14:04 +02:00
lndrschlz
b9bab27833
feat: expose complete step metadata through generated function (#2329)
* exposing step metadata through generator

* add metadata_generated.go

* fix step go test generation

* metadata fields added to generated files

* added generated files

* removed image placeholder from fortify step

* refactored step meta generation

* go generate

* fixed metadata generator and tests

* added output resource fields/tags to metadata generator

* fix string in metadata_generated

* go generate

* fixed generator

* go generate

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-11-20 08:13:59 +01:00
Kevin Stiehl
72dd34d910
rename versioning credential to "gitHttpsCredential" (#2185)
* rename versioning credential to "gitHttpsCredential"

* renmae secret to gitHttpsCredential

Co-authored-by: lndrschlz <leander.schulz01@sap.com>
2020-10-26 15:55:37 +01:00
Kevin Stiehl
24aafb0b69
add vaultSecretFileReferences (#2203)
* add vaultSecretFileReferences

* fix test

* fix test

* go generate

* remove code duplication

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-10-26 14:20:04 +01:00
Kevin Stiehl
3eae0c5f68
feat(vault): fetch secrets from vault (#2032)
* cloud-foundry & sonar from vault

* add vault development hint

* don't abort on vault errors

* cloudfoundry make credentialsId only mandatory when vault is not configured

* add vault ref to step ymls

* rename vaultAddress to vaultServerUrl

* rename PIPER_vaultRole* to PIPER_vaultAppRole*

* add resourceRef for detect step

* fix error when no namespace is set

* added debug logs

* added debug logs

* fix vault resolving

* add vaultCustomBasePath

* rename vault_test.go to client_test.go

* refactored vault logging

* refactored config param lookup for vault

* added tüddelchen

* rename vaultCustomBasePath to vaultPath

* fix tests

* change lookup path for group secrets

* fix interpolation tests

* added vault resource ref to versioning

* execute go generate

* rename Approle to AppRole

* change verbose back to false

Co-authored-by: Leander Schulz <leander.schulz01@sap.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-10-13 14:14:47 +02:00
Christopher Fenner
be90876b7c
feat(output): handle non-string output values (#2113)
* handle non-string values as JSON

* change value type to interface in resources

* regenerate code

* add test cases

* handle reading of json files

* write json data to json files

* fix assignment

* use GetResourceParameter

* add test case

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-10-05 15:33:28 +02:00
Oliver Nocon
19c1732826
Telemetry: report error category (#2085) 2020-09-29 13:49:40 +02:00
Christopher Fenner
b219fb6514
fix(typo): found by misspell (#2064)
* fix typos in step yamls

* fix typos in go files

* regenerate step code

* fix typos in md files

* fix typos in groovy files

* fix further typos
2020-09-24 07:41:06 +02:00
Kevin Stiehl
d589038206
Vault AppRole login (#1971)
* added interpolation package in config

* vault allow paths to hold config references

* allow referencing properties in vaultPaths

* fix small typo

* add approleAuth

* register resolved secrets to logger

* generate steps

* clean up

* add integration test

* add vault to context filter

* reduce Cognitive Complexity & added tests

* Update pkg/config/stepmeta_test.go

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>

* go generate

* go generate after merge

* rename VaultAppRole* to VaultRole*

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-09-16 14:50:09 +02:00
Oliver Nocon
bb8cd30c4c
Extend resource references to cater for secrets (#1718)
* Add secret references to step metadata

* Update references

* update generated files
2020-06-25 16:33:38 +02:00
Oliver Nocon
eafe383d54
Add error category parsing to cmd execution (#1703)
* Add error category parsing to cmd execution

It is now possible to define `ErrorCategoryMapping` as a `map[string][]string` on a `Command`.
The format contains the category as key which has a list of error patterns assigned.
Example:

```
cmd := Command{
  ErrorCategoryMapping: map[string][]string
    "build": {"build failed"},
    "compliance": {"vulnerabilities found", "outdated components found"},
    "test": {"some tests failed"},
  },
}
```

Setting this map triggers console log parsing when executing a command.
If a match is found the error category is stored and
it will automatically be added to the `errorDetails.json`.

* clean up go.mod

* fix test

* fix test

* Update DEVELOPMENT.md

* fix tests

* address long console content without line breaks

* scan condition update

* fix test

* add missing comment for exported function

* Update pkg/command/command.go

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-06-24 10:04:05 +02:00
Christopher Fenner
c42553593e
fix: remove unused variables from generated step coding (#1698)
* remove unused parameters from generated code

* update steps

* update steps

* correct golden files

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-06-23 18:05:21 +02:00
Stephan Aßmus
56699ab019
Run artifactPrepareVersion in maven container if necessary (#1634)
* The metadata for artifactPrepareVersion-go specifies a container for when the buildTool is maven.
* The alias to 'mavenExecute' was removed. The problem with this is that when a section containers is included in the metadata, dockerImage will always be picked up from mavenExecute, the conditional dependency on buildTool will not even be considered. Parameters such as m2Path, projectSettingsFile and globalSettingsFile should be configured in general/maven if necessary.
* When the step ends up being executed within dockerExecuteOnKubernetes, we need to preserve the .git folder. This folder would normally be excluded by the default excludes of the stash step. There was already a comment that suppressing this behavior by passing useDefaultExcludes: false was problematic (unfortunately without going into details), so I've added a new parameter to dockerExecute and dockerExecuteOnKubernetes named stashNoDefaultExcludes (note the reverted meaning to ease preserving the default behavior when this parameter is not provided). This parameter is passed to piperExecuteBin from the artifactPreferVersion groovy wrapper.
2020-06-08 17:08:05 +02:00
Maximilian Lenkeit
10e5bf98f8
artifactPrepareVersion: remember original artifact version (#1600)
* Update resources.go

* expose originalArtifactVersion parameter

* re-generate steps

* adjust golden files for generator changes
2020-05-28 10:05:22 +02:00
Sven Merk
af2a01c064
Fortify implementation in golang (#1428) 2020-05-25 19:48:59 +02:00
Oliver Nocon
7f7d0bce89
Piper steps: Issue SUCCESS message (#1478)
* Piper steps: Issue SUCCESS message

Log `SUCCESS` message in all steps by default if no error occured

* Update additional generated steps

* Update generated step
2020-05-18 21:52:54 +02:00
Sven Merk
911b8bfb26
Add resolution and storage of commit message (#1515)
* Add commit message

* Add ToDo for later refctoring
2020-05-07 11:21:29 +02:00
Oliver Nocon
f7fdd79fa7
artifactPrepareVersion: additional options (#1512)
* artifactPrepareVersion: additional options

it is now possible to use
* a Unix timestamp number
* a short commit hash

* Add dedicated Docker versioning scheme
2020-05-06 22:07:27 +02:00
lndrschlz
39e30ff1b4
Added hook configuration; added sentry hook and tests (#1497)
* Added hook configuration; added sentry hook and tests
* steps generated by step framework
* added comments and renamed correlationID
* formatted comment on NewSentryHook method
* fix generator step tests
2020-05-05 08:36:24 +02:00
Oliver Nocon
43947e6ef5
Pass golang error details to Jenkins pipeline (#1443)
Do not exit with os.Exit(1) but using log.Entry().Fatal() instead
* Golang: forward error details
* extend groovy wrapper to provide proper error message
* create closure for error handling
2020-04-28 07:42:02 +02:00
Kevin Hudemann
0be3225a3c
Enable the use of artifactPrepareVersion instead of artifactSetVersion (#1455)
This change enables the use of the artifactPrepareVersion go step instead
of the deprecated artifactSetVersion groovy implementation for the Cloud
SDK Pipeline. The groovy wrapper for the artifactPrepareVersion step now
enables the use of the download cache and minor issues are fixed.
2020-04-27 09:56:58 +02:00
Oliver Nocon
a728416e12
artifactPrepareVersion: define versioning scheme (#1457) 2020-04-24 20:52:16 +02:00
Oliver Nocon
75a10742d3
Step generator: correct error handling (#1429)
Do not exit with os.Exit(1) but using log.Entry().Fatal() instead
2020-04-20 16:52:57 +02:00
Daniel Kurzynski
ce4be51362
Mask secrets (#1382) 2020-04-16 14:37:45 +02:00
Oliver Nocon
b9781ce50c
artifactPrepareVersioning: support more buildTools (#1367)
* artifactPrepareVersioning: support more buildTools
2020-04-15 13:12:43 +02:00
Oliver Nocon
ef99f710ae
artifactPrepareVersion: documentation (#1365)
* artifactPrepareVersion: documentation

* Fix CodeClimate finding

* Update generated file

* Update versioning.yaml

* Update docs wrt PR 1374

* update generated step

* Update resources/metadata/versioning.yaml

Co-Authored-By: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Update resources/metadata/versioning.yaml

Co-Authored-By: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* update generated file

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-04-08 09:28:03 +02:00
Oliver Nocon
909c77045d
artifactPrepareVersion: new versioning option (#1374) 2020-04-07 13:27:36 +02:00
Oliver Nocon
999197b919
Add step to prepare the version for an artifact (#1343) 2020-04-03 16:34:40 +02:00