Commit Graph
100 Commits
Author SHA1 Message Date
Oliver NoconandGitHub 70d8c21525 Update README.md 2020-08-11 12:01:39 +02:00
Oliver NoconandGitHub 1a7de49cad compliance with reuse (#1910) 2020-08-11 11:53:42 +02:00
Oliver NoconandGitHub d34f92cd00 kubernetesDeploy: cover helm ImagePullSecrets (#1878)
* kubernetesDeploy: cover helm ImagePullSecrets

* update test

* fix formatting

* fix compatibility
2020-08-06 11:23:36 +02:00
Oliver NoconandGitHub 69b145e83f artifactPrepareVersion: improve error categorization (#1889)
* artifactPrepareVersion: improve error categorization
* go mod tidy
2020-08-06 11:13:19 +02:00
Oliver NoconandGitHub 1c61ba1de3 mtaBuild: improve error categorization (#1873) 2020-07-31 15:41:28 +02:00
Oliver NoconandGitHub 3318ba24e4 artifactPrepareVersion: improve error categorization (#1872) 2020-07-31 14:55:22 +02:00
77028bf50c kubernetesDeploy: fix commonPipelineEnvironment references (#1863)
* kubernetesDeploy: add step configuration alias

This is used for backward compatibility

* fix commonPipelineEnvironment references

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-07-29 15:48:55 +02:00
bcd450f06e kubernetesDeploy: add step configuration alias (#1857)
This is used for backward compatibility

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-07-29 11:21:32 +02:00
d691750931 Fix reading of hook configuration (#1854)
* Update piper.go

* Update piper_test.go

* Update config.go

* Update sentryHook.go

* Update piper.go

* go fmt

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-07-28 17:19:33 +02:00
Oliver NoconandGitHub d8553ab53d detectExecuteScan: update versioning (#1845)
* detectExecuteScan: update versioning

align with Fortify to also use the same versioning model by default.

* fix CodeClimate findings
2020-07-27 12:01:59 +02:00
Oliver NoconandGitHub 97737a2d8d detectExecuteScan: fix docs (#1835)
The step is golang based thus has no dependency to a Jenkins plugin.
2020-07-24 13:45:52 +02:00
Oliver NoconandGitHub e09b05b22e DocGenerator: fix links to secrets (#1825)
* DocGenerator: fix links to secrets
* Update docuHelper.go
2020-07-22 10:33:36 +02:00
220748d150 Step generation: allow usage of go generate (#1793)
* Step generation: allow usage of go generate

* Update DEVELOPMENT.md

* move generate command to main.go (#1809)

* Update main.go

* Update DEVELOPMENT.md

* Update piper.go

* align path usage

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-07-17 08:06:11 +02:00
Oliver NoconandGitHub c622b3dfac Documentation: fix parameter links & some spelling (#1807)
* Documentation: fix parameter links & some spelling
* Fix new line character
* update checkboxes
2020-07-16 11:34:02 +02:00
43f51ba90a Revamp documentation generation for golang steps (#1781)
* Revamp documentation generation for golang steps

* Add and update tests

* Add tests, add step outputs

* Add tests, add step outputs

* Update rendering

* fix merge conflict

* update generated files

* Update pkg/config/stepmeta.go

* Update pkg/config/stepmeta.go

* Update pkg/generator/helper/docuHelper.go

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Include PR feedback

* update tests

* Fix Jenkins dependecy indication

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-07-16 09:10:15 +02:00
Oliver NoconandGitHub 111e4de8c3 kanikoExecute - golang version (#1765)
* kanikoExecute golang version

* update entrypoint

* update entrypoint

* update entrypoint

* update entrypoint

* update entrypoint

* add command

* include PR feedback

* remove trailing spaces

* remove defaults for certificates

* Update generated file
2020-07-10 08:07:59 +02:00
Oliver NoconandGitHub 520f2cc975 Fix stashing of piper binary (#1764)
* Update piperExecuteBin.groovy

* update tests
2020-07-06 13:22:36 +02:00
49b7abb4ca Fix stashing issue with binrary execution (#1749)
Co-authored-by: Daniel Kurzynski <daniel.kurzynski@sap.com>
2020-07-02 12:08:56 +02:00
Oliver NoconandGitHub dc296b0727 Allow retrieving exit code from command execution (#1728)
* Allow retrieving exit code from command execution

This will be helpful to derive error categories in case
an executable provides context-specific error codes.

* make sure that we always have a non 0 exit code for errors
2020-06-30 12:57:27 +02:00
Oliver NoconandGitHub ac5ae40ecb kubernetesDeploy: categorize known errors & fix #1732 (#1727)
* kubernetesDeploy: categorize known errors

* hide Docker secret in log output
2020-06-30 10:48:30 +02:00
Oliver NoconandGitHub 54850d2ca0 sonarExecuteScan: add error categorization (#1715) 2020-06-26 07:38:27 +02:00
Oliver NoconandGitHub bb8cd30c4c Extend resource references to cater for secrets (#1718)
* Add secret references to step metadata

* Update references

* update generated files
2020-06-25 16:33:38 +02:00
eafe383d54 Add error category parsing to cmd execution (#1703)
* Add error category parsing to cmd execution

It is now possible to define `ErrorCategoryMapping` as a `map[string][]string` on a `Command`.
The format contains the category as key which has a list of error patterns assigned.
Example:

```
cmd := Command{
  ErrorCategoryMapping: map[string][]string
    "build": {"build failed"},
    "compliance": {"vulnerabilities found", "outdated components found"},
    "test": {"some tests failed"},
  },
}
```

Setting this map triggers console log parsing when executing a command.
If a match is found the error category is stored and
it will automatically be added to the `errorDetails.json`.

* clean up go.mod

* fix test

* fix test

* Update DEVELOPMENT.md

* fix tests

* address long console content without line breaks

* scan condition update

* fix test

* add missing comment for exported function

* Update pkg/command/command.go

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-06-24 10:04:05 +02:00
Oliver NoconandGitHub d8444d51f2 Add possibility to add category to failures (#1692)
* Add possibility to add category to failures

It is now possible to set the error category within the flow.
When exiting the program the error category can be used.

There is  a convenience function available for exiting with a previously set category,
for example

```
log.SetErrorCategory(log.ErrorCompliance)
...
log.FatalError(err, "configuration error")
```

* extend test

* go mod tidy

* add missing comment

* update information about error categories

* Update DEVELOPMENT.md
2020-06-22 08:21:38 +02:00
Oliver NoconandGitHub d602c6e2b9 extend security stage (#1687)
* extend security stage

Add new steps
* checkmarxExecuteScan
* fortifyExecuteScan

closes #1650

* update tests
2020-06-19 12:35:02 +02:00
8e74952166 Switch Detect scan to golang implementation (#1673)
* Switch Detect scan to golang implementation

* update workingDir

* use universal container as default

* Switch to official curl image

* fix code climate issue

* switch default image

* update docker options

* update CommonStepsTest

* remove invalid groovy test

* clean up defaults

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-06-16 17:21:59 +02:00
Oliver NoconandGitHub f125ba5b60 adding groovy wrapper for containerSaveImage (#1663) 2020-06-15 16:44:49 +02:00
Oliver NoconandGitHub 6116fe5528 Decouple config reading and retrieval (#1668)
There are use cases where we need to run `GetStepConfig()` multiple times.
In such cases it is more efficient to load the files once and then resolve the
respective step configuration.
2020-06-15 14:17:59 +02:00
Oliver NoconandGitHub 50b2a2d6fc Update deprecation message (#1656) 2020-06-12 15:19:13 +02:00
6053cae6c5 Cleanup go.mod (#1603)
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-05-28 11:04:28 +02:00
Oliver NoconandGitHub 5e27d26a7c whitesourceExecuteScan: add Docker image scanning (#1556)
* whitesourceExecuteScan: add Docker image scanning

* add test

* Update whitesourceExecuteScan.groovy

* Update WhitesourceConfigurationHelper.groovy
2020-05-27 16:27:36 +02:00
Oliver NoconandGitHub 243d196283 Add package for whitesource (#1577) 2020-05-26 17:30:45 +02:00
Oliver NoconandGitHub d5d510719d piperExecuteBin - prevent non-speaking exit code 1 error (#1584)
* piperExecuteBin - prevent non-speaking exit code 1 error

Make sure that error during context resolution gets more context.

* properly provide golang errors

* improve error message, when no details are available

* Add error category

* Update getConfig.go
2020-05-26 07:58:03 +02:00
Oliver NoconandGitHub f92771003c detectExecuteScan - switch off phone home (#1582) 2020-05-25 09:44:42 +02:00
Oliver NoconandGitHub 7e2ab744a5 mtaBuild - change default for docker image (#1574)
* mtaBuild - change default for docker image

* fix resolution of container default from step definition

* Properly fix default for container conditions
2020-05-22 09:48:11 +02:00
Oliver NoconandGitHub a0ded365c5 Fix container condition handling (#1562)
* Add additional tests
* Update handling of containers
* Properly consider existing config at root level
* Fix parameter filtering
* Fix tests for adapted filter handling
2020-05-20 10:50:35 +02:00
Oliver NoconandGitHub 09b2ddbafa piperExecuteBin: allow easier re-use (#1566)
* piperExecuteBin: allow easier re-use
* make binary path configurable in groovy step using `piperExecuteBin`
2020-05-19 16:49:21 +02:00
Oliver NoconandGitHub 7f7d0bce89 Piper steps: Issue SUCCESS message (#1478)
* Piper steps: Issue SUCCESS message

Log `SUCCESS` message in all steps by default if no error occured

* Update additional generated steps

* Update generated step
2020-05-18 21:52:54 +02:00
d5bff20b10 Create step to save container image (#1554)
* Create step to save container image

* add error handling for directory creation

* Update permissions

* remove indirection and add tests

* finalize tests

* Fix test to also run on linux environments

* Update cmd/containerSaveImage.go

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>

* Update resources/metadata/savecontainer.yaml

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>

* Update cmd/containerSaveImage.go

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>

* Update cmd/containerSaveImage_test.go

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>

* Update generated file

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-05-18 08:55:25 +02:00
Oliver NoconandGitHub 37a6d456fe Fix generator for custom steps (#1522) 2020-05-08 10:41:01 +02:00
Oliver NoconandGitHub f7fdd79fa7 artifactPrepareVersion: additional options (#1512)
* artifactPrepareVersion: additional options

it is now possible to use
* a Unix timestamp number
* a short commit hash

* Add dedicated Docker versioning scheme
2020-05-06 22:07:27 +02:00
Oliver NoconandGitHub 033ff8da6f go: align used yaml package (#1504) 2020-05-05 15:54:50 +02:00
Oliver NoconandGitHub 3f1a9d004a Allow proper usage of resilience feature in golang steps (#1487)
* Allow proper usage of resilience feature in golang steps
2020-04-30 10:39:27 +02:00
363f69da29 Handle illegal verbose value (#1485)
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-04-29 15:05:00 +02:00
Oliver NoconandGitHub a395cd75be Docker versioning: fall back to custom (#1484)
* Docker versioning: fall back to custom

* Update test and fix issue

* pass proper error message

* Fix test
2020-04-29 13:42:14 +02:00
e8c6baca42 Golang errors: improve error message in groovy part (#1482)
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-04-29 11:26:29 +02:00
Oliver NoconandGitHub 143162bf46 Revert breaking change with #1455 (#1472) 2020-04-28 08:04:51 +02:00
Oliver NoconandGitHub 43947e6ef5 Pass golang error details to Jenkins pipeline (#1443)
Do not exit with os.Exit(1) but using log.Entry().Fatal() instead
* Golang: forward error details
* extend groovy wrapper to provide proper error message
* create closure for error handling
2020-04-28 07:42:02 +02:00
Oliver NoconandGitHub d27e946280 Update artifactPrepareVersion.groovy (#1470) 2020-04-27 17:20:56 +02:00
b9450702af piperExecuteBin: fix reading environment (#1452)
* piperExecuteBin: fix reading environment
* update tests

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-04-24 22:41:34 +02:00
Oliver NoconandGitHub a728416e12 artifactPrepareVersion: define versioning scheme (#1457) 2020-04-24 20:52:16 +02:00
Oliver NoconandGitHub 75a10742d3 Step generator: correct error handling (#1429)
Do not exit with os.Exit(1) but using log.Entry().Fatal() instead
2020-04-20 16:52:57 +02:00
Oliver NoconandGitHub 3a9bc54711 protecodeExecuteScan - align user parameter (#1426)
align parameter name for scan user with other steps
2020-04-20 16:44:01 +02:00
Oliver NoconandGitHub 067eb0ba6a Improve error messages for test failures (#1417) 2020-04-17 11:59:51 +02:00
Oliver NoconandGitHub 4061881b53 UIVeri5 & Newman: provide better error messages (#1410) 2020-04-16 09:14:39 +02:00
bd0eed30ff stageWrapper: new error handling (#1401)
* handlePipelineStepErrors: allow disabling for stages
* update error handling in stages
* move file loading into try catch
* Cleanup DebugReport before test

Apparantly, the same DebugReport instance is loaded
during all Unit Tests. Avoid left-overs from previous
tests.

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-04-15 14:02:41 +02:00
Oliver NoconandGitHub b9781ce50c artifactPrepareVersioning: support more buildTools (#1367)
* artifactPrepareVersioning: support more buildTools
2020-04-15 13:12:43 +02:00
ef99f710ae artifactPrepareVersion: documentation (#1365)
* artifactPrepareVersion: documentation

* Fix CodeClimate finding

* Update generated file

* Update versioning.yaml

* Update docs wrt PR 1374

* update generated step

* Update resources/metadata/versioning.yaml

Co-Authored-By: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Update resources/metadata/versioning.yaml

Co-Authored-By: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* update generated file

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-04-08 09:28:03 +02:00
Oliver NoconandGitHub 909c77045d artifactPrepareVersion: new versioning option (#1374) 2020-04-07 13:27:36 +02:00
Oliver NoconandGitHub 999197b919 Add step to prepare the version for an artifact (#1343) 2020-04-03 16:34:40 +02:00
Oliver NoconandGitHub 91b6b0900f Config and Defaults via http(s) (#1302)
* Config and Defaults via http(s)

Add the possibility to use http(s) urls as path to
* config.yml
* defaults

* Remove comment

* Addressing PR feedback

* Fix error handling
2020-03-31 08:47:09 +02:00
Oliver NoconandGitHub 8fbeddb26c Golang step metadata: Config aliases for steps (#1293)
* Golang step metadata: Config aliases for steps

This will ease following scenarios:
* config migration due to step name changes
* re-use of more general config, e.g. `mavenExecute` in `mavenBuild`

* fix CodeClimate finding

* Fix panic if original stage config does not exist yet
2020-03-19 17:24:35 +01:00
Oliver NoconandGitHub 289bb58e66 Fix issue with pipeline resilience (#1288)
closes #1286 which was introduced with #1146
2020-03-18 08:32:28 +01:00
Oliver NoconandGitHub a49d8947c0 Streamline calling piper go binary (#1254) 2020-03-17 09:19:09 +01:00
Oliver NoconandGitHub a0223a2a0e Fix issue with deployment related stashing in piperPipeline (#1281)
relates to #1267
2020-03-16 12:52:58 +01:00
Oliver NoconandGitHub dd5cfcc1aa Added documentation of stepParameters being a Groovy map, requiring at least the script reference (#1248)
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-03-05 17:16:53 +01:00
Oliver NoconandGitHub 622d559eae Add ServiceKey deletion to cloudFoundryDeleteService step (#1177) 2020-03-05 15:35:43 +01:00
Oliver NoconandGitHub fd76f721ea Streamline implementation githubPublishRelease (#1191) 2020-02-19 09:05:53 +01:00
Oliver NoconandGitHub e81f40f645 Provide foundation for integration-like tests (#1131) 2020-02-18 15:19:50 +01:00
Oliver NoconandGitHub a5d4c64bf4 Correct step generator (#1190) 2020-02-18 08:31:54 +01:00
Oliver NoconandGitHub ec4fb1b186 Update generator (#1188)
Provide better starting point for new steps
2020-02-17 13:30:30 +01:00
Oliver NoconandGitHub 321f8fbbc1 More resilient handling of download errors for piper bin (#1147)
* fix #1114
* Update test/groovy/com/sap/piper/PiperGoUtilsTest.groovy
* add log output
2020-02-12 09:02:39 +01:00
Oliver NoconandGitHub d5bc8db50e Add missing pages to documentation (#1172)
* Create spinnakerTriggerPipeline.md
* Update mkdocs.yml
2020-02-12 08:53:32 +01:00
Oliver NoconandGitHub 61da9faf98 Support InfluxDB plugin in version 2.0 (#1130)
* Support InfluxDB plugin in version 2.0
closes #932
superseeds #933
2020-02-06 11:27:31 +01:00
cead027bfb Support yml and yaml extension of config files (#1153)
* Support yml and yaml extension of config files
* Update cmd/piper.go

Co-Authored-By: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-02-06 09:08:15 +01:00
9c1bd04752 Streamline step generation (#1142)
* Streamline step generation
* Include PR feedback, update DEVELOPMENT.md

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2020-02-04 10:46:43 +01:00
b542d15998 Enhance telemetry reporting (#1133)
* Enhance telemetry reporting
* Use central telemetry data object
* Add duration

Co-Authored-By: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-01-31 14:06:08 +01:00
Oliver NoconandGitHub 152c4dc9d2 Pass container image via CPE (#1113)
* Pass container image via CPE
* Update generated step
* update map name to better represent content
2020-01-29 16:03:18 +01:00
73ab887f25 Add kubernetesDeploy step (#1073)
* Add kubernetesDeploy step

Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2020-01-24 14:30:27 +01:00
a46b57e6b4 Persisted pipeline environment for golang library (#1091)
* Use commonPipelineEnvironment in go binary

* Update groovy part incl. tests

* Rework structure and naming

* Support influx resources in steps

* Update tests and some cleanups

* Add correct defer handling

* Address PR feedback

* Fix test

* Update resources.go

Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
2020-01-15 12:16:25 +01:00
Oliver NoconandGitHub 2670b4bc2e Merge branch 'master' into nevskrem-patch-1 2019-12-16 20:45:07 +00:00
Oliver NoconandGitHub 7597c7d228 Add step githubCreatePullRequest (#1068)
* Add step githubCreatePullRequest
2019-12-16 16:34:12 +00:00
Oliver NoconandGitHub 4f57738888 Detect: Add golang implementation (#1049)
Add detect golang implementation
2019-12-13 09:55:45 +00:00
Oliver NoconandGitHub df1c61dcd7 Add http package (#1033)
* Add http package

This serves for http calls in future piper steps implemented ingolang
2019-12-09 17:35:31 +01:00
Oliver NoconandGitHub 2aa34c7a3f Fix issue with removed parameter --stepName (#1007) 2019-11-25 08:58:16 +01:00
Oliver NoconandGitHub 8e29b70ea2 Fix parameters for githubPublishrelease (#1004) 2019-11-22 15:58:24 +01:00
Oliver NoconandGitHub 69bb2c9e60 Fix issues in config resolution (#1000)
* Respect defaults from step definition everywhere
* prevent empty config to overwrite default in aliasing

Address PR feedback
2019-11-22 10:30:44 +01:00
Oliver NoconandGitHub 3c29e91951 Update parameters to githubPublishRelease (#999) 2019-11-22 08:09:27 +01:00
Oliver NoconandGitHub e671094308 Use piper binary for githubPublishRelease (#990)
* Use piper binary for githubPublishRelease
2019-11-21 16:34:22 +01:00
Oliver NoconandGitHub 0f33f7a2ef Consider custom defaults in config.yml (#995)
* Consider custom defaults in config.yml
2019-11-21 16:09:57 +01:00
Oliver NoconandGitHub a822c2026e Add publishing of master binary to latest release (#972)
* Add publishing of master binary to latest release
* Update travis.yml
2019-11-12 16:34:05 +01:00
Oliver NoconandGitHub da0935c0e5 Patch Urls (#929)
* Patch Urls
* fix test
2019-11-11 17:40:23 +01:00
Oliver NoconandGitHub d7efd33a5b Fix bug in step generator (#964)
When using step generator outside of the current package the `PrepareConfig` call was not done using the prefix.
2019-11-11 09:42:47 +01:00
Oliver NoconandGitHub dca4a079b7 Refactor structure of generator (#960) 2019-11-08 15:35:11 +01:00
Oliver NoconandGitHub fedfb74910 Export step generator (#958)
* Export step generator
* Fix CodeClimate finding
2019-11-07 15:42:22 +01:00
Oliver NoconandGitHub ede322c8bb Export general configuration - part 2 (#957)
* Export general configuration - part 2

First part in #956 missed to export the elements of the struct ...

* Add comment for exported struct

* Add function for opening config files
2019-11-06 16:22:50 +01:00
Oliver NoconandGitHub a04489cd35 Add step spinnakerTriggerPipeline (#793) 2019-11-06 15:37:14 +01:00
Oliver NoconandGitHub bb230d3b9b Export general configuration (#956)
Allow for package external access to general configuration.

Use-case: re-use individual steps
2019-11-06 14:07:41 +01:00
Oliver NoconandGitHub d385015437 Merge branch 'master' into goGithub 2019-11-06 11:29:16 +01:00
Oliver NoconandGitHub de31cde9b8 Add PiperGoUtils for downloading piper binary (#928)
* Add PiperGoUtils for downloading piper binary

PiperGoUtils provide the link between a Jenkins library step and the library step execution running in a go binary.

It makes sure that an adequate binary is available.

* fix CodeClimate finding

* Remove Delimiter and add download resilience.
2019-11-06 11:28:10 +01:00