1
0
mirror of https://github.com/SAP/jenkins-library.git synced 2024-12-14 11:03:09 +02:00
Commit Graph

189 Commits

Author SHA1 Message Date
Daniel Kurzynski
fbf2b29a32
NpmExecuteScripts add script options (#1780) 2020-07-09 14:57:41 +02:00
Daniel Kurzynski
fb3c6b5978
Ignore wrong type of alias (#1778) 2020-07-09 14:13:06 +02:00
Kevin Stiehl
644f18b491
feat(vault): Added package to get secrets from vault (#1763)
* Added Vault package

* added support for logical path lookups instead of api paths

* added integration tests

* add integration tests and mock tests

* Replace mock with mockery generated one

	* update tests to use mockery
        * create mocks sub package
2020-07-08 08:20:15 +02:00
Oliver Nocon
49b7abb4ca
Fix stashing issue with binrary execution (#1749)
Co-authored-by: Daniel Kurzynski <daniel.kurzynski@sap.com>
2020-07-02 12:08:56 +02:00
Marcus Holl
0e19a3e71a
cf login additional opts (#1739)
provide facilities for handing over additional cf login and cf api parameters
2020-07-01 15:19:47 +02:00
Marcus Holl
580e6f8989
More acurate tests for cf login check (#1738)
- mock is used now for cf api commands
  - tests for cf api and cfk login are checking which
    commands are really executed
  - some minor simplifications wrt asserts
2020-07-01 14:19:21 +02:00
Marcus Holl
4f2ba73314
[refactoring] move the shell/command related interfaces into pkg/command (#1737)
* [refactoring] move the shell/command related interfaces into pkg/command

otherwise we are not able to use the corresponding mocks for the items contained in pkg since
these interfaces are not visible from the pkg folder

Co-authored-by: Daniel Kurzynski <daniel.kurzynski@sap.com>
2020-07-01 11:28:16 +02:00
Jordan Levin
34967c502c
Whitesource scan (MVP) (#1658)
* Whitesource MVP for Gradle, Golang, and NPM/Yarn

* Refactoring

* Refactor and cleanup, better error checking

* publish stepResults, use pkg/versioning, bubble up errors, add gomod versioning support

* Run gofmt and cleanup comments

* Resolve PR comments

* Update resources/metadata/whitesource.yaml

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Only determine project coordinates if they are missing

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>

* Gradle versioning artifact

* fix gradle artifact version regexp and refactor

* Fix token extraction from output buffer

* Fix some issues with pip and jsonfile versioning logic

* Remove useless spacing

* Remove unnecessary test file and fix naming style for JSONDescriptor

* Automatically download wss-unified-agent if file does not exist

* adds downloadVulnerabilityReport, checkSecurityViolations, minor refactoring

* adds config.ReportDirectoryName, improves readability

* Version-wide reporting for vulnerabilities and list of libraries.

* Refactor and improve build accuracy

* fix sed command

* Add includes file pattern config option

* Adds --exclude command line flag

* run go mod tidy and regenerate step framework

* Fix unit tests

* revert changes

* poll project status before downloading reports

* merge with master

* go mod tidy, go fmt, and fix whitesource unit test

* sync go.mod

* sync go.mod again

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-07-01 07:54:13 +02:00
Oliver Nocon
dc296b0727
Allow retrieving exit code from command execution (#1728)
* Allow retrieving exit code from command execution

This will be helpful to derive error categories in case
an executable provides context-specific error codes.

* make sure that we always have a non 0 exit code for errors
2020-06-30 12:57:27 +02:00
Oliver Nocon
ac5ae40ecb
kubernetesDeploy: categorize known errors & fix #1732 (#1727)
* kubernetesDeploy: categorize known errors

* hide Docker secret in log output
2020-06-30 10:48:30 +02:00
Marcus Holl
23fe4dcdcd
handle map[interface]interface{} in yaml utils substitute (#1725)
reality teaches us that we need to handle that type
2020-06-29 08:11:05 +02:00
Oliver Nocon
54850d2ca0
sonarExecuteScan: add error categorization (#1715) 2020-06-26 07:38:27 +02:00
Oliver Nocon
bb8cd30c4c
Extend resource references to cater for secrets (#1718)
* Add secret references to step metadata

* Update references

* update generated files
2020-06-25 16:33:38 +02:00
Marcus Holl
d1130a11ac
Support file modes in file system mock (#1716)
Support file mode in file system mock
2020-06-25 14:39:15 +02:00
Marcus Holl
d53ff9de0b
YamlUtil (prepares for cf deploy) (#1624)
Migrates basically `com/sap/piper/variablesubstitution/YamlUtils.groovy`.
2020-06-25 10:59:30 +02:00
Marcus Holl
295d526531
file system mock: Add capabilities for checks if a file has been written (#1706)
* Add capabilities for checks if a file has been written

With the current file system mock we cannot assert if
a file has been written. E.g. we cannot distiguish between
files added to the virtual file system before the test and files
explicitly written. In contrast to that we can check for deleted
files.

With the change here we get a func HasWritteFile(name).

[Q] Wouln't it be possible to check based on the file content
if a file has been written (the new file should have another
content as the file registered before).
[A] We should not assert some file content here since the
produced file content can be created by another "class" which
is unit tested somewhere else. With that approach we would test
the producer here again.

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-06-24 11:27:28 +02:00
Oliver Nocon
eafe383d54
Add error category parsing to cmd execution (#1703)
* Add error category parsing to cmd execution

It is now possible to define `ErrorCategoryMapping` as a `map[string][]string` on a `Command`.
The format contains the category as key which has a list of error patterns assigned.
Example:

```
cmd := Command{
  ErrorCategoryMapping: map[string][]string
    "build": {"build failed"},
    "compliance": {"vulnerabilities found", "outdated components found"},
    "test": {"some tests failed"},
  },
}
```

Setting this map triggers console log parsing when executing a command.
If a match is found the error category is stored and
it will automatically be added to the `errorDetails.json`.

* clean up go.mod

* fix test

* fix test

* Update DEVELOPMENT.md

* fix tests

* address long console content without line breaks

* scan condition update

* fix test

* add missing comment for exported function

* Update pkg/command/command.go

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-06-24 10:04:05 +02:00
Christopher Fenner
c42553593e
fix: remove unused variables from generated step coding (#1698)
* remove unused parameters from generated code

* update steps

* update steps

* correct golden files

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-06-23 18:05:21 +02:00
Oliver Nocon
d8444d51f2
Add possibility to add category to failures (#1692)
* Add possibility to add category to failures

It is now possible to set the error category within the flow.
When exiting the program the error category can be used.

There is  a convenience function available for exiting with a previously set category,
for example

```
log.SetErrorCategory(log.ErrorCompliance)
...
log.FatalError(err, "configuration error")
```

* extend test

* go mod tidy

* add missing comment

* update information about error categories

* Update DEVELOPMENT.md
2020-06-22 08:21:38 +02:00
Kevin Hudemann
ceb3dd0a04
Refactor pkg/npm and npmExecuteScripts (#1684)
This change refactors the npm pkg and npmExecuteScripts implementations
to be reusable for future steps, e.g., npmExecuteLint.

In addition, it fixes few small bugs related to unit test execution on
Windows and the fileUtils mocking implementation.

Co-authored-by: Daniel Kurzynski <daniel.kurzynski@sap.com>
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-06-18 17:30:17 +02:00
Florian Wilhelm
b5d4e7fd90
Avoid need of cd to install maven modules (#1677)
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-06-17 19:08:43 +02:00
Marcus Holl
2bd15543d0
Introduce interface for manifest (#1676)
We need the interface in order to inject mocks for tests.

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-06-17 15:19:40 +02:00
Marcus Holl
39d256d33c
Introduce manifest utils for cf (#1608)
Provide a manifest util for transforming legacy manifests for cf deployments.

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-06-16 17:50:26 +02:00
Marcus Holl
dcd9972516
No parallel test exec for file tests (#1675) 2020-06-16 14:50:11 +02:00
Daniel Kurzynski
0222bf83d1
Run npm scripts in virtual frame buffer and extend command.go to run executable asynchronously (#1669)
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
Co-authored-by: Florian Wilhelm <florian.wilhelm02@sap.com>
2020-06-16 11:42:51 +02:00
Florian Wilhelm
b0329cd2c0
Use absolute path for m2path in mta install (#1671) 2020-06-16 09:48:47 +02:00
Daniel Kurzynski
9a30758115
Consider stage config from custom defaults (#1670) 2020-06-16 09:06:37 +02:00
Oliver Nocon
6116fe5528
Decouple config reading and retrieval (#1668)
There are use cases where we need to run `GetStepConfig()` multiple times.
In such cases it is more efficient to load the files once and then resolve the
respective step configuration.
2020-06-15 14:17:59 +02:00
Florian Wilhelm
0fcf1654fe
Install maven artifacts after mta build (#1641)
Extend mta build step to install maven artefacts after build to allow re-using them in later stages (additional unit or integration tests which might not be running as part of the "build" life-cycle).
2020-06-15 12:46:54 +02:00
Stephan Aßmus
f855658e06
Enhance piperutils.Files and mock.FilesMock (#1664)
* Flesh out piperutils.Files and mock.FilesMock functionality
* Avoid a lot of code-duplication via embedding
2020-06-15 09:47:33 +02:00
Oliver Nocon
50b2a2d6fc
Update deprecation message (#1656) 2020-06-12 15:19:13 +02:00
Stephan Aßmus
d558db9106
Refactor maven.Evaluate() to take options (#1659)
Co-authored-by: Florian Wilhelm <florian.wilhelm02@sap.com>
2020-06-11 14:02:54 +02:00
Stephan Aßmus
c693c4c7dc
Increase default transport timeout to 3 mins (#1648) 2020-06-10 11:14:55 +02:00
Stephan Aßmus
f90a4f9eae
Provide an ExecRunner implementation for running commands in docker (#1606)
* ExecRunner implementation for executing commands within docker
* Add whole-file example as documentation
2020-06-02 14:24:06 +02:00
Stephan Aßmus
a24a7aad23
Fortify: Using mvn to auto-resolve classpath needs additional params (#1607)
* also reduce code duplication in token fetching
* concatenate classpaths from multi-maven projects

Co-authored-by: Daniel Kurzynski <daniel.kurzynski@sap.com>
2020-05-29 15:42:35 +02:00
Maximilian Lenkeit
10e5bf98f8
artifactPrepareVersion: remember original artifact version (#1600)
* Update resources.go

* expose originalArtifactVersion parameter

* re-generate steps

* adjust golden files for generator changes
2020-05-28 10:05:22 +02:00
Stephan Aßmus
e6f5544601
Implement YAMLfile.GetCooridnates() (#1604) 2020-05-27 17:20:34 +02:00
Florian Wilhelm
6cbc407af1
Use short warn log level (#1599)
Co-authored-by: Daniel Kurzynski <daniel.kurzynski@sap.com>
2020-05-27 14:06:09 +02:00
Marcus Holl
d1b717d7a4
suppress error in log message on case there is no error available (#1594) 2020-05-27 09:31:41 +02:00
Oliver Nocon
243d196283
Add package for whitesource (#1577) 2020-05-26 17:30:45 +02:00
Marcus Holl
2879d98502
Include error in log message (#1535)
* Include error in the log.

In case the default text formatter is used, the error
will be contained in the log, too.

* Avoid stupid nil token printed in error message
2020-05-26 11:06:52 +02:00
Sven Merk
af2a01c064
Fortify implementation in golang (#1428) 2020-05-25 19:48:59 +02:00
Oliver Nocon
7e2ab744a5
mtaBuild - change default for docker image (#1574)
* mtaBuild - change default for docker image

* fix resolution of container default from step definition

* Properly fix default for container conditions
2020-05-22 09:48:11 +02:00
Florian Wilhelm
bd1e0a1071
Fix npm registry config (#1564) 2020-05-20 13:41:23 +02:00
Oliver Nocon
a0ded365c5
Fix container condition handling (#1562)
* Add additional tests
* Update handling of containers
* Properly consider existing config at root level
* Fix parameter filtering
* Fix tests for adapted filter handling
2020-05-20 10:50:35 +02:00
Oliver Nocon
7f7d0bce89
Piper steps: Issue SUCCESS message (#1478)
* Piper steps: Issue SUCCESS message

Log `SUCCESS` message in all steps by default if no error occured

* Update additional generated steps

* Update generated step
2020-05-18 21:52:54 +02:00
Stephan Aßmus
5ab4dedcb2
Optionally ignore custom defaults (#1538)
* Add parameter "--ignoreCustomDefaults"
* Pass to piper customDefaults from config also via --defaultConfig
... and add "--ignoreCustomDefaults".
* Log output when ignoring customDefaults

Co-authored-by: Daniel Kurzynski <daniel.kurzynski@sap.com>
2020-05-14 10:50:58 +02:00
dominiklendle
ac732b3065
Add abapEnvironmentRunATCCheck step (#1454)
* Minor changes

* Changed groovy file

* Changed generated file

* Changed yaml with container config

* Changed groovy config

* minor changes

* minor changes

* Changed yaml with aliases

* minor changes

* minor changes

* minor changes

* minor changes

* minor changes

* minor changes

* minor changes

* minor changes

* minor changes

* minor changes

* Changed yaml aliases

* Adapted naming conventions

* Removed error code at the end

* Adapted configuration

* Minor changes

* Minor changes

* Minor changes

* Removed spaces

* Removed docker-related config from groovy file

* Minor changes

* Minor changes

* Removed container config

* Corrected testing function name

* Deleted unnecessary parts

* Changed service deletion message

* Changed service deletion message

* Logging out before throwing error service deletion step fails

* Minor changes

* Minor changes

* Minor changes

* Delete .DS_Store

* Delete .DS_Store

* Delete .DS_Store

* Delete .DS_Store

* Minor changes

* Minor changes

* Minor changes

* Added newline at end of file

* Added newline at end of file

* Changes for Pull request optimization

* added documentaion

* Adapted documentation

* Adapted documentation

* Adapted documentation

* Adapted documentation

* Adapted documentation

* Added CFDeleteServiceKeys

* Added ServiceKey deletion tests

* added cfServiceKeys flag explanation to documentation

* removed trailing spaces from documentation

* resolving conflicts

* Changed deletion message an variable naming

* Changed tests

* Changed tests

* Changed tests

* Changed tests

* Changed CloudFoundryDeleteServiceOptions to options

* Changed CloudFoundryDeleteServiceOptions to options

* Minor changes

* Minor changes

* Changed variable naming

* Changed error handling

* Changed error handling and logging

* Changed documentation

* Simplified code

* Fixed CodeClimate issues

* Changed from returning err to nil where no errur returned needed

* Add cloudFoundryCreateServiceKey Go Step

* Changed Groovy File

* Changed aliases

* Removed unneccessary parts

* Minor changes

* Minor changes

* Adapted documentation

* Adapted tests

* Adapted Groovy File

* Changed Groovy file

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Removed Groovy Tests for cfCreateServiceKey

* Minor changes

* Added ATC Check YAML

* Added ATC Check generated files

* Added test class

* Added abapEnvironmentRunATCCheck

* Minor changes

* Minor changes

* Changed groovy

* Minor changes

* Changed groovy

* Changed groovy

* Minor changes

* Adapted Groovy imports

* Adapted Groovy imports

* Adapted Groovy imports

* Adapted Groovy

* Getting ATC results

* Changed error message

* changed groovy

* removed trailing spaces

* Added login check

* Minor changes

* Added step to whitelistScriptReference

* Added ATC error message handling

* Added groovy file

* Added step to groovy tests

* corrected metadata file

* Debugging

* Debugging

* Added yaml config parameter for ATC run

* Adapted file location of ATC run config to jenkins specific location

* Implementing universal pipeline logic for finding yaml config regardless of pipeline

* Changed error handling for reading config yaml file

* Changed atcrunconfig alias

* minor changes

* Minor changes

* Minor changes

* Changed back to dynamic file reading

* Minor changes

* filepath changes

* Removing CF Login

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Minor changes

* Removed whitespaces

* Added CF functions unit tests

* Added invalid parameter handling

* Removed package and SC flag

* Minor changes

* Changed tests

* Changed tests

* Changed tests

* Minor changes

* Changed tests

* removed unnecessary logout

* Added documentation

* Changed docu

* Changed docu

* Changed docu

* Changed docu

* Changed docu

* Changed docu

* Changed docu

* Changed docu

* Changed docu

* Changed docu

* Removed trailing spaces

* Added newline at end of file

* code climate fixes

* code climate fixes

* code climate fixes

* Minor changes

* Minor changes

* Minor changes

* Changed tests

* Test changes

* Splitted Cloud Foundry functions into two classes

* Removed two steps from whtielistScriptReference

* removed atcrunConfig alias

* issue fixes

* Changed docu

* Changed docu

* Changed docu

* Removed trailing spaced from docu

* Changed docu

* Go generator run

* Issue fixes

* Remove unnecessary imports

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Update whitelistScript

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Adding piperutils for writing xml file

* Persisting ATC Results with piperutils

* Set failonMissingReports to true

* Refactoring for CodeClimate

* Changed result file name

* Changed credentials aliases

* changing secret name

* Removing trailing spaces

* Added secret name and alias to docu

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-05-13 14:51:48 +02:00
lndrschlz
4105f81f71
removed error level from sentry hook (#1528)
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-05-12 22:11:52 +02:00
Stephan Aßmus
80d53dae79
Reading defaults in go: Make sure all io.ReadClosers are closed (#1529) 2020-05-12 15:46:47 +02:00