1
0
mirror of https://github.com/SAP/jenkins-library.git synced 2024-12-14 11:03:09 +02:00
Commit Graph

213 Commits

Author SHA1 Message Date
Daniel Mieg
bfa601cd47
Improve testability of abap steps (#1840)
* Test

* Test

* Test abapEnvironmentPullGitRepo step

* Move mock functions

* Add package for mock

* Move mock
2020-07-31 14:43:23 +02:00
Marcus Holl
4b5fa80980
CfAPIOpts: remove leftovers (#1871)
we don't have any cf api calls anymore in the context of cf login.
2020-07-31 10:14:02 +02:00
Daniel Kurzynski
8ee0d358b9
Support maven params in detect scan (#1855)
Co-authored-by: Florian Wilhelm <florian.wilhelm02@sap.com>
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-07-30 10:35:46 +02:00
Oliver Nocon
d691750931
Fix reading of hook configuration (#1854)
* Update piper.go

* Update piper_test.go

* Update config.go

* Update sentryHook.go

* Update piper.go

* go fmt

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-07-28 17:19:33 +02:00
Daniel Kurzynski
bcab73d332
Support m2path for mtaBuild (#1850)
Co-authored-by: Florian Wilhelm <florian.wilhelm02@sap.com>
2020-07-28 17:06:17 +02:00
MuellerHenrik
e3f914e09d
Add abap utils pkg - ABAP Environment Steps (#1757)
* Add abaputils pkg and go files

* Add ReadServiceKeyAbapEnvironment function

* Fixes

* Add structs for SC, Pull and Branch

* Minor Improvements

* Adapt unit tests to new abaputils pkg

* Fixes

* Add adapted tests

* Fixes

* Fix cloudfoundry test

* Add check for host prefix (HTTPS)

* Fix tests + cleanup

* Fixes

* Fixes

* Fix

* Add mock for abaputils pkg unit tests

* Adapt abaputils comments

* Add unit test for missing params case

* Fix for missing mapping of CfSpace

* Fix host schema

* Remove LogoutOption param of unit tests and steps

* Fix unit test

* Fix unit test CF ReadServiceKey

Co-authored-by: Daniel Mieg <56156797+DanielMieg@users.noreply.github.com>
2020-07-23 10:26:50 +02:00
Kevin Stiehl
116a91ce28
Fetch config parameters from vault (#1782)
* added secret fetching from vault

* added rootPath and vault namespaces

* write tests for vault config loading
	* fix smaller bugs

* rename rootPath to basePath

* fix integration test

* make basePath optional

* write more tests

* remove unnecessary code

* make basePath configurable

* remove mixin for vault

* rewrite tests
* fix tests for searching over multiple paths

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-07-22 11:15:48 +02:00
Oliver Nocon
e09b05b22e
DocGenerator: fix links to secrets (#1825)
* DocGenerator: fix links to secrets
* Update docuHelper.go
2020-07-22 10:33:36 +02:00
Stephan Aßmus
61fed83475
Improve self-consistency of file system mock (#1815) 2020-07-20 14:23:33 +02:00
Oliver Nocon
220748d150
Step generation: allow usage of go generate (#1793)
* Step generation: allow usage of go generate

* Update DEVELOPMENT.md

* move generate command to main.go (#1809)

* Update main.go

* Update DEVELOPMENT.md

* Update piper.go

* align path usage

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-07-17 08:06:11 +02:00
Kevin Hudemann
58e7e4be44
Add buildDescriptorExcludeList parameter to npmExecuteScripts step (#1800)
This change adds a buildDescriptorExcludeList parameter to
npmExecuteScripts, to enable the exclusion of certain directories when
executing npm scripts. Previously, npmExecuteScripts could only execute
scripts in all packages.

Now it is possible to provide paths or patterns as elements of the
buildDescriptorExcludeList to exclude packages when executing npm scripts.
2020-07-16 17:16:55 +02:00
Marcus Holl
8af0540de2
Introduce chmod in file utils (#1808) 2020-07-16 14:25:01 +02:00
Oliver Nocon
c622b3dfac
Documentation: fix parameter links & some spelling (#1807)
* Documentation: fix parameter links & some spelling
* Fix new line character
* update checkboxes
2020-07-16 11:34:02 +02:00
Daniel Mieg
90571754bf
Add test cases for stepmeta.go (#1801)
* Add tests

* Remove obsolete comment
2020-07-16 11:09:49 +02:00
Oliver Nocon
43f51ba90a
Revamp documentation generation for golang steps (#1781)
* Revamp documentation generation for golang steps

* Add and update tests

* Add tests, add step outputs

* Add tests, add step outputs

* Update rendering

* fix merge conflict

* update generated files

* Update pkg/config/stepmeta.go

* Update pkg/config/stepmeta.go

* Update pkg/generator/helper/docuHelper.go

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Include PR feedback

* update tests

* Fix Jenkins dependecy indication

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-07-16 09:10:15 +02:00
Daniel Kurzynski
61a669778a
Apply alias to secrets (#1802) 2020-07-15 21:05:22 +02:00
Daniel Mieg
5dbb7be6b6
Fix commonPipelineEnvironment issue (#1798)
* Fix write to disk

* Change condition

* Test log output

* Fix

* Remove test log
2020-07-15 15:45:36 +02:00
Marcus Holl
37ccebeb77
No cf api for checking login state (#1785)
* Dont work upon a global command.Command instance inside cloudfoundry package

o Up to now we work on a private and shared instance of command.Command inside
  the cloudfounrdy package. We need to be able either configure this instance
  (environment variables) according to the use case. One option is to hand over
  an already configured instance which is used elsewhere. This is what we do with
  this commit.

o With this commit we remove the instance which is shared within the cloudfounrdy
  package and to provide an instance with a receiver which gets handed over to the
  functions. Hence we are thread save: parallel invoctation of e.g. Login will not
  affect each other.
2020-07-15 15:12:28 +02:00
Marcus Holl
49d4f6269a
Introduce stat command into Files utils and correspoding mock (#1721)
Introduce stat command into Files utils and correspoding mock
2020-07-15 10:31:36 +02:00
Daniel Mieg
7f69f4eb16
Support lists in commonPipelineEnvironment (#1797)
* Enable write Lists

* Enable read

* Add tests

* Fix unit test

* CodeClimate Refactor

* Add comment
2020-07-15 10:09:42 +02:00
Marcus Holl
38c5604798
file mock: Dont store content of written or deleted file (#1717)
Don't store the content of the written files with the written files
we can lookup that in the files map
2020-07-14 15:37:08 +02:00
Chris Bo
9558b47268
Introducing new step 'gctsExecuteABAPUnitTests' (#1532)
* add new step 'gctsExecuteABAPUnitTests'

* added new step entries

* small wording improvement

* regenerated

* fixed unit tests

* adjusted prerequisites

* removed telemetry

* remove command

* moved functions to http package

* newly generated

* remove commented code

* use credentials ID in config.yaml

* add supported protocols to docu

Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-07-14 10:58:57 +02:00
Marcus Holl
bda9a9ffba
Dont work upon a global command.Command instance inside cloudfoundry package (#1771)
o Up to now we work on a private and shared instance of command.Command inside
  the cloudfounrdy package. We need to be able either configure this instance
  (environment variables) according to the use case. One option is to hand over
  an already configured instance which is used elsewhere. This is what we do with
  this commit.

o With this commit we remove the instance which is shared within the cloudfounrdy
  package and to provide an instance with a receiver which gets handed over to the
  functions. Hence we are thread save: parallel invoctation of e.g. Login will not
  affect each other.
2020-07-10 16:31:17 +02:00
lndrschlz
db13b2f9ce
moved container context method to config package and expose for reuse (#1770) 2020-07-10 10:32:26 +02:00
Daniel Kurzynski
fbf2b29a32
NpmExecuteScripts add script options (#1780) 2020-07-09 14:57:41 +02:00
Daniel Kurzynski
fb3c6b5978
Ignore wrong type of alias (#1778) 2020-07-09 14:13:06 +02:00
Kevin Stiehl
644f18b491
feat(vault): Added package to get secrets from vault (#1763)
* Added Vault package

* added support for logical path lookups instead of api paths

* added integration tests

* add integration tests and mock tests

* Replace mock with mockery generated one

	* update tests to use mockery
        * create mocks sub package
2020-07-08 08:20:15 +02:00
Oliver Nocon
49b7abb4ca
Fix stashing issue with binrary execution (#1749)
Co-authored-by: Daniel Kurzynski <daniel.kurzynski@sap.com>
2020-07-02 12:08:56 +02:00
Marcus Holl
0e19a3e71a
cf login additional opts (#1739)
provide facilities for handing over additional cf login and cf api parameters
2020-07-01 15:19:47 +02:00
Marcus Holl
580e6f8989
More acurate tests for cf login check (#1738)
- mock is used now for cf api commands
  - tests for cf api and cfk login are checking which
    commands are really executed
  - some minor simplifications wrt asserts
2020-07-01 14:19:21 +02:00
Marcus Holl
4f2ba73314
[refactoring] move the shell/command related interfaces into pkg/command (#1737)
* [refactoring] move the shell/command related interfaces into pkg/command

otherwise we are not able to use the corresponding mocks for the items contained in pkg since
these interfaces are not visible from the pkg folder

Co-authored-by: Daniel Kurzynski <daniel.kurzynski@sap.com>
2020-07-01 11:28:16 +02:00
Jordan Levin
34967c502c
Whitesource scan (MVP) (#1658)
* Whitesource MVP for Gradle, Golang, and NPM/Yarn

* Refactoring

* Refactor and cleanup, better error checking

* publish stepResults, use pkg/versioning, bubble up errors, add gomod versioning support

* Run gofmt and cleanup comments

* Resolve PR comments

* Update resources/metadata/whitesource.yaml

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Only determine project coordinates if they are missing

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>

* Gradle versioning artifact

* fix gradle artifact version regexp and refactor

* Fix token extraction from output buffer

* Fix some issues with pip and jsonfile versioning logic

* Remove useless spacing

* Remove unnecessary test file and fix naming style for JSONDescriptor

* Automatically download wss-unified-agent if file does not exist

* adds downloadVulnerabilityReport, checkSecurityViolations, minor refactoring

* adds config.ReportDirectoryName, improves readability

* Version-wide reporting for vulnerabilities and list of libraries.

* Refactor and improve build accuracy

* fix sed command

* Add includes file pattern config option

* Adds --exclude command line flag

* run go mod tidy and regenerate step framework

* Fix unit tests

* revert changes

* poll project status before downloading reports

* merge with master

* go mod tidy, go fmt, and fix whitesource unit test

* sync go.mod

* sync go.mod again

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-07-01 07:54:13 +02:00
Oliver Nocon
dc296b0727
Allow retrieving exit code from command execution (#1728)
* Allow retrieving exit code from command execution

This will be helpful to derive error categories in case
an executable provides context-specific error codes.

* make sure that we always have a non 0 exit code for errors
2020-06-30 12:57:27 +02:00
Oliver Nocon
ac5ae40ecb
kubernetesDeploy: categorize known errors & fix #1732 (#1727)
* kubernetesDeploy: categorize known errors

* hide Docker secret in log output
2020-06-30 10:48:30 +02:00
Marcus Holl
23fe4dcdcd
handle map[interface]interface{} in yaml utils substitute (#1725)
reality teaches us that we need to handle that type
2020-06-29 08:11:05 +02:00
Oliver Nocon
54850d2ca0
sonarExecuteScan: add error categorization (#1715) 2020-06-26 07:38:27 +02:00
Oliver Nocon
bb8cd30c4c
Extend resource references to cater for secrets (#1718)
* Add secret references to step metadata

* Update references

* update generated files
2020-06-25 16:33:38 +02:00
Marcus Holl
d1130a11ac
Support file modes in file system mock (#1716)
Support file mode in file system mock
2020-06-25 14:39:15 +02:00
Marcus Holl
d53ff9de0b
YamlUtil (prepares for cf deploy) (#1624)
Migrates basically `com/sap/piper/variablesubstitution/YamlUtils.groovy`.
2020-06-25 10:59:30 +02:00
Marcus Holl
295d526531
file system mock: Add capabilities for checks if a file has been written (#1706)
* Add capabilities for checks if a file has been written

With the current file system mock we cannot assert if
a file has been written. E.g. we cannot distiguish between
files added to the virtual file system before the test and files
explicitly written. In contrast to that we can check for deleted
files.

With the change here we get a func HasWritteFile(name).

[Q] Wouln't it be possible to check based on the file content
if a file has been written (the new file should have another
content as the file registered before).
[A] We should not assert some file content here since the
produced file content can be created by another "class" which
is unit tested somewhere else. With that approach we would test
the producer here again.

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-06-24 11:27:28 +02:00
Oliver Nocon
eafe383d54
Add error category parsing to cmd execution (#1703)
* Add error category parsing to cmd execution

It is now possible to define `ErrorCategoryMapping` as a `map[string][]string` on a `Command`.
The format contains the category as key which has a list of error patterns assigned.
Example:

```
cmd := Command{
  ErrorCategoryMapping: map[string][]string
    "build": {"build failed"},
    "compliance": {"vulnerabilities found", "outdated components found"},
    "test": {"some tests failed"},
  },
}
```

Setting this map triggers console log parsing when executing a command.
If a match is found the error category is stored and
it will automatically be added to the `errorDetails.json`.

* clean up go.mod

* fix test

* fix test

* Update DEVELOPMENT.md

* fix tests

* address long console content without line breaks

* scan condition update

* fix test

* add missing comment for exported function

* Update pkg/command/command.go

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-06-24 10:04:05 +02:00
Christopher Fenner
c42553593e
fix: remove unused variables from generated step coding (#1698)
* remove unused parameters from generated code

* update steps

* update steps

* correct golden files

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-06-23 18:05:21 +02:00
Oliver Nocon
d8444d51f2
Add possibility to add category to failures (#1692)
* Add possibility to add category to failures

It is now possible to set the error category within the flow.
When exiting the program the error category can be used.

There is  a convenience function available for exiting with a previously set category,
for example

```
log.SetErrorCategory(log.ErrorCompliance)
...
log.FatalError(err, "configuration error")
```

* extend test

* go mod tidy

* add missing comment

* update information about error categories

* Update DEVELOPMENT.md
2020-06-22 08:21:38 +02:00
Kevin Hudemann
ceb3dd0a04
Refactor pkg/npm and npmExecuteScripts (#1684)
This change refactors the npm pkg and npmExecuteScripts implementations
to be reusable for future steps, e.g., npmExecuteLint.

In addition, it fixes few small bugs related to unit test execution on
Windows and the fileUtils mocking implementation.

Co-authored-by: Daniel Kurzynski <daniel.kurzynski@sap.com>
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-06-18 17:30:17 +02:00
Florian Wilhelm
b5d4e7fd90
Avoid need of cd to install maven modules (#1677)
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-06-17 19:08:43 +02:00
Marcus Holl
2bd15543d0
Introduce interface for manifest (#1676)
We need the interface in order to inject mocks for tests.

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-06-17 15:19:40 +02:00
Marcus Holl
39d256d33c
Introduce manifest utils for cf (#1608)
Provide a manifest util for transforming legacy manifests for cf deployments.

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-06-16 17:50:26 +02:00
Marcus Holl
dcd9972516
No parallel test exec for file tests (#1675) 2020-06-16 14:50:11 +02:00
Daniel Kurzynski
0222bf83d1
Run npm scripts in virtual frame buffer and extend command.go to run executable asynchronously (#1669)
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
Co-authored-by: Florian Wilhelm <florian.wilhelm02@sap.com>
2020-06-16 11:42:51 +02:00
Florian Wilhelm
b0329cd2c0
Use absolute path for m2path in mta install (#1671) 2020-06-16 09:48:47 +02:00