1
0
mirror of https://github.com/SAP/jenkins-library.git synced 2024-12-14 11:03:09 +02:00
Commit Graph

2817 Commits

Author SHA1 Message Date
Jordan Levin
34967c502c
Whitesource scan (MVP) (#1658)
* Whitesource MVP for Gradle, Golang, and NPM/Yarn

* Refactoring

* Refactor and cleanup, better error checking

* publish stepResults, use pkg/versioning, bubble up errors, add gomod versioning support

* Run gofmt and cleanup comments

* Resolve PR comments

* Update resources/metadata/whitesource.yaml

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

* Only determine project coordinates if they are missing

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>

* Gradle versioning artifact

* fix gradle artifact version regexp and refactor

* Fix token extraction from output buffer

* Fix some issues with pip and jsonfile versioning logic

* Remove useless spacing

* Remove unnecessary test file and fix naming style for JSONDescriptor

* Automatically download wss-unified-agent if file does not exist

* adds downloadVulnerabilityReport, checkSecurityViolations, minor refactoring

* adds config.ReportDirectoryName, improves readability

* Version-wide reporting for vulnerabilities and list of libraries.

* Refactor and improve build accuracy

* fix sed command

* Add includes file pattern config option

* Adds --exclude command line flag

* run go mod tidy and regenerate step framework

* Fix unit tests

* revert changes

* poll project status before downloading reports

* merge with master

* go mod tidy, go fmt, and fix whitesource unit test

* sync go.mod

* sync go.mod again

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-07-01 07:54:13 +02:00
MuellerHenrik
de3afe9476
Change parameters/alias for CfServiceKeyName (#1726)
* Fix alias cfServiceKeyName

* Adapt documentation

* Fix alias use in go files

Co-authored-by: Daniel Mieg <56156797+DanielMieg@users.noreply.github.com>
2020-06-30 15:46:07 +02:00
Oliver Nocon
dc296b0727
Allow retrieving exit code from command execution (#1728)
* Allow retrieving exit code from command execution

This will be helpful to derive error categories in case
an executable provides context-specific error codes.

* make sure that we always have a non 0 exit code for errors
2020-06-30 12:57:27 +02:00
Srinikitha Kondreddy
1387f15ed6
Upgrade cloud mbt tool docker image version (#1735) 2020-06-30 12:23:17 +02:00
Oliver Nocon
ac5ae40ecb
kubernetesDeploy: categorize known errors & fix #1732 (#1727)
* kubernetesDeploy: categorize known errors

* hide Docker secret in log output
2020-06-30 10:48:30 +02:00
Marcus Holl
23fe4dcdcd
handle map[interface]interface{} in yaml utils substitute (#1725)
reality teaches us that we need to handle that type
2020-06-29 08:11:05 +02:00
Alejandra Ferreiro Vidal
775f6d0b2e
add mtaExtensionDescriptor to cf target (#1720) 2020-06-26 15:04:35 +02:00
Oliver Nocon
54850d2ca0
sonarExecuteScan: add error categorization (#1715) 2020-06-26 07:38:27 +02:00
Oliver Nocon
bb8cd30c4c
Extend resource references to cater for secrets (#1718)
* Add secret references to step metadata

* Update references

* update generated files
2020-06-25 16:33:38 +02:00
Marcus Holl
d1130a11ac
Support file modes in file system mock (#1716)
Support file mode in file system mock
2020-06-25 14:39:15 +02:00
andre2007
f0babcade1
feat(sonar): upgrade sonar-scanner-cli to 4.4.0.2170 (#1710)
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-06-25 13:53:22 +02:00
Thorsten Duda
30302ff47a
mbt default version update (#1704)
With devxci/mbtci:1.0.14.1 docker image, UI5 projects
generated from Business Application Studio can be build
out of the box.
2020-06-25 12:16:02 +02:00
Marcus Holl
d53ff9de0b
YamlUtil (prepares for cf deploy) (#1624)
Migrates basically `com/sap/piper/variablesubstitution/YamlUtils.groovy`.
2020-06-25 10:59:30 +02:00
Kevin Hudemann
a4fc7bf4af
Fix error Handling in DefaultValueCache (#1719)
This change fixes the error handling in addDefaultsFromFiles(), i.e.,
using the correct path when calling the readFile step to return the
content of the unsuccessfully parsed yaml file.

Co-authored-by: Daniel Kurzynski <daniel.kurzynski@sap.com>
2020-06-25 07:58:29 +02:00
Daniel Mieg
f38a50739c
Fix extensions for Post stage of Abap Pipeline (#1714) 2020-06-24 17:44:15 +02:00
Stephan Aßmus
dc99676c8d
Init container map via YAML resource (#1695) 2020-06-24 17:04:58 +02:00
Marcus Holl
295d526531
file system mock: Add capabilities for checks if a file has been written (#1706)
* Add capabilities for checks if a file has been written

With the current file system mock we cannot assert if
a file has been written. E.g. we cannot distiguish between
files added to the virtual file system before the test and files
explicitly written. In contrast to that we can check for deleted
files.

With the change here we get a func HasWritteFile(name).

[Q] Wouln't it be possible to check based on the file content
if a file has been written (the new file should have another
content as the file registered before).
[A] We should not assert some file content here since the
produced file content can be created by another "class" which
is unit tested somewhere else. With that approach we would test
the producer here again.

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-06-24 11:27:28 +02:00
Alejandra Ferreiro Vidal
3d9606022f
add parallel execution to multicloudDeploy (#670)
* add parallelExecution to multicloudDeploy

adds parallel execution to the step multicloudDeploy

* add log

* fix warning

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
2020-06-24 10:56:36 +02:00
Oliver Nocon
eafe383d54
Add error category parsing to cmd execution (#1703)
* Add error category parsing to cmd execution

It is now possible to define `ErrorCategoryMapping` as a `map[string][]string` on a `Command`.
The format contains the category as key which has a list of error patterns assigned.
Example:

```
cmd := Command{
  ErrorCategoryMapping: map[string][]string
    "build": {"build failed"},
    "compliance": {"vulnerabilities found", "outdated components found"},
    "test": {"some tests failed"},
  },
}
```

Setting this map triggers console log parsing when executing a command.
If a match is found the error category is stored and
it will automatically be added to the `errorDetails.json`.

* clean up go.mod

* fix test

* fix test

* Update DEVELOPMENT.md

* fix tests

* address long console content without line breaks

* scan condition update

* fix test

* add missing comment for exported function

* Update pkg/command/command.go

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-06-24 10:04:05 +02:00
Christopher Fenner
1d1bf68d96
fix(security): remove duplicate stage wrapper (#1707)
* remove duplicate stage wrapper

fixes #1705

* Update piperPipelineStageSecurity.groovy

* fix trailing spaces
2020-06-24 08:21:41 +02:00
Christopher Fenner
c42553593e
fix: remove unused variables from generated step coding (#1698)
* remove unused parameters from generated code

* update steps

* update steps

* correct golden files

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-06-23 18:05:21 +02:00
Daniel Mieg
f348c53989
Add ABAP Environment Pipeline (#1555) 2020-06-23 15:56:56 +02:00
Christopher Fenner
1c1c141f3b
fix(influx): pass Sonar token to influx (#1699) 2020-06-23 14:16:44 +02:00
Daniel Kurzynski
654dea4b3e
Load global extensions in setupCommonPipelineEnvironment (#1688)
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
Co-authored-by: Florian Wilhelm <florian.wilhelm02@sap.com>
2020-06-22 17:52:11 +02:00
lydiazcheng
59f3cddd43
TMS: extend upload mta ext descriptor to node (#1635)
* add APIs for getNodes, getExtDescriptor, putExtDescriptor and uploadExtDescriptor
* add new params and extend the logic to enable upload or update extension descriptor to node
* extends unit tests


Co-authored-by: Zihe Cheng <zihe.cheng@sap.com>
Co-authored-by: Marcus Holl <marcus.holl@sap.com>
2020-06-22 11:34:10 +02:00
Florian Wilhelm
60eefabb20
Install dev deps in mta build (#1685)
Ensure npm dev dependencies are available after mtaBuild as they are required by certain tests.
Co-authored-by: Kevin Hudemann <kevin.hudemann@sap.com>
Co-authored-by: Daniel Kurzynski <daniel.kurzynski@sap.com>
2020-06-22 10:39:52 +02:00
Kevin Hudemann
60fa1d5bbf
Add Java-/Typescript linting capabilities (#1636)
This change adds linting support for Java-/TypeScript projects based on the checkDefaultLint and checkUserLint steps from Cloud SDK Pipeline.

Following options are supported:

1. Define a linting script named ci-lint in the package.json file(s) of the project
2. Provide a configuration for ESLint as part of the project
3. If none of the above: ESLint is executed with a general purpose configuration.

Co-authored-by: Daniel Kurzynski <daniel.kurzynski@sap.com>
2020-06-22 10:12:28 +02:00
Daniel Kurzynski
7c0c5242e3
Revert go mod change (#1696) 2020-06-22 10:04:25 +02:00
Oliver Nocon
d8444d51f2
Add possibility to add category to failures (#1692)
* Add possibility to add category to failures

It is now possible to set the error category within the flow.
When exiting the program the error category can be used.

There is  a convenience function available for exiting with a previously set category,
for example

```
log.SetErrorCategory(log.ErrorCompliance)
...
log.FatalError(err, "configuration error")
```

* extend test

* go mod tidy

* add missing comment

* update information about error categories

* Update DEVELOPMENT.md
2020-06-22 08:21:38 +02:00
Daniel Kurzynski
8f9ed0766c
Let json patcher fail if file not found (#1694)
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-06-19 20:13:20 +02:00
Daniel Kurzynski
0d1b94adb3
Add sysadmin capabilities to npmExecuteScripts (#1691) 2020-06-19 16:39:07 +02:00
Daniel Kurzynski
7c441f552f
Make version command a high level command like get-config without resolving the configuration (#1689) 2020-06-19 14:39:19 +02:00
Oliver Nocon
d602c6e2b9
extend security stage (#1687)
* extend security stage

Add new steps
* checkmarxExecuteScan
* fortifyExecuteScan

closes #1650

* update tests
2020-06-19 12:35:02 +02:00
Christopher Fenner
c5299f7a6c
add go report badge (#1682) 2020-06-18 19:52:52 +02:00
Kevin Hudemann
ceb3dd0a04
Refactor pkg/npm and npmExecuteScripts (#1684)
This change refactors the npm pkg and npmExecuteScripts implementations
to be reusable for future steps, e.g., npmExecuteLint.

In addition, it fixes few small bugs related to unit test execution on
Windows and the fileUtils mocking implementation.

Co-authored-by: Daniel Kurzynski <daniel.kurzynski@sap.com>
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-06-18 17:30:17 +02:00
Daniel Kurzynski
be01dd3869
Add schema patch step in go (#1683)
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-06-18 14:50:46 +02:00
Daniel Kurzynski
6dfd83d246
Add ability to load custom defaults from secured resource (#1680)
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-06-18 12:39:55 +02:00
Roland Stengel
f7015b562d
remove library statement from piperPipeline (#1595)
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-06-18 08:24:47 +02:00
Chris Bo
e9ad8c5e9c
Introducing new step 'gctsCloneRepository' (#1468)
* added new step gctsCloneRepository

* enhanced docu

* adjusted test case name

* enhanced docu

* refined error messages

* regenerated & remove unused Groovy imports

* small change

* remove execRunner

* Update documentation/docs/steps/gctsCloneRepository.md

* regenerated

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-06-18 07:45:22 +02:00
Florian Wilhelm
b5d4e7fd90
Avoid need of cd to install maven modules (#1677)
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-06-17 19:08:43 +02:00
Daniel Kurzynski
498e527c6c
Add npmExecuteScripts to integration stage (#1674)
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
2020-06-17 18:17:05 +02:00
Daniel Kurzynski
743fca43e1
Remove projectVersion from fortify documentation (#1628)
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-06-17 17:47:59 +02:00
Marcus Holl
2bd15543d0
Introduce interface for manifest (#1676)
We need the interface in order to inject mocks for tests.

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-06-17 15:19:40 +02:00
Marcus Holl
39d256d33c
Introduce manifest utils for cf (#1608)
Provide a manifest util for transforming legacy manifests for cf deployments.

Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
2020-06-16 17:50:26 +02:00
Oliver Nocon
8e74952166
Switch Detect scan to golang implementation (#1673)
* Switch Detect scan to golang implementation

* update workingDir

* use universal container as default

* Switch to official curl image

* fix code climate issue

* switch default image

* update docker options

* update CommonStepsTest

* remove invalid groovy test

* clean up defaults

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
2020-06-16 17:21:59 +02:00
Marcus Holl
dcd9972516
No parallel test exec for file tests (#1675) 2020-06-16 14:50:11 +02:00
Florian Wilhelm
1d3bd11621
Adapt mavenExecute example call to new api (#1672)
The old one won't work
2020-06-16 12:43:40 +02:00
Daniel Kurzynski
0222bf83d1
Run npm scripts in virtual frame buffer and extend command.go to run executable asynchronously (#1669)
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
Co-authored-by: Florian Wilhelm <florian.wilhelm02@sap.com>
2020-06-16 11:42:51 +02:00
Florian Wilhelm
b0329cd2c0
Use absolute path for m2path in mta install (#1671) 2020-06-16 09:48:47 +02:00
Daniel Kurzynski
9a30758115
Consider stage config from custom defaults (#1670) 2020-06-16 09:06:37 +02:00