stageName from parametersJSON was ignored up until now and there was a TODO in the code. The precedence from the TODO is not clear to me. It said "first env.STAGE_NAME, second: parametersJSON, third: flag". So maybe I implemented it wrongly, but it would be easy to change. Right now, STAGE_NAME is always in the ENV when piper is called from Jenkins, but to mimic the behavior of being able to pass `stageName` in the parameters and override the ENV value, this is what happens in go as well now.
The reason why `stageName` needs to be evaluated before other step parameters is that it affects config evaluation. So I've added a function which is called first in `PrepareConfig()`.
* Add abaputils pkg and go files
* Add ReadServiceKeyAbapEnvironment function
* Fixes
* Add structs for SC, Pull and Branch
* Minor Improvements
* Adapt unit tests to new abaputils pkg
* Fixes
* Add adapted tests
* Fixes
* Fix cloudfoundry test
* Add check for host prefix (HTTPS)
* Fix tests + cleanup
* Fixes
* Fixes
* Fix
* Add mock for abaputils pkg unit tests
* Adapt abaputils comments
* Add abapEnvironmentCheckoutBranch step setup
* Change description of abapEnvCheckoutBranch step
* Add http client code
* Disable code due to missing interace
* Add coding for use of abaputils
* Adapt checkout branch step
* Adapt URL for checkout_branch function import
* Fixes
* Add unit test for missing params case
* Fix for missing mapping of CfSpace
* Fix for missing mapping of CfSpace
* Add working code for a Branch Checkout
* Fix host schema
* Remove LogoutOption param of unit tests and steps
* Fix unit test
* Fix unit test CF ReadServiceKey
* Add abapEnvironmentCheckoutBranch step setup
* Change description of abapEnvCheckoutBranch step
* Add http client code
* Disable code due to missing interace
* Add coding for use of abaputils
* Adapt checkout branch step
* Adapt URL for checkout_branch function import
* Fixes
* Fix for missing mapping of CfSpace
* Add working code for a Branch Checkout
* Adapt changes of abautils pkg
* Add test for polling
* Minor fix
* Fix yaml spacing
* Add longdescription to yaml
* Refactor abaputil methods
* Refactoring
* Refactoring
* Minor fix
* Minor fixeds
* Adapt to new abaputils.AUtilsMock
* Delete obsolete initial checks for params
* Fix manageGitRepoUtils_test.go
* Adjust pollEntity tests
Co-authored-by: Daniel Mieg <56156797+DanielMieg@users.noreply.github.com>
Provide cloudFoundryDeploy step in GO layer.
Groovy part untouched. Groovy-Stub needs to be provided later (with a feature toggle in order to be able to switch back)
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
* If a step declares a parameter of type string, depending on how the config is written, it is no longer ignored, if it is interpreted by the yaml parser as integer or float value.
* If an expected parameter is present in the configuration, step execution will consistently fail if the parameter has the wrong type, no sensible conversion can take place, and it is known that the parameter will be ignored.
* For all type-mismatches that have no implemented conversion, a warning is logged. (It isn't known whether the conversion actually works, since it depends on both the yaml and json packages and future changes there.)
* Entries in the evaluated config with a value of nil are ignored.
* Whitesource MVP for Gradle, Golang, and NPM/Yarn
* Refactoring
* Refactor and cleanup, better error checking
* publish stepResults, use pkg/versioning, bubble up errors, add gomod versioning support
* Run gofmt and cleanup comments
* Resolve PR comments
* Update resources/metadata/whitesource.yaml
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
* Only determine project coordinates if they are missing
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
* Gradle versioning artifact
* fix gradle artifact version regexp and refactor
* Fix token extraction from output buffer
* Fix some issues with pip and jsonfile versioning logic
* Remove useless spacing
* Remove unnecessary test file and fix naming style for JSONDescriptor
* Automatically download wss-unified-agent if file does not exist
* adds downloadVulnerabilityReport, checkSecurityViolations, minor refactoring
* adds config.ReportDirectoryName, improves readability
* Version-wide reporting for vulnerabilities and list of libraries.
* Refactor and improve build accuracy
* fix sed command
* Add includes file pattern config option
* Adds --exclude command line flag
* run go mod tidy and regenerate step framework
* Fix unit tests
* revert changes
* poll project status before downloading reports
* merge with master
* go mod tidy, go fmt, and fix whitesource unit test
* sync go.mod
* sync go.mod again
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* Add error category parsing to cmd execution
It is now possible to define `ErrorCategoryMapping` as a `map[string][]string` on a `Command`.
The format contains the category as key which has a list of error patterns assigned.
Example:
```
cmd := Command{
ErrorCategoryMapping: map[string][]string
"build": {"build failed"},
"compliance": {"vulnerabilities found", "outdated components found"},
"test": {"some tests failed"},
},
}
```
Setting this map triggers console log parsing when executing a command.
If a match is found the error category is stored and
it will automatically be added to the `errorDetails.json`.
* clean up go.mod
* fix test
* fix test
* Update DEVELOPMENT.md
* fix tests
* address long console content without line breaks
* scan condition update
* fix test
* add missing comment for exported function
* Update pkg/command/command.go
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
This change adds linting support for Java-/TypeScript projects based on the checkDefaultLint and checkUserLint steps from Cloud SDK Pipeline.
Following options are supported:
1. Define a linting script named ci-lint in the package.json file(s) of the project
2. Provide a configuration for ESLint as part of the project
3. If none of the above: ESLint is executed with a general purpose configuration.
Co-authored-by: Daniel Kurzynski <daniel.kurzynski@sap.com>
* Add possibility to add category to failures
It is now possible to set the error category within the flow.
When exiting the program the error category can be used.
There is a convenience function available for exiting with a previously set category,
for example
```
log.SetErrorCategory(log.ErrorCompliance)
...
log.FatalError(err, "configuration error")
```
* extend test
* go mod tidy
* add missing comment
* update information about error categories
* Update DEVELOPMENT.md
* streamlined error category with standard term
* remove unused import from test
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* added new step gctsDeployCommit
* suggested PR fixes applied
* fixed test
* Remove unused imports
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
* added URL encoding for 'request' parameter
* regenerate after change
* minor changes
* renamed step to 'gctsDeploy'
* changed name
* remove space
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* Create step to save container image
* add error handling for directory creation
* Update permissions
* remove indirection and add tests
* finalize tests
* Fix test to also run on linux environments
* Update cmd/containerSaveImage.go
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
* Update resources/metadata/savecontainer.yaml
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
* Update cmd/containerSaveImage.go
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
* Update cmd/containerSaveImage_test.go
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
* Update generated file
Co-authored-by: Stephan Aßmus <stephan.assmus@sap.com>
* Add parameter "--ignoreCustomDefaults"
* Pass to piper customDefaults from config also via --defaultConfig
... and add "--ignoreCustomDefaults".
* Log output when ignoring customDefaults
Co-authored-by: Daniel Kurzynski <daniel.kurzynski@sap.com>
Avoid maven error `Unknown lifecycle phase \"-\"` when the value of a define contains `-`.
Don't split and trim maven arguments. Expect they come in as a list, keep them as list.
This is a breaking change compared to the old Groovy implementation which relied on using a shell for calling maven.
As an example, consider this diff:
```diff
- goals: 'org.apache.maven.plugins:maven-help-plugin:3.1.0:evaluate',
- defines: "-Dexpression=$pomPathExpression -DforceStdout -q",
+ goals: ['org.apache.maven.plugins:maven-help-plugin:3.1.0:evaluate'],
+ defines: ["-Dexpression=$pomPathExpression", "-DforceStdout", "-q"],
```
Do not exit with os.Exit(1) but using log.Entry().Fatal() instead
* Golang: forward error details
* extend groovy wrapper to provide proper error message
* create closure for error handling
With the step gctsCreateRepository it is possible to create a local gCTS repository on an ABAP server
Co-authored-by: Marcus Holl <marcus.holl@sap.com>
* Fix typos
* Support aliases also for secrets
* Adapt & extend Unit Tests
* Output deprecation warning for param/secret aliases
... if the alias is marked as 'deprecated'.
* Config and Defaults via http(s)
Add the possibility to use http(s) urls as path to
* config.yml
* defaults
* Remove comment
* Addressing PR feedback
* Fix error handling
Artifacts to upload are assembled for MTA projects and Maven projects with optional application sub-module. Then maven deploy:deploy-file is used as backend to upload bundles of artifacts plus sub-artifacts.
Co-authored-by: Florian Wilhelm <florian.wilhelm02@sap.com>
* Golang step metadata: Config aliases for steps
This will ease following scenarios:
* config migration due to step name changes
* re-use of more general config, e.g. `mavenExecute` in `mavenBuild`
* fix CodeClimate finding
* Fix panic if original stage config does not exist yet
* add static code checks for maven based projects as:
* pmd plugin
* spotBugs plugin
* test modules as unit-tests and integration-tests will be ignored by default. Additional modules to ignore are configurable
* for pmd: rulesets and excludes are configurable
* for spotBugs: includeFilter and excludeFilter are configurable
* Protecode as go implementation
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* Support yml and yaml extension of config files
* Update cmd/piper.go
Co-Authored-By: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
* Use commonPipelineEnvironment in go binary
* Update groovy part incl. tests
* Rework structure and naming
* Support influx resources in steps
* Update tests and some cleanups
* Add correct defer handling
* Address PR feedback
* Fix test
* Update resources.go
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
* Read the project config only if it exists
This avoid trying reading the file and have the control flow based
on errors. Beside that it helps troubleshooting when we have some
logging (debug level only).
* formatting only
* Adjust log level
* Export general configuration - part 2
First part in #956 missed to export the elements of the struct ...
* Add comment for exported struct
* Add function for opening config files
