* Fix logic for engineConfigurationID
Use the project-level value if no value is defined in the piper config.
Remove the default value of 1 for engineConfigurationID.
* Add the OS agnostic way of installing golangci-lint
* Fix
* Clean up
* Modify unit tests
* Rename downloaded archive
* Refactor
* Expose golangci-lint url as a parameter
* Clean up
* Rename parameter
* Update mock
* Fix golangci-lint version
* refactor: rebranding
rebranding from SAP Cloud Platform to SAP BTP
* refactor: rebranding from SCP to BTP
Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com>
* Add alias
* Add step and defaults
* Add closing }
* Add test
* Remove alias
* initial copy
* Change step activation
* Change test
* Read foreign config
* Fix Syntax
* Test
* Test
* Only consider stage config
* Test config
* Test
* Test
* Change echo
* Fix
* Test
* Default test
* Uncomment
* Test different outputs
* Adapt Config
Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com>
* changes to detectExec before master merge
* changes for detectExecuteScan
* self generated code added
* fix syntax errors and update docu
* added unit tests for fail and Group
* fix failOn bug
* add Groups as string array
* add Groups as string array
* tests and validation for groups, failOn
* Updated docs and added more tests
* documentation md files should not be changed
* Handle merge conflicts from PR 1845
* fix merge errors
* remove duplicate groups, merge error
* adding buildCode and buildTool as params
* switching build options
* building maven modules
* parameter correction
* parameter correction
* gnerate with new build parameter
* adding comments
* removing piper lib master and modifying goUtils to download 1.5.7 release
* first cleaning then installing
* multi module maven built
* multi module maven built removing unwanted code
* multi module maven built moving inside switch
* testing
* modifying the default use case to also call maven build
* modifying the default use case to also call maven build wih --
* corrected maven build command
* corrected maven build command with %v
* skipping test runs
* testing for MTA project with single pom
* adding absolute path to m2 path
* clean up
* adding switch for mta and maven and removing env from containers
* commiting changes for new detect step
* correting log message
* code clean up
* unit tests changes to detectExecute
* basic tests for new change
* restoring piperGoUtils to download correct piper binary
* code clean up
* code clean up
* protecodeExecuteScan -> Added authentication with user API key
* protecodeExecuteScan -> updating .yml file
* protecodeExecuteScan -> go generate fixed
* protecodeExecuteScan -> naming convention applied for UserAPIKey parameter
* protecodeExecuteScan -> extending groovy code for mapping jenkins credentials
Co-authored-by: D072410 <giridhar.shenoy@sap.com>
Co-authored-by: Keshav <anil.keshav@sap.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* add minscaninterval parameter
* update detectExec
* removed a single trailing space which caused a lint failure
* Add test case
* Ensure unmap is false
* fix test case
* update format of param value
* Initially generated tmsUpload<...> files
* First provisioning of parameters supported by tmsUpload step
* Refer to Go step from tmsUpload.groovy
* Initial client implementation
* Reverting line delimiters in tmsUpoad.groovy back to Unix ones
* Temporarily remove when-condition for Release stage
* Define useGoStep parameter in tmsUpload.groovy
* Unstash buildResult if useGoStep is true
* No unstashing and empty credentials, when using go step
* Register TmsUploadCommand in piper.go
* Cleanup groovy-related changes - they will be temporarily implemented in a different repo
* Make getting OAuth token success
* Look through the code and cleanup it a bit
* Read service key from Jenkins credentials store
* Provide initial set of unit tests for methods in /pkg/tms/tms.go file
* Minor improvements on logging response on http call error
* Check, if positive HTTP status code is as expected
* Cleanup tms.yaml file, provide additional unit test for tms.go
* Provide unit test for the case, when request body contains spaces
* Specify nodeExtDescriptorMapping parameter as of type map in tms.yaml
* Implement client method for getting nodes
* Write tests for GetNodes method
* Add GetMtaExtDescriptor client method and cover it with unit tests
* Provide first implementation for Update- and UploadMtaExtDescriptor
client methods
* Provide first implementation for Update- and UploadMtaExtDescriptor
client methods
* Provide UploadFile and UploadFileToNode client methods
* Provide tests for Update- and UploadMtaExtDescriptor client methods
* Write tests for FileUpload and FileUploadToNode client methods
* Minor corrections
* Remove some TODO comments
* Rename some of response structures
* Revert change for line delimiters in cmd/piper.go
* Add uploadType string parameter to UploadFile and UploadRequest methods
of uploader mock to reflect the changed Uploader implementation
* Start to implement execution logic in tmsUpload.go
* Changes in tms.yaml file
- remove resources from inputs in tms.yaml and implement mtaPath
parameter settings in the yaml file the same way, as it is done in
cloudFoundryDeploy.yaml
- rename tms.yaml to tmsUpload.yaml, since some generation policy
changed meanwhile
* Rename tms.yaml to tmsUpload.yaml and do go generate
* Use provided proxy on communication with UAA and TMS
* Set proxy even before getting OAuth token
* Further implementation of tmsUpload.go
* Continuation on implementing the tmsUpload.go executor
* Get mtarFilePath and git commitId from commonPipelineEnvironment, if
they are missing in configuration file + minor changes
* Implement a happy path test for tmsUpload logic
* Cover with unit tests main happy and error paths of tmsUpload.go logic
* Extend set of unit tests for tmsUpload.go
* Eliminate some TODOs, extend unit tests for tmsUpload.go
* Delete some TODOs
* Remove a couple of more TODOs from tms_test.go file
* Provide additional unit test for error due unexpected positive http
status code on upload
* Revert back line delimiters in cmd/piper.go
* Comment out file uploading calls in tmsUpload.go
* Run go generate to update generated files
* Convert line delimiters in tmsUpload.yaml to Unix ones, as well as
provide new line character in the end of the file, plus minor fix for
logging in tmsUpload.go file (pipeline complained)
* Correct description of a parameter in tmsUpload.yaml, extend unit tests
to check for trimming a slash in the end of TMS url for client methods
that do upload
* [minor] Add a comment in the test code
* Add stashContent parameter to do unstashing in tmsUpload.groovy, remove
some of the clarified TODOs
* Uncomment uploading file calls in tmsUpload.go, declare buildResult
stash in tmsUpload.yaml
* Remove clarified TODOs from the tmsUpload.go file
* Run go fmt for jenkins-library/pkg/tms
* Do not get explicitly values from common pipeline environment - all
configurations are provided in yaml file
* Remove unused struct from tmsUpload_test.go
* Run go fmt jenkins-library\pkg\tms
* Revise descriptions of parameters provided in tmsUpload.yaml file
* Specify STAGES scope for tmsUpload parameters
* Provide STAGES scope for the tmsUpload parameters, provide default value
for stashContent parameter
* Remove trailing space from tmsUpload.yaml
* Provide unit tests for proxy-related changes in http.go file
* Improve proxy implementation in tmsUpload.go file
* Make tmsServiceKey again a mandatory parameter
* Run go generate command to make the generated files correspond the yaml
state
* Change line delimiters back to Unix ones (were switched while resolving
the conflicts)
* Remove trailing spaces from tmsUpload.yaml
* Minor change in a comment to trigger pipelines with commit
* Improve checks for zero-structs and for empty maps, as well as use
different package to read files in the tests
* Revert line endings in http.go
* Revert comments formatting changes in files that do not belong to the tmsUpload step
* Add cds generated source code to Fortify scans.
This generated source code is needed to avoid false negatives when scanning code that uses the CAP framework.
* Also change documentation.
* Forgot comma.
* Run go generate.
* Change test.
Co-authored-by: sumeet patil <sumeet.patil@sap.com>
* WIP: Adapt bom names
* + WIP: Adapt bom filenames
* Upgrade cyclonedx gradle plugin and use cyclonedxBom config parameters
* Fix unit tests - use correct name in bom creation
* Fix pythonBuild bom name
* introduce and use npmBomFilename const
* Introduce and use mvnBomFilename const
* Introduce and use gradleBomFilename const
* Use build-tool names for bom suffix
* + Adapt tests (build tool suffix)
* Use BOM schema version 1.2 in gradleExecuteBuild
* Pin version of cyclonedx-maven-plugin to 2.7.1
* Adapt generated files
* Fix integration tests
* Fix integration tests
* Fix gradle build integration tests
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* passing registry username and password
* enhance the case for creating docker config json with user credentials
* refactoring code
* unit test and maintaing user provided docker config json file
* go generate
* remove addtional file addition to unit test
Co-authored-by: anilkeshav27 <you@example.com>
For running open source vulnerability scans in de-coupled processes
it is helpful to allow that steps only create
compliance reports to inform users/teams
but not fail the pipeline.
This can now be achieved constitently with the flag:
`failOnSevereVulnerabilities`
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Sets git reference and gitRemoteCommitId.
Jenkins has 2 strategies - 'Merging the pull request with the current target branch revision' and 'The current pull request revision'. When 'Merging the pull request with the current target branch revision' is run, Jenkins creates a local merge commit and runs a job for that particular merge commitId. This commitId is then used for codeql to upload sarif, on upload it throws an error as the merge commit does not exist in github. To resolve this we have introduces a new variable 'gitRemoteCommitId' in commonPipelineEnvironment which gives the remote merge commit id.
* enable build without values
* add sap-client as option
* use function from /net/url to add parameters
Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com>
* including comma seperated strings as arguments
* fix unit test
* adding unit test
* fix unit test no param case
Co-authored-by: anilkeshav27 <you@example.com>
* Add ans implementation
* Remove todo comment
* Rename test function
Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>
* Better wording
Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>
* Add reading of response body function
* Use http pkg ReadResponseBody
* Check read error
* Better test case description
* Fix formatting
* Create own package for read response body
* Omit empty nested resource struct
* Separate Resource struct from Event struct
* Merge and unmarshall instead of only unmarshalling
* Improve status code error message
* Remove unchangeable event fields
* Separate event parts
* Change log level setter function
* Restructure ans send test
* Revert exporting readResponseBody function
Instead the code is duplicated in the xsuaa and ans package
* Add check correct ans setup request
* Add set options function for mocking
* Review fixes
* Correct function name
* Use strict unmarshalling
* Validate event
* Move functions
* Add documentation comments
* improve test
* Validate event
* Add logrus hook for ans
* Set defaults on new hook creation
* Fix log level on error
* Don't alter entry log level
* Set severity fatal on 'fatal error' log message
* Ensure that log entries don't affect each other
* Remove unnecessary correlationID
* Use file path instead of event template string
* Improve warning messages
* Add empty log message check
* Allow configuration from file and string
* Add sourceEventId to tags
* Change resourceType to Pipeline
* Use structured config approach
* Use new log level set function
* Check correct setup and return error
* Mock http requests
* Only send log level warning or higher
* Use new function name
* One-liner ifs
* Improve test name
* Fix tests
* Prevent double firing
* Reduce Fire test size
* Add error message to test
* Reduce newANSHook test size
* Further check error
* Rename to defaultEvent in hook struct
* Reduce ifs further
* Fix set error category test
The ansHook Fire test cannot run in parallel, as it would affect the
other tests that use the error category.
* Change function name to SetServiceKey
* Validate event
* Rename to eventTemplate in hook struct
* Move copy to event.go
* Fix function mix
* Remove unnecessary cleanup
* Remove parallel test
The translation fails now and again when parallel is on.
* Remove prefix test
* Remove unused copyEvent function
* Fix ifs
* Add docu comment
* Register ans hook from pkg
* register hook and setup event template seperately
* Exclusively read eventTemplate from environment
* setupEventTemplate tests
* adjust hook levels test
* sync tests- wlill still fail
* migrate TestANSHook_registerANSHook test
* fixes
* Add ans send event step
* Fix tests
* Add groovy wrapper
* Add groovy wrapper test
* Fix function names
* Reduce ifs
* Fix docu
* We always set the timestamp
* Validate event
* Test unknown field in json
* Make test list test
* Set all event fields as separate parameters
* Generate and fix code
* Review fixes
* Format test file
* Format go code
* Fix common steps tests
* Print event to console if verbose
Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>
Co-authored-by: Roland Stengel <r.stengel@sap.com>
* add Step azureBlobUpload
* add azure sdk and unit tests
* add Documentation
* fix Groovy Wrapper
* adopt the requested changes from awsS3Upload
* fix lint tests
* downgrade azure sdk to go 1.17
* multiple fixes e.g. use of temporary files for tests
* fix tests
* Update cmd/azureBlobUpload.go
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
* Update cmd/azureBlobUpload.go
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
* Update documentation/docs/steps/azureBlobUpload.md
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
* Update documentation/docs/steps/azureBlobUpload.md
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
* Update documentation/docs/steps/azureBlobUpload.md
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
* Update documentation/docs/steps/azureBlobUpload.md
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
* requested changes
* use latest version of azure sdk after update to go 1.18
* change staticcheck from 1.1.0 to 1.2.0
* try to fix lint test by pre-compiling go 1.18
* fix caching for lint test
* improve error handling by dividing runner
* improve error handling and add validation
* multiple naming fixes
* add new test for unmarshalling JSON-Structs
* Update cmd/azureBlobUpload_test.go
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
* Update cmd/azureBlobUpload_test.go
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
* Update cmd/azureBlobUpload_test.go
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
* fix JSON unmarshall test
* Update documentation/docs/steps/azureBlobUpload.md
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
* Update cmd/azureBlobUpload_test.go
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
* Update cmd/azureBlobUpload.go
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
* fix uploadFunc
Co-authored-by: Thorsten Duda <thorsten.duda@sap.com>
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
* adding arguments
* splitting strings into args and checking position
* addtional check on adding arguments
* unit testing
* refactoring code
* unit test clean up
* add unit test for multiple params in multiple scripts
* unit test name
Co-authored-by: anilkeshav27 <you@example.com>
* Improvements were made
* fixed tests
* fixed issues
* fix versioning
* fix Inclusive Language warnings
* gradle support to fortifyExecuteScan. Classpath resolving
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* enhance build step
* Update abapEnvironmentBuild.go
* build with addonDescriptor
* use addondescriptor
* Use Addondescriptor
* add error
* improve logging
* rename intervall to interval
* update yaml
* Update abapEnvironmentBuild.go
* Update abapEnvironmentBuild.yaml
* insert generation phase
* we do not know what we have done
* Add createServiceKey to test
Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com>
Co-authored-by: Daniel Mieg <daniel.mieg@sap.com>
* fix(helmExecute): respect version from Chart
using version from CPE can create failure situations in case format is not semver.
This is the case for maven artifacts, for example.
* chore: simplify condition
* chore: cleanup
* chore: cleanup
* explicitly adding tar extension to project name when constructing the targetFilePath for whitesource docker image download
* comments
* correcting comment for better readability
* replace spaces in the project name with underscroe
* better comments
* passing legacy format download
* appending format to value
* keeping the download format for protecode as legacy
* improving docu
* keeping legacy format the default
* keeping tar file name same as project name to avoid duplicate names
* keeping legacy format download hard coded
Co-authored-by: anilkeshav27 <you@example.com>
* create virtual env
* adding bin bash source
* using sources from bin bash
* trying with bash
* appending filename to source
* using standard pip install
* not using root user
* adding path for pip
* using virtual env
* using virtual env name in path
* removing virtual env
* adding file path manually
* using root
* not using root and postpone removing venv
* trying to use the python from venv
* test to remove the venve
* seeing which python
* using symlink for python
* unit test
* python docu stub
* fix unit test and yaml extra line
* fixing unit test
* unit test success case fix
* unit test fix
* unit test fixes
* unit test and default publish flag
* fix integration test
Co-authored-by: anilkeshav27 <you@example.com>
* Add small fix
* fix unit-tests
* Add deploymentName and packageVersion as flags
* small fix
* Change getting version of helm chart
* small fix
Co-authored-by: “Vitalii <“vitalii.sidorov@sap.com”>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* add parameter for uploading multiple assets
* use assetPathList parameter
* add test case
* fix typo
* fix test case
* generage ghClient mock
* add test files
* make function testable
* add test case
* regenerate mock
* regenerate mocks
* feat(gitopsUpdateDeployment) forcePush
fix(gitopsUpdateDeployment) include registry
The push operation in this step can be forced to bypass branch-protection
Signed-off-by: Michael Sprauer <Michael.Sprauer@sap.com>
* add unit test
Signed-off-by: Michael Sprauer <Michael.Sprauer@sap.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* Remove --backend-type
* Delete CTS in isChangeDevelopment and change Dockerimage of CM-Client
* fix groovy unit tests
* another fix of groovy unit tests
* try to fix import of fork for Jenkins-Testing
* add workflow to create Go Binary for Jenkins-Server
* Change RepoOwner to test in Fork
* remove previous changes
* adjust docker image for TransportRequestCreate and Release
* Remove CTS from Documentation
Co-authored-by: Thorsten Duda <thorsten.duda@sap.com>
* adding PIP to BuildTool.groovy
* trying to run the container with root
* only creating sdist
* including wheel distribution
* adding settings info
Co-authored-by: anilkeshav27 <you@example.com>
* first version to download script from git
* unit test adjust
* adding git token
* info messages
* removing extra info message
* changing file permission for scrtips
* modying sources to handle https download
* adding script downloads
* commenting the file permission change
* changing persmission
* adding header to download file
* adding perimssions
* adding perimssions
* not touching file permissions
* adding to pipeline
* return file name
* changing script name
* adding file permission changes
* adding file permission changes
* using current directory
* file permission
* downloading in .pipeline folder
* removing permission handeling
* improving the step docu
* improving the step docu
* unit test and code cleaning
* fix typo
* adding read execute permission
* fix unit test
* fix unit test
* removing negative test
Co-authored-by: anilkeshav27 <you@example.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* Add helm dependency command
* Change name of flag for package command
Co-authored-by: “Vitalii <“vitalii.sidorov@sap.com”>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* gCTS Scenario and two images
* Updated gCTS Scenario
* updated gCTS_Scenario.md and images
* added checkstyle stage
* upated the gCTSExecuteABAPQualityChecks.yaml file with severity information
* updates to gCTSExecuteABAPQualityChecks and gCTSExecuteABAPUnitTests steps
* updated gCTS_Scenario.md and relevant images
* Add more explanation for workspace parameter, change atc variant default value to DEFAULT
* Fix a type in description of workspace parameter
* changes to gctsExecuteABApUnitTests.md gctsExecuteABAPQualityChecks.yaml and gctsExecuteABApUnitTests.yaml
* final updates to gCTS steps and scenario
* added env. to GIT_COMMIT in gCTSExecuteABAPQualityChecks.md and gCTSExecuteABAPUnitTests.md
* remove the gScenario description from the branch
* Correct column typo in yaml file and generate files
* Add a new line in docu for gCTSExecuteABAPUnitTests
* Remove blank line from docu in gctsExecuteABAPUnitTests
Co-authored-by: Rinita Asani <rinita.asani@sap.com>
Co-authored-by: Sarat Krishnan <78093145+sarat-krk@users.noreply.github.com>
* Implementation of pythonBuild step
* minor update and refactoring
* minor update
* add integration test and test project to testdata dir
* remove generated build data dir
* Rewrite some logic. Minor fix in integration tests for python
* Add new input parameters to pythonBuild.yaml
* rewrite logic
remove some checks
* rollback
* resolve merge conflict in piper.go
Update logic in python build. Create bom now works fine
* remove duplicate line
* refactoring fix
* resolve comment. Remove install build and change build command. Change twine upload command
* add groovy wrapper for pythonBuild step
* Rewrite tests. Remove some cheks from pythonBuild.go
* add some test to pythonBuild_test.go
* Add some parameters and credentials to the pythonBuild.groovy
* fix issue in unit tests
* add pythonBuild to fieldRelatedWhitelist
* update integration test for pythonBuild
* add imports
* update integration tests and add a new one
* minor fix
* fix some issues in integration tests
* update integration tests. Make it works again
Co-authored-by: Anil Keshav <anil.keshav@sap.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* Implemented bom creation
* Made small fixes. Added integration tests
* go generate
* minor fixes
* fix tests
* Added unit tests
* minor fixes
* use fileutils
* integration tests optimization
* change integraton tests timeout to 25m
* Fix Inclusive Language warnings
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* Add runHelmCommand
* Add dryRun for debug
* Add default case in helmExecute
* Fix unit-tests
* small fix
* Fix RunHelmAdd and change RunHelmPublish methods
* Fix RunHelmPublish
* Fix unit-tests
* Fix unit-test
* small fix
* small fix
* small fix
* Add LintFlag PackageFlag PublishFlag flags
* Add tests for httpClient.go
* test
* test
* smal fix
* small fix
* Add getting name and version from Chart.yaml
* Add test
* Fix
* small fix
* Fix according to comments
* small fix
Co-authored-by: “Vitalii <“vitalii.sidorov@sap.com”>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Vitalii Sidorov <vitalii_sidorov@sap.com>
* (fix) gitopsUpdateDeployment - integration test for kustomize
Signed-off-by: Michael Sprauer <Michael.Sprauer@sap.com>
* (fix) test: dash login shell is "-l"
Signed-off-by: Michael Sprauer <Michael.Sprauer@sap.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
this was already used in fortifyExecuteScan, but had no effect.
Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
