* Initial in progress
* compiling but not yet functional
* Missed file
* updated checkmarxone step
* Working up to fetching a project then breaks
* Missed file
* Breaks when retrieving projects+proxy set
* Create project & run scan working, now polling
* Fixed polling
* added back the zipfile remove command
* Fixed polling again
* Generates and downloads PDF report
* Updated and working, prep for refactor
* Added compliance steps
* Cleanup, reporting, added groovy connector
* fixed groovy file
* checkmarxone to checkmarxOne
* checkmarxone to checkmarxOne
* split credentials (id+secret, apikey), renamed pullrequestname to branch, groovy fix
* Fixed filenames & yaml
* missed the metadata_generated.go
* added json to sarif conversion
* fix:type in new checkmarxone package
* fix:type in new checkmarxone package
* removed test logs, added temp error log for creds
* extra debugging to fix crash
* improved auth logging, fixed query parse issue
* fixed bug with group fetch when using oauth user
* CWE can be -1 if not defined, can't be uint
* Query also had CweID
* Disabled predicates-fetch in sarif generation
* Removing leftover info log message
* Better error handling
* fixed default preset configuration
* removing .bat files - sorry
* Cleanup per initial review
* refactoring per Gist, fixed project find, add apps
* small fix - sorry for commit noise while testing
* Fixing issues with incremental scans.
* removing maxretries
* Updated per PR feedback, further changes todo toda
* JSON Report changes and reporting cleanup
* removing .bat (again?)
* adding docs, groovy unit test, linter fixes
* Started adding tests maybe 15% covered
* fix(checkmarxOne): test cases for pkg and reporting
* fix(checkmarxOne):fix formatting
* feat(checkmarxone): update interface with missing method
* feat(checkmarxone):change runStep signature to be able to inject dependency
* feat(checkmarxone): add tests for step (wip)
* Adding a bit more coverage
* feat(checkmarxOne): fix code review
* feat(checkmarxOne): fix code review
* feat(checkmarxOne): fix code review
* feat(checkmarxOne): fix integration test PR
---------
Co-authored-by: thtri <trinhthanhhai@gmail.com>
Co-authored-by: Thanh-Hai Trinh <thanh.hai.trinh@sap.com>
* add ascAppUpload step
add step that uploads an app binary to Application Support Center (ASC)
* re-run go generate
* fix typo in CODEOWNERS
* Update CODEOWNERS
* Initially generated tmsUpload<...> files
* First provisioning of parameters supported by tmsUpload step
* Refer to Go step from tmsUpload.groovy
* Initial client implementation
* Reverting line delimiters in tmsUpoad.groovy back to Unix ones
* Temporarily remove when-condition for Release stage
* Define useGoStep parameter in tmsUpload.groovy
* Unstash buildResult if useGoStep is true
* No unstashing and empty credentials, when using go step
* Register TmsUploadCommand in piper.go
* Cleanup groovy-related changes - they will be temporarily implemented in a different repo
* Make getting OAuth token success
* Look through the code and cleanup it a bit
* Read service key from Jenkins credentials store
* Provide initial set of unit tests for methods in /pkg/tms/tms.go file
* Minor improvements on logging response on http call error
* Check, if positive HTTP status code is as expected
* Cleanup tms.yaml file, provide additional unit test for tms.go
* Provide unit test for the case, when request body contains spaces
* Specify nodeExtDescriptorMapping parameter as of type map in tms.yaml
* Implement client method for getting nodes
* Write tests for GetNodes method
* Add GetMtaExtDescriptor client method and cover it with unit tests
* Provide first implementation for Update- and UploadMtaExtDescriptor
client methods
* Provide first implementation for Update- and UploadMtaExtDescriptor
client methods
* Provide UploadFile and UploadFileToNode client methods
* Provide tests for Update- and UploadMtaExtDescriptor client methods
* Write tests for FileUpload and FileUploadToNode client methods
* Minor corrections
* Remove some TODO comments
* Rename some of response structures
* Revert change for line delimiters in cmd/piper.go
* Add uploadType string parameter to UploadFile and UploadRequest methods
of uploader mock to reflect the changed Uploader implementation
* Start to implement execution logic in tmsUpload.go
* Changes in tms.yaml file
- remove resources from inputs in tms.yaml and implement mtaPath
parameter settings in the yaml file the same way, as it is done in
cloudFoundryDeploy.yaml
- rename tms.yaml to tmsUpload.yaml, since some generation policy
changed meanwhile
* Rename tms.yaml to tmsUpload.yaml and do go generate
* Use provided proxy on communication with UAA and TMS
* Set proxy even before getting OAuth token
* Further implementation of tmsUpload.go
* Continuation on implementing the tmsUpload.go executor
* Get mtarFilePath and git commitId from commonPipelineEnvironment, if
they are missing in configuration file + minor changes
* Implement a happy path test for tmsUpload logic
* Cover with unit tests main happy and error paths of tmsUpload.go logic
* Extend set of unit tests for tmsUpload.go
* Eliminate some TODOs, extend unit tests for tmsUpload.go
* Delete some TODOs
* Remove a couple of more TODOs from tms_test.go file
* Provide additional unit test for error due unexpected positive http
status code on upload
* Revert back line delimiters in cmd/piper.go
* Comment out file uploading calls in tmsUpload.go
* Run go generate to update generated files
* Convert line delimiters in tmsUpload.yaml to Unix ones, as well as
provide new line character in the end of the file, plus minor fix for
logging in tmsUpload.go file (pipeline complained)
* Correct description of a parameter in tmsUpload.yaml, extend unit tests
to check for trimming a slash in the end of TMS url for client methods
that do upload
* [minor] Add a comment in the test code
* Add stashContent parameter to do unstashing in tmsUpload.groovy, remove
some of the clarified TODOs
* Uncomment uploading file calls in tmsUpload.go, declare buildResult
stash in tmsUpload.yaml
* Remove clarified TODOs from the tmsUpload.go file
* Run go fmt for jenkins-library/pkg/tms
* Do not get explicitly values from common pipeline environment - all
configurations are provided in yaml file
* Remove unused struct from tmsUpload_test.go
* Run go fmt jenkins-library\pkg\tms
* Revise descriptions of parameters provided in tmsUpload.yaml file
* Specify STAGES scope for tmsUpload parameters
* Provide STAGES scope for the tmsUpload parameters, provide default value
for stashContent parameter
* Remove trailing space from tmsUpload.yaml
* Provide unit tests for proxy-related changes in http.go file
* Improve proxy implementation in tmsUpload.go file
* Make tmsServiceKey again a mandatory parameter
* Run go generate command to make the generated files correspond the yaml
state
* Change line delimiters back to Unix ones (were switched while resolving
the conflicts)
* Remove trailing spaces from tmsUpload.yaml
* Minor change in a comment to trigger pipelines with commit
* Improve checks for zero-structs and for empty maps, as well as use
different package to read files in the tests
* Revert line endings in http.go
* Revert comments formatting changes in files that do not belong to the tmsUpload step
* Add ans implementation
* Remove todo comment
* Rename test function
Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>
* Better wording
Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>
* Add reading of response body function
* Use http pkg ReadResponseBody
* Check read error
* Better test case description
* Fix formatting
* Create own package for read response body
* Omit empty nested resource struct
* Separate Resource struct from Event struct
* Merge and unmarshall instead of only unmarshalling
* Improve status code error message
* Remove unchangeable event fields
* Separate event parts
* Change log level setter function
* Restructure ans send test
* Revert exporting readResponseBody function
Instead the code is duplicated in the xsuaa and ans package
* Add check correct ans setup request
* Add set options function for mocking
* Review fixes
* Correct function name
* Use strict unmarshalling
* Validate event
* Move functions
* Add documentation comments
* improve test
* Validate event
* Add logrus hook for ans
* Set defaults on new hook creation
* Fix log level on error
* Don't alter entry log level
* Set severity fatal on 'fatal error' log message
* Ensure that log entries don't affect each other
* Remove unnecessary correlationID
* Use file path instead of event template string
* Improve warning messages
* Add empty log message check
* Allow configuration from file and string
* Add sourceEventId to tags
* Change resourceType to Pipeline
* Use structured config approach
* Use new log level set function
* Check correct setup and return error
* Mock http requests
* Only send log level warning or higher
* Use new function name
* One-liner ifs
* Improve test name
* Fix tests
* Prevent double firing
* Reduce Fire test size
* Add error message to test
* Reduce newANSHook test size
* Further check error
* Rename to defaultEvent in hook struct
* Reduce ifs further
* Fix set error category test
The ansHook Fire test cannot run in parallel, as it would affect the
other tests that use the error category.
* Change function name to SetServiceKey
* Validate event
* Rename to eventTemplate in hook struct
* Move copy to event.go
* Fix function mix
* Remove unnecessary cleanup
* Remove parallel test
The translation fails now and again when parallel is on.
* Remove prefix test
* Remove unused copyEvent function
* Fix ifs
* Add docu comment
* Register ans hook from pkg
* register hook and setup event template seperately
* Exclusively read eventTemplate from environment
* setupEventTemplate tests
* adjust hook levels test
* sync tests- wlill still fail
* migrate TestANSHook_registerANSHook test
* fixes
* Add ans send event step
* Fix tests
* Add groovy wrapper
* Add groovy wrapper test
* Fix function names
* Reduce ifs
* Fix docu
* We always set the timestamp
* Validate event
* Test unknown field in json
* Make test list test
* Set all event fields as separate parameters
* Generate and fix code
* Review fixes
* Format test file
* Format go code
* Fix common steps tests
* Print event to console if verbose
Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>
Co-authored-by: Roland Stengel <r.stengel@sap.com>
* add Step azureBlobUpload
* add azure sdk and unit tests
* add Documentation
* fix Groovy Wrapper
* adopt the requested changes from awsS3Upload
* fix lint tests
* downgrade azure sdk to go 1.17
* multiple fixes e.g. use of temporary files for tests
* fix tests
* Update cmd/azureBlobUpload.go
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
* Update cmd/azureBlobUpload.go
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
* Update documentation/docs/steps/azureBlobUpload.md
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
* Update documentation/docs/steps/azureBlobUpload.md
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
* Update documentation/docs/steps/azureBlobUpload.md
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
* Update documentation/docs/steps/azureBlobUpload.md
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
* requested changes
* use latest version of azure sdk after update to go 1.18
* change staticcheck from 1.1.0 to 1.2.0
* try to fix lint test by pre-compiling go 1.18
* fix caching for lint test
* improve error handling by dividing runner
* improve error handling and add validation
* multiple naming fixes
* add new test for unmarshalling JSON-Structs
* Update cmd/azureBlobUpload_test.go
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
* Update cmd/azureBlobUpload_test.go
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
* Update cmd/azureBlobUpload_test.go
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
* fix JSON unmarshall test
* Update documentation/docs/steps/azureBlobUpload.md
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
* Update cmd/azureBlobUpload_test.go
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
* Update cmd/azureBlobUpload.go
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
* fix uploadFunc
Co-authored-by: Thorsten Duda <thorsten.duda@sap.com>
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
* Implementation of pythonBuild step
* minor update and refactoring
* minor update
* add integration test and test project to testdata dir
* remove generated build data dir
* Rewrite some logic. Minor fix in integration tests for python
* Add new input parameters to pythonBuild.yaml
* rewrite logic
remove some checks
* rollback
* resolve merge conflict in piper.go
Update logic in python build. Create bom now works fine
* remove duplicate line
* refactoring fix
* resolve comment. Remove install build and change build command. Change twine upload command
* add groovy wrapper for pythonBuild step
* Rewrite tests. Remove some cheks from pythonBuild.go
* add some test to pythonBuild_test.go
* Add some parameters and credentials to the pythonBuild.groovy
* fix issue in unit tests
* add pythonBuild to fieldRelatedWhitelist
* update integration test for pythonBuild
* add imports
* update integration tests and add a new one
* minor fix
* fix some issues in integration tests
* update integration tests. Make it works again
Co-authored-by: Anil Keshav <anil.keshav@sap.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* Reorders getApiInformation, changes variables to get start time, adjusts and adds test cases
* Changes the way to get apiInformation and reduces number of requests
* Changes getting pipeline start time from correct env variable
* Refactors getApiInformation functionality
* Adds GetBuildReason() for Azure and Jenkins
* Updates JobURL for ADO
* Adjust local changed objects with commit history
* fix the righ history entry
* fix for local packages commit history
* Adapt unit tests and fix for retry after 500
* Rename gctsExecuteABAPUnitTests to gctsExecuteABAPQualityChecks
* Add unit test file for gCTSExecuteABAPUnitTest
* add a test step
* gcts Test change files
* Add unit test for gCTS test
* adapt gctsTEST
* deletegcts
* command
* maintain old step
* Adapt changes to documentation
* fix for go generate
* Implement helm step
* Create kubernetes package
* Refactoring helm.go
* Add package, test commands
* Add test for helm package
* Add tests for helm.go
* Add tests for helm.go
* Add tests for utils.go
* Add tests for helmExecute.go
* small fix
* Add helm lint
* small fix
* small fix
* Fix according to comments
* Fix test
* small fix
* Add helm add function
* Changes according to new comments
* Add helm push
* Add unit tests
* Add tests for helmExecute
* Add small fix
* small fix
* small fix
* Move DeployUtilsBundle from kubernetesDeploy to kubernetes package
* small fix
* small fix
* Add unit-tests
* Fix
* Update resources/metadata/helmExecute.yaml
* Update resources/metadata/helmExecute.yaml
* Add helm chart server parameterization
* small fix
* small fix
Co-authored-by: “Vitalii <“vitalii.sidorov@sap.com”>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* Checkin Go File - Push ATC Config
* ATC Push Config - Testfile
* Initial generated go files for ATC System Configuration
* ApiProxyUpload Command (#3295)
* ApiProxyUpload Command
* Code Review Fixes
* CodeReview Changes
* CodeReview Fixes
* YAML fixes
* CodeReview Fix
* Code Review Fixes
* CodeReview Fixes
* Code Climate Fixes
* Code Review Fixes
* Code Review Fixes
Co-authored-by: Roland Stengel <r.stengel@sap.com>
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
* Fetch Xcrsf-Token
* Update abapEnvironmentPushATCSystemConfig.yaml
* docs: fix urls (#3400)
* fix stash and adapt unit tests accordingly (#3403)
* Push ATC System Configuration - File read, send request, parse response
* fetchATCXcsrfToken
* fix(whitesourceExecuteScan): added missing credential declaration for the docker config (#3404)
* insert command in piper.go
* feat(cnbBuild): remove docker config after parsing (#3417)
Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>
* more funcs
* more funcs
* Revert "feat: Add debug output to getConfig (#3341)" (#3396)
This reverts commit b760bf48e7.
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* feat (kubernetesDeploy) allow secret creation in cluster with kubectl as deploy tool (#3398)
* defaulting deocker config json location
* function change
* harmonising secret creation via json config apply
* adding the env path to kaniko
* env var
* adding path
* adding path
* adding path
* rolling back changes for path
* adapting condition for secret creation
* conditions based on username and password only
* fix unit test as per new secret creation
* update documentation with regards to secret creation
* fixing yaml lint empty line
* fixing trailing line from kaniko yaml
* error condition when path of the docker config json file is not found
Co-authored-by: anilkeshav27 <you@example.com>
* readPipelineEnv -> Serialize Directly to os.Stdout (#3180)
* Serialize CPE Directly to os.Stdout
* Response Error Handling 1
* Add easy mode for AUnit & ATC (#3389)
* remove mandatory flag from config files
* Enable repo.yml as config
* Adapt to merge
* Refactoring
* Refactoring
* avoid panic
* Add comments
* Add easy mode for atc
* Add tests
* Add test
* Refactor
* Add test for MPS
* Updates
* Rename functions
* Add files to gitignore
* Rename
* Renaming
* Renaming
* Renaming
* Improve error messages
* Update documentation
* Add logging
* Rename
* Extend gitignore
* fix(sonar): fix file patterns for gcs upload (#3406)
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
* testing piper forks (#3420)
* developer doku update on how to test with forked repos
Co-authored-by: rosemarieB <45030247+rosemarieB@users.noreply.github.com>
* Error Messages parsing
* Update yaml parameter
* feat (npmExecuteScripts) create seperate npmrc file for publish to private repo (#3422)
* creating new npm rc file
* publishing to registry staging
* exposing base64 version of env variables
* changing encoding param
* fixing unit test for the new path
* debugging env var
* remove debug message
* update docu
* changing new npmrc file name
* adding new npmrc to ignore
* adding new npmrc to ignore
Co-authored-by: anilkeshav27 <you@example.com>
* minor
* small changes
* small changes
* UNIT Test
* Unit Test 2
* Update .gitignore
Co-authored-by: Daniel Mieg <56156797+DanielMieg@users.noreply.github.com>
* Update abapEnvironmentAddons.md: Unable to update target vectors with unchanged SWC version (#3392)
* Update abapEnvironmentAddons.md
* Update documentation/docs/scenarios/abapEnvironmentAddons.md
Co-authored-by: Daniel Mieg <56156797+DanielMieg@users.noreply.github.com>
* Update abapEnvironmentAddons.md
Co-authored-by: Daniel Mieg <56156797+DanielMieg@users.noreply.github.com>
* Returning earlier & parseOdataResp 1
* new Parameter for overwrite existing
* feat(cnbBuild): preserve maven test results in the workspace (#3429)
Co-authored-by: Pavel Busko <pavel.busko@sap.com>
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
* Update Introduction (#3433)
* Check Existence of configuration
* Quote strings in addon.yml (#3435)
* including vaultSecretFile to parameter generator docu (#3436)
Co-authored-by: anilkeshav27 <you@example.com>
* fix: wrong command line format for multiple extension files (#3434)
* feat(cnbBuild) containerImageName will be defaulted if possible (#3437)
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
Co-authored-by: Sumit Kulhadia <sumit.kulhadia@sap.com>
* Corr. check existence
* check existence 2
* existence handling 2
* refinement existence handling & param rename
* config yaml change - no mandatory for bool param
* PATCH for existing Configuration
* fix: correct typo in log message (#3439)
Closes#3438
* feat(protecode): add versioning model (#3373)
* changes to detectExec before master merge
* changes for detectExecuteScan
* self generated code added
* fix syntax errors and update docu
* added unit tests for fail and Group
* fix failOn bug
* add Groups as string array
* add Groups as string array
* tests and validation for groups, failOn
* Updated docs and added more tests
* documentation md files should not be changed
* Handle merge conflicts from PR 1845
* fix merge errors
* remove duplicate groups, merge error
* adding buildCode and buildTool as params
* switching build options
* building maven modules
* parameter correction
* parameter correction
* gnerate with new build parameter
* adding comments
* removing piper lib master and modifying goUtils to download 1.5.7 release
* first cleaning then installing
* multi module maven built
* multi module maven built removing unwanted code
* multi module maven built moving inside switch
* testing
* modifying the default use case to also call maven build
* modifying the default use case to also call maven build wih --
* corrected maven build command
* corrected maven build command with %v
* skipping test runs
* testing for MTA project with single pom
* adding absolute path to m2 path
* clean up
* adding switch for mta and maven and removing env from containers
* commiting changes for new detect step
* correting log message
* code clean up
* unit tests changes to detectExecute
* basic tests for new change
* restoring piperGoUtils to download correct piper binary
* code clean up
* code clean up
* protecodeExecuteScan :: versioning model draft - 1
* protecodeExecuteScan :: version model draft-2
* protecodeExecuteScan :: changing filename and version concatenation
* protecodeExecuteScan :: update documentation
* protecodeExecuteScan :: double URL encoding has been corrected & console messaging improved
* protecodeExecuteScan :: fixed Go/generate validation fail
* protecodeExecuteScan :: fixing failed unit tests
* protecodeExecuteScan :: Version field added
* protecodeExecuteScan :: Version field add => minor changes
* protecodeExecuteScan :: Version field add => fixing tests
Co-authored-by: D072410 <giridhar.shenoy@sap.com>
Co-authored-by: Keshav <anil.keshav@sap.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
* GCS Upload fixes (#3387)
* Fixed argument type in persist function
* Fixed gcp upload to be usable in internal piper
* Fixed import of packages
* Updated logs
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* Unit Test 2
* Unit Tests
* Add gcs upload to mavenExecuteIntegration step (#3432)
* Add gcs upload to mavenExecuteIntegration step
* go generate
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* Check times refine
* Permwrite 0644
* disable http retry (#3447)
* Update maxWaitTime timeout for WhiteSource (#3284)
* Update scanPolling.go
Changing maxWaitTime from 15 to 30 to overcome WhiteSource results reflection in the backend issue.
* Update configHelper.go
* Reset configHelper changes to fix PR 3284
Committer: raghunathd8
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: raghunathd8 <root@docker-evaluation.openstack.eu-nl-1.cloud.sap>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
* removed trailing spaces
* yaml Update
* minor mock for token & Unit Tests
* rename Parameter Patch
* lil refactor & Unit Test
* fixes PATCH & more Unit Tests
* add log at certain points
* not writing configuration file back
* feat(malwareExecuteScan): refactoring and docker support (#3421)
* feat(malwareExecuteScan): add support for scanning docker images
* refactoring
* print out finding if available
* generate toolrecord for malware scan
* persist scan report
* docs
* fix
* fix
* rollback cmd/init_unix.go
* auhenticated pull
* fix
* fix: report shall be consistent with the api model
* gcs upload
* fix linter
* Fix potential nil reference (#3460)
* Add gcs upload to karmaExecuteTests step (#3431)
* Add gcs upload to karmaExecuteTests step
* go generate
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* Add gcs upload to gaugeExecuteTests step (#3410)
* Add gcs upload to gaugeExecuteTests step
* go generate
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* Add gcs upload to newmanExecute step (#3408)
* Add gcs upload to newmanExecute step
* go generate
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* feat(GitHub): fetch commit statistics (#3381)
* fetch GH statistics
* move GH and Sonar integration tests to own files
* fix imports
* add integration test case
* add result type
* Apply suggestions from code review
* feat: add command to fetch default files (#3445)
* some ideas..
* Add getDefaults command (WIP) (#3444)
* add getYAML function for configs
* create getDefaults command(based on getConfig)
* add getDefaults command to CLI
* read defaults files, using github tokens as well
* write defaults to stdout as JSON object with YAMLs embedded
* catch case where no input files are given
* actually write output to file if outputFile is specified
* mark defaultsFile flag as required
* add basic tests
* add output (string) test
* adapt generateDefaults() to return output (used for test of previous commit)
* Changes to getDefaults() JSON output (#3449)
* change JSON output to contain separate fields
* filename -> filepath
* Apply suggestions from code review
* Apply suggestions from code review
* Update pkg/config/config_test.go
Co-authored-by: Jordi van Liempt <35920075+jliempt@users.noreply.github.com>
* BATCH request preparation
* Add gcs upload to mtaBuild step (#3405)
* Add gcs upload to mtaBuild
* Patterns was updated
* go generate
* removed patterns related to sapGenerateEnvironmentInfo step
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* Add gcs upload to mavenBuild step (#3394)
* Add gcs upload to mavenBuild step
* Patterns were updated
* go generate
* removed patterns related to sapGenerateEnvironmentInfo step
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* feat(gradleExecuteBuild): add stage scope to path parameter, fix project dir exist issue (#3401)
* add stage scope to path parameter, fix project dir exist issue
* fix unit test for gradleExecuteBuild
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* Add gcs upload to uiVeri5ExecuteTests step (#3409)
* Add gcs upload to uiVeri5ExecuteTests step
* go generate
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* Address further nil references (#3462)
* Address further nil references
* Message text
* Final checks
* fix (npmExecuteScripts) allow npm pack before npm publish (#3455)
* adding config to piperNpmr
* scope in cli
* adding scope to repo url and npmrc
* publish to scoped
* removing scope
* changing scope position
* adding scope to userconfig
* adding registry=
* pack and then tar
* not removing tmp folder
* adding flag
* pack before publish
* adding log
* debug
* debug with change directory
* publishing created tar ball
* debug
* üath
* adding main npmrc
* renaming old npmrc file
* error renaming old npmrc file
* renaming err
* correcting npmrc file path
* renaming file back to original
* current working directory
* renaming the npmrc file
* avoiding change directory
* with current working dir
* adding dot
* renaming npmrc and defer removal
* rename files
* Update pkg/npm/publish.go
* Update pkg/npm/publish.go
Co-authored-by: anilkeshav27 <you@example.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* Add testing for helm during acceptance stage (#3402)
* Add kubernetesDeploy to Acceptance
* Add more kubernetesDeploy
* Add helm tests
* Change documentation
* Fix docu
* Change generated
* Add tests
* Add groovy tests
* Fix tests
* Change tests
Co-authored-by: Thorsten Duda <thorsten.duda@sap.com>
* Batch Request for Patches
* Fix(gctsDeploy) : add client in config url, disable retry for create and pull (#3464)
* minor Unit Test correction
* lil cleanup
* lil refactoring
* removed nuneeded linebreaks
* refactoring Command on Entity in Batch
* some Unit Tests for Build Batch Request
* more Unittests - Build Batch Request
* fix(fortifyExecuteScan): User assignment based on PR ownership (#3472)
* Debug PR user details
* Check association
* Change to login
* Fix PR creator assignment
* Improve docs
* Fix test
* Update gradle version (#3476)
* Update gradle version
* Update generated file
* small correction Text & Test
* chore(kubernetesDeploy): add telemetry for deployTool (#3469)
* chore(kubernetesDeploy): add telemetry for deployTool
* Update cmd/kubernetesDeploy.go
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* rename Parameter patchIfExisting
* fix (mtaBuild) keep mtar artifact name in synch with maven gav (#3454)
* keep mtar name in synch with maven gav
* retaining slashes
* keeping dots in artifact name
Co-authored-by: anilkeshav27 <you@example.com>
* Update image for gradle build (#3479)
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* Last Changed not Zero check added
* Added pseudo_comment_policy in struct
* Unit Test adaption
* smaller refactoring, Unit Test, removed doubles
* fix(cnbBuild) do not add complete path to telemetry (#3487)
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>
* Add limitation that table keys are exported generically (#3490)
* Add limitation that table keys are exported generically
* Fix formatting
* Clarify delta calculation of different package types (#3482)
Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com>
* Add troubleshooting for missing add-on registration (#3491)
Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com>
* feat (Documentation) documentation for mavenBuild and npmExecuteScripts that consume dependencies from a private repo. (#3484)
* build depdency docu for maven and npm
* removing trailing white space in yaml
* Update resources/metadata/mavenBuild.yaml
* relative url for vault and mta docu
* running go generate
* keeping vault relative path
* go generate
* reverting to global paths
* go generate
* wild card for a higher level dir
* searching on top folder only
* relative level above
* pointing to infrastructure
* correcting links
Co-authored-by: anilkeshav27 <you@example.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* fix(kanikoExecute): tmp dir (#3478)
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* feat(golangBuild): add support for publishing binaries (#3495)
* fix(golangBuild): keep track of the artifactversion in binary names (#3498)
* fix(golangBuild): keep track of the artifactversion in binary names
* improve the naming scheme
* fix (build dependency) typo corrections (#3494)
* typo corrections
* trailing white space
* go generate
Co-authored-by: anilkeshav27 <you@example.com>
* Revert "fix(golangBuild): keep track of the artifactversion in binary names (#3498)" (#3499)
This reverts commit ec783b0da6.
* sonarqube coverage: additional metrics (#3465)
* sonarqube coverage: additional metrics
* sonarExecuteScan: add lines of code and language distribution to sonarscan.json
* sonarExecuteScan: consider branch in componentService requests
* SonarQube: Do not omit empty values in SonarCoverage
* sonarExecuteScan: Add integration tests for ComponentService getLinesOfCode
* fix tests
* sonarExecuteScan: use pullRequest in componentService
Co-authored-by: I550025 <r.kloe@sap.com>
Co-authored-by: Marc Bormeth <marc.bormeth@sap.com>
* fix(golangBuild): targetRepositoryUrl (#3502)
* fix(golangBuild): targetRepositoryUrl
* regen
* fix(httpclient): folder permissions (#3506)
* fix(golangBuild): use PUT instead of POST (#3504)
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* feat(golangBuild): publish to subfolder (#3503)
* feat(golangBuild): publish to subfolder
* edge case handling
* fix
* fix(golangBuild): expect status 201 instead of 200 (#3508)
* feat(kanikoExecute): allow building multiple images (#3443)
* feat(kanikoExecute): allow building multiple images
* enhance tests
* chore: allow running tests in parallel
* small fixes
* fix: fix destination bug
* update formatting and defaults
* fix yml formatting
* chore: change cpe parameter names
* chore: improve variable naming
* Changes GetJobName value to repositoryname (#3509)
* Update npmExecuteScripts step (#3211)
* Update npmExecuteScripts step
* Fixed failing build
* Fixed path issue
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Christian Volk <christian.volk@sap.com>
* Ignore violations if failOn is NONE (#3513)
* fix(sonarExecuteScan): error handling for report metrics (#3512)
* Generic step input from addon descriptor (#3467)
* enhance build step to run based on addon.yml in steampunk environment
Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com>
* feat(golangBuild): add BOM creation (#3496)
* Added BOM creation
* Fix cyclonedx install
* undo config.yml changes
* metadata was updated
* Update golangBuild.go
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* If statement dearrangement
Co-authored-by: Daniel Bernd <93763187+danManSAP@users.noreply.github.com>
Co-authored-by: Mayur Belur Mohan <68264167+mayurmohan@users.noreply.github.com>
Co-authored-by: Roland Stengel <r.stengel@sap.com>
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
Co-authored-by: Jordi van Liempt <35920075+jliempt@users.noreply.github.com>
Co-authored-by: Jesse Awan <jesse.awan@sap.com>
Co-authored-by: Pavel Busko <pavel.busko@sap.com>
Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>
Co-authored-by: Marc Bormeth <marc.bormeth@sap.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Anil Keshav <anil.keshav@sap.com>
Co-authored-by: anilkeshav27 <you@example.com>
Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com>
Co-authored-by: Daniel Mieg <56156797+DanielMieg@users.noreply.github.com>
Co-authored-by: Siarhei Pazdniakou <siarhei.pazdniakou@sap.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Peter Persiel <6087940+peterpersiel@users.noreply.github.com>
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
Co-authored-by: Marcus Holl <marcus.holl@sap.com>
Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
Co-authored-by: Sumit Kulhadia <sumit.kulhadia@sap.com>
Co-authored-by: Nikolay Grechanov <nikolay.grechanov@sap.com>
Co-authored-by: Umidjon Urunov <79094563+UmidjonUrunov@users.noreply.github.com>
Co-authored-by: D072410 <giridhar.shenoy@sap.com>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Sarat Krishnan <78093145+sarat-krk@users.noreply.github.com>
Co-authored-by: Raghunath Deshpande <raghunath.deshpande@sap.com>
Co-authored-by: raghunathd8 <root@docker-evaluation.openstack.eu-nl-1.cloud.sap>
Co-authored-by: Christian Volk <christian.volk@sap.com>
Co-authored-by: Eugene Kortelyov <eugene.kortelyov@sap.com>
Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>
Co-authored-by: Thorsten Duda <thorsten.duda@sap.com>
Co-authored-by: Matthias Scudlik <matthias.scudlik@gmail.com>
Co-authored-by: I550025 <r.kloe@sap.com>
Co-authored-by: ffeldmann <f.feldmann@sap.com>
Co-authored-by: ManjunathMS35 <manjunathms35@gmail.com>
Co-authored-by: Matthias Scudlik <matthias.scudlik@sap.com>
Co-authored-by: rosemarieB <45030247+rosemarieB@users.noreply.github.com>
* some ideas..
* Add getDefaults command (WIP) (#3444)
* add getYAML function for configs
* create getDefaults command(based on getConfig)
* add getDefaults command to CLI
* read defaults files, using github tokens as well
* write defaults to stdout as JSON object with YAMLs embedded
* catch case where no input files are given
* actually write output to file if outputFile is specified
* mark defaultsFile flag as required
* add basic tests
* add output (string) test
* adapt generateDefaults() to return output (used for test of previous commit)
* Changes to getDefaults() JSON output (#3449)
* change JSON output to contain separate fields
* filename -> filepath
* Apply suggestions from code review
* Apply suggestions from code review
* Update pkg/config/config_test.go
Co-authored-by: Jordi van Liempt <35920075+jliempt@users.noreply.github.com>
* Adds GetLog() function to orchestrator
* Fixes BUILD_NUMBER env variable
* Fixes correct env var for JENKINS_HOME
* Adds getEnv to read env variables with default value, adds test for jenkins GetLog() implementation
* Adds possibility to read errorJsons; updates splunk package for log files (WIP)
* Uncommenting dev code
* Adds GetLog() function to orchestrator
* Fixes BUILD_NUMBER env variable
* Fixes correct env var for JENKINS_HOME
* Adds getEnv to read env variables with default value, adds test for jenkins GetLog() implementation
* Adds possibility to read errorJsons; updates splunk package for log files (WIP)
* Uncommenting dev code
* Adds GetRequest function which holds the response in memory (not saved to disk)
* Implements GetLog() function for ADO, adds function to read PipelineRuntime
* PAT has been revoked
* Changes http package, s.t. if password only is required basic auth works too
* Adds env variable for azure token, error handling in case of unauthenticated/nil response
* Adds logging output in case env variable can not be read and fallback variable needs to be used
* Adds usage of environment variables for auth, uses jenkins api
* Adds init functionality for orchestrators, updates GetLog() and GetPipelineStartTime() function
* Adds initaliziation function for orchestrator authetnication
* Adds settings struct for orchestrator authentication
* Adds function to whole logfile to Splunk
* Struct for pipeline related telemetry information
* Increase messagebatch size to 10k
* Changes splunk package to a pointer based implementation, updates generated files and corresponding template and tests for splunk
* Changes telemetry package to pointer based implementation to have multiple telemetry objects, adjusted tests and splunk implementation
* Changes content type to txt
* Send telemetry independent of logfiles, increases amount of messages per file
* Adds JobURL for orchestrators and UnknownOrchestrator as fallback
* telemetry makes use of orchestrator specific information
* Adds orchestrator independent correlationID
* Adds custom fields for pipeline status
* go fmt
* Removes env var test - no env variables are read anymore
* Use UnknownOrchestratorConfigProvider in case the orchestrator can not be initalized
* Removes Custom fields from telemetry as these can not be reflected in SWA
* Adds custom telemetry information (piperHash,..) to each step telemetry information
* Removes falltrough in case no orchestrator has been found
* Updates tests for orchestrator package
* Adds orchestrator import in generated files
* Updates generator files for internal library
* Adds orchestrator telemetry information to steps
* Updates generated files, fatalHook writes to cpe
* Go generate from master, go fmt
* Adds Custom Data field LastErrorCode
* Removes GetLog() test
* Update init_unix.go
* Update docker_integration_test_executor.go
* Update integration_api_cli_test.go
* Reverts go1.17 fmt formatting
* Reverts go1.17 fmt formatting
* Reverts go1.17 fmt formatting
* Renames customTelemetryData to stepTelemetryData
* Adjustments to orchestrator-package, cleanup, adds JobName
* Adjusts commonPipelineEnvironment path
* Adds pipelineTelemetry struct to telemetry package, removes pipeline telemetry structs from splunk package
* Go fmt
* Changes path for errorDetails, adds debug information
* Removes custom fields from step, adds orchestrator, commithash to baseMetadata
* Adjusts tests for telemetry package
* Adds tests for orchestrator
* Updates generated files, initalization of splunk client only if its available in the config
* Fixes typo in helper go
* Update pkg/http/downloader.go
* Update pkg/http/downloader.go
* Update pkg/log/fatalHook.go
* Update fatalHook.go
* Update pkg/splunk/splunk.go
* Update pkg/telemetry/data.go
* Adds GetBuildStatus() and GetAPIInformation() to orchestrators
* error formatting
* Bugfix: dont send telemetry data if disabled, adjusts test
* go fmt
* Use correct error handling
* Update pkg/telemetry/telemetry.go
* Fixes telemetry disabled in the tests
* Fixes http tests
* Log fatal errors to logFile
* Adds CustomReportingConfig to hooks
* Cleanup comments in splunk package
* Adds possibility to send telemetry to custom endpoint
* Adds debug output for the payload
* Debug output for the payload as a string
* Adds test cases for changes in telemetry package
* go fmt
* Adds generated files for new step
* Reverts changes for http tests, causing problems with go1.15, changes need to be applied for newer go version >=1.17
* Adjusts test for sonarExecuteScan
* Adjusts test for sonarExecuteScan
* Adds explanation for customreportingConfig
* Makes disableing of customSend more obvious
* Adds custom step reporting to each step, updates generated files, adjusts helper testdata
* fixes unit test wrong usage of logging
* Send pipeline data altough there has been no error, adjust test cases
* Reverts changes for customReporting
* Updates generated files, removes customReporting
* Removes writing errorDetails to CPE
* Reverts usage of customreporting
* go fmt
* reverts changes in http_test
* reverts changes in http_test
* Skips integration cnb test
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* extend orchestator to provide stage name
* use orchestrator specific stage name
* fix test case
* remove comment
* fix test case
* prettify
* change something..
* do not exit
* Update pkg/orchestrator/azureDevOps.go
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* Implement cnbBuild step
Co-authored-by: Benjamin Haegenlaeuer <benjamin.haegenlaeuer@sap.com>
* Add cnbBuild groovy test
Co-authored-by: Benjamin Haegenlaeuer <benjamin.haegenlaeuer@sap.com>
* Add basic documentation template
Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>
* Support specifiying name, tag and registry
Co-authored-by: Pavel Busko <pbusko@users.noreply.github.com>
Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>
Co-authored-by: Pavel Busko <pbusko@users.noreply.github.com>
* Added checkIfStepActive step
* Implemented npmScripts condition. Code was refactored
* Added some unit tests
* Fixed go modules
* Fixed go modules
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* feat(config):read config/defaults with authentication
This change allows to use defaults and config files from a protected GitHub repository.
The options `--customConfig` and `--defaultConfig` already allowed to provide a link to an uprotected file.
Now, by passing a value in the form `<hostname>:<token>` to parameter `gitHubTokens` (this parameter can be passed multiple times) a token can be provided for dedicated hosts.
This makes it possible to use a link like
`https://api.github.com/repos/SAP/jenkins-library/contents/resources/my-defaults.yml?ref=master`
as reference to a default file or similarly as reference to a configuration file.
* update generation to allow protected config/defaults
* fix CodeClimate issues
* update missing generations
* new step integrationArtifactTriggerIntegrationTest
* add new step into allow list
* add the new step to main command
* refer cpe
* remove unused unit tests
* Check methods and URLs of http request
* Add TriggerIntegration to mockingutils
* Format code
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
Co-authored-by: Linda Siebert <linda.siebert@sap.com>
