* WIP: Adapt bom names
* + WIP: Adapt bom filenames
* Upgrade cyclonedx gradle plugin and use cyclonedxBom config parameters
* Fix unit tests - use correct name in bom creation
* Fix pythonBuild bom name
* introduce and use npmBomFilename const
* Introduce and use mvnBomFilename const
* Introduce and use gradleBomFilename const
* Use build-tool names for bom suffix
* + Adapt tests (build tool suffix)
* Use BOM schema version 1.2 in gradleExecuteBuild
* Pin version of cyclonedx-maven-plugin to 2.7.1
* Adapt generated files
* Fix integration tests
* Fix integration tests
* Fix gradle build integration tests
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* Cleanup of SBOM generation parameters
Adding `false` does not what is intended. If the parameters are added to the call, license texts and dev dependencies are included
* Fixed unit test
* fix(fortify): suppressed issues got "Unknown" category and state
* fix (fortify-sarif): classify findings into audit group
* fix(fortify-checkmarx-sarif): common properties bag for Fortify and Checkmarx (accepting the risk of empty value)
* fix (checkmarx-sarif): classify findings into audit group
* fix (sarif): formatting
* feat(cpe): provide go templating functions
* change type
* fix: type in test
* chore: add comment for exported function
* fix: ensure that custom returns string properly
* fix types and add tests
Co-authored-by: Anil Keshav <anil.keshav@sap.com>
* Update scanPolling.go
Changing maxWaitTime from 15 to 30 to overcome WhiteSource results reflection in the backend issue.
* Update configHelper.go
* Reset configHelper changes to fix PR 3284
Committer: raghunathd8
* ignoreSourceFiles to fileSystemScan
* Added ignoreSourceFiles param also
* minor adjustment
* minor adjustment again
* updated unit test
* tests fixed
* fmt-ed
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: raghunathd8 <root@docker-evaluation.openstack.eu-nl-1.cloud.sap>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: ffeldmann <f.feldmann@sap.com>
This commit replaces `ioutil.TempDir` with `t.TempDir` in tests. The
directory created by `t.TempDir` is automatically removed when the test
and all its subtests complete.
Prior to this commit, temporary directory created using `ioutil.TempDir`
needs to be removed manually by calling `os.RemoveAll`, which is omitted
in some tests. The error handling boilerplate e.g.
defer func() {
if err := os.RemoveAll(dir); err != nil {
t.Fatal(err)
}
}
is also tedious, but `t.TempDir` handles this for us nicely.
Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* enable build without values
* add sap-client as option
* use function from /net/url to add parameters
Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com>
* Add ans implementation
* Remove todo comment
* Rename test function
Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>
* Better wording
Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>
* Add reading of response body function
* Use http pkg ReadResponseBody
* Check read error
* Better test case description
* Fix formatting
* Create own package for read response body
* Omit empty nested resource struct
* Separate Resource struct from Event struct
* Merge and unmarshall instead of only unmarshalling
* Improve status code error message
* Remove unchangeable event fields
* Separate event parts
* Change log level setter function
* Restructure ans send test
* Revert exporting readResponseBody function
Instead the code is duplicated in the xsuaa and ans package
* Add check correct ans setup request
* Add set options function for mocking
* Review fixes
* Correct function name
* Use strict unmarshalling
* Validate event
* Move functions
* Add documentation comments
* improve test
* Validate event
* Add logrus hook for ans
* Set defaults on new hook creation
* Fix log level on error
* Don't alter entry log level
* Set severity fatal on 'fatal error' log message
* Ensure that log entries don't affect each other
* Remove unnecessary correlationID
* Use file path instead of event template string
* Improve warning messages
* Add empty log message check
* Allow configuration from file and string
* Add sourceEventId to tags
* Change resourceType to Pipeline
* Use structured config approach
* Use new log level set function
* Check correct setup and return error
* Mock http requests
* Only send log level warning or higher
* Use new function name
* One-liner ifs
* Improve test name
* Fix tests
* Prevent double firing
* Reduce Fire test size
* Add error message to test
* Reduce newANSHook test size
* Further check error
* Rename to defaultEvent in hook struct
* Reduce ifs further
* Fix set error category test
The ansHook Fire test cannot run in parallel, as it would affect the
other tests that use the error category.
* Change function name to SetServiceKey
* Validate event
* Rename to eventTemplate in hook struct
* Move copy to event.go
* Fix function mix
* Remove unnecessary cleanup
* Remove parallel test
The translation fails now and again when parallel is on.
* Remove prefix test
* Remove unused copyEvent function
* Fix ifs
* Add docu comment
* Register ans hook from pkg
* register hook and setup event template seperately
* Exclusively read eventTemplate from environment
* setupEventTemplate tests
* adjust hook levels test
* sync tests- wlill still fail
* migrate TestANSHook_registerANSHook test
* fixes
* Introduce necessary parameters
* Setup hook test
* Use file instead
* Adapt helper for ans
* Generate go files
* Add ans config to general config
* Change generator
* Regenerate steps
* Allow hook config from user config
Merges with hook config from defaults
* Remove ans flags from root command
* Get environment variables
* Generate files
* Add test when calling merge twice
* Update generator
* Regenerate steps
* Check two location for ans service key env var
* Re-generate
* Fix if
* Generate files with fix
* Duplicate config struct
* Add type casting test for ans config
* Fix helper
* Fix format
* Fix type casting of config
* Revert "Allow hook config from user config"
This reverts commit 4864499a4c497998c9ffc3e157ef491be955e68e.
* Revert "Add test when calling merge twice"
This reverts commit b82320fd07b82f5a597c5071049d918bcf62de00.
* Add ans config tests
* Improve helper code
* Re-generate commands
* Fix helper unit tests
* Change to only one argument
* Fix helper tests
* Re-generate
* Revert piper and config changes
* Re-generate missing step
* Generate new steps
* [ANS] Add servicekey credential to environment (#3684)
* Add ANS credential
* Switch to hooks and remove comments
* Add subsection for ans
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
* Remove changes to piper.go
* Remove formatting
* Add test for ANS
* Define hook credential seperately from step credential
* Add test for retrieval from general section
* Add comment
* Get ans hook info from DefaultValueCache
* [ANS] Add documentation (#3704)
* Add ANS credential
* Switch to hooks and remove comments
* Add subsection for ans
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
* Remove changes to piper.go
* Remove formatting
* Add test for ANS
* Define hook credential seperately from step credential
* Add test for retrieval from general section
* Add comment
* Add documentation
* Review changes
* Review comments
* Improve documentation further
* Add note of two event templates
* Add log level destinction
* Further improvements
* Improve text
* Remove unused things
* Add ANS credential
* Switch to hooks and remove comments
* Add subsection for ans
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
* Remove changes to piper.go
* Remove formatting
* Add test for ANS
* Define hook credential seperately from step credential
* Add test for retrieval from general section
* Add comment
* Get ans hook info from DefaultValueCache
* Improvements
Co-authored-by: Linda Siebert <linda.siebert@sap.com>
Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
* New lines
Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
Co-authored-by: Roland Stengel <r.stengel@sap.com>
Co-authored-by: Thorsten Duda <thorsten.duda@sap.com>
* Add ans implementation
* Remove todo comment
* Rename test function
Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>
* Better wording
Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>
* Add reading of response body function
* Use http pkg ReadResponseBody
* Check read error
* Better test case description
* Fix formatting
* Create own package for read response body
* Omit empty nested resource struct
* Separate Resource struct from Event struct
* Merge and unmarshall instead of only unmarshalling
* Improve status code error message
* Remove unchangeable event fields
* Separate event parts
* Change log level setter function
* Restructure ans send test
* Revert exporting readResponseBody function
Instead the code is duplicated in the xsuaa and ans package
* Add check correct ans setup request
* Add set options function for mocking
* Review fixes
* Correct function name
* Use strict unmarshalling
* Validate event
* Move functions
* Add documentation comments
* improve test
* Validate event
* Add logrus hook for ans
* Set defaults on new hook creation
* Fix log level on error
* Don't alter entry log level
* Set severity fatal on 'fatal error' log message
* Ensure that log entries don't affect each other
* Remove unnecessary correlationID
* Use file path instead of event template string
* Improve warning messages
* Add empty log message check
* Allow configuration from file and string
* Add sourceEventId to tags
* Change resourceType to Pipeline
* Use structured config approach
* Use new log level set function
* Check correct setup and return error
* Mock http requests
* Only send log level warning or higher
* Use new function name
* One-liner ifs
* Improve test name
* Fix tests
* Prevent double firing
* Reduce Fire test size
* Add error message to test
* Reduce newANSHook test size
* Further check error
* Rename to defaultEvent in hook struct
* Reduce ifs further
* Fix set error category test
The ansHook Fire test cannot run in parallel, as it would affect the
other tests that use the error category.
* Change function name to SetServiceKey
* Validate event
* Rename to eventTemplate in hook struct
* Move copy to event.go
* Fix function mix
* Remove unnecessary cleanup
* Remove parallel test
The translation fails now and again when parallel is on.
* Remove prefix test
* Remove unused copyEvent function
* Fix ifs
* Add docu comment
* Register ans hook from pkg
* register hook and setup event template seperately
* Exclusively read eventTemplate from environment
* setupEventTemplate tests
* adjust hook levels test
* sync tests- wlill still fail
* migrate TestANSHook_registerANSHook test
* fixes
* review - cleanup, reuse poke
* Apply suggestions from code review
* Change subject
* Review fixes
* Set stepName 'n/a' if not available
* Fix fire tests
Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>
Co-authored-by: Roland Stengel <r.stengel@sap.com>
hopefully that gives users a direct link back to the original fortify project+version
Co-authored-by: xgoffin <86716549+xgoffin@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* feat(fortifyExecuteScan): add a max number of retries for API calls in SARIF conversion
* feat(checkmarxExecuteScan): implement max number of retries on API call for descriptions in SARIF processing
* feat(checkmarx/fortify): extra logging line when failing an API request in SARIF conversion
* fix(fortifyExecuteScan): panic if undefined projectversion in sarif
* fix(fortifyExecuteScan): logging improvement
* fix(fortifyExecuteScan): wrong if condition caused crash
* fix(fortifyExecuteScan): do not log if retries hit -1, adjust logging
* fix(SARIF): commenting API calls for Checkmarx until a solution can be found for the API issues
* feat(SARIF): add omitempty to extensions
* Improvements were made
* fixed tests
* fixed issues
* fix versioning
* fix Inclusive Language warnings
* gradle support to fortifyExecuteScan. Classpath resolving
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* Add ans implementation
* Remove todo comment
* Rename test function
Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>
* Better wording
Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>
* Add reading of response body function
* Use http pkg ReadResponseBody
* Check read error
* Better test case description
* Fix formatting
* Create own package for read response body
* Omit empty nested resource struct
* Separate Resource struct from Event struct
* Merge and unmarshall instead of only unmarshalling
* Improve status code error message
* Remove unchangeable event fields
* Separate event parts
* Change log level setter function
* Restructure ans send test
* Revert exporting readResponseBody function
Instead the code is duplicated in the xsuaa and ans package
* Add check correct ans setup request
* Add set options function for mocking
* Review fixes
* Correct function name
* Use strict unmarshalling
* Validate event
* Move functions
* Add documentation comments
* improve test
Co-authored-by: Linda Siebert <39100394+LindaSieb@users.noreply.github.com>
Co-authored-by: Roland Stengel <r.stengel@sap.com>
* feat(fortfiyExecuteScan): proper XML unescaping, added rulepacks to SARIF, added kingdom/type/subtype to tags
* feat(fortifyExecuteScan): proper handling of severity, kinds, levels in SARIF
* fix(fortifyExecuteScan): edge case when handling properties taht could lead to a crash
* fix(fortifyExecuteScan): ensure SARIF processing is done after latest FPR is processed by SSC
* feat(checkmarxExecuteScan): respect SARIF standard more closely
* fix(checkmarxExecuteScan): edge case where message would be empty in SARIF
* fix(checkmarxExecuteScan): better message handling to ensure field is populated
* feat(checkmarxExecuteScan): SARIF file readability
* feat(checkmarxExecuteScan): include the helpURL as part of the Help object
* fix(sarif): remove wrong structure addition
* feat(checkmarxExecuteScan): safer handling of version in SARIF file
* feat(checkmarxExecuteScan): add CWE number to tags
* fix(helmExecute): respect version from Chart
using version from CPE can create failure situations in case format is not semver.
This is the case for maven artifacts, for example.
* chore: simplify condition
* chore: cleanup
* chore: cleanup
* explicitly adding tar extension to project name when constructing the targetFilePath for whitesource docker image download
* comments
* correcting comment for better readability
* replace spaces in the project name with underscroe
* better comments
* passing legacy format download
* appending format to value
* keeping the download format for protecode as legacy
* improving docu
* keeping legacy format the default
* keeping tar file name same as project name to avoid duplicate names
* keeping legacy format download hard coded
Co-authored-by: anilkeshav27 <you@example.com>
* feat(fpr_to_sarif & GHAS): adjustments to fit some rules
* feat(fortifyExecuteScan): fit GH ingestion rules better
* feat(fortifyExecuteScan): readability in SARIF report
* feat(fortifyExecuteScan): restore escaped chars in XML text
* feat(fortifyExecuteScan): properly replace threadflowlocations in each threadflow
* fix(fortifyExecuteScan): fixed missing threadflow in SARIF generation
* feat(fortifyExecuteScan): properly handle threadflows when a node has another node as Reason (node-in-node edge case)
* feat(fortifyExecuteScan): better sarif ruleID field
Co-authored-by: thtri <trinhthanhhai@gmail.com>
* including a artifact cpe type
* removing type kind related to PR 3717
* clean up
* eliminating local path
* go formatting fix
Co-authored-by: anilkeshav27 <you@example.com>
* fix(fortifyExecuteScan): check audit data length in all cases
* fix(fortifyExecuteScan): check audit data length in all cases
* feat(SARIF): logging improvements in debug mode
* fix(logging): readability
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
* Add small fix
* fix unit-tests
* Add deploymentName and packageVersion as flags
* small fix
* Change getting version of helm chart
* small fix
Co-authored-by: “Vitalii <“vitalii.sidorov@sap.com”>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* feat(gitopsUpdateDeployment) forcePush
fix(gitopsUpdateDeployment) include registry
The push operation in this step can be forced to bypass branch-protection
Signed-off-by: Michael Sprauer <Michael.Sprauer@sap.com>
* add unit test
Signed-off-by: Michael Sprauer <Michael.Sprauer@sap.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* feat(fortifyExecuteScan): query SSC once for batch audit data
* fix(fortifyExecuteScan): check audit data length in all cases
* feat(fortifyExecuteScan): in fpr_to_sarif, better detection of error cases, unit tests
* fix(log): comment useless error message
* fix(fortifyExecuteScan): clarify log message
* fix(fortifyExecuteScan): adapt unit tests
* Remove --backend-type
* Delete CTS in isChangeDevelopment and change Dockerimage of CM-Client
* fix groovy unit tests
* another fix of groovy unit tests
* try to fix import of fork for Jenkins-Testing
* add workflow to create Go Binary for Jenkins-Server
* Change RepoOwner to test in Fork
* remove previous changes
* adjust docker image for TransportRequestCreate and Release
* Remove CTS from Documentation
Co-authored-by: Thorsten Duda <thorsten.duda@sap.com>
* Add helm dependency command
* Change name of flag for package command
Co-authored-by: “Vitalii <“vitalii.sidorov@sap.com”>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* Reorders getApiInformation, changes variables to get start time, adjusts and adds test cases
* Changes the way to get apiInformation and reduces number of requests
* Changes getting pipeline start time from correct env variable
* Refactors getApiInformation functionality
* Adds GetBuildReason() for Azure and Jenkins
* Updates JobURL for ADO
* Implemented bom creation
* Made small fixes. Added integration tests
* go generate
* minor fixes
* fix tests
* Added unit tests
* minor fixes
* use fileutils
* integration tests optimization
* change integraton tests timeout to 25m
* Fix Inclusive Language warnings
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* Add runHelmCommand
* Add dryRun for debug
* Add default case in helmExecute
* Fix unit-tests
* small fix
* Fix RunHelmAdd and change RunHelmPublish methods
* Fix RunHelmPublish
* Fix unit-tests
* Fix unit-test
* small fix
* small fix
* small fix
* Add LintFlag PackageFlag PublishFlag flags
* Add tests for httpClient.go
* test
* test
* smal fix
* small fix
* Add getting name and version from Chart.yaml
* Add test
* Fix
* small fix
* Fix according to comments
* small fix
Co-authored-by: “Vitalii <“vitalii.sidorov@sap.com”>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Vitalii Sidorov <vitalii_sidorov@sap.com>
* fix(sarif): change format to fit omitempty cases better
* feat(fortifyExecuteScan): include category in sarif file
* fix(fortifyExecuteScan): access to undefined pointer in some cases
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
* (fix) match regexes in sliceContains to support vaultSecretNames
* add test for regex matching in sliceContains
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* Add bearer token retrieval function
Retrieving a bearer token from the xsuaa service on BTP is always the
same. With these functions one can retrieve a bearer token and set it
to the given header as 'Authorization'.
* CodeClimate fixes
* Refactor test
* Add basic auth to token retrieve request
Co-authored-by: Thorsten Duda <thorsten.duda@sap.com>
* Activates debug information for environment variables
* Adds tests for environment variable reading
* Reduces batch size to send messages to Splunk to 5000
* including negative conditions
* clean up and todos
* removing debug logging
* clean up
* fix unit test name
* fixing unit tests
* negative stage test
Co-authored-by: anilkeshav27 <you@example.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* Refactors logfile sending logic, renaming of fields, adds proper piper sourcetype
* Sets maximum retries to three and transport timeout to 10 seconds for azure and jenkins
* feat(checkmarx) : Checkmarx JSON Report
* Test cases with some fix
* Information total and audited test assertions
* feat(checkmarx): align total/audited with existing calculation
* fix(checkmarx): Reporting unit test
Co-authored-by: Sumeet PATIL <sumeet.patil@sap.com>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
this was already used in fortifyExecuteScan, but had no effect.
Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
* adds return in gcs upload in case error occurs e.g. no credentials, avoid nil pointer dereference
* Adds generated files
* Updates generated files
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* WIP
* New Logs
* Improving
* Determine log output based on available entities
* Increase width
* Add line
* Adapt TestPollEntity
* Format
* Fix query
* Adapt tests
* Fix test
* Improve formatting
* Retern early in case of no logs
* Remove duplicate log
* Implement helm step
* Create kubernetes package
* Refactoring helm.go
* Add package, test commands
* Add test for helm package
* Add tests for helm.go
* Add tests for helm.go
* Add tests for utils.go
* Add tests for helmExecute.go
* small fix
* Add helm lint
* small fix
* small fix
* Fix according to comments
* Fix test
* small fix
* Add helm add function
* Changes according to new comments
* Add helm push
* Add unit tests
* Add tests for helmExecute
* Add small fix
* small fix
* small fix
* Move DeployUtilsBundle from kubernetesDeploy to kubernetes package
* small fix
* small fix
* Add unit-tests
* Fix
* Update resources/metadata/helmExecute.yaml
* Update resources/metadata/helmExecute.yaml
* Add helm chart server parameterization
* small fix
* small fix
Co-authored-by: “Vitalii <“vitalii.sidorov@sap.com”>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* feat(FPRtoSARIF): boilerplate & comments
* Feat(Ingest): Build done, Vulnerabilities partway
* feat(Vulnerabilities): now entirely parsed
* feat(Ingestion): handle Description object
* feat(FprToSarif): integration in Piper step, full xml structure
* feat(fpr_to_sarif): base program. Need to replace names in messages
* feat(fpr_to_sarif): message substitution and custom definition integration
* fix(fpr_to_sarif): missing replacement in tools object
* fix(fortifyExecuteScan): unit tests
* fix(fpr_to_sarif): failing unit test
* Fix fortify folder creation for generating sarif
* deletion of unzip folder
* fix(fortifyExecuteScan): change logging to info
* feat(fpr_to_sarif): better unit test
* fix(fpr_to_sarif): pr tests failing
* feat(fpr_to_sarif): add specific properties to sarif
* feat(fpr_to_sarif): severity integration
* fix(fpr_to_sarif): unit test fixed
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
Co-authored-by: Sumeet PATIL <sumeet.patil@sap.com>
* enable detect 7 script
* unit test uses detect7 as default
* add detect6 test case
* add check for OSEnv detect version
* add test for OSEnv detect version
* update customEnvironmentVariables docu
* fix linting
Co-authored-by: ffeldmann <f.feldmann@sap.com>
* sonarqube coverage: additional metrics
* sonarExecuteScan: add lines of code and language distribution to sonarscan.json
* sonarExecuteScan: consider branch in componentService requests
* SonarQube: Do not omit empty values in SonarCoverage
* sonarExecuteScan: Add integration tests for ComponentService getLinesOfCode
* fix tests
* sonarExecuteScan: use pullRequest in componentService
Co-authored-by: I550025 <r.kloe@sap.com>
Co-authored-by: Marc Bormeth <marc.bormeth@sap.com>
* adding config to piperNpmr
* scope in cli
* adding scope to repo url and npmrc
* publish to scoped
* removing scope
* changing scope position
* adding scope to userconfig
* adding registry=
* pack and then tar
* not removing tmp folder
* adding flag
* pack before publish
* adding log
* debug
* debug with change directory
* publishing created tar ball
* debug
* üath
* adding main npmrc
* renaming old npmrc file
* error renaming old npmrc file
* renaming err
* correcting npmrc file path
* renaming file back to original
* current working directory
* renaming the npmrc file
* avoiding change directory
* with current working dir
* adding dot
* renaming npmrc and defer removal
* rename files
* Update pkg/npm/publish.go
* Update pkg/npm/publish.go
Co-authored-by: anilkeshav27 <you@example.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* add stage scope to path parameter, fix project dir exist issue
* fix unit test for gradleExecuteBuild
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* some ideas..
* Add getDefaults command (WIP) (#3444)
* add getYAML function for configs
* create getDefaults command(based on getConfig)
* add getDefaults command to CLI
* read defaults files, using github tokens as well
* write defaults to stdout as JSON object with YAMLs embedded
* catch case where no input files are given
* actually write output to file if outputFile is specified
* mark defaultsFile flag as required
* add basic tests
* add output (string) test
* adapt generateDefaults() to return output (used for test of previous commit)
* Changes to getDefaults() JSON output (#3449)
* change JSON output to contain separate fields
* filename -> filepath
* Apply suggestions from code review
* Apply suggestions from code review
* Update pkg/config/config_test.go
Co-authored-by: Jordi van Liempt <35920075+jliempt@users.noreply.github.com>
* fetch GH statistics
* move GH and Sonar integration tests to own files
* fix imports
* add integration test case
* add result type
* Apply suggestions from code review
* Fixed argument type in persist function
* Fixed gcp upload to be usable in internal piper
* Fixed import of packages
* Updated logs
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* changes to detectExec before master merge
* changes for detectExecuteScan
* self generated code added
* fix syntax errors and update docu
* added unit tests for fail and Group
* fix failOn bug
* add Groups as string array
* add Groups as string array
* tests and validation for groups, failOn
* Updated docs and added more tests
* documentation md files should not be changed
* Handle merge conflicts from PR 1845
* fix merge errors
* remove duplicate groups, merge error
* adding buildCode and buildTool as params
* switching build options
* building maven modules
* parameter correction
* parameter correction
* gnerate with new build parameter
* adding comments
* removing piper lib master and modifying goUtils to download 1.5.7 release
* first cleaning then installing
* multi module maven built
* multi module maven built removing unwanted code
* multi module maven built moving inside switch
* testing
* modifying the default use case to also call maven build
* modifying the default use case to also call maven build wih --
* corrected maven build command
* corrected maven build command with %v
* skipping test runs
* testing for MTA project with single pom
* adding absolute path to m2 path
* clean up
* adding switch for mta and maven and removing env from containers
* commiting changes for new detect step
* correting log message
* code clean up
* unit tests changes to detectExecute
* basic tests for new change
* restoring piperGoUtils to download correct piper binary
* code clean up
* code clean up
* protecodeExecuteScan :: versioning model draft - 1
* protecodeExecuteScan :: version model draft-2
* protecodeExecuteScan :: changing filename and version concatenation
* protecodeExecuteScan :: update documentation
* protecodeExecuteScan :: double URL encoding has been corrected & console messaging improved
* protecodeExecuteScan :: fixed Go/generate validation fail
* protecodeExecuteScan :: fixing failed unit tests
* protecodeExecuteScan :: Version field added
* protecodeExecuteScan :: Version field add => minor changes
* protecodeExecuteScan :: Version field add => fixing tests
Co-authored-by: D072410 <giridhar.shenoy@sap.com>
Co-authored-by: Keshav <anil.keshav@sap.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
Co-authored-by: Sven Merk <33895725+nevskrem@users.noreply.github.com>
* creating new npm rc file
* publishing to registry staging
* exposing base64 version of env variables
* changing encoding param
* fixing unit test for the new path
* debugging env var
* remove debug message
* update docu
* changing new npmrc file name
* adding new npmrc to ignore
* adding new npmrc to ignore
Co-authored-by: anilkeshav27 <you@example.com>
* fixed generated output parameters for influx
* change name to lower case
Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
* Add Changes for value of docker image
* Get docker image value
* Fix
* Fix unit
* Add chnages for kaniko and mta builds
* Fix
* Test changes
* Test
* Move func ResolveMetadata to stepmeta.go
* Fix
* Change getConfig.go
* Fix getting docker value for mta, npm and kaniko
* Fix according to suggestions
* Add func to get only value of docker image
* Test empty value of docker image
* Fix for getDockerImageValue
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* Removes unecessary fields from telemetry, restructuring splunk pkg
* Removes t.skip() and reactivates integration test
* Adjusts tests for fatalHook and helper functions, including log test
* Moves pipelinetelemetry to inner source, removes pipelineTelemetry from telemetry pkg, using generic map[string]interface for splunk
* Removes Read JSON from fatalHook -> moves to inner source
* Removed log output test
* go fmt
* log step telemetry data send to splunk
* Adjusts error logging
* Adds log information in case api information could not be retrieved
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* Adds GetLog() function to orchestrator
* Fixes BUILD_NUMBER env variable
* Fixes correct env var for JENKINS_HOME
* Adds getEnv to read env variables with default value, adds test for jenkins GetLog() implementation
* Adds possibility to read errorJsons; updates splunk package for log files (WIP)
* Uncommenting dev code
* Adds GetLog() function to orchestrator
* Fixes BUILD_NUMBER env variable
* Fixes correct env var for JENKINS_HOME
* Adds getEnv to read env variables with default value, adds test for jenkins GetLog() implementation
* Adds possibility to read errorJsons; updates splunk package for log files (WIP)
* Uncommenting dev code
* Adds GetRequest function which holds the response in memory (not saved to disk)
* Implements GetLog() function for ADO, adds function to read PipelineRuntime
* PAT has been revoked
* Changes http package, s.t. if password only is required basic auth works too
* Adds env variable for azure token, error handling in case of unauthenticated/nil response
* Adds logging output in case env variable can not be read and fallback variable needs to be used
* Adds usage of environment variables for auth, uses jenkins api
* Adds init functionality for orchestrators, updates GetLog() and GetPipelineStartTime() function
* Adds initaliziation function for orchestrator authetnication
* Adds settings struct for orchestrator authentication
* Adds function to whole logfile to Splunk
* Struct for pipeline related telemetry information
* Increase messagebatch size to 10k
* Changes splunk package to a pointer based implementation, updates generated files and corresponding template and tests for splunk
* Changes telemetry package to pointer based implementation to have multiple telemetry objects, adjusted tests and splunk implementation
* Changes content type to txt
* Send telemetry independent of logfiles, increases amount of messages per file
* Adds JobURL for orchestrators and UnknownOrchestrator as fallback
* telemetry makes use of orchestrator specific information
* Adds orchestrator independent correlationID
* Adds custom fields for pipeline status
* go fmt
* Removes env var test - no env variables are read anymore
* Use UnknownOrchestratorConfigProvider in case the orchestrator can not be initalized
* Removes Custom fields from telemetry as these can not be reflected in SWA
* Adds custom telemetry information (piperHash,..) to each step telemetry information
* Removes falltrough in case no orchestrator has been found
* Updates tests for orchestrator package
* Adds orchestrator import in generated files
* Updates generator files for internal library
* Adds orchestrator telemetry information to steps
* Updates generated files, fatalHook writes to cpe
* Go generate from master, go fmt
* Adds Custom Data field LastErrorCode
* Removes GetLog() test
* Update init_unix.go
* Update docker_integration_test_executor.go
* Update integration_api_cli_test.go
* Reverts go1.17 fmt formatting
* Reverts go1.17 fmt formatting
* Reverts go1.17 fmt formatting
* Renames customTelemetryData to stepTelemetryData
* Adjustments to orchestrator-package, cleanup, adds JobName
* Adjusts commonPipelineEnvironment path
* Adds pipelineTelemetry struct to telemetry package, removes pipeline telemetry structs from splunk package
* Go fmt
* Changes path for errorDetails, adds debug information
* Removes custom fields from step, adds orchestrator, commithash to baseMetadata
* Adjusts tests for telemetry package
* Adds tests for orchestrator
* Updates generated files, initalization of splunk client only if its available in the config
* Fixes typo in helper go
* Update pkg/http/downloader.go
* Update pkg/http/downloader.go
* Update pkg/log/fatalHook.go
* Update fatalHook.go
* Update pkg/splunk/splunk.go
* Update pkg/telemetry/data.go
* Adds GetBuildStatus() and GetAPIInformation() to orchestrators
* error formatting
* Bugfix: dont send telemetry data if disabled, adjusts test
* go fmt
* Use correct error handling
* Update pkg/telemetry/telemetry.go
* Fixes telemetry disabled in the tests
* Fixes http tests
* Log fatal errors to logFile
* Adds CustomReportingConfig to hooks
* Cleanup comments in splunk package
* Adds possibility to send telemetry to custom endpoint
* Adds debug output for the payload
* Debug output for the payload as a string
* Adds test cases for changes in telemetry package
* go fmt
* Adds generated files for new step
* Reverts changes for http tests, causing problems with go1.15, changes need to be applied for newer go version >=1.17
* Adjusts test for sonarExecuteScan
* Adjusts test for sonarExecuteScan
* Adds explanation for customreportingConfig
* Makes disableing of customSend more obvious
* Adds custom step reporting to each step, updates generated files, adjusts helper testdata
* fixes unit test wrong usage of logging
* Send pipeline data altough there has been no error, adjust test cases
* Reverts changes for customReporting
* Updates generated files, removes customReporting
* Removes writing errorDetails to CPE
* Reverts usage of customreporting
* go fmt
* reverts changes in http_test
* reverts changes in http_test
* Skips integration cnb test
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* feat: first parts of new run struct
* add parts for new stage condition handling
* update conditions
* feat: finalize conditions and tests
* feat(checkIfStepActive): support new CRD style conditions
* feat(docs): allow generating stage docs
* chore(docs): make step directory configurable
* fix: tests
* add option to output file
* Update checkIfStepActive_test.go
* create build settings for maven
* cases for when mavenBuild may be present
* fixing unit test for mavenBuild to include cpe
* changing position of buildSettngsJson to be called atfter build runs
* package
* extending the struct for other build types
* adding values for mta build settings
* changing config data type
* adding npm build settings
* unit tests
* fix trailing space
* typo correction in yaml
* Vitalii/build settings info (#3277)
* Add buildsettings package
* Improve buildSetting package for mta, npm
* Add unit-test
* Fix
* Fix
Co-authored-by: Vitalii Sidorov <vitalii.sidorov@sap.com>
* review changes
* removing buildTool param
* changing npm script name
* fix npmExecute tests
* including build settings info in npm struct
Co-authored-by: Your Name <you@example.com>
Co-authored-by: kingvvgo <56587879+kingvvgo@users.noreply.github.com>
Co-authored-by: Vitalii Sidorov <vitalii.sidorov@sap.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* Fixed validation for possibleValues option
* Change oneof-custom to possible-values
* go generate
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* new checks for commitIDs
* new checks for commitIDs
* relocate step from build stage to initial checks + refac
* log list
* fix log + check
* log format
* fix unit tests
Co-authored-by: Christian Luttenberger <42861202+bluesbrother84@users.noreply.github.com>
* empty cpe values from disk and cpe map on condition
* changing log level
* changing log level from info to edbug
* changing empty logic for empty string
* adding toBeEmptied condition
Co-authored-by: Your Name <you@example.com>
Co-authored-by: Oliver Nocon <33484802+OliverNocon@users.noreply.github.com>
* [WIP] bindings support for cnbBuild step
Co-authored-by: Pavel Busko <pavel.busko@sap.com>
* add unit tests
Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>
* switch to mapstruct for more meaningful errors
Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>
* add integration test for bindings
Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>
* Add documentation for cnbBuild.bindings
Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>
* fixed unit tests
Co-authored-by: Pavel Busko <pavel.busko@sap.com>
* apply codeclimate suggestions
Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>
* renamed field "secret" to "key"
Co-authored-by: Pavel Busko <pavel.busko@sap.com>
Co-authored-by: Pavel Busko <pavel.busko@sap.com>
* Adds retry for whitesource download in case of 404 not found
* Adds retry mechanism for 'forbidden errors' which occasional happen when downloading the unified agent
Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com>
