mirror of
https://github.com/SAP/jenkins-library.git
synced 2025-01-18 05:18:24 +02:00
Merge pull request #60 from CCFenner/publishTestResults
add new step testResultPublish
This commit is contained in:
Christopher Fenner
GitHub
commit
7b365aa27d
127
documentation/docs/steps/testsPublishResults.md
Normal file
127
documentation/docs/steps/testsPublishResults.md
Normal file
@ -0,0 +1,127 @@
|
||||
# testsPublishResults
|
||||
|
||||
## Description
|
||||
This step can publish test results from various sources.
|
||||
|
||||
## Prerequsites
|
||||
|
||||
* **test result files** - To use this step, there must be test result files available.
|
||||
* installed plugins:
|
||||
* [junit](https://plugins.jenkins.io/junit)
|
||||
* [jacoco](https://plugins.jenkins.io/jacoco)
|
||||
* [cobertura](https://plugins.jenkins.io/cobertura)
|
||||
* [performance](https://plugins.jenkins.io/performance)
|
||||
|
||||
## Pipeline configuration
|
||||
|
||||
none
|
||||
|
||||
## Explanation of pipeline step
|
||||
|
||||
Usage of pipeline step:
|
||||
|
||||
```groovy
|
||||
testsPublishResults(
|
||||
junit: [updateResults: true, archive: true],
|
||||
jacoco: [archive: true]
|
||||
)
|
||||
```
|
||||
|
||||
Available parameters:
|
||||
|
||||
| parameter | mandatory | default | possible values |
|
||||
| ----------|-----------|---------|-----------------|
|
||||
| junit | no | `false` | true, false |
|
||||
| jacoco | no | `false` | true, false |
|
||||
| cobertura | no | `false` | true, false |
|
||||
| jmeter | no | `false` | true, false |
|
||||
|
||||
* `junit` - Publishes test results files in JUnit format with the [JUnit Plugin](https://plugins.jenkins.io/junit).
|
||||
* `jacoco` - Publishes code coverage with the [JaCoCo plugin](https://plugins.jenkins.io/jacoco) .
|
||||
* `cobertura` - Publishes code coverage with the [Cobertura plugin](https://plugins.jenkins.io/cobertura).
|
||||
* `jmeter` - Publishes performance test results with the [Performance plugin](https://plugins.jenkins.io/performance).
|
||||
|
||||
Each of the parameters `junit`, `jacoco`, `cobertura` and `jmeter` can be set to `true` or `false` but also to a map of parameters to hand in different settings for the tools.
|
||||
|
||||
**junit**
|
||||
|
||||
| parameter | mandatory | default | possible values |
|
||||
| ----------|-----------|---------|-----------------|
|
||||
| pattern | no | `'**/target/surefire-reports/*.xml'` | |
|
||||
| archive | no | `false` | true, false |
|
||||
| updateResults | no | `false` | true, false |
|
||||
| allowEmptyResults | no | `true` | true, false |
|
||||
|
||||
**jacoco**
|
||||
|
||||
| parameter | mandatory | default | possible values |
|
||||
| ----------|-----------|---------|-----------------|
|
||||
| pattern | no | `'**/target/*.exec'` | |
|
||||
| include | no | `''` | `'**/*.class'` |
|
||||
| exclude | no | `''` | `'**/Test*'` |
|
||||
| archive | no | `false` | true, false |
|
||||
| allowEmptyResults | no | `true` | true, false |
|
||||
|
||||
**cobertura**
|
||||
|
||||
| parameter | mandatory | default | possible values |
|
||||
| ----------|-----------|---------|-----------------|
|
||||
| pattern | no | `'**/target/coverage/cobertura-coverage.xml'` | |
|
||||
| archive | no | `false` | true, false |
|
||||
| allowEmptyResults | no | `true` | true, false |
|
||||
| onlyStableBuilds | no | `true` | true, false |
|
||||
|
||||
**jmeter**
|
||||
|
||||
| parameter | mandatory | default | possible values |
|
||||
| ----------|-----------|---------|-----------------|
|
||||
| pattern | no | `'**/*.jtl'` | |
|
||||
| errorFailedThreshold | no | `20` | |
|
||||
| errorUnstableThreshold | no | `10` | |
|
||||
| errorUnstableResponseTimeThreshold | no | `` | |
|
||||
| relativeFailedThresholdPositive | no | `0` | |
|
||||
| relativeFailedThresholdNegative | no | `0` | |
|
||||
| relativeUnstableThresholdPositive | no | `0` | |
|
||||
| relativeUnstableThresholdNegative | no | `0` | |
|
||||
| modeOfThreshold | no | `false` | true, false |
|
||||
| modeThroughput | no | `false` | true, false |
|
||||
| nthBuildNumber | no | `0` | |
|
||||
| configType | no | `PRT` | |
|
||||
| failBuildIfNoResultFile | no | `false` | true, false |
|
||||
| compareBuildPrevious | no | `true` | true, false |
|
||||
| archive | no | `false` | true, false |
|
||||
| allowEmptyResults | no | `true` | true, false |
|
||||
|
||||
## Step configuration
|
||||
Following parameters can also be specified as step parameters using the global configuration file:
|
||||
|
||||
* `junit`
|
||||
* `jacoco`
|
||||
* `cobertura`
|
||||
* `jmeter`
|
||||
|
||||
## Return value
|
||||
none
|
||||
|
||||
## Side effects
|
||||
none
|
||||
|
||||
## Exceptions
|
||||
none
|
||||
|
||||
## Example
|
||||
```groovy
|
||||
// publish test results with coverage
|
||||
testsPublishResults(
|
||||
junit: [updateResults: true, archive: true],
|
||||
jacoco: [archive: true]
|
||||
)
|
||||
```
|
||||
|
||||
```groovy
|
||||
// publish test results with coverage
|
||||
testsPublishResults(
|
||||
junit: [pattern: '**/target/TEST*.xml', archive: true],
|
||||
cobertura: [pattern: '**/target/coverage/cobertura-coverage.xml']
|
||||
)
|
||||
```
|
||||
@ -84,3 +84,39 @@ steps:
|
||||
fail:
|
||||
high: '0'
|
||||
archive: false
|
||||
testsPublishResults:
|
||||
junit:
|
||||
pattern: '**/target/surefire-reports/*.xml'
|
||||
updateResults: false
|
||||
allowEmptyResults: true
|
||||
archive: false
|
||||
active: false
|
||||
jacoco:
|
||||
pattern: '**/target/*.exec'
|
||||
allowEmptyResults: true
|
||||
archive: false
|
||||
active: false
|
||||
cobertura:
|
||||
pattern: '**/target/coverage/cobertura-coverage.xml'
|
||||
onlyStableBuilds: true
|
||||
allowEmptyResults: true
|
||||
archive: false
|
||||
active: false
|
||||
jmeter:
|
||||
pattern: '**/*.jtl'
|
||||
errorFailedThreshold: 20
|
||||
errorUnstableThreshold: 10
|
||||
errorUnstableResponseTimeThreshold: ''
|
||||
relativeFailedThresholdPositive: 0
|
||||
relativeFailedThresholdNegative: 0
|
||||
relativeUnstableThresholdPositive: 0
|
||||
relativeUnstableThresholdNegative: 0
|
||||
modeOfThreshold: false
|
||||
modeThroughput: false
|
||||
nthBuildNumber: 0
|
||||
configType: 'PRT'
|
||||
failBuildIfNoResultFile: false
|
||||
compareBuildPrevious: true
|
||||
allowEmptyResults: true
|
||||
archive: false
|
||||
active: false
|
||||
|
||||
126
test/groovy/TestsPublishResultsTest.groovy
Normal file
126
test/groovy/TestsPublishResultsTest.groovy
Normal file
@ -0,0 +1,126 @@
|
||||
import org.junit.Before
|
||||
import org.junit.Ignore
|
||||
import org.junit.Rule
|
||||
import org.junit.Test
|
||||
import org.junit.rules.RuleChain
|
||||
|
||||
import com.lesfurets.jenkins.unit.BasePipelineTest
|
||||
|
||||
import static org.junit.Assert.assertEquals
|
||||
import static org.junit.Assert.assertTrue
|
||||
|
||||
import util.Rules
|
||||
|
||||
class TestsPublishResultsTest extends BasePipelineTest {
|
||||
Map publisherStepOptions
|
||||
List archiveStepPatterns
|
||||
|
||||
@Rule
|
||||
public RuleChain ruleChain = RuleChain.outerRule(Rules.getCommonRules(this))
|
||||
|
||||
def testsPublishResultsScript
|
||||
|
||||
@Before
|
||||
void init() {
|
||||
publisherStepOptions = [:]
|
||||
archiveStepPatterns = []
|
||||
// prepare checkResultsPublish step
|
||||
testsPublishResultsScript = loadScript('testsPublishResults.groovy').testsPublishResults
|
||||
helper.registerAllowedMethod('junit', [Map.class], {
|
||||
parameters -> publisherStepOptions['junit'] = parameters
|
||||
})
|
||||
helper.registerAllowedMethod('jacoco', [Map.class], {
|
||||
parameters -> publisherStepOptions['jacoco'] = parameters
|
||||
})
|
||||
helper.registerAllowedMethod('cobertura', [Map.class], {
|
||||
parameters -> publisherStepOptions['cobertura'] = parameters
|
||||
})
|
||||
helper.registerAllowedMethod('perfReport', [Map.class], {
|
||||
parameters -> publisherStepOptions['jmeter'] = parameters
|
||||
})
|
||||
helper.registerAllowedMethod('archiveArtifacts', [Map.class], {
|
||||
parameters -> archiveStepPatterns.push(parameters.artifacts)
|
||||
})
|
||||
}
|
||||
|
||||
@Test
|
||||
void testPublishNothingWithDefaultSettings() throws Exception {
|
||||
testsPublishResultsScript.call()
|
||||
|
||||
// ensure nothing is published
|
||||
assertTrue('WarningsPublisher options not empty', publisherStepOptions.junit == null)
|
||||
assertTrue('PmdPublisher options not empty', publisherStepOptions.jacoco == null)
|
||||
assertTrue('DryPublisher options not empty', publisherStepOptions.cobertura == null)
|
||||
assertTrue('FindBugsPublisher options not empty', publisherStepOptions.jmeter == null)
|
||||
}
|
||||
|
||||
@Test
|
||||
void testPublishNothingWithAllDisabled() throws Exception {
|
||||
testsPublishResultsScript.call(junit: false, jacoco: false, cobertura: false, jmeter: false)
|
||||
|
||||
// ensure nothing is published
|
||||
assertTrue('WarningsPublisher options not empty', publisherStepOptions.junit == null)
|
||||
assertTrue('PmdPublisher options not empty', publisherStepOptions.jacoco == null)
|
||||
assertTrue('DryPublisher options not empty', publisherStepOptions.cobertura == null)
|
||||
assertTrue('FindBugsPublisher options not empty', publisherStepOptions.jmeter == null)
|
||||
}
|
||||
|
||||
@Test
|
||||
void testPublishUnitTestsWithDefaultSettings() throws Exception {
|
||||
testsPublishResultsScript.call(junit: true)
|
||||
|
||||
assertTrue('JUnit options are empty', publisherStepOptions.junit != null)
|
||||
// ensure default patterns are set
|
||||
assertEquals('JUnit default pattern not set correct',
|
||||
'**/target/surefire-reports/*.xml', publisherStepOptions.junit.testResults)
|
||||
// ensure nothing else is published
|
||||
assertTrue('JaCoCo options are not empty', publisherStepOptions.jacoco == null)
|
||||
assertTrue('Cobertura options are not empty', publisherStepOptions.cobertura == null)
|
||||
assertTrue('JMeter options are not empty', publisherStepOptions.jmeter == null)
|
||||
}
|
||||
|
||||
@Test
|
||||
void testPublishCoverageWithDefaultSettings() throws Exception {
|
||||
testsPublishResultsScript.call(jacoco: true, cobertura: true)
|
||||
|
||||
assertTrue('JaCoCo options are empty', publisherStepOptions.jacoco != null)
|
||||
assertTrue('Cobertura options are empty', publisherStepOptions.cobertura != null)
|
||||
assertEquals('JaCoCo default pattern not set correct',
|
||||
'**/target/*.exec', publisherStepOptions.jacoco.execPattern)
|
||||
assertEquals('Cobertura default pattern not set correct',
|
||||
'**/target/coverage/cobertura-coverage.xml', publisherStepOptions.cobertura.coberturaReportFile)
|
||||
// ensure nothing else is published
|
||||
assertTrue('JUnit options are not empty', publisherStepOptions.junit == null)
|
||||
assertTrue('JMeter options are not empty', publisherStepOptions.jmeter == null)
|
||||
}
|
||||
|
||||
@Test
|
||||
void testPublishJMeterWithDefaultSettings() throws Exception {
|
||||
testsPublishResultsScript.call(jmeter: true)
|
||||
|
||||
assertTrue('JMeter options are empty', publisherStepOptions.jmeter != null)
|
||||
assertEquals('JMeter default pattern not set',
|
||||
'**/*.jtl', publisherStepOptions.jmeter.sourceDataFiles)
|
||||
|
||||
// ensure nothing else is published
|
||||
assertTrue('JUnit options are not empty', publisherStepOptions.junit == null)
|
||||
assertTrue('JaCoCo options are not empty', publisherStepOptions.jacoco == null)
|
||||
assertTrue('Cobertura options are not empty', publisherStepOptions.cobertura == null)
|
||||
}
|
||||
|
||||
@Test
|
||||
void testPublishUnitTestsWithCustomSettings() throws Exception {
|
||||
testsPublishResultsScript.call(junit: [pattern: 'fancy/file/path', archive: true, active: true])
|
||||
|
||||
assertTrue('JUnit options are empty', publisherStepOptions.junit != null)
|
||||
// ensure default patterns are set
|
||||
assertEquals('JUnit pattern not set correct',
|
||||
'fancy/file/path', publisherStepOptions.junit.testResults)
|
||||
assertEquals('JUnit default pattern not set correct',
|
||||
'fancy/file/path', publisherStepOptions.junit.testResults)
|
||||
// ensure nothing else is published
|
||||
assertTrue('JaCoCo options are not empty', publisherStepOptions.jacoco == null)
|
||||
assertTrue('Cobertura options are not empty', publisherStepOptions.cobertura == null)
|
||||
assertTrue('JMeter options are not empty', publisherStepOptions.jmeter == null)
|
||||
}
|
||||
}
|
||||
157
vars/testsPublishResults.groovy
Normal file
157
vars/testsPublishResults.groovy
Normal file
@ -0,0 +1,157 @@
|
||||
import com.cloudbees.groovy.cps.NonCPS
|
||||
|
||||
import com.sap.piper.ConfigurationHelper
|
||||
import com.sap.piper.ConfigurationMerger
|
||||
import com.sap.piper.MapUtils
|
||||
|
||||
import groovy.transform.Field
|
||||
|
||||
@Field List TOOLS = [
|
||||
'junit','jacoco','cobertura','jmeter'
|
||||
]
|
||||
|
||||
@Field def STEP_NAME = 'testsPublishResults'
|
||||
@Field Set STEP_CONFIG_KEYS = TOOLS
|
||||
@Field Set PARAMETER_KEYS = STEP_CONFIG_KEYS
|
||||
|
||||
/**
|
||||
* testResultsPublish
|
||||
*
|
||||
* @param script global script environment of the Jenkinsfile run
|
||||
* @param others document all parameters
|
||||
*/
|
||||
def call(Map parameters = [:]) {
|
||||
handlePipelineStepErrors (stepName: STEP_NAME, stepParameters: parameters) {
|
||||
def script = parameters.script
|
||||
if (script == null)
|
||||
script = [commonPipelineEnvironment: commonPipelineEnvironment]
|
||||
prepare(parameters)
|
||||
|
||||
// load default & individual configuration
|
||||
Map configuration = ConfigurationHelper
|
||||
.loadStepDefaults(this)
|
||||
.mixinStepConfig(script.commonPipelineEnvironment, STEP_CONFIG_KEYS)
|
||||
.mixin(parameters, PARAMETER_KEYS)
|
||||
.use()
|
||||
|
||||
// UNIT TESTS
|
||||
publishJUnitReport(configuration.get('junit'))
|
||||
// CODE COVERAGE
|
||||
publishJacocoReport(configuration.get('jacoco'))
|
||||
publishCoberturaReport(configuration.get('cobertura'))
|
||||
// PERFORMANCE
|
||||
publishJMeterReport(configuration.get('jmeter'))
|
||||
}
|
||||
}
|
||||
|
||||
def publishJUnitReport(Map settings = [:]) {
|
||||
if(settings.active){
|
||||
def pattern = settings.get('pattern')
|
||||
def allowEmpty = settings.get('allowEmptyResults')
|
||||
|
||||
if (settings.get('updateResults'))
|
||||
touchFiles(pattern)
|
||||
junit(
|
||||
testResults: pattern,
|
||||
allowEmptyResults: allowEmpty,
|
||||
healthScaleFactor: 100.0,
|
||||
)
|
||||
archiveResults(settings.get('archive'), pattern, allowEmpty)
|
||||
}
|
||||
}
|
||||
|
||||
def publishJacocoReport(Map settings = [:]) {
|
||||
if(settings.active){
|
||||
def pattern = settings.get('pattern')
|
||||
def allowEmpty = settings.get('allowEmptyResults')
|
||||
|
||||
jacoco(
|
||||
execPattern: pattern,
|
||||
inclusionPattern: settings.get('include', ''),
|
||||
exclusionPattern: settings.get('exclude', '')
|
||||
)
|
||||
archiveResults(settings.get('archive'), pattern, allowEmpty)
|
||||
}
|
||||
}
|
||||
|
||||
def publishCoberturaReport(Map settings = [:]) {
|
||||
if(settings.active){
|
||||
def pattern = settings.get('pattern')
|
||||
def allowEmpty = settings.get('allowEmptyResults')
|
||||
|
||||
cobertura(
|
||||
coberturaReportFile: pattern,
|
||||
onlyStable: settings.get('onlyStableBuilds'),
|
||||
failNoReports: !allowEmpty,
|
||||
failUnstable: false,
|
||||
failUnhealthy: false,
|
||||
autoUpdateHealth: false,
|
||||
autoUpdateStability: false,
|
||||
maxNumberOfBuilds: 0
|
||||
)
|
||||
archiveResults(settings.get('archive'), pattern, allowEmpty)
|
||||
}
|
||||
}
|
||||
|
||||
// publish Performance Report using "Jenkins Performance Plugin" https://wiki.jenkins.io/display/JENKINS/Performance+Plugin
|
||||
def publishJMeterReport(Map settings = [:]){
|
||||
if(settings.active){
|
||||
def pattern = settings.get('pattern')
|
||||
|
||||
perfReport(
|
||||
sourceDataFiles: pattern,
|
||||
errorFailedThreshold: settings.get('errorFailedThreshold'),
|
||||
errorUnstableThreshold: settings.get('errorUnstableThreshold'),
|
||||
errorUnstableResponseTimeThreshold: settings.get('errorUnstableResponseTimeThreshold'),
|
||||
relativeFailedThresholdPositive: settings.get('relativeFailedThresholdPositive'),
|
||||
relativeFailedThresholdNegative: settings.get('relativeFailedThresholdNegative'),
|
||||
relativeUnstableThresholdPositive: settings.get('relativeUnstableThresholdPositive'),
|
||||
relativeUnstableThresholdNegative: settings.get('relativeUnstableThresholdNegative'),
|
||||
modePerformancePerTestCase: false,
|
||||
modeOfThreshold: settings.get('modeOfThreshold'),
|
||||
modeThroughput: settings.get('modeThroughput'),
|
||||
nthBuildNumber: settings.get('nthBuildNumber'),
|
||||
configType: settings.get('configType'),
|
||||
failBuildIfNoResultFile: settings.get('failBuildIfNoResultFile'),
|
||||
compareBuildPrevious: settings.get('compareBuildPrevious')
|
||||
)
|
||||
archiveResults(settings.get('archive'), pattern, settings.get('allowEmptyResults'))
|
||||
}
|
||||
}
|
||||
|
||||
def touchFiles(){
|
||||
echo "[${STEP_NAME}] update test results"
|
||||
def patternArray = pattern.split(',')
|
||||
for(def i = 0; i < patternArray.length; i++){
|
||||
sh "find . -wholename '${patternArray[i].trim()}' -exec touch {} \\;"
|
||||
}
|
||||
}
|
||||
|
||||
def archiveResults(archive, pattern, allowEmpty) {
|
||||
if(archive){
|
||||
echo "[${STEP_NAME}] archive ${pattern}"
|
||||
archiveArtifacts artifacts: pattern, allowEmptyArchive: allowEmpty
|
||||
}
|
||||
}
|
||||
|
||||
@NonCPS
|
||||
def prepare(parameters){
|
||||
// ensure tool maps are initialized correctly
|
||||
for(String tool : TOOLS){
|
||||
parameters[tool] = toMap(parameters[tool])
|
||||
}
|
||||
return parameters
|
||||
}
|
||||
|
||||
@NonCPS
|
||||
def toMap(parameters){
|
||||
if(MapUtils.isMap(parameters))
|
||||
parameters.put('active', parameters.active == null?true:parameters.active)
|
||||
else if(Boolean.TRUE.equals(parameters))
|
||||
parameters = [active: true]
|
||||
else if(Boolean.FALSE.equals(parameters))
|
||||
parameters = [active: false]
|
||||
else
|
||||
parameters = [:]
|
||||
return parameters
|
||||
}
|
||||
Loading…
Reference in New Issue
Block a user