Merge pull request #1031 from rodibrin/J365

Custom Jenkins Setup

documentation/docs/customjenkins.md

@@ -0,0 +1,116 @@
+# Custom Jenkins Setup
+
+Although we recommend using the Cx Server, you can also run project "Piper" on your own Jenkins installation. In this case, however, you have to care for some settings the Cx Server gives you for free. Also, the support of non-Cx Server installations is challenging.
+
+This section describes the adjustments that might be necessary.
+
+## Requirements
+
+* You have installed Java Runtime Environment 8.
+* You have installed Jenkins v 2.60.3 or higher running on Linux. We've tested with debian-stretch.
+* A Jenkins user with administration rights.
+* Your Jenkins instance has access to [github.com][github].
+
+## Docker
+
+Most of the tools project "Piper" uses to build, test, and deploy your application are available as out-of-the-box Docker images. You don't need to manually install them on your Jenkins server or Jenkins nodes, nor care for updates. Instead, these are automatically pulled from *hub.docker.com*.
+
+Install Docker if you haven't installed it, yet. To install the newest version of Docker, see [Docker Community Edition][docker-install].
+**Note:** We've tested on Docker 18.09.6.
+
+If your Jenkins server already runs as Docker container, make sure the tools container can run on the Docker host. Extend the Docker call in the following way:
+
+```
+docker run ...  -v /var/run/docker.sock:/var/run/docker.sock ...
+```
+
+## Plugins
+
+Project "Piper" requires a set of plugins installed on your Jenkins server. This set may evolve in the future. Make sure that all plugins of the appropriate versions are installed.
+
+The Cx server repository contains an [up-to-date list][devops-cxs-plugins] of the required plugins. To ease the installation, download the list with the following command and use the [Jenkins client][jenkins-doc-client]:
+
+```
+curl -o plugins.txt https://raw.githubusercontent.com/SAP/devops-docker-cx-server/master/jenkins-master/plugins.txt
+```
+
+On the Jenkins server, run the following command with a user that has administration rights:
+
+```
+cat plugins.txt | awk '{system("java " "-jar jenkins-cli.jar -s http://localhost:8080 -auth ${ADM_USER}:${ADM_PASSWD} install-plugin " $1)}'
+```
+
+## Shared Library
+
+Shared libraries extending the Jenkins pipeline are defined within the Jenkins system configuration. A library is defined by a link to its source repository and an appropriate version identifier. To add the project "Piper"s library, execute the following steps:
+
+1. Open the Jenkins UI under `http://<jenkins-server-address>:<http-port>`, login with administration privileges, and choose **Manage Jenkins > Configure System**.
+   <p align="center">
+   ![Configure System](images/JenkinsHomeMenuManageConfig.png "Configure System")
+   </p>
+
+1. Scroll down to section **Global Pipeline Libraries** and choose the **Add** button. A new library is created.
+   <p align="center">
+   ![Add Library](images/JenkinsConfigSystemLibrary-Add.png "Add Library")
+   </p>
+
+1. For **Library Name**, enter `piper-lib-os`.
+
+1. For **Default Version**, enter the branch or tag you want to consume (e.g. `master` or `v0.1`).
+
+1. For **Retrieval Method**, choose **Modern SCM**.
+
+1. For **Source Code Management**, choose **Git**.
+
+1. For **Project Repository**, enter the GitHub URL of the project Piper shared library `https://github.com/SAP/jenkins-library`.
+   <p align="center">
+   ![Library Setup](images/JenkinsConfigSystemLibrary-Edit.png "Library Setup")
+   </p>
+
+1. Save your changes.
+
+**Result:** The library is available as `piper-lib-os`. To use it by any pipeline, add the following line to its `Jenkinsfile`:
+
+```groovy
+@Library('piper-lib-os') _
+```
+
+When the pipeline is launched, Jenkins downloads the corresponding library as source and compiles it before the pipeline is processed.
+
+## User Permission Issue
+
+Your native Jenkins installation defines the user `jenkins` as a service user. If it doesn't exist, it is created. In this case, the user ID is the next free number determined by `/etc/passwd` - probably starting from `100`.
+In contrast, the official [Jenkins Docker image][jenkins-docker-image] defines the user `jenkins` with the user ID `1000` as a service user inside the container.
+So, the service user ID of your native Jenkins server most likely differs from the user ID of the official Jenkins Docker image.
+
+This could have impacts.
+
+Project "Piper" runs many pipeline steps as Docker images. If a Docker container is created, the Jenkins Docker plugin passes the Jenkins user and group ID as a process owner into the Docker container.
+Binding a folder from the host machine into the container - used to exchange files between steps - results in file permission issues, if the user inside the container doesn't have rights for the folder on the host machine or vice versa.
+
+Although you won't face this issue with images of project "Piper" , some 3rd-party Docker images follow this convention and expect to be executed under userid `1000`, like [node.js][dockerhub-node], which is used by a set of additional steps.
+
+If you face such a [user permission issue][piper-issue-781], choose between the following options:
+
+- Change the ID of your Jenkins service user to `1000`.
+
+- [Create your own images][docker-getstarted] and solve the permission issues by removing the file system restrictions. Adjust the configuration accordingly, for example, adjust the `npmExecute` step of your project's YAML:
+
+```
+  npmExecute:
+    dockerImage: 'my-node:8-stretch'
+```
+
+- Set up a namespace. The user permission [issue 781][piper-issue-781] of the project "Piper" repository describes how to set up a Linux kernel user namespace to prevent the mismatch of user IDs. **Note:** This solution is experimental and should be well-considered.
+
+[github]: https://github.com
+[docker-install]: https://docs.docker.com/install
+[dockerhub-node]: https://hub.docker.com/_/node/
+[docker-getstarted]: https://docs.docker.com/get-started/
+[jenkins-doc-client]: https://jenkins.io/doc/book/managing/cli/
+[jenkins-docker-image]: https://github.com/jenkinsci/docker/
+[piper-library-pages]: https://sap.github.io/jenkins-library
+[piper-issue-781]: https://github.com/SAP/jenkins-library/issues/781
+
+[devops-docker-images]: https://github.com/SAP/devops-docker-images
+[devops-cxs-plugins]: https://github.com/SAP/devops-docker-cx-server/blob/master/jenkins-master/plugins.txt

documentation/docs/guidedtour.md

@@ -42,7 +42,7 @@ For more information on the Cx Server and how to customize your Jenkins, have a
 
 If you use your own Jenkins installation, you need to care for the configuration that is specific to project "Piper".
 This option should only be considered if you know why you need it, otherwise using the Cx Server life-cycle management makes your life much easier.
-If you choose to go this path, follow [my own Jenkins installation][guidedtour-my-own-jenkins] for some hints.
+If you choose to go this path, follow the [Custom Jenkins Setup guide][resources-custom-jenkins].
 
 **Note:** This option is not supported for SAP Cloud SDK projects.
 
@@ -206,6 +206,7 @@ Please also consult the blog post on setting up [Continuous Delivery for S/4HANA
 [guidedtour-sample.config]:          samples/cloud-cf-helloworld-nodejs/pipeline/config.yml
 [guidedtour-sample.jenkins]:         samples/cloud-cf-helloworld-nodejs/Jenkinsfile
 [guidedtour-sample.mta]:             samples/cloud-cf-helloworld-nodejs/mta.yaml
+[resources-custom-jenkins]:          customjenkins.md
 [resources-configuration]:           configuration.md
 [resources-step-mtabuild]:           steps/mtaBuild.md
 [resources-step-cloudFoundryDeploy]: steps/cloudFoundryDeploy.md

documentation/docs/jenkins/requiredPlugins.md

@@ -1,56 +0,0 @@
-# Required Plugins
-
-The following Jenkins plugins are needed in order to use the Piper Library.
-The list below contains the plugin Id and version of the plugin.
-
-**Plugins**
-
-- ace-editor 1.1
-- authentication-tokens 1.3
-- bouncycastle-api 2.16.2
-- branch-api 2.0.14
-- cloudbees-folder 6.2.1
-- credentials 2.1.16
-- credentials-binding 1.13
-- display-url-api 2.1.0
-- docker-commons 1.9
-- docker-workflow 1.10
-- durable-task 1.15
-- git 3.6.2
-- git-client 2.5.0
-- git-server 1.7
-- handlebars 1.1.1
-- icon-shim 2.0.3
-- jquery-detached 1.2.1
-- junit 1.21
-- mailer 1.20
-- matrix-project 1.12
-- momentjs 1.1.1
-- pipeline-build-step 2.5.1
-- pipeline-graph-analysis 1.3
-- pipeline-input-step 2.8
-- pipeline-milestone-step 1.3.1
-- pipeline-model-api 1.2.2
-- pipeline-model-definition 1.1.1
-- pipeline-model-extensions 1.1.1
-- pipeline-rest-api 2.6
-- pipeline-stage-step 2.2
-- pipeline-stage-tags-metadata 1.2.2
-- pipeline-stage-view 2.6
-- pipeline-utility-steps 1.3.0
-- plain-credentials 1.4
-- scm-api 2.2.3
-- script-security 1.34
-- ssh-credentials 1.13
-- structs 1.10
-- workflow-aggregator 2.5
-- workflow-api 2.23.1
-- workflow-basic-steps 2.6
-- workflow-cps 2.41
-- workflow-cps-global-lib 2.7
-- workflow-durable-task-step 2.17
-- workflow-job 2.12.2
-- workflow-multibranch 2.14
-- workflow-scm-step 2.6
-- workflow-step-api 2.13
-- workflow-support 2.16

documentation/docs/myownjenkins.md

@@ -1,51 +0,0 @@
-# My own Jenkins
-
-**Note:** This guide is not sufficient yet, to setup a Jenkins for project "Piper".
-
-## Requirements
-
-* Java Runtime Environment 8
-* Installation of Jenkins v 2.60.3 or higher running on Linux. We tested with debian-stretch.
-* Jenkins Plugins installed as described in the [Required Plugin][piper-library-pages-plugins] section.
-* A Jenkins user with administration privileges.
-* The Jenkins instance has access to [github.com][github].
-
-## Download and Installation
-
-To setup the shared library, you need to perform the following steps:
-
-1. Login to your Jenkins instance with administration privileges.
-1. Open the system configuration page (*Manage Jenkins > Configure System*).
-1. Scroll down to section *Global Pipeline Libraries* and add a new Library by clicking the *Add* button.
-    1. set *Library Name* to `piper-lib-os`
-    1. set *Default Version* to the branch or tag you want to consume (e.g. `master` or `v0.1`)
-    1. set *Retrieval Method* to `Modern SCM`
-    1. set *Source Code Management* to `Git`
-    1. set *Project Repository* to `https://github.com/SAP/jenkins-library`
-1. Save changes
-
-![Library Setup](images/setupInJenkins.png)
-
-Now the library is available as `piper-lib-os` and can be used in any
-`Jenkinsfile` by adding this line:
-
-```groovy
-@Library('piper-lib-os') _
-```
-
-Jenkins will download the library during execution of the `Jenkinsfile`.
-
-[github]: https://github.com
-[piper-library]: https://github.com/SAP/jenkins-library
-[devops-docker-images]: https://github.com/SAP/devops-docker-images
-[piper-library-scenario]: https://sap.github.io/jenkins-library/scenarios/ui5-sap-cp/Readme/
-[piper-library-pages]: https://sap.github.io/jenkins-library
-[piper-library-pages-plugins]: https://sap.github.io/jenkins-library/jenkins/requiredPlugins
-[piper-library-issues]: https://github.com/SAP/jenkins-library/issues
-[piper-library-license]: ./LICENSE
-[piper-library-contribution]: .github/CONTRIBUTING.md
-[jenkins-doc-pipelines]: https://jenkins.io/solutions/pipeline
-[jenkins-doc-libraries]: https://jenkins.io/doc/book/pipeline/shared-libraries
-[jenkins-doc-steps]: https://jenkins.io/doc/pipeline/steps
-[jenkins-plugin-sharedlibs]: https://wiki.jenkins-ci.org/display/JENKINS/Pipeline+Shared+Groovy+Libraries+Plugin
-[google-group]: https://groups.google.com/forum/#!forum/project-piper

documentation/mkdocs.yml

@@ -82,7 +82,7 @@ nav:
         - whitesourceExecuteScan: steps/whitesourceExecuteScan.md
         - xsDeploy: steps/xsDeploy.md
     - Resources:
-        - 'Required Plugins': jenkins/requiredPlugins.md
+        - 'Custom Jenkins Setup': customjenkins.md
 
 theme:
     name: 'material'