GitHub and Jenkins integration

Jakub StasApril 10th, 2015Last Updated: April 9th, 2015

1 46 7 minutes read

Jenkins is a well-know term in many teams around the world and has been for quite some time now. As many organizations migrate their source code form centralized version control systems to git, GitHub seems like a great tool to support and simplify work with git. Needless to mention its great support for code reviews by the concept of pull requests. Both Jenkins and GitHub are great tools on their own but it would be really nice to integrate them and leverage the power of automation even more. In this post I am going to describe my attempt to do so.

Working with git and GitHub

The way you work with git is important. Choosing proper branching model may and will influence the way you work and communicate with your teammates. If you use git now and carry any baggage from your Subversion times it’s time to rethink how you branch. Following post will rely on a suitable branching model to support the integration. Couple weeks ago, I came across an excellent article by Vincent Driessen with a title A successful Git branching model. I highly recommend checking it out since it provides a detailed description of one possible approach to branching with git that suits this post and works.

Gits branching model allows for smooth parallelization of work and pull requests in GitHub allow for discussions over the source code and effective code review process. In an ideal world, each developer merges the latest version of master code, runs full test suite against the feature they have worked on to prevent any problems and fixes that might occur after pushing to master. Unfortunately, this is not always the case so it makes sense to automate the whole process. What I propose here is to have Jenkins merge the pull request changes into a target branch, run a test suite and sonar/checkstyle analysis against this merged build and notify developers via HitBubs pull request listing if this pull request is ready for or even worthy of a code review.

Setting up Jenkins

First thing you need to do is to install additional plugins facilitating the integration. These plugins are either standalone plugins or dependencies of the plugins required to make the communication between the systems possible.

GIT client plugin (Shared library plugin for other Git related Jenkins plugins)
GIT plugin (This plugin integrates GIT with Jenkins)
GitHub API Plugin (This plugin provides GitHub API for other plugins)
GitHub plugin (This plugin integrates GitHub to Jenkins)
GitHub Pull Request Builder (This plugin builds pull requests in GitHub and report results)

Now you need to allow Jenkins to communicate with GitHub by configuring GitHub Web Hook access. Head over to Jenkins system configuration by clicking on Jenkins -> Manage Jenkins -> Configure system. You should see following entry near the bottom of the page:

Selecting Let Jenkins auto-manage hook URLs allows you to configure GitHub credentials to be used:

API URL: https://GITHUB_HOST:GITHUB_PORT/api/v3
Username: USER
OAuth token: TOKEN

There are several ways to obtain the OAuth token and one of them will be mentioned in a section about Jenkins configuration, so if you need help with it read on. You can test your configuration once you filled all the credentials.

Make Jenkins trust GitHub

In case you are running local instance of GitHub you may run into trouble with your self-signed certificates. To get over this issue it is required to export the target application’s SSL Certificate, import it into the Jenkins server’s JVM TrustStore, and restart Jenkins in order for Jenkins to trust the target application. Good news is that this needs to be done only once per Jenkins’s lifetime. To accomplish this, just follow these three steps:

Fetch the certificate from local GitHub:

openssl s_client -connect GITHUB_HOST:GITHUB_PORT < /dev/null | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > public.crt

Import the certificate into the keystore. After you fetched the certificate, import it into the JAVA_HOME used to run Jenkins:
```
JVM_PATH/bin/keytool -import -alias GITHUB_HOST -keystore JVM_PATH/jre/lib/security/cacerts -file public.crt
```
Don’t forget to restart Jenkins:
```
service jenkins restart
```

Please note that GITHUB_HOST, GITHUB_PORT and JVM_PATH need to be replaced by actual values specific to your environment.

Setting up GitHub

Before I start describing the way to setup GitHub there is one tip I would like to share in case your organisation is not practicing it. For purposes of integration and automation like this one, it is beneficial to create a dedicated system user to act on behalf of systems and scripts making the development process easier. For the purposes of Jenkins and GitHub integration, it is required to grant this user a collaborator role or Push & Pull rights. Once you have this user setup I encourage you to use it to setup your GitHub instance.

Once you log in to GitHub, head over to the repository of your choice and click on repository settings. After clicking on Webhooks & Services you will be presented with an option to add a new webhook or add a new service. Click on Add service and try to look for Jenkins (the name of the service was Jenkins (Git plugin) at the time of writing this post). Configuration is pretty straight forward and the only thing you need to provide is Jenkins hook URL and check Activate. The URL should look as follows http://JENKINS_HOST:JENKINS_PORT/github-webhook/.

You can test your configuration by clicking Test service. Head over to Jenkins system log to be sure that everything works well. You should be able to find log entries similar to these:

Mar 10, 2015 3:32:55 PM INFO com.cloudbees.jenkins.GitHubWebHook processGitHubPayload
Received POST for https://GITHUB_HOST:GITHUB_PORT/ORGANISATION/REPOSITORY

Mar 10, 2015 3:32:55 PM INFO com.cloudbees.jenkins.GitHubWebHook processGitHubPayload
Received POST for https://GITHUB_HOST:GITHUB_PORT/ORGANISATION/REPOSITORY

Given you followed all the previous steps successfully, you should see green check mark next to Jenkins (GitHub plugin) service.

Configuring Jenkins

Last step of the integration process is the configuration of Jenkins in terms of how to handle incoming web hooks and other stuff like comments or triggers. Head over to Jenkins system configuration by clicking on Jenkins -> Manage Jenkins -> Configure system and look for GitHub Pull Request Builder.

Start by filling out the basic configuration. Use before mentioned API URL https://GITHUB_HOST:GITHUB_PORT/api/v3 and the OAuth token. By clicking on Advanced… you can fine tune the behavior of this plugin. Most of the default setting are fine for what we want to achieve but I would suggest updating the Test phrase so it needs to match exactly.

One of the ways of generating the OAuth token is near bottom of this dialog. By providing the username and password you can generate an access token to be used in the earlier configuration. And that’s it. Completing all these steps should allow Jenkins to pick up pull request related events and act upon them.

Trade-offs of this integration

Faster bug detection
- One of the biggest improvements in the development process is the early bug detection. This is highly dependent on the quality of your tests (unit tests, integration tests, user acceptance tests, …). Having the option to test your code prior to merge allows you to prevent broken builds. This also comes in handy when your task involves environment modifications that need to be explicitly executed prior to running the application. Failing tests may point you to developing automated solution or informing your teammates and QA that an extra step is required from this point on in deployment process to make the application run as expected.
Removed need to roll back changes
- This point is connected to the previous one. Sometimes you break the build, push changes that make certain portions of the code run inefficiently, break the integration with a third-party software or simply screw up in a way that you need to roll your changes back from the master branch so it stays pure and fresh. Based on your test base these situation can be reduced dramatically and make the process more fluent.
Improved code review process
- Code review process involves many types of code checking. Running Sonar, PMD, CheckStyle or FindBugs type of analysis removes some burden from reviewers of your changes so they don’t have to bother with comments, indentation, brackets and other syntactical aspects of the code and can focus on the logic itself. Running this type of check before merging keeps master branch clean so its easy to use by anyone checking it out without creating the need to reformat or rewrite portions of code.
Customization options
- GitHubs webhook provides a handful of useful information you can use in your job definition. This allows for better integration of other processes as well as customization of what needs to be build by your job. My personal favorite is filtering by target branch.
Extra Jenkins jobs
- One thing to consider is the added load on Jenkins (depending on your infrastructure and hardware) which in turn manifests itself in following ways:
  - Added overhead
    - Creating new build jobs may negatively influence the speed of your Jenkins instance which may in turn require addition of a new Jenkins node.
  - New testing environment
    - Testing with a database that is not embedded or an integration with other systems might result in the need for new testing environment to be added to be used solely for the purpose of pull request checking.
  - Conflicts inflicted by the infrastructure
    - Not every company is willing to provide enough resources to facilitate proper testing processes and environments. This is the scenario when you might experience conflicts inflicted by infrastructure when two pull request checks test the code against single database or system. This might be resolved by throttling the builds using Throttle Concurrent Builds Plugin.

What’s next

This post should have given you an idea how to make the most of your current CI tools by describing what the integration may bring you and how to go about making GitHub and Jenkins talk to each other. It is a first half of what you need to do. In my next post called GitHub and Jenkins pull request checking I will tackle the Jenkins job configuration and how to make the necessary information available where you need it.

Reference:

GitHub and Jenkins integration from our JCG partner Jakub Stas at the Jakub Stas blog.