Jenkins vs Travis CI vs Circle CI vs TeamCity vs Codeship vs GitLab CI vs Bamboo
Why should you use a tool for your CI/CD workflow, and which one is the right tool for you?
More and more engineering teams are adopting agile development, pushing for shorter and faster release cycles. Code base growth and the frequency of creating new production builds led to the rise of Continuous Integration and Continuous Deployment/Delivery tools.
If you’re also thinking of making the switch to a fast release frequency, or if you’re not certain what other tools have to offer – we’ve got you covered. In the following post we’ll get familiar with some of the most popular CI/CD tools out there, and get to know each one a little bit better. Ship ahoy!
Table of contents
What’s CI/CD and what is it good for?
Before taking a deep dive into CI/CD automation tools, we first need to understand the concept at large. As we mentioned, Continuous Integration and Continuous Deployment usually go hand in hand with an agile development environment, where teams want to deploy different pieces of code to production as soon as they’re finished with them.
The use of CI/CD tools automates the process of building, testing and deploying new code. Each team member can get immediate feedback about the production readiness of their code, even if they just changed a single line or character. That way, each team member can push their code into production, while the process of building, testing and deploying is done automatically so they can move on to work on the next part of the application.
Adding automation to your workflow doesn’t end with deploying code into production. ou have to keep track of new errors before they significantly impact your users. For most teams, debugging in production is a manual and tedious task that requires their full attention for hours and days of sifting through logs. However, now it’s possible to automate root cause analysis at scale, knowing where, when and most importantly, why errors are happening.
If you’re a Java, Scala or a .NET developer, we have a special treat for you, check it out.
Now that we know why it’s important to implement automation in our workflow with a CI model, it’s time to see which tool is the right one for us.
1. Jenkins
Jenkins is one of the more known and common names in the CI market. It started as a side project by one of Sun’s engineers, and expanded into one of the biggest open source CI tools that helps engineering teams automate their deployments. Full disclosure: we at OverOps also use Jenkins, along with a homegrown CLI tool.
What does it do?
Just like a CI tool promises, with Jenkins you can automate your build, test and deploy tasks. The tool supports Windows, Mac OSX and various Unix systems, and can be installed using native system packages, as well as Docker or installed as a standalone on any machine with a Java Runtime Environment (JRE) installed.
On the practical side, Jenkins gives any member of the team the ability to push their code to the build, and get immediate feedback on whether it’s ready for production or not. In most cases, this will require some tinkering and tailoring of Jenkins according to your team’s custom requirements.
The place where Jenkins shines is with its rich plugin ecosystem. It offers an extended version with over 1,000 plugins, which allows integration with almost every tool and service that’s available on the market. Being an open source tool also gives you the option to custom fit it for a home-grown solution, just like we did. However, the need to spend time and some effort to make sure it is suitable for you might be a downside for some teams.
Price: Free
One more thing: We’ve said it once and we’ll say it again: open source + plugins = community. Any configuration, workflow, need or desire you can think of, you’ll have an option to create it with the help of Jenkins and its plugins. Also, great name for a band.
Bottom line: If you’re looking for a cheap (free!) CI solution, are willing to put in the work to customize your environment and need support from a community of users, Jenkins is the choice for you.
2. Travis CI
Travis CI is one of the more common names in the CI/CD ecosystem, created for open source projects and then expanded to closed source projects over the years. It’s focused on the CI level, improving the performance of the build process with automated testing and an alert system.
What does it do?
Travis-CI focuses on allowing users to quickly test their code as it’s deployed. It supports large and small code changes, and is designed to identify changes in building and testing. When a change is detected, Travis CI can provide feedback whether the change was successful or not.
Developers can use Travis CI to watch the tests as they run, run a number of tests in parallel, and integrate the tool with Slack, HipChat, Email and so on to get notified of issues or unsuccessful builds.
Travis CI supports container builds, and supports Linux Ubuntu and OSX. You can use it across different programming languages, such as Java, C#, Clojure, GO, Haskell, Swift, Perl and much more. It has a limited list of third party integrations, but since the focus is on CI rather than CD, it might not be an issue for your use case.
Price: While Travis CI offers free support for open source projects, the prices for private projects vary from $69/mo for the bootstrap version and up to $489/mo for the premium version.
One more thing: To make sure you always have a backup of your recent build, Travis CI clones your GitHub repository into a new virtual environment whenever you run a new build.
Bottom line: If your code is open source and you’re more concerned about the continuous integration of your builds, Travis CI is worth a try.
3. Circle CI
Circle CI is a cloud based tool that automates the integration and deployment process. It also focuses on testing every change to the code before it’s deployed, using a number of methods such as unit tests, integrations tests and functional tests. The tool supports containers, OSX, Linux and can run within a private cloud or your own data center.
What does it do?
Circle CI integrates with your current version control system, such as GitHub, Bitbucket and others, and runs a number of steps whenever a change is detected. These changes could be commits, opening PRs or any other change to the code.
Every code change creates a build and runs the tests in a clean container or in a VM, according to your initial configurations and preferences. Each build consists of a number of steps, including dependencies, testing and deployment. If the build passes the tests, it can be deployed through AWS CodeDeploy, Google Container Engine, Heroku, SSH or any other method of your choice.
The success or failure status of the builds and tests in question is sent via Slack, HipChat, IRC or a number of other integrations, so the team can stay updated. It’s important to note that Circle CI requires some tweaks and changes for a number of languages, so it’s best to go over the documentation for your language of choice.
Price: For Linux users, the first container is free, and each additional container costs $50/month. OSX prices start at $39/mo for teams building 1-5 builds/day and for private data centers or cloud, the price starts at $35/mo per user for an annual contract.
One more thing: Circle CI can auto-cancel redundant builds on GitHub. If a newer build is triggered on the same branch, the tool identifies it and cancels the older builds that are running or queued, even if the build in not finished.
Bottom line: If you’re looking for a GitHub friendly tool with a wide community behind it, that can also run within a private cloud or your inside own data center, Circle CI is worth checking out.
4. TeamCity
TeamCity is a CI/CD server, made by JetBrains. It offers continuous integration “out of the box”, as well as allowing users to best fit the tool to their own needs. It offers support for a number of languages (Java, .NET, Ruby and others), and has JetBrains to back the tool up support and documentation wise.
What does it do?
As a CI/CD tool, TeamCity aims to improve release cycles. With it you can review on-the-fly of test results, see code coverage and find duplicates, as well as customize statistics on build duration, success rate, code quality and other custom metrics.
Once TeamCity detects a change in your version control system, it adds a build to the queue. The server finds an idle compatible build agent and assigns the queued build to this agent, which executes the build steps.
While this process is running TeamCity server logs the different log messages, test reports, and other changes that are being made. The changes are saved and uploaded in real time, so users can know what’s happening with the build as they make changes to it. The tool also offers an option to run parallel builds simultaneously on different platforms and environments.
Price: The professional server license is offered for free, and it includes 20 build configurations, full access to all product features, support via forum and issue tracker and 3 build agents. The enterprise server license starts at $1,999 for a server with 3 agents, and the price increase according to the amount of agents you’re interested in.
One more thing: TeamCity comes with an option of gated commits, which can prevent developers from breaking sources in a version control system. This is done by running the build remotely for local changes, before committing it.
Bottom line: TeamCity has been gaining popularity over the past few years, offering a decent alternative to other CI tools in the market. And if you’re interested in viewing your builds and tests as they happen, or want a free and powerful CI solution, there’s no doubt that TeamCity is worth checking out.
5. Codeship
Codeship has a different view on CI/CD, and offers a Hosted Continuous Integration as a Service. This tool was originally built to give a Continuous Integration platform for Rails developers; hosting their code on GitHub and deploying to Heroku. Due to its popularity and demand, the company expanded over the years to support other technologies as well.
What does it do?
Codeship has 2 different variations of the product, each with its pros and cons. Codeship Basic allows setting up a CI/CD process by connecting a repository through a web UI and turnkey deployments. It offers support for a pre-configured CI environment, and allows multiple different builds to run on the same build VM.
Codeship Pro uses Docker to define a CI/CD environment, through it you can run your build pipeline. It comes with complete control over the build environment, letting you define what to run within it. The Pro version also allows pre branch caching, to set which image and which part of the workflow gets cached, along with parallel deployments.
As a whole, Codeship supports a number of languages, such as Java, Go, Node.js, Python, Ruby and others. When it comes to deployment, the Basic version supports AWS, Heroku, Azure and Kubernetes while the Pro also supports AWS ElasticBeanstalk, Google App Engine and DigitalOcean.
Price: The free plan includes 100 builds per month for unlimited projects, users and teams. It also offers one concurrent build and one parallel test pipeline.
Prices for the basic and pro plans range between $49-$79/mo, according to the number of concurrent build and parallel test pipeline you’re interested in.
One more thing: Codeship has a public collection of utilities, scripts and Docker images to use with the tool, and the company even points out that some of these can be used with other similar tools. This collection includes deployment scripts for external services that are customizable, scripts to install specific versions of software that are not included by default on the build VMs and more.
Bottom line: Offering 2 different tools under one domain might seem a bit odd, but it gives Codeship the option to focus on various elements that will better suit different kinds of customers. Since both the Basic and Pro are available for free, it’s an interesting choice for you CI needs.
6. GitLab CI
Soon after GitLab was launched, the team launched GitLab CI, the continuous integration service. Along with testing and building projects, the tool can also deploy the builds to your infrastructure, giving you an option to track your different deployments by knowing where each piece of code went.
What does it do?
GitLab CI is offered for free for as a part of GitLab, and can be set up rather quickly. To start using GitLab CI, you first need to add a .gitlab-ci.yml file to the root directory of your repository, as well as configure your GitLab project to use a Runner. Afterwards, every commit or push will trigger the CI pipeline that has three stages: build, test and deploy.
Each built can be divided into multiple jobs, and can run in parallel on multiple machines. The tool gives immediate feedback on the success or failure of the build, letting users know whether something went wrong or if something broke in the process.
Price: The Community Edition is offered for free. Prices start at $3.25/mo for a plan that includes issue boards, multiple approvals in code review, advanced syntax search and some more features.
One more thing: GitLab (and GitLab CI) is an open source project. In other words, you have access and the ability to modify the GitLab Community Edition, and the Enterprise Edition source code.
Bottom line: If you’re using GitLab, it’s almost a no-brainer to try the GitLab CI solution as part of the pack.
7. Bamboo
Bamboo is a part of the Atlassian product suite, and similarly to other tools, it offers building, testing and deploying code and supports numerous languages. It has strong integrations to other Atlassian products that are relevant for the CI cycle, such as JIRA and Bitbucket.
What does it do?
Build, test and deploy are all part of Bamboo’s package, and the testing part is done with the help of a Bamboo Agents. Similar to the agents in Java monitoring, Bamboo also offers two types; the local agents run as part of the Bamboo server as part of its process, while the remote agents run on other servers and computers. Each agent is assigned to the builds that match its capabilities, which allows assigning different agents to different builds.
The main advantage Bamboo offers is the strong ties to the rest of Atlassian’s products, such as JIRA and Bitbucket. With Bamboo you can see code changes and JIRA issues that have been introduced into the code since the last deployment. That way, developers can sync their workflow and always stay on track and know which version is next, and what (should) have been fixed.
Price: Bamboo pricing is based on the number of agents. The basic pricing starts at $10 for unlimited local agents, up to 10 jobs and no remote agents. The next tier is $800 for unlimited jobs and local agents, and 1 remote agent. Additional remote agents will spike up the price up to $44,000.
One more thing: Bamboo comes with Atlassian’s strong back, along with better workflows for the company’s existing products. If you want to add JIRA and Bitbucket to your CI process in a seamless way and are willing to pay for it, Bamboo is worth a try.
Bottom line: Bamboo is powerful as long as you’re using it with Bitbucket and JIRA, and are willing to pay for your CI solution.
Final thoughts
The need for faster and shorter release cycles leads to teams having to find tools and workflows that will support new methods for delivering software. Pushing to production on a weekly or even daily or hourly basis also means introducing new errors to production.
Now that you’ve moved to a CI/CD workflow, it’s time to think of the next step in this process; debugging in production and detecting errors as soon as they are introduced. If you want to know how to take your automation process one step further and utilize Automated Root Cause detection, you should check out OverOps.
| Published on Java Code Geeks with permission by Henn Idan, partner at our JCG program. See the original article here: Jenkins vs Travis CI vs Circle CI vs TeamCity vs Codeship vs GitLab CI vs Bamboo Opinions expressed by Java Code Geeks contributors are their own. |
