---
layout: markdown_page
title: "Jenkins"
---
## On this page
{:.no_toc}
- TOC
{:toc}
## Overview
<%= partial("includes/sdlc_comparison_tabs_vue_temp", :locals => { :source => "cloudbees_jenkins_electric_cloud" }) %>
## Summary
Jenkins is one of the most popular self-managed open source build automation and CI/CD developer tool in the world. It derives it's incredible flexibility from incorporating capabilities from it's hundreds of available plugins, enabling it to support building, deploying and automating any project.
At the Q3 2018 Jenkins World conference CloudBees (the primary maintainers of Jenkins) announced their intention to revive the competitiveness of Jenkins by splitting it and focusing on a cloud native version, as well as a simplified, opinionated version (Jenkins Evergreen). There is also a Jenkins sub-project called Jenkins X, meant to make running a pipeline out of the box with Kubernetes easier.
Jenkins X natively integrates Jenkins CI/CD server, Kubernetes, Helm, and other tools to offer a prescriptive CI/CD pipeline with best practices built-in, such as using GitOps to manage environments. It uses deployment of Jenkins into Kubernetes containers to get around the complexities of installing and integrating Jenkins. However, it is a complex pairing of many tools including the fragile Jenkins server.
In contrast, GitLab already provides more than what Jenkins is hoping to evolve to, by providing a fully integrated single application for the entire DevOps lifecycle. More than Jenkins' goals, GitLab also provides planning, SCM, packaging, release, configuration, and monitoring (in addition to Cthe I/CD that Jenkins is focused on).
## Gaps
* Extending the native functionality of Jenkins is done through plugins. Plugins are expensive to maintain, secure, and upgrade. In contrast, GitLab is [open core](https://about.gitlab.com/2016/07/20/gitlab-is-open-core-github-is-closed-source/) and anyone can contribute changes directly to the codebase, which once merged would be automatically tested and maintained with every change.
## Comments/Anecdotes
* HackerNews Thread - [Jenkins is Getting Old](https://news.ycombinator.com/item?id=19781251)
* Jenkinstein - From the article [DevOps World 2018: ‘Jenkinstein’ and a Cloud Native Jenkins](https://thenewstack.io/devops-world-2018-jenkinstein-and-a-cloud-native-jenkins/)
- Describing the snowflake server and the ["Brent"](http://devopsdictionary.com/wiki/Brent) situation and how it slows down everything. This is one of the main issues which Jenkins users face today and which Jenkins has no easy fix for:
> Acting as the gatekeeper for that channel is someone CloudBees CEO Sacha Labourey described as “the Jenkins guy… this superstar devoted to making Jenkins great on his team. Because this person is the authority on deployment within his organization, multiple teams come to rely on him to meet their scheduling goals. Yet this leads to a technical issue that few folks outside of IT operations take time to consider: The Jenkins Master . . . (the server managing a distributed scheme with multiple agents) becomes bloated, like an old telephone directory or the Windows XP System Registry.
>
> And because that organization’s Jenkins deployment is not only dependent upon the Guy, but somewhat bound to his choices of plug-ins, the result is what the CEO called “Frankenstein Jenkins,” and what other developers and engineers at the conference Tuesday had dubbed “Jenkinstein.”
- Again, CloudBees CEO Sacha Labourey describing the common problem with current Jenkins:
> Jenkins becomes bloated, slow to start. When it crashes, it takes forever to start. Hundreds of developers are pissed. And nobody wants to fix it, because if you touch it, you own it, right?”
* From note on 2018-08-31 from CloudBees CTO and Jenkins creator Kohsuke Kawaguchi ([https://jenkins.io/blog/2018/08/31/shifting-gears/](https://jenkins.io/blog/2018/08/31/shifting-gears/)):
- "Our Challenges. . . Service instability. . . Brittle configuration. . . Assembly required. . . Reduced development velocity. . ." (see above link for details on each)
- "Path forward. . . Cloud Native Jenkins. . . continue the incremental evolution of Jenkins 2, but in an accelerated speed"
- **Key takeaways**:
- They plan on BREAKING backward compatibility in their upcoming releases
- They plan on introducing a NEW flavor of Jenkins for Cloud native
- If you’re a Jenkins user today, it’s going to be a rough ride ahead
* From Jenkins Evergreen project page (identified in Kohsuke letter above as key for changes that need to come) - [https://github.com/jenkinsci/jep/blob/master/jep/300/README.adoc](https://github.com/jenkinsci/jep/blob/master/jep/300/README.adoc)
- "Pillars . . . Automatically Updated Distribution . . . Automatic Sane Defaults. . . Connected. . . Obvious Path to User Success"
- "The "bucket of legos" approach is . . . not productive or useful for end-users [5] who are weighing their options between running Jenkins, or using a CI-as-a-Service offering such as Travis CI or Circle CI."
- "existing processes around "Suggested Plugins", or any others for that matter, result in many "fiefdoms" of development rather than a shared understanding of problems and solutions which should be addressed to make new, and existing, users successful with Jenkins."
* From "Problem" definition page of Jenkins Evergreen project page ([https://github.com/jenkinsci/jep/blob/master/jep/300/README.adoc#problem](https://github.com/jenkinsci/jep/blob/master/jep/300/README.adoc#problem)):
> For novice-to-intermediate users, the time necessary to prepare a Jenkins environment "from scratch" into something productive for common CI/CD workloads, can span from hours to days, depending on their understanding of Jenkins and it’s related technologies. The preparation of the environment can also be very error prone and require significant on-going maintenance overhead in order to continue to stay up-to-date, secure, and productive.
>
> Additionally, many Jenkins users suffer from a paradox of choice [6] when it comes to deciding which plugins should be combined, in which ways, and how they should be configured, in order to construct a suitable CI/CD environment for their projects. While this is related to the problem which JEP-2 [7] attempted to address in the "Setup Wizard" introduced in Jenkins 2.0, Jenkins Evergreen aims to address the broader problem of providing users with a low-overhead, easily maintained, and solid distribution of common features (provided by a set of existing plugins) which will help the user focus on building, testing, and delivering their projects rather than maintaining Jenkins.
* GitLab blog on feedback about new Jenkins improvement efforts - [https://about.gitlab.com/blog/2018/09/03/how-gitlab-ci-compares-with-the-three-variants-of-jenkins/](https://about.gitlab.com/blog/2018/09/03/how-gitlab-ci-compares-with-the-three-variants-of-jenkins/)
* Project analysis on Jenkins, pointed to by CloudBees documentation as proof for need of change - [https://ghc.haskell.org/trac/ghc/wiki/ContinuousIntegration#Jenkins](https://ghc.haskell.org/trac/ghc/wiki/ContinuousIntegration#Jenkins)
> Pros
> - We can run build nodes on any architecture and OS we choose to set up.
>
> Cons
> - Security is low on PR builds unless we spend further effort to sandbox builds properly. Moreover, even with sandboxing, Jenkins security record is troublesome.
> - Jenkins is well known to be time consuming to set up.
> - Additional time spent setting up servers.
> - Additional time spent maintaining servers.
> - It is unclear how easy it is to make the set up reproducible.
> - The set up is not forkable (a forker would need to set up their own servers).
* From GitLab PMM
> - “Jenkins had to build a whole new separate project in order to work with Kubernetes. GitLab has natively adopted Kubernetes from the get-go.”
> - Jenkins X adoption is tiny. Most folks looking to go to Kubernetes will be on Jenkins proper, so the [Pinterest anecdote](/handbook/marketing/product-marketing/#deliver-value-faster) applies.
> - Although Jenkins X works with Kubernetes, it’s not a single application like GitLab. You still have to integrate to your PPM, SCM, security tools, etc. You have to manage permissions and access across all that which GitLab gives you out of the box, but Jenkins X does not ([value of a single app](/handbook/product/single-application/)).
## Strategy
[Jenkins objection handling](https://docs.google.com/document/d/18VKkaFRT2wo3JWL8OmHXMNOenl0vNYcUmoNoCsm9wHA/edit?usp=sharing) (GitLab access only)
[Jenkins objection handling Q&A](https://docs.google.com/document/d/1p94F9M2epeebyEwQfIhZDhQ0E2b4iqBPLkR5zbuiceU/edit) (GitLab access only)
## Resources
* [Jenkins OSS Website](https://jenkins.io/) - Open Source project website
* CloudBees Jenkins Website - [https://www.cloudbees.com/products/cloudbees-core](https://www.cloudbees.com/products/cloudbees-core)
* [Jenkins X Website](https://jenkins-x.io/)
## Pricing
* Jenkins OSS
- No cost (and Open Source)
- But Total Cost of Ownership is not zero, given maintenance requirements
* CloudBees Jenkins
(vague) - https://www.cloudbees.com/products/pricing
- CloudBees Core - Jenkins distribution with upgrade assistance on monthly incremental upgrades, cloud native architecture, centralized management, 24/7 support and training, enterprise-grade security and multi-tenancy, and plugin compatibility testing
- starting at $20k/year for 10 users, with tiered pricing for lower per-user cost for larger organizations
* Jenkins X
- No cost (and Open Source)
- But Total Cost of Ownership has cost (see [Pinterest anecdote](/handbook/marketing/product-marketing/customer-reference-program/#deliver-value-faster))
## Questions to ask
- "Do you use any Jenkins plugins to get the functionality that you need from Jenkins? How many plugins do you need to run?"
- Expect an average answer of 7-9.
- Follow on: "Most of the organizations we talk to tell us that managing Jenkins plugins is a nightmare. Each one has it's own development life cycle, and each time a new one comes out that they need they have to re-test the whole tool chain. We've heard stories like Group A needs plugin A, and the Jenkins administrator installs it, only to find out that it depends on a newer (and incompatible) version of the library that plugin B requires, which Group B needs. Solution? Give each group their own Jenkins server? But then you end up with Jenkins sprawl (more servers to manage). Not to mention all the maintenance and testing required to all the externally integrated tools in the tool chain." (expect to see a lot of nodding during this spiel)
## Comparison