Emacs, scripting and anything text oriented.

Version controlling Jenkins config

Kaushal Modi

Jenkins is an amazing free and open source continuous integration and deployment software. But its primary means of configuration is a web UI, and recently that cost me a lot of debug time. That set me down the path of figuring out a way to version control the Jenkins config (the $JENKINS_HOME directory).

Jenkins is a wonderful piece of software and I use it with my Bitbucket git repos for CI/CD.

Jenkins uses a web UI for its configuration. I dislike that because it’s difficult to document the configuration process without screenshots, and if I need to create a new server, it’s a manual process of clicking through tabs and filling in the text boxes. I didn’t mind this enough to do anything about it .. that is until I finally got bit by it.

What bit me #

Without going into too much detail, that issue was multi-fold:

  1. I had unknowingly messed up the Project-based Matrix Authorization Strategy such that other users in my team were not able to view the Jenkins jobs.
  2. I had also updated the Jenkins server that introduced a bug (JENKINS-68748) where the Test LDAP Settings failed with an error, but the LDAP authentication actually worked!
  3. I had also updated all the plugins after updating Jenkins. So if I rolled back the Jenkins versions, most of the plugins would fail because of incompatibility with the older Jenkins version. I had updated Jenkins after months!

That’s when I wished that my whole Jenkins was version-controlled. That would have allowed me to roll back to the last working “Jenkins image” with the Jenkin version, plugins' versions and my Jenkins config all in sync.

I had delayed doing this because my $JENKINS_HOME was more than 1GB in size and I didn’t have time or motivation to figure out what stuff I should commit and what I should ignore .. But no more — The time had finally come.

.gitignore for Jenkins config #

So I did what any good engineer would do .. start looking for a solution online. I found this StackOverflow answer for Is there a way to keep Hudson / Jenkins configuration files in source control?.

That answer shares a .gitignore that ignores files not necessary for configuring a Jenkins server — Example: job builds, workspace, log files, etc. But it didn’t work out of the box because the plugin version info wasn’t getting committed correctly. I had committed everything to git after using the suggested .gitignore and pushed to my git remote. But if I cloned that repo to a different area and attempted to start the Jenkins server from there, it crashed with this message:

2022-07-15 13:07:06.082+0000 [id=31]    SEVERE  jenkins.InitReactorRunner$1#onTaskFailed: Failed Loading global config
com.thoughtworks.xstream.mapper.CannotResolveClassException: hudson.security.ProjectMatrixAuthorizationStrategy
        at com.thoughtworks.xstream.mapper.DefaultMapper.realClass(DefaultMapper.java:81)
        at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:125)
        at com.thoughtworks.xstream.mapper.DynamicProxyMapper.realClass(DynamicProxyMapper.java:55)
        at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:125)
        at com.thoughtworks.xstream.mapper.PackageAliasingMapper.realClass(PackageAliasingMapper.java:88)
        at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:125)
        at com.thoughtworks.xstream.mapper.ClassAliasingMapper.realClass(ClassAliasingMapper.java:79)
        at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:125)
        at com.thoughtworks.xstream.mapper.ArrayMapper.realClass(ArrayMapper.java:74)
        at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:125)
        at com.thoughtworks.xstream.mapper.SecurityMapper.realClass(SecurityMapper.java:71)
        at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:125)
        at hudson.util.XStream2$CompatibilityMapper.realClass(XStream2.java:411)
        at hudson.util.xstream.MapperDelegate.realClass(MapperDelegate.java:46)
        at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:125)
        at com.thoughtworks.xstream.mapper.CachingMapper.realClass(CachingMapper.java:47)
        at hudson.util.RobustReflectionConverter.determineType(RobustReflectionConverter.java:521)
        at hudson.util.RobustReflectionConverter.doUnmarshal(RobustReflectionConverter.java:346)
Caused: jenkins.util.xstream.CriticalXStreamException:
---- Debugging information ----
cause-exception     : com.thoughtworks.xstream.mapper.CannotResolveClassException
cause-message       : hudson.security.ProjectMatrixAuthorizationStrategy
class               : hudson.model.Hudson
required-type       : hudson.model.Hudson
converter-type      : hudson.util.RobustReflectionConverter
path                : /hudson/authorizationStrategy
line number         : 12
version             : not available
Code Snippet 1: Snippet of Jenkins crash when attempting to run the server from the freshly cloned git repo

Jenkins Plugin Manager #

So I reached out for help on the Jenkins Community. One of the key contributors to Jenkins, Mark Waite, was tremendously helpful. He suggested using his jenkins-plugin-manager tool. After trying it out for a bit, I realized that this tool had everything I needed for version controlling the plugin versions:

  • Ability to save a list of installed Jenkins plugins and their versions to a file.
  • Ability to batch install all the plugins of the versions listed in a file.

This was like doing Python’s plugin management using requirements.txt, except that this was for Jenkins.

Full solution #

With a combination of the .gitignore that I started with from that SO answer, managing plugins using jenkins-plugin-manager, tweaking the .gitignore to my liking, and adding helper Bash scripts for downloading and running Jenkins server binaries, and doing the plugin management, I finally got what I needed:

The README on the repo has all the instructions.

Versions used: java 11.0.2 , jenkins 2.346.2
This is Day 37 of #100DaysToOffload.