Version controlling Jenkins config

Kaushal.Modi@fakeEmailToMakeValidatorHappy.com (Kaushal Modi) — Wed, 20 Jul 2022 00:18:00 -0400

Jenkins is an amazing free and open source continuous integration and deployment software. But its primary means of configuration is a web UI, and recently that cost me a lot of debug time. That set me down the path of figuring out a way to version control the Jenkins config (the $JENKINS_HOME directory).

Table of Contents

What bit me
.gitignore for Jenkins config
Jenkins Plugin Manager
Full solution

Jenkins is a wonderful piece of software and I use it with my Bitbucket git repos for CI/CD.

Jenkins uses a web UI for its configuration. I dislike that because it’s difficult to document the configuration process without screenshots, and if I need to create a new server, it’s a manual process of clicking through tabs and filling in the text boxes. I didn’t mind this enough to do anything about it .. that is until I finally got bit by it.

What bit me #

Without going into too much detail, that issue was multi-fold:

I had unknowingly messed up the Project-based Matrix Authorization Strategy such that other users in my team were not able to view the Jenkins jobs.
I had also updated the Jenkins server that introduced a bug (JENKINS-68748) where the Test LDAP Settings failed with an error, but the LDAP authentication actually worked!
I had also updated all the plugins after updating Jenkins. So if I rolled back the Jenkins versions, most of the plugins would fail because of incompatibility with the older Jenkins version. I had updated Jenkins after months!

That’s when I wished that my whole Jenkins was version-controlled. That would have allowed me to roll back to the last working “Jenkins image” with the Jenkin version, plugins' versions and my Jenkins config all in sync.

I had delayed doing this because my $JENKINS_HOME was more than 1GB in size and I didn’t have time or motivation to figure out what stuff I should commit and what I should ignore .. But no more — The time had finally come.

`.gitignore` for Jenkins config #

So I did what any good engineer would do .. start looking for a solution online. I found this StackOverflow answer for Is there a way to keep Hudson / Jenkins configuration files in source control?.

That answer shares a .gitignore that ignores files not necessary for configuring a Jenkins server — Example: job builds, workspace, log files, etc. But it didn’t work out of the box because the plugin version info wasn’t getting committed correctly. I had committed everything to git after using the suggested .gitignore and pushed to my git remote. But if I cloned that repo to a different area and attempted to start the Jenkins server from there, it crashed with this message:

2022-07-15 13:07:06.082+0000 [id=31] SEVERE jenkins.InitReactorRunner$1#onTaskFailed: Failed Loading global config
com.thoughtworks.xstream.mapper.CannotResolveClassException: hudson.security.ProjectMatrixAuthorizationStrategy
 at com.thoughtworks.xstream.mapper.DefaultMapper.realClass(DefaultMapper.java:81)
 at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:125)
 at com.thoughtworks.xstream.mapper.DynamicProxyMapper.realClass(DynamicProxyMapper.java:55)
 at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:125)
 at com.thoughtworks.xstream.mapper.PackageAliasingMapper.realClass(PackageAliasingMapper.java:88)
 at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:125)
 at com.thoughtworks.xstream.mapper.ClassAliasingMapper.realClass(ClassAliasingMapper.java:79)
 at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:125)
 at com.thoughtworks.xstream.mapper.ArrayMapper.realClass(ArrayMapper.java:74)
 at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:125)
 at com.thoughtworks.xstream.mapper.SecurityMapper.realClass(SecurityMapper.java:71)
 at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:125)
 at hudson.util.XStream2$CompatibilityMapper.realClass(XStream2.java:411)
 at hudson.util.xstream.MapperDelegate.realClass(MapperDelegate.java:46)
 at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:125)
 at com.thoughtworks.xstream.mapper.CachingMapper.realClass(CachingMapper.java:47)
 at hudson.util.RobustReflectionConverter.determineType(RobustReflectionConverter.java:521)
 at hudson.util.RobustReflectionConverter.doUnmarshal(RobustReflectionConverter.java:346)
Caused: jenkins.util.xstream.CriticalXStreamException:
---- Debugging information ----
cause-exception : com.thoughtworks.xstream.mapper.CannotResolveClassException
cause-message : hudson.security.ProjectMatrixAuthorizationStrategy
class : hudson.model.Hudson
required-type : hudson.model.Hudson
converter-type : hudson.util.RobustReflectionConverter
path : /hudson/authorizationStrategy
line number : 12
version : not available
-------------------------------

Code Snippet 1: Snippet of Jenkins crash when attempting to run the server from the freshly cloned git repo

Jenkins Plugin Manager #

So I reached out for help on the Jenkins Community. One of the key contributors to Jenkins, Mark Waite, was tremendously helpful. He suggested using his jenkins-plugin-manager tool. After trying it out for a bit, I realized that this tool had everything I needed for version controlling the plugin versions:

Ability to save a list of installed Jenkins plugins and their versions to a file.
Ability to batch install all the plugins of the versions listed in a file.

This was like doing Python’s plugin management using requirements.txt, except that this was for Jenkins.

Full solution #

With a combination of the .gitignore that I started with from that SO answer, managing plugins using jenkins-plugin-manager, tweaking the .gitignore to my liking, and adding helper Bash scripts for downloading and running Jenkins server binaries, and doing the plugin management, I finally got what I needed:

https://github.com/kaushalmodi/jenkins-minimal

The README on the repo has all the instructions.

jenkins on A Scripter's Notes

Version controlling Jenkins config

What bit me #

.gitignore for Jenkins config #

Jenkins Plugin Manager #

Full solution #

`.gitignore` for Jenkins config #