Quick Intro to Emacs Lisp Regression Testing— Kaushal Modi
A quick introduction to using ERT (Emacs Lisp Regression Testing) for your next elisp library.
ERT is a testing library that comes with Emacs i.e. you do not need to install from GNU Elpa or Melpa. The aim of this post is to serve as a quick guide on how to start using ERT with your Emacs Lisp library Here, a “library” could even be a set of custom Emacs Lisp code that you wrote for your Emacs config. .
The ERT manual
You can quickly visit the ERT manual from within Emacs using the
wonderful Info system by typing
C-h i m ert, you can visit Emacs
Manual – ERT in a web browser.
is really well-written and here’s how it introduces this testing
ERT is a tool for automated testing in Emacs Lisp. Its main features are facilities for defining tests, running them and reporting the results, and for debugging test failures interactively.
..it works well both for test-driven development and for traditional software development methods.
The part about test-driven development is so true! At the moment, I
am developing an Emacs library called
baser that does signed number
conversions among base 10, base 2 and base 16 formats. Before adding
new features, I first write the
ert tests, and then I develop and
refine the functions until they start passing.
In this post, I will use that baser library as an example of how to set up ERT – which I believe would be applicable to other Emacs Lisp projects too.
Test directory #
Below shows how the
test/ directory is created with respect to the
library being tested.
baser/ (repo root) ├── baser.el ├── .. ├── test/ │ ├── all-tests.el │ ├── .. │ └── tdec-hex.el └── Makefile
baser.elis the Emacs Lisp library being tested.
test/is the directory containing all the tests.
testtarget so that running tests is as easy as
Test Files #
test/ directory, I have a main
all-tests.el file that
requires all other test files named in
t***.el test files #
I like to follow the convention of starting all test file names with
I break up the test files such that each file tests only a particular
feature. In baser, I have
tdec-hex.el to test the conversions
between decimal and hexidecimal formats. Similarly I have
General structure of each feature test #
The structure of each
tfeature.el file looks like this:
- Lines 7 and 11 define the ert tests using
- The first
shouldmacro that checks if its body evaluates to non-nil. It is similar to constructs like
assert (something == 1);or
doAssert something == truein other programming languages.
- The second
ert_deftest uses kind the
should-errormacro that asserts if the evaluation of its body resulted in an error. Here, the test fails if the body does not throw an error. The optional
:type <error type>argument makes the test stricter; in addition to checking if evaluating that body throws an error, it also checks if the thrown error is of that specific type.
ert does not need to be required manually because
will autoload it.
Wrapper test file
A wrapper test file
all-tests.el is created for convenience so that
loading just this one file loads all the tests. Having such a wrapper
file is helpful as you will see in the Makefile section next.
This file simply requires all the feature test files like the ones explained above.
;; all-tests.el ;; If the directory happens to have both compiled and uncompiled ;; version, prefer to use the newer (typically the uncompiled) version. (setq load-prefer-newer t) (require 'tdec-hex) ;Require the dec<->hex conversion feature test ;; more feature tests ..
Finally, we create a
Makefile so that we can run
make test to run
the ERT tests.
make testto run all tests.
make test MATCH=footo run only the tests whose names match “foo”. The test names are the ones defined by the
EMACS ?= emacs TEST_DIR=$(shell pwd)/test # Run all tests by default. MATCH ?= .PHONY: test test: $(EMACS) --batch -L . -L $(TEST_DIR) -l all-tests.el -eval '(ert-run-tests-batch-and-exit "$(MATCH)")'
See ERT – Running Tests in Batch Mode for reference.
You can quickly create a nice test setup for your Emacs Lisp library
or Emacs config by adding a test file with tests defined using
should-error macros, and then
make test with the help of a
Makefile like the one shown
While this post might help you get started with using ERT quickly, do go through its manual — it’s short and sweet, and well-written, and covers a lot more than this post.