A Scripter's Notes ❚
Emacs, scripting and anything text oriented.

Quick Intro to Emacs Lisp Regression Testing

— Kaushal Modi

A quick introduction to using ERT (Emacs Lisp Regression Testing) for your next elisp library.

Table of Contents

ERT is a testing library that comes with Emacs i.e. you do not need to install from GNU Elpa or Melpa. The aim of this post is to serve as a quick guide on how to start using ERT with your Emacs Lisp library  Here, a “library” could even be a set of custom Emacs Lisp code that you wrote for your Emacs config. .

The ERT manual  You can quickly visit the ERT manual from within Emacs using the wonderful Info system by typing C-h i m ert, you can visit Emacs Manual – ERT in a web browser. is really well-written and here’s how it introduces this testing library:

ERT is a tool for automated testing in Emacs Lisp. Its main features are facilities for defining tests, running them and reporting the results, and for debugging test failures interactively.

..it works well both for test-driven development and for traditional software development methods.

The part about test-driven development is so true! At the moment, I am developing an Emacs library called baser that does signed number conversions among base 10, base 2 and base 16 formats. Before adding new features, I first write the ert tests, and then I develop and refine the functions until they start passing.

In this post, I will use that baser library as an example of how to set up ERT – which I believe would be applicable to other Emacs Lisp projects too.

Test directory #

Below shows how the test/ directory is created with respect to the library being tested.

baser/                (repo root)
├── baser.el
├── ..
├── test/
│  ├── all-tests.el
│  ├── ..
│  └── tdec-hex.el
└── Makefile
Code Snippet 1: Test directory structure from baser

Here,

  1. baser.el is the Emacs Lisp library being tested.
  2. test/ is the directory containing all the tests.
  3. Makefile defines a test target so that running tests is as easy as make test.

Test Files #

In the test/ directory, I have a main all-tests.el file that requires all other test files named in t***.el style.

Feature-specific t***.el test files #

I like to follow the convention of starting all test file names with t.

I break up the test files such that each file tests only a particular feature. In baser, I have tdec-hex.el to test the conversions between decimal and hexidecimal formats. Similarly I have tdec-bin.el and thex-bin.el.

General structure of each feature test #

The structure of each tfeature.el file looks like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

;; tfeature.el                    -*- lexical-binding: t; -*-

;; Require the emacs-lisp library being tested.
;; Example: (require 'baser)

;; Test things that *should* happen.
(ert-deftest addition-test ()
  (should (= (+ 1 2) 3)))

;; Test things that your library *should trigger errors* for.
(ert-deftest div-by-0-test ()
  (should-error (/ 0 0)
                :type 'arith-error))


(provide 'tfeature)
Code Snippet 2: Structure of each tfeature.el test file
  • Lines 7 and 11 define the ert tests using ert-deftest.
  • The first ert_deftest uses a should macro that checks if its body evaluates to non-nil. It is similar to constructs like assert (something == 1); or doAssert something == true in other programming languages.
  • The second ert_deftest​ uses kind the should-error macro that asserts if the evaluation of its body resulted in an error. Here, the test fails if the body does not throw an error. The optional :type <error type> argument makes the test stricter; in addition to checking if evaluating that body throws an error, it also checks if the thrown error is of that specific type.

ert does not need to be required manually because ert-deftest will autoload it.

Wrapper test file all-tests.el #

A wrapper test file all-tests.el is created for convenience so that loading just this one file loads all the tests. Having such a wrapper file is helpful as you will see in the Makefile section next.

This file simply requires all the feature test files like the ones explained above.

;; all-tests.el

;; If the directory happens to have both compiled and uncompiled
;; version, prefer to use the newer (typically the uncompiled) version.
(setq load-prefer-newer t)

(require 'tdec-hex)  ;Require the dec<->hex conversion feature test
;; more feature tests ..
Code Snippet 3: Wrapper test file all-tests.el to require all other tests

Makefile #

Finally, we create a Makefile so that we can run make test to run the ERT tests.

  1. Run make test to run all tests.
  2. Run make test MATCH=foo to run only the tests whose names match “foo”. The test names are the ones defined by the ert-deftest macro.

EMACS ?= emacs

TEST_DIR=$(shell pwd)/test

# Run all tests by default.
MATCH ?=

.PHONY: test

test:
    $(EMACS) --batch -L . -L $(TEST_DIR) -l all-tests.el -eval '(ert-run-tests-batch-and-exit "$(MATCH)")'
Code Snippet 4: make test recipe for running ERT tests

See ERT – Running Tests in Batch Mode for reference.

Summary #

You can quickly create a nice test setup for your Emacs Lisp library or Emacs config by adding a test file with tests defined using erf-deftest with should and should-error macros, and then running make test with the help of a Makefile like the one shown above.

While this post might help you get started with using ERT quickly, do go through its manual — it’s short and sweet, and well-written, and covers a lot more than this post.

References #