Defining tomelr – A library for converting Lisp expressions to TOML— Kaushal Modi
Creating a specification for an Emacs-Lisp library to convert Lisp data expressions into easy-to-read TOML strings.
ox-hugo has some custom code that generates TOML
I ❤️ TOML. As the makers of this config format put it.. “it’s a
format for humans”! — No need to deal with indentations, weird
syntax for multi-line strings, no prohibition on adding comments to
the config, or dealing with careful placement of commas and braces.
for Hugo front-matter, based on the Org keywords and other meta-data
that the user sets in their Org file. But this TOML generation code is
not generic enough for any TOML object type.
As I am writing up a definition on how to export
I felt it’s a good time to polish the whole Lisp data → TOML
conversion code, and may be package that into a separate library.
It’s kind of an ambitious project — I am calling it tomelr ✨
It’s a big undertaking to create a generic library for this kind of data format conversion. But even before I start coding or think if I can complete this project, I need to spec I am not sure if it’s a widely used verb, but to spec means to write a specification for something. it. I need to understand early-on how the S-exp (Symbolic lisp expression) would needed to look for each kind of generated TOML object — scalars, lists, lists of lists, maps, lists of maps, etc.
json-encode as reference #
The aim of the
tomelr library is to take some
(lisp data) and
convert that TOML. But I did not want to invent my own lisp data
convention for this! So I decided to stick with the lisp expression
conventions understood by the
json-encode function from the Emacs
Credit for the
json.el idea goes to this tweet by Piers Cawley:
I’d suggest that By “that”, he’s referring to adding support for exporting front-matter to JSON in
ox-hugo. , since emacs has built in JSON support these days, you don’t really have to worry about the commas and braces, just build the s-exp you want and export, but it’s you that’s writing the code and I’m just delighted that it exists.
Thank you for your efforts.
I am not sold on adding support of yet another front-matter format to
ox-hugo. I might not use
json.el for that, but it definitely
helped me a lot with coming up with this library’s spec 😆.
Mapping scalar data to TOML #
Figuring out the Lisp representation for scalar (plain key-value
pairs) TOML objects was easy.
json.el helped figure out how to deal
with nil and false values.
|(key removed in TOML)|
json.el defines a variable
json-false that’s set to the value
:json-false. This is because in JSON, the null value is different
from the boolean
- nil in Lisp → null in JSON
:json-falsein Lisp →
Inspired by that decision of
json.el, I am thinking of using
:false as the special value that will set the equivalent TOML value
TOML does not define a null value, but if the Lisp value is nil, that key will simply not be translated to TOML.
Mapping lists to TOML #
Mapping lists was simple.. because in Lisp, a list value of course
'((foo . (1 2 3 4 5)))
Mapping lists of lists to TOML #
I was going to use
'((lol_key . ((1 2) (3 4)))) as the reference Lisp expression for
lol_key = [ [1, 2], [3, 4] ]. But I found out that
json-encode throws an error if you pass it that expression! I don’t
understand the reason for that error, and so I have asked for help on
the help-gnu-emacs mailing list.
But while that question gets resolved, I wanted to move forward with
the spec definition. After some trial-and-error and
I knew how I wanted TOML to look. So I used an online JSON/TOML
converter to convert that TOML snippet to JSON, and then used
json-read to convert JSON to Lisp expression.
, I learned that list of list data needs to be represented using a Vector type, and so
'((lol_key . [(1 2) (3 4)])) would be the correct expression – Notice the use
of square brackets instead of parentheses for the outer vector.
Mapping other object types #
Once I figured out how to map the above data types, mapping Lisp data to TOML Tables aka dictionaries and arrays of Tables was a breeze Of course, the breeze is referring to the ease of writing the spec for these 😆. Implementation-wise, the tables, arrays of tables, and the especially nested variants of those are going to be the most challenging. .
It was fun coming up with an initial draft of the specification for
this library. My next steps would be to gradually add TOML generator
functions (as I find time) to this library, along with ERT tests!
Eventually, I will remove the existing TOML generation code from
ox-hugo and depend on this library.
Getting back to my plan for exporting
:LOGBOOK: drawers in
ox-hugo, based on this spec, I will need to construct this date in
(org_logbook . (((timestamp . 2022-04-08T14:53:00-04:00) (note . "This note addition prompt shows up on typing the `C-c C-z` binding.\nSee [org#Drawers](https://www.gnu.org/software/emacs/manual/html_mono/org.html#Drawers).")) ((timestamp . 2018-09-06T11:45:00-04:00) (note . "Another note **bold** _italics_.")) ((timestamp . 2018-09-06T11:37:00-04:00) (note . "A note `mono`."))))
.. will translate to this in TOML:
[[org_logbook]] timestamp = 2022-04-08T14:53:00-04:00 note = """This note addition prompt shows up on typing the `C-c C-z` binding. See [org#Drawers](https://www.gnu.org/software/emacs/manual/html_mono/org.html#Drawers).""" [[org_logbook]] timestamp = 2018-09-06T11:45:00-04:00 note = """Another note **bold** _italics_.""" [[org_logbook]] timestamp = 2018-09-06T11:37:00-04:00 note = """A note `mono`."""
Check out the below link where I have documented the equivalent expressions between Lisp, TOML and JSON for all the object types.