path: root/posts/2024-06-01-lisp-icing-or-cake.md

title: Lisp: Icing or Cake?
date: 2024-06-01 16:15:00
tags: lisp, scheme, guile, gamedev, gamejam, chickadee
summary: Is Lisp a thin layer on top of the stack or can it be (more of) the stack?
---
The [Spring Lisp Game Jam
2024](https://itch.io/jam/spring-lisp-game-jam-2024) ended one week
ago.  48 games were submitted, a new record for the jam!  This past
week has been a time for participants to play and rate each other’s
games.  As I explored the entries, I noticed two distinct
meta-patterns in how people approached building games with Lisp.  I
think these patterns apply more broadly to all applications of Lisp.
Let’s talk about these patterns in some detail, with examples.

## But first!

Here’s the breakdown of the [jam
submissions](https://itch.io/jam/spring-lisp-game-jam-2024/entries) by
language:

```
lang       entries    % (rounded)
----       -------    -----------
guile      15         31
fennel     10         21
clojure    5          10
cl         5          10
racket     4          8
elisp      4          8
s7         3          6
kawa       1          2
owl        1          2
```

I haven’t rolled up the various Schemes (Guile, Racket, S7, Kawa) into
a general `scheme` category because Scheme is so minimally specified
and they are all very distinct implementations for different purposes,
not to mention that Racket has a lot more going on than just Scheme.

For the first time ever, [Guile](https://gnu.org/software/guile) came
out on top with the most submissions!  There’s a very specific reason
for this outcome.  11 out of the 15 Guile games were built for the web
with [Hoot](https://spritely.institute/hoot), a Scheme-to-WebAssembly
compiler that I work on at the [Spritely
Institute](https://spritely.institute).  2 of those 11 were official
Spritely projects.  We [put out a
call](https://spritely.institute/news/make-a-game-with-hoot-for-the-lisp-game-jam.html)
for people to try making games with Hoot before the jam started, and a
lot of people took us up on it!  Very cool!

The next most popular language, which is typically *the* most popular
language in these jams, is [Fennel](https://fennel-lang.org/).  Fennel
is a Lisp that compiles to Lua.  It’s very cool, too!

Also of note, at least to me as a Schemer, is that three games used
[S7](https://ccrma.stanford.edu/software/snd/snd/s7.html).  Hmm, there
might be something relevant to this post going on there.

The patterns I’m about to talk about could sort of be framed as “The
Guile Way vs. The Fennel Way”, but I don’t want to do that.  It's not
an “us vs. them” thing.  It’s wonderful that there are so many flavors
of Lisp these days that anyone can find a great implementation that
suits their preferences.  Not only that, but many of these
implementations can be used to make games that anyone can easily play
in their web browser!  That was *not* the case several years ago.
Incredible!

I want to preface the rest of this post by saying that **both patterns
are valid**, and while I prefer one over the other, that is not to say
that the other is inferior.  I'll also show how these patterns can be
thought of as two ends of a spectrum and how, in the end, compromises
must be made.  Okay, let’s get into it!

## Lisp as icing

The icing pattern is using Lisp as a “scripting” language on top of a
cake that is made from C, Rust, and other static languages.  The
typical way to do this is by embedding a Lisp interpreter into the
larger program.  If you’re most interested in writing the high-level
parts of an application in Lisp then this pattern is the fastest way
to get there.  All you need is a suitable interpreter/compiler and a
way to add the necessary hooks into your application.  Since the
program is mainly C/Rust/whatever, you can then use emscripten to
compile it to WebAssembly and deploy to the web.  Instant
gratification, but strongly tied to static languages and their
toolchains.

S7 is an example of an embeddable Scheme.  Guile is also used for
extending C programs, though typically that involves dynamically
linking to libguile rather than embedding the interpreter into the
program’s executable.  Fennel takes a different approach, recognizing
that there are many existing applications that are already extensible
through Lua, and provides a lispy language that *compiles to* Lua.

## Lisp as cake

The cake pattern is using Lisp to implement as much of the software
stack as possible.  It’s Lisp all the way down... sorta.  Rather than
embedding Lisp into a non-Lisp program, the cake pattern does the
inverse: the majority of the program is written in Lisp.  When
necessary, shared libraries can be called via a foreign function
interface, but this should be kept to a minimum.  This approach takes
longer to yield results.  Time is spent implementing missing libraries
for your Lisp of choice and writing wrappers around the C shared
libraries you can’t avoid using.  Web deployment gets trickier, too,
since the project is not so easily emscriptenable.

(You may recognize this as the classic embed vs. extend debate.
You’re correct!  I'm just adding my own thoughts and applying it
specifically to some real-world Lisp projects.)

I mentioned Guile as an option for icing, but Guile really shines best
as cake.  The initial vision for Guile was to Emacsify other programs
by adding a Scheme interpreter to them.  These days, the best practice
is to write your program in Scheme to begin with.  Common Lisp is
probably the best example, though.  Implementations like SBCL have
good C FFIs and can compile efficient native executables, minimizing
the desire to use some C for performance reasons.

## Case studies

Let’s take a look at some of the languages and libraries used for the
Lisp Game Jam and evaluate their icing/cake-ness.

### Fennel + love2d

[love2d](https://love2d.org/) has been a popular choice for solo or
small team game development for many years.  It is a C++ program that
embeds a Lua interpreter, which means it’s a perfect target for
Fennel.  Most Linux distributions package love2d, so it’s easy to run
`.love` files natively.  Additionally, thanks to emscripten, love2d
games can be deployed to the web.  Thus most Fennel games use love2d.
[./soko.bin](https://jleightcap.itch.io/sokobin) and [Gnomic
Vengeance](https://alexjgriffith.itch.io/gnomic-vengeance) are two
games that use this stack.

Fennel + love2d is a perfect example of Lisp as icing.  Fennel sits at
the very top of the stack, but there’s not really a path to spread
Lisp into the layers below.  It is also the most successful Lisp game
development stack to date.

### S7 + raylib

This stack is new to me, but two games used it this time around:
[GhostHop](https://gcmas.itch.io/ghosthop) and [Life
Predictor](https://illusion-fisherman.itch.io/life-predictor).  (You
really gotta play GhostHop, btw.  It’s a great little puzzle game and
it is playable on mobile devices.)  [Raylib](https://www.raylib.com/)
is a C library with bindings for many higher-level languages that has
become quite popular in recent years.  S7 is also implemented in C and
is easily embeddable.  This makes the combination easy to deploy on
the web with emscripten.

S7 + raylib is another example of Lisp as icing.  I’m curious to see
if this stack becomes more popular in future jams.

### Guile + Chickadee

This is the stack that I helped build.
[Chickadee](/projects/chickadee.html) is a game library for Guile that
implements almost all of the interesting parts in Scheme, including
rendering.  Two games were built with Chickadee in the most recent
jam: [Turbo Racer 3000](https://etenil.itch.io/turbo-racer-3000) and
[Bloatrunner](https://snamellit.itch.io/bloatrunner).

Guile + Chickadee is an example of Lisp as cake.  Chickadee wraps some
C libraries for low-level tasks such as loading images, audio, and
fonts, but it is written in pure Scheme.  All the matrix and vector
math is in Scheme.  Chickadee comes with a set of rendering primitives
comparable to love2d and raylib but they’re all implemented in Scheme.
I’ve even made progress on rendering vector graphics with Scheme,
whereas most other Lisp game libraries use a C library such as
[nanosvg](https://github.com/memononen/nanosvg).  Chickadee has pushed
the limits of Guile’s compiler and virtual machine, and Guile has been
improved as a result.  But it’s the long road.  Chickadee is mostly
developed by me, alone, in my very limited spare time.  It is taking a
long time to reach feature parity with more popular game development
libraries, but it works quite well for what it is.

### Hoot + HTML5 canvas

I also helped build this one.  Hoot is a Scheme-to-WebAssembly
compiler.  Rather than compile the Guile VM (written in C) to Wasm
using emscripten, Hoot implements a complete Wasm toolchain and a new
backend for Guile’s compiler that emits Wasm directly.  Hoot is
written entirely in Scheme.  Unlike C programs compiled with
emscripten that target Wasm 1.0 with linear memory, Hoot targets Wasm
2.0 with GC managed heap types.  This gives Hoot a significant
advantage: Hoot binaries **do not ship a garbage collector** and thus
are much smaller than Lisp runtimes compiled via emscripten.  The Wasm
binary for my game weighs in at < 2MiB whereas the love2d game I
checked had a nearly 6MiB `love.wasm`.  Hoot programs can also easily
interoperate with JavaScript.  Scheme objects can easily be passed to
JavaScript, and vice versa, as they are managed in the same heap.
With all of the browser APIs just a Wasm import away, an obvious
choice for games was the built-in HTML5 canvas API for easy 2D
rendering.

11 games used Hoot in the jam, including (shameless plug)
[Cirkoban](https://davexunit.itch.io/cirkoban) and [Lambda
Dungeon](https://fluxharmonic.itch.io/lambda-dungeon).

Hoot + HTML5 canvas is mostly dense cake with a bit of icing.  On one
hand, it took a year and significant funding to [boot
Hoot](https://wingolog.org/archives/2024/05/16/on-hoot-on-boot).  We
said “no” to emscripten, built [our own
toolchain](https://wingolog.org/archives/2024/05/24/hoots-wasm-toolkit),
and extended Guile’s compiler.  It's Lisp all the way until you hit
the browser runtime!  We even have a Wasm interpreter that runs on the
Guile VM!  Hoot rules!  It was a risk but it paid off.  On the other
hand, the canvas API is very high-level.  The more cake thing to do
would be to use Hoot’s JS FFI to call WebGL and/or WebGPU.  Indeed,
this is the plan for the future!  Wasm GC needs some improvements to
make this feasible, but my personal goal is to get Chickadee ported to
Hoot.  I want Chickadee games to be easy to play natively and in
browsers, just like love2d games.

## The cake/icing spectrum

I must acknowledge the limitations of the cake approach.  We’re not
living in a world of Lisp machines, but a world of glorified PDP-11s.
Even the tallest of Lisp cakes sits atop an even larger cake made
mostly of C.  All modern Lisp systems bottom out at some point.  Emacs
rests on a C core.  Guile’s VM is written in C.  Hoot runs on mammoth
JavaScript engines written in C++ like V8.  Games on Hoot currently
render with HTML5 canvas rather than WebGL/WebGPU.  Good luck using
OpenGL without libGL; Chickadee uses guile-opengl which uses the C FFI
to call into libGL.  Then there’s libpng, FreeType, and more.  Who the
heck wants to rewrite all this in Lisp?  Who even has the resources?
Does spending all this time taking the scenic route matter at all, or
are we just deluding ourselves because we have fun writing Lisp code?

I think it *does* matter.  Every piece of the stack that can be
reclaimed from the likes of C is a small victory.  The parts written
in Lisp are much easier to hack on, and some of those things become
**live hackable** while our programs are running.  They are also
memory safe, typically, thanks to GC managed runtimes.  Less FFI calls
means less overhead from traversing the Lisp/C boundary and more
safety.  As more of the stack becomes Lisp, it starts looking less
like icing and more like cake.

Moving beyond games, we can look to the [Guix](https://guix.gnu.org)
project as a great example of just how tasty the cake can get.  Guix
took the functional packaging model from the Nix project and made a
fresh implementation, replacing the Nix language with Guile.  Why?
For code staging, code sharing, and improved hackability.  Guix also
uses an init system written in Guile rather than systemd.  Why?  For
code staging, code sharing, and improved hackability.  These are real
advantages that make the trade-off of not using the industry-standard
thing worth it.

I’ve been using Guix since the early days, and back then it was easy
to make the argument that Guix was just reinventing wheels for no
reason.  But now, over 10 years later, the insistence on maximizing
the usage of Lisp has been key to the success of the project.  As a
user, once you learn the Guix idioms and a bit of Guile, you unlock
extraordinary power to craft your OS to your liking.  It’s the closest
thing you can get to a Lisp machine on modern hardware.  The cake
approach paid off for Guix, and it could pay off for other projects,
too.

If Common Lisp is more your thing, and even if it isn’t, you’ll be
amazed by the [Trial](https://shirakumo.github.io/trial/) game engine
and how much of it is implemented in Common Lisp rather than wrapping
C libraries.

There’s also projects like [Pre-Scheme](https://prescheme.org/) that
give me hope that one day the layers below the managed GC runtime can
be implemented in Lisp.  Pre-Scheme was developed and successfully
used for [Scheme 48](https://s48.org/) and I am looking forward to a
modern revival of it thanks to an [NLnet
grant](https://nlnet.nl/project/Pre-Scheme/).

## I'm a cake boy

That’s right, I said it: I’m a cake boy.  I want to see projects
continue to push the boundaries of what Lisp can do.  When it comes to
the Lisp Game Jam, what excites me most are not the games themselves,
but the small advances made to reclaim another little slice of the
cake from stale, dry C.  I intend to keep pushing the limits for Guile
game development with my Chickadee project.

It’s *not* a piece of cake to bake a lispy cake, and the way is often
hazy, but I know we can’t be lazy and just do the cooking by the book.
Rewrite it in Rust?  No way!  Rewrite it in Lisp!