Two years ago I wrote a blog post to announce that the GHC wasm backend had been merged upstream. I’ve been too lazy to write another blog post about the project since then, but rest assured, the project hasn’t stagnated. A lot of improvements have happened after the initial merge, including but not limited to:

• Many, many bugfixes in the code generator and runtime, witnessed by the full GHC testsuite for the wasm backend in upstream GHC CI pipelines. The GHC wasm backend is much more robust these days compared to the GHC-9.6 era.
• The GHC wasm backend can be built and tested on macOS and aarch64-linux hosts as well.
• Earlier this year, I landed the JSFFI feature for wasm. This lets you call JavaScript from Haskell and vice versa, with seamless integration of JavaScript async computation and Haskell’s green threading concurrency model. This allows us to support Haskell frontend frameworks like reflex & miso, and we have an example repo to demonstrate that.

And…the GHC wasm backend finally supports Template Haskell and ghci!

Show me the code!

$ nix shell gitlab:ghc/ghc-wasm-meta?host=gitlab.haskell.org
$ wasm32-wasi-ghc --interactive
GHCi, version 9.13.20241102: https://www.haskell.org/ghc/  :? for help
ghci>

Or if you prefer the non-Nix workflow:

$ curl https://gitlab.haskell.org/ghc/ghc-wasm-meta/-/raw/master/bootstrap.sh | sh
...
Everything set up in /home/terrorjack/.ghc-wasm.
Run 'source /home/terrorjack/.ghc-wasm/env' to add tools to your PATH.
$ . ~/.ghc-wasm/env
$ wasm32-wasi-ghc --interactive
GHCi, version 9.13.20241102: https://www.haskell.org/ghc/  :? for help
ghci>

Both the Nix and non-Nix installation methods default to GHC HEAD, for which binary artifacts for Linux and macOS hosts, for both x86_64 and aarch64, are provided. The Linux binaries are statically linked so they should work across a wide range of Linux distros.

If you take a look at htop, you’ll notice wasm32-wasi-ghc spawns a node child process. That’s the “external interpreter” process that runs our Template Haskell (TH) splice code as well as ghci bytecode. We’ll get to what this “external interpreter” is about later, just keep in mind that whatever code is typed into this ghci session is executed on the wasm side, not on the native side.

Now let’s run some code. It’s been six years since I published the first blog post when I joined Tweag and worked on a prototype compiler codenamed “Asterius”; the first Haskell program I managed to compile to wasm was fib, time to do that again:

ghci> :{
ghci| fib :: Int -> Int
ghci| fib 0 = 0
ghci| fib 1 = 1
ghci| fib n = fib (n - 2) + fib (n - 1)
ghci| :}
ghci> fib 10
55

It works, though with $O(2^n)$ time complexity. It’s easy to do an $O(n)$ version, using the canonical Haskell fib implementation based on a lazy infinite list:

ghci> :{
ghci| fib :: Int -> Int
ghci| fib = (fibs !!)
ghci|   where
ghci|     fibs = 0 : 1 : zipWith (+) fibs (drop 1 fibs)
ghci| :}
ghci> fib 32
2178309

That’s still boring isn’t it? Now buckle up, we’re gonna do an $O(1)$ implementation… using Template Haskell!

ghci> import Language.Haskell.TH
ghci> :{
ghci| genFib :: Int -> Q Exp
ghci| genFib n =
ghci|   pure $
ghci|     LamCaseE
ghci|       [ Match (LitP $ IntegerL $ fromIntegral i) (NormalB $ LitE $ IntegerL r) []
ghci|       | (i, r) <- zip [0 .. n] fibs
ghci|       ]
ghci|   where
ghci|     fibs = 0 : 1 : zipWith (+) fibs (drop 1 fibs)
ghci| :}
ghci> :set -XTemplateHaskell
ghci> :{
ghci| fib :: Int -> Int
ghci| fib = $(genFib 32)
ghci| :}
ghci> fib 32
2178309

Joking aside, the real point is not about how to implement fib, but rather to demonstrate that the GHC wasm backend indeed supports Template Haskell and ghci now.

Here’s a quick summary of wasm’s TH/ghci support status:

• The patch has landed in the GHC master branch and will be present in upstream release branches starting from ghc-9.12. I also maintain non-official backport branches in my fork, and wasm TH/ghci has been backported to 9.10 as well. The GHC release branch bindists packaged by ghc-wasm-meta are built from my branches.
• TH splices that involve only pure computation (e.g. generating class instances) work. Simple file I/O also works, so file-embed works. Side effects are limited to those supported by WASI, so packages like gitrev won’t work because you can’t spawn subprocesses in WASI. The same restrictions apply to ghci.
• Our wasm dynamic linker can load bytecode and compiled code, but the only form of compiled code it can load are wasm shared libraries. If you’re using wasm32-wasi-ghc directly to compile code that involves TH, make sure to pass -dynamic-too to ensure the dynamic flavour of object code is also generated. If you’re using wasm32-wasi-cabal, make sure shared: True is present in the global config file ~/.ghc-wasm/.cabal/config.
• The wasm TH/ghci feature requires at least cabal-3.14 to work (the wasm32-wasi-cabal shipped in ghc-wasm-meta is based on the correct version).
• Our novel JSFFI feature also works in ghci! You can type foreign import javascript declarations directly into a ghci session, use that to import sync/async JavaScript functions, and even export Haskell functions as JavaScript ones.
• If you have c-sources/cxx-sources in a cabal package, those can be linked and run in TH/ghci out of the box. However, more complex forms of C/C++ foreign library dependencies like pkgconfig-depends, extra-libraries, etc. will require special care to build both static and dynamic flavours of those libraries.
• For ghci, hot reloading and basic REPL functionality works, but the ghci debugger doesn’t work yet.

What happens under the hood?

For the curious mind, -opti-v can be passed to wasm32-wasi-ghc. This tells GHC to pass -v to the external interpreter, so the external interpreter will print all messages passed between it and the host GHC process:

$ wasm32-wasi-ghc --interactive -opti-v
GHCi, version 9.13.20241102: https://www.haskell.org/ghc/  :? for help
GHC iserv starting (in: {handle: <file descriptor: 2147483646>}; out: {handle: <file descriptor: 2147483647>})
[             dyld.so] reading pipe...
[             dyld.so] discardCtrlC
...
[             dyld.so] msg: AddLibrarySearchPath ...
...
[             dyld.so] msg: LoadDLL ...
...
[             dyld.so] msg: LookupSymbol "ghczminternal_GHCziInternalziBase_thenIO_closure"
[             dyld.so] writing pipe: Just (RemotePtr 2950784)
...
[             dyld.so] msg: CreateBCOs ...
[             dyld.so] writing pipe: [RemoteRef (RemotePtr 33)]
...
[             dyld.so] msg: EvalStmt (EvalOpts {useSandboxThread = True, singleStep = False, breakOnException = False, breakOnError = False}) (EvalApp (EvalThis (RemoteRef (RemotePtr 34))) (EvalThis (RemoteRef (RemotePtr 33))))
4
[             dyld.so] writing pipe: EvalComplete 15248 (EvalSuccess [RemoteRef (RemotePtr 36)])
...

Why is any message passing involved in the first place? There’s a past blog post which contains an overview of cross compilation issues in Template Haskell, most of the points still hold today, and apply to both TH as well as ghci. To summarise:

• When GHC cross compiles and evaluates a TH splice, it has to load and run code that’s compiled for the target platform. Compiling both host/target code and running host code for TH is never officially supported by GHC/Cabal.
• The “external interpreter” runs on the target platform and handles target code. Messages are passed between the host GHC and the external interpreter, so GHC can tell the external interpreter to load stuff, and the external interpreter can send queries back to GHC when running TH splices.

In the case of wasm, the core challenge is dynamic linking: to be able to interleave code loading and execution at run-time, all while sharing the same program state. Back when I worked on Asterius, it could only link a self-contained wasm module that wasn’t able to share any code/data with other Asterius-linked wasm modules at run-time.

So I went with a hack: when compiling each single TH splice, just link a temporary wasm module and run it, get the serialized result and throw it away! That completely bypasses the need to make a wasm dynamic linker. Needless to say, it’s horribly slow and doesn’t support cross-splice state or ghci. Though it is indeed sufficient to support compiling many packages that use TH.

Now it’s 2024, time to do it the right way: implement our own wasm dynamic linker! Some other toolchains like emscripten also support dynamic linking of wasm, but there’s really no code to borrow here: each wasm dynamic linker is tailored to that toolchain’s specific needs, and we have JSFFI-related custom sections in our wasm code that can’t be handled by other linkers anyway.

Our wasm dynamic linker supports loading exactly one kind of wasm module: wasm shared libraries. This is something that you get by compiling C with wasm32-wasi-clang -shared, which enables generation of position-independent code. Such machine code can be placed anywhere in the address space, making it suitable for run-time code loading. A wasm shared library is yet another wasm module; it imports the linear memory and function table, and you can specify any base address for memory data and functions.

So I rolled up my sleeves and got to work. Below is a summary of the journey I took towards full TH & ghci support in the GHC wasm backend:

• Step one was to have a minimum NodeJS script to load libc.so: it is the bottom of all shared library dependencies, the first and most important one to be loaded. It took me many cans of energy drink to debug mysterious memory corruptions! But finally I could invoke any libc function and do malloc/free, etc. from the NodeJS REPL, with the wasm instance state properly persisted.
• Then load multiple shared libraries up to libc++.so and running simple C++ snippets compiled to .so. Dependency management logic of shared libraries is added at this step: the dynamic linker traverses the dependency tree of a .so, spawns async WebAssembly.compile tasks, then sequentially loads the dynamic libraries based on their topological order.
• Then figure out a way to emit wasm position-independent-code from GHC’s wasm backend’s native code generator. The GHC native code generator emits a .s assembly file for the target platform, and while assembly format for x86_64 or aarch64, etc. is widely taught, there’s really no tutorial nor blog post to teach me about assembly syntax for wasm! Luckily, learning from Godbolt output examples was easy enough and I quickly figured out how the position-independent entities are represented in the assembly syntax.
• The dynamic linker can now load the Haskell ghci shared library! It contains the default implementation of the external interpreter; it almost worked out of the box, though the linker needed some special logic to handle the piping logic across wasm/JS and the host GHC process.
• In ghci, the logic to load libraries, lookup symbols, etc. are calling into the RTS linker on other platforms. Given all the logic exists on the JS side instead of C for wasm, they are patched to call back into the linker using JSFFI imports.
• The GHC build system and driver needed quite a few adjustments, to ensure that shared libraries are generated for the wasm target when TH/ghci is involved. Thanks to Matthew Pickering for his patient and constructive review of my patch, I was able to replace many hacks in the GHC driver with more principled approaches.
• The GHC driver also needs to learn to handle the wasm flavour of the external interpreter. Thanks to the prior work of the JS backend team here, my life is a lot easier when adding wasm external interpreter logic.
• The GHC testsuite also needed quite a bit of work. In the end, there are over 1000 new test case passes after I flip on TH/ghci support for the wasm target.

What comes next?

The GHC wasm backend TH/ghci feature is way faster and more robust than what I hacked in Asterius back then. One nice example I’d like to show off here is pandoc-wasm: it’s finally possible to compile our beloved pandoc tool to wasm again since Asterius is deprecated.

The new pandoc-wasm is more performant not only at run-time, but also at compile-time. On a GitHub-hosted runner with just 4 CPU cores and 16 GB of memory, it takes around 16min to compile pandoc from scratch, and the time consumption can even be halved on my own laptop with peak memory usage at around 10.8GB. I wouldn’t doubt that time/memory usage can triple or more with legacy GHC-based compilers like Asterius or GHCJS to compile the same codebase!

The work on wasm TH/ghci is not fully finished yet. I do have some things in mind to work on next:

• Support running the wasm external interpreter in the browser via puppeteer. So your ghci session can connect to the browser, all your Haskell code runs in the browser main thread, and all JSFFI logic in your code can access the browser’s window context. This would allow you to do Haskell frontend livecoding using ghci.
• Support running an interactive ghci session within the browser. Which means a truly client side Haskell playground in the browser. It’ll only support in-memory bytecode, since it can’t invoke compiler processes to do any heavy lifting, but it’s still good for teaching purposes.
• Maybe make it even faster? Performance isn’t my concern right now, though I haven’t done any serious profiling and optimization in the wasm dynamic linker either, so we’ll see.
• Fix ghci debugger support.

You’re welcome to join the Haskell wasm Matrix room to chat about the GHC wasm backend. Do get in touch if you feel it is useful to your project!