One of the key features of Nickel is the merge system, which is a clever way of combining records, and allows defining complex configurations in a modular way. This system is inspired by (amongst others) the NixOS module system, which are the magic bits that tie together NixOS configurations and gives NixOS its insane flexibility. Nickel merging and NixOS modules work slightly differently under the hood, and target slightly different use-cases:

• Nickel merge is designed to combine several pieces of configurations which all respect the same contract. This allows an application developer to define the interface of its configuration, and have the user write it in a modular way.
• NixOS modules, on the other hand, are designed to combine pieces which not only define a part of the final configuration, but also a part of the contract. This is a must-have for big systems like NixOS, where defining the whole schema in one place isn’t possible.

Even in Nickel, it would be sometimes desirable to get the full modularity of NixOS modules. For instance, Organist is based on this idea of having individual pieces, each defining both a part of the final schema and a part of the configuration — the files module will define the interface for declaring custom config files, and hook into the base Nix system to declare a flake app based on it, the editorconfig module declares an interface for managing the .editorconfig file, and hooks into the files interface for the actual generation of the file, etc.

Fortunately, it is actually trivial to implement an equivalent of the NixOS module system in Nickel by leveraging the merge system. Not only that, but this will only be a very light abstraction over built-in features. This means that it will come with the joy of being understood by the LSP, giving nice auto-completion, quick and relevant error messages, and so on. Also, the lightness of the abstraction makes it very flexible, allowing to easily build variants of the system.

Before explaining how that works, let’s define a bit more precisely what we want with a module system.

Module systems

A “module system”, in the NixOS sense, is a programming paradigm which allows exploding a complex configuration into individual components. Each component (a “module”) can define both a piece of the schema for the overall configuration, and a piece of the configuration. The schemas of all the modules are combined to form the final schema, and the same goes for configurations. The only constraint is that the final configuration matches the final schema.

For instance, here is an instantiation of a (very) simplified version of the NixOS module system, written in Nix:

mergeModules {
   module1 = {...}: {
     options.foo = mkOption { type = int; };
     options.bar = mkOption { type = string; };
     config.bar = "world";
   };
   module2 = {config, ...}: {
      options.baz = mkOption { type = string; };
      config.foo = 3;
      config.baz = "Hello " + config.bar;
    };
 }

# Result
=> {
  foo = 3;              # defined by module1, set by module2
  bar = "world";        # defined by module1, set by module1
  baz = "Hello world";  # defined by module2, set by module2 using `bar` from module 1
}

We can see that each module defines both a piece of the final schema (the options field) and a piece of the final config (the config field). They have the ability to set and refer to options defined in another module.

Assigning a value of the wrong type to an option gives an error. For instance, changing config.foo = 3 to config.foo = "3" yields:

error:
       … while evaluating the attribute 'value'
         at /nix/store/axfhzzkixwwdmxlrma7k8f65214acvml-source/lib/modules.nix:809:9:
          808|     in warnDeprecation opt //
          809|       { value = builtins.addErrorContext "while evaluating the option `${showOption loc}':" value;
             |         ^
          810|         inherit (res.defsFinal') highestPrio;

       … while evaluating the option `foo':

       … while evaluating the attribute 'mergedValue'
         at /nix/store/axfhzzkixwwdmxlrma7k8f65214acvml-source/lib/modules.nix:844:5:
          843|     # Type-check the remaining definitions, and merge them. Or throw if no definitions.
          844|     mergedValue =
             |     ^
          845|       if isDefined then

       (stack trace truncated; use '--show-trace' to show the full, detailed trace)

       error: A definition for option `foo' is not of type `signed integer'. Definition values:
       - In `<unknown-file>': "foo"

Shortcomings of the NixOS module system

This system is incredibly powerful, and rightfully serves not only the whole of NixOS, but also a number of other projects in the Nix world, such as home-manager, nix-darwin, flake parts and terranix.

However, it suffers from some limitations, mostly due to it being a pure library-side encoding instead of a Nix built-in.

• The error messages can get quite daunting (just look at the example above). Great effort has been put to improve the Nix error messages (and it shows), but the situation is still far from ideal.
• Because it has nearly zero support from the language side, no LSP server is truly able to understand it, meaning that things like autocompletion or in-editor error messages are very much best-effort, if they exist at all.
• Finally, it is often too dynamic for its own good. The fact that it only cares about global consistency makes it nearly impossible to reason about a module individually. The only way, for instance, to know whether a given module is correct is to evaluate the whole module system to know whether all the options it refers to are defined somewhere. Likewise, it is absolutely impossible to know the exact consequences of flipping an option somewhere as it might have an impact on any other module.

Implementing a (better) module system in Nickel

Given the usefulness of this module system, and its weaknesses, it would be great if we could have the same thing in Nickel – and even better if we could fix the shortcomings on the way.

It turns out that we can, and in a whopping one line of code:

{ Module = { Schema | not_exported = {}, config | Schema } }

This probably needs some explanation, so let’s look at how it works, and how it can be used.

Before anything else, let’s be honest: the simplicity of that line is rather deceptive since it hides the big heavy lifting done by the runtime, in particular the merge and contract systems.

This defines a contract called Module that represents values with:

• A Schema field, which is a record contract;
• A config field, matching the contract defined by Schema.

We can use it as such:

let Module = { Schema | not_exported = {}, config | Schema } in

{
  module1 = {
    Schema.foo | Number,
    Schema.bar | String,
    config.bar = "world",
  },
  module2 = {
    Schema.baz | String,
    config.bar,
    config.foo = 3,
    config.baz = "Hello " ++ config.bar,
  },

  module3 | Module = module1 & module2,
}.module3.config

# Result
=> {
  "bar": "world",
  "baz": "Hello world",
  "foo": 3
}

This is very similar to the Nix example above. What happens if we make a mistake on one of the fields, say replace config.foo = 1 with config.foo = "x"?

$ nickel export main.ncl
error: contract broken by the value of `foo`
   ┌─ /tmp/tmp.U0m6Ro8Vvo/main.ncl:13:18
   │
 5 │     Schema.foo | Number,
   │                  ------ expected type
   ·
13 │     config.foo = "x",
   │                  ^^^ applied to this expression
   │
   ┌─ <unknown> (generated by evaluation):1:1
   │
 1 │ "x"
   │ --- evaluated to this value

So the contract system is aware of what we want to check, and reports us an error accordingly. Better, it points to the right place in the code. Even better, the LSP server is aware of the error, and can point right at it:

Global vs local consistency

This is definitely great, and solves the LSP integration issue, as well as part of the error messages one. What it doesn’t do is alleviate the lack of local consistency: module1 and module2 are still implicitly depending on each other, and the only way to know that is to see that — for instance — module2 sets bar, which is declared in module1.

We can, however, enforce something stricter: In our example, we only require module1 & module2 to be a valid module. But we could require each of them to be individually consistent by enforcing that their config field matches their Schema field:

--- a/main.ncl
+++ b/main.ncl
@@ -4,11 +4,13 @@ let Module = { Schema | not_exported = {}, config | Schema } in
   module1 = {
     Schema.foo | Number,
     Schema.bar | String,
+    config | Schema,
     config.bar = "world",
   },
   module2 = {
     Schema.baz | String,

+    config | Schema,
     config.bar,
     config.foo = 3,
     config.baz = "Hello " ++ config.bar,

Doing so will yield an error:

$ nickel export main.ncl
error: contract broken by the value of `config`
       extra fields `bar`, `foo`
   ┌─ /tmp/tmp.U0m6Ro8Vvo/main.ncl:13:14
   │
13 │     config | Schema,
   │              ------ expected type
   │
   ┌─ <unknown> (generated by evaluation):1:1
   │
 1 │ { bar, baz = %<closure@0x55fa68e718c8>, foo = 3, }
   │ -------------------------------------------------- evaluated to this value
   │
   = Have you misspelled a field?
   = The record contract might also be too strict. By default, record contracts exclude any field which is not listed.
     Append `, ..` at the end of the record contract, as in `{some_field | SomeContract, ..}`, to make it accept extra fields.

Indeed, module2 isn’t consistent. It is using foo and bar, but doesn’t know about them as they are declared in module1. But we can fix that by making it explicitly depend on module1:

--- a/main.ncl
+++ b/main.ncl
@@ -7,7 +7,7 @@ let Module = { Schema | not_exported = {}, config | Schema } in
     config | Schema,
     config.bar = "world",
   },
-  module2 = {
+  module2 = module1 & {
     Schema.baz | String,

     config | Schema,

And now everything gets completely consistent, and we can confidently write a fully modular configuration, with the assurance that the language will have our back and give us early warnings in case anything goes wrong.

Besides, since the language really knows about everything that’s going on, the LSP can do its magic and help us with autocompletion. Let’s add a new option to module1:

--- a/main.ncl
+++ b/main.ncl
@@ -4,6 +4,12 @@ let Module = { Schema | not_exported = {}, config | Schema } in
   module1 = {
     Schema.foo | Number,
     Schema.bar | String,
+    Schema.my_option
+      | { _ : String }
+      | doc m%"
+          The set of things that will wobble up when
+          stuff wiggles down and bubbles sideways
+        "%,
     config | Schema,
     config.bar = "world",
   },

We can now refer to it from module2, and have our editor tell us everything we want to know:

Conclusion

We’ve seen how we can easily implement an equivalent of the NixOS module system on top of Nickel. We’ve also seen how this maps nicely to the semantics of the language and gives us access to all the benefits of good error messages and editor integration.

This is obviously just scratching the surface, and there’s a ton of extra features of NixOS modules that haven’t been covered here. Some would be trivial to implement, like submodules (left as an exercise to the reader). Others would require more work, or even changes to the underlying language like custom types with custom merge functions. However, even this simple formalization is already tremendously powerful. It is used in an experimental branch of Organist to provide a principled way of combining different, independent, but related pieces of functionality.

A great possible follow-up work would be to hook that up with existing NixOS modules (maybe using a mixture of jsonschema-converter and json-schema-to-nickel) to allow writing one’s NixOS configuration directly in Nickel… maybe soon?