This is the first in a series of blog posts intended to provide a gentle introduction to flakes, a new Nix feature that improves reproducibility, composability and usability in the Nix ecosystem. This blog post describes why flakes were introduced, and give a short tutorial on how to use them.

Flakes were developed at Tweag and funded by Target Corporation and Tweag.

What problems do flakes solve?

Once upon a time, Nix pioneered reproducible builds: it tries hard to ensure that two builds of the same derivation graph produce an identical result. Unfortunately, the evaluation of Nix files into such a derivation graph isn’t nearly as reproducible, despite the language being nominally purely functional.

For example, Nix files can access arbitrary files (such as ~/.config/nixpkgs/config.nix), environment variables, Git repositories, files in the Nix search path ($NIX_PATH), command-line arguments (--arg) and the system type (builtins.currentSystem). In other words, evaluation isn’t as hermetic as it could be. In practice, ensuring reproducible evaluation of things like NixOS system configurations requires special care.

Furthermore, there is no standard way to compose Nix-based projects. It’s rare that everything you need is in Nixpkgs; consider for instance projects that use Nix as a build tool, or NixOS system configurations. Typical ways to compose Nix files are to rely on the Nix search path (e.g. import <nixpkgs>) or to use fetchGit or fetchTarball. The former has poor reproducibility, while the latter provides a bad user experience because of the need to manually update Git hashes to update dependencies.

There is also no easy way to deliver Nix-based projects to users. Nix has a “channel” mechanism (essentially a tarball containing Nix files), but it’s not easy to create channels and they are not composable. Finally, Nix-based projects lack a standardized structure. There are some conventions (e.g. shell.nix or release.nix) but they don’t cover many common use cases; for instance, there is no way to discover the NixOS modules provided by a repository.

Flakes are a solution to these problems. A flake is simply a source tree (such as a Git repository) containing a file named flake.nix that provides a standardized interface to Nix artifacts such as packages or NixOS modules. Flakes can have dependencies on other flakes, with a “lock file” pinning those dependencies to exact revisions to ensure reproducible evaluation.

The flake file format and semantics are described in a NixOS RFC, which is currently the best reference on flakes.

Trying out flakes

Flakes are currently implemented in an experimental branch of Nix. If you want to play with flakes, you can get this version of Nix from Nixpkgs:

$ nix-shell -I nixpkgs=channel:nixos-21.05 --packages nixUnstable

Since flakes are an experimental feature, you also need to add the following line to ~/.config/nix/nix.conf:

experimental-features = nix-command flakes

or pass the flag --experimental-features 'nix-command flakes' whenever you call the nix command.

Using flakes

To see flakes in action, let’s start with a simple Unix package named dwarffs (a FUSE filesystem that automatically fetches debug symbols from the Internet). It lives in a GitHub repository at https://github.com/edolstra/dwarffs; it is a flake because it contains a file named flake.nix. We will look at the contents of this file later, but in short, it tells Nix what the flake provides (such as Nix packages, NixOS modules or CI tests).

The following command fetches the dwarffs Git repository, builds its default package and runs it.

$ nix shell github:edolstra/dwarffs --command dwarffs --version
dwarffs 0.1.20200406.cd7955a

The command above isn’t very reproducible: it fetches the most recent version of dwarffs, which could change over time. But it’s easy to ask Nix to build a specific revision:

$ nix shell github:edolstra/dwarffs/cd7955af31698c571c30b7a0f78e59fd624d0229 ...

Nix tries very hard to ensure that the result of building a flake from such a URL is always the same. This requires it to restrict a number of things that Nix projects could previously do. For example, the dwarffs project requires a number of dependencies (such as a C++ compiler) that it gets from the Nix Packages collection (Nixpkgs). In the past, you might use the NIX_PATH environment variable to allow your project to find Nixpkgs. In the world of flakes, this is no longer allowed: flakes have to declare their dependencies explicitly, and these dependencies have to be locked to specific revisions.

In order to do so, dwarffs’s flake.nix file declares an explicit dependency on Nixpkgs, which is also a flake. We can see the dependencies of a flake as follows:

$ nix flake metadata github:edolstra/dwarffs
Description: A filesystem that fetches DWARF debug info from the Internet on demand
…
github:edolstra/dwarffs/d11b181af08bfda367ea5cf7fad103652dc0409f
├───nix: github:NixOS/nix/3aaceeb7e2d3fb8a07a1aa5a21df1dca6bbaa0ef
│   └───nixpkgs: github:NixOS/nixpkgs/b88ff468e9850410070d4e0ccd68c7011f15b2be
└───nixpkgs: github:NixOS/nixpkgs/b88ff468e9850410070d4e0ccd68c7011f15b2be

So the dwarffs flake depends on a specific version of the nixpkgs flake (as well as the nix flake). As a result, building dwarffs will always produce the same result. We didn’t specify this version in dwarffs’s flake.nix. Instead, it’s recorded in a lock file named flake.lock that is generated automatically by Nix and committed to the dwarffs repository.

Flake outputs

Another goal of flakes is to provide a standard structure for discoverability within Nix-based projects. Flakes can provide arbitrary Nix values, such as packages, NixOS modules or library functions. These are called its outputs. We can see the outputs of a flake as follows:

$ nix flake show github:edolstra/dwarffs
github:edolstra/dwarffs/d11b181af08bfda367ea5cf7fad103652dc0409f
├───checks
│   ├───aarch64-linux
│   │   └───build: derivation 'dwarffs-0.1.20200409'
│   ├───i686-linux
│   │   └───build: derivation 'dwarffs-0.1.20200409'
│   └───x86_64-linux
│       └───build: derivation 'dwarffs-0.1.20200409'
├───defaultPackage
│   ├───aarch64-linux: package 'dwarffs-0.1.20200409'
│   ├───i686-linux: package 'dwarffs-0.1.20200409'
│   └───x86_64-linux: package 'dwarffs-0.1.20200409'
├───nixosModules
│   └───dwarffs: NixOS module
└───overlay: Nixpkgs overlay

While a flake can have arbitrary outputs, some of them, if they exist, have a special meaning to certain Nix commands and therefore must have a specific type. For example, the output defaultPackage.<system> must be a derivation; it’s what nix build and nix shell will build by default unless you specify another output. The nix CLI allows you to specify another output through a syntax reminiscent of URL fragments:

$ nix build github:edolstra/dwarffs#checks.aarch64-linux.build

By the way, the standard checks output specifies a set of derivations to be built by a continuous integration system such as Hydra. Because flake evaluation is hermetic and the lock file locks all dependencies, it’s guaranteed that the nix build command above will evaluate to the same result as the one in the CI system.

The flake registry

Flake locations are specified using a URL-like syntax such as github:edolstra/dwarffs or git+https://github.com/NixOS/patchelf. But because such URLs would be rather verbose if you had to type them all the time on the command line, there also is a flake registry that maps symbolic identifiers such as nixpkgs to actual locations like https://github.com/NixOS/nixpkgs. So the following are (by default) equivalent:

$ nix shell nixpkgs#cowsay --command cowsay Hi!
$ nix shell github:NixOS/nixpkgs#cowsay --command cowsay Hi!

It’s possible to override the registry locally. For example, you can override the nixpkgs flake to your own Nixpkgs tree:

$ nix registry add nixpkgs ~/my-nixpkgs

or pin it to a specific revision:

$ nix registry add nixpkgs github:NixOS/nixpkgs/5272327b81ed355bbed5659b8d303cf2979b6953

Writing your first flake

Unlike Nix channels, creating a flake is pretty simple: you just add a flake.nix and possibly a flake.lock to your project’s repository. As an example, suppose we want to create our very own Hello World and distribute it as a flake. Let’s create this project first:

$ git init hello
$ cd hello
$ echo 'int main() { printf("Hello World"); }' > hello.c
$ git add hello.c

To turn this Git repository into a flake, we add a file named flake.nix at the root of the repository with the following contents:

{
  description = "A flake for building Hello World";

  inputs.nixpkgs.url = github:NixOS/nixpkgs/nixos-20.03;

  outputs = { self, nixpkgs }: {

    defaultPackage.x86_64-linux =
      # Notice the reference to nixpkgs here.
      with import nixpkgs { system = "x86_64-linux"; };
      stdenv.mkDerivation {
        name = "hello";
        src = self;
        buildPhase = "gcc -o hello ./hello.c";
        installPhase = "mkdir -p $out/bin; install -t $out/bin hello";
      };

  };
}

The command nix flake init creates a basic flake.nix for you.

Note that any file that is not tracked by Git is invisible during Nix evaluation, in order to ensure hermetic evaluation. Thus, you need to make flake.nix visible to Git:

$ git add flake.nix

Let’s see if it builds!

$ nix build
warning: creating lock file '/home/eelco/Dev/hello/flake.lock'
warning: Git tree '/home/eelco/Dev/hello' is dirty

$ ./result/bin/hello
Hello World

or equivalently:

$ nix shell --command hello
Hello World

It’s also possible to get an interactive development environment in which all the dependencies (like GCC) and shell variables and functions from the derivation are in scope:

$ nix develop
$ eval "$buildPhase"
$ ./hello
Hello World

So what does all that stuff in flake.nix mean?

• The description attribute is a one-line description shown by nix flake metadata.

• The inputs attribute specifies other flakes that this flake depends on. These are fetched by Nix and passed as arguments to the outputs function.

• The outputs attribute is the heart of the flake: it’s a function that produces an attribute set. The function arguments are the flakes specified in inputs.

  The self argument denotes this flake. Its primarily useful for referring to the source of the flake (as in src = self;) or to other outputs (e.g. self.defaultPackage.x86_64-linux).

• The attributes produced by outputs are arbitrary values, except that (as we saw above) there are some standard outputs such as defaultPackage.${system}.

• Every flake has some metadata, such as self.lastModifiedDate, which is used to generate a version string like hello-20191015.

You may have noticed that the dependency specification github:NixOS/nixpkgs/nixos-20.03 is imprecise: it says that we want to use the nixos-20.03 branch of Nixpkgs, but doesn’t say which Git revision. This seems bad for reproducibility. However, when we ran nix build, Nix automatically generated a lock file that precisely states which revision of nixpkgs to use:

$ cat flake.lock
{
  "nodes": {
    "nixpkgs": {
      "info": {
        "lastModified": 1587398327,
        "narHash": "sha256-mEKkeLgUrzAsdEaJ/1wdvYn0YZBAKEG3AN21koD2AgU="
      },
      "locked": {
        "owner": "NixOS",
        "repo": "nixpkgs",
        "rev": "5272327b81ed355bbed5659b8d303cf2979b6953",
        "type": "github"
      },
      "original": {
        "owner": "NixOS",
        "ref": "nixos-20.03",
        "repo": "nixpkgs",
        "type": "github"
      }
    },
    "root": {
      "inputs": {
        "nixpkgs": "nixpkgs"
      }
    }
  },
  "root": "root",
  "version": 5
}

Any subsequent build of this flake will use the version of nixpkgs recorded in the lock file. If you add new inputs to flake.nix, when you run any command such as nix build, Nix will automatically add corresponding locks to flake.lock. However, it won’t replace existing locks. If you want to update a locked input to the latest version, you need to ask for it:

$ nix flake lock --update-input nixpkgs
$ nix build

To wrap things up, we can now commit our project and push it to GitHub, after making sure that everything is in order:

$ nix flake check
$ git commit -a -m 'Initial version'
$ git remote add origin [email protected]:edolstra/hello.git
$ git push -u origin master

Other users can then use this flake:

$ nix shell github:edolstra/hello -c hello

Next steps

In the next blog post, we’ll talk about typical uses of flakes, such as managing NixOS system configurations, distributing Nixpkgs overlays and NixOS modules, and CI integration.

Nix Flakes Series

• Part 1: An introduction and tutorial
• Part 2: Evaluation caching
• Part 3: Managing NixOS systems