Tweag

The right words in the right place

2 May 2024 — by Valentin Gagarin

tl;dr You may not believe it, but Nix documentation is getting better. Nixpkgs and NixOS still need more time.

Table of contents

Overview

This is a retrospective of my and many other people’s work on documentation in the Nix ecosystem between October 2022 and March 2024. It serves as a showcase of what we achieved together, and gives an impression of what’s involved in improving the user experience in a complex software system. A lot has happened during that time, so this text is quite lengthy.

The details of this report will mainly be interesting to power users and active contributors, or people working on software projects that are in a similar situation as Nix. For everyone else, I’d summarise the effort so far as “success in slow motion that indicates compounding effects”. Much of it was made possible through ongoing sponsorship from Antithesis, Tweag, and many individuals, along with the incredible commitment of numerous volunteers.

  • nix.dev is now the official documentation hub for the Nix ecosystem, designed to follow the Diátaxis documentation framework.
  • There are a number of new in-depth tutorials and practical guides, with more in the making.
  • The contribution process got smoother and is documented much better.
  • The Nix manual substantially improved in terms of structure, clarity, and detail. It can now be accessed by release version on nix.dev.
  • The Nixpkgs and NixOS manuals are now entirely written in Markdown, after two years of migrating away from XML.
  • More people than ever have contributed improvements to documentation.
  • The official NixOS Wiki has launched under wiki.nixos.org.

It may not seem like much, but these are big changes that posed enough of a challenge to arrive at. It’s still not easy or fast to learn all the things needed to wield the full power of Nix. However, there are reports of people grasping most of it within a couple of weeks on their own, which would be a significant improvement over what previously seemed to take months. And as more contributors chime in, things are continuously getting better.

The next steps will be expanding reference documentation, developing more suitable technical infrastructure, adding more tutorials, shaping an information architecture that connects the ecosystem… and simply figuring a lot of things out and writing them down.

Motivation

What originally compelled me to start working on improving Nix documentation was a vague feeling that there was something profound about Nix, something that was merely obscured by confusing explanations. It wasn’t just the fact that the Nix ecosystem permanently solves many problems in computing I believe should never have existed in the first place. It was the question: What underlying principle makes it possible for it to do so? Being in the right place at the right time often enough has allowed me to pursue a few hunches as part of my day job at Tweag for the past two years. And that time was needed.

First, it took me a while to fully grasp how beautifully simple Nix is conceptually:

  • The Nix store has a computation engine that operates on file system trees, disguised as a system for caching sandboxed execve calls.
  • The Nix language’s most distinguishing features are, not coincidentally, first-class facilities for dealing with file system paths and strings.
  • Nixpkgs is the world’s largest knowledge base for getting open source software to run.
  • NixOS offers a uniform user interface for configuring that open source software.

Second, after staring at it long enough, it turns out that the underlying powerful idea is programming itself! More precisely, disciplined programming:

  • The Nix store enforces referential integrity on the file system and constrains processes to act like pure functions, and it facilitates distributed, incremental computation. Read more on that in Taming Unix with functional programming.
  • Nixpkgs is written in the purely functional Nix language, and keeps scaling due to a substantial amount of automation.
  • The magic behind NixOS (and related tools, such as Home Manager) is the module system, which combines a rich type system with modular composition. Together with virtual machines, it enables systematic integration testing of a large part of the ecosystem’s artifacts.

From the outside, all of this seems terribly messy. It has grown organically over 20 years, and most of it is still only sparsely documented. Some parts are hard to understand; but that’s often because programming can be fundamentally hard. And it doesn’t help that things are just as messy on the inside. There are places where I’d say “the code doesn’t understand itself”.

The same things that make Nix hard to learn and use also make it hard to document and teach: There are many ways of doing certain things with Nix, and often none of them is clearly superior. A large part of that problem must eventually be solved at the implementation level, but that is subject to the Lisp Curse. In a volunteer-driven open source software community, many technical issues reduce to social issues. The real challenge is coordination!

That’s not exactly what I had originally subscribed to. But in retrospect, maybe it’s not surprising that, apart from finding the right words and the right place for them, the path to better documentation is paved with debates, design documents, fundraisers, UX workshops, governance meetings — that is, listening and talking to lots of people.

Making the ecosystem more approachable for a mass audience may as well end up to be teaching “disciplined software development with Nix”. But even that will require us, as a community-of-practice, to agree on (rather than just find) answers to many open questions. Until then, this report is meant to be a panorama of what that process has entailed so far, from my perspective. I hope that it will encourage you to try for yourself and participate, or otherwise help out.

Statistics

Before getting into a lengthy narrative, let’s look at recorded documentation activities of the past two years. tl;dr: numbers go up, this is good.

The following three charts show the number of documentation-related pull requests merged in the three repositories the team has worked on: Nix, Nixpkgs, and nix.dev. The charts have the same vertical scale and each bar stands for a quarter of a year since the beginning of 2022. Documentation work on Nix is underrepresented prior to Q3, since we only then began systematically labeling those pull requests. For nix.dev all pull requests are counted. The most recent period accounts for at most half of the quarter, because this is when I compiled the data.

Notably, Nixpkgs documentation activities have grown roughly with (and possibly even slower than) the overall number of users and contributors. This sample shows all pull requests automatically labeled as being related to documentation, which often comes with package updates and are often inconsequential in practice.

In contrast, the frequency of changes in nix.dev and the Nix manual greatly increased since the documentation team was founded.

255f52a1 612d 4224 b3b9 865c089371dc

3d3dc368 cf07 4873 94a0 fd788ce52b37

84480102 9633 48d2 9551 d00a2dff3751

The next three charts show the number of GitHub issues related to documentation that were opened or closed in a given quarter, again for Nix, Nixpkgs, and nix.dev. These charts also have the same scale. Here, one can observe that Nixpkgs documentation activities, while more pronounced overall due to the greater number of people involved, are significantly driven by the team. There is also some up-and-down due to alternating between more exploratory and more implementation-oriented periods.

eb0e2611 35aa 42d1 ba77 98c18faca5e2

3c0136b6 82a3 441e 9a5a 0a2d7dadd237

b618968c c4a3 4ef0 9058 7fec0d662c43

The final three charts show the number of new contributors and those who made repeated contributions related to documentation, again per quarter and for Nix, Nixpkgs, and nix.dev. These charts are not to scale with each other because including Nixpkgs would have made the others unreadable. This is the most notable change, and in my opinion the most important success: We now have around 10 people working consistently on Nix reference documentation and introductory materials — more than ever!

contributors nix

contributors nixpkgs

contributors nix dev

And now for the full story, which is long because we have made and learned so many things.

Retrospective

In 2022, I started out with the hypothesis that there needs to be a comprehensive document that one could read cover to cover to understand the Nix ecosystem. With Nix inventor Eelco Dolstra’s blessing and Tweag’s funding I started an effort to write what we called The Nix Book.

It turned out not to be that simple at all. The ecosystem is too large, the spectrum of use cases too wide, and the details too messed up to just sit down and blurt out everything one knows, and then wait for others to fix the typos. In fact, there have been multiple such attempts by different people.

A more systematic approach was needed. In addition to an initial literature survey, I also ran a small usability study and evaluated the 2022 community polls. During that time, the documentation team was founded, and became the forum for developing and refining goals for documentation in the Nix ecosystem, and working towards their achievement.

As of today, the documentation team meeting was held more than a hundred times. I’m convinced: we’re delivering. You can feel the change in pace and atmosphere, and the numbers support it.

Over the period covered by this report, we formulated multiple sets of goals. They reflect our evolving understanding of the problem space and shifts in priorities.

Because all other attempts to make a comprehensive tractable review turned out to be impractical, I will first list the goals in the order they were published, for reference. Then, based on the statements reworded and regrouped to avoid overlap, I will present results for each category separately.

From the proposal for The Nix Book April 2022
  • Write a book actually explaining Nix and its surrounding ecosystem at a high level of abstraction
  • Overhaul the Nix manual to make it a focused technical reference
  • Improve discoverability of existing learning material
From the documentation team announcement August 2022
  • Ease Nix learning, increase onboarding success and user retention
  • Improve organisation of Nix knowledge
  • Lead, guide, and support community efforts
From my NixCon talk Flattening the learning curve October 2022
  • Reference documentation should be complete, correct, and easy to update
  • Guides should cover all major use cases
  • Tutorials should teach all key skills necessary to solve problems on one’s own, and require minimal effort from learners
  • Documentation should be versioned in lockstep with the code being documented
  • Each area should have a dedicated maintainer; someone who knows the whole thing by heart and knows where to put what
From the call for maintainers March 2023
  • Ease onboarding for users of Nix tools and contributors to their documentation:
    • Improve discoverability of relevant documentation for major use cases
    • Find dedicated owners for each part of the documentation
    • Markdown everywhere
  • Create a coherent vision for documentation in the Nix ecosystem, and derive an implementation strategy and roadmap, guided by the Diátaxis framework for technical documentation:
    1. Reference: Design an information architecture for reference documentation
    2. Tutorials: Draft a complete onboarding and learning journey
    3. Guides: Define a curation model for how-to guides
    4. Explanation: Devise a plan for developing a book on the intellectual history of the Nix ecosystem
From the Learning Journey project April 2023
  • Develop a curriculum draft
  • Categorise existing documentation materials
  • Break down tasks for the writing phase
  • Prepare a contributor workflow
  • Link or migrate existing documentation into a central location, as far as possible.
  • Prepare and publish a call for contributors for the writing phase

Documentation goals for the Nix ecosystem

I chose the following categories for evaluating documentation activities, because they emerge from the concrete goals recorded in the past. I ordered them taking into account their relative importance and the sequence of events; but mostly all of this is merely a narrative device to condense a lot of information.

Improve discoverability

Many of the most obvious issues with Nix-related documentation revolve around obstacles to finding relevant information. We made notable progress on improving discoverability, by more clearly separating the roles of different information sources, reworking navigation paths, and adding cross-references. But we also ran into deeper structural and technical problems, which we started to address from various angles.

Details (17 items)

Increase coverage of reference documentation

The Nix ecosystem is a large piece of software. The goal is to eventually capture all of its interfaces in reference documentation.

We mainly added much more information to the Nix reference manual, but have not yet started fully documenting the Nix language syntax. There were many incremental improvements to Nixpkgs reference manual, and work is ongoing to further increase coverage. The people involved with the documentation team did not work on the NixOS manual, and the official NixOS Wiki has only recently launched.

Details (14 items)

Ensure correctness of reference documentation

Merely having interface documentation is not enough, the information also has to be correct and up to date.

Contributors corrected countless errors and omissions in all documentation resources. Work is ongoing to render all of the Nixpkgs library documentation from the source code. There are technical obstacles to increase automatic presentation of interface documentation in the Nix reference manual.

The biggest impact on correctness would be made by automatically testing examples as part of continuous integration, but there was no progress on that apart from exchanging ideas. We’ve been updating examples while working on other things, to make them more self-contained so they could be tested eventually.

Details (8 items)

Improve the contributor experience

I claim that one of the most important use cases of any community-driven open source project is contributing to that project. Much of our efforts revolved around easing contributions, in particular to documentation. We added and improved contributor guides for each component of the Nix ecosystem, and added automation to further reduce friction. Substantial parts of official documentation are now actively maintained, and the team holds regular meetings to review and merge pull requests. As a consequence, in the second half of 2023 we got much more work done than ever, and more people than ever joined and stayed to help.

Details (14 items)

Teach important skills

Part of the documentation team’s mission is to provide more and better learning materials. Thanks to many volunteers’ patience and endurance, the group produced roughly one tutorial every two months between end of 2022 and end of 2023. This reportedly improved the learning experience — especially where we took the time to build a solid foundation in reference documentation. That way, we have addressed a good chunk of fundamental use cases.

But there is still much ground to cover, even for what we consider essential skills and workflows in the Nix ecosystem. Some articles could be a lot shorter. Nonetheless, there are now more and much better tutorials than ever, all of which are actively maintained to incorporate feedback and keep up with best practices.

Details (9 items)

Explain concepts

The Nix ecosystem approaches many problems in a unique way, which requires introducing new terms and explaining why certain things are the way they are. While that was my original motivation to start working on documentation, explanations have shown to be less of an important tool for serving immediate needs. As a result, only a few things were worked on.

Details (3 items)

Reduce onboarding time

The primary mission of the documentation team is to reduce onboarding time for beginners. That plays into addressing the other challenges I have talked about, such as attracting more contributors. We figured that a sequence of lessons focused on widely applicable skills would be an effective resource for users to become self-sufficient in the ecosystem.

To that end, there was a coordinated project to design what we called the Learning Journey, which was supported by many people through a fundraiser on Open Collective. Despite some setbacks, we achieved the main goals and will continue with implementation by writing more tutorials for the curriculum.

Details (7 items)
  • With support of the NixOS Foundation, the documentation team applied for Google Season of Docs with the goal to develop a Nix curriculum called the Learning Journey.
  • We did not win the grant, but Ron Efroni (@ronef, Flox), in his capacity as NixOS Foundation board member, swiftly helped running a successful fundraiser to get the project started regardless. Within a few months, we collected more than 18 000 Euro from 57 supporters.
  • The team organised participation, processed applications, and were excited to have found the ideal candidates for the editorial lead and technical expert positions.
  • Unfortunately, after helping to kickstart the project, our editorial lead had to permanently leave for health reasons. Instead of searching for a replacement, we decided to shift focus, and reallocated part of the budget to fund work on technical infrastructure.
  • A dedicated Learning Journey volunteer working group formed around the project, led by Zach Mitchell (@zmitchell, Flox). Zach published monthly updates during that time. The remaining budget was used for expert reviews and support work, as originally intended, which helped keep up the momentum.
  • By the end of 2023, the Learning Journey group produced the key deliverable: a curriculum for learning Nix. This was accompanied by new and improved tutorials and guides, updated navigation structure on nix.dev, a well-tried contribution workflow, and a better overview of existing documentation resources.
  • We didn’t do another usability study as originally planned. That will be more sensible after filling up more of the curriculum scaffold with more tutorials.

Summary

All in all, we can claim success: The number of new and regular contributors to Nix documentation and nix.dev has increased significantly every quarter. The average turnaround time for documentation contributions seems to have decreased. We’ve come quite far with smoothing out the first couple of days and weeks of using vanilla Nix. Based on feedback and heuristics, such as questions asked on Discourse, I conclude that we have substantially reduced the time required for onboarding with Nix. I’m convinced that discoverability of learning materials and reference documentation has also improved a lot. Some of these results have yet to be validated with more evidence.

Thoughts on future work

I think that Nix documentation is on the right trajectory to eventually shake off its negative reputation and thus help unleash the power of Nix onto the world. I suggest to continue following the goals discussed in this report, and in order to reach them more quickly:

  • Finish things that have been started.
  • Put more emphasis on technical infrastructure and automation.
  • Focus on fewer topics, prioritise reference documentation.
  • Cultivate code ownership and empower more people to become maintainers.
  • Write more tutorials outlined in the Learning Journey.
  • Help establish a consistent information architecture across the Nix ecosystem.
  • Secure more funding to support volunteer efforts.
  • Systematically measure effectiveness of documentation efforts.

Acknowledgements

This whole endeavor has only been possible to sustain thanks to ongoing financial support. I’d like to thank the leadership of Antithesis, Tweag, Flox, and Determinate Systems for funding me, Silvan, Zach, and Luc to lead and assist the documentation team’s efforts over the past 18 months. Many thanks also to all our supporters on Open Collective for your confidence in our mission and your crucial role in speeding up progress.

An enormous amount of work has also been done by numerous volunteers, who did not just contribute to multiple articles and many small improvements, but were also involved in arriving at technical decisions and implementing them. Some of them stay around to incorporate corrections to their texts and code. Thank you so much for your indispensable contributions to our collective endeavor:

About the author

Valentin Gagarin

After an artistic career in education, film making and graphics design, Valentin grew into informatics, where he quickly developed a mathematical and engineering mindset as well as a passion for functional programming, reproducibility, and effective communication.

If you enjoyed this article, you might be interested in joining the Tweag team.

This article is licensed under a Creative Commons Attribution 4.0 International license.

Company

AboutOpen SourceCareersContact Us

Connect with us

© 2024 Modus Create, LLC

Privacy PolicySitemap