I have been a user of Nix (a “reproducible” package manager) and NixOS (the Linux distribution based on this package manager) for almost two years. I use it in a personal and professional context. In this article I will focus on the “pain points” in my use of Nix and NixOS. While there are many positives, the NixOS experience is far from being a smooth ride. I still mostly enjoy using Nix and NixOS, and this article is not about blaming anyone. I’m also a part of the problem, as I could probably participate more actively in the project.
Poor documentation¶
The documentation of Nix and NixOS is rather poor. To understand how a service or an option works, it is very often necessary to read the source code (`.nix’ files) because there are no explanations in the manual, or the explanations are not very comprehensible.
On this point, the use of GitHub to host project repositories makes this search sometimes difficult. For example, a search on the keyword “pass-wayland” should return a result in the “Code” tab, but it does not, because Github does not index the entire repository (you can check that it is present by using your browser search feature on the all-packages.nix file)
The Nix / NixOS wiki is rarely up to date. It mixes information related to specific versions of nixpkgs (e.g. 22.05, unstable, …) without necessarily specifying which version it is. As it is not an “official” resource, its pages contain many references to the use of third party tools (such as the home-manager program or the cachix service) or non-stable features (such as flakes). Specific configurations are posted directly to the wiki (e.g. https://nixos.wiki/wiki/Sway) without any explanation of the choices made or their specificity, which the declarative aspect of the NixOS configuration tends to encourage.
A porous evaluation¶
The reproducibility of NixOS / Nix / Nixpkgs is very perfectible. One might think that by default NixOS is ‘reproducible’ (for some definition of reproducible). This is not really the case: two machines that have the same configuration files (/etc/nixos/configuration.nix) can differ widely due to dependency on other configuration files such as ~/.config/nixpkgs/config.nix, the ability to change Nix’s behaviour using environment variables, command line arguments or even due to using different versions of Nix.
Since the Nixpkgs repository (which contains all the recipes related to the packages present in NixOS) is different from the Nix repository (which contains the source code of the Nix package manager), using a specific commit of Nixpkgs is not enough, it is also necessary to pay attention to the version of Nix used, or else regressions might appear. From this point of view, Guix has a better approach in my opinion, a single repository being used for the package manager and the package “recipes”.
Fragile sources¶
The reproducibility of NixOS / Nix is also only assured if the sources used remain truly accessible. A recent example underlines the fragility of these sources, and the fragility of Nix which does not exploit solutions such as Software Heritage (where Guix does for at least three years). This means, in a scientific context for example, the user, that is to say me, still has to care for source preservation if they wants to be sure five or ten years after they developed their code and used nix, it still runs.
No binary reproducibility¶
One of the problems with Nix / NixOS / Nixpkgs regarding the backup of these sources is simply the discovery of the sources to archive. Indeed, it is not possible to easily extract the set of dependencies (e.g. URLs used in the recipes of Nixpkgs packages) to archive them, as a certain number of these URLs are only “computed” at evaluation time, which requires to evaluate the whole Nixpkgs set, in all its possible configurations, to make sure that one is actually archiving what needs to be archived (part of this work is currently done by Hydra, but unfortunately broken). And that’s without even mentioning Fixed Output Derivation. It should be remembered that the reproducibility of Nix / NixOS / Nixpkgs is only a reproducibility of the sources: if the sources change, one is warned, but it is not a question of the reproducibility of the binaries (which can change at each build).
This binary reproducibility of Nix / NixOS / Nixpkgs is indeed not really tested, at least not systematically. The website “Is NixOS Reproducible?” points out that the NixOS ISO is 99.77% reproducible, but this is a check on a tiny percentage, less than 2%, of all packages. Again, Guix is much better with stats on all packages in the repository. And even more classical distributions such as Archlinux or Debian are better at this than Nix / NixOS / Nixpkgs ! As of now :
- Archlinux is
78% reproduciblebetween 86% and 90% reproducible on amd64 packages (Thanks Foxboron for the correction); - Debian is 95.7% reproducible on amd64 packages ;
- Both distribution actually test binary reproducibility, and offer good dashboards to follow the reproducibility of the packages.
Managing state could be easier¶
To handle « state », e.g. data that is not software but which is linked to certain software versions, e.g., postgresql data, NixOS uses the system.stateVersion option which specifies how compatible is the state with software versions. For example, for postgresql, the package version depends on system.stateVersion, unless overwritten manually :
mkDefault (if versionAtLeast config.system.stateVersion "22.05" then pkgs.postgresql_14
else if versionAtLeast config.system.stateVersion "21.11" then pkgs.postgresql_13
else if versionAtLeast config.system.stateVersion "20.03" then pkgs.postgresql_11
else if versionAtLeast config.system.stateVersion "17.09" then mkThrow "9_6"
else mkThrow "9_5");
Because any module can depend on this stateVersion, this means updating between different stateVersions is harder than it should be, in an ideal world. The NixOS FAQ states (haha) you can update system.stateVersion when :
- You have read all release notes starting from your stateVersion.
- You have verified all instances of stateVersion in the code in
<nixpkgs/nixos>. - You have made all manual interventions as required by the changes previously inventoried.
While reading release notes is ok, verifying the source code of <nixpkgs/nixos>, especially when the Github Search is broken, is less ok. Currently, there is also no policy on how long a given system.stateVersion should be supported. This seems harder than it should be.
Nix and NixOS: it’s all or nothing¶
When we use NixOS, and we want to use a package that is not available under NixOS / Nixpkgs (which happens quite often), we have only one choice: pack it into NixOS. And, to be clear, by « pack it into NixOS », I mean writing a derivation for the package, whether it uses buildFHSEnv or not. It is generally not possible to take a shortcut and use precompiled binaries, which means that it is not possible to work with constraints (such as time constraints) on Nix / NixOS, unless you are a motivated enough user to sometimes package several levels of dependencies.
To give an example, the project you want to package might depend on pytest-postgresql or aiohttp-utils, which are not packaged, and which depend in turn on mirakuru and webtest-aiohttp, which are not packaged.
Packaging all this tends to take time. And, of course, because Nix or NixOS are radically different from more classical package management, you can’t really mix them together in a single project and expect to have a good user experience. Using only NixOS is thus not tenable, for me: I need to have access to a second machine, under a more classical distribution. That would also be true on Guix System.
Nix* « Tool Fatigue »¶
I originally didn’t include this section in this article, so if you have been here before and you don’t remember seeing this, it’s normal.
Finally, and probably one of the least pleasant parts of working with Nix*, is what I (and probably others) call « tool fatigue ». This is not a problem limited to Nix*, but Nix* is particularly affected.
I put behind this expression the bazillion of tools built around Nix*: tools to manage secrets in NixOS, tools to deploy NixOS machines, frameworks to package the dependencies of certain languages (e.g. python or rust) in Nix, opinionated tools for the management of development environments… tools that are not really « endorsed » by Nix and NixOS, and are not « official ». As someone told me once on the Nix matrix room, « “official” means a lot less than you seem to think. », and I think that’s a problem, because there is no incentive for the « community » (whatever its form) to work together on a specific piece of software to make it better, which would help with e.g. documentation, integration, and so on. And because the Nix* ecosystem is moving so fast, you may have to « bet » on the best tools to use because you have not many guarantees with regards to their longevity.
For example, it has become a kind of running joke in the Nix* world that there are more deployments tools than deployment tools users. Yes, you have NixOps that is the kind of official tool (as in « living under the NixOS namespace on Github ») to deploy configuration to possibly remote machines, but it has not seen a stable release in more than three and half years. Instead, you get new deployment tools every year, which is nice, in a way, because it means the community is active, but is less nice because a newcomer has too many choices to choose from.
There is no « standard », safe answer to the question « what do I use to manage and deploy NixOS configurations », because NixOps is lagging behind because we’re in a state where mostly everybody use flakes, which are unstable, but the NixOps does not mention them a single time. Instead, you can deploy NixOS configurations with tools such as colmena, deploy-rs, morph, nixus, nix-deploy, bento, krops, pushnix, …
And, it’s the same with language dependencies : to package rust in your development environment, you could use devenv, rust-overlay, fenix, nixpkgs-mozilla, dream2nix, cargo2nix, and so on…
It feels, to me, the Nix* community lacks a bit of focus on the ways of achieving things, and, as usual, this problem is difficult because it is not a technical one but a human one.