NixOS: A Field Report

Why?

NixOS is a relatively new Linux distribution based on the Nix package manager. A Nix package is hashed based on its content, dependencies, compiler , etc. and stored immutably in the Nix store. The Nix package manager uses this Nix store, a patched dynamic linker, and a buttload of symlinks to isolate installs. You can have multiple versions of applications and libraries living on the same machine without conflict.

The Nix package manager also maintains a history of installations and—since the Nix store is full of immutable installs of applications and libraries—Nix provides a way to rollback installs to a previously working configuration. NixOS extends this concept all the way to the Linux kernel. When you change the system configuration for your machine you get a new GRUB entry for that entire profile of your computer: kernel, applications, libraries, and even your configuration files. If something goes badly you just reboot and select the GRUB entry for your previous configuration. Check out section 1.4 of the NixOS manual for some cool details http://nixos.org/nixos/manual/#sec-upgrading.

But wait there's more! These installations are atomic, so it either completes successfully or not at all. So basically what we get here is something like an ACID MVCC for the software on your computer.

There are definitely practical benefits to all this. There have been times that I've done some riskier upgrades. Moving to a new Ubuntu release is one of those times. There's always a bit of a question as to what may or may not be broken. I usually back up my files and do a fresh install. I've also been bitten a couple of times by a bad video driver upgrade that was annoying to recover from. That said, I don't know that I've really needed an ACID MVCC for software all that often.

Aside from riskier installs you can also setup a Nix configuration for each project you work on. You can chroot into this Nix environment. The Nix configuration can be version controlled, so that when you checkout an old branch you can use the environment that matches that version of your project. You can also take this Nix configuration and create a VM from it, or send it off to an EC2 node. Admittedly you can do all of this stuff just using the Nix package manager on Linux or OSX without needing NixOS. (Here is how http://zef.me/5966/setting-up-development-environments-with-nix.)

So, really...Why?

So all this stuff is great, but maybe not worth switching to NixOS for your day-to-day desktop machine. I have moved for a couple of reasons. Lately I've been getting disillusioned with the direction Ubuntu is headed. I barely made it through the last LTS upgrade. I prefer to use Gnome and XMonad, but Ubuntu comes with Unity these days and it's hard to get a plain Gnome install (though I made it work). The flip side is that Ubuntu is a really smooth install where most everything just works out of the box, but that's not really enough to keep me there. I'm basically forced to consider some kind of a major change whether it is a change in distribution or the tools I use or something.

I also like to try to change things up every once in a while. I've been on Ubuntu for close to 10 years and Gnome+XMonad for maybe 3 years. I need to shake things up a bit, learn some new stuff, and feel like a hopeless newb. Boy did I ever accomplish that!

Attempt 1: No Gnome & Miserable Failure with Wireless

I downloaded and burned the NixOS 13.10 minimal CD. It booted up fine and gives you hints to walk you through the process. The CD boots with a copy of the manual on one of the Linux consoles so you can jump back and forth to reference the manual as you're installing. The process is pretty simple, too. You format a partition, mount it, run a config generation program, and customize the generated config. Once the config is set you run an install command and boot into your new NixOS installation.

However, Gnome doesn't install. Based on a reading of the docs you might be led to think that it does, but after 13.10 was released this commit was made to the Nix packages: https://github.com/NixOS/nixpkgs/commit/a734f32fa1efce21a008e391cfb10695c4d738cd, so it's not even an option in more recent versions of NixOS.

Coincidentally XMonad doesn't install either. Well it seems like it does, and if you use it without any custom config then it's probably fine, but to use a custom config you need a Haskell compiler and the XMonad packages. For some reason when the XMonad packages install they don't install the dependencies that they need (which seems broken given how Nix is supposed to work). I had to iteratively run `ghc-pkg check` and install a bunch of packages manually. Perhaps I was missing something. Also, not to give away the ending, but this was with 13.10, so it may be a different story with the latest unstable NixOS.

Finally, I had wireless issues. The installation CD booted and I was able to get an install, but once I tried to find some combination of CLI and/or GUI software to configure my wireless adapter I wedged something, and decided to just reboot and reinstall, but the installation CD was hanging when starting the WPA supplicant. It seemed maybe there was some state stored in the wireless card that was causing the WPA supplicant to hang, but I couldn't boot NixOS in order to configure the card. After many hours fiddling with this to try to get the wireless working and being very frustrated I very nearly gave up on NixOS. As a last ditch effort I decided to try using the unstable version of NixOS.

Attempt 2: Using Unstable NixOS

Out of the gate Unstable NixOS was more stable than the 13.10 release. I was able to actually boot the thing! I made it through the installation process. I had decided that I was going to use Xfce+i3 instead of Gnome+XMonad. Changing my desktop and window manager at the same time as changing my distribution was going to make this whole process more "interesting."

Configuring NixOS is relatively simple once you know what the configuration options are, but finding them isn't as easy as it could probably be. The NixOS manual has a section of configuration options at the end. That was an important reference. I don't know if it documents every possible option, so sometimes I would take a look at the Nix package source. There are also configuration options that affect some packages that aren't entirely obvious unless you look at the package source. I think in the future it would be nice (in addition to the way the documentation is currently organized) to have documentation that can be browsed per package, so you could see for a particular package what options affect it.

Having a working distribution is since, but there were still some annoyances. Searching for packages is also a bit unsatisfying. The recommended way to search seems to be `nix-env -qaP '*' | grep NEEDLE` which works, but it's not the most performant, and you also can only search by package name; you can't search by description or configuration options or anything else. Even when you've found the package you want more detail on, it is not obvious how to find the package. You can browse the nixpkgs source, but it is organized by categories. The best way seems to be doing a `find / -name NEEDLE` or something. None of this is really a deal breaker (for me), but just lessons learned on how to find information.

Finally, on the subject of finding information the nix-dev mailing list is a good resource. I ended up searching it with google "site:http://lists.science.uu.nl/pipermail/nix-dev/ NEEDLE". I guess it is also possible to search the Gmane archive http://news.gmane.org/gmane.linux.distributions.nixos.

Adaptations

There were definitely tools that I'm used to using that were available and worked great on NixOS, among them were Conkeror, Emacs, OpenJDK, Ruby, Tmux, and Skype.

There were replacements I had to make. Obviously Gnome+XMonad became Xfce+i3, and this had a cascading effect.

• I had a raise-or-run script that used wmctrl to activate a program's window, but wmctrl doesn't work with i3, so I had to replace that script with one that uses i3msg.
• I spent a bit of time tuning my i3 config to have bindings similar to the ones I was using with XMonad.
• Previously I was using gnome-panel for workspaces, wireless control, sound volume control, date/time, etc.; that got replaced by i3bar and i3status.
• I was using swarp to shuffle the mouse cursor out of the way in Gnome+XMonad, and that package wasn't available on NixOS, so I replaced that functionality with xdotool.
• I tried to get NetworkManager up and running, and there appear to be configuration options to get it set it up on NixOS, but I ran into issues. I ended up just installing the wpa_gui tool to manage my wireless connections.

I also had a cache of scripts that I used for various things. Some of them were for building programs from source, but I think I can drop those and just use Nix to accomplish the same thing. Others I had to replace '#!/bin/bash' with '#!/usr/bin/env bash' since NixOS doesn't have things at fixed, standard locations like other Linux distributions.

There are some other minor issues to work out. For one my sound card seems to be detected just fine, and Skype will use it, but Xfce doesn't seem to be playing any sounds. I just get a system beep for "sound" events. I think this is an issue with the configuration of Xfce and its sound theme or something. Not really a major issue for me, but I'd like to figure it out eventually.

I would also like to get Dropbox installed. It actually worked fine on 13.10, but for some reason it isn't working so well on unstable. I haven't had a lot of time to look into it, and again not a high priority. I may take the opportunity to look into OwnCloud instead.

Conclusion

I went into this expecting it to be rough at times, and expecting to make some changes to the tools I was used to using. That was no surprise. I was a bit disappointed in how the latest official release of NixOS turned out. The flip side is that the Unstable release has worked great so far, and I'm a little more comfortable following the bleeding edge of NixOS given that (theoretically) I should be able to rollback any changes that bork my system. I'm hoping that turns out to be true.

If you're willing to learn something new and possibly snag your sweater on some rough edges, then NixOS could be a fun experience for you. :) If you use some of the same tools as me, then hopefully what I've discovered can help you move to NixOS. If you want something a little more polished, then you may want to wait on NixOS.

References

My configuration.nix
https://gist.github.com/pjstadig/8183688
Dotfile changes moving from Ubuntu+Gnome+XMonad to NixOS+Xfce+i3
https://github.com/pjstadig/new.dotfiles/compare/455374...master