Daniel Siepmann: Nix Journey Part 1

Introduction ¶

This should be the first blog post about my Nix Journey, let's see if I’ll continue, there are no promises, except that I hope and plan to continue.

This first post will explain how I got introduced to the hole Nix ecosystem. I try to explain where I started, why I had a look and how it continued. It might help you to see how easy or difficult it might be to switch, But it probably only is explaining my background. Still I try to outline some great benefits that are available right from the beginning. I also hope it will reduce the fear that most often comes when thinking about switching stuff. The post also contains many references to get others started.

How it started ¶

My Nix Journey started some years ago when I visited the TYPO3 Camp Switzerland where Andri introduced me to a new Linux Distribution called NixOS. I felt overwhelmed seeing the potential, but I was still on my way from MacOS to Linux and didn't feel like I was ready for NixOS. I then continued my Journey to Linux using Ubuntu as my distribution of choice. One day I got introduced to https://suckless.org/ and their approach to software.I already switched from Gnome to i3wm as window manager in between and felt ready for suckless tools st as terminal and dmenu as application launcher.

Suckless tools have one drawback for conventional distributions.The user should change the source code instead of providing external configuration files. That's not possible (for me) using a distribution like Ubuntu. I therefore had to find other ways and discovered Nix (https://nixos.org/manual/nix/stable/introduction.html), the package manager. I could install and use it along on my existing Ubuntu installation to install st and dmenu and apply patches. Nix itself is a package manager that has recipes (called derivations) to compile software. That way it is easy to apply patches to the source code before compiling. I installed both tools via nix with my patches applied and forgot it for a year or so.

I then discovered NixOS again. I can't remember why, probably because I was unhappy with Canonicals (the Company behind Ubuntu) direction and the potential future of Ubuntu. I checked many distributions and none were promising for me. Until I've discovered NixOS and remembered that I already had Nix in use (without updating the two packages for a year).

How I finally started with Nix (not NixOS) ¶

I read the website, studied documentations (official ones and none official ones). I learned that there is a huge ecosystem, with the most important parts being:

NixOS. A Linux distribution build on Top of Nix.
Nix. A package manager used by NixOS, that could be installed alongside on other Linux distributions. It even has MacOS Support, but I never really tried that one.
Nix. The language used by Nix the package manager.
Nixpkgs. A huge repository providing expressions on how to compile software, written in the Nix language. It is the default repository used by Nix the package manager and NixOS.

I already had Nix the package manager in use, together with Nixpkgs. I decided to dive into it.

I set up a git repository (https://git.daniel-siepmann.de/danielsiepmann/nixpkgs) to track my progress and easily switch back and keep track of my learnings. I also hoped that this might serve as a more or less good tutorial for others. Many commits have proper commit messages explaining what was done. The first commit was done on Mon, 31 Jan 2022 12:02:07 CET and added the existing modifications for the st (https://git.daniel-siepmann.de/danielsiepmann/nixpkgs/commit/4f7b9e64837dff7f7af13d8c347b4b79fd5ef8d6) and dmenu (https://git.daniel-siepmann.de/danielsiepmann/nixpkgs/commit/29d43dceba2825f2f8f566541fce233165a6a30a) packages.

Soon I added more packages and create a custom “package” called derivation within Nix to install all the packages in a declarative way. I learned how to improve the situation step by step.

Adding home-manager on top ¶

I installed some packages through nix. Which was done more or less like any other package manager I was used to. I didn't have a proper state of the system, except that I wrapped all packages as dependencies of a custom package which I had installed. That wasn't satisfying to me. One great benefit of Nix and its ecosystem is that it is declarative. I then discovered home-manager (https://nix-community.github.io/home-manager/) a community project build on top of Nix (the language and package manager) and Nixpkgs. It allows controlling the user environment, its home folder. A user can declarative define the user environment, install packages, define systemd services, create configuration files and a lot more.

That's when I switched my setup to home-manager. It was exactly what I always wished, and even better. I always tracked my dotfiles, but not my whole user setup. Nor was I able to easily convert any system into a specific state. Furthermore, home-manager is sitting on top of nix and its concepts. Some concepts I want to outline as they are groundbreaking for me:

The system is converted with a huge amount of symlinks, making switching system state more or less instant, once the state was build upfront.
Switching between system states is instant as well, allowing me to roll back to an earlier if something broke.
Allowing me to configure everything within a single language, instead of ini format for one tool, json for another, yaml for another, …
Allowing me to patch any software I install, actually benefiting from open source and software freedom.
Migrating my existing setup step by step. There is no need to make a single system switch at a certain time.
It allows me to not only manage configurations, but installed software, services (systemd) and system settings.
It provides so-called Modules, a way to describe available options that will be converted to a system state, which is also used by NixOS. That way I didn't have to deep dive into Linux internals. I could read the man pages (generated from the modules) and inspect the options. I guess Modules need a dedicated blog post as they are such a great feature, at least for me

That was a very long process, because my setup wasn't defined, it was in a floating state. I started migrating my configuration files, discovering software and migrating the software. I removed the software from Ubuntu whenever possible. Right now my home-manager setup is running on my Dell XPS 13 on top of Ubuntu, as well as on my MacBook Air 2013 on top of NixOS. That way I've the exact same setup, same tools, config, within one repository and one tool. It defines my whole user environment, starting from the login. It will start my window manager i3wm, services like applets and provides everything else on a user level. Only a few tools seem to be too coupled to Ubuntus internals, so I can't install them with home-manager on Ubuntu, but they work fine on NixOS.

Adding development specifics ¶

I'm a web developer doing PHP backend development most of the time. I'm working for different customers with different PHP version from 7.4 to 8.2. The projects are mostly TYPO3 installations and sometimes need a Solr server in addition to a web server (Apache) and PHP (FPM) as well as a database server (MySQL/MariaDB). Those are still installed via Ubuntu as that was my existing setup. Still I need composer as package manager for PHP packages on a per-project level, and there was a time when some projects still needed to use composer version 1, while others were using version 2. Projects also use different PHP versions on command line as well.

I started with shell.nix (https://nixos.org/manual/nix/stable/command-ref/nix-shell.html) as my go-to solution. Every project got its own file declaring the concrete PHP and composer version, as well as Node.js if necessary in order to compile the frontend assets. That way I no longer needed to switch PHP versions on CLi provided by Ubuntu (I had a small dmenu script which made the process a matter of a couple of keystrokes).

There are other solutions I considered, but that should be another blog post.

I than started with a POC setup on NixOS. That was very cool and you can see the current state at https://git.daniel-siepmann.de/danielsiepmann/nixpkgs/src/branch/main/systems/hikari/web-development if you are not afraid of reading Nix language. A basic TYPO3 project looks like this right now:

{ pkgs, lib, config, ... }:
let
  php = pkgs.php82;
in import ./../lib/create-typo3.nix {
  inherit config lib pkgs php;

  domain = "daniel-siepmann.own.localhost";
  relativeDocumentRoot = "own/daniel-siepmann/project/public/";
  databaseName = "own_danielsiepmann";
}

It contains everything that's needed to setup the database, virtual host and fpm pool. I still use shell.nix for the projects on command line level. (My journey is not over and I don't like the duplication of PHP versions, but that's the current state while writing this post).

Feedback welcome ¶

Please provide feedback if something feels too unimportant or bloated. Also provide feedback if something is lacking or unclear. Check out the different ways to contact me.

Acknowledgements ¶

Thanks to @array@fosstodon.org as he was looking for a new distribution replacing his existing solution. I already wanted to blog more about my journey but never found the motivation or a proper goal for blog posts. But his thread on the Fediverse helped me. I hope this blog will help him and others as well.

Blog Post

Nix Journey Part 1