#+TITLE: Functional Package and Configuration Management with GNU Guix #+AUTHOR: David Thompson #+EMAIL: davet@gnu.org #+DATE: Wednesday, January 20th, 2016 #+DESCRIPTION: #+KEYWORDS: #+LANGUAGE: en #+OPTIONS: H:1 num:t toc:nil \n:nil @:t ::t |:t ^:t -:t f:t *:t <:t #+OPTIONS: TeX:t LaTeX:t skip:nil d:nil todo:t pri:nil tags:not-in-toc #+INFOJS_OPT: view:nil toc:nil ltoc:t mouse:underline buttons:0 path:http://orgmode.org/org-info.js #+EXPORT_SELECT_TAGS: export #+EXPORT_EXCLUDE_TAGS: noexport #+LINK_UP: #+LINK_HOME: #+startup: beamer #+LaTeX_CLASS: beamer #+LaTeX_CLASS_OPTIONS: [bigger] #+COLUMNS: %40ITEM %10BEAMER_env(Env) %9BEAMER_envargs(Env Args) %4BEAMER_col(Col) %10BEAMER_extra(Extra) #+LATEX_HEADER: \beamertemplatenavigationsymbolsempty #+BEAMER_THEME: metropolis * About me - GNU project volunteer - GNU Guile user and contributor since 2012 - GNU Guix contributor since 2013 - Day job: Ruby + JavaScript web development / “DevOps” * Overview - Problems with application packaging and deployment - Intro to functional package and configuration management - Towards the future - How you can help * User autonomy and control It is becoming increasingly difficult to have control over your own computing: - GNU/Linux package managers not meeting user needs - Self-hosting web applications requires too much time and effort - Growing number of projects recommend installation via =curl | sudo bash= [fn:2] or otherwise avoid using system package managers - Users unable to verify that a given binary corresponds to the source code * User autonomy and control “Debian and other distributions are going to be that thing you run Docker on, little more.” [fn:7] * User autonomy and control This is very bad for desktop users and system administrators alike. We must regain control! * What’s wrong with Apt/Yum/Pacman/etc.? - Global state (=/usr=) that prevents multiple versions of a package from coexisting - Non-atomic installation, removal, upgrade of software - No way to roll back - Nondeterminstic package builds (though this is changing!) - Reliance on pre-built binaries provided by a single point of trust - Requires superuser privileges * The problem is bigger Proliferation of *language-specific package managers* and *binary bundles* that complicate secure system maintenance. * Web applications Web applications are particularly painful. * Web applications It’s common for today’s web applications to require *two or more package managers* to get all dependencies. * Web applications Integrating a web application packaged only for a language-specific manager into a system package manager proves difficult. NodeJS is particularly frightening. [fn:6] * Web applications There’s a growing number of popular web applications (Hadoop, Chef Server, Cloudera, etc.) that *no one knows how to build from source*! [fn:3] * Deployment How do we automate application deployment without going crazy? * Chef/Puppet/Ansible/etc. are pretty good, right? Building on top of mainstream package managers and distros yields an unstable foundation. * Problems with configuration management software - Imperative config management makes is overly-complex and brittle (idempotence is hard) - More reliable builds require spawning new machines and building from scratch each time (sledgehammer) - Made primarily for developers for server maintenance, but all users could benefit * Docker? Surely Docker addresses these issues? * Docker? I’m afraid not. * Problems with Docker - Still imperative (though resulting images are immutable) - Dockerfile DSL is not expressive - Promotes one disk image per application to cover up underlying package management mess [fn:4] - No provenance - Image layering is an ineffective caching strategy - Does not compose (containers are only one important use-case) * Problems with Docker - Reliance on DockerHub binaries proves to be insecure [fn:5] * Well that was pessimistic Computers are hard. Maybe we should just farm potatoes instead. * Meet GNU Guix Guix is the functional package management tool for the GNU system. It is based on the pioneering work of the Nix project. [fn:8] * What does “functional” mean? “Functional” in this context means treating package builds as functions, in the mathematical sense. =emacs = f(gcc,make,coreutils,…)= * Functional package management Benefits: - Build reproducibility - Atomic upgrades and roll backs - No single point of trust - Unprivileged package management - Multiple variants of the same software may coexist * Functional package management The *complete dependency graph* is captured, precisely, down to the *bootstrap binaries*. No SAT solver or other complex algorithm for dependency resolution. * Functional package management To view package builds this way, Guix performs builds in an *isolated container* in which *only the specified dependencies* are accessible. This maximizes *build reproducibility*. * Reproducible builds Reproducible builds produce *bit-identical binaries* when performed multiple times under the same conditions. Allows for *independent verification* that a given binary corresponds to its alleged source code. * Why? WRITEME Mention reproducible-builds.org * Demo guix package guix challenge * Hacking Guix is made to be maximally hackable, taking inspiration from Emacs. We seek to intentionally blur the line between user and developer. * Choice of language Guix is rather special in its choice of implementation language. * Philosophy It’s better to extend an existing language for package recipes and configuration files rather than making a new, domain-specific one. * Embedded vs. External DSLs Using an extensible programming language as a host has several advantages compared to external DSLs: - No new parser, interpreter/compiler, editor tools, etc. to maintain - Access to all available libraries of the host language - Extensions to the host language can be used as a library by others Not all general-purpose programming languages are suitable for embedding new languages, [fn:1] so which did we choose? * Guile Scheme - GNU Guile is a Scheme implementation and the official extension language of the GNU project - It’s a great choice for EDSLs because of Scheme’s hygienic macro system - It’s a great choice for Guix because purely functional programming is well-supported in Scheme * Guile goes with everything Guix uses Guile for nearly everything: - Initial RAM disk - Init system (GNU Shepherd, formerly GNU dmd) - Package recipes (including build scripts!) - Command line tools - Low-level POSIX/Linux utilities (such as =call-with-container=) * Guix as a library - Guix is a big collection of Guile modules - Packages are first-class Scheme objects - Anyone can use Guix as a library to write new Guile programs that manipulate package recipes, create new user interfaces (like a web UI), etc. * Example package recipe WRITEME * Demo build package at the REPL in Emacs * Other user interfaces Demo Emacs UI, web prototype * Importing packages from elsewhere guix import guix refresh * Demo Import a package from PyPI * Development environments guix environment * Full-system configuration WRITEME * Example system configuration WRITEME * Demo guix system vm * Project status WRITEME * Future WRITEME * Join us! We need interested hackers to help us: - Add new packages - Upgrade existing packages - Write system services - Improve the UI - Add new tools - Translate to new languages - Maintain the web site - Other stuff! * Join us! We are currently collecting donations via the FSF to purchase new servers for our build farm! https://gnu.org/software/guix/donate/ * Join us! Chat with us in the =#guix= channel on Freenode or on the =guix-devel@gnu.org= and =help-guix@gnu.org= mailing lists. * Thank you! Visit https://gnu.org/software/guix for source code, documentation, past talks, etc. \begin{center} \huge{Questions?} \end{center} * Legal © 2016 David Thompson == This presentation is licensed under the Creative Common Attribute Share-Alike 4.0 International license. * Footnotes [fn:1] “How to be a good host: miniKanren as a case study” https://www.youtube.com/watch?v=b9C3r3dQnNY [fn:2] http://curlpipesh.tumblr.com/ [fn:3] “Your big data toolchain is a big security risk!” http://www.vitavonni.de/blog/201504/2015042601-big-data-toolchains-are-a-security-risk.html [fn:4] “The sad state of sysadmin in the age of containers” http://www.vitavonni.de/blog/201503/2015031201-the-sad-state-of-sysadmin-in-the-age-of-containers.html [fn:5] “Over 30% of Official Images in Docker Hub Contain High Priority Security Vulnerabilities” http://www.banyanops.com/blog/analyzing-docker-hub/ [fn:6] “Let’s Package jQuery: A Javascript Packaging Dystopian Novella” http://dustycloud.org/blog/javascript-packaging-dystopia/ [fn:7] “ownCloud and distribution packaging” http://lwn.net/Articles/670566/ [fn:8] http://nixos.org/nix/