diff options
Diffstat (limited to 'posts/2015-08-30-ruby-on-guix.md')
| -rw-r--r-- | posts/2015-08-30-ruby-on-guix.md | 302 |
1 files changed, 302 insertions, 0 deletions
diff --git a/posts/2015-08-30-ruby-on-guix.md b/posts/2015-08-30-ruby-on-guix.md new file mode 100644 index 0000000..29804c6 --- /dev/null +++ b/posts/2015-08-30-ruby-on-guix.md @@ -0,0 +1,302 @@ +title: Ruby on Guix +date: 2015-08-30 15:00 +tags: gnu, guix, scheme, guile, ruby, wsu +summary: How to use Guix + some elbow grease to replace RVM and Bundler on GNU/Linux +--- + +I’ve been working with Ruby professionally for over 3 years now and +I’ve grown frustrated with two of its most popular development tools: +RVM and Bundler. For those that may not know, RVM is the Ruby version +manager and it allows unprivileged users to download, compile, +install, and manage many versions of Ruby instead of being stuck with +the one that is installed globally by your distro’s package manager. +Bundler is the tool that allows developers to keep a version +controlled “Gemfile” that specifies all of the project’s dependencies +and provides utilities to install and update those gems. These tools +are crucial because Ruby developers often work with many applications +that use different versions of Ruby and/or different versions of gems +such as Rails. Traditional GNU/Linux distributions install packages +to the global `/usr` directory, limiting users to a single version of +Ruby and associated gems, if they are packaged at all. Traditional +package management fails to meet the needs of a lot of users, so many +niche package managers have been developed to supplement them. + +Taking a step back, it becomes apparent that dependency isolation is a +general problem that isn’t confined to software written in Ruby: Node +has npm and nvm, Python has pip and virtualenv, and so on. A big +limitation of all these language-specific package managers is that +they cannot control what is outside of their language domain. In +order to use RVM to successfully compile a version of Ruby, you need +to make sure you have the GCC toolchain, OpenSSL, readline, libffi, +etc. installed using the system package manager (note: I’ve seen RVM +try to build prerequisites like OpenSSL before, which I then disabled +to avoid duplication and security issues and I recommend you do the +same.) In order to use Bundler to install Nokogiri, you need to make +sure libxml2 has been installed using the system package manager. If +you work with more than a single language, the number of different +package management tools needed to get work done is staggering. For +web applications, it’s not uncommon to use RVM, Bundler, NPM, Bower, +and the system package manager simultaneously to get all of the +necessary programs and libraries. Large web applications are +notoriously difficult to deploy, and companies hire a bunch of +operations folk like me to try to wrangle it all. + +Anyway, let’s forget about Node, Python, etc. and just focus on Ruby. +Have you or someone you work with encountered hard to debug issues and +Git merge conflicts due to a problem with `Gemfile.lock`? Bundler’s +fast and loose versioning in the `Gemfile` (e.g. `rails >= 4.0`) +causes headaches when different users update different gems at +different times and check the resulting auto-generated `Gemfile.lock` +into the repository. Have you ever been frustrated that it’s +difficult to deduplicate gems that are shared between multiple bundled +gem sets? Have you looked at the [RVM home page](https://rvm.io) and +been frustrated that they recommend you to `curl | bash` to install +their software? Have you been annoyed by RVM’s strange system of +overriding shell built-ins in order to work its magic? I’m not sure +how you feel, dear reader, but my Ruby environments feel like one +giant, brittle hack, and I’m often enough involved in fixing issues +with them on my own workstation, that of my colleagues, and on +production servers. + +So, if you’re still with me, what do we do about this? How can we +work to improve upon the status quo? Just use Docker? Docker is +helpful, and certainly much better than no isolation at all, but it +hides the flaws of package management inside an opaque disk image and +restricts the environments in which your application is built to +function. The general problem of dependency isolation is orthogonal +to the runtime environment, be it container, virtual machine, or “bare +metal.” Enter functional package management. What does it mean for a +package manager to be functional? GNU Guix, the functional package +manager that I contribute to and recommend, has this to say: + +> GNU Guix is a functional package management tool for the GNU +> system. Package management consists of all activities that relate +> to building packages from sources, honoring their build-time and +> run-time dependencies, installing packages in user environments, +> upgrading installed packages to new versions or rolling back to a +> previous set, removing unused software packages, etc. +> +> The term functional refers to a specific package management +> discipline. In Guix, the package build and installation process +> is seen as a function, in the mathematical sense. That function +> takes inputs, such as build scripts, a compiler, and libraries, +> and returns an installed package. + +Guix has a rich set of features, some of which you may find in other +package managers, but not all of them (unless you use another +functional package manager such as Nix.) Gem/Bundler can do +unprivileged gem installation, but it cannot do transactional upgrades +and rollbacks or install non-Ruby dependencies. Dpkg/yum/pacman can +install all build-time and runtime dependencies, but it cannot do +unprivileged package installation to isolated user environments. And +none of them can precisely describe the *full* dependency graph (all +the way down to the C compiler’s compiler) but *Guix can*. + +Guix is written in Guile, an implementation of the Scheme programming +language. The upcoming release of Guix will feature a Ruby build +system that captures the process of installing gems from `.gem` +archives and a RubyGems import utility to make it easier to write Guix +packages by using the metadata available on https://rubygems.org. +Ruby developers interested in functional package management are +encouraged to try packaging their gems (and dependencies) for Guix. + +Now, how exactly can Guix replace RVM and Bundler? Guix uses an +abstraction called a “profile” that represents a user-defined set of +packages that should work together. Think of it as having many `/usr` +file system trees that can be used in isolation from the others +(without invoking virtualization technologies such as virtual machines +or containers.) To install multiple versions of Ruby and various +gems, the user need only create a separate profile for them: + +```sh +guix package --profile=project-1 --install ruby-2.2 ruby-rspec-3 +# Hypothetical packages: +guix package --profile=project-2 --install ruby-1.9 ruby-rspec-2 +``` + +A profile is a “symlink forest” that is the union of all the packages +it includes, and files are deduplicated among all of them. To +actually use the profile, the relevant environment variables must be +configured. Guix is aware of such variables, and can tell you what to +set by running the following: + +```sh +guix package --search-paths --profile=project-1 +``` + +Additionally, you can also create ad-hoc development environments with +the `guix environment` tool. This tool will spawn a sub-shell (or +another program of your choice) in an environment in which a set of +specified packages are available. This is my preferred method as it +automagically sets all of the environment variables for me and Guix is +free to garbage collect the packages when I close the sub-shell: + +```sh +# Launch a Ruby REPL with ActiveSupport available. +guix environment --ad-hoc ruby ruby-activesupport -E irb +``` + +In order to make this environment reproducible for others, I recommend +keeping a `package.scm` file in version control that describes the +complete dependency graph for your project, as well as metadata such +as the license, version, and description: + +```scheme +(use-modules (guix packages) + (guix licenses) + (guix build-system ruby) + (gnu packages) + (gnu packages version-control) + (gnu packages ssh) + (gnu packages ruby)) + +(package + (name "cool-ruby-project") + (version "1.0") + (source #f) ; not needed just to create dev environment + (build-system ruby-build-system) + ;; These correspond roughly to "development" dependencies. + (native-inputs + `(("git" ,git) + ("openssh" ,openssh) + ("ruby-rspec" ,ruby-rspec))) + (propagated-inputs + `(("ruby-pg" ,ruby-pg) + ("ruby-nokogiri" ,ruby-nokogiri) + ("ruby-i18n" ,ruby-i18n) + ("ruby-rails" ,ruby-rails))) + (synopsis "A cool Ruby project") + (description "This software does some cool stuff, trust me.") + (home-page "https://example.com") + (license expat)) +``` + +With this package file, it is simple to an instantiate a development +environment: + +```sh +guix environment -l package.scm +``` + +I’m not covering it in this post, but properly filling out the blank +`source` field above would allow for building development snapshots, +including running the test suite, in an isolated build container using +the `guix build` utility. This is very useful when composed with a +continuous integration system. Guix itself uses +[Hydra](https://nixos.org/hydra/) as its CI system to perform all +package builds. + +As mentioned earlier, one of the big advantages of writing Guix +package recipes is that the full dependency graph can be captured, +including non-Ruby components. The pg gem provides a good example: + +```scheme +(define-public ruby-pg + (package + (name "ruby-pg") + (version "0.18.2") + (source + (origin + (method url-fetch) + (uri (rubygems-uri "pg" version)) + (sha256 + (base32 + "1axxbf6ij1iqi3i1r3asvjc80b0py5bz0m2wy5kdi5xkrpr82kpf")))) + (build-system ruby-build-system) + (arguments + '(#:test-target "spec")) + ;; Native inputs are used only at build and test time. + (native-inputs + `(("ruby-rake-compiler" ,ruby-rake-compiler) + ("ruby-hoe" ,ruby-hoe) + ("ruby-rspec" ,ruby-rspec))) + ;; Native extension links against PostgreSQL shared library. + (inputs + `(("postgresql" ,postgresql))) + (synopsis "Ruby interface to PostgreSQL") + (description "Pg is the Ruby interface to the PostgreSQL RDBMS. It works +with PostgreSQL 8.4 and later.") + (home-page "https://bitbucket.org/ged/ruby-pg") + (license license:ruby))) +``` + +Note how the recipe specifies the PostgreSQL dependency. Below is the +dependency graph for ruby-pg as produced by `guix graph`, excluding +the GCC compiler toolchain and other low-level tools for brevity. +Pretty neat, eh? + + + +Given that Guix doesn’t yet have many gems packaged (help wanted), it +can still be advantageous to use it for getting more up-to-date +packages than many distros provide, but in conjuction with Bundler for +fetching Ruby gems. This gets RVM out of your hair whilst creating a +migration path away from Bundler at a later time once the required +gems have been packaged: + +```sh +cd my-project/ +guix environment --ad-hoc ruby bundler libxml2 libxslt # etc. +# A small bash script can be used to make these gem sets. +mkdir .gems +export GEM_HOME=$PWD/.gems +export GEM_PATH=$GEM_HOME:$GEM_PATH +export PATH=$GEM_HOME/bin:$PATH +bundle install +``` + +As you’ve seen in the above package snippets, Guix package definitions +are typically very short and rather easy to write yourself. The +`guix import gem` tool was made to lower the barrier even more by +generating most of the boilerplate code. For example: + +```sh +guix import gem pry +``` + +Produces this Scheme code: + +```scheme +(package + (name "ruby-pry") + (version "0.10.1") + (source + (origin + (method url-fetch) + (uri (rubygems-uri "pry" version)) + (sha256 + (base32 + "1j0r5fm0wvdwzbh6d6apnp7c0n150hpm9zxpm5xvcgfqr36jaj8z")))) + (build-system ruby-build-system) + (propagated-inputs + `(("ruby-coderay" ,ruby-coderay) + ("ruby-method-source" ,ruby-method-source) + ("ruby-slop" ,ruby-slop))) + (synopsis + "An IRB alternative and runtime developer console") + (description + "An IRB alternative and runtime developer console") + (home-page "http://pryrepl.org") + (license expat)) +``` + +One still has to package the propagated inputs if they aren’t yet +available, add the necessary inputs for building native extensions if +needed, and fiddle with the native inputs needed to run the test +suite, but for most pure Ruby gems this gets you close to a working +package quickly. + +In conclusion, while support for Ruby in Guix is still in its early +days, I hope that you have seen the benefits that using a +general-purpose, functional package manager can bring to your Ruby +environments (and all other environments, too.) For more information +about Guix concepts, installation instructions, programming interface, +and tools, please refer to the +[official manual](https://gnu.org/software/guix/manual/). Check out +the [help page](https://gnu.org/software/guix/help/) for ways to +contact the development team for help or to report bugs. If you are +interested in getting your hands dirty, please +[contribute](https://gnu.org/software/guix/contribute/). Besides +contributions of code, art, and docs, we also need +[hardware donations](https://gnu.org/software/guix/donate/) to grow +our build farm to meet the needs of all our users. Happy hacking! |