Creating a Gem

So one of my teammates at Replay asked me to write a step by step on the process when we decided to publish my work with RSpec and SimpleCov integration as a gem. There are a lot of guides available for this already, but this will at least be the most up to date one when it’s published :)

When this guide starts I already have some working code. An earlier draft of the code in question can be seen in my previous post about RSpec and SimpleCov integration.

What I want to do now is make a gem that includes the code and publish it so anyone can add and use it with ease. And the first step is to get a gem skeleton up…

Skeleton

You can create your gem skeleton in many ways, manually, using some specific gem for it, copying an existing gem … One fast and easy one is to use all our favourite tool; Bundler.

Bundler has a gem command that lets you create a new gem from scratch, just give it the name you want for your gem and away you go:

$ bundler gem rspec-simplecov
  create  rspec-simplecov/Gemfile
  create  rspec-simplecov/Rakefile
  create  rspec-simplecov/LICENSE.txt
  create  rspec-simplecov/README.md
  create  rspec-simplecov/.gitignore
  create  rspec-simplecov/rspec-simplecov.gemspec
  create  rspec-simplecov/lib/rspec/simplecov.rb
  create  rspec-simplecov/lib/rspec/simplecov/version.rb
Initializing git repo in /Users/pixie/replay/rspec-simplecov

So this is your default gem setup. The main file of note here is the gemspec file, which we will get to in a minute. You also get a bunch of default files, including a default version file, rspec-simplecov/lib/rspec/simplecov/version.rb that I will talk about further down, and even a default “implementation” of your main file, rspec-simplecov/lib/rspec/simplecov.rb …

Git

As you can see the bundle gem command also initialises a git repository in the newly created gem folder. I’ll assume you are using Git for your revision needs here, if you are not I’m sure you can translate the theory into something that works with your revision system :)

When making a gem, at least when you intend to release it publicly, it’s nice to tag the commits with a version. I’ll let you read the Git book chapter on how that works. The point is that you tag the commits that constiture a release version with that version so you can later do git checkout v0.0.1 and get the code at that version.

This rocks for finding bugs introduced in certain versions of your code since it’s super easy to check the code out at that version, not having to care about what happened later in the project history…

Gemspec

rspec-simplecov/rspec-simplecov.gemspec holds all the meta info on your gem, such as name, author, homepage and the description that will end up on rubygems, as well as being the initiation point when people require your gem.

As per usual in Ruby world the gemspec file is really just a ruby file so you can do all the fancy things you like here. One of the defaults, that I don’t use, is this:

spec.files = `git ls-files -z`.split("\x0")

This line lists all the files you have added in the git repo for the gem and lists them as the gems files. Pretty neat for simple setups but has the drawbacks of making the gems dependencies opaque, I can’t just look in the gemspec to see what files are relevant, I need to use git. In certain places, notably online, it’s hard to do the git ls-files and find out what files are actually in there. It also makes the gem contain ALL the files you track using git and personally I’m not sure I think that the distributed gem should contain its complete test suite and all the vector originals for your fancy icons etc. So I usually list them old-school, by hand as strings in an array…

Version

Another convention is that of the gem version. It has varied over time but nowadays most gem authors seem to follow the lib/[gem name]/version.rb strategy. That basically means that you have a version.rb file that is nested under your gems namespace (more on namespaces later) that basically holds a constant, VERSION, that is the gem version. It looks like this for our example:

module RSpec
  module SimpleCov
    VERSION = "0.0.1"
  end
end

And the lines in the gemspec that use it look like this:

require 'rspec-simplecov/version'

and

spec.version = RSpec::SimpleCov::VERSION

Default values

You also have to fill in some default info in the gemspec before you publish your gem. Things like author and email, unless your local git config already had them, and the description and summary of your gem. Especially the description, the long version of your descriptive texts, is important since that is what ends up on the rubygems.org page when you publish your gem.

Dependencies and the Gemfile

Ok so we all know to add our dependencies in the Gemfile right? But not when you are building a gem (!?!). Still, there is a gemfile, and you do use bundler … This is all so confusing so here it is, straight up and down:

Bundler was not part of the original ruby ecosystem, it got added later for VERY good reasons and has been so successful at solving the problem of version managing our gems that we now take it for granted.

Gems on the other hand have pretty much been there all along, so they had to solve this issue as well, right? So since all the metadata for a gem is in the gemspec, so are all the dependencies. You basically do spec.add_dependency "rspec", "~> 3.3" for your production gems and spec.add_development_dependency "bundler", "~> 1.7" for your development dependencies.

Bundler then piggybacks on this with the special-only-for-gems-development gemspec command. That command makes bundler read the gemspec and get the gem information from there instead. Simple and easy to use. Just don’t add your gems directly to the Gemfile or you’re in for trouble…

Packaging

When you make a gem you will normally want to package it at some point. You can run gems unpacked from local folders, which might be fine for your project, but for a wider audience you want to package.

Normally when we use gems we add them to a Gemfile or install them directly with gem install. In both cases we look up the gem by name at rubygems.org, download the gemfile and install (ie. unpack) it locally in our systems gem folder.

From there it’s basically the same as running from another local folder. You can go to the gem location and read or even change the source of your favourite gems … Let’s package our gem now, just to see how it works.

Dry run

Now that we have filled in all the crud in the gemspec we can make sure it’s correct, at least syntactically, by building the gem using gem build rspec-simplecov. This should give us a gem file:

$ gem build ./rspec-simplecov.gemspec
  Successfully built RubyGem
  Name: rspec-simplecov
  Version: 0.0.1
  File: rspec-simplecov-0.0.1.gem

So, that rspec-simplecov-0.0.1.gem file is the actual gem. This is what we upload to rubygems and what others download when they bundle or gem install our gem.

It’s nothing fancy though, just a slightly schizophrenic zip file. Try renaming it to .zip instead of .gem and open it in your favorite archiving tool. I bet it won’t take you 2 minutes to figure out what the gem build command does :)

Bundler

We can also try it out with bundler to make sure we have registered all of our dependencies correctly.

For the rspec-simplecov gem I have specified rspec and simplecov and runtime dependencies as well as bundler, rake and guard-rspec as development dependencies:

spec.add_dependency "rspec", "~> 3"
spec.add_dependency "simplecov", "~> 0.10"

spec.add_development_dependency "bundler", "~> 1.7"
spec.add_development_dependency "rake", "~> 10.0"
spec.add_development_dependency "guard-rspec", "~> 4.6"

It’s easy to test, jsut run bundle install as normal and make sure 1) It doesn’t fall over and die and 2) it installs/lists all the gems you want/need:

$ bundle install
Fetching gem metadata from https://rubygems.org/............
Fetching additional metadata from https://rubygems.org/..
Resolving dependencies...
Using rake 10.4.2
Using bundler 1.7.9
Using coderay 1.1.0
Using diff-lcs 1.2.5
Installing docile 1.1.5
Installing ffi 1.9.10
Using formatador 0.2.5
Installing rb-fsevent 0.9.5
Using rb-inotify 0.9.5
Installing listen 3.0.3
Using lumberjack 1.0.9
Installing nenv 0.2.0
Installing shellany 0.0.1
Installing notiffany 0.0.7
Using method_source 0.8.2
Using slop 3.6.0
Using pry 0.10.1
Using thor 0.19.1
Installing guard 2.13.0
Installing guard-compat 1.2.1
Installing rspec-support 3.3.0
Installing rspec-core 3.3.2
Installing rspec-expectations 3.3.1
Installing rspec-mocks 3.3.2
Installing rspec 3.3.0
Installing guard-rspec 4.6.4
Using json 1.8.3
Installing simplecov-html 0.10.0
Installing simplecov 0.10.0
Using rspec-simplecov 0.0.1 from source at .
Your bundle is complete!
Use `bundle show [gemname]` to see where a bundled gem is installed.

And that looks about correct to me :)

What we have so far

Ok, so my rspec-simplecov.gemspec, with some minor edits as per above, now looks like this:

# encoding: utf-8
lib = File.expand_path('../lib', __FILE__)
$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
require 'rake'
require 'rspec/simplecov'

Gem::Specification.new do |spec|
  spec.name          = "rspec-simplecov"
  spec.version       = RSpec::SimpleCov::VERSION
  spec.authors       = ["Jonas Schubert Erlandsson"]
  spec.email         = ["jonas@accodeing.com"]
  spec.summary       = "Integrates SimpleCov with RSpec so that low code coverage fails the test suite."
  spec.description   = "Creates an after suite hook in RSpect that dynamically creates and injects a new test case that expects the actual code coverage to be at least the lower limit set in SimpleCov."
  spec.homepage      = "https://github.com/replaygaming/rspec-simplecov"
  spec.license       = "MIT"

  spec.files         = FileList['lib/**/*.rb','spec/**/*.rb'].to_a
  spec.test_files    = spec.files.grep(%r{^spec/})
  spec.require_paths = ["lib"]

  spec.add_dependency "rspec", "~> 3"
  spec.add_dependency "simplecov", "~> 0.10"

  spec.add_development_dependency "bundler", "~> 1.7"
  spec.add_development_dependency "rake", "~> 10.0"
  spec.add_development_dependency "guard-rspec", "~> 4.6"
end

We have already covered the important stuff in there but if you want to have a complete list of available things you can set and what they mean have a look at the official docs, they are great :)

Two more things before we move on: What happens when someone uses your gem and that nagging issue about nested namespaces.

require ‘then/what?’

Ok, so … When your gem is included in someone’s Gemfile the paths to that gem and the relative paths in the gemspec require_paths = ... array are added to the load paths of the project.

This means that if my gem is installed at /users/pixie/ruby/gems/rspec-simplecov and since it has a require_paths value of ["lib"] the complete path that ends up in the load paths is /users/pixie/ruby/gems/rspec-simplecov/lib.

When the host project later require the gem, like require 'rspec/simplecov' in this case, the require gets resolved against the projects load paths and it finds /users/pixie/ruby/gems/rspec-simplecov/lib/rspec/simplecov.rb. And THAT is what is actually loaded.

What happens from here is up to you. That file might include some other files, monkey patch String, set some stuff up automatically or just expose some constant to the including app for later use.

In the case of rspec-simplecov that file looks like this:

require "rspec/simplecov/configuration"
require "rspec/simplecov/setup"
require "rspec/simplecov/version"

module RSpec
  module SimpleCov
    class << self

      def start( simplecov_instance = ::SimpleCov, &block )
        configuration = Configuration.new( simplecov_instance, caller.to_a, &block )

        Setup.do( configuration )
      end

    end
  end
end

As you can see it includes some other files, creates a couple of new modules and a method. That method is what will allow users to call RSpec::SimpleCov.start to affect the RSpec/SimpleCov setup before running their suite.

The configuration inside of it is a minimal DSL that allows users to give a block to the start method to set configuration parameters for the gem, it also stores the values.

Setup holds the actual work and the call to do on it is what triggers the changes to the RSpec configuration that makes it all work.

Nested namespaces

This gem is called rspec-simplecov, by convention that translates into rspec/simplecov for paths and Rspec::Simplecov for constants (but in our case we are using the names from two existing gems so the constants become RSpec::SimpleCov instead, even though that normally would mean the gem should be named r_spec-simple_cov:)).

As you can see hyphen, -, translates into a namespace separator and underscore, _, translates into capitalization on the same constant.

When you create a new gem you will probably not give it a nested namespace, there is normally no reason for it, but in this case it is more of a plugin to an existing gem and rspec plugins are usually, though not always, nested in the RSpec module. So thats what I did.

Remember that the require has a slash in it, not a hyphen. Forgetting that it is a common source of grief, at least for me.

Adding your sauce

Ok, so with all of that out of the way all you have to do is add your actual code. In my case this was simple, since I already had it working in another project. I’d say the best way to create gems is to make the thing you want to do work in the context of a project and then extract it into a gem.

Look at it like sketching before drawing the real thing. You make sure you get all the proportions right and that the big picture works before you get bogged down with details. And if you like TDD you can see it as the design spike. After which you know how you want it to work and can test drive the gem from there.

If your gem is meant for public consumption, or even for an internal team project, consider documenting your code. I won’t write a lot about it here, but consider how much easier all of this was just because there is good documentation for thinks like Git, Bundler and rubygems …

Check out your favourite gem for documentation examples or read about RDoc and consider using it in your code, even if it’s not a gem for public consumption :)

Finally

That about wraps it up. I think you should go make your own gem. I bet you have some part of some application that should be a gem. Remember that you wont have to be mentioned on Ruby5 or make the first page on rubygems.org with every gem. They are excellent for keeping code organised inside a project as well.

Nothing quite says “Use the public f*ng API” as putting the code in a gem, even if it’s just for your team.

You don’t have to publish your gems to enjoy life with bundler. You can install them locally by hand or use the excellent gem server that comes built in with the gem command. Just punch gem server into your favourite shell and watch it go (maybe read some docs as well?).

My point is this: I know you want to make the next rave gem. But building gems that work well and follow common practices normally requires some practise. So build a few gems for you and your team, publish some smaller gems and generally build up the the big reveal.

Making gems isn’t hard, just scary. Make it part of your life now and it will just be routine when the time comes when you want to release a gem to the public and take your shot at the front page :)

Best of luck!

Thanks

Thank you for reading to the end, it’s really appreciated :) I’d also like to thank Pranav from the Replay team for inspiring this article and to Paul at Replay for letting me write blog posts as part of my job!

Also thanks to Pranav, Paul and Felipe for proof reading and commenting, resulting in some new sections on Git and documentation.