When not to use instance variables in RSpec

Using RSpec there is some confusion about the differences between let, let!, and instance variables in specs. I'd like to focus on how instance variables work in RSpec in combination with before :context blocks, and in what kind of scenarios you should and should not use them.

Why use instance variables?

The advantage of declaring instance variables in before :context (or before :all) blocks is that whatever value is assigned is only queried or calculated once for many specs. The before :context block is only executed once for all specs in that context. Those specs can use the instance variable without repeating the same setup for every spec, which should speed up the test suite.

Reading the above it might be tempting to put a lot of spec setup in before :context blocks. A fast test suite creates happy developers, right? But there's a downside to using instance variables in RSpec, which could make for very unhappy developers.

From the RSpec docs:

It is very tempting to use before(:context) to speed things up, but we recommend that you avoid this as there are a number of gotchas, as well as things that simply don't work.

[...]

Instance variables declared in before(:context) are shared across all the examples in the group. This means that each example can change the state of a shared object, resulting in an ordering dependency that can make it difficult to reason about failures.

The RSpec docs give us a warning about changing values of the instance variable. State can leak between specs using instance variables defined in a before :context block this way.

Let's look at some examples of specs using instance variables and in what scenarios in will break.

Specs sharing instance variables

In the example below specs only assert if the value of the instance variable matches the expected value. Since the instance variable is not changed, all the specs will pass. If the instance variable value took a long time to query or calculate we have saved that time for two specs in this file.

# spec/lib/example_1_spec.rb
describe "Example 1" do
  before :context do
    # Imagine this being a complex value to prepare
    # This block is only run once in the `describe "Example 1"` block
    @my_instance_variable = :my_value
  end

  it "spec 1" do
    expect(@my_instance_variable).to eql(:my_value)
  end

  it "spec 2" do
    expect(@my_instance_variable).to eql(:my_value)
  end
end

$ bundle exec rspec spec/lib/example_1_spec.rb --order defined
Finished in 0.00223 seconds
2 examples, 0 failures

(I'm using --order defined in the examples in this post so that the spec execution order is predictable and reproducible.)

But specs can be more complicated than this. They may pass the instance variable to some other part of the app, which modifies the given value. This is where things go wrong, what the RSpec docs warn us about.

Reassigning the instance variable

If changing the instance variable is the problem, let's reassign it and see what happens in other specs.

In the example below the "spec 1" spec changes the instance variable to test a slightly different scenario.

# spec/lib/example_2_spec.rb
describe "Example 2" do
  before :context do
    # Imagine this being a complex value to prepare
    @my_instance_variable = :my_value
  end

  it "spec 1" do
    @my_instance_variable = :new_value
    expect(@my_instance_variable).to eql(:new_value)
  end

  it "spec 2" do
    expect(@my_instance_variable).to eql(:my_value)
  end
end

$ bundle exec rspec spec/lib/example_2_spec.rb --order defined
Finished in 0.00223 seconds
2 examples, 0 failures

In this example "spec 2" does not fail, even though "spec 1"—which runs before "spec 2"—changes the instance variable. There was no need for us to reset the original value of the instance variable at the end of the spec even though we changed it.

The way that RSpec runs the specs ensures that every spec uses the original instance variables. Every spec in RSpec is its own Ruby class, in which the spec is performed. Before the spec class run, RSpec sets the instance variables from the before :context block on the spec class. When an instance variable is reassigned in a spec, it only reassigns it on that spec class instance. It doesn't not reassign the instance variable on the same scope as the before :context instance variables are stored, and so does not interfere with any other specs. An example of how this looks can be found later on in this post.

This behavior doesn't always quite work though. Let's see what happens if we use a bit more complex value. That way we know the limitations of using instance variables in RSpec.

Modifying instance variable values

In the next example the @my_instance_variable is assigned a more complex value: an array with multiple values. We will intentionally break the spec in this scenario.

In "spec 1" we're testing a slightly different scenario again, modifying the value before running the assertion. Instead of reassigning the variable we're adding a value to the array on @my_instance_variable.

# spec/lib/example_3_spec.rb
describe "Example 3" do
  before :context do
    @my_instance_variable = [:one, :two]
  end

  it "spec 1" do
    @my_instance_variable << :three
    expect(@my_instance_variable).to eql([:one, :two, :three])
  end

  it "spec 2" do
    expect(@my_instance_variable).to eql([:one, :two])
  end
end

$ bundle exec rspec spec/lib/example_3_spec.rb
Failures:

  1) Example 3 spec 2
     Failure/Error: expect(@my_instance_variable).to eql([:one, :two])

       expected: [:one, :two]
            got: [:one, :two, :three]

       (compared using eql?)

       Diff:
       @@ -1 +1 @@
       -[:one, :two]
       +[:one, :two, :three]

     # ./spec/lib/example_3_spec.rb:14:in `block (2 levels) in <top (required)>'

Finished in 0.01596 seconds (files took 0.10576 seconds to load)
2 examples, 1 failure

Failed examples:

rspec ./spec/lib/example_3_spec.rb:12 # Example 3 spec 2

Unlike before, the "spec 2" spec has now failed. It fails because the instance variable still has the value from the first spec. State has leaked from "spec 1" into "spec 2". Let's look at how the values from these instance variables have moved between specs.

How instance variables work in RSpec

To recap what we learned from the examples earlier:

Assigning instance variables in before :context means they'll only be assigned once, as the before :context block is only once run before all specs in the spec context.
Every spec in RSpec is performed as its own class, with its own scope. Instance variable from the before :context block are set on the spec class before it is performed.
Instance variable values do not leak between specs when the values are basic Ruby objects such as Symbols, numbers, etc.
Instance variable values do leak between specs when the values are more complex objects such as Arrays, Strings, Class instances, etc.

Let's take a closer look at how RSpec handles instance variables for spec classes to see how this could break in our test suite.

How RSpec handles instance variables

When RSpec runs specs in a context, it first runs the before :context blocks. After a before :context block is executed RSpec then stores the list of the instance variables on the class it creates for the context.

When RSpec then starts a spec in that context it creates a new class for that spec and sets instance variables of that context on the spec class.

Let's look at how this works using the same scenario from the second example, where we reassigned the instance variables.

class Context # RSpec context class
  def self.before_context_ivars
    # Where the `before :context` instance variables are stored
    @ivars ||= {}
  end

  def self.set_before_context_ivars_on(spec_instance)
    # Set instance variables from `before :context` on spec instance
    before_context_ivars.each do |name, value|
      spec_instance.instance_variable_set(name, value)
    end
  end
end

class Spec # RSpec spec class
  def run # During spec
    @var = 2 # Reassign instance variable
  end
end

# Mock a `before :context` block and
# store an instance variable on the Context class
Context.before_context_ivars[:@var] = 1 # Basic Ruby object value
# Initialize the spec class
spec_instance = Spec.new
# Set the `before :context` instance variables on the spec
Context.set_before_context_ivars_on(spec_instance)

puts "Context @var before spec:",
  Context.before_context_ivars[:@var].inspect

# Run the spec
spec_instance.run

puts "Spec @var:", spec_instance.instance_variable_get(:@var).inspect
puts "Context @var after spec:",
  Context.before_context_ivars[:@var].inspect

(Simplified example for demonstration purposes.)

Context @var before spec:
1
Spec1 @var:
2
Context @var after spec:
1

RSpec sets the original value of the instance variable on every spec class instance. Reassigning the instance variable in the spec class does not modify the value of the instance variable on the context. It only changes it on the spec class instance.

When the instance variable value was modified however, the value stored on the Context class is also modified, as it is still the same value.

# Using the same context class as from the previous example

# Mock a `before :context` block and
# store an instance variable on the Context class
Context.before_context_ivars[:@var] = [1, 2] # More complex Ruby object
# Initialize the spec class
spec_instance = Spec.new
# Set the `before :context` instance variables on the spec
Context.set_before_context_ivars_on(spec_instance)

puts "Context @var before spec:",
  Context.before_context_ivars[:@var].inspect

# Run the spec
spec_instance.run

puts "Spec @var:", spec_instance.instance_variable_get(:@var).inspect
puts "Context @var after spec:",
  Context.before_context_ivars[:@var].inspect

Context @var before spec:
[1, 2]
Spec @var:
[1, 2, 3]
Context @var after spec:
[1, 2, 3]

In this example we can see the instance variable's value has changed, because the value was modified, rather than the instance variable being reassigned. This value was modified in memory, also changing the value in the context instance variable storage, which causes the modified state to leak into other specs.

Pointers

What happens is, RSpec (or rather Ruby) is assigning pointers to the values in memory when it sets the instance variable values on the spec class instance. Variables in Ruby are pointers to a place in the application's memory. Assigning a value to another variable does not make a copy of it, but points to the same location in memory.

When RSpec sets the instance variables, it doesn't set a copy of the original Array value. Instead it sets the pointer to the Array value in memory. If the Array in memory has changed during the spec run, it will set not the original value for the next spec, but the modified Array instead. This is part of how Ruby works, this is not something RSpec can "fix". And which is why the RSpec docs warn us about using instance variables in before :context blocks.

Alternatives

To prevent state from leaking into other specs by modified values, it's possible to "freeze" objects in Ruby. If we freeze an Array, String or other object instance, Ruby will not allow any modifications.

var = [1, 2].freeze
var << 3
# Raises an error to prevent modification
# => FrozenError (can't modify frozen Array: [1, 2])

But this will be more difficult to do for larger objects with nested objects, as it only freezes the top object and not all nested objects.

var = [[1, 2], [4, 5]].freeze
var[0] << 3
puts var.inspect
# => [[1, 2, 3], [4, 5]] # The nested value was modified

Alternatively it's possible to deep clone or dup the object. The problem with this is that it will take up a lot more memory, as every object will be kept in memory multiple times, so I can't recommend it.

Conclusion

RSpec scoping instance variables between specs in classes is a great help from RSpec for basic Ruby objects, but this behavior shouldn't be relied upon for more complex Ruby objects such as Arrays, Strings, and other object instances.

If a spec is modifying an instance variable value, you can't be sure what the value of that instance variable will be in the next spec. State may leak to other specs, breaking them in unexpected ways. This will be especially difficult to track down when the specs are run in a random order each time.

Make sure that if you use instance variables, you absolutely do not modify any value set on the instance variable if you want a predictable and reproducible test suite. And that's something we should all want. I'm all for fast test suites, but what I like more is a stable test suite.

A big thanks to Benoit Tigeot for fact checking this article!

Blog