How we write readable feature tests with RSpec

Chris Zetter, a developer in the FutureLearn product team, talks about how the team writes feature tests after moving from Cucumber to RSpec in the pursuit of maintainable and readable tests.

Testing is an integral part of building and maintaining a large platform. When we build new parts of the FutureLearn platform we write automated feature tests to document how the new feature should work and to let us know when it doesn’t.

Cucumber Love-hate

Cucumber is a commonly used tool for writing feature specs, and seemed like the obvious choice to use when we started the project. It let us write high level behaviour driven tests from the user’s perspective:

We liked Cucumber because it meant the tests were very easy to create from our user stories, and once written, easy to read. However, there were a few downsides to using Cucumber. Firstly, we were already using RSpec for the rest of the project, so having Cucumber meant another test framework dependency that we had to maintain. Secondly, there was a cognitive overhead of switching between them because of the different DSLs and test runners. Finally, we particularly didn’t like Cucumber’s use of regular expressions, because they made it harder to follow what code is being executed when tests are run as compared to standard method invocation in Ruby.

Writing better RSpec features

So, how could we stop using Cucumber but keep the readable tests that we liked?

We had already started using RSpec features instead of Cucumber, and often they looked like this:

These had a tendency to be quite long, making it difficult to understand what they were testing. There is also no clear separation between the Arrange, Act, Assert parts of the test (known as ‘Given’, ‘When’ and ‘Then’ in the language of Cucumber). We tried adding comments to the steps in the code, but found that these suffered the same fate as comments often do in application code: after time they get out of sync with the actual code.

Typically, a method this long elsewhere in our application code would have been a red flag, which we’d normally refactor through the therapeutic act of method extraction. So why not do just that? Let’s extract those lines and name each method in the same style as a Cucumber step:

What we found

We’ve since removed all of our Cucumber features and converted most of them to these new style RSpec feature specs. They retain the benefit of readability that Cucumber previously provided, yet are much easier to write and maintain.

We made a conscious decision not to optimise for code-reuse of our extracted methods between different feature files as we were worried that it would make the tests harder to follow. We have found that some code reuse tends to happen naturally when writing multiple scenarios around a single feature.

Category Making FutureLearn

Comments (20)

0/1200

  • Aref Aslani

    Amazing 🙂 That’s really good…

  • Vlad

    Do you try to use https://github.com/jimweirich/rspec-given ?

    • (sorry I didn’t see you comment earlier)

      We didn’t know about rspec-given when we choose to write this style of specs. It looks like it’s trying to achieve a similar thing to what we’re doing, and you could even use them together: `Given { there_is_a_course }` but I think it might only be worth the extra syntax if you found that you needed to re-use the methods alot with different Given/and/then prefixes (which we don’t so far).

  • Kosmas Chatzimichalis

    Chris, it is possible to use cucumber with text expressions if you don’t like regular expressions.

    And it makes it much easier for non programmers to read than rspec.

    • Hi Kosmas,
      We found that it was only the programmers in our team that were ever writing and reading the Cucumber features. If this wasn’t it the case it certainly would have been a good reason for us to keep using Cucumber.
      I agree that the `$` syntax is easier to write and maintain than regular expressions, but I think we would still have the same problem that I talked about- the use of variables in the step definition makes the execution path hard to follow as compared to method invocation in Ruby because you cannot just search for the method name.

  • Lukas Oberhuber

    We’ve hit a few of these problems, but have stuck with ‘Cucumber’. We use Turnip (https://github.com/jnicklas/turnip), which uses the Cucumber (well, really Gherkin syntax) and runs in rspec so you only have to maintain one spec_helper.rb. Then, on top of Turnip, we’ve created Rutabaga which allows the Gherkin tests to be called from inside rspec files (https://github.com/simplybusiness/rutabaga). This allows us to run Turnip tests against Rails controllers.

    But the main reason we stick with Gherkin is that we want business users to read and modify our features and rspec doesn’t really facilitate that.

    And the second reason is that we haven’t found a great way to create more extensive rspec comments for rspec tests.

    One final point: I don’t like using Cucumber tests for integration tests or long running tests. For me, Cucumber should be about testing business rules, and integration tests are about making sure it all hangs together. Instead, tag your integration tests with ‘integration’ and by default exclude them in your spec_helper.rb.

    Sorry this is long winded, but hopefully helpful.

  • Thanks for everyone’s comments! It’s really interesting to know what others are doing for their feature specs, and what problems this approach might have.

  • Andrey

    We write with the same way tests on the project:)

  • Gavin Morrice

    Isn’t this a pain to change down the line though?

    What if – using the given example – you were to remove the need for a Course to be present first.

    So `given_there_is_a_course` is out. Now you have to rename your `and_i_am_logged_in_as_a_learner` method to `given_i_am_logged_in_as_a_learner`, or define an alias method for testing both cases. Which is a bit of a pain to manage.

    If that was the case, I’d start to write my own DSL – which would start to look a bit like Cucumber

    • Yes, I was worried about that too. So far I don’t think it has been a problem. The nature of the feature tests means that it’s unlikely that you would have a test that was `Given A, And B` and another test that was `Given B` within the same file where both specs couldn’t start with `Given B`. You can also use RSpec before blocks and contexts to extract setup away from scenarios where this could be a problem. In the specific case of login (which is required setup for the majority of our feature specs) we have extracted it to a block-style rspec helper so this particular problem wouldn’t happen.

      I’ve looked across our features and there’s one place where we have had this problem and we used an `alias_method`.

      • Gavin Morrice

        Cool – well I’ll give this a try 🙂

  • cyle

    Whoah, I literally started doing this on my own just last week. That’s hilarious, glad it made sense to somebody else too. :p

  • Scott Radcliff

    I’ve been doing something similar lately, but without the Gherkin syntax.

    Also, I throw my methods in external files. Either spec_helper or some other helper file that I require. I like to make sure they are accessible on all my specs as needed. You may be doing that too, but I can’t tell from your example.

    My methods are typically like ‘create_new_product`. And then I move all the steps required for that one process into that method. If the steps for creating a product ever change, I have one place to edit, regardless of how many specs depend on creating a product.

    That’s the awesome thing about software dev, there is rarely a right or wrong way. Thanks for sharing.

  • Nicolas

    Very interesting, thanks for sharing.

  • Philippe

    Interesting approach. I would keep using cucumber for the following reasons:

    – 1) cucumber features are often integration tests, so they are slower than an rspec test suite. I run my rspec test suite often – cucumber features from time to time.

    – 2) you can inject parameters in a cucumber step, using methods makes it harder to read:
    `given_there_is_a_course(name: “TDD”, with_student_count: 10)`
    vs
    `Given there is a “TDD” course with 10 students)

    3) you can define cucumber steps using strings:
    `Given “there is a course” do`.
    It even support parameters:
    `Given %|there is a “$course_name” course with $students_count students| do |course_name, students_count|`

    4) When the 3rd step fails, Cucumber formatter will show that the first 2 worked and the 3rd one failed; RSpec will just report that the overall scenario failed. It might be harder to know where the error comes from.

    • Mike Williams

      On point 4: yeah, when you have long-running tests (i.e. not unit tests) in RSpec, it can be difficult to see progress, or understand where/why a test fails. I had similar problems on a project where we were using RSpec with Capybara.

      So, I wrote “rspec-longrun” (https://github.com/mdub/rspec-longrun), which allows you to define “steps”, providing more insight into the progress of a test.

    • Thanks Philippe- I’ve tried to give my thoughts on your points:
      1) We have our features in a separate directory and can run them (or just our other specs) with a rake task so this isn’t a problem for us.
      2) I agree, so we have avoided doing that- instead of your example we’d either write `given_there_is_a_course_named_TDD_with_ten_students` or, since the specifics don’t normally matter when your testing at a high level, `given_there_is_a_course_with_students`
      3) I agree that strings are a bit easier to read to snake case method names but I don’t think by much- remember that we’re writing and reading ruby methods every day in the rest of the code base. It might be different if non-rubyists were writing feature specs. As to the parameter passing I find that it makes Cucumber harder to use and maintain and creates more questions- when do I re-use steps?, what part of the step do I vary?
      4) You’d get a ruby stack trace that would include the line number and name of the failing method/step. I agree that it won’t be as nicely formatted as Cucumber, but again, we write Ruby every day and the stack trace is in the same format as all our other specs.

  • Adam

    Nice idea. Since i saw cucumber i thought i wont use it , but this approach gives you all cucumber adventages with using rspec. +1

  • Marc

    Great writeup. I’ve done similar transitions on both Ruby and Scala code bases. It seems like an obvious solution, hope it catches on more widely.

  • Josh

    That’s an interesting approach. The long method names hurt my eyes, though. 😉