Update February 13th: Join the discussion on Hacker News
If you’re part of the Ruby On Rails community for a long time, you’ve probably read tons of articles about testing Rails application (less these days, though). Although there always have been diverging opinions on the matter, it seems the common wisdom was to say that you had to test everything: models, controllers, views and full-stack tests. Oh, and you had to do all of this with a TDD/BDD mindset as well.
I tried to do this myself and quickly concluded it would lead me right into the abyss. You see, it took me way too long to accept that being a freelancer was not the same as being employed in a trendy company with lots of money and resources. Employees will get paid anyway. If they want they can easily convince themselves and their bosses that testing ERB templates and getting 100% code coverage is the most essential thing in the world. In the past I even heard people say that “every real developers” were striving for 100% code coverage and that “they could not even imagine” a rails developer today not doing TDD. At some point it became a sort of religion. You had the heroes on one side, those who wrote more tests in their lives than actual code, and on the other end you had the “undesirable”, those lazy and bad programmers not committed enough to testing.
If I am all by myself, every single thing I do in my work has to bring me real value, otherwise I am losing my energy, time and money.
One day, I was writing a controller spec to make sure that calling the “index” method with a “get” request would return a 200 status code when I realized how absurd it was.
What the heck was I doing? Where was the value of this test? There was none. If the index method returns a 404, it’s because I didn’t create the damn template yet. Why would I deploy my application at this stage? Someone could object that this test will be useful if I somehow delete the index template by mistake. But come on, do we really want to write tests to defend against this kind of stuff? I know I don’t.
Even though I know there are probably ways to write more valuable controller tests, I decided to drop them and concentrate on other tests. Testing views prove to be an even greater waste of time so I dropped them as well.
What was left for me to test? Unit and full-stack tests. Both give me value but of those two, full-stack tests prove to be the most valuable of all.
Full-stack tests are the ones who give me the most value
For me the main purpose of testing is just to obtain an acceptable level of confidence in my overall application. I don’t want (and don’t have the time) to test every single object on every single case in every single part of the stack.
Here is my preferred and almost too simple workflow:
- Think about the feature
- Write the feature
- Test the feature (RSpec and Capybara)
- Deploy with acceptable level of confidence
The testing part is in #3 exactly where it belongs. That’s right, this means no TDD for me. Doesn’t mean TDD isn’t good, it just means it isn’t essential in order to write good and solid code. Experience and some programming skills is what it takes to do that. And whilst it’s true that I could reverse the order of step #2 and #3, the thing is that with me the “thinking” part often blends with the “writing” part. I think the overall feature, then I start writing and continue my thinking along the way, improving the solution I had thought up initially. When I’m happy with the result, I add my feature tests.
Also, even if Full-stack tests are valuable to me, I don’t test everything. Again, time is my most precious resource, I don’ want to waste it in testing mundane stuff.
My tests will target the specific features I am writing on a given project. The workflow of the feature is what matters more to me. I will write tests to make sure that everything happens in the correct order, and in the correct manner as it was thought up by my brain (Thinking. That’s point #1 in my workflow above!). I will write the “happy path” first and then will write some unhappy tests to make sure that the correct error messages / feedback is given to the user.
Once I have that, I have something valuable and it’s enough for me. I can forget the project and come back a few weeks/months later with a level of confidence high enough to refactor or add new features.
After some years as freelance I came to the same conclusion, just test the features.
I was in some project where the test ratio grew up to 3.x. There was no real benefit from so many test but wasted time. Since then just test the features and not the framework.
There are cases where testing shouldn’t be omitted, #update and #create actions in controllers, helps you to spot changes in validations or attributes.
The love 100% test coverage because it’s a simple number they can optimize for.
Sadly, you can have 100% coverage and things still fail:
x = read_from_stdin()
y = read_from_stdin()
save(x/y)
Easy to get 100% there. Easy to do so without checking for the x/0 scenario.
Precisely, it gives you a false sense of security.
So I tend to agree but I actually came across a really good use case for this recently working on a side project. A client asked that instead of having a one to one relationship for a thing they ended up wanting a many to many relationship at the end of a project. HAD there been tests for this switching ‘title’ to ‘titles’ would have been a little more bearable across a ton of files.
Can you elaborate on this?
did you mean you use integration tests, but there was a case you craved for unit testing?
I meant there were no tests specifically for ‘things that would never break’ mainly templates. Unit tests would have saved the day I think.
Uh, I get what you mean now
I’m in a very similar situation. I don’t always TDD features I work on. But rather than using integration tests, I default to unit tests. I feel like unit tests give me the largest benefit with the least amount of overhead.