Want to stay on top? Ruby Weekly is a once-weekly e-mail newsletter covering the latest Ruby and Rails news.
     Feed Icon

Heckle: Tortures Your Tests For Revealing Confessions

By Peter Cooper / December 21, 2006

Cmbiz041

Heckle is a great new library from Kevin Clark (though Ryan Davis wrote a proof of concept at RubyConf) that 'torments your tests'. It uses ParseRuby and RubyToRuby to rip your code apart and forces random data into your code (currently it does this for strings, symbols, regexps, ranges, booleans, and numbers) to see how good your tests really are.

Simply, it's fuzz testing for your Ruby tests, and that's a good thing if you want your test suites to be strong and far-reaching.

Comments

  1. meekish says:

    Does Heckle have any usefulness if you don't strive for 100% testing coverage?

  2. Peter Cooper says:

    I dare say it'd be an interesting test of code robustness and/or ability to sanitize input. This could be important on, say, a library that interacts with remote services.

  3. matt says:

    If you don't strive for 100% testing coverage, then probably not.

    If you even have to think about testing coverage as something you strive for, then probably not.

    Heckle is designed for Agile shops, where testing coverage is a by-product of the test-first development process. When TDD gets in your blood, you read comments like the one above and wonder how programmers can be writing code without first writing a test to prove that they need to fix something.

    4gauge.blogspot.com

  4. zenspider says:

    Some corrections: It uses ParseTree, not ParseRuby; it is a tool, not a library (you invoke it on the command-line); more important (imo) than mutating literals, it also mutates all branching/looping logic; and while I wrote the proof of concept and Kevin did most of the work since then, I'm still working on it.

  5. zenspider says:

    I should add, YES, heckle is still useful when you're not pushing for 100% test coverage (although, I must ask--why not?). You can invoke heckle on a single method to make sure that it and only it are heckle-proof.

  6. meekish says:

    I brought up the testing coverage question on Aslak Hellesoy's blog as well: http://blog.aslakhellesoy.com/articles/2006/12/19/heckling-with-rspec

    The question of 100% test coverage was first brought to my attention by Jonathan Conway http://www.noodlesinmysandals.com/2006/11/13/so-you-think-you-re-agile

    But I have to agree, if coding with a BDD or TDD mentality, shouldn't 100% test coverage be built in? Perhaps Jonathan's argument implies that testing coverage drops below 100% after refactoring?

Other Posts to Enjoy

Twitter Mentions