Jasmine + mocha hell: test don’t fail anymore!

This week I worked on a jascript project and was surprised to be able to write wrong assertions in tests and keep a green build.

The test project was using jasmine/mocha/chai/webpack for tests.

After few investigations it seemed the test was correctly written so the behavior was really surprising. Debugging further I noticed we were passing in a try/catch and the exception was passed to mocha done(e) as expected to say the asynchronous test was failing (if there is a parameter it is considered as failing).

Just to share the idea the test was doing something like:

const execAsync = (done, callback) => {
  runAsyncHelper(() => {
    try {
      callback();
    } catch (ex) {
      done(ex);
    }
  });
};

So all was fine so what was the error? At that moment I was very suspicious on that done function but how to ensure it is the one I expected? In java I would have checked the source of the .class to ensure it was the one I expected but in js? One easy solution I found was to dump the function and compare its content to the one I expected. To do that I added in my catch block this code:

(function(suspiciousFunction){
  console.log(suspiciousFunction.toString());
})(done);

It dumped:

LOG: 'function () {
      if (!called) {
        called = true;
        fn();
      }
    }'

Then quickly checking the source of that function using the "if (!called)" with a grep allowed me to identify the source:

$ grep 'if (!called) {' -R node_modules/
node_modules/jasmine-core/lib/jasmine-core/jasmine.js:      if (!called) {

Jasmine! Jasmine being loaded with mocha overrides its done() function and therefore the behavior is not the expected one at all since Jasmine doesn't support this error parameter to make the build failing.

One easy fix is to replace the done(ex) by a failling assert or done.fail():

const execAsync = (done, callback) => {
  runAsyncHelper(() => {
    try {
      callback();
    } catch (ex) {
      done.fail(ex);
      // or expect(ex).toEqual(null);
    }
  });
};

Last tip: I would recommand you to try to dump all you can in this catch block like page source etc to let it be debuggable without starting the debugger each time (appreciated on continuous integration systems ;)).