tl;dr
you are using poltergeist and your konacha tests fail to start with an error like the following,
Timed out waiting for response to {"name":"visit","args":["http://127.0.0.1:58265/"]}....
then you should add
Capybara.register_driver :slow_poltergeist do |app|
Capybara::Poltergeist::Driver.new(app, :timeout => 2.minutes)
end
Konacha.configure do |konacha|
require 'capybara/poltergeist'
konacha.driver = :slow_poltergeist
end
to your konacha configuration file.
The Details
Here at SmartLogic, we embrace automated testing. We use an in-house continuous integration (CI) server running Jenkins to test each of our projects on every commit. We isolate projects from one another on this server by having each project run in a dedicated Vagrant virtual machine.
Some Background
For projects in which make heavy use of JavaScript, we write both the code and the tests in CoffeeScript. As always, we strive for well-defined separation of responsibilities in our code, so the specs for each class only include the relevant application code, rather than having each spec require the entirety of the application code. On this project, we also happen to be using backbone, so there are a non-trivial number of small JavaScript classes. Because PhantomJS is headless (not to mention very VERY fast) we use it on our CI server as the Capybara driver whenever possible.
A Problem Arises
Recently the tests for one of our projects started flapping (sometimes passing and sometimes failing) because the JavaScript tests wouldn’t start executing. Individual tests weren’t failing, the entire set of tests just failed to start. Tellingly, when we ran the same tests on our development boxes (beefy iMacs), they never displayed this behavior. Everything started smoothly and tests passed.
Over time, the flapping failures on the CI server became more frequent until eventually they always failed (homework: think about why our initial response to the flapping failures of “oh, just re-run the tests” was a very bad thing). Since we could not reproduce the failure on any of our development machines, we stopped the test job for that project and ssh-ed into the CI server, and from there into the test VM itself.
Investigating the Problem
Running “rake konacha:run” manually produced the same failure we were seeing in CI, so
we started the rails server from the VM and visited it in a web browser. That worked, so we re-ran “rake konacha:run.” Surprisingly, the tests started promptly and ran successfully. In fact, once we did this, the tests ALWAYS started up. The only way we could reproduce the failure to start was by starting a fresh VM. Now we had a clue.
Carefully comparing the rails logs from our development machine and the fresh VM while running “rake konacha:run” on both, we noticed that both logs contained many entries like
Served asset /models/walrus.js - 200 OK (102ms)
Served asset /views/walrus_pen.js - 200 OK (53ms)
we ALSO saw entries in the log from the fresh VM like
Compiled models/walrus.js (705ms) (pid 5588)
Compiled views/walrus_pen.js (276ms) (pid 5588)
Understanding the Problem
Now we understood the problem. When rails receives a request for “/models/walrus.js,” it looks first in the public/assets folder for such a file. If one is not found, it will attempt to generate that file by preprocessing (compiling) app/assets/javascripts/models/walrus.js.coffee. Since we were making a deliberate effort to limit dependencies between our classes, each test file makes a separate request for its associated application code. Note the difference from a production environment, where all the individual files are combined and served as a single application.js file. The test VM was just slow enough that phantomjs’ timeout was being exceeded.
Solving the Problem
To fix the problem, we needed to tell PhantomJS that it is OK if the initial page load takes a little bit of time, like so:
Within our existing config/initializers/konacha.rb, we defined a new Capybara driver which will use PhantomJS with a longer initial timeout than the default, and then set konacha to use that driver instead.
Capybara.register_driver :slow_poltergeist do |app|
Capybara::Poltergeist::Driver.new(app, :timeout => 2.minutes)
end
Konacha.configure do |konacha|
require 'capybara/poltergeist'
konacha.driver = :slow_poltergeist
end
Instantly, our CI jobs resumed running (and passing!).
