UPDATE: using in-dyno postgres puts Heroku CI on par with running Rspec locally!
Experimenting with using Heroku CI for a Rails app, we were a little surprised just how slow Rspec specs run.
As you can see however the runtime cost is about the same since the largest dyno performance-l costs twice as much as the default dyno used by Heroku CI.
Unfortunately, we cannot test using in-dyno Postgres since Heroku does not seem to offer a way to specify the Postgres version for in-dyno instances.
|Mac OS X||Heroku CI "M" dyno||Heroku CI "L" dyno|
|Heroku cost||$ 0.34||$0.36|
Next Step: If we can get parallel tests to run, it will be interesting to see if the overall runtime (and cost) remains constant so we can shorten the test to say 2 minutes by running 16 dynos.
If so that may be GREAT news for using Heroku CI vs Travis CI or Circle CI, whose fixed monthly cost is HUGE for even a few concurrent instances.