Potentially broader role for Selenium
Document Actions
Submitted by
jenn.
on 2005-05-05 13:56.
As I've worked more with Selenium, I've become more confident that we can do substantial testing with it. We can at least use Selenium for developer acceptance testing, to catch obvious mechanical problems before the new code gets to the human testers.
My biggest concern with automation is still that it removes the random
elements introduced by individual testers. There's no way to tell
Selenium, for example, "hit random modules until you find one with
pop-up solutions"; you pretty much have to give it a specific module to
test. A lot of our deepest bugs are found through serendipity: a tester
chooses an odd object to work on, or an unusual path to a feature, and
finds emergent properties where many variables collide.The test suites are DTML documents, though, so it's conceivable that we could introduce some dynamism there. And the test-running mechanism is all JavaScript. It could be cool to hack the test running mechanism to have it pop up prompts for certain variables, like module IDs or search terms.
Brent had a great idea that early in the summer we should have a test-writing day where all the developers sit in a room and convert the existing test plans to Selenium. The basic syntax for Selenium is very simple; the biggest challenges will be in addressing page elements that don't have ids, and in encountering subtle Selenium bugs, of which there are still quite a few as of the current 0.3 release.
If we have enough people and it's going well enough, we should be able to add ids and element names as we go. If we can provide hooks on the spot for new test steps, writing tests should go much faster.
This will of course bring a lot more brains into the middle of the test-writing process. I'm sure at some point, someone will say "We can't make it do exactly X, so would Y accomplish the same thing?" I'll need to write up the guidelines I use for generating test plans from interfaces, so that when questions arise we can answer them systematically.
