Steven's Notebook

Look Ma - No Hands!

Web Automation, A First Start

written a while back, not posted until today; this was about a year or so ago

My software testing (QA) team has grown over the years, often at a frantic pace. Like many others, I’m sure, there’s always been more work to do than time to do it in, and so saw sharpening has never gotten high enough on the priority stack. Instead the answer was to work more hours or hire more people, and continue with manual testing. As you can imagine that becomes unsustainable and stressful. At one point, while the development team was in the midst of a large re-write project, I carved out a bit of time get started with something that we had been talking about for years: automating some of our testing. This is the beginning of this tale, a story that — for that team — is not yet complete.

Where to start? The products under development are web applications, all custom-built using Ajax and JQuery running on a home-grown web servers and not created with automation in mind at all. One of the developers had even told me that there was “no way” I’d be able to drive the app from any tool or framework. The developers were as busy as ever, so I started poking around to see if I could find any methods that could work for us.

My goals were a bit loose at this point, but I knew that we needed a way to drive our application and check results for regression testing. We would need to be able to do it across multiple browsers, and that it would need to be done — at least initially — by the testing team without much assistance from the developers. Oh, and because this was still a side project, it needed to be free.

I looked at several tools and combinations of tools and decided to start with a combination of Ruby and Watir. My decision was based, I will admit, partially on Watir’s documentation and claims of ease but also on the fact that I wanted an excuse to learn Ruby. Let’s take a look at a few of those first faltering steps together.

The first difficulty was that of determining when the application was actually ready for interaction. This application never loads a new page or changes the URL at all. My first attempts were admittedly ugly and involved sleeping in loops waiting for elements to be created. Bad, bad, bad. Don’t do this. On the upside, here at least the developers had added id properties that I could use to locate the elements I needed on the login page.

require 'watir-webdriver'
username = 'AUserName'
password = 'APassword'
b = Watir::Browser.new :ie
puts 'loading page'
b.goto('https://testsite.mycompany.org/')
puts 'logging on as ' + username
b.text_field(:id => 'UserNameText').set username
b.text_field(:id => 'PasswordText').set password
b.checkbox(:id, 'agreeCheckBox').set
b.button(:id, 'LoginButton').click
d = b.div :id,'documents'
until  d.exists?
  # puts 'watiting for documents page'
  sleep 1
end
puts 'login complete'

The next step was to navigate to a specific page. Here I’m performing some user actions and then waiting for a particular item to become available. This is a slightly better method than the sleep above; looping until a specific element is set to a known text string. Error handling takes care of things until that condition becomes true, which makes the loop a bit longer, and perhaps more ugly, but it’s at least a bit informative. My debugging-via-stdout lines remain for your entertainment:

# go to Passenger List
# click the menus
b.link(:id, 'navPassenger').click
b.link(:id, 'PassengerList').click
# wait for the page
d = b.div(:class,'pageTitle')
begin
  until 'Passenger List'.eql?(d.text)
  end
  rescue Watir::Exception::UnknownObjectException
    $stdout.write('.')
    retry
  rescue Selenium::WebDriver::Error::ElementNotVisibleError
    $stdout.write(',')
    retry
  rescue Selenium::WebDriver::Error::StaleElementReferenceError
    $stdout.write('+')
    retry
end
s = b.select_list(:id => 'jumpPerPage').select '100'
puts 'List displayed: ' + d.text 
d = nil

At this point I was in a place to actually look at what’s important – to make sure that all the rows displayed had a memberID. There’s nothing here to see if the data’s actually correct; just to see that the app’s not returning any records that don’t have data in a particular column. This does assume that this page has all the data displayed, that there’s no pagination. Also, looping through through the .count method isn’t speedy, but for this proof-of-concept that wasn’t a concern.

# Get the rows of the table (assuming there is just one dataTable)
table_trs = b.div(:class, 'dataTable').table.tbody.trs
rows = table_trs.count
puts 'total rows on page: ' + rows.to_s

#Find how many rows have data in the 5th cell
rows_with_data = table_trs.count{ |tr| tr.td(:index, 4).text != '' }
puts 'rows with MemberID: ' + rows_with_data.to_s

Here’s where I began to run into the second challenge, one that is pretty prevalent not only in this application’s code but wherever developers don’t have testability as an up-front design priority. Since the app’s pages are written to be as multi-purpose as possible — from the developers’ point of view — there are many places where they “just know” the structure of the elements and fill them with the correct data, the elements aren’t given id properties at all. That means that I also had to know the structure and code to it, and that when the application changes this test code will need to change as well. This application also lets the user determine which columns are shown, which will further complicate matters to say the least. Suffice to say, there’s a lot of room for improvement here.

Note that there’s no actual testing (or checking) being done here; no desired outcomes are defined and the script doesn’t report any sort of failure or success, it just shows some information about what the application’s displaying. At this point, though, I had enough to show my testing team what could be done if they’d dig into the world of automation, and to show our management what could be done if they’d provide some time for us to do so. It was also pretty obvious that I needed to get the development team involved, discussing ways of making the application more testable from the get-go, using elements, controls, and properties in a more well-defined and consistent manner. We also started looking at other tools, expanding our goals and perhaps even adding some funding.

As they say on TV, “to be continued…” — though the rest of that team’s story will written by others.

Updated: April 24, 2013 — 10:16 am
Steven's Notebook © 2000-2014 Frontier Theme