Web Automation, A First Start

written a while back, not posted until today; this was about a year or so ago

My software testing (QA) team has grown over the years, often at a frantic pace. Like many others, I’m sure, there’s always been more work to do than time to do it in, and so saw sharpening has never gotten high enough on the priority stack. Instead the answer was to work more hours or hire more people, and continue with manual testing. As you can imagine that becomes unsustainable and stressful. At one point, while the development team was in the midst of a large re-write project, I carved out a bit of time get started with something that we had been talking about for years: automating some of our testing. This is the beginning of this tale, a story that — for that team — is not yet complete.

Where to start? The products under development are web applications, all custom-built using Ajax and JQuery running on a home-grown web servers and not created with automation in mind at all. One of the developers had even told me that there was “no way” I’d be able to drive the app from any tool or framework. The developers were as busy as ever, so I started poking around to see if I could find any methods that could work for us.

My goals were a bit loose at this point, but I knew that we needed a way to drive our application and check results for regression testing. We would need to be able to do it across multiple browsers, and that it would need to be done — at least initially — by the testing team without much assistance from the developers. Oh, and because this was still a side project, it needed to be free.

I looked at several tools and combinations of tools and decided to start with a combination of Ruby and Watir. My decision was based, I will admit, partially on Watir’s documentation and claims of ease but also on the fact that I wanted an excuse to learn Ruby. Let’s take a look at a few of those first faltering steps together.

The first difficulty was that of determining when the application was actually ready for interaction. This application never loads a new page or changes the URL at all. My first attempts were admittedly ugly and involved sleeping in loops waiting for elements to be created. Bad, bad, bad. Don’t do this. On the upside, here at least the developers had added id properties that I could use to locate the elements I needed on the login page.

require 'watir-webdriver'
username = 'AUserName'
password = 'APassword'
b = Watir::Browser.new :ie
puts 'loading page'
puts 'logging on as ' + username
b.text_field(:id => 'UserNameText').set username
b.text_field(:id => 'PasswordText').set password
b.checkbox(:id, 'agreeCheckBox').set
b.button(:id, 'LoginButton').click
d = b.div :id,'documents'
until  d.exists?
  # puts 'watiting for documents page'
  sleep 1
puts 'login complete'

The next step was to navigate to a specific page. Here I’m performing some user actions and then waiting for a particular item to become available. This is a slightly better method than the sleep above; looping until a specific element is set to a known text string. Error handling takes care of things until that condition becomes true, which makes the loop a bit longer, and perhaps more ugly, but it’s at least a bit informative. My debugging-via-stdout lines remain for your entertainment:

# go to Passenger List
# click the menus
b.link(:id, 'navPassenger').click
b.link(:id, 'PassengerList').click
# wait for the page
d = b.div(:class,'pageTitle')
  until 'Passenger List'.eql?(d.text)
  rescue Watir::Exception::UnknownObjectException
  rescue Selenium::WebDriver::Error::ElementNotVisibleError
  rescue Selenium::WebDriver::Error::StaleElementReferenceError
s = b.select_list(:id => 'jumpPerPage').select '100'
puts 'List displayed: ' + d.text 
d = nil

At this point I was in a place to actually look at what’s important – to make sure that all the rows displayed had a memberID. There’s nothing here to see if the data’s actually correct; just to see that the app’s not returning any records that don’t have data in a particular column. This does assume that this page has all the data displayed, that there’s no pagination. Also, looping through through the .count method isn’t speedy, but for this proof-of-concept that wasn’t a concern.

# Get the rows of the table (assuming there is just one dataTable)
table_trs = b.div(:class, 'dataTable').table.tbody.trs
rows = table_trs.count
puts 'total rows on page: ' + rows.to_s

#Find how many rows have data in the 5th cell
rows_with_data = table_trs.count{ |tr| tr.td(:index, 4).text != '' }
puts 'rows with MemberID: ' + rows_with_data.to_s

Here’s where I began to run into the second challenge, one that is pretty prevalent not only in this application’s code but wherever developers don’t have testability as an up-front design priority. Since the app’s pages are written to be as multi-purpose as possible — from the developers’ point of view — there are many places where they “just know” the structure of the elements and fill them with the correct data, the elements aren’t given id properties at all. That means that I also had to know the structure and code to it, and that when the application changes this test code will need to change as well. This application also lets the user determine which columns are shown, which will further complicate matters to say the least. Suffice to say, there’s a lot of room for improvement here.

Note that there’s no actual testing (or checking) being done here; no desired outcomes are defined and the script doesn’t report any sort of failure or success, it just shows some information about what the application’s displaying. At this point, though, I had enough to show my testing team what could be done if they’d dig into the world of automation, and to show our management what could be done if they’d provide some time for us to do so. It was also pretty obvious that I needed to get the development team involved, discussing ways of making the application more testable from the get-go, using elements, controls, and properties in a more well-defined and consistent manner. We also started looking at other tools, expanding our goals and perhaps even adding some funding.

As they say on TV, “to be continued…” — though the rest of that team’s story will written by others.

Moored Once More

Forms have been completed. anchors are down, and the next phase of my career has begun. This week, I signed aboard my new post as part of telerik’s software testing tools group.

at the office in Austin TX  (photo hosted by Flickr)

The department I’m joining is in Austin Texas. No, we’re not moving. I’ll be working out of my home office (and Atlanta Hartsfield-Jackson International Airport). As telerik’s Test Studio Evangelist, I’ll be engaging with the testing community: attending and speaking at conferences, visiting customer sites, blogging, webcasting, and the like, all with the purpose of raising awareness of methods & techniques of software test automation.

Most of all, I’m looking forward to learning a lot. Fortunately I’m going to be surrounded by some of the best people in software development & testing.

The adventure begins!

TDD is a Habit

Up-front admission: I am not “a software developer.” I dabble in software development. I work with software developers; I test and support their code and, though I write some code (and some of that has even shipped to customers as part of various products) I am not “officially” a developer. I don’t even play one on TV. What I am is someone who enjoys working with, learning from, and exploring the software development zeitgeist. For the past year or so, perhaps longer, that has included Test Driven Development (TDD).

I'm test-driven!The basic idea behind TDD is that a developer should not only write code to test their code (in itself a recommended practice, though not widely enough embraced), but that those tests should be written first. In a way, this falls into the “measure twice, cut once” idea of making sure you’re doing the right thing. If you’re a software developer and haven’t heard of TDD, stop right now and go start reading.

Friday I was working on a project, a small application I’ve been writing to compare data on multiple systems. I had started off earlier in the week rather idealistically, designing a few classes and writing the tests for them first. That all went well. Then I got to the “real work” of the app, opening connections to the databases, getting data from them, combining & comparing data. I ran into some difficulties, and I must have spent about two hours trying to figure it out including about an hour of one of our senior developers’ time. Once we found the problem, it was one of those simple “oh, I should have seen that!” moments. Problem solved, only three “man-hours” spent.

Then this morning, over my Cheerios and coffee, I’m reading a little of Robert Martin‘s Clean Code, specifically Chapter 7 on Error Handling. There’s a section on writing one’s try-catch-finally statements first, and while I’m thinking about that I realize that – though I did that Friday – what I didn’t do was to write any tests for all that database code. I was catching exceptions and all that, but I’d never actually tested what the catch was reporting. Doh! If I’d taken half an hour to do things “right” on Friday morning, the afternoon would have been a few hours shorter.

So why didn’t I do it “right” the first time? Because it’s not a habit. It’s not the way I’m used to working. Even though I know I should, when things get rolling it’s just not the first thing that my brain and hands jump to. And again, I’m not a full-time, “real” developer with years and years of ingrained habits.

Peter Provost writes that Kata are the Only Way to Learn TDD. I don’t know about only, but without forcing yourself into some repetition, the habits won’t be formed and you’ll too easily find yourself three hours into fixing a problem that shouldn’t have existed in the first place.

Intro/Refresher Resources for SQL

SQL iconI was asked to find a few references for our people here; folks who are new to SQL or just haven’t used it in a while. Rather than have my notes tied up in email, I figured I’d put them here for all to use. We use both Oracle Database (10g now, 11g soon) and Microsoft SQL Server.

For a good SQL intro/refresher, I would recommend Murach’s SQL Server 2008 for Developers: Training & Reference by Bryan Syverson and Joel Murach. It is, as the title suggests, targeted toward the use of Microsoft SQL Server, but the 90+% of SQL Language features will apply to other SQL-based databases as well.

added later: also check out Craig Earl‘s SQL for Testers.

Another very good resource is Learning SQL, by Alan Beaulieu; this book covers the SQL language as a whole, without being specific to any one implementation.

[added later] As you progress, these also look to be good if you’re interested in learning more about SQL development:

For a practice environment, you can download and setup either MS SQL-Server Express or Oracle Express on your computer.

There are some basic differences between Oracle and Microsoft SQL Server, I’d recommend wandering through these if you, like us, will be playing in both worlds:

The first difference that we will see right away (from the StackOverflow question above): Transaction control. In Oracle everything is a transaction and it is not permanent until you COMMIT. In SQL Server, there is (by default) no transaction control. An error half way through a stored procedure WILL NOT ROLLBACK the DDL in previous steps. There are settings that can change it, but this is the default behavior.

There are also excellent courses available at Pluralsight, and they have a 10-day free trial available. You may also wish to check out the SQL University (start here).

Enjoy, and let me know if you have other resources that have worked well for you.

Don’t Just Say…

Advice for my software testers, though it applies in many other parts of life as well:

Don’t just say “this doesn’t work.”

Use something like “This is what I did, this is what I expected, this is what I saw instead, here are the things I’ve tried, and this is what I think is wrong.”

Optimally, follow with “and here’s how I think it can be fixed.”

It’s important to Learn Why Things Work.

Practices of an Agile Developer

This past week, I read Practices of an Agile Developer, not because I’m currently employed as a developer, nor because my employer uses them, but because I believe the guidelines are good ones to be practiced no matter what the profession. Check out some readers’ notes, then invest some time in the book yourself, especially if you are in the software development arena.

Is your site intentionally broken?

As a follow-up to my previous post, Is Your Text Broken, see also John Gruber’s article today on Tynt, the Copy/Paste Jerks:

All they’re really doing is annoying their readers. Their websites are theirs, but our clipboards are ours. Tynt is intrusive, obnoxious, and disrespectful. I can’t believe some websites need to be told this.

Another example of intentionally broken websites. It’s sad, really.

It’s the Little Things

Application development, especially User Interface, is very much about “the little things.” The things that make life easier for users. Here’s a simple example, one that bugs me almost every time I use a browser.

There are probably URLs that you type often and, for whatever reason, you don’t bother to make into bookmarks or favorites. Maybe you’re just a keyboard person, and Control-L type-type-type is easier for you. Maybe you’ve got them memorized and there are too many to fit nicely in your tool bar. Whatever.

So you alt-Tab to your Firefox window, click in the address bar (or use the Control-L keyboard shortcut) and start typing a URL. If the letters you type are part of an address you’ve used before, it shows you a list.

screenshot from Firefox

This is all well and good, and Internet Explorer does something similar. It’s fine. You type, you mouse or cursor down, you go on with your day.

Until you use Chrome.

Control-L, type type type, and…

Screenshot from Chrome
(There really was also a menu, Chrome just didn’t leave it down for my my screen-capture software to grab in the image.)

Hey, check that out – the URL that matches my typing and that I use the most is already in my address bar. I don’t have to cursor- or mouse-down to it, I can just drop my right pinkie on the Enter key and zoom, away we go. Imagine that. Control-L, face[enter] and I’m on my way to Facebook. Control-L, netf[enter] and I’m looking at my Netflix queue. No more hunting through a drop-down list unless I’m looking for something different than usual.

It’s not a big deal but as with most conveniences, once you’ve gotten used to it everything else is just a little more frustrating.

So – what’s frustrating your users/customers, what’s your competitor is making easier for them?