Software Testing Social Network

Article Index
How to approach Web Service Testing ?
Page 2
All Pages

In many ways testing a Web service is no different then testing anything else. You still do all the same steps of the test method, but some of them can take on a bit of a different flavor.

Here are a couple of tidbits for testing Web services:

Determining coverage and oracles

When testing Web services, you will find that scheme coverage becomes important. You still probably care about all the same coverage you cared about with non-Web service testing (scenario coverage, requirements coverage, code coverage, and application data coverage), but now you get to add schema coverage to the list.

Schema means not only performing a schema validation against all the response XMLs that come back from the service you're testing, it also includes testing for the right number of maximum and minimum repeating elements, correct reference identification tag values, and other subtle nuances for XML. This can sometimes create an oracle problem. You will find often comparing against a schema along with a mapping document of some sort. A mapping document tells you what data should be stored in what element and when it should be there.

Determining the test procedures

One of the initial problems to be solved when performing Web service testing is management of all the data files (typically XML, but not always). There are tools that do this file management for you (covered briefly below), but you should have a game plan for files that need to be managed manually. When you are testing, typically track three files: a request file, an expected response file, and an actual response file.

A good naming convention is to put a "-rq.xml", "-rs.xml", or "-result.xml" on the end of all my files. That way for each file name you have a set of XMLs that paint the entire picture for that test case. Once you get a naming convention worked out, get the files into source control if you don't have a tool that keeps them all straight for you.

Once you have all these XMLs (sometimes hundreds, sometimes thousands), now you get the joy of keeping them up to date. You will find that schema changes can happen quite often on a project, as can mapping changes. Whenever one of those changes occur, you get the pleasure of updating all those XML files. And remember, if you have 100 test cases, you have 200 files -- assuming you don't bother to update your actual result file since you'll be re-running the test case.

A way to handle most schema changes is to have a Ruby script handy that you can use to make the updates for you programmatically. If you do it enough times, you eventually build a library of scripts for just about every type of change that comes your way. There are some exceptions to that where you may need to do some changes manually, but you will find those changes don't normally affect all the test cases, just a subset.

< Prev