| Article Index |
|---|
| Testing Portal functionality without using automated tools |
| Page 2 |
| Page 3 |
| All Pages |
Here are a few tactical approaches to consider as you design your testing efforts -- these are not in any particular order intentionally, as you'll need to think through each and then revisit how these tactics work together.
Risk analysis. Web analytics. Priority levels. History. Exploratory testing. Issue tracking.
A risk analysis is a great step in trying to determine what's important. It is found that a risk analysis when shared with stakeholders might bring tough topics out into discussion as we hash through what is a risk and what ranking we might assign to each risk. Sometimes the risks are obvious and sometimes I've uncovered surprises just by hosting the conversation about risk.
Explain to project stakeholders that not all testing will be addressed but that testing will follow the results of the risk analysis. Both of these facts are important and as obvious as it might sound to a tester, this isn't always obvious to project stakeholders. Risk analysis drives priority in testing. Sometimes people -- upper management and stakeholders especially -- will falsely believe all testing will somehow be accomplished. It is best to communicate that testing will likely never be done but will be executed in order of priority and that a risk analysis is not just a process but an important way to determine that priority order.
Browser analytics. What do customers do on the site? Weblogs can give you lots of information about what takes place on the website you're testing. Common navigations, most frequently purchased items, typical session length, number of pages viewed, browser and O/S usage can all be learned from weblogs. Try to design testing around usage. Weblogs can give you those directional signals.
These two tactics -- assessing risk and then knowing what actually happens on the website help can help guide what testing needs to be covered. Once the workload begins to become larger than you think there will ever be enough time to address, it's time to plan for reality. Design testing in priority bands or levels. What's essential to be tested for every release, even a small release when you have been assured no code change could have impacted X where X equals essential functionality, or functionality that has been assessed high in terms of risk. For instance, on an e-commerce site, the shopping cart is always at the top of risk and priority. Along with cart processing, typically credit card processing and protecting customer data fall into that same high priority band/level. Identify areas that must be tested for each release and learn what the minimum amount of time and staff it takes to plow through that workload. I also like to keep project stakeholders aware of what the bare effort looks like to get through a release.
The number of priority levels or bands that you define may vary. Try to keep things simple and work with one to four levels (even fewer if you can). Level one is must, level two is important but not critical, level three is good to cover and level four is "everything." When a release involves a patch in a certain area of code, that affected area becomes its own high priority.
History. Review defects found and look for similar issues on subsequent releases. People often make similar mistakes and bugs found in one area will likely appear in other areas. Shift testing to cover conditions you suspect based on your experience with the product and the developers. Once some areas of code have "tightened up," shift your test ideas again. This is one reason exploratory testing works so well. Product changes and bugs move with the product. Here's an example based on experience -- credit card processing of expired cards. Once multiple bugs were found in this area and so this area remained high on the priority list for multiple releases. Once the issues had been cleared, still a test or two has to be executed because it was a high risk area but downshift how much time you spent on this area and move onto something else. Historical product knowledge can be a great help in planning testing.
Issue tracking. As your website continues to roll out releases into production, keep a healthy list of what you're not accomplishing -- which is often what we don't enjoy talking about. But it is best to discuss where you feel risks may not be addressed. Share this list with your stakeholders so that they're not lulled into thinking you've got it all figured out. Also keep in touch with customer support to know what issues have been found in production. You might alter your testing based on this feedback. You might be able to identify to your stakeholders the areas that could be covered with more time, staff or a tool.
This issue list will be your list to help grow your team and advocate for potential automated tool purchases. It isn't an excuse list or a list to hide behind, it's a reality list. You will never meet a product that was tested for every condition that you could think of, but you can find many products and websites that worked well under production use. What doesn't get tested is a potential business risk and those risks should be discussed in ongoing dialogs.
Subscribe to this comment's feed
| < Prev | Next > |
|---|
