| Article Index |
|---|
| Practicing Exploratory Testing |
| Page 2 |
| All Pages |
The external structure of Exploratory Testing is easy enough to describe. Over a period of time, a tester interacts with a product to fulfill a testing mission, and reporting results. There you have the basic external elements of Exploratory Testing: time, tester, product, mission, and reporting. The mission is fulfilled through a continuous cycle of aligning ourselves to the mission, conceiving questions about the product that if answered would also allow us to satisfy our mission, designing tests to answer those questions, and executing tests to get the answers. Often our tests dont fully answer the questions, so we adjust the tests and keep trying (in other words, we explore). We must be ready to report our status and results at any time.
An exploratory test session often begins with a charter, which states the mission and perhaps
some of the tactics to be used. The charter may be chosen by the tester himself, or assigned by the test lead or test manager. Sometimes charters are written down. In some organizations,
test cases and procedures are documented so vaguely that they essentially serve as charters for exploratory testing.
Here are some example testing charters for DecideRight, a decision analysis product:
Explore and analyze the product elements of DecideRight. Produce a test coverage
outline.
Identify and test all claims in the DecideRight manual. (either use checkmark/X/?
notation on the printed manual, or list each tested claim in your notes)
Define work flows through DecideRight and try each one. The flows should represent
realistic scenarios of use, and they should collectively encompass each primary
function of the product.
We need to understand the performance and reliability characteristics of DecideRight
as decision complexity increased. Start with a nominal scenario and scale it up in
terms of number of options and factors until the application appears to hang, crash, or
gracefully prevent user from enlarging any further.
Test all fields that allow data entry (you know the drill: function, stress, and limits,
please)
Analyze the file format of a DecideRight scenario and determine the behavior of the
application when its elements are programmatically manipulated. Test for error
handling and performance when coping with pathological scenario files.
Check UI against Windows interface standards.
Is there any way to corrupt a scenario file? How would we know its corrupted?
Investigate the feasibility of writing an automatic file checker. Find out if the
developers have already done so.
Test integration with external applications, especially Microsoft Word.
Determine the decision analysis algorithm by experimentation and reproduce it in
Excel. Then, use that spreadsheet to test DecideRight with complex decision
scenarios.
Run DecideRight under AppVerifier and report any errors.
These above charters might be ambiguous. They are intended to communicate the mission of a test session clearly and succinctly to testers who have already been trained in the expectations, vocabulary, techniques and tools used by the organization.
Remember, in Exploratory Testing we make maximum use of skill, rather than attempting to represent every action in written form.
In freestyle exploratory testing, the only official result that comes from a session of Exploratory Testing is a set of bug reports. In session-based test management, each session of Exploratory Testing also results in a set of written notes that are reviewed by the test lead. It may also result in updated test materials or new test data. If you think about it, most formal written test procedures were probably created through a process of some sort of exploratory testing.
The outer trappings, inputs and outputs to exploratory testing are worth looking at, but it is the inner structure of Exploratory Testing that matters mostthe part that occurs inside the mind of the tester.
Thats where Exploratory Testing succeeds or fails; where the excellent explorer is distinguished from the amateur. This is a complex subject, but here are some of the basics:
Test Design: An exploratory tester is first and foremost a test designer. Anyone can design a test accidentally, the excellent exploratory tester is able to craft tests that systematically explore the product. That requires skills such as the ability to analyze a product, evaluate risk, use tools, and think critically, among others.
Careful Observation: Excellent exploratory testers are more careful observers than novices, or for that matter, experienced scripted testers. The scripted tester need only observe what the script tells him to observe. The exploratory tester must watch for anything unusual or mysterious. Exploratory testers must also be careful to distinguish obervation from inference, even under pressure, lest they allow preconceived assumptions to blind them to important tests or product behavior.
Critical Thinking: Excellent exploratory testers are able to review and explain their
logic, looking for errors in their own thinking. This is especially important when
reporting the status of a session of exploratory tests, or investigating a defect.
Diverse Ideas: Excellent exploratory testers produce more and better ideas than
novices. They may make use of heuristics to accomplish this. Heuristics are mental
devices such as guidelines, generic checklists, mnemonics, or rules of thumb. The diversity of tester temperaments and backgrounds on a team can also be harnessed by savvy exploratory testers through the process of group brainstorming to produce better test ideas.
Rich Resources: Excellent exploratory testers build a deep inventory of tools,
information sources, test data, and friends to draw upon. While testing, they remain
alert for opportunities to apply those resources to the testing at hand.
| < Prev | Next > |
|---|
