| Article Index |
|---|
| Automated Software Testing |
| Page 2 |
| All Pages |
Probably most of the problems listed above will apply. One way to avoid them in the first place would have been to figure out how this system looks inside.
Is there a database? If so, the verification should probably not be performed against the UI but against the database.
Do we need to interface with a supplier? If so, how should this interaction look?
Is the same functionality available via an API? If so, it should be tested through the API, and the UI should just be checked to interact with the API correctly.
This will probably yield a higher number of tests, some of them being much 'smaller' in their resource requirements and executing far faster than the full end-to-end tests. Applying these simple questions will allow us to:
write many more tests through the API, e.g., to cover many boundary conditions,
execute multiple threads of tests on the same machine, giving us a chance to spot race-conditions,
start earlier with testing the system, as we can test each interface when it becomes 'quasi-stable',
makes maintenance of tests and debugging easier, as the tests break closer to the source of the problem,
require fewer machine resources, and still execute in reasonable time.
Here we are not advocating the total absence of UI tests here. The user interface is just another interface, and so it deserves attention too. However we do think that we are currently focusing most of our testing-efforts on the UI. The common attitude, that the UI deserves most attention because it is what the user sees, is flawed. Even a perfect UI will not satisfy a user if the underlying functionality is corrupt.
Neither should we abandon our end-to-end tests. They are valuable and no system can be considered tested without them. Again, the question we need to ask ourselves is the ratio between full end-to-end tests and smaller integration tests.
Unfortunately, there is no free lunch. In order to change the style of test-automation we will also need to change our approach to testing. Successful test-automation needs to:
start early in the development cycle,
take the internal structure of the system into account,
have a feedback loop to developers to influence the system-design.
Some of these points require quite a change in the way we approach testing. They are only achievable if we work as a single team with our developers. It is crucial that there is an absolute free flow of information between the different roles in this team.
In previous projects we were able to achieve this by
removing any spatial separation between the test engineers and the development engineers. Sitting on the next desk is probably the best way to promote information exchange,
using the same tools and methods as the developers,
getting involved into daily stand-ups and design-discussions.
This helps not only in getting involved really early (there are projects where test development starts at the same time as development), but it is also a great way to give continuous feedback. Some of the items in the list call for very development-oriented test engineers, as it is easier for them to be recognized as a peer by the development teams.
To summarize, we figured out that a successful automation project needs:
to take the internal details and exposed interface of the system under test into account,
to have many fast tests for each interface (including the UI),
to verify the functionality at the lowest possible level,
to have a set of end-to-end tests,
to start at the same time as development,
to overcome traditional boundaries between development and testing (spatial, organizational and process boundaries), and
to use the same tools as the development team.
Subscribe to this comment's feed
| < Prev | Next > |
|---|
