Black-Box testing on Window-based Application
Editable Fields Checking and Validation:
- Valid/invalid characters/strings data in all editable fields
- Valid minimum/maximum/mid range values in fields
- Null strings (or no data ) in required fields
- Record length (character limit)in text/memo fields
- Cut/copy/paste into/from fields when possible
- {mosgoogle left}
Not Editable Fields Checking:
- Check for all test/spelling in warnings and error messages/dialogs
- Invoke/check all menu items and their options
Application Usability:
- Appearance an outlook (Placement an alignment of objects on screen)
- User Interface Test (open all menus, check all items)
- Basic functionality checking (File+Open+Save, etc..)
- Right mouse clicking sensitivity
- Resize/min/max/restore app, windows (check min app size)
- Scrollability when applicable (scrollbars, keyboard, autoscrolling)
- Keyboard and mouse navigation, highlighting, dragging, drag/drop
- Print in landscape an portrait modes
- Check F1, What's This , Help menu
- Short-cut and Accelerator keys
- Tab Key order and Navigation in all dialog boxes and menus
Web-Enabled Development and Test automation
Web-Enabled application go further in these areas:
- Web-enabled application are meant to be stateless. HTTP was designed to be stateless. Each request from a Web-enabled application is meant to be atomic and not rely on any previouse requests. This has huge advantages for system architecture and datacenter provisioning. When requests are stateless, then any sserver can respond to the request and any request handler on any server may service the request.
- Web-enabled application are platform independent. The client application may be written for Windows, Macintosh, Linux, and any other platform that is capable of implementing the command protocol and network connection to the server.
- Web-enabled application expect the client application to provide presentation rendering and simple scripting capabilities. The client application is usually a browser, however, it may also be a dedication client application such as a retail cash register, a Windows-based data analysis tool, ot an electronic address book in your mobile phone.
The missing context in a Web-enabled application test automation means that software developers and QA technicians must manually script tests for each Web-enalbled application. Plus, they need to maintain the test scriots as the application changes. Web-enabled application test automation tools focus on making the scriot writing and maintenance tasks easier. The test automation tool offer these features:
- A friendly, graphical user interface to integrate the record, edit, and run-time script functions.
- A recorder that watches how an application is used and writes a test script for you.
- A playback utility that drives a Web-enalbed application by processing the test script and logging. The playback utility also provides the facility to play back several concurrently running copies of the same script to check the system for scalability and load testing.
- A report utility to show how the playback differed from the original recording. The differences may be slower or faster performance times, errors, and incomplete transactions.
Testing Web Sites Applications
Many developers and testers are making the transition from testing traditional client/server, PC, and/or mainframe systems, to testing rapidly changing Web applications.
Web test: This testing forcuses on how well all parts of the web site hold together, whether inside and outside the website are working and whether all parts of the website are connected.
Web sites Server consist of three back-end layer:
- Web server
- Application server
- Data layers
FUNCTIONAL SYSTEM TESTING TECHNIQUES
Requirements Testing Technique :
Usage " To ensure that system performs correctly
" To ensure that correctness can be sustained for a considerable period of time.
" System can be tested for correctness through all phases of SDLC but incase of reliability the programs should be in place to make system operational.
Objectives " Successfully implementation of user requirements
" Correctness maintained over considerable period of time
" Processing of the application complies with the organization's policies and procedures.
" Secondary users needs are fulfilled -
Security officer
DBA
Internal auditors
Record retention
Comptroller
How to Use " Test conditions created
These test conditions are generalized ones, which becomes test cases as the SDLC progresses until system is fully operational.
Test conditions are more effective when created from user's requirements.
Test conditions if created from documents then if there are any error in the documents those will get incorporated in Test conditions and testing would not be able to find those errors.
Test conditions if created from other sources (other than documents) error trapping is effective.
" Functional Checklist created.
When to use " Every application should be Requirement tested
" Should start at Requirements phase and should progress till operations and maintenance phase.
" The method used to carry requirement testing and the extent of it is important.
Examples " Creating test matrix to prove that system requirements as documented are the requirements desired by the user.
" Creating checklist to verify that application complies to the organizational policies and procedures.
Regression Testing Technique, :
Background " One segment of system is developed and thoroughly tested.
" Another segment is changed which has disastrous effect on the tested segment.
" The implemented change, new data or parameters have created change in the already tested segment.
Usage " All aspects of system remain functional after testing.
" Change in one segment does not change the functionality of other segment.
Objectives " Determine -
system documents remain current
System test data and test conditions remain current.
Previously tested system functions properly without getting effected though changes are made in some other segment of application system.
How to Use " Test cases, which were used previously for the already tested segment is, re-run to ensure that the results of the segment tested currently and the results of same segment tested earlier are same.
" Test automation is needed to carry out the test transactions (test condition execution) else the process is very time consuming and tedious.
" In this case of testing cost/benefit should be carefully evaluated else the efforts spend on testing would be more and payback would be minimum.
When to use " When there is high risk that the new changes may effect the unchanged areas of application system.
" In development process: Regression testing should be carried out after the pre-determined changes are incorporated in the application system.
" In Maintenance phase : regression testing should be carried out if there is a high risk that loss may occur when the changes are made to the system
Examples " Re-running of previously conducted tests to ensure that the unchanged portion of system functions properly.
" Reviewing previously prepared system documents (manuals) to ensure that they do not get effected after changes are made to the application system.
" Obtaining printout of data dictionary to ensure that the data elements reflect the changes being incorporated.
Disadvantage " Time consuming and tedious if test automation not done.
Black Box testing for web-based application: (1)
1. Browser functionality:
- Is the browser compatible with the application design?
- There are many different types of browsers available.
2. User Interface 3.Instructions 4. Site map or navigational bar 5. Content 6. Colors/backgrounds |
Black Box testing for web-based application: (2)
7. Images
Whether it's a screen grab or a little icon that points the way,
a picture is worth a thousand words. Sometimes, the best way to tell the user
something is to simply show them. However, bandwidth is precious to the client
and the server, so you need to conserve memory usage. Do all the images add
value to each page, or do they simply waste bandwidth? Can a different file type
(.GIF, .JPG) be used for 30k less?
In general, you don't want large pictures
on the front page, since most users who abandon a page due to a large load will
do it on the front page. If you can get them to see the front page quickly, it
will increase the chance they will stay.
8. Tables
You also want to verify that tables are setup properly. Does
the user constantly have to scroll right to see the price of the item? Would it
be more effective to put the price closer to the left and put miniscule details
to the right? Are the columns wide enough or does every row have to wrap around?
Are certain columns considerably longer than others?
9. Wrap-around
Finally, you will want to verify that wrap-around occurs
properly. If the text refers to "a picture on the right", make sure the picture
is on the right. Make sure that widowed and orphaned sentences and paragraphs
don't layout in an awkward manner because of pictures.
10. Functionality
The functionality of the web site is why your company
hired a developer and not just an artist. This is the part that interfaces with
the server and actually "does stuff".
11. Links
A link is the vehicle that gets the user from page to page. You
will need to verify two things for each link: that the link brings you to the
page it said it would and that the pages you are linking to actually exists. It
may sound a little silly but I have seen plenty of web sites with internal
broken links.
12. Forms
When a user submits information through a form it needs to work
properly. The submit button needs to work. If the form is for an online
registration, the user should be given login information (that works) after
successful completion. If the form gathers shipping information, it should be
handled properly and the customer should receive their package. In order to test
this, you need to verify that the server stores the information properly and
that systems down the line can interpret and use that information.
13. Data verification
If the system verifies user input according to
business rules, then that needs to work properly. For example, a State field may
be checked against a list of valid values. If this is the case, you need to
verify that the list is complete and that the program actually calls the list
properly (add a bogus value to the list and make sure the system accepts it).
14. Cookies
Most users only like the kind with sugar, but developers love
web cookies. If the system uses them, you need to check them. If they store
login information, make sure the cookies work. If the cookie is used for
statistics, verify that totals are being counted properly. And you'll probably
want to make sure those cookies are encrypted too, otherwise people can edit
their cookies and skew your statistics.
Application specific functional
requirements Most importantly, you want to verify the application specific
functional requirements. Try to perform all functions a user would: place an
order, change an order, cancel an order, check the status of the order, change
shipping information before an order is shipped, pay online, ad naseum.
This
is why your users will show up on your doorstep, so you need to make sure you
can do what you advertise.
Black Box testing for web-based application: (3)
16. Interface Testing
Many times, a web site is not an island. The site
will call external servers for additional data, verification of data or
fulfillment of orders.
16. Server interface
The first interface you should test is the interface
between the browser and the server. You should attempt transactions, then view
the server logs and verify that what you're seeing in the browser is actually
happening on the server. It's also a good idea to run queries on the database to
make sure the transaction data is being stored properly.
17. External interfaces
Some web systems have external interfaces. For
example, a merchant might verify credit card transactions real-time in order to
reduce fraud. You will need to send several test transactions using the web
interface. Try credit cards that are valid, invalid, and stolen. If the merchant
only takes Visa and MasterCard, try using a Discover card. (A script can check
the first digit of the credit card number: 3 for American Express, 4 for Visa, 5
for MasterCard, or 6 for Discover, before the transaction is sent.) Basically,
you want to make sure that the software can handle every possible message
returned by the external server.
18. Error handling
One of the areas left untested most often is interface
error handling. Usually we try to make sure our system can handle all of our
errors, but we never plan for the other systems' errors or for the unexpected.
Try leaving the site mid-transaction - what happens? Does the order complete
anyway? Try losing the internet connection from the user to the server. Try
losing the connection from the server to the credit card verification server. Is
there proper error handling for all these situations? Are charges still made to
credit cards? Is the interruption is not user initiated, does the order get
stored so customer service reps can call back if the user doesn't come back to
the site?
19. Compatibility
You will also want to verify that the application can
work on the machines your customers will be using. If the product is going to
the web for the world to use, you will need to try different combinations of
operating system, browser, video setting and modem speed.
20. Operating systems
Does the site work for both MAC and IBM-Compatibles?
Some fonts are not available on both systems, so make sure that secondary fonts
are selected. Make sure that the site doesn't use plug-ins only available for
one OS, if your users will use both.
21. Browsers
Does your site work with Netscape? Internet Explorer? Lynx?
Some HTML commands or scripts only work for certain browsers. Make sure there
are alternate tags for images, in case someone is using a text browser. If
you're using SSL security, you only need to check browsers 3.0 and higher, but
verify that there is a message for those using older browsers.
22. Video settings
Does the layout still look good on 640x400 or 600x800?
Are fonts too small to read? Are they too big? Does all the text and graphic
alignment still work?
23. Modem/connection speeds
Does it take 10 minutes to load a page with a
28.8 modem, but you tested hooked up to a T1? Users will expect long download
times when they are grabbing documents or demos, but not on the front page. Make
sure that the images aren't too large. Make sure that marketing didn't put 50k
of font size -6 keywords for search engines.
Black Box testing for web-based application: (4)
23. Printers
Users like to print. The concept behind the web should save
paper and reduce printing, but most people would rather read on paper than on
the screen. So, you need to verify that the pages print properly. Sometimes
images and text align on the screen differently than on the printed page. You
need to at least verify that order confirmation screens can be printed properly.
24. Combinations
Now you get to try combinations. Maybe 600x800 looks good
on the MAC but not on the IBM. Maybe IBM with Netscape works, but not with
Lynx.
If the web site will be used internally it might make testing a little
easier. If the company has an official web browser choice, then you just need to
verify that it works for that browser. If everyone has a T1 connection, then you
might not need to check load times. (But keep in mind, some people may dial in
from home.) With internal applications, the development team can make
disclaimers about system requirements and only support those systems setups.
But, ideally, the site should work on all machines so you don't limit growth and
changes in the future.
25. Load/Stress
You will need to verify that the system can handle a large
number of users at the same time, a large amount of data from each user, and a
long period of continuous use. Accessibility is extremely important to users. If
they get a "busy signal", they hang up and call the competition. Not only must
the system be checked so your customers can gain access, but many times crackers
will attempt to gain access to a system by overloading it. For the sake of
security, your system needs to know what to do when it's overloaded and not
simply blow up.
Many users at the same time
If the site just put up the
results of a national lottery, it better be able to handle millions of users
right after the winning numbers are posted. A load test tool would be able to
simulate large number of users accessing the site at the same time.
Large
amount of data from each user
Most customers may only order 1-5 books from
your new online bookstore, but what if a university bookstore decides to order
5000 different books? Or what if grandma wants to send a gift to each of her 50
grandchildren for Christmas (separate mailing addresses for each, of course.)
Can your system handle large amounts of data from a single user?
Long period
of continuous use
If the site is intended to take orders for flower
deliveries, then it better be able to handle the week before Mother's Day. If
the site offers web-based email, it better be able to run for months or even
years, without downtimes.
You will probably want to use an automated test
tool to implement these types of tests, since they are difficult to do manually.
Imagine coordinating 100 people to hit the site at the same time. Now try
100,000 people. Generally, the tool will pay for itself the second or third time
you use it. Once the tool is set up, running another test is just a click away.
26. Security
Even if you aren't accepting credit card payments, security
is very important. The web site will be the only exposure some customers have to
your company. And, if that exposure is a hacked page, they won't feel safe doing
business with you.
Black Box testing for web-based application: (5)
27. Directory setup
The most elementary step of web security is proper
setup of directories. Each directory should have an index.html or main.html page
so a directory listing doesn't appear.
One company I was consulting for
didn't observe this principal. I right clicked on an image and found the path
"...com/objects/images". I went to that directory manually and found a complete
listing of the images on that site. That wasn't too important. Next, I went to
the directory below that: "...com/objects" and I hit the jackpot. There were
plenty of goodies, but what caught my eye were the historical pages. They had
changed their prices every month and kept the old pages. I browsed around and
could figure out their profit margin and how low they were willing to go on a
contract. If a potential customer did a little browsing first, they would have
had a definite advantage at the bargaining table.
SSL Many sites use SSL for
secure transactions. You know you entered an SSL site because there will be a
browser warning and the HTTP in the location field on the browser will change to
HTTPS. If your development group uses SSL you need to make sure there is an
alternate page for browser with versions less than 3.0, since SSL is not
compatible with those browsers. You also need to make sure that there are
warnings when you enter and leave the secured site. Is there a timeout limit?
What happens if the user tries a transaction after the timeout?
28 Logins
In order to validate users, several sites require customers to
login. This makes it easier for the customer since they don't have to re-enter
personal information every time. You need to verify that the system does not
allow invalid usernames/password and that it does allow valid logins. Is there a
maximum number of failed logins allowed before the server locks out the current
user? Is the lockout based on IP? What if the maximum failed login attempts is
three, and you try three, but then enter a valid login? What are the rules for
password selection?
Behind the scenes, you will need to verify that server logs are working properly. Does the log track every transaction? Does it track unsuccessful login attempts? Does it only track stolen credit card usage? What does it store for each transaction? IP address? User name?
30. Scripting languages
Scripting languages are a constant source of security holes. The details are different for each language. Some exploits allow access to the root directory. Others allow access to the mail server. Find out what scripting languages are being used and research the loopholes. It might also be a good idea to subscribe to a security newsgroup that discusses the language you will be testing.
31. Web Server Testing Features
* Feature: Definition
* Transactions: The nunber of times the test script requested the current URL
* Elapsed time: The number of seconds it took to run the request
* Bytes transferred: The total number of bytes sent or received, less HTTP headers
* Response time: The average time it took for the server to respond to each individual request.
* Transaction rate: The average number of transactions the server was able to handle per second.
* Transferance: The average number of bytes transferred per second.
* Concurrency: The average number of simultaneous connections the server was able to handle during the test session.
* Status code nnn: This indicates how many times a particular HTTP status code was seen.
Subscribe to this comment's feed
| < Prev | Next > |
|---|
