Software Testing Social Network

In this article, I will describe the usage of external files for storing of test data and their usage in TestComplete scripts. I will describe some good and bad variants for storing data and thus explain why I chose external files. Also, I will give general examples of functions that work with *.csv files and an example of building and using of *.csv file for search options.

Databases are an essential part of the modern software such as client-server software or web-applications.

Our team has some experience in testing of applications that work with databases. The purpose of this article is to sum up the gathered experience and to share it with those who faced the similar task for the first time.

The article is a small manual with practical ideas that can be useful while writing the test plan for your application. Test cases are gathered with no relation to some definite type of application or DBMS (Database Management System); that is why they do not claim to be taken “as is”. After reading this article, you will be able to adapt them to your concrete product.

Nowadays, software virtualization systems have become widespread. The end user can use virtual applications and desktops. When creating such virtualization systems, developers take into account the customer preferences (the end-user preferences) and usability. That is why, testers face a lot of diverse functionality when testing them. One such functionality is the redirection of sound to the terminal session.

In this article, we will examine the specifics of testing of sound redirection in terminal sessions. We will also consider main notions of digital sound, sound settings, and instruments, which are required for testing.

In this article, we will examine the specific of testing of the software virtualization systems. We will touch upon the following questions:

• test environment configuration;
• testing types;
• nuances of each of testing types for the virtualization systems.

So, we will examine those important things that should be taken into account when thinking over the testing strategy of the software virtualization system.

This material can be useful for those people who test  systems of described type and also for the managers of these projects.

In this article, we will examine the specific of testing of the software virtualization systems. We will touch upon the following questions:

• test environment configuration;
• testing types;
• nuances of each of testing types for the virtualization systems.

So, we will examine those important things that should be taken into account when thinking over the testing strategy of the software virtualization system.

This material can be useful for those people who test  systems of described type and also for the managers of these projects.

In the process of creation of a successful software product, there is an inevitable problem of finding a balance between the quality and the release date of the software product. The testing allows of obtaining a product that satisfies all requirements. But the covering of each product risk with various test cases and compiling them take too long. The correctly prepared testing process should provide a required quality level without exceeding project time and budget . If the time for testing was estimated wrongly, it can lead you either to the late product delivery, or to the decrease of its quality and competitiveness . The estimation of time resources in software testing is a rather complicated and volumetric process but its significance for the creation of the successful project shouldn’t be underestimated.

This article contains recommendations that can help you to obtain more realistic and functional time estimates for the testing of a new project.

This article is devoted to the testing of programs that use virtual desktop acceleration to improve performance. Here we will consider features of virtualization, testing scenarios, some useful tools and tips, and also will give examples of real test cases.

Introduction

If you prepared the testing reports without making the preliminary analysis of the findings before, please start doing it. The correctly prepared report on the results of testing is a powerful tool for the optimization of the software development process. That is why let’s pay special attention to this process.

This article includes practical recommendations concerning the preparation of different types of testing reports and their ready examples.

 

Introduction

If you prepared the testing reports without making the preliminary analysis of the findings before, please start doing it. The correctly prepared report on the results of testing is a powerful tool for the optimization of the software development process. That is why let’s pay special attention to this process.

This article includes practical recommendations concerning the preparation of different types of testing reports and their ready examples.

Many corporations, companies, firms, and other institutions of different kinds of activity need to control access to their computer networks, devices that store information, and to the information itself. In future this problem can be solved by means of inventing new devices. These devices will let working only those people, whose voice, fingerprints or the retina picture match those in the database. However, because of comparatively high prices of these devices, applications, which limit the user access to devices and I/O ports, are used to protect the information. Such applications also let you protect the corporate computer networks from the influence of harmful viruses.

Applications limiting the user access to devices and ports are not only complicated in development but also require special treatment while testing.

This article will be useful for both testing specialists and newbies as soon as the process of GUI testing is represented here as completely as possible.

GUI testing can be performed from three points:

• Usability
• Design, appearance
• Element arrangement, layout.

The first two points are discussed well in the article by Elena Osadchaya “GUI Testing: Basic tips” [1]. Information concerning element arrangement testing can be found on the Microsoft site [2] and that is what we will talk about in this article.

 

Written by:
Elena Futornyak, Junior Tester

Introduction

1. Composing test plan for GUI testing

2. Tools and documents

3. Reporting

3.1. Test documentation

3.2. Bug reports for GUI element arrangement testing

3.3. GUI element arrangement testing report

Conclusion

References

Introduction

Imagine a window where all elements are aligned perfectly and the style remains consistent in all of elements, tabs and boxes. Such harmony will be very pleasant to the eye and a user will enjoy it for sure.

In this article you can find answers for such questions:

• How to compose test plan for element arrangement testing
• Whattoolsarebettertouseandwhy
• Howtocompose test documentation and report testing results

1. Composing test plan for GUI testing

As we mentioned above we have an important task - to test element arrangement up to a single pixel. And first we should compose the main document, which will describe the testing process – test plan.

First of all we should choose the certain rule sets that are the most important for us and formulate them clearly. Then we divide an interface into parts: windows, tabs, functionalities… The individual test plan can be composed for each element in correspondence with the chosen documentation scheme. We will compose testing documentation by the plan [1]:

1.   Elements

- Buttons

- Checkboxes and radiobuttons

- Listboxes and comboboxes

- Editboxes

- Menus

- Windows, tabs, blocks

- Pictograms

- State line

- Input forms

- The structure of interface forms

- Text

- Interaction

- Keyboard

- Visual representation

- Indication

2. Arrangement

- Visual hierarchy

- Effective usage of screen space

- Element sizes.

Each point includes a list of nuances to pay attention at and the values of distance between elements in pixels. In a window where there are only checkboxes we check the corresponding requirements, skip the rest from “Elements” section and continue testing. If the interface part includes a lot of different elements we proceed by the list skipping the absent elements.

Let’s consider two tabs of computer properties window My computer -> Properties and test their interface (Fig. 1 and Fig. 2).

Figure 1. Automatic Updates Tab

Figure 2. RemoteTab

Using the list given above let’s make a draft of basic plan for composing the test plan:

• Properties windows
  • • Buttons OK, Cancel and Apply
    • Distance between buttons in pixels
    • Distance between buttons and borders in pixels
    • Buttons Help and Close
    • Tab Automatic updates
• Radiobuttons
• Comboboxes
• Pictograms
• Structure of interface forms
• Text
• Tab Remote
  • Block alignment
  • Block Remote Assistance
    • Distance between the button Advanced and frame borders
    • Allow Remote Assistance checkbox
    • Text alignment inside the form
    • Block caption alignment in pixels
• Block Remote Desktop
  • • • • Distance between the button Select Remote Users and frame borders
        • Allow users to connect checkbox
    • Text alignment inside the form
    • Block caption alignment in pixels

Now we will look through this list along with the Microsoft recommendations and make the specific test cases. See the results below.

2. Tools and documents

Before start testing let’s se what tools can help us to count the pixels on the screen.

There are 3 the most widely used approaches:

1. Ruler JRuler
2. Microsoft Paint
3. Adobe Photoshop

Let’s consider each one in detail.

JRuler is a small application to easily measure distance or sizes in pixels (Fig. 3). It’s interface is really a ruler where each pixel is marked by a scale line. A ruler can be both horizontal and vertical. The value that corresponds to the current cursor position on the screen is displayed inside. Holding the mouse left button you can easily move a ruler to the proper place. It is good to mach the ruler left border with the first pixel of the considered interface part.

Figure 3. JRuler interface

Advantages of JRuler:

1. Compact.
2. There is Always on top option.
3. Easy to use.
4. Good-looking interface.

Disadvantages:

- The one pixel difference is very hard for eye to catch and so you have to scrutinize.

- The number of pixels is displayed in only one fixed place. If to put a ruler down you cannot see these numbers.

There are a lot of analogues: Jruler PRO, JR Screen Ruler, UDRuler and others.

Microsoft Paint. To use Paint we should first make a screenshot with application window and paste it to Paint. Then we choose View -> Zoom -> Large. And one more step: View -> Zoom -> Show grid. Now we can count the pixels (Fig. 4).

Рис 4. Large Zoom with grid in Paint.

We can also use Pencil tool to mark examined pixels by bright colors and thus make it easier to count these small countless cells.

Advantages of Paint:

1. The size of pixels can be made larger.
2. There is a grid.
3. Color marking makes it easier to demonstrate the bug found.

Disadvantages:

- You have to make screenshots (additional time, efforts and attention).

- Zooming is not smooth.

- There is no ruler.

Adobe Photoshop. This program is a powerful graphics editor and thus it’s not surprising that it’s the easiest tool to deal with pixels. To start working we should make a screenshot and paste it to the new Photoshop document (Fig. 5).

Then choose:

1. View -> Show -> Grid.
2. View -> Rulers.
3. Edit -> Preferences -> Guide, Grid & Slices ->

-> In Grid block we choose Grid every 1 pixel.

4) Edit -> Preferences -> Units & Rulers. In Units block we choose

Rulers: pixels, Type: pixels.

5) Zoom in.

Figure 5. Editing in Photoshop.

Advantages of Photoshop:

1. Smooth zooming.
2. Ruler.
3. Grid.
4. Additional tools.

Disadvantages:

- Program window has to be maximized for the proper work.

- Screenshots should be made.

- It’s not cheap commercial product.

- You should have some special knowledge to work with this program.

As usual, each tool has its advantages and disadvantages. The best way is to have all of them and choose one each time depending on the specific situation.

3. Reporting

3.1. Test documentation

Let’s match the described above basic plan with the Microsoft distance table and represent as Excel spreadsheet.

After that we perform all tests using above mentioned tools and put the results to the table (Table 1). Each test performed has a mark in Status column: passed or failed. Test results can be aggregated to obtain the current quality level of tested object; we can use the formula:

QL = (Passed / Total) * 100%.

Then, as usual, we make a ticket for each failed test, make bug report and put it to the bug-tracking system. Below there are examples of bug reports. It may seems like these both tickets describe the same problem – failed distance – and one ticket would be enough. But actually the Microsoft table has two points:

- what distance between buttons should be;

- what distance between a button and window border should be.

And thus if you just indicate “failed” in test plan you will not give the clear answer what exactly is wrong. This situation also results in two peculiarities of making tickets about element arrangement:

1. Pay close attention on details.
2. Give a screenshot by all means.

When all test cases are passed we form the project report, where all necessary testing details are described. Usually this report is sent to all developers and testers in the project team as well as project manager.

And finally the examples of bug reports and project GUI testing report are given below.

Table 1. Test plan and test passing results.

Responsible: Futornyak Elena
Configuration: Windows XP SP3 x86
Quality Level =	86,67%
Total =	30
Passed =	26
Failed =	4
Test name	Status
My computer -> Properties
OK button have 75 x 23 size in pixels	passed
Cancel button have 75 x 23 size in pixels	passed
Apply button have 75 x 23 size in pixels	passed
OK, Cancel and Apply buttons have the same vertical aligning	passed
Distance between OK and Cancel buttons is 7 pixels	failed
Distance between Cancel and Apply buttons is 7 pixels	failed
Distance between Apply button and bottom window border is 11 pixels	failed
Distance between Apply button and right window border is 11 pixels	failed
Help button is in the right top corner of the window	passed
Close button is in the right top corner of the window	passed
Automatic updates tab
Name of tab has center aligning	passed
4 radio-buttons have the same horizontal aligning	passed
There is no radio-button selected by default	passed
Two combo-boxes are disabled when first option is not selected	passed
After selecting first option two combo-boxes are enabled	passed
Date combo-box has default value	passed
Time combo-box has default value	passed
Red and green icons have the same size	passed
Red and green icons have the same horizontal aligning	passed
First paragraph and radio-buttons have the same horizontal aligning	passed
Remote access tab
Two frames have the same width	passed
Two buttons in frames have the same right aligning	passed
Checkboxes in two frames have the same horizontal aligning	passed
Remote helper frame
Distance between Advanced button and right frame border is 11 pixels	passed
Distance between Advanced button and bottom frame border is 11 pixels	passed
Distance between frame left border and frame name is 9 pixels	passed
Checkbox and frame name have the same horizontal aligning	passed
Remote managing frame
Distance between Advanced button and right frame border is 11 pixels	passed
Distance between frame left border and frame name is 9 pixels	passed
Checkbox and frame name have the same horizontal aligning	passed

3.2. Bug reports for GUI element arrangement testing

[Ticket#1] Distance between OK and Cancel buttons is more than 7 pixels.

Module: My computer properties.

Version: Windows XP SP3 x86

Type: Design

Priority: Low

Steps to reproduce:

1. Right-click My computer, open Properties window.
2. Measure the distance between OK and Cancel button in pixels.

Actual result: The distance is 8 pixels. See screenshot 1.

Expected result: The distance is 7 pixels.

Note: The same distance is between Cancel and Apply buttons.

Screenshot 1.

[Ticket#2] Distance between Apply button and right window border is less than 11 pixels.

Module: My computer properties.

Version: Windows XP SP3 x86

Type: Design

Priority: Low

Steps to reproduce:

1. Right-click My computer, open Properties window.
2. Measure the distance between Apply button and right window border in pixels.

Actual result: The distance is 7 pixels. See screenshot 2.

Expected result: The distance is 11 pixels.

Note: The same problem is between Apply button and bottom window border. Distance is 8 pixels instead of 11.

Screenshot 2.

3.3. GUI element arrangement testing report

Testing report

Module: My computer Properties window

Version: ftp://storage/out/Testing/2009-12-16/

Configuration: Windows XP SP3 x86

Type of testing: GUI testing

Results:

Automatic updates and Remote tabs were tested.

See Results in table:

Quality Level =	86,67%
Total =	30
Passed =	26
Failed =	4

There were founded 2 new bugs:

[Ticket#1] Distance between OK and Cancel buttons is more than 7 pixels.

[Ticket#2] Distance between Apply button and right window border is less than 11 pixels.

Problems:

There were no problems.

Common vision:

Interface looks very good. GUI corresponds to almost all of Microsoft standards.

There are a few low-priority bugs.

Conclusion

In this article we managed to:

1. Ground the complexity and importance of GUI element arrangement testing.
2. Give an algorithm to compose test plan for testing of element arrangement.
3. Compose and execute short test plan for an example.
4. Consider three the most popular tools to measure distance in pixels.
5. Illustrate words with examples of bug reports and project testing report.

References

1. Elena Osadchaya  «GUI Testing: Basic tips» http://www.apriorit.com/our-experience/articles/10-qa-articles/63-gui-testing
2. http://msdn.microsoft.com/en-us/library/aa511279.aspx

Working with the software starts with its installation on the computer. During the installation, user gets the first impression of a product. That’s why this process is very important and claims certain attention.

This article includes practical recommendations on testing of software installation process and also includes a bulk of ready tests that are a good basis for a valid test plan preparation.

 

Written by: QA Specialists of Apriorit Testing Team (Network Direction),

Finally prepared by: Liliya Smolyar, Junior Tester.

Contents

• Installation and uninstallation process notions
• Types of installation
• Testing of the software installation
• Peculiarities of installation testing for various applications
• How OS security tools affect software installation
• Testing of repeated installation
• Installation wizard testing peculiarities
• Conclusion

Introduction

Working with the software starts with its installation on the computer. During the installation, user gets the first impression of a product. That’s why this process is very important and claims certain attention.

This article includes practical recommendations on testing of software installation process and also includes a bulk of ready tests that are a good basis for a valid test plan preparation.

Installation and uninstallation process notions

Installer is a program that performs certain actions for software installation on a user computer such as:

• file copying to a destination folder on a hard disk;
• shortcuts creation;
• changes in the Registry;
• system files changing.

Widespread types of installers:

• Windows: Windows Installer, InstallShield, Macrovision InstallAnywhere, Wise, SetupBuilder, Actual Installer, Smart Install Maker
• Linux: apt, rpm, dpkg, portage.

Software uninstallation  means deletion of all program parts from the system (records in the Registry and in other system files, DLL libraries in the WINDOWS/SYSTEM folder, etc.)

Types of installation

Manual installation is a case when the software installation is performed without the installer help or requires considerable user participation.

Automatic installation is an installation that does not require any user actions except its starting (password entering, license agreement acceptance, etc.).

Self-installation is a fully automatic installation that does not require an initial start of the installation process.

Silent installation. Messages and windows are not displayed during the installation of this type.

Remote installation is a type of installation when the control is performed from the remote computer that is connected via LAN or serial cable.

Clear installation is an installation that is performed in the absence of interfering factors (files of the previous installation of this program, unstable OS mode, etc.).

Testing of the software installation

During the first launch of the installer, it proposes you to choose the installation mode.

The principal moment, on which you should concentrate during the testing, is the control of correspondence of installed components  to those chosen by the user:

• Typical: For less experienced users, components are installed by default.
• Custom: For advanced users, only the components chosen by the user are installed.
• Full: Full installation of all components. All options are included.
• After the installation, it is necessary to check recording of the new data to the system Registry for the correctness, and successful registration of a new application.
• Test it on different OS because the same installation package can install different sets of files for different OS.

Peculiarities of installation testing for various applications

While testing, you should take into account specific of installations of various applications.

Program packages

Pay close attention to the work of the license agreement, i.e. check the program behavior at the illegal license key entering. If the program validates the key via the security server using the internet connection then the following tests can be also performed:

• Try to install the program package with the disabled internet connection.
• Check if it is possible to reuse the same license key on several machines.

Antivirus software

Antivirus software in most cases conflicts with other installed antiviruses. That’s why starting the antivirus installation testing you can single out two main test cases:

• The system already has another antivirus application installed. In this case, the launched installation must warn the user about the possible conflict with the installed antivirus and propose to remove it or to cancel the installation.
• If the firewall is the only PC security tool, check that the antivirus can redefine the protection responsibilities by default to itself during the installation. This can be checked in the Security Center (Control Panel -> Security Center).

Drivers

Driver installation testing should be performed carefully and attentively.

• Make sure in the operability of the device after installation, in case the OS installs standard drivers for it automatically. Usually there are no problems in such cases.
• Test the incompatibility of drivers and the third-party driver installation in case you use “Hardware Installation wizard” when a user is proposed to choose driver and device it will be installed for.
• During the installation of drivers, which are supplied with the software and hardware, check only the correctness of installation and the operability of the device in the end of installation process.
• Check the possibility and correctness of rollback after the canceling of unsigned driver installation by the system after warning about the possible incompatibility.
• Test the installation of unsigned drivers in case such installation is disabled (Control Panel -> System -> Hardware -> Driver Signing).
• Make sure those temporary files, which are not necessary for the work of application (*.tmp…), are deleted in case the program contains files signed with MS certificate.

Games   

Full-screen 3D games are the most requiring for the PC hardware. Many games collect the information about the system at the preliminary stage of installation.

• Check the installation of a game on PC with low technical characteristics (e.g. install it on the corresponding virtual machine). As a result a user should be informed about the mismatch of system requirements.

Mail clients

The installation of mail clients implies the choice of basic settings of work with a client.

• Check various configurations: type of the server access protocol (POP, IMAP4, MAPI), type of authentication and the connection port.

How OS security tools affect software installation

Installation under different users

As is known, for every Windows user there is a folder with his own profile where different system settings are stored. It refers also to the programs installed. That’s why installation testing and starting a program for different users are the separate interesting classes of testing. In this case, the following situations may occur:

• The program is installed for all users by default regardless of who installs it.
• The program is installed by default only for the user who launches the installation.
• There is an option in the installation: “To install the program for all users”/ “To install the program only for the current user”.

It should be mentioned in the specification which of the installation variants your program uses. If it is missed it is necessary to focus on common sense and the program functionality.

Thus, the following tests should  be executed:

• Install the program under one user and then log in as another user. Check the program behavior: either the program launches or it is unavailable and its shortcuts must not appear in Start menu, Desktop, etc.
• Test the program launch for users with limited rights in case it was installed for all users. In this case, the program can simply fall at the attempt of performing some actions (e.g., at the first attempt to write data to logs).

User permissions

• Also it is necessary to test the installation and the first launch of a program for users with different rights. If the user rights are not sufficient for starting of installation, the installation should not run. Special case: a user has rights for program installation but installation ends unsuccessfully. Reason: during the installation program executes different actions with objects for which additional rights are needed, e.g., the right to write in certain folders or to connect to the data base.

UAC impact

User Account Control is a Microsoft Windows component that appeared first in Windows Vista OS. This component inquires the confirmation of actions that require the administrator’s rights to protect the computer from unauthorized use.

• Check the success of software installation with enabled UAC.

There is a certain list of actions that call the UAC message, e.g., writing to the system folders or Registry branches, adding accounts, etc. If the program performs some actions from this list, it must response correctly to the UAC reaction; especially it concerns the silent installation.

It can be enabled in such a way:

Vista:

• Select Control Panel -> User Accounts.
• Turn User Account Control on or off.
• Select the options and click on the OK button.
• Restart your computer to apply the UAC settings.

Win7 has 4 UAC gradations (which differ in the list of events that call UAC message), it is necessary to run testing on the strictest gradation (Always notify). It can be enabled in such way:

• Select Control Panel -> User Accounts.
• Change User Account Control settings.
• Move the slider to the very top.
• Click OK.
• Restart the computer.

DEP impact

Data Execution Prevention is a set of software and hardware techniques that allow you to execute additional testing of RAM contents and to prevent launching of a harmful code.

DEP allows blocking viruses and other malicious software that attempt executing their code out of system memory space that is reserved for Windows and other authorized programs. After discovering that a program uses system memory incorrectly, the DEP closes the program forcibly and displays the appropriate message. Unlike the firewall or antivirus program the DEP feature does not prevent the installation of potentially unsafe programs. Instead of it monitoring is performed to make the programs use system memory in the proper way. By default the DEP feature is enabled only for basic programs and services of Windows OS.

• Test the installation and the first launch of the program with enabled DEP function to make sure that your product uses the memory correctly and the end user will not face problems if DEP is enabled.

DEP can be enabled in such way:

1. Right click on My Computer.

2. Select Properties.

3. In the Advanced tab, select Performance and click Settings.

4. On the Data Execution Prevention tab, select the Turn on DEP for all programs and services except those I select option.

Remote installation and Firewall

Your program must be ready to be installed not from the local computer.

• Try to install the program from the network drive or the share folder. In this case, installation must run successfully, just like from the local disk.

Special case: programs that allow the installation from a single computer to all computers in the network, domain, certain range of ip addresses, etc. Such installation is not rather an object of installation testing, but a separate functionality and is tested accordingly.

Anyway, the remote installation first of all faces the use of Firewall.

• Test the possibility of remote installation with enabled Windows Firewall. Usually, having discovered, that the application connects via the network, firewall proposes to enable or to reject or to add an exception. You can try all 3 cases to look how the user will work with this application.

To  enable Firewallgo to Control Panel -> Windows Firewall -> Change settings. Then launch the remote installation.

During the configuration of environment for testing of remote installations pat attention to Exceptions in the Firewall settings: Control Panel -> Windows Firewall -> Change Settings -> Exceptions. Pay attention to all options that begin with the word Remote. To perform the remote installation Firewall is often disabled but if these options are included to Exceptions they will function and will not allow to execute the installation.

Testing of repeated installation

During the repeated installation it is often proposed to delete, update, recover or modify the installed software.

Uninstall (Remove) – deletion of a program.

• Check if all the components, which the program entered to the system (files, registry records, library links, etc.), were deleted after its uninstallation. Here special utilities for system status monitoring can help.
• Check if the data, created during the work with application, were saved.
• Check if the warning message appears when trying to uninstall running application.

Repair – an application recovery if it was damaged.

• Delete some data from the installation folder and the Registry. After using Repair, all the data must be recovered.

Modify (change) – change of  installation options after its termination.

• Install only the part of available options during the first installation and while using the Modify function add the rest (or cancel the installation of chosen options).

Update – update the program version.

• Check for the existent version update after using Update.
• Make sure that after the installation of updates all created earlier objects open and function correctly.
• Check the possibility of saving the document in the old format in the updated version.
• The program must be tested for correctness after the canceling the update process.

Installation wizard testing peculiarities

It is very important to test the installation wizard as well.

• Test the navigation on wizard pages. Expected results: entered data must be saved; having changed the data on the previous page check for the appropriate change (or the answers reset) on the next pages.
• Make sure that wizard blocks the Next moves if the data were not entered properly or the answer was chosen incorrectly.
• Check the operability of the Cancel (Close) button on any wizard page.
• Perform validation testing for entry field: Unicode and special symbols, empty field; if it is the field to enter the path: inexistent path, path with incorrect structure, etc.

Conclusion

We made sure that software installation is really a very important moment that has its own nuances and bottlenecks. That’s why the installation testing must be performed with account taken of all peculiarities of installation process. The basic moments were covered in this article. Let’s summarize.

Performing the installation testing it is necessary to take into account the following things:

• type of the installation being tested (e.g., how it is executed);
• pecularities of the application which installation must be tested
• features of the OS where the software is being installed
• interaction of the software being installed with the OS security tools
• pecularities of the installation wizard testing
• also the testing nuances of the repeated installation (Uninstall, Repair, Modify, Update)

Wish your product be of high quality!

The licensing mechanism is very important part of the commercial product. Proper testing of it prevent the unauthorized utilizing of your intellectual property.

This article includes brief description of the software licensing types and tools accompanied by the notes about what things are worth paying more attention.

Written by:
Viktoria Paschenko,
Junior Tester of Apriorit Inc.
http://www.apriorit.com

What is licensing and what is it for?

Software just like other objects of the intellectual property, such as music and literature works, is protected from the unauthorized copying by the laws about the author’s rights. These laws suppose that the software publisher keeps some exclusive rights, one of them is the right to produce software copies.

Buying the software user actually buys the license that gives him a right to use this software. The terms and conditions of utilizing the software (for example, the possibility to move it to the other PC, use previous versions) are fixed in the agreement, which accompanies the software delivery. The most widespread is EULA - End User License Agreement. EULA is the contract between a user and the software publisher, which describes the possible ways of software utilizing and also limitations for it.

The Agreement can:

• Describe what changes user can perform on the product
• Limit the user right of copying the product
• Allow user to transfer the product or install it to the portable PC
• Indicate if a user can update the product
• Provide a user with the special rights of the network utilizing of the product.

Usually EULA is provided in the digital form and is shown on the first start of the product. User should agree to its terms before the product installation.

Buying a license a user in fact gets the additional services from the publisher such as free technical support, updates and others.

All licenses are divided into two big groups: free software and proprietary software. There are also different types depending on the number of licensed copies (individual, volume, enterprise licenses etc.), additional services volume (standard, ultimate etc.), the type of the object the license is applied to (per user, per machine, per server etc.).

Software protection tools

Methods of the software protection can be divided into software-based and hardware-based. Software-based methods don’t consider the physical properties of the data storages, special hardware etc. Hardware-based methods use specific hardware (e.g. electronic keys connected to the computer ports) or physical peculiarities of the data storages (CDs, floppy disks) to identify the original software version and protect the product from the illegal utilizing.

Electronic key (hardware key, dongle) is the hardware tool intended for software protection from copying, illegal usage and unauthorized distribution.

Generally hardware key is a microchip or microcontroller that has unique algorithms of functioning. The keys are often plugged via USB or LPT interfaces.

Key is plugged to the certain computer interface. Then the protected program sends data to it by means of the special driver. This data is processed in accordance with the specified algorithm and returned back. If the key response is correct then the program continues working, otherwise it can perform any actions implemented by the developers – for example, turn to the demonstration mode, block the access to some features.

To provide the security of the network software special digital keys are used. It’s enough to use just one key for protection and licensing of the network product (limitation of the number of the program copies, which are working in the network). This key is installed on any workstation or network server.

Serial number or license key . Usually it is the text string (but can be a file) with the specific structure. This key is provided to the customer together with the program version (for example in the box package) or separately. Then, just after the installation, the program asks user to enter the key (or provide the path to the key file) and checks the authenticity of the key by the certain criteria. 

Subject to the check method we can name the following license key types:

Standard key/serial number. In this case program checks the key provided by a user using several conditions, for example: the key should include 25 symbols, 12 of them should be the numbers and 13 should be the letters, number sum should be equal to 60 and the sum of the numeric equivalents for the letters should be equal to 100. If this approach is used per se (without some additional security actions like activation via Internet) then the key can be used for installing the program to the other computers.

Key with the reference to the serial numbers of the computer hardware components. As a rule, vendor uses the mechanism when a user fills the survey at the vendor’s site and sends to the same site the specific computer identifier (hardware id). Using this hardware id the key is generated. Usually the key contains encrypted information about the user, product, number of licenses etc.

If a user upgrades his computer, the protection fails. The authors of the many programs protected by hardware reference are ready to provide a user with the new key. Besides the hardware id vendor can also use serial number of the hard drive, MAC address of the netcard, BIOS control sum and other system properties.

Activation via Internet. Here the unique product serial number is used. When a user installs an application, it asks to enter serial number and then establish Internet connection with the vendor’s system to check if the provided serial number is listed in the actual numbers.

License server. It is the specialized server application or hardware-software complex that enable to centralize license management. If there is big loading then a separated physical server can be provided for the license server. License server stores all licenses bought for the specified number of the product copies and provides a license to the each program start.

Licensing check-list and testing guidelines

When you start the work on the licensing test you should first of all determine what scheme you have, i.e. what protection tools are used and what are their combinations.

Let’s consider the simplest most widespread licensing schemes: with text key, with key file and with hardware key. More complicated schemes are the partial intersection of the following sets or are specific for the each individual product.

Text key

The trial period is often present in this scheme. We’ll consider the check-list with it as it’s more complicated – for the scheme without trial period some tests will be simply omitted.

It’s recommended to check:

- the limitations of the application functionality (if there is such during the trial period) and correspondingly the availability of the full functionality after the successful licensing.

- the possibility of licensing during the trial period

- application response to the input of the incorrect registration data (if the key consists of the several fields then you should make the separate test for the each of them):

а. Key is corrupted

b. Key is absent

c. Key is typed in the wrong case

d. Key contains the extra symbols at the end and/or at the beginning

- application response to the input of the correct registration data

- application response to the input of the registration data provided for the previous versions of the tested application

- unlicensed application behavior after it has been reinstalled during the trial period

- application behavior after the trial period is over

- unlicensed application behavior after it has been reinstalled after the trial period is over

- application response to the system time change forward/backward (the most important here is to keep the trial period correct). Take into account that the system time change should be performed in BIOS because the one performed in the operating system does not have much effect.

- the possibility to register application after the trial period is over.

- application behavior after the registration data has been deleted, for example from the registry (usually after registration the deletion of these data is disabled).

- licensed application behavior after it was reinstalled.

Key file

Here you should pay attention on the aspects:

- Using a file with the correct name but with incorrect content and vice versa.

- Starting application after the file was deleted.

- Saving the file while the application is being reinstalled.

- Application behavior after the file has been replaced:

a. correct -> incorrect,

b. correct -> correct,

c. incorrect -> correct,

d. incorrect -> incorrect.

Hardware key

We should check:

- Application functioning with no special software for hardware key installed. 

- Application functioning without hardware key.

- Unplugging hardware key while the application is running (here we should take into account 2 situations: application has some process run and application is in the standby mode):

a. Permissible waiting time for the key plugging.

b. Correct application behavior after the unplugged key is returned.

с. Correct application behavior if the waiting time is over and the key was not plugged.

- Application functioning if there are some other devices akin to the hardware key plugged (for example, if the key is USB device then it’s good to check its functioning together with flash drive).

General aspects

Frequently it’s required to perform countdown not only of the trial time but also of the program starts during the trial period. Here it’s important to control that after the number of starting attempts is over the trial period is finished too regardless of the actual number of trial days left.

Most software products put the information regarding the registration to the About form (usually called by the About menu item). We should check if this information is present and is correct for any licensing scheme in every state.

If HardwareID is used in the registration process it makes sense to try to propose the HardwareID of the other computer.

Conclusion

Testing licensing you should not forget to check some main things:

• correct finishing of the trial period (if it is present);
• user cannot use a program after trial period is over without licensing;
• limitations of functionality in demo-version;
• user cannot renew the trial period using the system time changing or program reinstallation;
• user cannot license a program with incorrect data.

The correct work of licensing is a guarantee of the proper protection of your product. Don’t spare resources to protect your program from the unauthorized utilizing.

You should remember that especially for this type of testing it’s not enough just to check the correct functioning of the licensing mechanism. Tester should think as a user and try to hack the existent protection by any means. Only such approach gives some confidence in the licensing reliability.

Let your products be of high quality!