S
safety: The capability of the software product to achieve acceptable levels of risk of harm to
people, business, software, property or the environment in a specified context of use. [ISO
9126]
safety testing: The process of testing to determine the safety of a software product.
sanity test: See smoke test.
scalability: The capability of the software product to be upgraded to accommodate increased
loads. [After Gerrard]
scalability testing: Testing to determine the scalability of the software product.
scenario testing: See use case testing.
scribe: The person who has to record each defect mentioned and any suggestions for
improvement during a review meeting, on a logging form. The scribe has to make ensure
that the logging form is readable and understandable.
scripting language: A programming language in which executable test scripts are written,
used by a test execution tool (e.g. a capture/replay tool).
security: Attributes of software products that bear on its ability to prevent unauthorized
access, whether accidental or deliberate, to programs and data. [ISO 9126]
security testing: Testing to determine the security of the software product.
serviceability testing: See maintainability testing.
severity: The degree of impact that a defect has on the development or operation of a
component or system. [After IEEE 610]
simulation: The representation of selected behavioral characteristics of one physical or
abstract system by another system. [ISO 2382/1]
24
simulator: A device, computer program or system used during testing, which behaves or
operates like a given system when provided with a set of controlled inputs. [After IEEE
610, DO178b] See also emulator.
smoke test: A subset of all defined/planned test cases that cover the main functionality of a
component or system, to ascertaining that the most crucial functions of a program work,
but not bothering with finer details. A daily build and smoke test is among industry best
practices. See also intake test.
software quality: The totality of functionality and features of a software product that bear on
its ability to satisfy stated or implied needs. [After ISO 9126]
software quality characteristic: See quality attribute.
software feature: See feature.
software test incident: See incident.
software test incident report: See incident report.
source statement: See statement.
specification: A document that specifies, ideally in a complete, precise and verifiable manner,
the requirements, design, behavior, or other characteristics of a component or system, and,
often, the procedures for determining whether these provisions have been satisfied. [After
IEEE 610]
specification-based testing: See black box testing.
specification-based test design technique: See black box test design technique.
specified input: An input for which the specification predicts a result.
stability: The capability of the software product to avoid unexpected effects from modifications
in the software. [ISO 9126] See also maintainability.
standard software: See off-the-shelf software.
standards testing: See compliance testing.
state diagram: A diagram that depicts the states that a component or system can assume, and
shows the events or circumstances that cause and/or result from a change from one state to
another. [IEEE 610]
state table: A grid showing the resulting transitions for each state combined with each
possible event, showing both valid and invalid transitions.
state transition: A transition between two states of a component or system.
state transition testing: A black box test design technique in which test cases are designed to
execute valid and invalid state transitions. See also N-switch testing.
statement: An entity in a programming language, which is typically the smallest indivisible
unit of execution.
statement coverage: The percentage of executable statements that have been exercised by a
test suite.
statement testing: A white box test design technique in which test cases are designed to
execute statements.
static analysis: Analysis of software artifacts, e.g. requirements or code, carried out without
execution of these software artifacts.
25
static analyzer: A tool that carries out static analysis.
static code analysis: Analysis of program source code carried out without execution of that
software.
static code analyzer: A tool that carries out static code analysis. The tool checks source code,
for certain properties such as conformance to coding standards, quality metrics or data flow
anomalies.
static testing: Testing of a component or system at specification or implementation level
without execution of that software, e.g. reviews or static code analysis.
statistical testing: A test design technique in which a model of the statistical distribution of
the input is used to construct representative test cases. See also operational profile testing.
status accounting: An element of configuration management, consisting of the recording and
reporting of information needed to manage a configuration effectively. This information
includes a listing of the approved configuration identification, the status of proposed
changes to the configuration, and the implementation status of the approved changes.
[IEEE 610]
storage: See resource utilization. {mosgoogle}
storage testing: See resource utilization testing.
stress testing: Testing conducted to evaluate a system or component at or beyond the limits
of its specified requirements. [IEEE 610]
structural coverage: Coverage measures based on the internal structure of the component.
structural test design technique: See white box test design technique.
structural testing: See white box testing.
structured walkthrough: See walkthrough.
stub: A skeletal or special-purpose implementation of a software component, used to develop
or test a component that calls or is otherwise dependent on it. It replaces a called
component. [After IEEE 610]
subpath: A sequence of executable statements within a component.
suspension criteria: The criteria used to (temporarily) stop all or a portion of the testing
activities on the test items. [After IEEE 829]
suitability: The capability of the software product to provide an appropriate set of functions
for specified tasks and user objectives. [ISO 9126] See also functionality.
Software Usability Measurement Inventory (SUMI): A questionnaire based usability test
technique to evaluate the usability, e.g. user-satisfaction, of a component or system.
[Veenendaal]
syntax testing: A black box test design technique in which test cases are designed based upon
the definition of the input domain and/or output domain.
system: A collection of components organized to accomplish a specific function or set of
functions. [IEEE 610]
system integration testing: Testing the integration of systems and packages; testing
interfaces to external organizations (e.g. Electronic Data Interchange, Internet).
26
system testing: The process of testing an integrated system to verify that it meets specified
requirements. [Hetzel]
T
technical review: A peer group discussion activity that focuses on achieving consensus on
the technical approach to be taken. A technical review is also known as a peer review.
[Gilb and Graham, IEEE 1028]
test: A set of one or more test cases [IEEE 829]
test approach: The implementation of the test strategy for a specific project. It typically
includes the decisions made that follow based on the (test) projects goal and the risk
assessment carried out, starting points regarding the test process, the test design techniques
to be applied, exit criteria and test types to be performed.
test automation: The use of software to perform or support test activities, e.g. test
management, test design, test execution and results checking.
test basis: All documents from which the requirements of a component or system can be
inferred. The documentation on which the test cases are based. If a document can be
amended only by way of formal amendment procedure, then the test basis is called a frozen
test basis. [After TMap]
test bed: See test environment.
test case: A set of input values, execution preconditions, expected results and execution
postconditions, developed for a particular objective or test condition, such as to exercise a
particular program path or to verify compliance with a specific requirement. [After IEEE
610]
test case design technique: See test design technique.
test case specification: A document specifying a set of test cases (objective, inputs, test
actions, expected results, and execution preconditions) for a test item. [After IEEE 829]
test case suite: See test suite.
test charter: A statement of test objectives, and possibly test ideas. Test charters are amongst
other used in exploratory testing. See also exploratory testing.
test comparator: A test tool to perform automated test comparison.
test comparison: The process of identifying differences between the actual results produced
by the component or system under test and the expected results for a test. Test comparison
can be performed during test execution (dynamic comparison) or after test execution.
test completion criterion: See exit criteria.
test condition: An item or event of a component or system that could be verified by one or
more test cases, e.g. a function, transaction, quality attribute, or structural element.
test coverage: See coverage.
test data: Data that exists (for example, in a database) before a test is executed, and that
affects or is affected by the component or system under test.
test data preparation tool: A type of test tool that enables data to be selected from existing
databases or created, generated, manipulated and edited for use in testing.
test design: See test design specification.
27
test design specification: A document specifying the test conditions (coverage items) for a
test item, the detailed test approach and identifying the associated high level test cases.
[After IEEE 829]
test design tool: A tool that support the test design activity by generating test inputs from a
specification that may be held in a CASE tool repository, e.g. requirements management
tool, or from specified test conditions held in the tool itself.
test design technique: A method used to derive or select test cases.
test driver: See driver.
test environment: An environment containing hardware, instrumentation, simulators,
software tools, and other support elements needed to conduct a test. [After IEEE 610]
test evaluation report: A document produced at the end of the test process summarizing all
testing activities and results. It also contains an evaluation of the test process and lessons
learned.
test execution: The process of running a test by the component or system under test,
producing actual result(s).
test execution automation: The use of software, e.g. capture/playback tools, to control the
execution of tests, the comparison of actual results to expected results, the setting up of test
preconditions, and other test control and reporting functions.
test execution phase: The period of time in a software development life cycle during which
the components of a software product are executed, and the software product is evaluated
to determine whether or not requirements have been satisfied. [IEEE 610]
test execution schedule: A scheme for the execution of test procedures. The test procedures
are included in the test execution schedule in their context and in the order in which they
are to be executed.
test execution technique: The method used to perform the actual test execution, either
manually or automated.
test execution tool: A type of test tool that is able to execute other software using an
automated test script, e.g. capture/playback. [Fewster and Graham]
test fail: See fail.
test generator: See test data preparation tool.
test harness: A test environment comprised of stubs and drivers needed to conduct a test.
test incident: See incident.
test incident report: See incident report.
test infrastructure: The organizational artifacts needed to perform testing, consisting of test
environments, test tools, office environment and procedures.
test item: The individual element to be tested. There usually is one test object and many test
items. See also test object.
test item transmittal report: See release note.
test level: A group of test activities that are organized and managed together. A test level is
linked to the responsibilities in a project. Examples of test levels are component test,
integration test, system test and acceptance test. [After TMap]
28 {mosgoogle}
test log: A chronological record of relevant details about the execution of tests. [IEEE 829]
test logging: The process of recording information about tests executed into a test log.
test manager: The person responsible for testing and evaluating a test object. The individual,
who directs, controls, administers plans and regulates the evaluation of a test object.
test management: The planning, estimating, monitoring and control of test activities,
typically carried out by a test manager.
Test Maturity Model (TMM): A five level staged framework for test process improvement,
related to the Capability Maturity Model (CMM) that describes the key elements of an
effective test process.
Test Process Improvement (TPI): A continuous framework for test process improvement
that describes the key elements of an effective test process, especially targeted at system
testing and acceptance testing.
test object: The component or system to be tested. See also test item.
test objective: A reason or purpose for designing and executing a test.
test oracle: A source to determine expected results to compare with the actual result of the
software under test. An oracle may be the existing system (for a benchmark), a user
manual, or an individuals specialized knowledge, but should not be the code. [After
Adrion]
test outcome: See result.
test pass: See pass.
test performance indicator: A metric, in general high level, indicating to what extent a
certain target value or criterion is met. Often related to test process improvement
objectives, e.g. Defect Detection Percentage (DDP).
test phase: A distinct set of test activities collected into a manageable phase of a project, e.g.
the execution activities of a test level. [After Gerrard]
test plan: A document describing the scope, approach, resources and schedule of intended{mosgoogle}
test activities. It identifies amongst others test items, the features to be tested, the testing
tasks, who will do each task, degree of tester independence, the test environment, the test
design techniques and test measurement techniques to be used, and the rationale for their
choice, and any risks requiring contingency planning. It is a record of the test planning
process [After IEEE 829]
test planning: The activity of establishing or updating a test plan.
test policy: A high level document describing the principles, approach and major objectives
of the organization regarding testing.
test point analysis (TPA): A formula based test estimation method based on function point
analysis. [TMap]
test procedure: See test procedure specification.
test procedure specification: A document specifying a sequence of actions for the execution
of a test. Also known as test script or manual test script. [After IEEE 829]
test process: The fundamental test process comprises planning, specification, execution,
recording and checking for completion. [BS 7925/2]
test record: See test log.
29
test recording: See test logging.
test repeatability: An attribute of a test indicating whether the same results are produced
each time the test is executed.
test report: See test summary report.
test run: Execution of a test on a specific version of the test object.
test run log: See test log.
test result: See result.
test situation: See test condition.
test script: Commonly used to refer to a test procedure specification, especially an automated
one.
test specification: A document that consists of a test design specification, test case
specification and/or test procedure specification.
test stage: See test level.
test strategy: A high-level document defining the test levels to be performed and the testing
within those levels for a programme (one or more projects).
test suite: A set of several test cases for a component or system under test, where the post
condition of one test is often used as the precondition for the next one.
test summary report: A document summarizing testing activities and results. It also contains
an evaluation of the corresponding test items against exit criteria. [After IEEE 829]
test target: A set of exit criteria.
test tool: A software product that supports one or more test activities, such as planning and
control, specification, building initial files and data, test execution and test analysis.
[TMap] See also CAST.
test type: A group of test activities aimed at testing a component or system regarding one or
more interrelated quality attributes. A test type is focused on a specific test objective, i.e.
reliability test, usability test, regression test etc., and may take place on one or more test
levels or test phases. [After TMap]
testability: The capability of the software product to enable modified software to be tested.
[ISO 9126] See also maintainability.
testability review: A detailed check of the test basis to determine whether the test basis is at
an adequate quality level to act as an input document for the test process. [After TMap]
testable requirements: The degree to which a requirement is stated in terms that permit
establishment of test designs (and subsequently test cases) and execution of tests to
determine whether the requirements have been met. [After IEEE 610]
tester: A technically skilled professional who is involved in the testing of a component or
system.
testing: The process consisting of all life cycle activities, both static and dynamic, concerned
with planning, preparation and evaluation of software products and related work products
to determine that they satisfy specified requirements, to demonstrate that they are fit for
purpose and to detect defects.
30
testware: Artifacts produced during the test process required to plan, design, and execute
tests, such as documentation, scripts, inputs, expected results, set-up and clear-up
procedures, files, databases, environment, and any additional software or utilities used in
testing. [After Fewster and Graham]
thread testing: A version of component integration testing where the progressive integration
of components follows the implementation of subsets of the requirements, as opposed to
the integration of components by levels of a hierarchy.
time behavior: See performance.
traceability: The ability to identify related items in documentation and software, such as
requirements with associated tests. See also horizontal traceability, vertical traceability.
top-down testing: An incremental approach to integration testing where the component at the
top of the component hierarchy is tested first, with lower level components being simulated
by stubs. Tested components are then used to test lower level components. The process is
repeated until the lowest level components have been tested.
U
understandability: The capability of the software product to enable the user to understand
whether the software is suitable, and how it can be used for particular tasks and conditions of
use. [ISO 9126] See also usability.
unit testing: See component testing.
unreachable code: Code that cannot be reached and therefore is impossible to execute.
usability: The capability of the software to be understood, learned, used and attractive to the
user when used under specified conditions. [ISO 9126]
usability testing: Testing to determine the extent to which the software product is
understood, easy to learn, easy to operate and attractive to the users under specified
conditions. [After ISO 9126]
use case testing: A black box test design technique in which test cases are designed to
execute user scenarios.
user acceptance testing: See acceptance testing.
user scenario testing: See use case testing.
user test: A test whereby real-life users are involved to evaluate the usability of a component
or system.
V
V-model: A framework to describe the software development life cycle activities from
requirements specification to maintenance. The V-model illustrates how testing activities
can be integrated into each phase of the software development life cycle.
validation: Confirmation by examination and through provision of objective evidence that
the requirements for a specific intended use or application have been fulfilled. [ISO 9000]
variable: An element of storage in a computer that is accessible by a software program by
referring to it by a name.
verification: Confirmation by examination and through the provision of objective evidence
that specified requirements have been fulfilled. [ISO 9000]
31
vertical traceability: The tracing of requirements through the layers of development
documentation to components.
volume testing: Testing where the system is subjected to large volumes of data. See also
resource-utilization testing.
W
walkthrough: A step-by-step presentation by the author of a document in order to gather
information and to establish a common understanding of its content. [Freedman and
Weinberg, IEEE 1028]
white box test design technique: Documented procedure to derive and select test cases based
on an analysis of the internal structure of a component or system.
white box testing: Testing based on an analysis of the internal structure of the component or
system.
Wide Band Delphi: An expert based test estimation technique that aims at making an
accurate estimation using the collective wisdom of the team members.
32
Annex A (Informative)
Index of sources; the following non-normative sources were used in constructing this
glossary:
[Abbott] J. Abbot (1986), Software Testing Techniques, NCC Publications.
[Adrion] W. Adrion, M. Branstad and J. Cherniabsky (1982), Validation, Verification and
Testing of Computer Software, in: Computing Surveys, Vol. 14, No 2, June 1982.
[Bach] J. Bach (2004), Exploratory Testing, in: E. van Veenendaal, The Testing Practitioner
2nd edition, UTN Publishing, ISBN 90-72194-65-9.
[Beizer] B. Beizer (1990), Software Testing Techniques, van Nostrand Reinhold, ISBN 0-442-
20672-0
[Chow] T. Chow (1978), Testing Software Design Modelled by Finite-Sate Machines, in:
IEEE Transactions on Software Engineering, Vol. 4, No 3, May 1978.
[CMM] M. Paulk, C. Weber, B. Curtis and M.B. Chrissis (1995), The Capability Maturity
Model, Guidelines for Improving the Software Process, Addison-Wesley, ISBN 0-201-
54664-7
[CMMI] M.B. Chrissis, M. Konrad and S. Shrum (2004), CMMI, Guidelines for Process
Integration and Product Improvement, Addison Wesley, ISBN 0-321-15496-7
[Fenton] N. Fenton (1991), Software Metrics: a Rigorous Approach, Chapman & Hall,
London.
[Fewster and Graham] M. Fewster and D. Graham (1999), Software Test Automation,
Effective use of test execution tools, Addison-Wesley, ISBN 0-201-33140-3.
[Freedman and Weinberg] D. Freedman and G. Weinberg (1990), Walkthroughs, Inspections,
and Technical Reviews, Dorset House Publishing, ISBN 0-932633-19-6.
[Gerrard] P. Gerrard and N. Thompson (2002), Risk-Based E-Business Testing, Artech House
Publishers, ISBN 1-58053-314-0.
[Gilb and Graham] T. Gilb and D. Graham (1993), Software Inspection, Addison-Wesley,
ISBN 0-201-63181-4.
[Grochtmann] M. Grochtmann (1994), Test Case Design Using Classification Trees, in:
Conference Proceedings STAR 1994.
[Hetzel] W. Hetzel (1988), The complete guide to software testing, 2nd edition, QED
Information Sciences, ISBN 0-89435-242-3.
[McCabe] T. McCabe (1976), A complexity measure, in: IEEE Transactions on Software
Engineering, Vol. 2, pp. 308-320.
[Musa] J. Musa (1998), Software Reliability Engineering Testing, McGraw-Hill Education,
ISBN 0-07913-271-5.
[Myers] G. Myers (1979), The Art of Software Testing, Wiley, ISBN 0-471-04328-1.
[TMap] M. Pol, R. Teunissen, E. van Veenendaal (2002), Software Testing, A guide to the
TMap Approach, Addison Wesley, ISBN 0-201-745712.
[Veenendaal] E.P.W.M. van Veenendaal (2004), The Testing Practitioner 2nd edition, UTN
Publishing, ISBN 90-72194-65-9.
{mosgoogle}
Subscribe to this comment's feed
| < Prev | Next > |
|---|
