Maintainability
A test focus area defined as the ability to locate and fix an error in the system. Can also be the ability to make dynamic changes to the system environment without making system changes.
Master Test Plan
A plan that addresses testing from a high-level system viewpoint. It ties together all levels of testing (unit test, integration test, system test, acceptance test, systems integration, and operability). It includes test objectives, test team organization and responsibilities, high-level schedule, test scope, test focus, test levels and types, test facility requirements, and test management procedures and controls.
Operability
A test focus area defined as the effort required (of support personnel) to learn and operate a manual or automated system. Contrast with Usability.
Operability Testing
A level of dynamic testing in which the oper ations of the system are validated in the real or closely simulated production environ ment. This includes verification of produc tion JCL, installation procedures and operations proc edures. Operability Testing con siders such fac¬tors as performance, resource con sumption, adherence to standards, etc. Operability Testing is normally performed by Operations to assess the readiness of the system for implementation in the produc tion environment.
Operational Testing
A structural type of test that verifies the ability of the application to operate at an acceptable level of service in the production-like environment.
Parallel Testing
A functional type of test, which verifies that the same input on “old” and “new” systems, produces the same results. It is more of an implementation that a testing strategy.
Path Testing
A white box testing technique that requires all code or logic paths to be executed once. Complete path testing is usually impractical and often uneconomical.
Performance
A test focus area defined as the ability of the system to perform certain functions within a prescribed time.
Performance Testing
A structural type of test which verifies that the application meets the expected level of performance in a production-like environment.
Portability
A test focus area defined as ability for a system to operate in multiple operating environments.
Problem
(1) A call or report from a user. The call or report may or may not be defect oriented. (2) A software or process deficiency found during development. (3) The inhibitors and other factors that hinder an organization’s ability to achieve its goals and critical success factors. (4) An issue that a project manager has the authority to resolve without escalation. Compare to ‘defect’ or ‘error’.
Quality Plan
A document which describes the organization, activities, and project factors that have been put in place to achieve the target level of quality for all work products in the application domain. It defines the approach to be taken when planning and tracking the quality of the application development work products to ensure conformance to specified requirements and to ensure the client’s expectations are met. A
Regression Testing
A functional type of test, which verifies that changes to one part of the system have not caused unintended adverse effects to other parts.
Reliability
A test focus area defined as the extent to which the system will provide the intended function without failing.
Requirement
(1) A condition or capability needed by the user to solve a problem or achieve an objective. (2) A condition or capability that must be met or possessed by a system or system component to satisfy a contract, standard, specification, or other formally imposed document. The set of all requirements forms the basis for subsequent development of the system or system component.
Review
A process or meeting during which a work product, or set of work products, is presented to project personnel, managers, users or other interested parties for comment or approval.
Root Cause Analysis
See Causal Analysis.
Scaffolding
Temporary programs may be needed to create or receive data from the speci fic pro gram under test. This approach is called scaf fold¬ing.
Security
A test focus area defined as the assurance that the system/data resources will be protected against accidental and/or intentional modification or misuse.
Security Testing
A structural type of test which verifies that the application provides an adequate level of protection for confidential information and data belonging to other systems.
Software Quality
(1) The totality of features and characteristics of a software product that bear on its ability to satisfy given needs; for example, conform to specifications. (2)The degree to which software possesses a desired combination of attributes. (3)The degree to which a customer or user perceives that software meets his or her composite expectations. (4)The composite characteristics of software that determine the degree to which the software in use will meet the expectations of the customer.
Software Reliability
(1) The probability that software will not cause the failure of a system for a specified time under specified conditions. The probability is a function of the inputs to and use of the system as well as a function of the existence of faults in the software. The inputs to the system determine whether existing faults, if any, are encountered. (2) The ability of a program to perform a required function under stated conditions for a stated period of time.
Statement Testing
A white box testing technique that requires all code or logic statements to be executed at least once.
Static Testing
(1) The detailed examination of a work product's characteristics to an expected set of attributes, experiences and standards. The product under scrutiny is static and not exercized and therefore its behaviour to changing inputs and environments cannot be assessed. (2) The process of evaluating a program without executing the program. See also desk checking, inspection, walk-through.
Stress / Volume Testing
A structural type of test that verifies that the application has acceptable performance characteristics under peak load conditions.
Structural Function
Structural functions describe the technical attributes of a system.
Structural Test Types
Those kinds of tests that may be used to assure that the system is techni cally sound.
Stub
(1) A dummy program element or module used during the development and testing of a higher level element or module. (2) A program statement substituting for the body of a program unit and indicating that the unit is or will be defined elsewhere.
The inverse of Scaffolding.
Sub-system
(1) A group of assemblies or components or both combined to perform a single function. (2) A group of functionally related components that are defined as elements of a system but not separately packaged.
System
A collection of components organized to accomplish a specific function or set of functions.
Systems Integration Testing
A dynamic level of testing which ensures that the systems integration activities appropriately address the integration of application subsystems, integration of applications with the infrastructure, and impact of change on the current live environment.
System Testing
A dynamic level of testing in which all the components that comprise a system are tested to verify that the system functions together as a whole.
Test Bed
(1) A test environment contaning the hardware, instrumentation tools, simulators, and other support software necessary for testing a system or system component. (2) A set of test files, (including databases and reference files), in a known state, used with input test data to test one or more test conditions, measuring against expected results.
Test Case
(1) A set of test inputs, execution conditions, and expected results developed for a particular objective, such as to exercize a particular program path or to verify compliance with a specific requirement. (2) The detailed objectives, data, pro cedures and expected results to conduct a test or part of a test.
Test Condition
A functional or structural attribute of an application, system, network, or component thereof to be tested.
Test Conditions Matrix
A worksheet used to formulate the test conditions that, if met, will produce the expected result. It is a tool used to assist in the design of test cases.
Test Conditions Coverage Matrix
A worksheet that is used for planning and for illustrating that all test conditions are covered by one or more test cases. Each test set has a Test Conditions Coverage Matrix. Rows are used to list the test conditions and columns are used to list all test cases in the test set.
Test Coverage Matrix
A worksheet used to plan and cross check to ensure all requirements and functions are covered adequately by test cases.
Test Data
The input data and file conditions associated with a specific test case.
Test Environment
The external conditions or factors that can directly or indirectly influence the execution and results of a test. This includes the physical as well as the operational environments. Examples of what is included in a test environ ment are: I/O and storage devices, data files, programs, JCL, communication lines, access control and security, databases, reference tables and files (version controlled), etc.
Test Focus Areas
Those attributes of an application that must be tested in order to assure that the business and structural requirements are satisfied.
Test Level
See Level of Testing.
Test Log
A chronological record of all relevant details of a testing activity
Test Matrices
A collection of tables and matrices used to relate functions to be tested with the test cases that do so. Worksheets used to assist in the design and verification of test cases..
Test Objectives
The tangible goals for assur ing that the Test Focus areas previously selected as being relevant to a particular Business or Struc tural Function are being validated by the test.
Test Plan
A document prescribing the approach to be taken for intended testing activities. The plan typically identifies the items to be tested, the test objectives, the testing to be performed, test schedules, entry / exit criteria, personnel requirements, reporting requirements, evaluation criteria, and any risks requiring contingency planning.
Test Procedure
Detailed instructions for the setup, operation, and evaluation of results for a given test. A set of associated procedures is often combined to form a test procedures document.
Test Report
A document describing the conduct and results of the testing carried out for a system or system component.
Test Run
A dated, time-stamped execution of a set of test cases.
Test Scenario
A high-level description of how a given business or technical requirement will be tested, including the expected outcome; later decomposed into sets of test conditions, each in turn, containing test cases.
Test Script
A sequence of actions that executes a test case. Test scripts include detailed instructions for set up, execution, and evaluation of results for a given test case.
Test Set
A collection of test conditions. Test sets are created for purposes of test execution only. A test set is created such that its size is manageable to run and its grouping of test conditions facilitates testing. The grouping reflects the application build strategy.
Test Sets Matrix
A worksheet that relates the test conditions to the test set in which the condition is to be tested. Rows list the test conditions and columns list the test sets. A checkmark in a cell indicates the test set will be used for the corresponding test condition.
Test Specification
A set of documents that define and describe the actual test architecture, elements, approach, data and expected results. Test Specification uses the various functional and non-functional requirement documents along with the quality and test plans. It provides the complete set of test cases and all supporting detail to achieve the objectives documented in the detailed test plan.
Test Strategy
A high level description of major system-wide activities which collectively achieve the overall desired result as expressed by the testing objectives, given the constraints of time and money and the target level of quality. It outlines the approach to be used to ensure that the critical attributes of the system are tested adequately.
Test Type
See Type of Testing.
Testability
(1) The extent to which software facilitates both the establishment of test criteria and the evaluation of the software with respect to those criteria. (2) The extent to which the definition of requirements facilitates analysis of the requirements to establish test criteria.
Testing
The process of exercising or evaluating a program, product, or system, by manual or automated means, to verify that it satisfies specified requirements, to identify differences between expected and actual results.
Testware
The elements that are produced as part of the testing process. Testware includes plans, designs, test cases, test logs, test reports, etc.
Top-down
Approach to integration testing where the component at the top of the component hierarchy is tested first, with lower level components being simulated by stubs. Tested components are then used to test lower level components. The process is repeated until the lowest level components have been tested.
Transaction Flow Testing
A functional type of test that verifies the proper and complete processing of a transaction from the time it enters the system to the time of its completion or exit from the system.
Type of Testing
Tests a functional or structural attribute of the system. E.g. Error Handling, Usability. (Also known as test type.)
Unit Testing
The first level of dynamic testing and is the verification of new or changed code in a module to determine whether all new or modified paths function correctly.
Usability
A test focus area defined as the end-user effort required to learn and use the system. Contrast with Operability.
Usability Testing
A functional type of test which verifies that the final product is user-friendly and easy to use.
User Acceptance Testing
See Acceptance Testing.
Validation
(1) The act of demonstrating that a work item is in compliance with the original require ment. For example, the code of a module would be validated against the input requirements it is intended to imple¬ment. Validation answers the question "Is the right system being built?” (2) Confirmation by examination and provision of objective evidence that the particular requirements for a specific intended use have been fulfilled. See "Verifica tion".
Variance
A mismatch between the actual and expected results occurring in testing. It may result from errors in the item being tested, incor rect expected results, invalid test data, etc. See "Error".
Verification
(1) The act of demonstrating that a work item is satisfactory by using its predecessor work item. For example, code is verified against module level design. Verification answers the question "Is the system being built right?” (2) Confirmation by examination and provision of objective evidence that specified requirements have been fulfilled. See "Validation".
Walkthrough
A review technique characterized by the author of the object under review guiding the progression of the review. Observations made in the review are documented and addressed. Less formal evaluation technique than an inspection.
White Box Testing
Evaluation techniques that are executed with the knowledge of the implementation of the program. The objective of white box testing is to test the program's state ments, code paths, conditions, or data flow paths.
Work Item
A software development lifecycle work product.
Work Product
(1) The result produced by performing a single task or many tasks. A work product, also known as a project artifact, is part of a major deliverable that is visible to the client. Work products may be internal or external. An internal work product may be produced as an intermediate step for future use within the project, while an external work product is produced for use outside the project as part of a major deliverable. (2) As related to test, a software deliverable that is the object of a test, a test work item.
