Software Testing

Overview

To frame our discussion, consider:

Why do we need a conceptual structure to guide the testing process?

How do black-box and white-box testing differ?

What is the economic guidelines of software testing, and how does it relate to different methods?

OUTLINE

• The Purpose of Software Testing
• Testing in the Lifecycle
• Objectives
• Testing Principles
• White-Box Testing
• Basis Path Testing
• Loop Testing
• Black-Box Testing
  
  
• References

What is the purpose of software testing?

a. To demonstrate that the product performs each function intended;
b. To demonstrate that the internal operation of the product performs according to specification and all internal components have been adequately exercised;
c. To increase our confidence in the proper functioning of the software.
d. To show the product is free from defect.
e. All of the above.

Testing in the Lifecycle

Objectives

1. Testing is a process of executing a program with the intent of finding an error.

2. A good test case is one that has a high probability of finding an as yet undiscovered error.

3. A successful test is one that uncovers an as yet undiscovered error.

A Problem of Economics

With large systems, it is almost always true that more testing will find more defects. The question is not whether all the defects have been found, but whether the cost of discovering the remaining defects can be justified. Hence, strategies for testing should be adopted that optimize the effort expended.

Testing Principles

1. All tests should be traceable to customer requirements.
2. Tests should be planned long before testing begins.
3. 80% of errors are traceable to 20% of the modules (Pareto Principle - Error prone modules)
4. Testing should begin in the small and progress to larger components.
5. Exhaustive testing is NOT possible.
6. Testing is more effective when conducted by an independent party. (SQA?)

White-Box Testing

1) guarantee all independent paths in a module have been tested (exercised) at least once;

2) exercise all logical decisions for both true and false conditions;

3) execute all loops at their boundary values and within their operational bounds;

4) exercise internal data structures to ensure their validity.

Basis Path Testing

Method

2. Determine the cyclomatic complexity of the flow graph.

3. Determine the basis set of independent paths. (The cyclomatic complexity indicates the number of paths required.)

4. Determine a test case that will force the execution of each path.

Flow Graphs

Sequence
IF
While
Repeat
Case

Example

Procedure Average
* This procedure computes the average of 100 or fewer
numbers that lie between bounding values.  It also
computes the sum and the total number of valid entries.
INTERFACE RETURNS  average, total.input, total.valid;
INTERFACE ACCEPTS  value, minimum, maximum;
TYPE  value[1:100] IS SCALAR ARRAY;
TYPE average, total.input, total.valid, minimum, maximum,
sum IS SCALAR;
TYPE  i IS INTEGER;

i=1;
total.input=total.valid=0;
sum=0;
DO WHILE value[i]<>-999 and total.input<100 
increment total.input by
1; IF value[i]>=minimum AND value[i]<=maximum
     THEN increment total.valid by 1;
             sum=sum+value[i]
     ELSE  skip
ENDIF
increment i by 1;
ENDDO
IF total.valid>0
     THEN average=sum/total.valid;
     ELSE average=-999;
ENDIF
END AVERAGE

Step 1: Construct Flow Graph
a) Identify predicate nodes

Procedure Average
* This procedure computes the average of 100 or fewer
numbers that lie between bounding values.  It also
computes the sum and the total number of valid entries.
INTERFACE RETURNS  average, total.input, total.valid;
INTERFACE ACCEPTS  value, minimum, maximum;
TYPE  value[1:100] IS SCALAR ARRAY;
TYPE average, total.input, total.valid, minimum, maximum,
sum IS SCALAR;
TYPE  i IS INTEGER;
i=1;
total.input=total.valid=0; { 1 }
sum=0;
DO WHILE 

value[i]<>-999  { 2}

and 

total.input<100 { 3}

increment total.input by 1;  { 4 }
IF 

value[i]>=minimum   { 5}

AND 

value[i]<=maximum  { 6}

     THEN increment total.valid by 1;  { 7 }
             sum=sum+value[i]
     ELSE  skip

ENDIF
increment i by 1;  { 8 }

ENDDO  { 9 }

IF total.valid>0  { 10 }

     THEN average=sum/total.valid;  { 11 }

     ELSE average=-999;  { 12 }

ENDIF  { 13 }
END AVERAGE

Step 2: Determine Cyclomatic Complexity

V(G) = E - N + 2

V(G) = 17 - 13 + 2 = 6

Step 3: Determine the basis set of independent paths.

1-2-10-11-13

1-2-10-12-13

1-2-3-10-11-13

1-2-3-4-5-8-9-2 ...

1-2-3-4-5-6-8-9-2 ...

1-2-3-4-5-6-7-8-9-2 ...

Step 4: Prepare test cases.

Path	Test Case
1-2-10-11-13
1-2-10-12-13
1-2-3-10-11-13
1-2-3-4-5-8-9-2 ...
1-2-3-4-5-6-8-9-2 ...
1-2-3-4-5-6-7-8-9-2 ...

Loop Testing

Black Box Testing

Tests are designed to answer questions such as:

1) How is functional validity tested?

2) What classes of input make good test cases?

3) Is the system particularly sensitive to certain input values?

4) How are the boundaries of data classes isolated?

5) What data rates or volumes can the system tolerate?

6) What effect will specific combinations of data have on system operation?

Equivalence Partitioning

Guidelines for equivalence partitioning -

1) If an input condition specifies a range, one valid and two invalid equivalence classes are defined.

2) If an input condition requires a specific value, one valid and two invalid equivalence classes are defined.

3) If an input condition specifies a member of a set, one valid and one invalid equivalence class are defined.

4) If an input condition is boolean, one valid and one invalid class are defined.

Example

Area Code	Blank or three-digit number
Prefix	Three-digit number, not beginning with 0 or 1
Suffix	Four-digit number
Password	Six-character alphanumeric
Commands	"Check", "deposit", "pay", etc.

Data Item	Input Condition	Remarks
Area Code	boolean	The area code may or may not be present
	range	Values between 200 and 999 with area codes requiring 0, 1 in second position
Prefix	range	Specified value > 200
Suffix	value	Four-digit length
Password	boolean	Password may or not be present
	value	Six-character string
Commands	set

Data Item	Input Condition	Equivalence Class
Area Code	boolean	1) no area code given 2) area code given
	range	1) valid - in range specified 2) invalid - greater 3) invalid - less than
Prefix	range	1) valid - between 200 and 999 (inclusive) 2) invalid - greater than 999 3) invalid - less than 200
Suffix	value	1) valid - a four-digit number 2) invalid - five-digit number 3) invalid - three-digit number
Password	boolean	1) no password given 2) password given
	value	1) valid - a six-character string 2) invalid - five-character string 3) invalid - seven-character string
Commands	set	1) valid - command in command set 2) invalid - command not in command set

Boundary Value Analysis

Guidelines for boundary value analysis -

1) If an input condition specifies a range bounded by values a and b, test cases should be designed with values a and b, and values just above and just below and b.

2) If an input condition specifies a number of values, test cases should be developed that exercise the minimum and maximum numbers. Values above and below the minimum and maximum are also tested.

3) Apply the above guidelines to output conditions. For example, if the requirement specifies the production of an table as output then you want to choose input conditions that produce the largest and smallest possible table.

4) For internal data structures be certain to design test cases to exercise the data structure at its boundary. For example, if the software includes the maintenance of a personnel list, then you should ensure the software is tested with conditions where the list size is 0, 1 and maximum (if constrained).

Example

Cause-Effect Graphs

Guidelines for cause-effect graphs -

1) Causes and effects are listed for a modules and an identifier is assigned to each.

2) A cause-effect graph is developed (special symbols are required).

3) The graph is converted to a decision table.

4) Decision table rules are converted to test cases.

Sample Symbols

Identity
Not
Or
And

Go To Lecture [Outline] [Overview]

Go To [Course Outline]