Once all of the test cases have been identified, they should be reviewed and validated to ensure accuracy and to identify redundant or missing test cases. Then, once they are approved, the final step is to assign actual data values. Without test data, test cases can't be implemented or executed; they are just descriptions of conditions, scenarios, and paths. Therefore, it is necessary to identify actual values to be used in implementing the final tests.
Table 3: Test Case Matrix for the Register for Courses Use Case
Section 3 Using state diagram to generate Test cases
State Chart Test Case Generation
This section discusses the application from UML state diagrams in UML class testing. A set of coverage criteria is proposed based on control and data flow in UML state diagrams and it is shown to how to generate test cases satisfying these criteria from UML state diagrams. First, control flow is identified by transforming UML state diagrams to extended finite state machines (EFSM). The hierarchical and concurrent structure of states is flattened and the broadcast communication are eliminated in the resulting EFSMs. Second, data flow is identified by transforming EFSMs into flow graphs to which conventional data flow analysis techniques can be applied.
UML state diagrams are widely used for specifying the dynamic behaviour of classes and are substantially based on state charts which have been successfully applied to reactive systems. UML state diagrams provide several concepts that distinguish themselves from conventional FSMs. These include the hierarchical and concurrent structure of states, the communication mechanism through events broadcasting, and the actions associated with states and transitions.
An integral part of class testing is the construction of test cases as sequence of messages from a given specification. Because there exists an infinite number of possible sequence of messages, exhaustive testing is impossible to achieve and it is necessary to have systematic coverage criteria which select a reasonable number of message sequences satisfying certain conditions.
The set of states in UML state diagrams represents both basic states and composite states which contain other states as substates. A composite state is classified as either an or-state or an and-state . An or-state has substates that are related to each other by an exclusive-or relation. In Figure 3, the state CVM consists of ‘Off’ and ‘On’ with ‘Off’ as the default state. Being in CVM implies being in ‘Off’ or in ‘On’, but not in both. An and-state has substates, called orthogonal components. Being in the and-state ‘On’ implies in Supply and Money simultaneously.
States can have actions associated with them. Actions are performed in response to events received while the object is in the state, without changing the state. Three reserved events are used in the action compartment : entry, exit and do. The entry (exit) event is used to specify actions performed at the entry (exit) of a state. The do event is used to specify an action performed while in a given state.
Figure 3: An example of UML state diagram
Transitions in UML state diagrams are represented by arrows between states and are labeled by event [guard] / action Λ send. We classify events into external events that are generated by the environment of an object, i.e. other objects, and internal events that are generated by the object itself. A special event, tm(interval), is used to represent the passage of a period of time. Guard is a Boolean expression that must be satisfied for the transition to occur. Action is a list of operations executed as a result of the transition being taken and is assumed to be atomic. Send is a list of events generated when the transition is fired.
An enabled transition is enabled by an event, and it originates from an active state. An enabled transition is triggered when there exists at least one full path from the source state to the target state.
Four kinds of event can be specified : call events, signal events, time events and change events. A call event represents the reception of a request to synchronously invoke a specific operation. A signal event represents the reception of a particular (synchronous) signal. A time event represents the passage of a designated period of time after a designated event (often the entry of the current state) or the occurrence of a given date and time. A change event models an event that occurs when an explicit Boolean expression becomes true as a result of a change in value of one or more attributes or associations.
The semantics of UML state diagrams is based on the notion of steps. External events generated by the environment of an object are accepted by an events queue. The semantics assumes that the events in the queue are processed in sequence at a time. Once an event is dispatched, one or multiple transitions may be enabled. The state machine selects and fires a maximal set of enabled transitions that are mutually non-conflicting. The basic transformation is called a step. Actions that result from taking a transition may cause events to be generated for this and other objects. Events for an object are broadcast within the present state diagram. After a previous step has completed, the next external event in the queue is dispatched to the state diagram
Generating Test Cases Based on Control Flow
To flatten the hierarchical and concurrent structure of states and eliminate the broadcast communication in UML state diagrams, EFSMs are used as intermediate forms in the transformation. AN EFSM is a tuple <GStates, C0, GTrans> such that
-
GStates is a set of global states
-
C0 ∈ GStates is the initial global state
-
GTrans is a set of global transitions
From a given UML state diagram, we identify an EFSM as follows. First, the set of global states GStates corresponds to the set of configurations in UML state diagrams. For example, there are five configurations in Figure 3 and these constitute the global states of the EFSM in Figure 4.
gta1 = (C3, λ, true, l=Λl, C3) gta2 = (C5, λ, true, l=Λl, C5)
Figure 4 : An example of extended FSMs
We say that a global transition represent the set of transition T in UML state diagrams. Intuitively, a global transition is a set of transitions that are executed by the occurrence of one event. For example, consider the event power-on and the configuration C1 = {CVM, Off} in Figure 3. We have a global transition gt1 = (C1, power-on, true, money = 0, C2) in Figure 4. For another example, consider the event power-off and the configuration C3. We have a global transition gt23 = (C3, power-off, true, light = off, C1) because t2 exists from the state Busy whose exist action is light = off.
For each global state C, we have a global transition gta = (C1, e, g, a, C2) such that
-
C1 = C2 = C
-
e = λ, where λ is the null event
-
g = true
-
a = ∪s∈C do(s) i.e., where a is the set of all do actions in C
The global transition gta means that the set of do actions in a global state C is performed repeatedly as long as the class is in the global state. For example, we have two global transitions gta1 and gta2 for the global states C3 and C5, because C3 and C5 include the state Busy whose do action is “light = Λ light”.
Remark 1. (Enabledness) When simulating or generating reachability graphs using UML state diagrams, the enabledness should be determined by the current values of variables as well as the current configuration. However, we do not consider the values of the variables because our work centres on identifying possible control and data flow.
Remark 2. (Ignoring broadcasting) When defining global transitions, we eliminate the broadcasting through internal events by treating external and internal events in the same way. This elimination is conservative in the sense that the transformed EFSMs include all the possible execution sequences in UML state diagrams.
Control flow in UML state diagrams is identified in terms of the paths in EFSMs. Let <GStates, C0, GTrans> be an EFSM, and let gsi ∈ GStates and gti = (Ci, ei, gi, ai, C’i) ∈ GTrans for 0 ≤ i ≤ n. A sequence (gs0, gt0), (gs1, gt1), …, (gsn, gtn) is called a path if gs0 = C0 and gsi = Ci and gsi+1 = C’i, for 0 ≤ i ≤ n – 1. In general, there are infinitely many paths in EFSMs and hence it is impossible to cover all these paths. We explore the following coverage criteria out of a potentially infinite family of criteria. Let P be a set of paths.
-
P satisfies path coverage if P contains all possible paths through EFSMs. This is the strongest criterion and generally impossible to achieve.
-
P satisfies state coverage (global state coverage) if P includes every s ∈ States (gs ∈ GStates).
-
P satisfies transition coverage (global transition coverage) if P includes every t ∈ Trans (gt ∈ GStates).
Test cases satisfying these criteria can be constructed in terms of simple breadth or depth first searches over EFSMs. The following shows a test cases generation method for global state coverage by traversing EFSMs in the breadth first order. WE can similarly define the methods for generating test cases satisfying other coverage criteria. Let <GStates, C0, GTrans> be an EFSM.
make-state-coverage-testing-tree (TT)
begin
if all gs ∈ GStates are visited then return;
for each gt = (Ci, ei, gi, ai, C’i) ∈ GTrans do begin
if Ci = TT and C’i is not visited then
make C’i as a child of TT;
end
for each child cTT of TT do
make_state_coverage_testing_tree (cTT)
end
For example, figure 5 shows the testing tree constructed by the above algorithm in which the set of paths {p1, p2} such that p1 = (C1, gt1), (C2, gt31) and p2 = (C1, gt1), (C2, gt51), (C4, gt32) satisfies global state coverage. The path p1 in the EFSM correspond to the message sequence of (power-on, coffee) in the UML state digram of Figure 3 and the path p2 corresponds to the message (more precisely, an object instantiated from the class), we can traverse all the global states of the coffee vending machine.
In general, certain parts in EFSMs may be unexecutable or infeasible. For example, the path p1 = (C1, gt1), (C2, gt31) is infeasible because the value of money is set as 0 by gt1 and thus the condition money > 0 of gt31 cannot be satisfied. Of course, the selection of feasible paths is undecidable, thus making the application of these criteria undecidable.
Figure 5 : An example of testing trees
Generating Test Cases Based on Data Flow
Data flow in conventional programs is based on the notions of definitions and uses of variables in the statements. In UML state diagrams, variables can be defined and used in the actions of states and in transitions.
-
A variable x is defined (used) in an action a of a state if a assigns a value to x (references x)
-
A variable x is defined (used) in a transition t if action(t) assigns a value to x (guard(t) or action(t) references x)
Table 4 below shows the definitions and uses of the variables money and light in Figure 3:
Table 4
Let <GStates, C0, GTrans> be an EFSM. Let gt = (C, e, g, a, C’) ∈ GTrans such that represents a set of Transitions T in UML state diagrams. Recall that g is the conjunction of the guards of the transitions in T and a is the union of the actions of the transitions in T and the entry and exit actions that are executed by T.
-
We say that a variable x is defined in gt ∈ GTrans if x is defined in at least one of the actions in a.
-
We say that a variable x is used in gt ∈ GTrans if x is used in at least one of the guards in g or the actions a.
In addition to variables, UML state diagrams introduce a new type of data flow, which we call “data flow through states”. In UML state diagrams, both states and variables can affect the occurrence of transitions in the same way. That is, the pre-condition and post-condition of a transition are defined in terms of its source and target states as well as the value of variables. Precisely, data flow through states is defined as follows :
-
A state s is defined in a transition t if s is a descendent of scope (t), i.e., s ∈ p*(scope(t)).
-
A state s is used in a transition t if s is an ancestor of the source state of t, i.e., source(t) ∈ p*(scope(s)).
Table 5 shows the definitions and uses of the states in Figure 3. Consider the transition t3 in Figure 3. Since scope(t3) = Supply, we say that Supply, Idle, and Busy are defined by t3. Intuitively, Supply and its children can be changed by t3 while the states outside Supply are not changed by t3. We say that Idle, Supply, On, and CVM are used by t3, because source(t3) = Idle. Intuitively, the source state and all its ancestors are the pre-condition of transitions, i.e., they should be included in the current configuration for the transition to occur.
Table 5
Let <GStates, C0, GTrans> be an EFSM. Let gt be a global transition representing a set of transitions T in UML state diagrams.
-
We say that a state s is defined in gt ∈ GTTrans if x is defined in at least one transition t ∈T.
-
We say that a state s is defined in gt ∈ GTTrans if x is used in at least one transition t ∈T.
Now we can transform EFSMs into flow graphs. A flow graph is a tuple <V, N, E, def, use> such that
-
V is a set of variables and states in UML state diagrams
-
N = Ns ∪ Nt is a set of nodes. Each node in Ns is called a s-node and each node in Nt is called a t-node.
-
E = Est ∪ Ets is a set of edges such the Est = {(s,t) ⎪s ∈ Ns, t ∈ Nt} and Ets = {(t,s) ⎪t ∈ Nt, s ∈ Ns
-
def : N → 2V identifies the variables and states defined for each node.
-
use : N → 2V identifies the variables and states used for each node.
From a given EFSM <GStates, C0, GTrans>, we identify the flow graphs as follows :
- Ns = GStates
- Nt = GTrans
-
For each global transition gt = (Ci, ei, gi, ai, C’i) ∈ GTrans, we have two edges e1 and e2 such that e1 = (Ci, gti) ∈ Est and e2 = (gti, C’i) ∈ Ets
-
For s ∈ Ns, def(s) = 0
-
For t ∈ Nt, def(t) is the set of all the variables and states defined in t.
-
For s ∈ Ns, use(s) = 0
-
For t ∈ Nt, use(t) is the set of all the variables and states defined in t.
Figure 6 shows the part of the flow graphs which is identified by considering C2, C3, gt31 and gta1 in Figure 4.
Figure 6 : A part of flow graphs
Now we can readily generate test cases based on data flow from UML state diagrams, by apply conventional data flow analysis techniques to the resulting flow graphs. We can also reuse a number of coverage criteria that have been extensively studied and compared in the testing literature such as all-definition, all-use, and all def-use paths coverage. By applying existing data flow techniques and coverage criteria, we can generate test cases as a set of paths that cover the associations between the definitions and uses of each variable and state in UML state diagram.
For example, in Figure 6, we can identify two def-use associations of light : (gt32, gta2) and (gta2, gta2). That is, the definitions in gt32 and gta2 can reach to the use in gta2. A path (C1, gt1), (C2, gt51), (C5, gta2), (C5, gta2), which corresponds to the sequence of (power-on, inc, coffee, a2, a2) in Figure3, can be used as the test case that covers these associations. Table 6 shows all the def-use associations of money such as (gt1, gt31) that occurs because of the hierarchical structure of states and (gt1, gt31) that occurs because of the hierarchical structure of states (gt51, gt32) that occurs because of the concurrent structure of states in UML state diagrams.
Table 6 : Def-use associations the variables in Figure 3
Section 4 Using collaboration diagram to generate Test cases
UML collaboration diagrams represent a significant opportunity for testing because they precisely describe how the functions the software provides are connected in a form that can be easily manipulated by automated means. The most novel aspect of this is that tests can be generated automatically from the software design, rather than the code or the specifications. Criteria are defined for both static and dynamic testing of specification-level and instance-level collaboration diagrams.
These criteria allow a formal integration tests to be based on high level design notations.
Previous work has focused on generating tests from specifications; the current work focuses on design descriptions.
Generating test data from high level design notations has several advantages over code-based generation:
Design notations can be used as a basis for output checking, significantly reducing one of the major costs of testing.
The process of generating tests from design will often help the test engineer discover problems with the design itself. If this step is done early, the problems can be eliminated early, saving time and resources.
Generating tests during design also allows testing activities to be shifted to an earlier part of the development process, allowing for more effective planning and utilization of resources. Another advantage is that the test data is independent of any particular implementation of the design.
This paper presents a model for performing static analysis and generating test inputs from Unified Modeling Language (UML) collaboration diagram specifications.
The state-chart models the reactions of the class to events received from the environment. Our current research is focused on analyzing the behavior of a set of interacting objects.
Collaboration diagrams provide the following six pieces of information:
1. The objects that are involved in an interaction and the structure of these objects.
2. Instances of allowable sequences of operation calls to an object.
3. The semantics of an operation.
4. The operations that is imported from other classes, thus enabling collaboration with objects of the other class.
5. The communication pattern of objects in collaboration (synchronous or asynchronous).
6. The execution characteristics of objects (parallel or sequential).
In particular, it is suitable to consider collaboration diagrams for integration testing since collaboration diagrams specify the interactions among a set of objects.
Software Testing
Testing and test design, as parts of quality assurance, should also focus on fault prevention. There are various ways to classify adequacy criteria. One of the most common is by the source of information used to specify testing requirements and in the measurement of test adequacy. Hence, an adequacy criterion can be specification-based, design-based, or program-based.
Collaboration Diagrams
A collaboration diagram is a graphical representation of collaboration. The objects in a collaboration diagram are instances of classes in a class diagram.
Without the interaction part, a collaboration diagram is similar to a class diagram. However, they are not the same. For collaboration, there need not be an object of every class, because some classes will be irrelevant to the particular collaboration being considered. There may be two or more different objects of the same class.
A collaboration diagram has two forms, which are specification level collaboration diagram and instance level collaboration diagram
A realization is a relationship between a specification and its implementation. When collaboration diagrams are used to describe the realization of use cases, they describe only the externally visible actions and their sequences.
Since collaboration diagrams include both the messages that are passed between objects and their sequences, collaboration diagrams provide both design level data flow and design level control flow information. Traditionally, data flow and control flow information are obtained from the source code. Data flow and control flow information have had significant impact on testing. Hence, using collaboration diagrams in testing helps us in many ways. Obvious benefits are generating test data using existing data flow and control flow-testing techniques before the code generation, static checking of specification itself, and static checking of code.
Testing Criteria
Collaboration diagrams describe the structure and behavior of the system. A UML collaboration specifies what requirements must be fulfilled by the objects in a system, and what communications must take place between the objects for a specific task to be performed.
Static Checking
Testing can be either static or dynamic. Most discussions of testing focus on dynamic testing, wherein the software is executed on some inputs. Static testing refers to checking some aspects of the software without execution, usually by evaluating the source code.
We have identified four items that should be used to statically check the code. They are described as follows:
1. ClassifierRoles: In a collaboration, if two ClassifierRoles originate from the same base class, then they should be distinct in terms of their requested operations and values of attributes. Since they originate from the same class, it is possible that they might be mistaken for each other. For this reason, ClassifierRoles that originate from the same class should be tested to see if they have all the required attributes and operations.
2. Collaborating Pairs: The links on a collaboration diagram depict the structural constraints of collaborating pairs and their communication messages. The association tells if it is a one-to-one, one-to-many, or many-to-many relationship. The relationship reflects a constraint in the requirement specification. Hence, testing the structural relationship between objects can serve as verification of requirements. Each collaborating pair on the collaboration diagram should be checked or tested at least once.
3. Message or stimulus: Messages provide information on:
- return value type
- thread of control
- operation or method name to be invoked on the target object
- parameters to the invoked operation or method
Testing of a message itself may reveal most integration problems. A stimulus is an instance of a message. A stimulus could be a signal, a method or operation call, or an event that causes an object to be created or destroyed. Besides carrying a specific data, messages have a direction. That is, a stimulus originates from a source object (sender) and resides on a target object (receiver). We can see that stimulus provides early information for integration testing, thus each stimulus should be used as a basis for generating test inputs.
4. Local Variable Definition-Usage Link Pairs: Checking variable definition-usage link pairs let’s us find data flow anomalies at the design level. The following link pairs should all be checked for data flow anomalies:
- Global Variable Definition-Use Pairs
- Object Creation-Use Pairs
- Object Use-Destruction Pairs
- Object Creation-Destruction Pairs
Dynamic Testing
Collaboration diagrams provide a complete path for a use case or the realization of an operation. Assume that there is one collaboration diagram per operation, and the implementation of an operation conforms to the collaboration.
Test criterion: For each collaboration diagram in the specification, there must be at least one test case t such that when the software is executed using the software that implements the message sequence path of the collaboration diagram must be executed.
Criteria of this type are normally used in one of two ways. The criterion can be used as a guide for generating tests, or the criterion can be used as a metric for measuring externally created tests.
Figure 7 Message Sequence Path
Figure 8
The Instrumentation Algorithm
Figure above gives an algorithm for instrumentation. The algorithm attempts to achieve the following goals:
1. insert instruments for each link
2. help the tester to keep track of the run-time interaction traces
Section 5 Conclusions and Future Work
The described procedures resemble a baseline approach to the goal of automated generation of test cases based on UML models. It increases the efficiency with which test cases are generated and supports iterative development processes by reducing testing efforts. It encourages developers to develop more detailed models by offering (potential) tool support in both modeling and model-based generation.
An additional benefit of applying the approach aside from obtaining test cases systematically is its rather formal and easily comprehensible documentation of the software’s usage and interactive behavior. This fact can be considered a slight reduction of the overhead testing imposes on the software design process.
In current practice, Use Case is associated with the front end of the software development lifecycle and test cases are typically associated with the latter part of the lifecycle. By leveraging use cases to generate test cases, however, testing teams can get started much earlier in the lifecycle, allowing them to identify and repair defects that would be very costly to fix later, ship on time, and ensure that the system will work reliably.
The resulting set of test cases generated from State Chart provides the capability of checking that classes are correctly implemented against specifications written in UML state diagrams by testing whether class implementations establish the desired control and data flow specified in the specifications. It focuses on unit testing of classes and do not consider inter-relationships between classes, and it is possible to extend the works to support object-oriented integration testing using state diagrams.
The State Chart test set have better capability of revealing unit level faults than the Collaboration Diagram test set, and the collaboration diagram test set have better capability of revealing integration level faults than the State Chart test set. The experimental data also shows that the State Chart are resulted in more test cases than the collaboration diagram.
In conclusion, UML provides three diagrams to specify communication between classes: sequence, collaboration and activity diagrams. Testing techniques using these diagrams should be developed to complete the testing of dynamic behavior specified in UML. It is to be further developed to the testing of generalization and specialization of classes through inheritance.
Reference:
LIGHTVIEWS-visual interactive Internet environment for learning OO software testing
Automated generation of statistical test cases from UML state diagrams
Analysis and Testing of Web Applications
Structural testing of Web Applications
Generating Test Cases from UML
Automatic Test Case Generation f rom UML State charts
Using a model-based test generator to test for standard conformance
Automated Test Case Generation from Dynamic Models
Use Cases and Test Cases: Jumpstart Software Testing
UML-Based Statistical Test Case Generation
B. Beizer. Software Testing Techniques. Van Nostrand Reinhold, Inc, New York NY, 2nd edition, 1990. ISBN 0-442-20672-0.
Rational Software Corporation. Rational Rose 98: Using Rational Rose. Rational Rose Corporation, Cupertino CA, 1998.
Gregor Engels, L. P. J. Groenewegen, and G. Kappal. Object-oriented specification of coordinated collaboration. In Proceedings of the IFIP World Conference on IT Tools, pages 437{449, Canberra, Australia, September 1996.
Gregor Engels, Roland Hucking, Stefan Sauer, and Annika Wagner. Uml collaboration diagrams and their transformation to java. In Proceedings of the Second IEEE International Conference on the Unified Modeling Language (UML99), pages 473{488, Fort Collins, CO, October 1999. IEEE Computer Society Press.
P. G. Frankl and E. J. Weyuker. An applicable family of data flow testing criteria. IEEE Transactions on Software Engineering, 14(10):1483{1498, October 1988.
Object Management Group. OMG UML Specification Version 1.3, June 1999.
http://www.omg.org/uml/.
NV, October 1999. IEEE Computer Society Press.
Gunnar Overgaard. A Formal Approach to Collaborations in the Unified Modeling Language. In Proceedings of the Second IEEE International Conference on the Unified Modeling Language (UML99), pages 99{115, Fort Collins, CO, October 1999. IEEE Computer Society Press.
Hong Zhu, Patrick A. V. Hall, and John H. R. May. Software unit test coverage and adequacy. ACM Computing Surveys, 29(4):366{427, December 1997.
The UML Collaboration, a Standard for Role Modeling
A Study on the Effectiveness of Test Cases Generated from UML Diagram
UML-Based Integration Testing
Software Development Process Using UML
Generating Tests from UML Specifications
Using UML Collaboration Diagrams for Static Checking and Test
UML Tutorial: Collaboration Diagrams
