This paper describes methods for specification-based testing and testing measurement for real-time reactive systems. One of the goals of this work is to automate these methods and integrate them into TROMLAB, a rigorous framework being prototyped for the development of real-time reactive systems.
In the context of real-time creactive systems, which are mostly safety-critical, the reliability of the testing process would indicate the extent of dependability and correctness achieved in the implementation of safety and time-dependent behavior. To achieve a high level of reliability, testing and measurement must be based on rigorous methods. Ourapproach is anchored on the formal object-oriented design specification method TROM (Timed Object Reactive Model) [1]. In TROM formalism reactive objects are described as labeled transition systems augmented with ports, attributes, logical assertions on the attributes, and time constraints. TROMLAB [2] is a framework for real-time reactive systems development built on TROM formalism. The framework includes a number of tools to promote a rigorous development of real-time reactive systems. The methods discussed in this paper lead to the development of the two tools: (1) a tool to automatically generate test cases that exercise timing behavior, and (2) a tool for testing measurement when a degree of adequacy is associated with each test set.
We define two test adequacy criteria - state cover and transition cover. These criteria ensure that test cases that satisfy the criteria will test every state and every transition in a TROM. With a dense notion of time, the labeled transition system TROM that models the behavior of a reactive object will have infinite number of states. Corresponding to a TROM we generate a grid automaton that has only finite number of states and is equivalent in behavior to the TROM. We discuss an algorithm to generate the grid automaton from a TROM, and an algorithm to generate test cases from a grid automaton such that the two test adequacy criteria are satisfied.
The testing measurement in TROMLAB environment provides measures of test suite adequacy in terms of coverage. We use metric-based coverage measures to evaluate how well the test suite approximates its target. For this purpose, the test case domain is abstracted as a metric space. The proximity measure is the distance metric expressed in terms of elementary transformations minimally needed to transform an abstraction of one test case into another. The distance metric satisfies the axioms of measurement theory, thus guaranteeing the theoretical validity of the testing distance measure. Test case selection algorithm that can identify the optimal set of test cases (from the automatically generated test cases domain), based on test cases proximity measure, is proposed.
We illustrate our testing techniques and testing measurement on a Train-Gate-Controller example, a bench-mark case study in real-time research community. The paper will present a survey of the related work, and include all the results that we have obtained on the above two issues.
1. R. Achuthan, A Formal Model for Object-Oriented Development of Real-Time Reactive Systems. Ph.D. Thesis, Concordia University, Montreal, Canada, October 1995.
2. V.S. Alagar, R. Achuthan, D. Muthiayen, TROMLAB: A Software Development Environment for Real-Time Reactive Systems, Technical Report, (first version 1996, revised 1999), Concordia University, Montreal, Canada.'