Abstract
Usage-based testing focuses quality assurance on highly used parts of the software. The basis for this are usage profiles based on which test cases are generated. There are two fundamental approaches in usage-based testing for deriving usage profiles: either the system under test (SUT) is observed during its operation and from the obtained usage data a usage profile is automatically inferred, or a usage profile is modeled by hand within a model-based testing (MBT) approach. In this article, we propose a third and combined approach, where we automatically infer a usage profile and create a test data repository from usage data. Then, we create representations of the generated tests and test data in the test model from an MBT approach. The test model enables us to generate executable Testing and Test Control Notation version 3 (TTCN-3) and thereby allows us to automate the test execution. Together with industrial partners, we adopted this approach in two pilot studies. Our findings show that usagebased testing can be applied in practice and greatly helps with the automation of tests. Moreover, we found that even if usage-based testing is not of interest, the incorporation of usage data can ease the application of MBT.
1 Introduction
One of the major challenges in quality assurance is the efficient spending of available resources to get the largest benefit out of them. One strategy is to put the user-experienced quality at center stage through usage-based testing [26]. The idea is to focus the quality assurance on the highly used parts of a system under test (SUT), and, thereby, assure that the core functionalities often required by the users are well tested. The core of usage-based testing are usage profiles. A usage profile is a stochastic description of the utilization of a SUT. The usage profile obtains information on which events are triggered by the users. Within this article, we consider the testing of service oriented architectures (SOAs). Hence, the events which model the usage of the SUT are the requests and responses exchanged between services within a service orchestration.
7 Conclusion
Within this article, we discussed the application of usagebased testing in practice. To this aim, we proposed an approach for the application of usage-based testing to SOA. The first part of our approach is a usage monitor for SOA that is non-intrusive and easy to integrate. Then, we use the obtained usage data from the monitor to train a usage profile from which we can generate test sequences. Moreover, we create a repository with test data from the usage data. We combine the usage-based testing with an MBT approach based on the MIDAS DSL, a UML and UTP-based language for the definition of tests. Through this combination, we were able to automatically generate executable tests. We evaluated the approach in two pilot studies, in which we gained valuable insights into the practical feasibility of usage-based testing.
