New Journal Article on Test-Case Generation and Fault Propagation

Our new article on fault propagation and its relation to model-based test-case generation was just published:

Bernhard K. Aichernig, Elisabeth Jöbstl, and Martin Tappler. Does this fault lead to failure? Combining refinement and input-output conformance checking in fault-oriented test-case generation. Journal of Logical and Algebraic Methods in Programming, 85(5, Part 2):806–823, 2016. (PDF) (doi:10.1016/j.jlamp.2016.02.002)

The publisher Elsevier provides a free download of the original article until December 2.

The paper describes how we generate test-cases for reactive systems by

1. injecting faults into the models (mutation testing)
2. generating a test-case that triggers this fault (refinement checking)
3. extending the test-case until an observational failure could be produced (ioco checking).

We show that this is more efficient than previous approaches that search for observational equivalences directly, i.e. skipping Step 2.

Our case study of a car-alarm controller shows that for some subtle faults we need up to nine additional interactions until a fault becomes visible at the interface.

New journal article on model-based testing

Today, our new journal article appeared in the Science of Computer Programming journal of Elsevier.

It deals with the problem of automatically generating test-cases from mutated models:

Bernhard K. Aichernig, Elisabeth Jöbstl, and Stefan Tiran. Model-based mutation testing via symbolic refinement checking. Science of Computer Programming. 97 (2015): 383-404, Elsevier, 2015.

Here is a link to a free copy of it that works for 50 days.

Highlights:

• We deal with model- and mutation-based test case generation.
• The main focus lies on optimizations of the underlying conformance check.
• We explain the construction of test cases based on the conformance check.
• We allow for non-determinism in the test models.
• We demonstrate the effectiveness of our optimizations on industrial case studies.

Enjoy!

Habilitation

Yesterday, I have completed my habilitation process at TU Graz. The last step was a public 45 minutes talk on

"Model-Based Mutation Testing: From Semantics to Automated Test-Case Generation"

given to the faculty of computer science and the habilitation commission. After the talk I answered questions and defended my habilitation thesis.

The habilitation thesis with the title "Model-Based Mutation Testing: Theory and Application" was submitted in January. It consists of selected papers and a survey chapter.

The survey chapter is available for download: Survey Chapter Habilitationsschrift Aichernig.

The habilitation gives me the right to lecture any topic covered by my habilitation area "Practical Computer Science and Formal Methods in Computer Science". "Practical Computer Science" includes topics like Software Engineering and Programming Languages. "Formal Methods" are mathematically based techniques for the specification, development and verification of software and hardware systems.

More details regarding the history and concept of habilitation can be found on Wikipedia.

Integrating model-based testing and analysis tools via test case exchange

Our first publication in the MBAT project was presented at TASE 2012 and in a seminar given at UNU-IIST:

Bernhard K. Aichernig, Florian Lorber, and Stefan Tiran. Integrating model-based testing and analysis tools via test case exchange. In TASE 2012, 6th IEEE International Symposium on Theoretical Aspects of Software Engineering, July 4–6, Beijing, China, pages 119-126. IEEE Computer Society, 2012. (PDF)

In this paper we show an integration of two tools via test cases. For example test cases are generated from a model in Tool 1 and then model checked in another Tool 2. This is useful when a modeling tool does not support model checking as it is the case with most UML tools.

Abstract: Europe’s industry in embedded system design is currently aiming for a better integration of tools that support their development, validation and verification processes. The idea is to combine model-driven development with model-based testing and model-based analysis. The interoperability of tools shall be achieved with the help of meta-models that facilitate the mapping between different modelling notations. However, the syntactic and semantic integration of tools is a complex and costly task. A common problem is that different tools support different subsets of a language. Furthermore, semantic differences are a major obstacle to sound integration efforts.

 In this paper we advocate an alternative, more pragmatic approach. We propose the exchange of test cases generated from the models instead of exchanging the models themselves. The advantage is that test cases have a much simpler syntax and semantics, and hence, the mapping between different tools is easier to implement and to maintain. With a formal testing approach with adequate testing criteria a set of test cases can be viewed as partial models that can be formally analysed. We demonstrate an integration of our test case generator Ulysses with the CADP toolbox by means of test case exchange. We generate test cases in Ulysses and verify properties in CADP. We also generate test cases in CADP and perform a mutation analysis in Ulysses.

Towards Symbolic Model-Based Mutation Testing: Combining Reachability and Refinement Checking

We present our newest paper on model-based mutation testing accepted at MBT 2012.

Model-based mutation testing uses altered test models to derive test cases that are able to reveal whether a modelled fault has been implemented. This requires conformance checking between the original and the mutated model. This paper presents an approach for symbolic conformance checking of action systems, which are well-suited to specify reactive systems.

We also consider non-determinism in our models. Hence, we do not check for equivalence, but for refinement. We encode the transition relation as well as the conformance relation as a constraint satisfaction problem and use a constraint solver in our reachability and refinement checking algorithms.

Explicit conformance checking techniques often face state space explosion. First experimental evaluations show that our approach has potential to outperform explicit conformance checkers.

This research is part of the TRUFAL project.

Download PDF

Protocol conformance testing a SIP registrar

Photo: Tiptel

In recent years we have successfully applied model-based mutation testing to several implementations of communication protocols. In our work reported in

Bernhard K. Aichernig, Bernhard Peischl, Martin Weiglhofer, and Franz Wotawa. Protocol conformance testing a SIP registrar: an industrial application of formal methods. In Mike Hinchey and Tiziana Margaria, editors, Fifth IEEE International Conference on Software Engineering and Formal Methods (SEFM 2007), 10-14 September 2007, London, England, UK, pages 215–226. IEEE Computer Society, 2007. (PDF) (doi:10.1109/SEFM.2007.31)

we tested parts of the Session Initiation Protocol (SIP) used in internet telephony in order to register and connect communication partners.

Our case study targeted a commercial and an open-source (OpenSER) SIP registrar. Furthermore, we compared the mutation approach to a standard model-based test case generation technique. Our mutation technique generated 124 test cases, compared to 6000 test cases generated by the standard approach.

The 6000 tests discovered nine different faults in the commercial SIP registrar and four in the open-source registrar OpenSER.

Interesting was the fact that the comparatively small set of 124 mutation tests uncovered one additional fault in the commercial implementation that was not detected by the 6000 tests of the classical approach.

This is a further indication that mutation testing could play an important role in a mature testing process.

Another important contribution of this paper was to show how our mutation technique could be scaled to larger models. We sliced the models with the help of markers around the injected faults.

A new EU project started: MBAT

On November 1st a new EU project started: MBAT - Combined Model-based Analysis & Testing of Embedded Systems.

MBAT is a three-years ARTEMIS joint undertaking project with 41 partners. I am the key researcher and project manager for TU Graz. MBAT combines the latest model-based testing technologies with static analysis techniques leading to a new model-based validation & verification method in the transportation domain (aerospace, automotive, railways).

My team at TU Graz brings in our expertise and tools in model-based mutation testing from the MOGENTES and TRUFAL projects. One aim is to combine model-based mutation testing with structural test case generation approaches. We will also investigate the relation of fault-models to the step-wise refinement in model-driven development.

In model-based mutation testing we inject faults in models of the system under test and automatically generate test sequences that will detect these faults in an implementation. Hence, our tests prevent that the faulty model has been implemented. For details see, e.g., our paper "Efficient mutation killers in action" that can be found in the list of our conference papers. This work describes the case-study of testing a car-alarm system.

In MBAT we will also investigate the current testing practice of our industrial partners and develop a process model where the different testing techniques of MBAT best fit. This will facilitate the adoption of the tools and techniques in industry.

We closely collaborate with the following Austrian project partners: AIT, AVL, and VIF.

TAP 2011, Zurich

Photo: Daniel Gasienica

I have been invited to serve on the Program Committee of TAP 2011, the 5th International Conference on Tests & Proofs, to be held in Zurich, Switzerland, June 30 - July 1, 2011.

The TAP conference is devoted to the convergence of proofs and tests. It combines ideas from both sides for the advancement of software quality.

We had a paper at TAP 2009 on concolic execution of distributed systems (see my conference papers).

To prove the correctness of a program is to demonstrate, through impeccable mathematical techniques, that it has no bugs; to test a program is to run it with the expectation of discovering bugs. The two techniques seem contradictory: if you have proved your program, it's fruitless to comb it for bugs; and if you are testing it, that is surely a sign that you have given up on any hope to prove its correctness.

Accordingly, proofs and tests have, since the onset of software engineering research, been pursued by distinct communities using rather different techniques and tools.

And yet the development of both approaches leads to the discovery of common issues and to the realization that each may need the other. The emergence of model checking has been one of the first signs that contradiction may yield to complementarity, but in the past few years an increasing number of research efforts have encountered the need for combining proofs and tests, dropping earlier dogmatic views of incompatibility and taking instead the best of what each of these software engineering domains has to offer.

The conference will include a mix of invited and submitted presentation, and a generous allocation of panels and informal discussions. All papers will be published in Springer's LNCS series.

SCENARIOS 2011, Berlin

Photo: Ishtar Gate, Pergamon Museum, Berlin 2010.

I have been invited to serve on the Program Committee of SCENARIOS 2011, the 1st international workshop on Scenario-Based Testing . SCENARIOS 2011 is a satellite worksop of ICST 2011 to be held in Berlin, Germany, March 2011.

Nowadays, more and more techniques rely on human intervention so as to ensure the applicability of testing approaches. Indeed, a trade-off has to be found between the relevance of the test cases (requiring strong human intervention) and automation. Scenario-Based Testing addresses this issue by looking for more automation, with human intervention restricted to insightful activities. Over the years, different scenario based testing techniques have been developed to assist the validation engineer, using textual scenarios such as regular expressions, graphical notations such as UML interaction diagrams or test purposes specified as transitions systems. All these techniques represent an efficient solution to the state space explosion problem by restricting the possible executions of the system to a limited, and hopefully more accurate, subset of admissible traces. In addition, they make it possible to capture the human expertise for testing specific situations that can not be targeted by a systematic model coverage approach.

This first edition of the workshop on Scenario-Based Testing aims at gathering ideas and techniques in the area of semi-automated testing. It is intended to assess the effectiveness of available techniques and promote original research ideas that can be concretized into an industrial context.

MBT 2011, April 2-3, 2011, Saarbruecken, Germany

Photo: Wolfgang Staudt

I have been invited to serve on the Program Committee of MBT 2011, the 7th Workshop on Model-Based Testing. MBT 2011 is a satellite worksop of ETAPS 2011 to be held in Saarbrücken, Germany.

The workshop is devoted to model-based testing of both software and hardware. Model-based testing is closely related to model-based specification. Models are used to describe the behavior of the system under consideration and to guide such efforts as test selection and test results evaluation. Both testing and verification are used to validate models against the requirements and check that the implementation conforms to the specification model.

Model-based testing has gained attention with the popularization of models in software/hardware design and development. Of particular importance are formal models with precise semantics, such as state-based formalisms. Testing with such models allows one to measure the degree of the product's conformance with the model.

Techniques to support model-based testing are drawn from diverse areas, like formal verification, model checking, control and data flow analysis, grammar analysis, and Markov decision processes.

The intent of this workshop is to bring together researchers and users of models for to discuss the state of the art in theory, applications, tools, and industrialization of model-based specification, testing and verification.

Original submissions are solicited from industry and academia. They are invited to present their work, plans, and views related to model-based testing. The topics of interest include but are not limited to:

• Online and offline test sequence generation methods and tools
• Test data selection methods and tools
• Runtime verification
• Model-based test coverage metrics
• Automatic domain/partition analysis
• Combination of verification and testing
• Models as test oracles
• Scenario based test generation
• Meta programming support for testing
• Formalisms suitable for model-based testing
• Application of model checking techniques in model-based testing
• Game-theoretic approaches to testing
• Model-based testing in industry: problems and achievements

ICFEM 2010, Nov 16-19, Shangai

Photo: Wolfgang Staudt

Another Shanghai conference I have been invited to serve on the Program Committee: ICFEM 2010, the 12th International Conference on Formal Engineering Methods. ICFEM 2010 will take place in Shanghai, right after UTP 2010. Submission deadline for abstracts is 28th of May 2010. The conference chair is my former colleague at UNU-IIST He Jifeng (East China Normal University, China), PC-cochairs are Jin Song Dong (National University of Singapore, Singapore) and Huibiao Zhu (East China Normal University, China).

ICFEM brings together those interested in the application of formal engineering methods to computer systems. Researchers and practitioners, from industry, academia, and government, are encouraged to attend, and to help advance the state of the art. We are interested in work that has been incorporated into real production systems, and in theoretical work that promises to bring practical, tangible benefit.

Submissions related to the following principal themes are encouraged, but any topics relevant to the field of formal methods and their support environments will also be considered:

• Formal model-based development and code generation
• Abstraction and refinement
• Formal specification and modelling
• Software verification
• Formal approaches to software testing
• Software model checking
• Formal methods for object and component systems
• Analysis and models for concurrency
• Formal methods for cloud computing
• Tool development and integration
• Software safety, security and reliability
• Experiments involving verified systems
• Applications of formal methods

UTP 2010, Nov 15-16, Shanghai

Photo: stevechasmar

I have been invited to serve on the Program Committee of UTP 2010, the 3rd International Symposium on Unifying Theories of Programming. UTP 2010 will take place in Shanghai. Submission deadline for abstracts is 4th of June 2010. The PC-chair is Shengchao Qin, University of Durham, UK.

Based on the pioneering work on Unifying Theories of Programming by Tony Hoare, He Jifeng, and others, the aims of the UTP Symposium series are to continue to reaffirm the significance of the ongoing UTP project, to encourage efforts to advance it by providing a focus for the sharing of results by those already actively contributing, and to raise awareness of the benefits of such unifying theoretical frameworks among the wider computer science and software engineering communities.

Of particular interest is how unification may be used to meet the goals and difficulties to be encountered in the Grand Challenges of Computing, with particular reference to the UK's "GC6: Dependable Systems Evolution", its international cousin the "Verified Software Initiative", and their plan to develop a Verified Software Repository. To this end the UTP2010 Symposium welcomes contributions on the above themes as well as others which can be related to them. Such additional themes include, but are not limited to, relational semantics, relational algebra, healthiness conditions, normal forms, linkage of theories, algebraic descriptions, incorporation of probabilistic programming, timed calculi and object-based descriptions, as well as alternative programming paradigms such as functional, logical, data-flow, and beyond. In all cases, the UTP approach should be compared and advantages/disadvantages discussed.

SEFM 2010, 13-17 September, Pisa, Italy

Photo: Pisa 2007

I have been invited to serve on the Program Committee of SEFM 2010, the 8th IEEE International Conference on Software Engineering and Formal Methods. SEFM 2010 will take place in Pisa. Submission deadline for abstracts is 22 March 2010. The PC-chairs are Jose Luis Fiadeiro and Stefania Gnesi.

The aim of the conference is to bring together practitioners and researchers from academia, industry and government to advance the state of the art in formal methods, to facilitate their uptake in the software industry and to encourage their integration with practical engineering methods. Papers that combine formal methods and software engineering are especially welcome.

I served as a PC-Cochair for SEFM 2005. For information on the past SEFM conferences visit the general SEFM webpage.

Here the call for papers will be published as soon as it is out.

Fault-based Test Case Generation for Component Connectors

Photo: Matthew Stinson

Sun Meng at CWI had the idea to translate my UTP-theory of mutation testing to REO connectors. This resulted in a TASE 2009 paper published in July:

Bernhard K. Aichernig, Farhad Arbab, Lacramioara Astefanoaei, Frank S. de Boer, Sun Meng, and Jan Rutten. Fault-based test case generation for component connectors. In TASE 2009, Third IEEE International Symposium on Theoretical Aspects of Software Engineering, Tianjin, China, July 29–31, pages 147–154. IEEE Computer Society, July 2009. Copyright by IEEE. (PDF) (doi:10.1109/TASE.2009.14)

The paper demonstrates that the idea of formalizing basic software engineering concepts in Hoare and He's Unifying Theories of Programming really helps to transfer knowledge and technology. In this case mutation testing found a new application domain: component connectors for modeling coordination in service-oriented architectures.

Abstract. The complex interactions appearing in service-oriented computing make coordination a key concern in serviceoriented systems. In this paper, we present a fault-based method to generate test cases for component connectors from specifications. For connectors, faults are caused by possible errors during the development process, such as wrongly used channels, missing or redundant subcircuits, or circuits with wrongly constructed topology. We give test cases and connectors a unifying formal semantics by using the notion of design, and generate test cases by solving constraints obtained from the specification and faulty connectors. A prototype symbolic test case generator serves to demonstrate the automatizing of the approach.

More conference papers.

FMCO 2008: Conformance Testing of Distributed Concurrent Systems with Executable Designs

Photo: *clairity*

In August our FMCO 2008 paper was published:

Bernhard K. Aichernig, Andreas Griesmayer, Einar Broch Johnsen, Rudolf Schlatte, and Andries Stam. Conformance testing of distributed concurrent systems with executable designs. In Formal Methods for Components and Objects: 7th International Symposium, FMCO 2008, Sophia Antipolis, France, October 21–23, 2008, Revised, volume 5751 of Lecture Notes in Computer Science, pages 61–81, Berlin/Heidelberg, August 2009. Springer. (PDF) (doi:10.1007/978-3-642-04167-9_4)

The paper is a result of the CREDO project. It shows our testing process based on the modeling language Creol. Concolic execution on the model generates test stimuli and these are then executed on a system under test. During a test run the event traces are recorded and then replayed on the model. A small GUI helps to identify the controllable events for generating the test driver for Creol.

Abstract. This paper presents a unified approach to test case generation and conformance test execution in a distributed setting. A model in the object-oriented, concurrent modeling language Creol is used both for generating test inputs and as a test oracle. For test case generation, we extend Dynamic Symbolic Execution (also called Concolic Execution) to work with multi-threaded models and use this to generate test inputs that maximize model coverage. For test case execution, we establish a conformance relation based on trace inclusion by recording traces of events in the system under test and replaying them in the model. User input is handled by generating a test driver that supplies the needed stimuli to the model. An industrial case study of the Credo project serves to demonstrate the approach.

More conference papers.

MBT 2010, March 21, Paphos, Cyprus

Photo: Grabowski

I have been invited to serve on the Program Committee of MBT 2010, the 6th Workshop on Model-Based Testing. MBT 2010 is a satellite worksop of ETAPS 2010 to be held at Paphos, the birthplace of Aphrodite.

The workshop is devoted to model-based testing of both software and hardware. Model-based testing is closely related to model-based specification. Models are used to describe the behavior of the system under consideration and to guide such efforts as test selection and test results evaluation. Both testing and verification are used to validate models against the requirements and check that the implementation conforms to the specification model.

The call for papers is out. Submission deadline: December 10, 2009

Model-based testing has gained attention with the popularization of models in software/hardware design and development. Of particular importance are formal models with precise semantics, such as state-based formalisms. Testing with such models allows one to measure the degree of the product's conformance with the model.

Techniques to support model-based testing are drawn from diverse areas, like formal verification, model checking, control and data flow analysis, grammar analysis, and Markov decision processes.

The intent of this workshop is to bring together researchers and users of models for to discuss the state of the art in theory, applications, tools, and industrialization of model-based specification, testing and verification.

Original submissions are solicited from industry and academia. They are invited to present their work, plans, and views related to model-based testing. The topics of interest include but are not limited to:

• Online and offline test sequence generation methods and tools
• Test data selection methods and tools
• Runtime verification
• Model-based test coverage metrics
• Automatic domain/partition analysis
• Combination of verification and testing
• Models as test oracles
• Scenario based test generation
• Meta programming support for testing
• Formalisms suitable for model-based testing
• Application of model checking techniques in model-based testing
• Game-theoretic approaches to testing
• Model-based testing in industry: problems and achievements

Journal Paper: Fault-Based Conformance Testing in Practice

Photo: marcusuke

We got our work on mutation testing of communication protocols published in the International Journal of Software and Informatics.

The article is part of a special double issue on Formal Methods edited by Prof. Dines Bjorner. The journal is published by the Chinese Academy of Science and has an international editing board.

Martin Weiglhofer, Bernhard Aichernig, and Franz Wotawa. Fault-based conformance testing in practice. International Journal of Software and Informatics, 3(2–3):375–411, June/September 2009. Copyright by Institute for Software, Chinese Academy of Science. (PDF)

In this work we present our results on testing two communcation protocols the Session Initiation Protocol and the Conference Protocol. The protocols are modeled in LOTOS and then mutation testing is applied on the LOTOS specs for generating test cases. We present detailed data of the test process including the nice result that we were able to find a new bug in an implementation that was not detected by random and scenario-based testing.

Paper abstract: Conforming to protocol specications is a critical issue in modern distributed software systems. Nowadays, complex service infrastructures, such as Voice-over-IP systems, are usually built by combining components of dierent vendors. If the components do not correctly implement the various protocol specications, failures will certainly occur. In the case of emergency calls this may be even life-threatening. Functional black-box conformance testing, where one checks the conformance of the implemented protocol to a specication becomes therefore a major issue. In this work, we report on our experiences and ndings when applying fault-based conformance testing to an industrial application. Besides a discussion on modeling and simplications we present a technique that prevents an application from implementing particular faults. Faults are modeled at the level of the specication. We show how such a technique can be adapted to specications with large state spaces and present results obtained when applying our technique to the Session Initiation Protocol and to the Conference Protocol. Finally, we compare our results to random and scenario based testing.

AI meets Formal Methods: Qualitative Reasoning and Action Systems

Photo: cokada

We got our work on combining Qualitative Reasoning Techniques with Action Systems accepted at ICFEM 2009 in Rio.

This year the conference was highly competitive with an acceptance rate below 30 percent. The paper integrates Qualitative Differential Equations into Back's Action System formalism for modeling hybrid systems. The work forms part of the MOGENTES Project and aims at model-based testing of hybrid systems.

To our knowledge this is the first time that Qualitative Differential Equations have been merged into a formal development technique. The results are not limited to Action Systems, but apply also to similar formalisms like e.g. Event-B.

Bernhard K. Aichernig, Harald Brandl, and Willibald Krenn. Qualitative action systems. In Proceedings of ICFEM 2009: 11th International Conference on Formal Engineering Methods, Dec 9-12, 2009, Rio de Janeiro, Lecture Notes in Computer Science. Springer-Verlag, 2009. in press. (PDF)

Paper abstract: An extension to action systems is presented facilitating the modeling of continuous behavior in the discrete domain. The original action system formalism has been developed by Back et al. in order to describe parallel and distributed computations of discrete systems, i.e. systems with discrete state space and discrete control. In order to cope with hybrid systems, i.e. systems with continuous evolution and discrete control, two extensions have been proposed: hybrid action sys- tems and continuous action systems. Both use dierential equations (relations) to describe continuous evolution. Our version of action systems takes an alternative approach by adding a level of abstraction: continuous behavior is modeled by Qualitative Dierential Equations that are the preferred choice when it comes to specifying abstract and possibly non-deterministic requirements of continuous behavior. Because their solutions are transition systems, all evolutions in our qualitative action systems are discrete. Based on hybrid action systems, we develop a new theory of qualitative action systems and discuss how we have applied such models in the context of automated test-case generation for hybrid systems.

TAP 2010, June 28 - July 2, Malaga, Spain

Photo: Pat McDonald

I have been invited to serve on the Program Committee of TAP 2010, the 4th International Conference on Tests & Proofs.

The TAP conference is devoted to the convergence of proofs and tests. It combines ideas from both sides for the advancement of software quality.

Gordon Fraser, a former colleague of mine at TU Graz, is co-chairing the PC with Angelo Gargantini. The conference chairs are Yuri Gurevich, Microsoft Research, USA and Bertrand Meyer, ETH Zuerich, Switzerland.

Last year, we had a paper at TAP 2009 on concolic execution of distributed systems (see my conference papers).

To prove the correctness of a program is to demonstrate, through impeccable mathematical techniques, that it has no bugs; to test a program is to run it with the expectation of discovering bugs. The two techniques seem contradictory: if you have proved your program, it's fruitless to comb it for bugs; and if you are testing it, that is surely a sign that you have given up on any hope to prove its correctness.

Accordingly, proofs and tests have, since the onset of software engineering research, been pursued by distinct communities using rather different techniques and tools.

And yet the development of both approaches leads to the discovery of common issues and to the realization that each may need the other. The emergence of model checking has been one of the first signs that contradiction may yield to complementarity, but in the past few years an increasing number of research efforts have encountered the need for combining proofs and tests, dropping earlier dogmatic views of incompatibility and taking instead the best of what each of these software engineering domains has to offer.

The conference will include a mix of invited and submitted presentation, and a generous allocation of panels and informal discussions. All papers will be published in Springer's LNCS series.

Modeling and Testing Multi-Threaded Asynchronous Systems with Creol

Bosphorus bridge in Istanbul viewing from Asia to Europe.

After a long waiting period, yesterday our TTSS 2008 paper was published by Elsevier. The paper is a result of the CREDO project.

Bernhard Aichernig, Andreas Griesmayer, Rudolf Schlatte, and Andries Stam. Modeling and testing multi-threaded asynchronous systems with Creol. In Proceedings of the 2nd International Workshop on Harnessing Theories for Tool Support in Software (TTSS 2008), Istanbul, Turkey, 30 August 2008, volume 243 of Electronic Notes in Theoretical Computer Science, pages 3–14. Elsevier, July 2009. (PDF) (doi:10.1016/j.entcs.2009.07.002)

The paper describes a model-based testing technique for asynchronous, concurrent systems and shows how the modeling language Creol can serve as a test oracle. Creol is a nice object-oriented modeling language for distributed systems by University of Oslo. We applied the technique to a commercial agent-based information system.

The basic steps of our method are:

• Create an abstract model of the system under test (SUT) in Creol.
• Instrument the source code of the SUT by aspect-oriented programming techniques (here AspectC) for recording events.
• Execute tests on the SUT and record the events.
• Translate the event traces to Creol test cases.
• Replay the test case on the model.
• If successful: the test passed;
• If it fails: model check if there is a different non-deterministic path for this test case, if not the test failed.

Check out my other conference papers.