Embedded software often interacts with multiple inputs from various sensors whose dependency is often complex or partially known to developers. With incomplete information on dependency, testing is likely to be insufficient in detecting errors. We propose a method to enhance testing coverage of embedded software by identifying subtle and often neglected dependencies using information contained in usage log. Usage log, traditionally used primarily for investigative purpose following accidents, can also make useful contribution during testing of embedded software. Our approach relies on first individually developing behavioral model for each environmental input, performing compositional analysis while identifying feasible but untested dependencies from usage log, and generating additional test cases that correspond to untested or insufficiently tested dependencies. Experimental evaluation was performed on an Android application named Gravity Screen as well as an Arduino-based wearable glove app. Whereas conventional CTM-based testing technique achieved average branch coverage of 26% and 68% on these applications, respectively, proposed technique achieved 100% coverage in both.

Software testing is an important problem to design a large software system and it is difficult to be solved due to its computational complexity. We try to use program nets to approach this problem. As the first step towards solving software testing problem, this paper provides a technique to generate subnets of a program net and applies this technique to software testing. Firstly, definitions and properties of program nets are introduced based on our previous works, and the explanation of software testing problem is given. Secondly, polynomial algorithms are proposed to generate subnets that can cover all the given program net. Finally, a case study is presented to show how to find subnets covering a given program net by using the proposed algorithms, as well as to show the input test data of the program net for software testing.

For the past couple of decades, the usage of the Web as a platform for deploying software products has become incredibly popular. Web applications became more prevalent, as well as more complex. Countless Web applications have already been designed, developed, tested, and deployed on the Internet. However, it is noticeable that many common functionalities are present among these vast number of applications. This paper proposes an approach based on a database containing information from previous test artifacts. The information is used to generate test scenarios for Web applications under test. We have developed a tool based on our proposed approach, with the aim of reducing the effort required from software test engineers and professionals during the test planning and creation stage of software engineering. We evaluated our approach from three viewpoints: comparison between our approach and manual generation, qualitative evaluation by professional software engineers, and comparison between our approach and two open-source tools.

This letter discusses the applicability of boolean satisfiability (SAT) solving to pairwise testing in practice. Due to its recent rapid advance, using SAT solving seems a promising approach for search-based testing and indeed has already been practiced in test generation for pairwise testing. The previous approaches use SAT solving either for finding a small test set in the absence of parameter constraints or handling constraints, but not for both. This letter proposes an approach that uses a SAT solver for constructing a test set for pairwise testing in the presence of parameter constraints. This allows us to make full use of SAT solving for pairwise testing in practice.

Test coverage is an important indicator of whether software has been sufficiently tested. However, there are several problems with the existing measurement tools for test coverage, such as their cost of development and maintenance, inconsistency, and inflexibility in measurement. We propose a consistent and flexible measurement framework for test coverage that we call the Open Code Coverage Framework (OCCF). It supports multiple programming languages by extracting the commonalities from multiple programming languages using an abstract syntax tree to help in the development of the measurement tools for the test coverage of new programming languages. OCCF allows users to add programming language support independently of the test-coverage-criteria and also to add test-coverage-criteria support independently of programming languages in order to take consistent measurements in each programming language. Moreover, OCCF provides two methods for changin the measurement range and elements using XPath and adding user code in order to make more flexible measurements. We implemented a sample tool for C, Java, and Python using OCCF. OCCF can measure four test-coverage-criteria. We also confirmed that OCCF can support C#, Ruby, JavaScript, and Lua. Moreover, we reduced the lines of code (LOCs) required to implement measurement tools for test coverage by approximately 90% and the time to implement a new test-coverage-criterion by over 80% in an experiment that compared OCCF with the conventional non-framework-based tools.

Many existing software reliability models (SRMs) are based on the assumption that fault correction activities take a negligible amount of time and resources, which is often invalid in real-life situations. Consequently, the estimated and predicted software reliability tends to be over-optimistic, which could in turn mislead management in related decision-makings. In this paper, we first make an in-depth analysis of real-life software testing process; then a Markovian SRM considering fault correction process is proposed. Parameter estimation method and software reliability prediction method are established. A numerical example is given which shows that by using the proposed model and methods, the results obtained tend to be more appropriate and realistic.

In the OTS/CafeOBJ method, software specifications are described in CafeOBJ executable formal specification language, and verification is done by giving scripts to the CafeOBJ system. The script is called a proof score. In this study, we propose a test case generator from an OTS/CafeOBJ specification together with a proof score. Our test case generator gives test cases by analyzing the proof score. The test cases are used to test whether an implementation satisfies the specification and the property verified by the proof score. Since a proof score involves important information for verifying a property, the generated test cases are also expected to be suitable to test the property.

A mobile ad hoc network is an autonomous wireless network which consists of mobile nodes without any base stations. Many routing schemes and services have been proposed for mobile ad hoc networks. However, since these schemes tend to be evaluated through simulation experiments, it is not known whether they work effectively in real environments or not. Therefore, in order to verify their practical use in mobile ad hoc networks, it is necessary to perform field experiments using actual mobile nodes. If the network size is large, it is difficult to perform field experiments due to problems with limited battery, difficulty of topology control and so on. Realization of rapid topology change of the ad hoc networks topology is especially difficult. In order to solve this problem, this paper proposes a testing environment for mobile ad hoc network software, which emulates field experiments in wired networks.

According to the proliferation of ubiquitous computing, various products which contain large-size embedded software have been developed. One of most typical features of embedded software is concurrency of software and hardware factors. That is, software has connected deeply into hardware devices. The existence of various hardware make quality assurance of embedded software more difficult. In order to assure quality of embedded software more effectively, this paper discusses features of embedded software and an effective method for quality assurance for embedded software. In this paper, we first analyze a failure distribution of embedded software and discuss the effects of hardware devices on quality of embedded software. Currently, in order to reduce hardware related faults, huge effort for testing with large number of test items is required. Thus, one of the most important issues for quality assurance of embedded software is how to reduce the cost and effort of software testing. Next, focusing on hardware constraints as well as software specifications in embedded software, we propose an evaluation metrics for determinating important functions for quality of embedded software. Furthermore, by referring to the metrics, undesirable behaviors of important functions are identified as root nodes of fault tree analysis. From the result of case study applying the proposed method to actual project data, we confirmed that test items considering the property of embedded software are constructed. We also confirmed that the constructed test items are appropriate to detect hardware related faults in embedded systems.

In order to respond to the active market's needs for software with various new functions, the system testing must be completed within a limited period. Additionally, important faults, which are closely related to essential functions for users or the target system, have to be removed, preferably in system testing. Many techniques have been proposed to date for effective software testing. Among them, selective software testing is one of the most cost effective techniques. However, most of the previous techniques cannot be applied to short-term development and initial development of software with various new functions because much cost is needed for their testing preparation. In this paper, we propose a new method for selective system testing in which priorities assigned to functions play an essential role in the execution of testing. The priorities are determined based on the evaluation results of three metrics for functions: the frequency of use, the complexity of use scenario, and the fault impact to users. Detailed testing instructions are assigned to test items with high priority, and short and ordinal instructions are assigned to those with low priority. The difference in the volume of testing instruction controls the effort of checking test items. As a result of experimental application to actual software testing in a certain company, we have confirmed that the proposed selective system testing can detect both fatal faults related to key functions and critical faults for the system.

Comparison of intelligent and random testing in data inputting is still under discussion. Little is also known about testing for the whole software and empirical testing methodology when random testing used. This study research not only for data inputting testing, but also operation of software (called transitions) in order to test the whole GUI software by intelligent and random testing. Methodology of this study is that we attempt to research efficiency of random and intelligent testing by Chinese postman problem. In general, random testing is considered straightforward but not efficient. Chinese postman problem testing is complicated but efficient. The comparison between random and intelligent testing would give further recommendation for software testing methodology.

Software and its systems are more complicated than a decade ago, and the systems are used for mission critical business, flight control and so on which often require high assurance systems. In this circumstance, we often use black-box testing. The question now arises that black-box testing does not generate numerical value of testing result but empirical. Thus, in this research, we develop and enhance FSM (Finite State Machine) testing method which can produce code coverage rate as numerical value. Our developed FSM testing by code coverage focuses on not only software system behavior but also data. We found higher code coverage rate, which indicates quality of system, by this method than existing black box testing method.

2-Factor covering designs, a type of combinatorial designs, have recently received attention since they have industrial applications including software testing. For these applications, even a small reduction on the size of a design is significant, because it directly leads to the reduction of testing cost. In this letter, we report ten new designs that we constructed, which improve on the previously best known results.

Conformance testing techniques are required for the efficient production of reliable communication protocols. A lot of conformance testing techniques have been developed. However, most of them can only decide whether an implemented protocol conforms to its specification. That is, the exact locations of faults are not determined by them. This paper presents some conditions that enable to find locations of multiple faults, and then proposes a test sequence generation technique under such conditions. The correctness proof and complexity analysis of the proposed technique are also given. The characteristics of this technique are to generate test sequences based on protocol specifications and interim test results, and to find locations of multiple faults in protocol implementations. Although the length of the test sequence generated by the proposed technique is a little longer than the one generated by the previous one, the class to which the proposed technique can be applied is larger than that to which the previous one can be applied.

This paper describes a new metric for evaluating the cost effectiveness of software reviews. The proposed metric is based on the degree to which costs needed for detecting all faults from software are reduced by software reviews in a project. The metric can be interpreted as combining two conventional metrics proposed by Fagan (1976) and Collofello and Woodfield (1989). As the proposed metric is normalized by virtual testing cost, we can compare the values of it among any different kind of projects. Using an experimental evaluation of the conventional metrics and the proposed metric for data collected in an industrial environment, we show the validity and usefulness of the proposed metric. In addition, we present a method to estimate a value of the proposed metric by using only the values which can be computed during the software reviews.