To identify problems in a software development process, we have been developing an automated measurement tool called TaskPit, which monitors software development tasks such as programming, testing and documentation based on the execution history of software applications. This paper introduces the system requirements, design and implementation of TaskPit; then, presents two real-world case studies applying TaskPit to actual software development. In the first case study, we applied TaskPit to 12 software developers in a certain software development division. As a result, several concerns (to be improved) have been revealed such as (a) a project leader spent too much time on development tasks while he was supposed to be a manager rather than a developer, (b) several developers rarely used e-mails despite the company's instruction to use e-mail as much as possible to leave communication records during development, and (c) several developers wrote too long e-mails to their customers. In the second case study, we have recorded the planned, actual, and self reported time of development tasks. As a result, we found that (d) there were unplanned tasks in more than half of days, and (e) the declared time became closer day by day to the actual time measured by TaskPit. These findings suggest that TaskPit is useful not only for a project manager who is responsible for process monitoring and improvement but also for a developer who wants to improve by him/herself.

There are many roles to play in the bug fixing process in open source software development. A developer called “Committer”, who has a permission to submit a patch into a software repository, plays a major role in this process and holds a key to the successfulness of the project. Despite the importance of committer's activities, we suspect that sometimes committers can make mistakes which have some consequences to the bug fixing process (e.g., reopened bugs after bug fixing). Our research focuses on studying the consequences of each committer's activities to this process. We collected each committer's historical data from the Eclipse-Platform's bug tracking system and version control system and evaluated their activities using bug status in the bug tracking system and commit log in the version control system. Then we looked deeper into each committer's characteristics to see the reasons why some committers tend to make mistakes more than the others.

To detect the theft of Java class files efficiently, we propose a concept of Java birthmarks, which are unique and native characteristics of every class file. For a pair of class files p and q, if q has the same birthmark as p's, q is suspected as a copy of p. Ideally, the birthmarks should satisfy the following properties: (a) preservation - the birthmarks should be preserved even if the original class file is tampered with, and (b) distinction - independent class files must be distinguished by completely different birthmarks. Taking (a) and (b) into account, we propose four types of birthmarks for Java class files. To show the effectiveness of the proposed birthmarks, we conduct three experiments. In the first experiment, we demonstrate that the proposed birthmarks are sufficiently robust against automatic program transformation (93.3876% of the birthmarks were preserved). The second experiment shows that the proposed birthmarks successfully distinguish non-copied files in a practical Java application (97.8005% of given class files were distinguished). In the third experiment, we exploit different Java compilers to confirm that the proposed Java birthmarks are core characteristics independent of compiler-specific issues.

This letter describes system architecture of a measurement environment called GINGER that helps programmer productivity by automatically collecting and analyzing data about software development activities. Based on the result of analysis, the environment gives programmers feedback so that they can recognize and improve their activities. The first prototype system of GINGER is currently implemented in a UNIX environment. We expect that the proposed environment will improve the overall productivity of programmers and the quality of resulting products.

This paper proposes to use eye movements to characterize the performance of individuals in reviewing software documents. We design and implement a system called DRESREM, which measures and records eye movements of document reviewers. Based on the eye movements captured by eye tracking device, the system computes the line number of the document that the reviewer is currently looking at. The system can also record and play back how the eyes moved during the review process. To evaluate the effectiveness of the system we conducted an experiment to analyze 30 processes of source code review (6 programs, 5 subjects) using the system. As a result, we have identified a particular pattern, called scan, in the subject's eye movements. Quantitative analysis showed that reviewers who did not spend enough time on the scan took more time to find defects on average.

This paper describes a new metric for evaluating the cost effectiveness of software reviews. The proposed metric is based on the degree to which costs needed for detecting all faults from software are reduced by software reviews in a project. The metric can be interpreted as combining two conventional metrics proposed by Fagan (1976) and Collofello and Woodfield (1989). As the proposed metric is normalized by virtual testing cost, we can compare the values of it among any different kind of projects. Using an experimental evaluation of the conventional metrics and the proposed metric for data collected in an industrial environment, we show the validity and usefulness of the proposed metric. In addition, we present a method to estimate a value of the proposed metric by using only the values which can be computed during the software reviews.

Nowadays, software development societies have given more precedence to Open Source Software (OSS). There is much research aimed at understanding the OSS society to sustain the OSS product. To lead an OSS project to a successful conclusion, researchers study how developers change source codes called patches in project repositories. In existing studies, we found an argument in the conventional patch acceptance detection procedure. It was so simplified that it omitted important cases from the analysis, and would lead researchers to wrong conclusions. In this research, we propose an algorithm to overcome the problem. To prove out our algorithm, we constructed a framework and conducted two case studies. As a result, we came to a new and interesting understanding of patch activities.

We present a comparison of correlated failures for multiversion software using community error recovery (CER) and software breeding (SB). In CER, errors are detected and recovered at checkpoints which are inserted in all the versions of the software. SB is analogous to the breeding of plants and animals. In SB, versions consist of loadable modules, and a driver exchanges the modules between versions to detect and eliminate faulty modules. We formulate reliability models to estimate the probability of failure for software using either CER or SB. Our reliability models assume failures in the checkpoints in CER and the driver in SB. We use beta-binomial distribution for modeling correlated failures of versions, because much of the evidence suggests that the assumption that failures in versions occur independently is not always true. Our comparison indicates that multiversion software using SB is more reliable than that using CER when the probability of failure in the checkpoints in CER or the driver in SB is 10-7.