Within information hiding technology, digital watermarking is one of the most important technologies for copyright protection of digital content. Many digital watermarking schemes have been proposed in academia. However, these schemes are not used, because they are not practical; one reason for this is that the evaluation criteria are loosely defined. To make the evaluation more concrete and improve the practicality of digital watermarking, watermarking schemes must use common evaluation criteria. To realize such criteria, we organized the Information Hiding and its Criteria for Evaluation (IHC) Committee to create useful, globally accepted evaluation criteria for information hiding technology. The IHC Committee improves their evaluation criteria every year, and holds a competition for digital watermarking based on state-of-the-art evaluation criteria. In this paper, we describe the activities of the IHC Committee and its evaluation criteria for digital watermarking of still images, videos, and audio.

In the field of software reengineering, many component identification approaches have been proposed for evolving legacy systems into component-based systems. Understanding the behaviors of various component identification approaches is the first important step to meaningfully employ them for legacy systems evolution, therefore we performed an empirical study on component identification technology with considerations of their similarity measures, clustering approaches and stopping criteria. We proposed a set of evaluation criteria and developed the tool CIETool to automate the process of component identification and evaluation. The experimental results revealed that many components of poor quality were produced by the employed component identification approaches; that is, many of the identified components were tightly coupled, weakly cohesive, or had inappropriate numbers of implementation classes and interface operations. Finally, we presented an analysis on the component identification approaches according to the proposed evaluation criteria, which suggested that the weaknesses of these clustering approaches were the major reasons that caused components of poor-quality.

Intrusion Detection Systems (IDS) have been received considerable attention among the network security researchers as one of the most promising countermeasures to defend our crucial computer systems or networks against attackers on the Internet. Over the past few years, many machine learning techniques have been applied to IDSs so as to improve their performance and to construct them with low cost and effort. Especially, unsupervised anomaly detection techniques have a significant advantage in their capability to identify unforeseen attacks, i.e., 0-day attacks, and to build intrusion detection models without any labeled (i.e., pre-classified) training data in an automated manner. In this paper, we conduct a set of experiments to evaluate and analyze performance of the major unsupervised anomaly detection techniques using real traffic data which are obtained at our honeypots deployed inside and outside of the campus network of Kyoto University, and using various evaluation criteria, i.e., performance evaluation by similarity measurements and the size of training data, overall performance, detection ability for unknown attacks, and time complexity. Our experimental results give some practical and useful guidelines to IDS researchers and operators, so that they can acquire insight to apply these techniques to the area of intrusion detection, and devise more effective intrusion detection models.

Performance evaluation criterion is one of the most important issues for design of network traffic control mechanisms and algorithms. Due to multiple performance objectives of network traffic control, performance evaluation criteria must include multiple performance metrics executed simultaneously, which is called integrated performance evaluation criteria. In this paper, we analyze various performance metrics of network traffic control, and propose three integrated performance evaluation criteria. One is the improvement on original Power formula; our new Power formula is based on the multi-service-class model. Another is about the fairness of user's QoS (Quality of Service) requirements (queuing delay and loss rate); especially the detailed discussion on Proportional Fairness Principle is given. And the third one is the integration of preceding two, in which the throughput, queuing delay, packet loss rate, and the fairness are considered simultaneously.