This paper presents the results of a continuing research work on the practical characterization of operating systems (OS) behavior in the presence of software faults in OS components, such as faulty device drivers. The methodology used is based on the emulation of software faults in device drivers and observation of the behavior of the overall system regarding a comprehensive set of failure modes, analyzed according to different dimensions related to multiple user perspectives. The emulation of the software faults is done through the injection of specific mutations at machine-code level that reproduce the code generated by compilers when typical programming errors occur in the high level language code. Two important aspects of this methodology are the independence of source code availability and the use of simple and established practices to evaluate operating systems failure modes, thus allowing its use as a dependability benchmarking technique. The generalization of the methodology to any software system built of discrete and identifiable components is also discussed.

The application of fault injection in the context of dependability benchmarking is far from being straightforward. One decisive issue to be addressed is to what extent injected faults are representative of the considered faults. This paper proposes an approach to analyze the effects of real and injected faults.