In this paper, we show that motion sharpening phenomenon can be explained as a form of visual masking for a special case where a video sequence is composed of alternate frames with different level of sharpness. A frame of higher sharpness behaves to mask the ambiguity of a subsequent frame of lower sharpness and hence preserves the perceptive quality of the whole sequence. Borrowing the mechanism for visual masking, we formulated a quantitative model for deriving the minimum spatial frequency conditions which preserves the subjective quality of the frames being masked. The quantitative model takes into account three fundamental properties of the video signals, namely the size of motion, average luminance and the power of each frequency components. The psychophysical responses towards the changes of these properties are obtained through subjective assessment tests using video sequences of simple geometrical patterns. Subjective experiments on natural video sequences show that more than 75% of viewers could make no distinction between the original sequence and the one processed using the quantitative model.

We present a method for the objective quality evaluation of noise-reduced speech in wideband speech communication services, which utilize speech with a wider bandwidth (e.g., 7 kHz) than the usual telephone bandwidth. Experiments indicate that the amount of residual noise and the distortion of speech and noise, which are quality factors, influence the perceived quality degradation of noise-reduced speech. From the results, we observe the principal relationships between these quality factors and perceived speech quality. On the basis of these relationships, we propose a method that quantifies each quality factor in noise-reduced speech by analyzing signals that can be measured and assesses the overall perceived quality of noise-reduced speech using values of these quality factors. To verify the validity of the method, we perform a subjective listening test and compare subjective quality of noise-reduced speech with its estimation. In the test, we use various types of background noise and noise-reduction algorithms. The verification results indicate that the correlation between subjective quality and its objective estimation is sufficiently high regardless of the type of background noise and noise-reduction algorithm.

VoIP is one of the key technologies for recent telecommunication services. In addition to the migration from the conventional PSTN to IP networks, mobile networks will follow the PSTN in moving to an IP-based infrastructure. Due to limited radio resources, the speech bitrate in mobile networks must be more strongly compressed than is true in PSTN. This will lead to a heterogeneous network environment, in which different speech codecs are employed in fixed and mobile networks. Therefore, from the viewpoint of designing and managing the QoE (Quality of Experience) of end-to-end telephony services, establishing a method to evaluate the quality of VoIP in such a heterogeneous network environment is very important. The quality of speech communication services should be discussed in subjective terms. Subjective quality assessment is time-consuming and expensive, however, so objective quality assessment which estimates subjective quality without carrying out subjective quality experiments is desirable. To establish an objective method to evaluate the end-to-end quality of speech in a heterogeneous network environment, this paper proposes a method for estimating the end-to-end listening quality based on the quality in each individual segment. This method is very important because conventional technologies such as the E-model, which was standardized as ITU-T Recommendation G.107, cannot accurately estimate overall quality based on segmental qualities. The experimentals show that the proposed method offers better performance in terms of quality estimation than the conventional method.

Interactive audio-video applications over IP networks have subjective tradeoffs between fidelity and latency owing to packet buffering at the receiver. Increasing the buffering time improves the fidelity, whereas it degrades the latency. This paper makes the subjective tradeoff between fidelity and latency clear in a quantitative way. In addition, we examine the effect of tasks on the subjective tradeoff. In evaluating the effect of tasks, we use two tasks according to ITU-T Recommendation P.920. An experiment was conducted to measure user-level QoS of an interactive application with the psychometric methods. We then investigate the subjective tradeoff quantitatively by QoS mapping. The experimental results confirm that there exists the buffering time which makes user-level QoS the highest. The results also show that the optimum buffering time depends on the kind of task.

The performance of a force feedback system is disturbed by delay that arises from the time required for transmission and processing of data. We used a psychophysical method to measure how much a user's subjective impression of elasticity associated with delays of feedback force deviated from the original physical elasticity. The results show that users' point of subjective equality (PSE) for their subjective impression of elasticity decreased as the delay of feedback force increased. We proposed a model that estimates the PSE of elasticity from the variables that can be physically measured. Another experiment was conducted to examine the model's prediction, which the results supported.

The Video Quality Experts Group (VQEG) is preparing a programme of subjective multimedia quality tests. The results from these tests will be used to evaluate the performance of competing objective multimedia quality metrics. The reliability of the subjective test data is of great importance for VQEG's task. This paper provides an overview of VQEG's multimedia ad-hoc group. The work of this group will require subjective tests to be performed by laboratories located in Europe, Asia and North America. For VQEG's multimedia work to be successful, the subjective assessment methodology must be precisely defined and produce reliable and repeatable subjective quality data. Although international standards covering multimedia quality assessment methods are in force, there remains some uncertainty regarding the most effective approach to assessing the subjective quality of multimedia. A review of existing methods is provided. Two experiments are presented investigating the suitability of alternative subjective assessment methods (single-stimulus ACR and SAMVIQ). The results of these experiments are discussed within the context of the VQEG multimedia testing programme.

This paper provides an overview of the new tendencies in the subjective assessment of the quality of video for Multimedia applications. New subjective assessment methods are here described together with the description of the new general approaches. Some motivations of these new approaches are also here provided.

Visualizing occluded objects is a useful applications of Mixed Reality (MR), which we call "see-through vision." For this application, it is important to display occluded objects in such a manner that they can be recognized intuitively by the user. Here, we evaluated four visualization methods for see-through vision that can aid the user to recognize occluded objects in outdoor scenes intuitively: "elimination of occluding objects," "ground grid," "overlaying model of occluding object," and "top-down view." As we used a new handheld MR device for outdoor see-through vision, we performed subjective experiments to determine the best combination of methods. The experimental results indicated that a combination of showing the ground grid, overlaying wireframe models of occluding objects, and top-down view to be optimal, while it was not necessary to display occluding objects for outdoor see-through vision with a handheld device, because users can see them with the naked eye.

In this paper, we discuss the issue of empathy-inducing effect brought by Japanese Haiku, regarded as the world-shortest poetry. Its condensed form of word combination sometimes arouses deep impression and empathy in readers. Its possibilities as communication media would be enhanced in multimedia contexts, when considering educational purpose or cross-cultural interchanges. An experimental result is shown for evaluating multimedia effect on combined presentation of a CG drawing and its associated haiku in English. We confirmed that various aspects of visual impression and empathy were sometimes enhanced with certain haiku text as linguistic stimuli, compared with the condition of viewing a CG drawing only. The reason of raised empathy was analyzed from the various aspects of the content provided by the associated haiku text complementary for the drawing. Moreover, we examined the effects of poetic styles on subjects' empathy, using Japanese prose and conventional haiku style (5-7-5 syllabic rhyme), both of them were translated from the same haiku in English.

This paper proposes a subjective model for estimating the quality of video streaming services with dynamic bit-rate control. In a subjective quality assessment test, we clarify users' perceptions of distributed video signals whose quality is time-variant due to dynamic bit-rate control. Using this result, we constructed an estimation model considering the following three characteristics: 1) the influence of the video section where quality degradation is large will strongly affect the overall quality, 2) the impression of a past quality weakens with the passage of time, and 3) the range of evaluation scores becomes wider when the time duration of an evaluation is longer. We found that the proposed model enables the accuracy of estimating overall subjective quality to be dramatically improved compared with that of a model that averages segmental quality. The estimation error of the proposed model is less than the statistical reliability of the subjective score even for verification data. We also show that our findings are applicable to QoS design/management issues for video streaming services with dynamic bit-rate control.

We propose the concept of an opinion model for interactive multimodal services and apply it to an audiovisual communication service. First, psychological factors of an audiovisual communication service were extracted by using the semantic differential (SD) technique and factor analysis. Forty subjects participated in subjective tests and performed point-to-point conversational tasks on a PC-based video phone that exhibited various network qualities. The subjects assessed those qualities on the basis of 25 pairs of adjectives. Two psychological factors, i.e., an aesthetic feeling and a feeling of activity, were extracted from the results. Then, quality impairment factors affecting these two psychological factors were analyzed. We found that the aesthetic feeling was affected by IP packet loss and video coding bit rate, and the feeling of activity depended on delay time, video packet loss, video coding bit rate, and video frame rate. Using this result, we formulated an opinion model derived from the relationships among quality impairment factors, psychological factors, and overall quality. The validation test results indicated that the estimation error of our model was almost equivalent to the statistical reliability of the subjective score.

We proposed and evaluated a speech packet loss concealment method which predicts lost segments from speech included in packets either before, or both before and after the lost packet. The lost segments are predicted recursively by using linear prediction both in the forward direction from the packet preceding the loss, and in the backward direction from the packet succeeding the lost segment. Predicted samples in each direction are smoothed by averaging using linear weights to obtain the final interpolated signal. The adjacent segments are also smoothed extensively to significantly reduce the speech quality discontinuity between the interpolated signal and the received speech signal. Subjective quality comparisons between the proposed method and the the packet loss concealment algorithm described in the ITU standard G.711 Appendix I showed similar scores up to about 10% packet loss. However, the proposed method showed higher scores above this loss rate, with Mean Opinion Score rating exceeding 2.4, even at an extremely high packet loss rate of 30%. Packet loss concealment of speech degraded with G.729 coding, and babble noise mixed speech showed similar trends, with the proposed method showing higher qualities at high loss rates. We plan to further improve the performance by using adaptive LPC prediction order depending on the estimated pitch, and adaptive LPC bandwidth expansion depending on the consecutive number of repetitive prediction, among many other improvements. We also plan to investigate complexity reduction using gradient LPC coefficient updates, and processing delay reduction using adaptive forward/bidirectional prediction modes depending on the measured packet loss ratio.

Next generation codecs, benchmarked by the H.264/AVC standard, are providing substantial compression efficiency for the coding and transmission of video. Coupled with technologies offering larger transmission bandwidths over DSL, wireless and satellite networks, the capability of delivering high quality video services to the home is now a reality. The perceptual quality of the content delivered over communications networks will be crucial in ensuring a first-class customer experience. It is therefore important to assess the advantages and disadvantages of the optional features offered by next generation codecs. This paper describes a subjective assessment that was carried out to investigate the perceptual effects of switching the in loop de-blocking filter within the H.264/ AVC CODEC on or off. Although the filter is believed to substantially improve the perceptual quality of video, it has been suggested that in some cases negative perceptual effects can be produced. The H.264/AVC architecture allows de-blocking to be switched off in cases where there are limited processing resources or it is considered a negative perceptual effect may be introduced. This paper describes a study that examined the perceptual effects of de-blocking by employing a standardised subjective assessment methodology. The Absolute Category Rating (ACR) method was used to capture Difference Mean Opinion Scores (DMOS) for a range of video. Content was selected to span a wide and representative range of coding complexity. This content was then encoded at a variety of bit-rates to represent high, medium and low qualities. Results were used to examine the end-user perception of video quality when the de-blocking filter is switched on or off. The experimental design allowed the overall effects of the de-blocking filter to be examined and additionally the relationship between content and quality on the filter performance. The experiment found that the performance of the de-blocking filter was content-dependent. Results were used to discuss the advantages and disadvantages of in-loop de-blocking and there is an examination of content properties (e.g. spatial and temporal complexity) that influence the performance of de-blocking.

VoIP is one of the key technologies for recent telecommunication services. The quality of its services should be discussed in subjective terms. Since subjective quality assessment is time-consuming and expensive, however, objective quality assessment which estimates subjective quality without carrying out subjective quality experiments is desirable. This paper discusses the performance of the objective quality measure that was standardized as ITU-T Recommendation P.862 and clarifies the quality factors that can be evaluated with satisfactory accuracy based on it. We found that P.862 can be applied to the evaluation of coding distortion, tandeming of codecs, transmission bit-errors, packet loss, and silence compression in a codec, at least for clean Japanese speech. In addition, we propose a method of estimating the subjective quality evaluation value from objective measurement results and show the validity of this method. We also evaluate the uniqueness of objective quality assessment based on P.862 from the viewpoints of the effect of measurement noise and the variation of test speech samples, and propose how to improve the reproducibility of objective quality assessment.

The measurement of the 3-dimensional behavior of early reflections in a sound field has been an important issue in auditorium acoustics since the reflection profile has been found to be strongly correlated with the subjective responsiveness of a listener. In order to detect the incidence angle and relative amplitude of reflections, a 4-point microphone system has conventionally been used. A new measurement system is proposed in this paper, which has 5 microphones. Microphones are located on each four apex of a tetrahedron and at the center of gravity. Early reflections, including simultaneously incident reflections,which previous 4-point microphone system could not discriminate as individual wavefronts, were successfully found with the new system. In order to calculate accurate image source positions, it is necessary to determine the exact peak positions from measured impulse responses composed of highly deformed and overlapped impulse trains. For this purpose, a peak-detecting algorithm, which finds dominant peaks in the impulse response by an iteration method, is introduced. In this paper, the theoretical background and features of the 5-microphone system are described. Also, some results of experiments using this system are described.

This paper assesses the media synchronization quality of recovery control schemes from asynchrony, which are referred to as reactive control schemes here, in terms of objective and subjective measures. We deal with four reactive control techniques: skipping, discarding, shortening and extension of output duration, and virtual-time contraction and expansion. We have carried out subjective and objective assessment of the media synchronization quality of nine schemes which consist of combinations of the four techniques. The paper makes a comparison of media synchronization quality among the schemes. It also clarifies the relations between the two kinds of quality measures.

We proposed a new method to evaluate display legibility as a function of resolution. This method was able to evaluated display legibility without being restricted to the display resolution. Using this method, subjective psychological experiments were carried out to investigate display resolution, which provides legibility, in observing small characters. Samples viewed by subjects were images displayed on a high-resolution TFT-LCD that we developed, CRT images and printed documents for comparison. We have found that TFT-LCD legibility was much better than that of CRT, and that minimum resolution of about 175 dpi was needed for use in legible document viewers.

In this paper, we describe an experimental evaluation of visual fatigue in a binocular disparity type 3-D display system. To evaluate this fatigue, we use a subjective assessment method and focus on mismatching between convergence and accommodation, which is a major weakness of binocular disparity 3-D displays. For this subjective assessment, we use a newly-developed binocular disparity 3-D display system with a compensation function for accommodation. Because this equipment only allowed us to compare the terms of the mismatching itself, the evaluation is more accurate than similar previous works.

Cell loss is one of the most important metrics of quality of service in ATM mobile communication systems. This loss can be suppressed by introducing buffer memories in the network, but that sacrifices delay. This paper proposes a lossless handover scheme for ATM mobile communication networks that can suppress delay fluctuations, and presents a subjective evaluation of MPEG2 images with various buffer memory sizes.

This paper presents results of subjective assessment of the media synchronization quality in the virtual-time rendering (VTR) media synchronization algorithm. For the assessment, stored voice and video streams were transmitted as two separate transport streams from a source to a destination on various traffic conditions in an experimental system. At the destination, they were output after synchronization control. We subjectively assessed the quality of media synchronization in a systematic way. This paper examines the effects of the difference between methods of recovery from asynchrony on the media synchronization quality. The paper also clarifies the relationships between the subjective and objective performance measures. Furthermore, it examines the effect of the difference in scene between media streams and that of the modification of the target output time on the media synchronization quality.