The human visual system exhibits a characteristic known as the Helmholtz-Kohlrausch (H-K) effect: even if the hue and the lightness retain the same values, the actual lightness (perceived lightness) changes with changes in the color saturation. Quantification of this effect is expected to be useful for the future development and evaluation of high-quality displays. We have been studying the H-K effect in natural images projected by LED projectors, which play important roles in practical uses. To verify the effectiveness of the determinations of the H-K effect for natural images, we have performed a subjective-evaluation experiment by method of adjustment for natural images and compared the experimental values with values calculated from extended form of Nayatani's equation to apply to natural images. In general, we found a high correlation between the two, although there was a low correlation for some images. Therefore, we obtained a correction function derived from the subjective evaluation experiment value of 108 color (hue: 12 × saturation: 3 × lightness: 3) patterns and have applied it to estimate the equation H-K effect.

In recent years, driver's visual attention has been actively studied for driving automation technology. However, the number of models is few to perceive an insight understanding of driver's attention in various moments. All attention models process multi-level image representations by a two-stream/multi-stream network, increasing the computational cost due to an increment of model parameters. However, multi-level image representation such as optical flow plays a vital role in tasks involving videos. Therefore, to reduce the computational cost of a two-stream network and use multi-level image representation, this work proposes a single stream driver's visual attention model for a critical situation. The experiment was conducted using a publicly available critical driving dataset named BDD-A. Qualitative results confirm the effectiveness of the proposed model. Moreover, quantitative results highlight that the proposed model outperforms state-of-the-art visual attention models according to CC and SIM. Extensive ablation studies verify the presence of optical flow in the model, the position of optical flow in the spatial network, the convolution layers to process optical flow, and the computational cost compared to a two-stream model.

High-performance deep learning-based object detection models can reduce traffic accidents using dashcam images during nighttime driving. Deep learning requires a large-scale dataset to obtain a high-performance model. However, existing object detection datasets are mostly daytime scenes and a few nighttime scenes. Increasing the nighttime dataset is laborious and time-consuming. In such a case, it is possible to convert daytime images to nighttime images by image-to-image translation model to augment the nighttime dataset with less effort so that the translated dataset can utilize the annotations of the daytime dataset. Therefore, in this study, a GAN-based image-to-image translation model is proposed by incorporating self-attention with cycle consistency and content/style separation for nighttime data augmentation that shows high fidelity to annotations of the daytime dataset. Experimental results highlight the effectiveness of the proposed model compared with other models in terms of translated images and FID scores. Moreover, the high fidelity of translated images to the annotations is verified by a small object detection model according to detection results and mAP. Ablation studies confirm the effectiveness of self-attention in the proposed model. As a contribution to GAN-based data augmentation, the source code of the proposed image translation model is publicly available at https://github.com/subecky/Image-Translation-With-Self-Attention

The authors have evaluated a method of expanding the bit depth of image signals called SGRAD, which requires fewer calculations, while degrading the sharpness of images less. Where noise is superimposed on image signals, the conventional method for obtaining high bit depth sometimes incorrectly detects the contours of images, making it unable to sufficiently correct the gradation. Requiring many line memories is also an issue with the conventional method when applying the process to vertical gradation. As a solution to this particular issue, SGRAD improves the method of detecting contours with transiting gradation to effectively correct the gradation of image signals which noise is superimposed on. In addition, the use of a prediction algorithm for detecting gradation reduces the scale of the circuit with less correction of the vertical gradation.

An interactive support system for image quality enhancement to adjust display equipments according to the user's own subjectivity is developed. Interactive support system for image quality enhancement enable the parameters based on user's preference to be derived by only selecting user's preference images without adjusting image quality parameters directly. In the interactive support system for image quality enhancement, the more the number of parameters is, the more effective this system is. In this paper, lightness, color and sharpness are used as the image quality parameters and the images are enhanced by increasing the number of parameters. Shape of tone curve is controlled by two image quality adjustment parameters for lightness enhancement. Images are enhanced using two image quality adjustment parameters for color enhancement. The two parameters are controlled in L* a* b* color space. Degree and coarseness of image sharpness enhancement are adjusted by controlling a radius of mask of smoothing filter and weight of adding. To confirm the effectiveness of the proposed method, the image quality and derivation time of the proposed method are compared with a manual adjustment method.

In previous studies, the retrieval accuracy of large image databases has been improved as a result of reducing the semantic gap by combining the input sketch with relevance feedback. A further improvement of retrieval accuracy is expected by combining each stroke, and its order, of the input sketch with the relevance feedback. However, this leaves as a problem the fact that the effect of the relevance feedback substantially depends on the stroke order in the input sketch. Although it is theoretically possible to consider all the possible stroke orders, that would cause a realistic problem of creating an enormous amount of data. Consequently, the technique introduced in this paper intends to improve retrieval efficiency by effectively using the relevance feedback by means of conducting data mining of the sketch considering the similarity in the order of strokes. To ascertain the effectiveness of this technique, a retrieval experiment was conducted using 20,000 images of a collection, the Corel Photo Gallery, and the experiment was able to confirm an improvement in the retrieval efficiency.

We propose a method that detects vehicles from in-vehicle monocular camera images captured during nighttime driving. Detecting vehicles from their shape is difficult at night; however, many vehicle detection methods focusing on light have been proposed. We detect bright spots by appropriate binarization based on the characteristics of vehicle lights such as brightness and color. Also, as the detected bright spots include lights other than vehicles, we need to distinguish the vehicle lights from other bright spots. Therefore, the bright spots were distinguished using Random Forest, a multiclass classification machine-learning algorithm. The features of bright spots not associated with vehicles were effectively utilized in the vehicle detection in our proposed method. More precisely vehicle detection is performed by giving weights to the results of the Random Forest based on the features of vehicle bright spots and the features of bright spots not related to the vehicle. Our proposed method was applied to nighttime images and confirmed effectiveness.

Recently, Intelligent Transport Systems (ITS) are being researched and developed briskly. As a part of ITS, detecting vehicles from images taken by a camera loaded on a vehicle are conducted. From such backgrounds, authors have been conducting vehicle detection in nighttime. To evaluate the accuracy of this detection, gold standards of the detection are required. At present, gold standards are created manually, but manually detecting vehicles take time. Accordingly, a system which detects vehicles accurately without human help is needed to evaluate the accuracy of the vehicle detection in real time. Therefore the purpose of this study is to automatically detect vehicles in nighttime images, taken by an in-vehicle camera, with high accuracy in offline processing. To detect vehicles we focused on the brightness of the headlights and taillights, because it is difficult to detect vehicles from their shape in nighttime driving scenes. The method we propose uses Center Surround Extremas, called CenSurE for short, to detect blobs. CenSurE is a method that uses the difference in brightness between the lights and the surroundings. However, blobs obtained by CenSurE will also include objects other than headlights and taillights. For example, streetlights and delineators would be detected. To distinguish such blobs, they are tracked in inverse time and vehicles are detected using tags based on the characteristics of each object. Although every object appears from the same point in forward time process, each object appears from different places in images in inverse time processing, allowing it to track and tag blobs easily. To evaluate the effectiveness of this proposed method, experiment of detecting vehicles was conducted using nighttime driving scenes taken by a camera loaded on a vehicle. Experimental results of the proposed method were nearly equivalent to manual detection.

The present study proposes a method for estimation of subjective image quality, for combinations of display physical factors, based on the Mahalanobis-Taguchi system in the field of quality engineering. The proposed method estimates subjective image quality by the estimated equation based on the Mahalanobis-Taguchi System and subjective evaluation experiments using the method of successive categories for images of which parameters are combinations of gamma, maximum luminance and minimum luminance. The estimated image quality is in good agreement with the experimental subjective image quality.

The Helmholtz-Kohlrausch (H-K) effect is a phenomenon in which the perceived brightness levels induced by two stimuli are different even when two color stimuli have the same luminance and different chroma in a particular hue. This phenomenon appears on display devices, and the wider the gamut these devices have, the more the perceived brightness is affected by the H-K effect. The quantification of this effect can be expected to be useful for the development and evaluation of a wide range of display devices. However, quantification of the H-K effect would require considerable subjective evaluation experimentation, which would be a major burden. Therefore, the authors have derived perceived brightness maps for natural images using an estimation equation for the H-K effect without experimentation. The results of comparing and analyzing the calculated maps and ground truth maps obtained through subjective evaluation experiments confirm strong correlation coefficients between such maps overall. However, a tendency for the estimation of the calculation map to be poor on high chroma strongly influenced by the H-K effect was also confirmed. In this study, we propose an accuracy improvement method for the estimation of the H-K effect by correcting the calculation maps using a correction coefficient obtained by focusing on this tendency, and we confirm the effectiveness of our method.

The purpose of this study is to estimate the noise level of every pixel in a single noisy image, that is superimposed independent and non-identically distributed random variables with normal distribution. The method makes a set of similar pixels in the local region to the interest pixel using the approximate function of noise variance, and estimates with regard to the noise level. As the results show, the proposed method is effective in estimation of noise level of every pixel for any images.

Overall picture quality of a liquid crystal display (LCD) is changed by viewing conditions. An evaluation method of overall picture quality was proposed for LCDs. However, the estimation values calculated by the evaluation method differed from the subjective evaluation values, when viewing angle changed in the vertical direction. In this study, the coefficients of impairment in the evaluation equation were obtained by the multiple regression analysis using subjectively evaluated data. The evaluation method of overall picture quality was improved by weighting factors using these coefficients. As the result, the good estimation accuracy was obtained even if the viewing angle was changed both in the horizontal and vertical directions individually.

A set of three optical filters was designed and a camera system was developed using these filters in order to capture high-fidelity colors within the gamut of vision. Photographs of a number of highly saturated colors and a combination of the Macbeth chart and 18 pieces of clothing samples of various colors were taken. A 39 matrix was used to convert the camera output signals into XYZ tristimulus data. The tristimulus values of the colors were compared with those of the images captured by the camera. The average color difference, ΔE, for these samples were found to be 2.16 and 1.18, respectively.

We previously proposed a query-by-sketch image retrieval system that uses an edge relation histogram (ERH). However, it is difficult for this method to retrieve partial objects from an image, because the ERH is a feature of the entire image, not of each object. Therefore, we propose an object-extraction method that uses edge-based features in order to enable the query-by-sketch system to retrieve partial images. This method is applied to 20,000 images from the Corel Photo Gallery. We confirm that retrieval accuracy is improved by using the edge-based features for extracting objects, enabling the query-by-sketch system to retrieve partial images.

This study examines the robustness of image quality factors in various types of environment illumination using a parameter design in the field of quality engineering. Experimental results revealed that image quality factors are influenced by environment illuminations in the following order: minimum luminance, maximum luminance and gamma.

There has recently been much research on content-based image retrieval (CBIR) that uses image features including color, shape, and texture. In CBIR, feature extraction is important because the retrieval result depends on the image feature. Query-by-sketch image retrieval is one of CBIR and query-by-sketch image retrieval is efficient because users simply have to draw a sketch to retrieve the desired images. In this type of retrieval, selecting the optimum feature extraction method is important because the retrieval result depends on the image feature. We have developed a query-by-sketch image retrieval method that uses an edge relation histogram (ERH) as a global and local feature intended for binary line images. This histogram is based on the patterns of distribution of other line pixels centered on each line pixel that have been obtained by global and local processing. ERH, which is a shift- and scale-invariant feature, focuses on the relation among the edge pixels. It is fairly simple to describe rotation- and symmetry-invariant features, and query-by-sketch image retrieval using ERH makes it possible to perform retrievals that are not affected by position, size, rotation, or mirroring. We applied the proposed method to 20,000 images in the Corel Photo Gallery. Experimental results showed that it was an effective means of retrieving images.