This paper focuses on the potential value and future prospects of using virtual reality (VR) technology in online education. In detailing online education and the latest VR technology, we focus on metaverse construction and artificial intelligence (AI) for educational VR use. In particular, we describe a virtual university campus in which on-demand VR lectures are conducted in virtual lecture halls, automated evaluations of student learning and training using machine learning, and the linking of multiple digital campuses.

Privacy violations via spy cameras are becoming increasingly serious. With the recent advent of various smart home IoT devices, such as smart TVs and robot vacuum cleaners, spycam attacks that steal users' information are being carried out in more unpredictable ways. In this paper, we introduce a new spycam attack on a mobile WebVR environment. It is performed by a web attacker who maliciously accesses the back-facing cameras of victims' mobile devices while they are browsing the attacker's WebVR site. This has the power to allow the attacker to capture victims' surroundings even at the desired field of view through sophisticated content placement in VR scenes, resulting in serious privacy breaches for mobile VR users. In this letter, we introduce a new threat facing mobile VR and show that it practically works with major browsers in a stealthy manner.

The progress of immersive technology enables researchers and developers to construct work spaces that are freed from real-world constraints. This has motivated us to investigate the role of the human body. In this research, we examine human cognitive behaviors in obtaining an understanding of the width of their virtual body through simple yet meaningful experiments using virtual reality (VR). In the experiments, participants were modeled as an invisible board, and a spherical object was thrown at the participants to provide information for exploring the width of their invisible body. Audio and visual feedback were provided when the object came into contact with the board (body). We first explored how precisely the participants perceived the virtual body width. Next, we examined how the body perception was generated and changed as the trial proceeded when the participants tried to move right or left actively for the avoidance of collision with approaching objects. The results of the experiments indicated that the participants could become successful in avoiding collision within a limited number of trials (14 at most) under the experimental conditions. It was also found that they postponed deciding how much they should move at the beginning and then started taking evasive action earlier as they become aware of the virtual body.

The aim of this study is to calculate optimal walking routes in real space for users partaking in immersive virtual reality (VR) games without compromising their immersion. To this end, we propose a navigation system to automatically determine the route to be taken by a VR user to avoid collisions with surrounding obstacles. The proposed method is evaluated by simulating a real environment. It is verified to be capable of calculating and displaying walking routes to safely guide users to their destinations without compromising their VR immersion. In addition, while walking in real space while experiencing VR content, users can choose between 6-DoF (six degrees of freedom) and 3-DoF (three degrees of freedom). However, we expect users to prefer 3-DoF conditions, as they tend to walk longer while using VR content. In dynamic situations, when two pedestrians are added to a designated computer-generated real environment, it is necessary to calculate the walking route using moving body prediction and display the moving body in virtual space to preserve immersion.

360-degree video delivery in Virtual Reality is very challenging due to the fact that 360-degree videos require much higher bandwidth than conventional videos. To overcome this problem, viewport-adaptive streaming has been introduced. In this study, we propose a new adaptation method for tiling-based viewport-adaptive streaming of 360-degree videos. For content preparation, the Cubemap projection format is used, where faces or parts of a face are encoded as tiles. Also, the problem is formulated as an optimization problem, in which each visible tile is weighted based on how that tile overlaps with the viewport. To solve the problem, an approximation algorithm is proposed in this study. An evaluation of the proposed method and reference methods is carried out under different tiling schemes and bandwidths. Experiments show that the Cubemap format with tiling provides a lot of benefits in terms of storage, viewport quality across different viewing directions and bandwidths, and tolerance to prediction errors.

A new hybrid brain-computer interface (BCI), which is based on sequential controls by eye tracking and steady-state visual evoked potentials (SSVEPs), has been proposed for high-speed spelling in virtual reality (VR) with a 40-target virtual keyboard. During target selection, gaze point was first detected by an eye-tracking accessory. A 4-target block was then selected for further target selection by a 4-class SSVEP BCI. The system can type at a speed of 1.25 character/sec in a cue-guided target selection task. Online experiments on three subjects achieved an averaged information transfer rate (ITR) of 360.7 bits/min.

In this letter, we propose an acoustic distance rendering (ADR) algorithm that can efficiently create the proximity effect in virtual reality (VR) systems. By observing the variation of acoustic cues caused by the movement of the sound source in the near field, we develop a model that can closely approximates the near-field transfer function (NFTF). The developed model is used to efficiently compensate for the near-field effect on the head related transfer function (HRTF). The proposed algorithm is implemented and tested in the form of an audio plugin for a VR platform and the test results confirm the efficiency of the proposed algorithm.

The quality of visual comfort and depth perception is a crucial requirement for virtual reality (VR) applications. This paper investigates major causes of visual discomfort and proposes a novel virtual camera controlling method using visual saliency to minimize visual discomfort. We extract the saliency of each scene and properly adjust the convergence plane to preserve realistic 3D effects. We also evaluate the effectiveness of our method on free-form architecture models. The results indicate that the proposed saliency-guided camera control is more comfortable than typical camera control and gives more realistic depth perception.

Mass-market head mounted displays (HMDs) are currently attracting a wide interest from consumers because they allow immersive virtual reality (VR) experiences at an affordable cost. Flying over a virtual environment is a common application of HMD. However, conventional keyboard- or mouse-based interfaces decrease the level of immersion. From this motivation, we design three types of immersive gesture interfaces (bird, superman, and hand) for the flyover navigation. A Kinect depth camera is used to recognize each gesture by extracting and analyzing user's body skeletons. We evaluate the usability of each interface through a user study. As a result, we analyze the advantages and disadvantages of each interface, and demonstrate that our gesture interfaces are preferable for obtaining a high level of immersion and fun in an HMD based VR environment.

A low-cost prototypic Kinect-based rehabilitation system was developed for recovering balance capability of stroke patients. A total of 16 stroke patients were recruited to participate in the study. After excluding 3 patients who failed to finish all of the rehabilitation sessions, only the data of 13 patients were analyzed. The results exhibited a significant effect in recovering balance function of the patients after 3 weeks of balance training. Additionally, the questionnaire survey revealed that the designed system was perceived as effective and easy in operation.

An earthquake is a destructive natural disaster, which cannot be predicted accurately and causes devastating damage and losses. In fact, many of the damages can be prevented if people know what to do during and after earthquakes. Earthquake education is the most important method to raise public awareness and mitigate the damage caused by earthquakes. Generally, earthquake education consists of conducting traditional earthquake drills in schools or communities and experiencing an earthquake through the use of an earthquake simulator. However, these approaches are unrealistic or expensive to apply, especially in underdeveloped areas where earthquakes occur frequently. In this paper, an earthquake drill simulation system based on virtual reality (VR) technology is proposed. A User is immersed in a 3D virtual earthquake environment through a head mounted display and is able to control the avatar in a virtual scene via Kinect to respond to the simulated earthquake environment generated by SIGVerse, a simulation platform. It is a cost effective solution and is easy to deploy. The design and implementation of this VR system is proposed and a dormitory earthquake simulation is conducted. Results show that powerful earthquakes can be simulated successfully and the VR technology can be applied in the earthquake drills.

Due to ease of implementation for various user interactive applications, much research on motion recognition has been completed using Kinect. However, one drawback of Kinect is that the skeletal information obtained is provided under the assumption that the user faces Kinect. Thus, the skeletal information is likely incorrect when the user turns his back to Kinect, which may lead to difficulty in motion recognition from the application. In this paper, we implement a highly accurate human motion capture system by installing six Kinect sensors over 360 degrees. The proposed method enables skeleton to be obtained more accurately by assigning higher weights to skeletons captured by Kinect in which the user faces forward. Toward this goal, the front vector of the user is temporally traced to determine whether the user is facing Kinect. Then, more reliable joint information is utilized to construct a skeletal representation of each user.

We present a linear-time algorithm for treating collision response of articulated rigid bodies in physically based modeling. By utilizing the topology of articulated rigid bodies and the property of linear equations, our method can solve in linear time the system of linear equations that is crucial for treating collision response.

In this letter, we propose a enhanced framework for a Personalized User Interface (PUI). This framework allows users to access and customize virtual objects in virtual environments in the sense of sharing user centric context with virtual objects. The proposed framework is enhanced by integrating a unified context-aware application for virtual environments (vr-UCAM 1.5) into virtual objects in the PUI framework. It allows a virtual object to receive context from both real and virtual environments, to decide responses based on context and if-then rules, and to communicate with other objects individually. To demonstrate the effectiveness of the proposed framework, we applied it to a virtual heritage system. Experimental results show that we enhance the accessibility and the customizability of virtual objects through the PUI. The proposed framework is expected to play an important role in VR applications such as education, entertainment, and storytelling.

The "Proactive Desk" is a new human-machine interface for the desktop operations of computers. It provides users with tactile sensation in addition to visual sensation. Two linear induction motors underneath the desk generate a two-dimensional force to move objects and control their positions on the desktop using feedback control, and users feel tactile sensation while handling those objects. In this paper, we examined the effects of adding haptic information to simple mouse operation using the Proactive Desk. In our experiment, we used a button-type visual stimulus with and without haptic information. When using haptic conditions, three types of force feedback pattern were displayed: "Edge," "Resistance to motion" and "Attractive force," and each had three force strength conditions: no, half and full. The subject was asked to push buttons twenty times as the buttons were shown one after the other on the desk as quickly as possible. Consequently, the reaction times for pushing the button for all haptic conditions, except for the half-force condition of "Attractive force," were significantly faster than no-force (without haptic information) condition. This result shows that the haptic information was advantageous for easy operation.

In this paper, we propose a framework for virtual reality, I2-NEXT, which enables users to interact with virtual objects by tangible objects in immersive networked virtual environment. The primary goal of this framework is to support rapid development of immersive and interactive virtual reality systems as well as various types of user interfaces. The proposed framework consists of user interface for interactions, immersive virtual environment, and networking interface. In this framework, we adopt several design patterns to guarantee that either developers or users (artists) can easily implement their VR applications without strong knowledge of VR techniques such as programming, libraries etc. One of the key features of this framework is the presence of the device module which supports a natural user interaction in a virtual environment. For example, the proposed framework provides users with tangible objects so that the users are able to manipulate virtual objects by touching real objects. The proposed framework also supports large scale stereoscopic display through clustering technique. To realize the effectiveness of the proposed framework, we have been developing an application for digital heritage reconstruction. Having been through development of the system, we believe that virtual reality technology is one of the promising technologies which enable users to experience realities in a digital space. Detailed explanations of each component and system architecture are presented.

We propose a shape resolution control method applying a tolerance caused by movement to object's shape and texture in order to represent efficiently a textured object that has a detailed structure. It is generally difficult to perceive the error of shape or texture of the object that is moving. Our method applies this error as a tolerance. The efficient object's representation is realized by the shape resolution control that tolerates errors of contour shape and textured surface by the tolerance caused by movement and reduces object's data. It was shown better experimental results of processing time and of the quality of images in comparison with other methods. Thus, it was proved that the method applying the tolerance caused by movement to the object's shape and texture is effective in the representation of textured object that has a detailed structure.

An interactive 3-D sound processing system and its implementation is described, which is to provide virtual auditory environments to listeners. While conventional 3-D sound processing systems require high performance workstations or large DSP arrays, the proposed system is reduced in hardware size for practical applications. The proposed system is implemented using a prevailing IBM-compatible PC and a single DSP. Since the organization of the proposed system is independent of implementation details such as operation precision and number of audio tracks, the proposed system can be ported to various hardware entities. In addition, an easy-to-use user interface is also implemented on PC software for realtime input of 3-D sound movement. Owing to these features, the presented system is valuable as a prototype for various implementation of 3-D sound processing systems, while the current implementation is useful as a 3-D sound content production system.

This paper presents a new model which computes the deformation and the feedback force of high-resolution biomedical volumetric objects consisting of hundreds of thousands of volume elements. The main difficulty in the simulation of these high-resolution volumetric objects is to compute and generate stable feedback force from the objects within a haptic update time (1 msec). In our model, springs are used in order to represent material properties of volume elements and cylinders are used to activate corresponding springs according to the amount of deformation. Unlike in a mass-spring model, springs in our model have constraint conditions. In our model, the deformation is calculated locally and then is propagated outward through object's volume as if a chain is pulled or pushed. The deformed configuration is then used to compute the object's internal potential energy that is reflected to the user. The simple nature of our model allows the much faster calculation of the deformation and the feedback force from the volumetric deformable object than the conventional model (an FEM or a mass-spring model). Experiments are conducted with homogenous and non-homogenous volumetric cubic objects and a volumetric human liver model obtained from CT data at a haptic update rate of 1000 Hz and a graphic update rate of 100 Hz to show that our model can be utilized in the real-time volume haptic rendering. We verify that our model provides a realistic haptic feeling for the user in real time through comparative study.

It is important to look for alternative forms of physical movement of people and equipments in order to assure diagnosis and maintenance tasks, especially in an environment where workers are subject to danger. An evident and classical solution is the use of the tele-operation and tele-robotics. If the tele-operation helps to solve a lot of real and technical problems, it still remains insufficient to assure an appropriate remote diagnosis and maintenance. The use of virtual reality techniques with the tele-operation can be the solution for an effective remote maintenance and diagnosis. In this paper we show the inefficiency occurred with the use of only tele-operation in the remote maintenance, we introduce our original new system where we use virtual reality techniques and 2D-3D matching (2D camera image-3D virtual objects) with tele-operation to remotely collect machinery vibration data. We explain its structure, implementation and its advantages. We finished by experimenting the system, measuring the different operating times and precision and discuss the results.