Realistic scene reconstruction under different depths through light field imaging for virtual reality

Virtual reality (VR) immerses users in digital environments, transforming the entertainment, education, healthcare, and engineering industries. VR content has traditionally been created using methods like computer-generated imagery and conventional imaging techniques such as panoramic and omnidirectional imaging. However, computer-generated images often lack realism, and conventional imaging captures only 2D spatial information, limiting the immersion and overall potential of VR experiences.

Advanced technologies like light field (LF) imaging address these limitations by capturing both 2D spatial and 2D angular information of light rays in the form of 4D LF images. This study conducts a comprehensive review that explores various VR content creation methods, categorised into traditional and LF imaging-based approaches. The review highlights the advantages and disadvantages of each technique. It discusses the challenges of creating immersive and realistic LF imaging-based VR content, such as image size, processing speed, precise calibration, and depth reconstruction. Additionally, it identifies a gap in current LF-based VR content concerning the lack of depth perception, an essential feature for providing an immersive and comfortable user experience.

Building upon the findings of the literature review, this study leverages LF imaging to create realistic VR content. A depth reconstruction model derived from LF imaging is proposed to reconstruct realistic objects at different depths. The model begins by applying aberration correction techniques to the LF images, which are subsequently used to extract sub-aperture images for depth reconstruction but also delivers a lifelike experience. Furthermore, the proposed depth reconstruction model reduces the size of LF images, thereby reducing processing time. These depth-reconstructed images are integrated into the VR content creation process through a translation mechanism, enabling the refocusing feature and enhanced depth perception.

To ensure the generation of better-quality depth images, a comprehensive calibration procedure is conducted for both the LF imaging system and the depth reconstruction model. The calibration aim is to ensure the creation of precise, better-quality depth images within the specified range. Experiments were conducted on realistic objects in various settings to validate the proposed model's performance under realistic scenarios. It demonstrates the model's effectiveness in different settings. Furthermore, a comparative analysis with other depth reconstruction methods indicates that the proposed model outperforms existing techniques.

A user feedback study was conducted to assess the user experience, particularly focusing on immersion and motion sickness. Three questionnaires were employed for this assessment: the User Experience Questionnaire to assess general experience, the iGroup Presence Questionnaire for immersion, and the Simulator Sickness Questionnaire to quantify motion sickness. It indicates that the integration of realistic objects and the refocusing feature in the VR environment enhanced realism and depth perception, resulting in a more immersive and comfortable user experience compared to the computer-generated VR content. Finally, this study shows improvement in immersive VR content creation by integrating LF imaging along with interaction capabilities and enhancing the depth perception. By overcoming traditional limitations, it contributes to reducing motion sickness and improving overall user comfort and experience. The proposed methods and findings have implications for academic research and real-world applications in entertainment, education, healthcare, and assistive technologies.