CMIC student and staff present their paper and demo "Real-Time Mixed Reality Rendering for Underwater 360° Videos” at ISMAR 2019

CMIC present the Underwater Toolkit, a mixed reality (MR) toolkit that enables seamless blending of virtual objects into underwater 360° videos (360-video) in real time.

CMIC student Stephen Thompson and CMIC post-doc researcher Daniel Medeiros had the opportunity to showcase their latest VR work at the ISMAR 2019 conference in China.

Real-time mixed reality underwater rendering into a 360° video. The top left image is rendered without our method, and the top right image is rendered with our underwater lighting method applied. The bottom row shows each of the underwater lighting effects individually.

Stephen presented the Underwater Toolkit at the paper program. The toolkit provides real-time underwater lighting (caustics, god rays, fog, and particulates) to ensure that the virtual objects are lit and blend similarly to each frame of the underwater video semiautomatically and in real time. The toolkit’s user-friendly interface enables users to fine-tune the underwater lighting parameters so they can match the lighting observed in the 360-video for improved visual quality and seamless blending. In the demonstration program, Stephen and Daniel ran a demo which users was able to immerse themselves into underwater 360-videos using a HMD. Using motion controllers, the users was able to interact with fish by feeding or catching them. An additional user was able to interact with our toolkit, changing underwater lighting parameters to seamlessly blend fish into the 360-video in real time. For details, please check the project page.


ISMAR 2019, 18th IEEE International Symposium on Mixed and Augmented Reality was held from October 14th - 18th, 2019 at the Beijing, China.