Virtual Worlds Lab

At the School of Design Innovation’s Virtual Worlds Lab, staff and students explore ways to create interactive digital experiences using emerging technologies.

Lab overview

New developments with virtual reality (VR) technology highlight a vast territory for immersive storytelling, gaming, and many other forms of interactive experiences such as simulated education, medical therapy, and scientific visualisation. While Sony, Samsung, and Oculus Rift (Facebook) have delivered the hardware, start-ups and research labs are playing catch-up with the development of content. Local Wellington company 8i for example, has received over 20 million dollars in seed funding, and Weta Digital is also investing in this new domain.

At the lab, Design staff and students research new methods for storytelling, interaction, and immersive experience, presented via emerging technologies like head-mounted displays.

Cinematic and theatrical narrative

How do directors tell design-driven stories in this medium? Which cinematic tropes work for VR? What are the formal strategies for transitioning through time in a cinematic VR? What are successful techniques for transitioning locations?

This research group will develop projects demonstrating how narrative story telling can work with a VR representative. Content will include both photographic and computer generated videos. The Virtual Worlds Lab (VWL) seeks to engage with writers, dancers, theatre performers, and other film practitioners to craft tomorrow’s entertainment experiences.

Immersive audio

What techniques can be used to generate the soundscapes presented in a VR experience? VWL explores and demonstrates how environmental sound can be captured and presented in a way that locates audio in space in a plausible way for use in VR. Students in this research stream work with audio engineers and musicians to create next-generation music video experiences, utilising live performance and computer generated material.

Gaming

Which aspects of narrativity and ludology—discourses that have already happened in the gaming community—can be applied towards one now taking place in VR? What new experiences can be crafted through VR gaming affordances (including games for training and education)? Perhaps more so than any other medium, games have the potential to point the way forward in the development of VR experiences. Staff and students experiment with different aspects of immersion (first-person perspective, head tracking, etc.) through VR systems.

Lab research aims and objectives

Aims

The primary aim of this VWL is to produce new forms of design content through VR technology utilizing the plethora of new display and interaction systems coming onto market.

Objectives

• Completion of MDI degrees with high grades and marks of distinction
• Posters and conference papers published in high quality conferences such as SIGGRAPH, EuroVR and IEEEVR
• Exhibitions and performances at internationally recognized venues and performance spaces for art, design and science: SIGGRAPH, AWE, FILE and SMTE
• VR designs that support journal publications such as Leonardo.

Lab postgraduate learning objectives

1. Have strong technical knowledge in design-related VR/VFX studio practices
2. Be able to communicate effectively, visually and verbally, to the design concepts and techniques in VR/VFX disciplines
3. Have excellent creative and critical thinking skills in relation to building a personal design strategy
4. Have good group/team work skills and be a valued member of a studio environment
5. Be able to undertake responsibility, act ethically and demonstrate accountability in relation to Te Tiriti when relating design to other social and cultural concerns
6. Have the professional and enterprise skills necessary for working effectively in professional organisations.

Lab projects

Cinema of realities

The lack of diversity in movies and declining theater attendance has brought into question the viability of cinema as a relevant art form. If the experience of sitting in a theater no longer captures our imagination, perhaps it is because cinema is a medium based on outdated myths. And new myths, which we are still in the process of forming, are pointing away from the collective experience, towards something centered on a singular consciousness.

As the next generation of virtual reality tools become increasingly accessible, VR is poised as the storytelling medium of choice for both filmmakers and audiences wanting to embrace a more embodied and user-driven way of telling stories. But what is the relationship between the two mediums? If movies have taught us how to tell designer-led stories, can VR teach movies how to acknowledge the agency of audiences seeking a more individually driven experience of narrativity?

In order to trace the rapidly changing formal language of cinema, this project examines a series of recent feature films that have prioritised formal strategies such as first-person point-of-view, the long take, and a re-examination of the gaze. Such a deconstruction will help us better understand the tools required to build a new language for storytelling in virtual reality. Research outcomes aim for publication in academic journals.

Staff

• Raqi Syed

Circle versus square: VR as the third experience

Much of the technology behind filmmaking has changed in the last 150 years, but the formal language of cinema and its predecessor, theater, have come to be defined by the shape of the rectangle. On the other hand, the formal language of Virtual Reality, while still fluid, is developing largely around the sphere.

For VR, this opens up a series of opportunities and challenges. Can the experience of VR be built around our present understanding of cinema and theater? And how do we create the artifice most fictional storytelling requires, while still concealing the apparatus needed to construct it in a spherical environment?

Presently, the tools required to erase this apparatus are experimental and expensive. Light field technology for example, is very promising and may well be the post-production tool of choice to eliminate crew, lights, and other rigging materials from the virtual space. This project aims to explore a different way of utilizing the spherical virtual space—one in which the apparatus of production is preserved and exploited even—as part of the storytelling process.

Is it possible audience understanding of narrativity has progressed far enough to allow for this kind of modern deconstruction? By imagining the virtual sphere as both a theatrical stage and a post-production studio in which the magic is exposed, we hope to gain a deeper understanding of how narrativity and real-time user based experiences function in Virtual Reality.

This project aims to utilize 3D printing technologies, 360 camera rigs, 3D soundscapes, and live performance to produce a VR short for the purpose of exhibition.

Staff

• Raqi Syed
• Kevin Romond