The Wiri Project brings together a range of academic fields at the University: Psychology, Computer Science, Theatre Studies, and Design. In collaboration with academic and technical staff from Te Iho ki Motukairangi—Miramar Creative Centre, the researchers are using cutting-edge technology to capture a vast spectrum of real human emotional reactions. That data informs advanced machine-learning generative models and aims to bridge the gap between live actors and their digital counterparts.
Bringing together the distinctive approaches of their respective disciplines, and drawing on Mātauranga Māori, the Wiri Project researchers are aiming to develop a unique resource that sits at the intersection of indigenous knowledge, research science, and technology.
The Wiri Project is named after a distinctive element of Māori performance—the wiri is a quivering hand movement that represents the shimmering heat haze which rises from the earth after rainfall. In practice, the wiri is adopted to accompany songs, dance and rituals of celebration—denoting feelings of love, welcome, grief, anger, resistance, and despair. It’s an apt name for a project that centres Mātauranga Māori and explores the full range of emotions.
The struggle to create emotionally engaging characters in games
Principal investigator Areito Echevarria from the School of Design Innovation is an Academy Award-winning visual effects artist who researches motion capture and generative learning approaches to gaming design. He says interactive animation has always struggled to create characters that are physically plausible and emotionally engaging.
“Traditionally, in the context of video games, getting a believable performance out of a character is very complicated. How do you make this character behave in a believable way that you can empathise with?,” he says. “Anytime you want to add depth and complexity to an interactive character, the workload explodes, so we wanted to solve that problem.”
Associate Professor Hedwig Eisenbarth, Wiri Project principal investigator from the School of Psychological Sciences, says the project followed an earlier collaboration that focused on how people show empathy through body motion.
“With help from the University’s Research Office to develop the idea, we quickly realised we would need people with expertise in performance and computer science, to really advance the project,” says Dr Eisenbarth.
They were connected with Dr Nicola Hyland (Te Atihaunui-a-Pāpārangi, Ngāti Hauiti), a senior lecturer in theatre and performance from the School of English, Film, Theatre, Media Communication, and Art History, whose research focuses on Māori performance and theatre. They also linked up with Bastiaan Kleijn, a Professor of Machine Learning from the School of Engineering and Computer Science, and the Wiri Project was born.
Dr Eisenbarth says each of the disciplines involved in the research approaches the problem in different ways.
“We are taking a systematic approach to understanding how emotion is portrayed in the body, and also applying a te ao Māori perspective, which probably hasn’t been done before,” she explains. “The next step is to build generative models that can then create something new.”
Collecting data to inform emotion
Dr Hyland has developed a tool, Te Kaupae Whitu o Whakatinanatanga—the Seven Levels of Embodiment, which the team calls 7LE. Each kaupae, or level, describes a different range of physicality from low energy to full tension. Each is drawn from the movement qualities of different creatures in te ao Māori, such as the pūpūharakeke, or flax snail (level 1) or the taiaroa, or albatross (level 3). Using these levels as a basis for the physical movement of the performers, the team then layers specific rongo (emotions) on top: riri (anger), harikoa (happiness), pōuri (sadness), uruwehi (fear), and whakarihariha (disgust), which showcases how distinct movements can influence the expression of a certain emotion.
Dr Hyland says the 7LE tool enables the research team to derive a holistic digital expression of the emotional human performances.
“The ruru [morepork], for example, represents level seven of 7LE, which is Kaumingomingo—chaos. It’s the final level but it doesn’t look as intense as level 6 because it comes after the big physical emotions, at a point where you’re stricken but also incredibly tense,” she explains. “As a tohu [symbol], the ruru is a predator—it barely moves but when it does, it’s very intense. Translating that into a character’s body gives them the sense of holding something very deep inside that could come out at any point.”
Using wearable technology, data was collected from the actors that captured their movements, voice, facial expressions, skin temperature, heart rate, and self-reported emotional state. That information is then matched to the particular emotional field and level of embodiment they are acting. The computational model learns the association between the labelled emotions and the physical data and can then use that information to closely recreate a specific movement in a digital character that replicates a human response. The tool the team is working on will then be able to generate new body motion based on its underlying model.
A rich resource that comes from a special collaboration
The Wiri Project had funding from the Ministry of Business, Innovation and Employment, which ended in 2025. The artificial intelligence model is now being used in research by students from the School of Design Innovation, who are working to create new games using this technology, but there are currently no plans to commercialise the project.
“What we’re developing is really just a rich resource. It’s currently at proof-of-concept stage, but the potential is that the gaming industry will pick it up and carry on developing it,” says Dr Eisenbarth.
She says she has learned a lot from the collaborative approach. “We each bring our different lenses to the project, and when that is layered with te ao Māori we get a really unique way of looking at emotions,” she says.
“Building this collaboration across disciplinary boundaries and bringing together different perspectives to find a common shared language has been fun and exciting.”
The team says the Wiri Project will contribute to elevating the quality and productivity of Aotearoa’s interactive media sector, such as games and animation, by providing cutting-edge technological innovation.
“This particular group of people and this particular interaction of disciplines is unique to this university, and it would be a very difficult combination of areas of expertise to assemble any other way. It would be basically impossible in industry to get computer scientists, psychologists, designers and theatre practitioners working together in the same team,” says Areito.
“That’s one of the special, unique things about doing this kind of research in the university context—we can build these interdisciplinary teams from seemingly quite diverse backgrounds. It’s where the magic happens.”
Watch the video online or below:
Video credits:
Creative lead, Ted Whitaker
Crew, Hans Weston
Editor, Michelle Cameron
Te Herenga Waka—Victoria University of Wellington Image Services