Working with CMIC industry partners
OLM Digital (Anime/3DCG/VFX)
SQUARE ENIX (game)
IMAGICA GROUP (media solution)
Future studio solution
Develop advanced solutions for production studio using computer graphics, simulation, and applied mathematics.
Mesh saliency / Mrs Saliency (MRSS)
The latest developments in rendering, capture, and fabrication technology have made it possible to create highly detailed three-dimensional meshes for use not only in offline media such as films and animations, but also in interactive multimedia including games and other reactive content, such as extended reality.
While highly complex meshes have become a staple in production, their use entails significant bottlenecks. In effect, loading and manually editing models with millions of vertices in interactive software require significant memory and computing time. Moreover, using these models in simulation and real-time rendering is not feasible, even with modern hardware.
Especially in real-time applications, you want to represent the target with the minimum required polygons. To meet this need, a technology called polygon reduction has been researched and developed for a long time. Although it is extremely difficult to significantly reduce the number of polygons without impairing the characteristics of the original shape, recently, there is an increasing need to significantly reduce the total number of polygons while maintaining the quality as close as possible to the original data.
CMIC team aim to develop a novel, efficient multi-scale method to compute mesh saliency at both coarse and finer scales. Our original tool "Mrs. Saliency (MRSS) achieves speedups of up to three orders of magnitude over prior approaches.
Detailed facial animation
CMIC industry partners from leading film and game studios report significant bottlenecks in realistic animation production. While performance capture technology offers a potential solution, the processing and editing of the recorded data are often too expensive for broad application.
We aim to develop novel software that automatically layers fine-scale details over early-stage animation. To realise this software, we will undertake applied research that untangles the complexity of realistic facial movements, such as subtle movements of the eyebrows and slight movements of the lips, realistic human-like movement the person is not even aware of.
Illumination browser: a library for radiance maps
Radiance maps (RM) are the lighting properties captured from real-world environments. Databases of RMs are useful for various rendering applications such as look development, live-action compositing, mixed reality, and machine learning. Traditional RM databases are not helpful if not organized in a meaningful way.
To address this, we introduce the illumination space, a feature space that arranges RM databases based on illumination properties. We avoid manual labelling by automatically extracting features from a RM that provides a concise and semantically meaningful representation of its typical lighting effects. This is made possible with the following novel contributions: a method to automatically extract a small set of dominant and ambient lighting properties from RMs, and a low-dimensional (5D) light feature vector summarising these properties to form the illumination space. Our method is motivated by how the RM illuminates the scene as opposed to describing the textural details of the RM image.
Directable cartoon lighting
Cel animation remains a prevalent form of artistic expression. The freedom offered by manual drawing continues to attract content creators to develop hand-drawn comic and cartoon style characters for feature films and interactive media such as games.
Using digital tools, artists create 3D models from character artwork for automated rendering from any viewpoint. However, many manual appearance edits must be made to keep 3D stylized rendering faithful to hand-drawn media.
One of the key challenges of 3D cartoon media is controlling the light and shadow areas generated by conventional toon shading. Even in realistic rendering styles, or interactive stylized media and games, artists require fine-grained, non-physical, shadow shape control.
We developed the Shading Rig, a framework for designing and animating dynamic art-directed 3D toon shading. We design a novel parametric model for generating shading edits, that allows for continuous, real-time, art-directed shadow animation.