In January 2020, the spread of the coronavirus meant that theaters were no longer able to offer in-person viewing experiences. Noh, as an art form, is only fully realized when performed on the Noh stage, so simply streaming in 2D could risk losing the true essence of the performance. In response, we developed a 180-degree multi-angle streaming system, creating a unique digital experience by enabling "free viewpoint" viewing.
The actual filming was conducted with high-resolution cameras equipped with 180-degree lenses, capturing three angles simultaneously. The audio was also recorded in a spatial format, designed so that the sound differs according to each viewpoint, in an effort to replicate the live experience as faithfully as possible.
While 360-degree video captured the experience, we also utilized Matterport to 3D scan the entire Noh theater, offering a more immersive experience. Although audience members cannot physically step onto the Noh stage, this digital approach enables them to explore even the backstage areas of the stage.
Using photogrammetry, we created a viewing experience where masks used in performances, such as Ko-omote and Hannya, can be examined as if held in one’s own hands. This technique generated 3D data from photos taken from 360 degrees, faithfully reproducing the detailed texture and even the rarely seen reverse side of the masks, typically viewed only by Noh performers.
