С��Ƭ��Ƶ

In the spring of 2022, Aviva Avnisan, a former Emerging Media and Technology faculty member taught the class Children’s Lit in Augmented Reality. For their projects, students used children’s picturebooks to inspire designs for an original augmented reality storytelling platform. The course was co-scheduled with Design Innovation and students’ work was showcased in the DI Hub’s Blank_Lab. 

Executive Director of С��Ƭ��Ƶ’s Design Innovation Initiative, J.R. Campbell was intrigued by Aviva’s work in immersive installation as well as the hologramatic capture work of another faculty member, Karl Kosko, Ph.D., of the College of Education, Health and Human Services. Kosko brought the technology company Depthkit to Campbell’s attention.

Depthkit uses volumetric capture to record video of a subject from multiple angles. The software then combines this footage to create a realistic 3D volumetric video that can be refined, shaped and moved into rendering software such as Unity or Blender.

Campbell recognized the power of the tech right away. “It’s one of the most powerful setups for being able to properly capture full 3D videos, holograms essentially,” he said. He got to work and convened a team of 24 collaborators to work on a grant proposal for the State of Ohio’s Rapids Grant program, which eventually brought Depthkit to С��Ƭ��Ƶ.

Fast-forward to Fall 2024 Semester, when Associate Professor Jeffrey Rockland of the School of Theatre and Dance, wanted to explore ideas for virtual, immersive dance pieces. Inspired by MASH Jerusalem Dance House’s work in creating virtual dance recitals during the COVID quarantine years, Rockland wanted to do something similar but different. Campbell introduced Rockland to the director of the School of Emerging Media and Technology, Michael Beam.

The resulting collaboration came together the school’s Interdisciplinary Projects class. The course sets collaborative, interdisciplinary teams of students to work providing an innovative solution to a real-world problem using technology.

Learn more about the project and in this video.

The team of students Kyle Bartlett, Collin Lewis, Kaitlyn Perkovic and project leader John Comi, spent the first part of the semester learning the intricacies of the software and the video camera setup. At the same time, working in collaboration with Rockland, they wrote and storyboarded the content for the final product, taking note of the music cues to create something that was artistically competent as well as technologically impressive.

Using 10 Femto Bolt capture devices, they utilized the space created in the DI Hub’s first floor “XR Collaboratory” to capture video of 11 dancers from Professor Rockland’s class. The cameras were daisy-chained together in a circle and spoke to each other through the software, registering their location and calibrating based on the location of the other nine cameras.

Taken together these 10 capture devices create a 3D holographic video of whatever is inside a set space that they surround. Managing a complicated schedule within a tight time frame, the team scheduled dancers from Professor Rockland's class to visit the DI Hub and dance inside the Depthkit space. From there, the team transferred the volumetric capture into the Blender software and then set about refining the video images they had. 

The results were used by Professor Rockland and his class in the staging of their recital at the end of Fall 2024 Semester in a piece called “Oy, Mi Gente” that combined live-action dancing with computer-generated sequences of the same dancers, dancing on rooftops, climbing walls and being spun around on the planet. 

"This is a technology with great potential," said Associate Professor Enrico Gandolfi, who teaches the Interdisciplinary Projects course. "Endless potential. Education, professional development in any field, our ability to teach and learn online will be more immersive. We can absolutely use this technology in healthcare for public health or mental health. Video games, journalism, movies – the new Blade Runner movie used many volumetric videos to make its aesthetics way more immersive. This is the future of extended realities."

Project leader John Comi agrees. “What traditional photography did for communication and information, volumetric video like Depthkit could very well transform the world even more radically. This team's ability to not only develop with and understand this tool but work with it in such a creative context, is something I am immensely proud of.

"I cannot wait to see where this technology will take us next.”