By Jen Knights

A team of University of Iowa professors, students, staff, and alumni recently presented at the world’s largest trade show event for live entertainment design and technology.  

The LDI (Live Design International) conference was held in November 2022 in Las Vegas. The team presented their creation Media Clown, an evolving, multi-year, multimedia production and research project that utilizes augmented reality and virtual greenscreen for live broadcast to break the fourth wall between the physical and digital worlds. 

Media Clown co-creator Department of Theatre Arts Associate Professor Daniel Fine, who has a joint appointment in the Department of Dance and specializes in digital media in performance, says, “It started off as clowning using technology, but then also became a story that is clowning technology, like poking fun at it. It tells the story of an oddball tech entrepreneur, Perceval, who has created the YuVerse, a digital platform where you can upload your ‘digital essence’ and develop it to become a better person.” 

Fine and co-creator Theatre Associate Professor Paul Kalina, both of the UI College of Liberal Arts and Sciences, began conceptualizing the project in 2016, and it has since morphed and grown as they’ve added collaborators, experimented with new technology, and responded to the changing technological world. 

“Our concept started before the Metaverse was a thing. The idea—our sense of the need for commentary on technology and society—was already in the creative ether and lots of us were making sense of it in different ways,” Kalina says.

When the COVID pandemic came along and thrust everyone into a more digital existence, the Media Clown team was uniquely prepared to respond. 

“We were ready for it,” Fine said. “It really didn’t feel like a big pivot for us. It felt natural, like ‘Oh, this is perfect; this is already the story we’re telling.’ We couldn’t go out in real life, but in the world of Media Clown we imagined our avatars could go out into the YuVerse and interact and not get COVID."

MEDIA CLOWN GOES TO VEGAS—AND STAYS HOME 

Typically, at LDI, Fine says, speakers give a show-and-tell presentation of a past production or performance that they’ve worked on. Media Clown unfolded live, with LDI’s Vegas conference attendees having the opportunity to see both the audience view—what was happening on Zoom—while also getting a simultaneous live feed view of the behind-the-scenes work happening in the studio. 

Kalina, who is Head of Acting in the UI Department of Theatre Arts, performed the parts of Perceval and multiple versions of his digital avatar, de.Z, live in the UI motion-capture studio, surrounded by a technical team of students and professional staff operating various elements of the production on laptops, iPads, and sound boards using tech tools like UnReal Engine, Touch Designer, Zoom, Zoom OSC, Qlab, Lens Studio, and motion capture via Optitrack integration. Meanwhile, Zoom audience members logged on from all over the world and interacted with the show in real time, whether as participants in an imaginary game show or as bettors who could help determine the game’s results via a custom-built web app.  

Fine and other members of the team traveled to Las Vegas to present the show in real time at the conference and host a question and answer session with the team, which included additional collaborators zooming in from other locations. 
 

RESEARCH COLLABORATIONS 

Media Clown’s interdisciplinary research team is made of collaborators from theatre, art, engineering, and computer science, along with industry professionals. The purpose of the ongoing project is to integrate the newest digital technologies, specifically motion capture, Augmented Reality, virtual production, live broadcast, and real-time systems, with traditional/theatrical clown techniques, exploring the relevance of an analog clown in the digital age and the nature of codependency between live, physical performance and live digital media systems. 

“This is our research. Creating new work and exploring technology is pretty freaking complicated, but it’s what we’re about,” Kalina explains. “A lot of pieces of software have to talk to each other and we’re pushing things in new directions. Every time we imagine a new twist, we have to devise a way and work with our tech collaborators to create it on the stage or screen.”

“One great thing about being embedded in a major research university is that we have access to amazing technology and we have scientists and engineers that want to get creative with us. It’s great for our students because they gain hands-on experience here working with the latest technology, and then they go into the industry with professional skills, whether they are studying stage management, design, acting, or directing,” he added. 

STUDENT EXPERIENCE 

This project offers current and former students a diverse pedagogical and practical experience where they contribute their creative, technological and organizational skills in faculty-led research. This research pays students and alumni stipends and allows them to contribute at the highest levels to one of the first international projects to bring the power of the analog clown into the digital age. Student researchers are supported through funding from the Iowa Center for Research by Undergraduates. 

The acting and theatre arts landscape continues to be changed by technology and world events, but the UI’s acting curriculum is ready and ahead of the curve. “We are developing actors who can integrate technology naturally in their performance, transitioning in and out of different media and formats. That’s an art form unto itself,” Fine says. “Every student actor here can have an opportunity to perform with motion capture. Nobody else is doing that.” 

The University of Iowa currently offers a Certificate in Public Digital Arts, which emphasizes building skills to apply cutting-edge technologies and multiple media platforms to new forms of artistic expression. The Department of Theatre Arts is currently developing a new MFA program in the Digital Arts. To learn more, visit pda.uiowa.edu. 

MEDIA CLOWN TEAM  

Faculty

• Daniel Fine, Theatre Arts and Dance Associate Professor - Creator/Director/Producer 
• Paul Kalina, Theatre Arts Associate Professor - Creator/Performer/Producer 

Industry Partner 

• Matthew Ragan (SudoMagic) - Director of Technology 

Staff

• Emily Berkheimer (2021 BA), Associate Director of Technology 
• Bri Atwood (2015 BA), Sound Design and Audio integration 

Students

• Jordan Jones (MFA, stage management) - Production Stage Manager 
• Victor Maldonado (MFA, design) - Electrician 
• Ellen Oliver (MDA, dance) - Run Crew 

Alumni

• Leigh Marshall (2020 MFA, playwriting) - Lead Writer 
• Chelsea June (2020 MFA design) - Co-Director of Art Department (Costume & Character Design) 
• Courtney Gaston (2020 MFA, design) - Co-Director of Art Department (Physical and virtual world set, lighting design) 
• Travis Kelley (2020 BS Engineering) - Associate Director of Technology 
• Sarah Hamilton (2020 MFA, directing) - Associate Director