ARStudio: AR Broadcasting System based on Multiple RGB-D Cameras

Abstract

Many television shows have been produced that synthesize virtual objects or make use of interactions between cast members and virtual objects. Recently, many attempts have been made to apply augmented reality (AR) technology to the production of interactive broadcast content using virtual objects and special effects, even during a live broadcast. In this study, we present ARStudio, an AR-based broadcasting system that can transmit virtual objects by synthesizing them in real time to the 3D structure of the broadcast set space recognized by multiple cameras. Also, we introduce a new calibration method that can easily and quickly merge multiple broadcast cameras and RGB-D cameras into one coordinate system. Additionally, we built a spatial editing and control tool for traditional broadcast producers to help them easily implement AR broadcasting. We applied the proposed system to a real broadcasting production environment by working with a Korean national broadcasting company, EBS, and demonstrated its functionality through a public TV network.