Transforming screens into stages with AR
What’s ALTstage AR App?
ALTstage is the world’s first technology that brings AR together with live events and video content. It came to life using Azure Kinect hardware and volumetric videos.
Its main goal is to let the watchers enjoy the three-dimensional views of performers, whether at home or live. Just with the help of their mobile phones.
Client’s Wants & Needs
Breathe Life into Live Video Performances
Due to the Coronavirus outbreak, more live performances are being broadcasted than ever. However, the two-dimensionality of videos quickly became an issue. Videos take away the thrill of the theater as they just can’t convey the beauty and grace of a performer at work.
Combine Live Performance with AR Technology
WE/AR Studio was tasked with developing an app that would solve this problem with AR technology. The app was required to allow viewers to see performers in motion in augmented reality.
Accessible 3D AR App
WE/AR Studio’s clients wanted the audience to have three-dimensional views of performers, whether at home or live. The app was required to be compatible with modern gadgets and also suitable for low-budget projects.
Scalable ALTstage Platform
The ALTstage app is the first step in a broad project. WE/AR Studio clients’ end goal is to develop a big ALTstage platform that will be used by performing artists around the world.
How WE/AR Studio Did It
1. Client’s Needs
As always, the first step for the WE/AR Studio team during any project is getting a clear idea of what their client hopes to achieve. They worked hand-in-hand with the Ukrainian Platform of Contemporary Dance and the German Post Theater from start to finish to ensure that they achieved the best results.
2. Video Recording
After understanding the project, WE/AR Studio proceeded to a Chromakey studio to record a volumetric video. The company then processed it to procure a material in which the performers were taken out of the background. That way, they were able to add any background they desired.
3. Shooting a Volumetric Video for AR
WE/AR Studio got this project done using the Azure Kinect hardware. The team had to use a 12-megapixel camera along with an infrared sensor.
WE/AR Studio recorded 2D images with the camcorder and measured the distance between the device and the pixels of the image with the sensor. That way, they were able to transform the video from 2D to 3D.
4. Synchronization
WE/AR Studio developed a software environment that synchronizes AR with the performance on a screen. Two devices were needed for this; a laptop for playing the video and a mobile device reproducing the augmented reality.
The synchronization in space was done with the aid of markers. They transmit the details of the desired location and size of the AR object to the app via the camera of a smartphone or tablet.
However, time synchronization is a new technology created by the ALTstage team. The principle behind it is simple: the laptop’s video player and the ALTstage app receive instructions from a server to play video and AR. This way, the performers in the video and AR can interact.
5. Debugging of UI & UX
This process was all about making the technology easy to use. The importance of this was to allow viewers to enjoy live events instead of concentrating on the complexity of the technology.
Challenges That WE/AR Studio Overcame
Identifying the goals of the project
Converting 2D images to 3D
Developing time synchronization
Mastering volumetric video technology
Combining two modes (live and home) in one AR application
Building a foundation for the ALTstage platform
Project Results
This technology isn’t restricted to the art industry alone. As long as your event has an audience, you can give them spectacular views with ALTstage.
After the presentation is held in a few cities across Ukraine, the ALTstage platform will be fully launched so that anyone can use it. What this implies is that performing artists around the globe will be allowed to create their projects on the ALTstage platform.
The results of the ALTtage project were presented in July 2021 in Kyiv.
During this event, the audience was given a demonstration of how both the home and live versions of the technology works.
COMPANY: WE/AR Studio
