Telepresence And Robotic Training

Enterprise, Training / Education


VR - Tethered Headsets

Telepresence And Robotic Training

Telepresence And Robotic Training


A large industrial manufacturer needed a simulation environment that could easily and efficiently be used to train a simple robot extension arm and grabber. As the project progressed the client became interested in adopting increasingly more complex solutions that would provide additional functionality and value.


Initially the task for this robot was to train the arm and grabber. A human operator would manipulate the arm, record the movement and then use that data to train the virtualized robotic system. The process required the development of various 3D objects—cylinders, boxes of various size and mass, etc.—to be moved around within the simulation. 

Shortly FS Studio team decided to expand the scope of the project and create a digital twin of the complete robotic system and all its sensors. This system would still train, but also facilitate telepresence of the robot. To accomplish the assignment RSS imported all of the robot’s data from a URDF file and constructed a robot from the file definition and supplied assets. FS Studio ensured that the robot had proper IK and control systems implemented and the various sensors as well. The virtualized robot was then incorporated into simulated environments—living rooms, kitchens, institutional spaces—for operator training in a variety of situations. 

After evaluating the success of these projects, the client had an additional request. Could FS Studio create a system to place the virtual robot in a real-world environment? RSS was happy to help. FS Studio developed a solution to migrate the virtual simulation into an augmented reality experience on the HoloLens and Magic Leap HMDs. Since the latest AR systems incorporate robust SLAM and dynamic Spatial Mapping it was pretty straightforward process. With the addition of a new user interface the virtual robot could now easily maneuver in the real-world with features like collision detection, obstacle avoidance, and route finding. 


The clients were completely thrilled with the entirety of the project. The had initially tapped RSS to develop a simple grabber arm training program and through collaboration and brainstorming now had at their disposal an end to end simulation that not only provided virtual training but also a proof of concept that their products operated successfully in real world environments.


By | July 12, 2021

Need help choosing the right XR partner? Just ask us!