Can operating a machine using teleoperation be productive enough? 

Youtube video by Voysys’ customer Cargotec/HIAB. The system is in commercial use.


The shift towards teleoperation in logistical flows including delicate work tasks raise questions with regards to productivity. Is it possible to perform a certain work task at the same speed when using a teleoperation system?

Our experience is that it is possible even for very difficult work tasks like loading timber. There are a number of keys for success in order to match productivity of real life.

 

How do we create a visual experience equal to real life?

 

Voysys solution

Need


  • Support for using any type of camera in a 3D pair

  • Calibration process for 3D camera pairs

  • Stereoscopic 3D support in VR headsets

  • Stereoscopic 3D support on computer screens (active 3D)

  • Stereoscopic 3D support on projected screens (passive 3D) including spherical and cylindrical domes

  • Support for adjustable level of depth (could be better than in real life)

  • Depth assessment should be as good as in real life or better


  • Support for projection on spherical domes

  • Field Of View should cover the area of work including a large vertical FOV for excavators and similar machines


  • Support for most of the fast cameras on the market (Voysys is camera agnostic)

  • Fast encryption

  • Latency-optimized adaptive streaming

  • Fast decryption

  • Fast 3D rendering

  • Glass-to-glass latency close to 50 ms on fast networks

  • Glass-to-glass latency well below 100 ms over 4G/LTE

  • Latency should be kept at a minimum

Measurements of glass-to-glass delay over LTE/4G.


  • Plugin architecture for ingestion of dynamic information directly into the Voysys 3D render engine

  • All machine information visible in the real life environment must be visible to the operator also if using a VR headset


 

Are there productivity problems in real life that can be solved by technology when using teleoperation?

 

Voysys solution

Problem


  • Ultimate “virtual driving position” per work situation
    Example: Low driving position when the scoop of an excavator is at a low position, high driving position when the scoop is on a high position.

  • Fast switching between driving positions

  • Video area optimization per driving position

  • The operator in a machine sometimes suffer from a bad driving position, maybe he/she has a bad angle towards the area of interest.


  • Camera positions outside of obstacles creating transparent sections of the machine

  • Virtual driving positions outside of obstacles

  • Additional 2D camera views rendered as overlays

  • The view of an operator in a machine is sometimes obstructed by the machine itself and parts of it


  • Support for rendering sensor data on top of video to make dust and smog semi transparent

  • The visibility around a machine may sometimes get limited due to dust and smog.


 

Can the use of a large digital screen allow for technical features that can increase productivity?

 

Voysys solution

Idea


  • HDMI input support

  • Overlay renderings

  • Editor for scaling and positioning of overlays

  • Cloud services like Google maps running on a separate computer could assist the operator


  • IP video input support

  • Overlay renderings

  • Editor for scaling and positioning of overlays

  • Cameras positioned at strategic locations other than on the machine could be used to assist the operator, could give better overview or better angles. (Drone cameras?)


  • HDMI input support

  • 3D model support

  • Editor for scaling and positioning of 3D models

  • Plugin architecture for dynamic insertion directly into the Voysys 3D render engine

  • Pre-scanned 3D models of the environment around the machine could be used to create high-quality birdeye views etc.


  • Plugin architecture for dynamic insertion directly into the Voysys 3D render engine

  • Augmented Reality features could be used to guide the operator through his work tasks, and to increase productivity