Justified Text Example

Instinctive Robot Control via Hololens2


Ivan Alberico     Michael Baumgartner     Jonathan Becker     Seif Ismail    

Computer Vision and Geometry LAB, ETH Zurich


Abstract

The objective of this project revolves around the development of an immersive mixed reality interface for the Microsoft HoloLens 2 (HL2). The end goal is to empower users to remotely control a robotic arm, facilitating the execution of fundamental assembly tasks through a combination of hand and eye tracking. The integration of C#, Unity, and the Mixed Reality Toolkit (MRTK) is instrumental in crafting an intuitive interface that seamlessly interfaces with the Hololens 2.





As a pivotal element of this project, the Robot Operating System (ROS) takes center stage in establishing communication between the mixed reality interface and the physical robotic arm. This integration enables users to exert control over the robotic arm from a remote location, thereby facilitating a versatile and user-friendly experience. Within the realm of object pose estimation, Python and OpenCV come into play, leveraging the distinctive qualities of ArUco markers as fiducial markers. This pairing of technologies allows for the accurate determination of the poses of physical objects in the tangible environment, a prerequisite for the successful execution of assembly tasks.



Quaternion Averaging


However, a critical facet of this project revolves around the nuanced incorporation of quaternion averaging techniques. This intricate process serves a paramount role in refining the accuracy of pose estimation. Specifically, quaternion averaging plays a pivotal role in mitigating jittering and ensuring the fluidity of transitions within the virtual representation of physical objects. This becomes particularly salient in scenarios demanding precision, such as the control of a robotic arm for intricate assembly operations. The significance of quaternion averaging is underscored by its ability to address tracking inaccuracies and variations in pose estimation that may arise during user interactions within the augmented environment. By effectively managing orientation data, quaternion averaging emerges as a robust solution to instill stability and cohesiveness, ultimately contributing to a seamless and visually satisfying mixed reality experience.


Without quaternion averaging:


With quaternion averaging:


Key Idea



In essence, this comprehensive amalgamation of C#, Unity, MRTK, ROS, Python, and OpenCV as part of this project seeks to yield a fluid, user-centric mixed reality interface for remote robotic arm control. The deliberate emphasis on quaternion averaging is a testament to the commitment to precision and stability, ensuring that the virtual representation aligns seamlessly with the physical reality, and enhancing the overall user experience within the mixed reality environment.