Constrained hand-multiple-eyes calibration

This paper addresses the problem of calibrating multiple visual sensors mounted on a robotic manipulator, a task critical for accurate robot perception and interaction. We present a  novel approach to hand-multiple-eyes calibration that incorporates closed-loop constraints to ensure consistency between the sensors’ poses. Unlike traditional hand-eye calibration  methods that handle individual sensor pairs independently, our method leverages a unified optimization framework that simultaneously optimizes the relative poses of all sensors while  enforcing a loop closure constraint to each pose triplet. The core of our approach is a least squares approach to solve multiple hand-eye matrix equations of the form AX = XB, further  enhanced with the method of Lagrangian multipliers to account for loop-closure constraints. We apply this idea to a minimal setup involving one hand and two eyes and demonstrate its  effectiveness in improving the accuracy of pose estimation for various levels of noisy measurements.