学生混合模型（SMM）视觉伺服：用整幅图像的光度数据代替几何特征

Classical Image-Based Visual Servoing (IBVS) makes use of geometric image features like point, straight line and image moments to control a robotic system.

Robust extraction and real-time tracking of these features are crucial to the performance of the IBVS.

Moreover, such features can be unsuitable for real world applications where it might not be easy to distinguish a target from the rest of the environment.

Alternatively, an approach based on complete photometric data can avoid the requirement of feature extraction, tracking and object detection.

In this work, we propose one such probabilistic model based approach which uses entire photometric data for the purpose of visual servoing.

A novel image modelling method has been proposed using Student Mixture Model (SMM), which is based on Multivariate Student's t-Distribution.

Consequently, a vision-based control law is formulated as a least squares minimisation problem.

Efficacy of the proposed framework is demonstrated for 2D and 3D positioning tasks showing favourable error convergence and acceptable camera trajectories.

Numerical experiments are also carried out to show robustness to distinct image scenes and partial occlusion.

Visual servoing [1, 2] is one such approach where a control law regulates an error in the image space to achieve desired positioning of the robot. To achieve this, a set of visual features has to be selected from the image of the object of interest as a primary step. Following this, the current and desired pose can be defined in terms of the selected visual features. Once the feature selection is done, an error function between desired and current image features is modeled.

The IBVS commonly uses points [6],[7],[8], lines, contours [9],[10] optical flow estimates, etc. as the features. For such features real-time feature tracking and correspondence are important but computationally involved steps of vision-based control. Various approaches and advancements have been proposed for feature tracking [9, 11, 12] to overcome this problem upto a limit but it is an unavoidable step. An extension of these approaches has been proposed in [13] where shape and moments are considered as visual features. These features provide more intuitive than the geometric features but a segmentation step is still required before extraction of features. Vision-based control technique applicable to real life environments with diverse objects and robust to noises are the main challenges of IBVS. Therefore, the present work proposes a solution to overcome these challenges. Previous approaches rely mainly on geometric features. However, additional computations for feature extraction, matching and real-time tracking are some overheads for these visual servoing methods.

In this regard, researchers have concentrated towards feature less tracking to avoid the drawbacks of geometric approaches. A work in that direction has been initially presented in [14] and [15] where image intensity was used as indirect visual features to perform control action. Later image intensity was directly considered to perform 2D matching of images without any feature extraction in [16] and [17]. Here, a kernel projected value of intensities in image at each pixel and information theoretic approach of mutual information were illustrated. A similar method called Photometric visual servoing developed in [18] used image intensities as visual features. Use of the concept of Sum of Conditional Variance [19] results in a direct visual servoing task which is easy and fast to compute. Even though it is robust towards non-linear illumination variations, it is less robust towards local variations. A depth map obtained from a range sensor was used as a visual feature in [20]. This approach is both direct (without any 3D pose estimation) and dense (without feature extraction) but it requires an expensive range sensor which may have problems like noise in data and the absence of measurements. The above limitation is overcome in [21] where a histogram is considered as visual feature. Here, the applicability of the framework was shown to many histograms, like, intensity histograms, Histograms of Oriented Gradients (HOG) and colour histograms. However, this method suffers from low convergence and impact from uninformative image regions of an image under consideration.

A new approach using feature based mixture models in visual servoing has been initially studied in [26]. Here, feature points extracted from current and desired images were represented by two separate mixture models and the visual servoing minimized the difference between the two mixture models. It is advantageous since it considers feature position uncertainties by probabilistic models. However, the feature point detection is still a mandatory step and points extracted from the current images has to be the same as the points extracted from the desired image. Furthermore, it is implemented to control only three degrees-of-freedom (DOF) of the robot and also lacks an analytical formulation of interaction matrix. In [27], dense features were used to model mixture models alternatively to the above method. The main idea of this method is to model every pixel in the image as Gaussian function, and the algorithm tries to reduce the error between desired and current gaussian function of images. However, suffers from the drawback that it requires tuning of an extension parameter experimentally for providing overlapping of image regions.

In this paper, we propose a novel approach to visual servoing (see Fig. 1) using student t𝑡t-distribution mixture model. In literature, it can be seen that t𝑡t-distribution mixture models are successfully used in image processing applications like image registration [28]. Geometrically a t𝑡t-distribution function has heavily-tailed alternative to the normal distributions with same symmetric and bell-shaped density function. Such a characteristics of t𝑡t-distribution guarantees inclusion of values which is far from its mean value.

1. A novel SMM image modelling method. A novel image modelling method has been proposed using Student Mixture Model (SMM), which is based on Multivariate Student's t-Distribution.

2. Analytical interaction matrix and least squares control law. An analytical formulation of interaction matrix using the proposed novel SMM feature model is introduced for visual servoing. The proposed method relies on the minimization of distances between probability density functions defined by SMM of current and desired images. Thus the control law is modeled as an optimization problem for minimizing the cost function.

3. Verified in 2D/3D tasks and realistic simulation. Efficacy of the proposed framework is demonstrated for 2D and 3D positioning tasks showing favourable error convergence and acceptable camera trajectories. Numerical experiments are also carried out to show robustness to distinct image scenes and partial occlusion.

In this paper, a student t𝑡t-distribution mixture model based framework is proposed for visual servoing. Instead of using image features, the entire image is modeled as a function using SMM. The SMM model enables a method to deal with non-textured/textured objects and also it serves as a replacement of methods where extracting visual features is too complex or too costly. An analytical formulation of interaction matrix using the proposed novel SMM feature model is introduced for visual servoing. The proposed method relies on the minimization of distances between probability density functions defined by SMM of current and desired images. Thus the control law is modeled as an optimization problem for minimizing the cost function.

This control law is able to perform servoing operations in successful manner even with the influence of image lighting changes, texture changes and occlusions. To prove its efficacy, the proposed algorithm was initially implemented on a numerical simulation model for 2D, 3D cases and later a realistic simulation is performed in gazedo using UR5 manipulator. The experimental results showed the effectiveness and accuracy of the proposed method for different images, different poses, different image size and under occlusions. While the proposed system works well on the numerical model, execution on the real-world setup and detailed stability analysis will be taken up as future work. The method will also be extended for visual servoing towards an object which is under motion.