Research: Robotics

Visual Servoing to an Arbitrary Pose with Respect to an Object Given a Single Known Length


A method is presented to attach reference frames to piecewise planar objects in view a single camera. This method uses Euclidean homography relationships and a single known geometric length on a single object. By attaching reference frames to objects in the scene, the method is useful in positionbased visual servo control, where it allows control of pose with respect to an object. The method is distinguished from methods that require a detailed model of the object/scene to give camera pose relative to an object, and it is distinguished from methods that can only give current camera pose with respect to a pose where a reference image was taken. Simulations of the method for camera-in-hand and camera-to-hand visual servo control tasks are presented. Experiments are presented where the reconstruction method is used to estimate the pose of a vehicle. These experiments represent the initial steps in a visionbased vehicle following controller.

Experimental Results

In conjunction with the Center for Intelligent Machines and Robotics at the University of Florida, we are in the initial stages of implementing the method presented in the paper in a vehicle following (i.e., platooning) applications. A leader vehicle is fixed with a known target, and a follower vehicle carries an on-board camera. By tracking the target points on the leader vehicle, the pose estimation method presented in this paper is used to estimate relative pose and velocity of the leader vehicle with respect to follower vehicle. The follower vehicle uses a predefined optimization function to trace its path in the environment. The relative pose estimation allows the follower to maintain a fixed distance between the leader and follower vehicle. The throttle and brakes can be controlled with a PD controller, using a relative pose and velocity estimation data, to avoid collision with leader vehicle or losing track of leader vehicle in the field.

Early experiments demonstrate the effectiveness of the vision based estimation in this scheme. Field tests of the complete platooning system will be conducted in the coming weeks. An experiment using a moving vehicle in Fig. 1 is performed to demonstrate the pose estimation method as used in the platooning application. Four bright LED arrays were fixed to the back of a truck to facilitate simple image segmentation, where the centroid of each detected array provides four feature points used to construct the Homography matrix. Each of the four centroids is indicated in Fig. 1 by a cross and a number. The truck was equipped with a differential GPS unit to provide a reference to compare the pose estimation. The road was marked at approximately 20 feet (6.1 m) intervals and the car was driven forward and stopped approximately every 20 feet.

The results of the experiment are seen in Fig. 2. The expected periodically increasing step function along the camera frame z direction (i.e., the optical axis) is evident. Furthermore, the change in pose estimate agrees closely to the GPS Northing measurement. There is also a small periodic step increase estimated in the camera x direction. The estimate degrades as the distance to the vehicle increases. This is primarily due to sensor noise. As the car moves farther from the camera, the perceived lights become dimmer and it is harder to extract the centroids. This increases the effects of pixilation (i.e., quantization noise). There are currently several avenues of development on the vision system. The first is the introduction of brighter LED arrays, which will mitigate the problem of feature noise at large distances. New set of LED's are visible at a distance up to 100 meters. Additional LED targets will be affixed to the sides of the lead vehicle to maintain target tracking during turns or aggressive maneuvers. The next experiment will involve 5 targets at the back, and 4 each on left and right sides of the leader vehicle. Five targets on the back plane will be used to distinguish back plane from side planes in an image. Efforts are being made to replace artificial targets with natural features like sharp corners.

Figure 1: A processed video frame from the pose estimation experiment.

Figure 2: Results of experiment to estimate the pose of a moving vehicle.


Gans, N.R.; Dani, A.P.; Dixon, W.E., "Visual servoing to an arbitrary pose with respect to an object given a single known length," American Control Conference, 2008 , vol., no., pp.1261,1267, 11-13 June 2008

More Publications