Sensing is clearly an important mechanism for a robot's ability to
operate in a dynamic and uncertain environment. Yet most traditional
approaches do not consider the issue of sensing at the motion planning
stage. This results in motion plans that do not utilize the sensing
effectively or consider sensor limitations. We have been developing a
framework for closely integrating sensing with motion planning. The
basis of this approach is extending the notion of configuration
space of a robot to include a set of measurable sensor
parameters. These parameters are derived, for example, from the
visual image of the end-effector of a robot manipulator. The resulting
manifold structure that we term "perceptual control manifold"
(PCM) can be used for defining vision-based planning problems. The
motion plans thus developed can exploit properties of the sensed data
and can also be linked to appropriately designed vision-based control
laws. We have demonstrated the motion planning approach for tasks
such as the interception of an arbitrarily moving target. We are
exploring efficient representations and computational schemes,
e.g. use of randomization, for dealing with the higher dimensional
space involved and neural networks for learning the PCM . We would also like to extend this framework to
include active sensing and tolerance to calibration errors.