Introduction to Cognitive Robotics

Consider the robot depicted in Figure 1. This robot has a camera that can be pointed into different directions and orientation, a wheeled navigation system two arms that can be moved with a gripper that can be opened and closed. If you had a game controller with which you can navigate the robot, orient the camera, and control the arms and hand of the robot you could remote control the robot to accomplish  various manipulation tasks such as setting and cleaning the table, loading and unloading the dishwasher, making popcorn, assembling   toy airplane, replenishing shelves in a retail store, to name only a few.


Fig. 1: PR2 body parts and sensors

In this interactive text book and the accompanying learning environment we want to acquire the competence of understanding, designing, and even implementing control programs that enable robots to accomplish such tasks autonomously.

This control program has to solve the body motion problem: given a task request such as "set the table'' determine a body motion of the robot that accomplishes the desired effects and avoids unwanted side effects. Desired effects are that all items needed are intact, placed on the table, and arranged in the proper way. Unwanted side effects are that the robot breaks objects, damages the environment, or spills stuff.

To reach the competence level of humans the control programs have to solve the body motion problem under challenging conditions.

First, the body motions generated by control program must be perception guided. The program has only incomplete, inaccurate, and possibly faulty knowledge about the environment. Therefore, it has to search and find the objects it needs to manipulate. The execution of body motions is inaccurate and yield some nondeterminism and as a consequence the program is uncertain about the physical effects that the body motions cause. To act robustly despite of these complications has to perceive the environment and monitor its actions to fill in knowledge gaps and revise its beliefs given new perceptual evidence. The control program has to specify how the robot is to respond to what it perceives in order to accomplish its tasks.

Second, the action categories such as pouring are general and can be applied in many contexts. We can appreciate this generality by looking at the breadth and depth of skill with which humans accomplish tasks, such as pouring substances: humans can pour water out of a pot and pancake mix into a pan; they can separate egg yolk from the egg white, extinguish fire, neutralize acid, and pour beer into a glass, to name only a few variations. These pouring tasks involve different substances being poured, different containers, and different tools. They serve different purposes and have different effects. Each variation of the pouring task requires its own specific behavior patterns.

Third, performing actions at the competence level of humans is knowledge intensive and the knowledge has to be turned into body motions.

Fourth, to achieve good performance it is often necessary to learn and be prospective.

Fifth, we require the robot control system to "understand'' what it is doing, by which we mean that it can answer questions about what it is doing, why, how, what it expects to happen, what might go wrong, etc.