Book
Introduction to Cognitive Robotics
Completion requirements
1. Introduction
1.2. Robot agents
In this section we consider how robot agents accomplish different kinds of tasks.
... making popcorn
As an example for a task we consider a sequence of pick and place actions, e.g., making popcorn. To make popcorn, the robot needs to:
- Pick up an empty pot and place it on the hotplate.
- Turn the hot plate on.
- Pick up the bowl with corn, put the corn into the empty pot, and place the bowl back on the table.
- Pick up the lid and place it on the pot.
- Shake the pot after some time so the popcorn doesn't burn.
- Wait until the popcorn is done.
- Pick up the warm pot, put the finished popcorn into a bowl, and place the pot on a cold area of the hotplate.
Fig. 1: Grasping a pot.While all these actions pick and place actions, they are dependent on the corresponding context and vary in how exactly they have to be executed. For instance, the bowl and the pot need to be grasped in different ways and they need to be placed in different orientations and locations. The location and the state of each individual object has an impact on how the pick and place action has to be performed to be successful. For example, the robot can grasp a pot that is empty and cold in many ways, but when the popcorn is done and the pot is still hot, the robot needs to know how to grasp it by the handles.
A robot that is able to cope with context-dependent actions in a dynamic environment needs to be implemented as a so-called cognition-enabled agent. A cognition-enabled agent has the skills to decide which actions it has to do and also how it should perform the actions according to the given context. In addition, the agent is aware at any time what action needs to be done to reach the goal and can also explain why it is performing those actions. To enable the robot to be cognitive, a complex software system is required which allows splitting the activity in pick and place actions. This generates the specific required movements. The system needs to handle a fast and reactive interplay of hardware and software components to let the robot cope with the context related actions in a complex task such as popcorn making.
To make our discussions more tangible we will take a home assistance robot as our running example, which is supposed to prepare popcorn in a kitchen environment. In order to translate an instruction, such as "make popcorn'', into action a robot has to infer missing information including missing action steps and their appropriate order. In order to get a better intuition about the complexity and details of such manipulation tasks, consider the snapshots of the cooking activity, which are depicted in Figure 1. Here is a video featuring a robot performing the complete popcorn preparation task:
A robot that is able to cope with context-dependent actions in a dynamic environment needs to be implemented as a so-called cognition-enabled agent. A cognition-enabled agent has the skills to decide which actions it has to do and also how it should perform the actions according to the given context. In addition, the agent is aware at any time what action needs to be done to reach the goal and can also explain why it is performing those actions. To enable the robot to be cognitive, a complex software system is required which allows splitting the activity in pick and place actions. This generates the specific required movements. The system needs to handle a fast and reactive interplay of hardware and software components to let the robot cope with the context related actions in a complex task such as popcorn making.
To make our discussions more tangible we will take a home assistance robot as our running example, which is supposed to prepare popcorn in a kitchen environment. In order to translate an instruction, such as "make popcorn'', into action a robot has to infer missing information including missing action steps and their appropriate order. In order to get a better intuition about the complexity and details of such manipulation tasks, consider the snapshots of the cooking activity, which are depicted in Figure 1. Here is a video featuring a robot performing the complete popcorn preparation task:
Fig. 2: Action steps for popcorn making: (1) putting the cooking pot on the stove, (2) opening the drawer, (3) pouring the corn into the pot, (4) switching on the drawer, (5) grasping the lid, (6) putting the lid on the pot, (7) distributing the corn evenly in pot, (8) pouring the popcorn onto the plate, (9) salting the popcorn.
Competently making popcorn requires robots to solve a number of challenging reasoning tasks. The robot has to infer implicit goals and unmentioned subactions, including that in order to cook popcorn the unpopped corn has to be put into a cooking pot, the pot be placed on a stove top, the knob for the respective stove be turned in order to switch the stove on. The robot has to wait for the popcorn to be done, the lid has to be removed from the pot in order to pour it onto a plate.
Other kinds of needed knowledge might be that a pot with a lid on top is heavier than without it and that a filled pot should be held upright. When pouring the pot should not be held too high above the plate to avoid spilling. Also, a hot pot should be only grasped by its handles. For grasping objects the robot should choose the grasp type and position that is most reliable, efficient, and controllable. Additional task knowledge includes knowledge about where the popcorn is stored, that the salt grinder has to be turned upside down, and the the drawers open outwards.
Other kinds of needed knowledge might be that a pot with a lid on top is heavier than without it and that a filled pot should be held upright. When pouring the pot should not be held too high above the plate to avoid spilling. Also, a hot pot should be only grasped by its handles. For grasping objects the robot should choose the grasp type and position that is most reliable, efficient, and controllable. Additional task knowledge includes knowledge about where the popcorn is stored, that the salt grinder has to be turned upside down, and the the drawers open outwards.