SHRDLU the Robot

SHRDLU is a computer program constructed at MIT by Terry Winograd. SHRDLU controls a robot arm that operates above a table top strewn with colored play blocks of various shapes and sizes. Both the arm and its environment are simulated by a computer which also synthesizes a TV display of the arm's activities. Communication between human and robot is via a keyboard, with the computer's replies appearing as 'subtitles' on the TV screen. Below is a representative dialog between SHRDLU and Winograd, which demonstrates SHRDLU's ability to handle natural English instructions and discuss the "motives"; of its own actions.

Winograd's questions are shown in boldface. SHRDLU's responses are shown in capital letters indicated by a green dot (). Winograd's comments are italicized in parentheses.

Pick up a big red block.

 OK.

(On the screen, the robot arm swings into action. Two red blocks are 
visible, one small, one large, as on figure above.  The large one has a 
green cube stacked on top of it. The robot first transfers the green cube 
to the table top, and then picks up the red block.)

Grasp the pyramid.

 I DON'T UNDERSTAND WHICH PYRAMID YOU MEAN.

(There are three pyramids in the scene, and SHRDLU knows that phrases
beginning with "the" are intended to refer to a specific object the speaker
has in mind.)

Find a block which is taller than the one you are holding and put it
into the box.

(SHRDLU has to figure out what is meant by "one" and "it", as well as using
deduction to decide which block it is holding.)

 BY 'IT', I ASSUME YOU MEAN THE BLOCK WHICH 
IS TALLER THAN THE ONE I AM HOLDING.

(SHRDLU puts down the red block, picks up a taller blue one, and places 
it in the box.)

What does the box contain?

 THE BLUE PYRAMID AND THE BLUE BLOCK.

What is the pyramid supported by?

 THE BOX.

(There are in fact two other pyramids on the table, a red one and a green 
one. SHRDLU rightly assumes that the question concerns the pyramid it 
just mentioned.)

How many blocks are not in the box?

 FOUR OF THEM.

Is at least one of them narrower than the one which I told you to pick up?

(The phrase "one of them" must be analyzed to refer back to the previous
question, and "the one which..." phrase must also refer to blocks.  The memory
of past events is used to deduce which block this refers to.)

 YES, THE RED CUBE.

Is it supported?

(SHRDLU has heuristics about what information is relevant to a person who
asks a question, even if the person does not provide that information
specifically.)

 YES, BY THE TABLE.

Can the table pick up blocks?

 NO.

(SHRDLU's semantic rules include the knowledge that "picking up" must be
done by an animate object, and that the table is not animate.)

Can a pyramid be supported by a block?

 YES.

(SHRDLU finds an actual example, so it knows this is possible.)

Can a pyramid support a pyramid?

 I DON'T KNOW.

(There is no example of this in the scene.)

Stack up two pyramids.

(The robot tries and fails.)

 I CAN'T.

The blue pyramid is mine.

 I UNDERSTAND.

(Possession is an unknown property of objects about which SHRDLU can be
told.  It stores this information in its data base.)

I own all blocks which are not red, but I don't own anything which supports
a pyramid.

 I UNDERSTAND.

(The result of understanding these sentences is the creation of new
procedures within SHRDLU for deducing whether you own something.  The next
four questions give SHRDLU an opportunity to display its powers of
deduction and logic.)

Do I own the box?

 NO.

Do I own anything in the box?

 YES, TWO THINGS: THE BLUE BLOCK AND THE BLUE PYRAMID.

Is there anything which is bigger than every pyramid but is not as wide 
as the thing that supports it?

 YES, THE BLUE BLOCK.

Does the shortest thing the tallest pyramid's support supports support 
anything green?

(SHRDLU took less than twenty seconds to make sense of this; I needed 
rather longer.)

 YES, THE GREEN PYRAMID.

What colour is it?

(SHRDLU sensibly assumes that the question is not about the green pyramid.)

 BY 'IT', I ASSUME YOU MEAN THE SHORTEST 
THING THE TALLEST PYRAMID'S SUPPORT SUPPORTS. RED.

Does a steeple

 SORRY, I DON'T UNDERSTAND THE WORD 'STEEPLE'.

(SHRDLU interrupts as soon as it encounters a word that it doesn't know.)

A 'steeple' is a stack which contains two green cubes and a pyramid.

(The quote marks indicate that you are defining a new word.  The definition
is added to SHRDLU's dictionary in a form compatible with all the words it
already knows.)

 I UNDERSTAND.

Are there any steeples now?

 NO.

Will you please stack up both of the red blocks and either a green cube 
or a pyramid?

 OK.

(SHRDLU opts to stack the red blocks with a green cube. In the course of 
building its stack, it has to move the green pyramid off one of the red 
blocks.  Remember this.)

Which green block is sitting on the table?
 THE ONE WHICH SUPPORTS THE RED PYRAMID.

Put the littlest pyramid on top of it.

 OK.

(SHRDLU duly picks up the tiny green pyramid.)

Had you touched any pyramid before you put one on the green block?

 YES, THE GREEN ONE.

(SHRDLU deduces from its memory what event is being referred to, and can
relate other events to it in time.)

When did you pick it up?

 WHEN I WAS STACKING UP A RED CUBE, A LARGE 
GREEN CUBE AND A LARGE RED BLOCK.

Why?

(SHRDLU remembers immediately previous sentences to understand questions
like this and the following ones.)

 TO GET RID OF IT.

Why did you do that?

("That" refers to the action mentioned in SHRDLU's last answer.)

 TO CLEAR OFF THE RED CUBE.

Why did you clear off that cube?

(Here, "that" means the cube just referred to.)

 TO PUT IT ON A LARGE GREEN CUBE.

Why did you do that?

 TO STACK UP THE RED CUBE, A LARGE RED BLOCK, AND A
LARGE GREEN CUBE.

Why did you do that?

 BECAUSE YOU ASKED ME TO.