Experimental Investigation 1
This version has now been deprecated and is superceded by Empirical Investigations
- Source: Deliverable D2.1: A Roadmap for the Development of Cognitive Capabilities in Humanoid Robots, Section 17
Situation: The icub "sits" in a special chair that gives support to the head and legs while the arms are free to move. The surrounding is a homogenously coloured field.
- 1. A face pattern is introduced into the peripheral visual field (30° from the centre) . The visual angle corresponds to that of a real face at 0.5 m. When this happens, the icub moves the eyes and head to position the face at the centre of the visual field. They both starts at the same time, but the eyes arrive first to its new position. When the eyes are at the final position and the head moves there, gaze stay at the fixation object while the eyes counter rotate until they look straight ahead again. The same thing should also happen when a colourful object (3-8° visual angle) is introduced into the visual field or when a sounding object is introduced to the side of the robot (30-50°). New objects that the robot has not seen before will attract the gaze more than familiar objects.
- 2. The robot turns its head (10-20°) while fixating an object or a face (10-30°). The eyes of the robot will then counter rotate so that the gaze is unaffected by the body movements (learning)
- 3. An object moves into the visual field. Its average velocity is 8-25°/s. The robot makes a saccade to the object and then starts tracking it. The tracking will involve both head and eyes. When the object makes repetitive turns the robot should turn its eyes with the motion with no lag. When the turn is unexpected, a lag is acceptable but not greater than 0.1 s. The amplitude of the gaze adjustments may have smaller amplitude than the object motion and the difference will then be compensated with catch up saccades to the object. Learning is involved. With training, the amplitude of the gaze adjustments will better adjusted to the object motion.
- 4. An object moves in the visual field and gets temporarily occluded behind some other objects. The robot stops the eyes at the disappearance point and then makes a saccade to the other side of the occluder. The saccade will predict when and where the object will appear.
The situation is as above.
- 5. The robot extends one of its arms-hand into the visual field and then turns its head towards it. The robot will move the arm-hand and try to keep its eyes on the hand all the time (learning). Both arm-hands should be involved in this activity (first single limbs, then both limbs simultaneously). The robot should touch the other arm and hand when it is looking at it.
- 6. The robot moves the arms to different parts of its own body and touches it. The hand opens up before or during the extension of the arm. This activity is carried out both when the robot looks at the different body parts and when it does not. The purpose of this activity is to build a body map (learning). The icub will also touch body parts that lie outside the visual field.
- 7. A ball or a cube (4-5 cm in diameter) is presented on a string or stick and gently moved up and down in front of the eyes. The robot turns the eyes and head towards it. It also extends one (or both) arms towards the object. The hand opens up during the extension of the arm and the fingers of the hand extends to make the touch surface larger. (When the robot learns to reach, it might be an advantage to make the robot always start the approach at a similar position. We have observed that the infants tend to retreat the hand closer to the body between attempts to get to the object but they do not seem to have a favourite lateral or vertical starting position. Another simplification of the reaching task is to lock the elbow joint. This has been reported in the literature but we have not observed it. It is possible that in special situations where the object is at a position where it can be attained without adjusting the elbow joint, the infant will only adjust the shoulder joint). When the hand of the robot touches the object, this activity will be repeated again and again with variation (that is, the robot retreats the hand a bit and makes a new approach) (learning). If the object is to the right, the right hand will be involved and if the object is to the left, the left hand will be involved. If the object is positioned straight ahead, one or both arms will extend towards it. Note that the focus of pre-reaching activity is on the arm. The hand acts as a feeler.
- 8. The distance and lateral position to the ball or cube is varied from half the length of the arms to 1.5 the length of the arms. The robot will learn to plan an efficient trajectory to the object. To begin with only a part of the trajectory will be planned ahead. At the end of this part, a new segment will be planned, etc. In the end, a continuous movement to the goal will be performed. If the distance to the object is larger than the arms, the robot will not reach for the object.
Reach and grasp
The situation is as above.
- 9. Objects of different sizes are introduced into the visual field of the robot. The robot extends one or both hands towards the object and then grasps it. The duration of the approach will be 3 s or less. The robot hand will slow down towards the end of the approach and just before grasping the object, the velocity will be close to zero. The robot will fixate the object to be grasped during the approach.
- 10. The hand will first open up during the approach of the object and then begin to close around it. All fingers will be engaged. To begin with, the hand will open to its full extent during the approach before starting to close. Later on during training, the maximal opening of the hand will be adjusted to the size of the object. The maximum opening of the hand should always be larger than the object to be grasped to make it easier to slide the hand over the object. It is important that the grasping begins before the touch otherwise there is a risk that the hand of the robot will push away the object as a consequence of the touch. The last part of the closing of the hand will take place as the robot-hand is in contact with the object. If the object is large (<10 cm diameter) both hands will participate in grasping the object. In order not to have the two hands compete for grasping the object, it might be desirable to develop some laterality.
- 11. Objects of different forms are introduced into the visual field of the robot (cylinders with a 2 cm and 5 cm diameter, and egg-shaped object with maximum diameter of 6 cm, an irregular object). The robot-hand will rotate during the approach in order to grasp the object over the most convenient opposition space. If the object is a rod, the grasp will take place around its longitudinal axis.
- 12. The object to be grasped moves. The velocity of the object motion will vary from 5 to 60 cm/s. The object will either approach on a vertical trajectory or a horizontal one. The hand moves towards a future position of the object where the hand and the object will meet. If the object comes from the left, it is the right arm-hand that will grasp it and if it comes from the right, it is the left arm-hand that will grasp it. The other hand will help to secure the object after the active hand has caught it (stopped it).
- 13. Small round objects (0.5 to 2.0 cm diameter) will be introduced into the visual field. The robot will then only engage the thumb and the index finger in the act of grasping them.
- 14. After the object is grasped, the robot will examine the object by turning it around. Both hands will participate in this activity. One hand will hold the object in a fixed position while the other hand is moved over it in order to feel its surface and examine its interior. Through this activity the robot will built an object representation of familiar objects.
- 15. The object will be transferred from one hand to the other while the robot fixates the object (maybe also transferred repeatedly between the hands). The transfer should be as smooth and continuous as possible. This means that the delivering hand should let go of the object at the same time as the receiving hand grasps it.
- 16. After grasping an object, the robot will move it to another position and deposit it there. The robot will turn its gaze towards the goal position of the action while the object is moved there. If the object is moved to its final position via an obstacle, the robot will fixate the obstacle and when the hand with the object has cleared the obstacle, the gaze will go to the final position.
Reaching and posture
(not in this version) The robot sits without support.
- 17. compensation for inertia and gravity.
- 18. Leaning forward.
- 19. Using the other hand to counterbalance.
Postural control in action
(not in this version) The icub sits independently and moves by crawling.
- 20. crawling. R crawls and prepares a reach during crawling. Crawl to sit.
- 21. sitting and balancing.
- 22. Balancing during action. Posture: the body is stabilized so when R grasps the other hand counterbalances.
Manipulation and affordances
(not in this version)
- 23. R. acts on an object: grasps it and drops it on the floor, looking.
- 24. R. combines two objects to see how they fit