Difference between revisions of "ICub Cognitive Architecture"

From Wiki for iCub and Friends
Jump to: navigation, search
(Explanation of the iCub Cognitive Architecture and its relationship to the software architecture)
 
 
(46 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 +
{{lowercase|iCub Cognitive Architecture}}
 +
 +
__TOC__
 +
 +
==The Evolution of the Architecture==
 +
 +
The iCub cognitive architecture is the result of a detailed design process founded on the developmental psychology and neurophysiology of humans, capturing much of what is known about the neuroscience of action, perception, and cognition. This process and the final outcome is documented in [http://www.robotcub.org/index.php/robotcub/more_information/deliverables/deliverable_2_1_pdf Deliverable D2.1: A Roadmap for the Development of Cognitive Capabilities in Humanoid Robots].
  
{{lowercase|iCub Cognitive Architecture}}
 
__NOTOC__
 
  
The iCub cognitive architecture is the result of a detailed design process founded on the developmental psychology and neurophysiology of humans so that it encapsulates what is currently known about the neuroscience of action, perception, and cognition. This process and the final outcome is documented in [http://www.robotcub.org/index.php/robotcub/more_information/deliverables/deliverable_2_1_pdf Deliverable D2.1: A Roadmap for the Development of Cognitive Capabilities in Humanoid Robots].  
+
The architecture itself is realized as a set of [[ICub Cognitive Architecture#Cognitive Architecture Modules Documentation | YARP executables]], connected by YARP ports.  Early prototypes were developed at a RobotCub project meeting at the University of Hertfordshire in July 2007 as an exercise in consolidating the software development effort of the project partners. Several subsequent versions were produced at the [[VVV07 | RobotCub Summer School 2007]] VVV '07. These prototypes were developed in parallel with the D2.1 Roadmap effort mentioned above.  These two strands of design effort converged in the cognitive architecture shown below (Version 0.4). Previous versions can be accessed via the links at the end of the page.  [[VVV09 Cognitive Architecture Group | VVV '09]] addressed the development of the architecture's [[Auto-associative Memory Specification | (auto-associative) episodic]] and [[Hetero-associative Procedural Memory Specification | (hetero-associative) procedural]] memories.
  
The architecture itself comprises a set of [[iCub YARP module specifications | YARP executables]], typically connected by YARP ports.  Early prototypes were developed at a RobotCub project meeting at the University of Hertfordshire in July 2007 as an exercise in consolidating the software development effort of the project partners.  Several subsequent versions were produced at the [[VVV07 | RobotCub Summer School 2007]] VVV '07.  These prototypes were developed in parallel with the Roadmap effort mentioned above.  These two strands of design effort converged in the cognitive architecture shown below (version 1.0). Previous versions can be accessed via the links at the end of the page.
 
  
The immediate purpose in developing the software architecture is to create a core software infrastructure for the iCub so that it will be able to exhibit a set of target behaviours for an [[Experimental Investigation 1]].
+
The immediate purpose in developing the cognitive architecture is to create a core software infrastructure for the iCub so that it will be able to exhibit a set of target behaviours for an [[Empirical Investigations]].
  
[[Image:iCub_cognitive_architecture_v1.0.jpg]]
+
[[Image:Icub_cognitive_architecture_v0.4.jpg]]
  
 +
Figure 1: iCub cognitive architecture, version 0.4.
  
==Modifications==
+
==Differences from previous version==
* Removed the <code>tracker</code>
+
* Removed the <code>tracker</code> (should be handled by attention/salience sub-system)
 +
* Removed the <code>face localization</code> (should be handled by attention/salience sub-system)
 +
* Removed the <code>hand localization</code> (should be handled by attention/salience sub-system)
 +
* Removed the <code>sound localization</code> (should be handled by salience module)
 +
* Removed the <code>attention selection</code>  
 
* Added <code>Exogenous Salience</code> and <code>Endogenous Salience</code>
 
* Added <code>Exogenous Salience</code> and <code>Endogenous Salience</code>
 
* Added <code>Locomotion</code>  
 
* Added <code>Locomotion</code>  
 +
* Added <code>Matching</code>
 +
* Added <code>Auto-associative episodic memory</code>
 +
* Added <code>Hetero-associative procedural memory</code>
 +
* Added <code>Affective state</code>
 +
* Added <code>Action selection</code>
 +
 +
==Notes==
 +
* Gaze implies 7 DoF: head and eyes
 +
* Locomotion paradigm: “go where you are looking”
 +
* Reaching paradigm: “reach where you are looking”
 +
* Endogenous and exogenous salience implies salience based on internal and external events, respectively
 +
* Gaze, reaching, and locomotion motor activities condition endogenous salience: i.e. motor states condition attention
 +
* Sensory inputs condition exogenous salience: i.e. attention conditions motor states
 +
* Episodic memory is memory of autobiographical events. Initially, this is purely visual and implemented as an auto-associative memory.  Later it will be multimodal and will include sound as well as associated emotions. It will then have to be implemented as a hetero-associative network of unimodal auto-associative memories.
 +
* Episodic memory storage is conditioned by poor matching and high salience
 +
* Procedural memory is defined to mean perception-action event sequence
 +
* Procedural memory recall:
 +
** Event A & Event D inputs recall sequence of intermediate events
 +
** Event A input  recalls Event B (subsequent event)
 +
* Affective state is a competitive network of three motives:
 +
** Curiosity (exogenous salience prevalent)
 +
** Experimentation (endogenous salience prevalent)
 +
** Social engagement (exogenous and endogenous salience balanced)
 +
* Action selection is not a winner-take-all process: one or more actions are disinhibited
 +
* The developmental drive is to construct a procedural memory that improves prediction
 +
 +
A more detailed description of the behaviour of each module and circuit in this architecture will be added in due course (both here on the iCub wiki and in [http://www.robotcub.org/index.php/robotcub/more_information/deliverables/deliverable_2_1_pdf Deliverable D2.1: A Roadmap for the Development of Cognitive Capabilities in Humanoid Robots]).
 +
 +
 +
==YARP Module Implementation==
  
 +
The iCub cognitive architecture is implemented as a set of iCub YARP modules, as follows.
  
==Observations==
 
* attentionSelection is likely to expand (and/or sub-divide) into, first, <code>automaticActionSelection</code>, and then to <code>actionSelection</code>
 
* <code>egoSphere</code> is likely to expand (and/or sub-divide) significantly in the future. First, it might become a full expression of peripersonal space, encompassing proprioceptive and exterioceptive perception in a temporally stable fashion. This stability would be maintained despite re-orientation by the iCub. Second, it might then become some form of [[#Notes | allocentric>]] mechanism, effected possibly by some mutual association of many egocentric representations.  This allocentric mechanism would be stable despite movement of the iCub around its environment.
 
* The <code>controlGaze -> salience -> attentionSelection</code> circuit is a fast retinotopic circuit. It is one of the circuits by which motoric state modulates attentional capacity.
 
* <code>soundLocalization</code> at present implies simply a binaural localization of the direction of arrival of a sound, expressed in head-centric spherical coordinates.  It should ultimately effect some sensori-motor capability that acts to re-orient the iCub towards a localized sound.
 
* <code>faceLocalization</code> would ideally be implemented as a ‘three blob’ detection; at present it is intended to utilize an OpenCV face detector for the immediate future.
 
* <code>Reaching -> salience -> egoSphere -> attentionSelection -> controGaze</code> is the primary circuit for achieving visually-guided reaching.
 
* <code>visualSalience</code> and <code>attentionSelection</code> are tightly-coupled.
 
  
==Notes==
 
''Timbre'', in the sense of time-frequency characteristics of a signal, typically focussing on the pattern of  growth and decay of harmonics
 
  
''Allocentric'', in the sense of addressing aspects of the iCub’s sensory world that have been experienced but are not presently within its sensory compass.
+
[[Image:Icub_cognitive_architecture_modules_B.jpg]]
 +
 
 +
[[Image:Icub_cognitive_architecture_modules_A.jpg]]
  
==Open Issues==
 
At some point, we need to figure out what segmentation means, where it fits in this architecture, and how it is effected.
 
  
The same applies for  object recognition.
+
Figure 2: iCub cognitive architecture implementation with YARP modules.
  
 
==Links==
 
==Links==
 +
 +
 +
===Cognitive Architecture Modules Documentation===
 +
 +
* [http://wiki.icub.org/iCub/dox/html/group__icub__actionSelection.html Action Selection Module]
 +
* [http://wiki.icub.org/iCub/dox/html/group__icub__affectiveState.html Affective State Module]
 +
* [http://wiki.icub.org/iCub/dox/html/group__icub__attentionSelection.html Attention Selection Module]
 +
* [http://wiki.icub.org/iCub/dox/html/group__icub__attentionDistributed.html Attention Application]
 +
* [http://wiki.icub.org/iCub/dox/html/group__icub__controlGaze2.html Control Gaze Module]
 +
* [http://wiki.icub.org/iCub/dox/html/group__icub__endogenousSalience.html  Endogenous Salience Module]
 +
* [http://wiki.icub.org/iCub/dox/html/group__icub__episodicMemory.html  Episodic Memory Module] and [[Auto-associative Memory Specification | Episodic Memory Specification]] and [http://wiki.icub.org/iCub/dox/html/group__icub__episodicMemoryApplication.html Episodic Memory Application]
 +
* [http://wiki.icub.org/iCub/dox/html/group__icub__proceduralMemory.html  Procedural Memory Module] and [[Hetero-associative Procedural Memory Specification | Procedural Memory Specification]]
 +
* [http://wiki.icub.org/iCub/dox/html/group__icub__rectification.html  Rectification Module]
 +
* [http://wiki.icub.org/iCub/dox/html/group__icub__crossPowerSpectrumVergence.html Vergence Module] and    [http://wiki.icub.org/iCub/dox/html/group__icub__crossPowerSpectrumVergenceApplication.html Vergence Application]
 +
 +
 +
* See also the current draft [[iCub YARP module specifications]] and the [http://www.icub.org/doc/icub-main iCub brain] (current source code documentation).
 +
 +
 +
===Past Versions of the Cognitive Architecture===
  
 
* [[iCub cognitive architecture version 0.1]]
 
* [[iCub cognitive architecture version 0.1]]
 +
* [[iCub cognitive architecture version 0.2]]
 +
* [[iCub cognitive architecture version 0.3]]
 +
* [[iCub cognitive architecture version 0.4]]
  
The following links are to early versions of the iCub "software architecture", a design for an iCub application (i.e. a set of YARP modules) that approximated some of the elementary aspects of the [[iCub cognitive architecture]] which now supercedes them.  These early version have all now been deprecated, as has the title "software architecture" in this context.  Software Architecture now refers, as it originally did, to the [http://eris.liralab.it/wiki/Deliverable_8.3 YARP] system.
+
 
 +
===Past Versions of the Software Architecture===
 +
 
 +
The following links are to early versions of the iCub "software architecture", a design for an iCub application (i.e. a set of YARP modules) that approximated some of the elementary aspects of the [[iCub cognitive architecture]] which now supercedes them.  These early versions have all now been deprecated, as has the title "software architecture" in this context.   
 +
 
 +
The term ''Software Architecture'' now refers, as it originally did, to the [http://wiki.icub.org/wiki/Deliverable_8.3 YARP] system.
  
 
* [[iCub software architecture version 0.1]]
 
* [[iCub software architecture version 0.1]]
 
* [[iCub software architecture version 0.2]]
 
* [[iCub software architecture version 0.2]]
 
* [[iCub software architecture version 0.3]]
 
* [[iCub software architecture version 0.3]]
* [[iCub software architecture version 0.4]]
+
* [[iCub software architecture | iCub software architecture version 0.4]]
 +
 
  
 +
===Summer School (VVV '09) Cognitive Architecture Group===
  
* See also the current draft [[iCub YARP module specifications]]
+
* [[VVV09 Cognitive Architecture Group]]
* [http://eris.liralab.it/brain iCub brain] - current source code documentation
 

Latest revision as of 16:04, 25 May 2018

The correct title of this article is iCub Cognitive Architecture. The initial letter is shown capitalized due to technical restrictions.

The Evolution of the Architecture

The iCub cognitive architecture is the result of a detailed design process founded on the developmental psychology and neurophysiology of humans, capturing much of what is known about the neuroscience of action, perception, and cognition. This process and the final outcome is documented in Deliverable D2.1: A Roadmap for the Development of Cognitive Capabilities in Humanoid Robots.


The architecture itself is realized as a set of YARP executables, connected by YARP ports. Early prototypes were developed at a RobotCub project meeting at the University of Hertfordshire in July 2007 as an exercise in consolidating the software development effort of the project partners. Several subsequent versions were produced at the RobotCub Summer School 2007 VVV '07. These prototypes were developed in parallel with the D2.1 Roadmap effort mentioned above. These two strands of design effort converged in the cognitive architecture shown below (Version 0.4). Previous versions can be accessed via the links at the end of the page. VVV '09 addressed the development of the architecture's (auto-associative) episodic and (hetero-associative) procedural memories.


The immediate purpose in developing the cognitive architecture is to create a core software infrastructure for the iCub so that it will be able to exhibit a set of target behaviours for an Empirical Investigations.

Icub cognitive architecture v0.4.jpg

Figure 1: iCub cognitive architecture, version 0.4.

Differences from previous version

  • Removed the tracker (should be handled by attention/salience sub-system)
  • Removed the face localization (should be handled by attention/salience sub-system)
  • Removed the hand localization (should be handled by attention/salience sub-system)
  • Removed the sound localization (should be handled by salience module)
  • Removed the attention selection
  • Added Exogenous Salience and Endogenous Salience
  • Added Locomotion
  • Added Matching
  • Added Auto-associative episodic memory
  • Added Hetero-associative procedural memory
  • Added Affective state
  • Added Action selection

Notes

  • Gaze implies 7 DoF: head and eyes
  • Locomotion paradigm: “go where you are looking”
  • Reaching paradigm: “reach where you are looking”
  • Endogenous and exogenous salience implies salience based on internal and external events, respectively
  • Gaze, reaching, and locomotion motor activities condition endogenous salience: i.e. motor states condition attention
  • Sensory inputs condition exogenous salience: i.e. attention conditions motor states
  • Episodic memory is memory of autobiographical events. Initially, this is purely visual and implemented as an auto-associative memory. Later it will be multimodal and will include sound as well as associated emotions. It will then have to be implemented as a hetero-associative network of unimodal auto-associative memories.
  • Episodic memory storage is conditioned by poor matching and high salience
  • Procedural memory is defined to mean perception-action event sequence
  • Procedural memory recall:
    • Event A & Event D inputs recall sequence of intermediate events
    • Event A input recalls Event B (subsequent event)
  • Affective state is a competitive network of three motives:
    • Curiosity (exogenous salience prevalent)
    • Experimentation (endogenous salience prevalent)
    • Social engagement (exogenous and endogenous salience balanced)
  • Action selection is not a winner-take-all process: one or more actions are disinhibited
  • The developmental drive is to construct a procedural memory that improves prediction

A more detailed description of the behaviour of each module and circuit in this architecture will be added in due course (both here on the iCub wiki and in Deliverable D2.1: A Roadmap for the Development of Cognitive Capabilities in Humanoid Robots).


YARP Module Implementation

The iCub cognitive architecture is implemented as a set of iCub YARP modules, as follows.


Icub cognitive architecture modules B.jpg

Icub cognitive architecture modules A.jpg


Figure 2: iCub cognitive architecture implementation with YARP modules.

Links

Cognitive Architecture Modules Documentation



Past Versions of the Cognitive Architecture


Past Versions of the Software Architecture

The following links are to early versions of the iCub "software architecture", a design for an iCub application (i.e. a set of YARP modules) that approximated some of the elementary aspects of the iCub cognitive architecture which now supercedes them. These early versions have all now been deprecated, as has the title "software architecture" in this context.

The term Software Architecture now refers, as it originally did, to the YARP system.


Summer School (VVV '09) Cognitive Architecture Group