Our robot will learn "affordances", sort of.
It will infer an object from its appearance. Then it will learn how that appearance predicts how the object responds to actions directed towards that object.
What it will learn is a mapping from object and action to consequence. We propose to represent the approximately 10d object, action, state mapping using and modifying the GMM library already in iCub. GMMs have the advantage of quickly learning high dimensional non-linear mappings. For motor actions we modify the Action Primitives library.
Vision, state estimation and data association will be done with IQR and some other bits (guys?).
As a final demo, it will play "golf" with the object to get it to a target location - hopefully it will do this at above random capability after learning with the object for a bit.
We'll be modifying ActionPrimitivesExample or CHRIS equivalent and exposing the interface to fiddle and fumble around!
Module name: iFumbly
Module port: /iFumbly/rpc:i [fidl] * x (double): target good ol' x [m]. * y (double): target good ol' y [m]. * d (double): the hand distance to target used initially [m]. * a (double): the hand angle used initially (frame?) [degrees]. * v (double): the reference velocity [m/s]. * fa (double): the final hand orientation (abs/rel frame?) [degrees].
Module motto: "Fee-fi-fo-fum"
This is the Learning interface, not to be confused with the [Trajectory learner and replayer].
You can send this interface N-dimensional datapoints from which it will a build probability density function over the N-dimensional space. It will learn in an ongoing fashion, rebuilding its pdf representation as it needs to - but this will be opaque to you, the user, who will merely send datapoints on one port and ask for inferences on another port. The only thing you can't do is ask for inferences before enough data has been sent (about 3 datapoints).
The inference occurs when you give it partial vectors and it infers a simple probability distribution over the remaining dimensions using Gaussian Mixture Regression - and returns the mean of this distribution.
- On this port you send the learner bottles of the form "DATAPOINT" ndim dim_1_value dim_2_value ... dim_ndim_value
- Should not block too long... debugging that...
- To this port you can request an inference to be done by sending a bottle of the form:
- "INFER" ndim comp_dim_1 comp_dim_2 ... comp_dim_ndim input_val_1 input_val_2 ... input_val_ndim
- So you have to give it a list of those dimensions being provided with the input vector, as well as those dimensions values.
- Infer will then give one of the following responses:
- "NOTREADY" (if the learner hasn't learnt anything yet)
- "UNKNOWN COMMAND" (if someone is being silly, or)
- "REGRESSED" ndim output_val_1 output_val_2 ... output_val_ndim
The latter interface has not yet been tested, and the former seems to be a bit slow - perhaps the semaphores are issumatic?
Behaviours not yet implemented (but will be, via RPC):
- save-dataset name
- load-dataset name
- save-distribution name
- load-distribution name