• Home :

SENSOPAC is one of the most ambitious projects explicitly joining neuroscience, informatics and robotics. Neuroscience has been directed toward the main goal of extracting salient computational rules to be applied to robotic control and, in doing so, has made relevant progresses on its own. There have been four laboratories working on neuroscience – EMC (Netherlands), Pavia (Italy), Lund (Sweden), and Rehovot (Israel) – addressing the core issue on how the cerebellum elaborates signals coming through the main afferent pathways. The cerebellum is a key brain structure comparing commands to their execution, with the main outcome of regulating movement and learning how to correct execution errors. By doing so, a third fundamental operation is expected to be the implicit reconstruction of the concept of an object. The basis of these computations have to reside into the elementary operations of the network, which have been addressed in detail. During SENSOPAC, we have defined (1) the nature of input spike patterns to the cerebellum, (2) the operation mode of the main cerebellar neurons, (3) the properties of the synapses therein, and (4) the  learning rules at these synapses. Moreover, we have (5) defined relationships between these synaptic properties and elementary behaviours like eye-blink conditioning and VOR. Finally, (6) the results have been translated into computational models and elaborated in order to be applied to robotic control systems.  At the technological level, major achievements have been recordings form cerebellar neurons in vivo, network recordings using voltage-sensitive dies and multi-electrode arrays, chronic recordings during behaviour.

The current picture of the cerebellar network function is that signals come as bursts and are then processed generating specific temporal sequences  limited by the inhibitory circuit and regulated by synaptic plasticity (Time window – matching hypothesis). During repetitive activation, the network fragments time sequences generating coherent oscillations on the theta-band (Theta resonance/oscillation hypothesis). Local non-linear dynamics lead to reorganize cerebellar activity depending on the input patterns  (Spatio-temporal filtering hypothesis) allowing for the emergence of elementary operations of coincidence detection and spatial pattern separation. Following this first stage, which occurs in the granular layer, activity is then processed by the Purkinje cells, which fine-tune their basal discharge frequency and generate the final output.  Alterations in this pathway (mossy fibers, granule cells and Purkinje cells), obtained by genetic manipulation, cause alterations in sensory-motor adaptation. Implementation of these principles into large-scale network simulations is now allowing to testing their impact on the robotic control.

Within SENSOPAC, advanced statistical machine learning techniques act as a glue, translating fundamental principles from observed biological motor control and neurophysiological experiments into computational principles that can be used for much more sophisticated and robust control of anthropomorphic robotic systems. Novel actuator designs and antagonistic principles require that control has to be appropriately adapted to the redundancies. Moreover, control under multiple contexts is something biological systems achieve seamlessly. The fundamental aim of the machine learning driven techniques of the project is to automatically extract sensor-dynamics contingencies indexed on context, come up with appropriate low dimensional representations of these contingencies and exploit this both for context identification and control -- all this in a purely data driven fashion.

We have developed and applied (1) advanced, online non-parametric techniques for acquiring actuator dynamics from data in real time settings. Importantly, we have developed (2) techniques that can identify switching of contexts and apply appropriate learned models for robust tracking and control of complex, high dimensional anthropomorphic movement systems. Contexts have been explored under various increasing levels of sophistication, such as (a) different loads, (b) different inertial properties etc. We have developed a framework where the object dynamic properties can be used to identify and switch contexts. (3) Context switching has been realised in ODE based full physics simulation of robotic limbs and we are in process of extending this analysis to high density tactile sensor arrays. (4) One of the key challenges is 'How to extract low dimensional context parameters from complex, multi dimensional and non-linearly transformed sensor arrays' like the tactile sensors. Various dimensionality reduction techniques have been explored to tackle these issues and Information Theory based machine learning methods are being used to (i) identify maximally informative and discriminative movements (ii) identify and disambiguate context with this and (c) use this for efficient control of antagonistic robotic arms. (5) There is also significant effort into compartmentalised modelling of the various cerebellar substrates (or micro-complexes) such that LSAM models can be used to generate a biologically realistic implementation of this context switching paradigm.

For more information see Part 2.

Biological musculoskeletal systems are antagonistic. Each joint is activated by a group of muscles, pulling the bones via tendons. Apparently for reasons of linearisation of the dynamics, this architecture is rather complex: e.g., for planar motion of the elbow and shoulder only, the human arm uses a total of 19 muscles, forming a complex of agonist, antagonist, monoarticulate and biarticulate muscles. From a biological point of view, there are many reasons to use antagonistic drive principles for generating motion. Of course, a strong one is the fact that muscles, responsible for vertebrate motion, can only pull and not push. But apart from that restriction, the antagonistic approach leads to two important advantages: the system is energy-optimal for various tasks, and the joints are intrinsically flexible.

In order to obtain dynamical properties very similar to musculoskeletal systems, we therefore consider antagonistic drive principles of key importance for future robotics. Using this principle, DLR is constructing within SENSOPAC a revolutionary hand-arm system which mimics the kinematic and dynamic properties of the human hand-arm complex as closely as possible. Using biological drive and joint concepts, this new hand-arm system consists of a hand with 19 active degrees of freedom (DoF) attached to a robot arm with 7 active degrees of freedom, all with weight, size and strength properties equal or close to that of humans.

DLR's approach to the combination of precision of movements with the compatibility of the system for its integrated use in the human body is a hand-arm-system where the architecture and coordination of bones and muscles will be designed as in a human hand along with similar flexibility and adaptability.

To reach the same accuracy and speed in finger and hand movements we have realised a precise kinematic model of the human hand and its respective bone and joint positions in grasping and other movements, generated from in vivo MRT-data that takes into account every single bone and its 3D orientation in space in relation to the time of each grasping action or other movement.

This newly researched detailed kinematic model enables us to design a hand-arm-system in realistic size weight and architecture that will not only move as precise as the original but also be able to perform with similar efficiency as the essential basis for its integration in human.

Detailed results can be found in Part 3.