Several web sites interesting for my domain, including:
Some prospective texts about:
Researches related to Neuroscience at INRIA
written end of 2006.
Several project-teams are directly implicated in researches related to Neuroscience:
- In the theme BIO: ANUBIS, ASCLEPIOS, DEMAR, ODYSSEE, VISAGES
- In the theme COG-A: CORTEX, MAIA, MISTIS, SELECT
- In the theme COG-B: PERCEPTION
- In the theme COG-D: I3D
- In the theme COM-D: ALCHEMY
- In the theme NUM-A: APICS, e-MOTION
- In the theme NUM-C: OMEGA
while others are interested in this topic because they share common formalism or methodology, or they could collaborate to or be end-users of such researches:
Modelling what ?
- elementary mechanisms: activities of individual neurons and of populations of different sizes (microscopic, mesoscopic and macroscopic scales), binding and synchronization, emergence of properties, self-organization
- sensor responses: fMRI, DMRI, MEG, EEG, Optical Imaging, Multi-electrode recordings, other emerging techniques such as Confocal and Two-Photon Microscopy
- integrated physiological, behavioral, perceptive-motor functions of the central and peripheral nervous system in humans and other species. Modelling visual perception is of particular interest: 3D and binocular vision, eye movements and visual attention, visual-auditory integration, visual-motor loop. Also its interaction with other cognitive functions such as speech, reading, reasoning, etc....
- disfunctions of the central nervous system, e.g. epilepsy, Parkinson, Alzheimer, etc....
Modelling how ?
- modelling from data is the major source of inspiration: parameter estimation, statistics and stochastic modelling, machine learning, signal processing and interpretation, inverse problem, identification, large distributed database construction and exploitation, multi-model and multi data source fusion
- modelling from knowldege is also considered: knowledge from various domains of Neurosciences but also from geometry (for biological vision modelling). New knowledge can also come from Virtual Reality and advanced interfaces that allow for original experiments.
- using related tools and formalisms: automatic control, applied mathematics and computer science: variational methods, optimization, statistics and probability and other approaches in machine learning, but also model checking and constraints programming; Tools also come from other domains: physiology, circuit design; all these skills require a multidisciplinary approach.
Structuration of activities at INRIA: Three double axes:
The scientific activity of the 15 project-teams directly implicated in researches related to neurosciences plus 9 other teams interested in this topic can be read as follows:
- Modeling of the neuro-physiological and neuro-psychological phenomena
- Bottom-up models of observed biological phenomena
- Top-down models of generic properties of the neuronal systems
- Measure of the cerebral activity and related neuro-signals
- For neurosciences investigations
- For clinical applications
- Implication of neurosciences in computer science
- Bio-inspired systems and new emerging computational architecture and paradigms.
- Human and brain computer interface, using neurosciences knowledge and technology in computer science.
Some comments about the present structuration:
- Such approaches avoid any epistemological mistake about "cognitivism" (<<the brain, as far as its intrinsic operations are concerned, does no information processing but is a specific biological organ and its specific neuro-biological processes cause specific forms of intentionality; in the brain, intrinsically, there are neuro-biological processes and sometimes they cause consciousness>> computational properties is not to be discovered in the nervous system, but assigned to it, in order to simulate some specific sensori-motor function).
- Modeling of phenomena observed in neurosciences is indeed to be realized in deep collaboration with biologists, this is already the case and is to be always increased in the future: association with neurosciences teams or medical teams gets deeper and deeper with time.
- Although weak versus strong interactions between computer and neuro sciences is to be clearly made explicit (e.g. 1.1 versus 1.2 or 3.1 versus 3.2) both complementary approaches are obviously mandatory to push the state of the art and INRIA research teams, all together, have this precious advantage to link both domains, at different scale, different level, etc..
- increase the fundamental knowledge in neurosciences (new theories, better behavioral and physiological data) and related medical fields and technologies (psychiatry, prosthesis, surgery, electrophysiology)
- develop new theories and models in control and computer science: new models of computation, asynchronous, adaptative and distributed computation, hardware implementation, circuits, new interfaces
- The special case of Virtual Reality (VR) and Human-Computer Interface (HCI): VR and HCI are of special interest: they can be very useful to get new experimental data (e.g., perceptive-motor loops, physiological data); they can be used in applications (e.g. brain computer interfaces) or rehabilitation procedures (e.g. cyber-therapies); their design can also benefit from such inspiration (ex:enhanced gaze-contingent displays, pseudo-haptic feedback, presence/immersion), etc..
Towards the next five years ?
- Now mature enough, it is proposed for the INRIA neurosciences modeling activity to refer more clearly to the neurosciences domains they contribute to: single neuron investigation (understanding the neural code), sensori-motor processes (particularly visual functions, the role of action on perception, multimodal integration), memory systems, phylogenetic vs ontogenetic approaches (e.g. evolution through species, developmental models), neuronal diseases understanding, etc. Accordingly, it is recommanded to choose the biological or clinical domains to collaborate with and to explain our impact in the domain.
- New tools in the measure of cerebral activity (e.g. extrinsic optical-imaging, high angular resolution diffusion imaging, actively controlled electrodes, ..) after recent techniques such as MEEG are emerging with strong needs in terms of signal and image processing, generating new challenges for teams working in dynamic and volumic image processing.
- Not only taking new modalities into account but also the challenge of realizing a "meta-neuronal-scanner" is a string perspective of the present works: since each modality has complementary properties, their integration and combination will not only add but multiply their capability to provide pertinent measures of the neuronal activity.
- The study of large assemblies of neurons provides a brute force computational tool, including at the hardware level, since chips simulating event-based neuronal networks 105 faster than real-time arrive soon. Without a deep understanding of their properties, tools to properly define their behavior, including at the architecture level, these developments will be vain.
- If perspectives for neuronal-computer-interface have to be confirmed using non-invasive tools (e.g. EEG), it is clear that new emerging technologies (e.g. functional electrical stimulation (FES), TMS, etc.) are to be considered and seriously understood for all their very promising (or dangerous, thus to be avoided !) applications.
- Non invasive brain-computer interface also really means to understand how the brain behaves in front of a computer interface (e.g. in virtual reality devices) in order to optimize such interfaces, provide accessibility to disable people, etc..
Perception, Action, Cognition at LORIA/INRIA-Lorraine:
written end of 2006.
Contribution by LORIA project-teams: CORTEX, MAGRIT, MAIA, MERLIN, and PAROLE.
Many modern technological systems integrate an increasing number of characteristics related to human perception, action and cognition. This is particularly important for systems dedicated to operate with human-like performances or to interact with human beings.
Modeling human perception has a long history in informatics. More precisely, computer vision and speech processing have been studied for a long time and offer a large variety of frameworks to study perception from sensors and emulate perception tasks. The underlying models are numerical models, like stochastic, multi-agent or neural models and are related to the geometry and the temporal evolution of signals. They are designed to translate the sensory signals into a more elaborated information that can be used by a decision module.
Primarily, action refers to the robotic domain, where decisions are often expressed by movements that might be complex to realize. More recently, action has also been considered as an interesting way to improve perceptive performances. This is for example the case with articulatory hints that can be used to disambiguate speech perception and with selective attention and other oculomotor strategies used to focus on specific parts of the visual scene or to integrate the position of the observer in the scene. The models can be built with explicit geometrical considerations or, when the geometry is too complex or not known, they can be inferred with implicit relations acquired by learning through experiences in the environment.
In the picture, cognition is certainly the widest field since it can correspond to implement very simple reactive agents, directly relating perception to action, up to more deliberative or communicating systems where a decision can depend on experience or other external context, including other agents or human beings. In between, cognitive strategies are necessary to emulate advanced perception tasks, where perceptive cues are the basis for the interpretation of the environment. Such adaptive strategies can be obtained by machine learning and other artificial intelligence techniques.
Interaction is an important keyword to introduce here: it can be related to the design of new sensorimotor loops, for example including multimodal perception, to offer more natural and simple interfaces to humans. It can also be a framework to describe and integrate the external context evoked above, implemented as interacting external agents. Both levels rely on modeling human behaviors, with physiological and cognitive aspects.
The domain of modeling and simulating humans in their environment can be related to the domain of complex systems. It can be also presented as a multi-model domain along several dimensions:
- Some models are explicit and describe a physical or physiological phenomenon with symbolic equations. Other models are implicit and learn from data when the explicit approach is not possible.
- A multi-domain approach must be used when the task is too complex to be tackled by a unique model. Instead, models of different nature must be designed for various aspects of the task, together with corresponding boundary conditions.
- A multi-model approach, combining several approaches, is also often used to increase recognition or discrimination performance
- The multi-modal nature of human perception, action and cognition is also a strong incentive to combine models of different kinds.
The major challenge in the domain is that human perception, action and cognition are deeply related one to the others. Accordingly, the three domains must be considered as a whole to design efficient systems.
In a domain known to generate many different kinds of models, an important challenge is to integrate and make collaborate models of very different natures. A prerequisite is to master the formalism of each model and to develop deeper models. This has also implications on the models themselves (conditions on the boundary), on data structures and software platforms to make models communicate and on the power of computation that has to be available.
Fundamental considerations must also be drawn from this integrative perspective. There is a need for theories coming from human modeling, when the corresponding knowledge is available and easy to integrate in models, to explain how the perceptive, active and cognitive dimension should be integrated. This can originate from brain theories or from more behavioral domains. It is important to develop such theories and to translate them for informatics. When the knowldge is missing, technological interactive platforms can also be used to elaborate realistic models of human behavior using empirical or experimental approaches.
Another fundamental point is related to the nature of computations: they are numerical, they are distributed along several domains, each one regarding only a subpart of the problem and they must respect time constraints (be consistent with the dynamics of interaction, try to relate phenomena not necessarily occuring at the same moment). Such considerations lead to look for the emergence of properties from the self-organization of distributed elementary computations, as it is for example the case with cellular automata. The substratum of computation and its possible distributed and asynchronous dimensions must be also considered as a model of computation but also as a way to embed such systems in adapted architectures.
The experimental dimension of such researches has also strong consequences on the association of the modeling phase together with simulation and interaction during the simulation. Adapted interfaces and other technological platforms like autonomous robots must be employed and interaction during simulation is only possible if adapted efficient computations are developped.
- Improve the efficiency and realism of human like information processing systems
- Develop more natural and easy to use interfaces for interacting with tomorrow smart artifacts
- Propose new knowledge about human functioning, on the physiological as well as cognitive side
- Master and combine a variety of models, explicit and implicit, and propose adequate computational formalism and architecture to overcome their complexity
- Validate the appropriateness of these multimodels by confronting them to the actual usage of interactive devices
- Integrate such a technology in tomorrow artificial assistants and realistic training systems, for the eldery, handicaped people and more generally for the citizen
Why should it be a part of the INRIA strategic plan?
Future technological systems will integrate more perceptive, active and cognitive components. A fundamental question is to know whether this progress will allow, at the same time, to increase knowledge in human sciences, to offer more natural and simple interaction with these systems for the users together with more efficient and robust performances. Our assumption is that this will be possible only if the various issues are considered as a whole.