Virtual Reality for cognitive neuroscience & electrical neuroimaging
Virtual reality (VR) technologies offer various solutions to simulate real or imaginary places and to allow people to explore actively these virtual environments (VE). VR covers a broad range of display and sensing technologies, including; 3D real time rendering, stereoscopic display with projector or head mounted displays (HMD), motion capture of one or several body parts, and integration of multimodal interaction devices.
At LNCO, we use VR to place experimental subjects in pseudo-ecological conditions (illusion of reality especially of the human body) while manipulating precisely the spatio-temporal aspects of stimulation. We work with different stereoscopic (perception of depth in 3D) visualization systems, with audio (such as spatial sound), and with technologies for tracking the body in real time and in a wide range of static and dynamic conditions (turning, walking, sitting, standing, lying, etc). We have setup multiple VR solutions to satisfy the different needs and constraints of our experimental work:
VR capture room; simulation with full body motion tracking
* Large rear projection screen (~4.0×2.5 m).
* Active Optical tracking system (Reactor, capture area: 3.0*3.0*2.4m) for real time full-body tracking subjects.
* Also equipped with a treadmill & cameras.
Desktop VR; simulation with upper-limbs motion tracking
* 50″ stereoscopic TV (active glasses)
* 3 DOF orientation sensors (Freespace, OS5000) and stereoscopic camera tracking (home made) to track the head or hands of subjects
Rotating chair embedded VR; simulation during full body rotation
* 26″ stereoscopic monitor (active glasses NV 3D Vision)
* Optical high speed eye tracker (EyeLink 2000)
* Bidirectionnal communication with control PC (serial, ethernet)
Portable VR; simulation in first person perspective
* Small field of view ( VReal Viewer) or large field of view HMD (Fake Space)
* Cameras (mono, stereoscopic, or hybrid 3D encoder VirtualFx 3D Converter)
* Can be combined with a tracking system
All the above systems are compatible with EEG recording. Particular care is taken to avoid electromagnetic noise as every system is embedded in a Faraday cage (IAC isolation chambers or home made).
These hardware platforms are open to various software solutions to create and generate the real time simulation (E-Prime, Presentaiton, LabView, Motion Builder, etc.). In addition, we develop an in-house tool for the experimental design of VR-based experiments; ExpyVR is a graphical tool to create and edit an experiment (accessible to non-programmers) and the pylnco toolkit offer a standardized process for the development of hardware or experimental specific components (for Python programmers).
Functional magnetic resonance imaging (fMRI) is a neuroimaging technique which provides high resolution, non-invasive measures of neural activity detected by a blood oxygen level dependent (BOLD) signal. The increased blood flow to the local vasculature that accompanies neural activity in the brain results in a corresponding local reduction in deoxyhemoglobin because the increase in blood flow occurs without an increase of similar magnitude in oxygen extraction. Deoxyhemoglobin acts as an endogenous contrast enhancing agent, and serves as the source of the signal for fMRI. This technique is characterized by high spatial resolution (up to 1x1x1 mm3), but poor temporal resolution (2-3 s).
To carry on its research, the LNCO uses the most up-to-date MR technologies provided by the Centre d’Imagerie BioMédicale (CIBM), including a 3T scanner equipped with a 32-channel head coil, located at the University hospital (CHUV), and a 7T scanner located in the EPFL campus.
During LNCO investigations, MR scanners are often used in conjunction with MR-compatible robots which have been designed in collaboration with robotics labs both at the EPFL (LRSO, Prof. Bleuler) and ETH (RELab, Prof. Gassert). Robotic devices guarantee an exquisite time and spatial precision in controlling the stimulation and, combined with neuroimaging techniques, are a powerful tool for exploring brain functions.
256 Channel Virtual Reality-EEG
Our Virtual Reality (VR) facility is equipped with an Immersive Virtual Reality system,combined with a 256-channel EEG recording system (BioSemi. Inc Netherlands). Our VR setup contains an Active Optical tracking system(Capture area: 3.0*3.0*2.4m) for real time tracking of the experimental subject. We use Stereo3D Projection on a large Passive Rear Projection screen (~4.0×2.5 m).We have also incorporated the Head mounted device(Iglasses video3D Pro) which offers the capability to view interlaced 3D Video in True Stereoscopic 3D,in tandem with the 3D encoder (VirtualFx 3D Converter) which converts images into holographic like 3D projection in real time. We are currently exploring this state of the art VR technology in combination with electrical neuroimaging to investigate the functional and neural mechanisms involved in embodiment, self consciousness, and “presence”.
200 Channel EEG
Evoked potential mapping (or EP mapping) has the great advantage of measuring electrophysiological brain signals with a temporal resolution in the range of milliseconds. EP mapping is based on the examination of the global electric field at each moment in time recorded from 200 scalp electrodes. Any neuronal activity in the brain generates electric current flow. Since the brain is a volume conductor, the sum of all currents at any given moment in time is expressed by an electric potential distribution on the scalp, i.e. a momentary electric field. The electroencephalogram is basically the recording of this electric field potential at discrete scalp positions. With sufficient electrodes, the surface electric field can be reconstructed (so-called EEG/EP Mapping). Any change of the configuration of this field over time or between experimental conditions is due to a change of the active neuronal populations in the brain. EP mapping defines changes in map configuration over time and between conditions in our cognitive experiments. EEG/EP Mapping is combined with the applications of different source localization algorithms as well as intracranial recordings in epileptic patients (in collaboration wit Prof. M. Seeck) in order to localize the generators of the maps in human cortex.
Neuropsychology studies the behavioral effects of brain damage and electrical cortical stimulation on human brain functions (such as language, body knowledge, visuo-spatial cognition) and phenomenological states (illusions, hallucinations, delusions). In comparison to neuroimaging data that present correlative evidence for the implication of a given area at a given time for a certain brain function, neuropsychological data provide causal evidence for the implication of a given area for a certain brain function. Neuropsychology thus has the power to demonstrate whether a brain region is necessary for a certain function.
We use classical tests such as paper and pencil type tests, computer-based stimulus presentation, and virtual reality techniques (under development). We use recording of eye movements, recording of arm and body movements in order to improve reliability and accuracy, also to making the testing more enjoyable for the patients.Neuropsychological techniques are complemented by neuroimaging techniques such as functional MRI and evoked potential mapping. For example, patients with brain injuries may take part in neuroimaging studies to investigate the effects of a lesion on the functioning of other, undamaged areas of the brain.
Vestibular cues provide information about head movement in space and the orientation of gravity. They are implicated in postural control (body stabilization/orientation), gaze stabilization (vestibulo-ocular reflex) and verticality judgments (such as visual verticality and own body verticality). There is increasing evidence that vestibular cues influence many cortical areas (parieto-insular vestibular cortex, superior temporal gyrus, inferior parietal lobule, etc.) and that this vestibular cortex is implicated in spatial cognition. Data from our Laboratory suggest that disturbed cortical vestibular processing is also an important factor in causing illusory own body perceptions, embodiment and self location (Blanke et al., Brain 2004; Blanke and Mohr, Brain Research Reviews 2005).
We use high resolution EEG to understand how the vestibular cortex responds to natural vestibular stimulations (dynamic: rotating chair; static: whole body tilt) and artificial vestibular stimulations (galvanic vestibular stimulations with electrodes placed on the mastoid processes; saccular stimulations with short tone bursts). One of the main focuses of the Laboratory is to determine how these natural and artificial vestibular stimulations can influence subjects’ performance in spatial tasks such as mental rotation of visual objects, body imagery, and detection of biological motion. We also perform behavioral studies to understand how vestibular cues are integrated with visual and somatosensory cues in brain-damaged patients with visuo-spatial deficits (such as visuo-spatial neglect), and how vestibular stimulations interact with the performance of neurological patients in high-level spatial tasks. We are currently investigating the optimality of visual and vestibular cue integration within a probabilistic model based on Bayes’ theorem. More specifically, we are interested in how top-down cognitive factors such as habituation, prior experience or experimentally induced priming might interfere with optimal sensory integration.
Psychophysical methodologies are commonly used to explore the relationship between the observer’s psychological states, assessed via their responses in a given task, to finely controlled manipulations of the physical stimulus.
Stimuli are carefully chosen to target only the specific perceptual processes of interest. In the Laboratory, we investigate in such a way visual and audio-visual biological motion perception. Different thresholds for different stimuli can help to reveal why some stimuli are easier or harder to perceive than others. Such results can inspire new computational and neural models of perceptual processing in the brain, and further test their predictions.
Psychophysical experiments can be run using only a computer and a good display and important insights into brain function can be obtained just by testing small groups of normal healthy adults. However, also here the power is enhanced if psychophysics are combined with neuroimaging techniques such as evoked potential mapping and functional MRI, to observe how the activity of specific areas in the brain is correlated with observer’s performance as a function of the stimulus.
Lesion analysis techniques have been developed to study common brain regions involved in a group of patients with a particular neurological syndrome. The neurological patients are recruited at the Department of Neurology (Prof. T. Landis) at the Geneva University Hospital. We use several types of procedures:
1. Overlap analysis using segmentation of ‘native’ (un-normalized) brain images using anatomical landmarks in collaboration with Dr L. Spinelli ( Geneva University Hospital).
2. Overlap analysis using segmentation, normalization, statistical parametric mapping (SPM), and voxel-based symptom mapping in collaboration with Dr L. Spinelli (Geneva University Hospital).
3. Development of new quantitative ways of lesion mapping in collaboration with the Laboratory of Laboratory of Signal processing V (Prof. J.-P Thiran).
These methods have recently been applied to the study of illusory own body and self perceptions, auditory verbal hallucinations, as well as motion blindness and motion deafness.
We developed a behavioral task to measure the boundaries of peripersonal space in humans, based on audio-tactile and visuo-tactile interactions. For a complete description of the set up, please follow the the link: