The Handbook of Multimodal-Multisensor Interfaces, Volume 1. Sharon Oviatt
Читать онлайн книгу.
using computer interfaces. Our brains are highly plastic in terms of integrating these bodily extensions into a pre-existing system, although they evolved to capitalize on relatively primitive forms of these instruments.
Glossary
Active behaviors: Overt movements performed by the observer. In this chapter, we restrict this definition to include intentional, or goal-directed actions (as opposed to reflexive actions).
Amodal stimulus properties: Properties of an object that can be conveyed by more than one sensory system [Lickliter and Bahrick 2004]. By some definitions, amodal also refers to information that is no longer connected to the sensory modality by which it was encoded.
Binding problem: Theoretical problem stemming from the required ability of the brain to bind sensory signals that trigger neuronal firing to the environmental stimulus from which those signals originate. This also entails the problem of how the brain combines signals from multiple senses into a unified percept, given that they detect different features of the stimuli.
Blood-oxygen-level-dependent (BOLD): The primary measure of interest in fMRI. BOLD computes the ratio of deoxygenated to oxygenated hemoglobin in the brain. The more oxygen an area is consuming, the more active its neurons.
Constructivism: A broad theoretical approach that considers the organism to be constructing meaning based on interactions with objects and people in the world.
Convergence: The unique property of neural connections by which more than one type of unisensory neuron connects to the same neuron and may independently evoke neural activity from that neuron. This can be contrasted with integration, whereby the sensory signals are combined to produce a single response based on more than one input. Convergence results in multiple separable signals in a given neuron, whereas integration refers to combining multiple inputs into a single response (see Figure 2.12).
Experience-dependent: Changes in the brain that are caused by experience, which may involve active or passive interaction with the environment.
Feedforward-feedback loop: Information travels in at least a bi-directional manner in the cortex and subcortex. Information affects, and is affected by, information included in these loops.
Functional connectivity: Statistical dependencies in time between spatially distinct neural regions. Used on a BOLD signal from fMRI, this method can result in revealing brain regions whose activities are correlated in time, leading to the inference that they are working together for that given behavior.
Functional magnetic resonance imaging (fMRI): A neuroimaging method broadly used in medical and psychological sciences to non-invasively observe extremely small changes in tissue properties, particularly changes in ferromagnetism associated with changes in the ratio of deoxygenated to oxygenated hemoglobin, that can be used to make inferences concerning the functioning of cortical structures.
Fusiform gyrus: A gyrus in the brain that lies on the ventral aspect of the temporal lobe, spans from the occipital lobe to midway along the temporal lobe and, is located approximately along the midline of the temporal lobe. In functional terms, it is associated with object perception and recognition.
Grounded cognition: A central concept embedded in a range of cognitive theories, including simulation theory, situated cognition, and embodied cognition that opposes the existence of abstract knowledge systems. Instead, they consider knowledge systems to be represented implicitly by sensory and motor mechanisms during interaction with the world.
Hebbian learning: A well-documented neural mechanism of learning in which axonal connections between neurons undergo activity-dependent changes. There are two basic tenants: (1) when two neurons repeatedly fire in a coordinated manner, the connections between them are strengthened, effectively increasing the likelihood of firing together in the future; and (2) when two neurons repeatedly fire in an uncoordinated manner, the connections between them weaken, effectively reducing the likelihood of firing together in the future.
Integration: The unique property of some multisensory neurons by which they combine converging input signals from more than one sensory modality and output an integrated signal. Input may be from primary sensory organs, secondary regions, or association areas. The integrated output combines the signals into a single output. See also definition for convergence for a comparison (see Figure 2.12).
Intersensory redundancy: Recent theory of perceptual development that evades the binding problem by portraying the human perceptual system as a multimodal system. It does so by responding to unisensory and multisensory inputs with differential weightings: greater weight on multisensory input and less weight on unisensory input.
Lateraloccipital complex (LOC): A broad functional zone encompassing the anatomical regions of lateral occipital cortex and the ventral and lateral surfaces of temporal cortex. It is traditionally defined as cortex that is more active when presented with pictures of intact objects than pictures of textures or of scrambled objects.
Mental rotation: A form of mental imagery in which an observer rotates imaginary visual representations of two- or three-dimensional objects in their mind.
Modality-specific stimulus properties: Properties of objects that may only be detected by one sensory system. An example is wavelengths of light, which are only detectable by the visual system.
Multimodal (cognitive neuroscience meaning): Combined sensory and motor system input to the brain, for example, the interaction of vision and motor systems during encoding of an event. On the other hand, passive viewing of visually presented stimuli is not generally considered multimodal, because it can be accomplished without motor input or input from other sensory systems.
Multimodal-multisensory: See multimodal and multisensory definitions.
Multisensory: The involvement of at least two sensory modalities in a given process. For example, vision and haptics are combined and provide multisensory input to the brain when we see and touch objects. Multisensory processes are most often multimodal. However, multisensory processes can occur without motor input. For example, temporally coincident visual and auditory cues need not be accompanied by head or eye movements.
Multisensory depression: Information about objects and events in the environment gleaned by relying upon spatiotemporal disparities across more than one sense (e.g., the visual experience of color is not always coincident with tactile cues), which leads to a depressed (decreased) neural response. That is, the neural response to the same multisensory input in the future will be reduced. This process effectively increases the response to modality-specific stimulus properties, such as color, because it decreases the physiological response to multisensory signal combinations, such as simultaneous visual-auditory stimuli.
Multisensory enhancement: Information about objects and events in the environment gleaned by relying upon spatiotemporal coincidence of more than one sense (e.g., vision and haptics that coincide during visually guided touch), which leads to an enhanced (increased) neural response. That is, the neural response to the same multisensory input in the future will be increased. This process effectively increases the response to amodal stimulus properties, such as visual-haptic cues, because it increases the physiological response to amodal stimulus properties. Multisensory enhancement is distinct from Super-additivity (see Chapter 1 of this volume), which occurs when a multisensory response is greater than the sum of the unimodal responses. Multisensory enhancement provides a developmental mechanism for multisensory perception whereas supper-additivity describes the neural response of a population of neurons responding to multisensory stimuli.
Multisensory neurons: A neuron that responds to a stimulus from more than one