Authors:

Naci, L., PULVERMULLER, F., Taylor, K., SHTYROV Y., HAUK, O. & Tyler, L.

Reference:

Year of publication:

2006

CBU number:

6453

Abstract:

A central issue in cognitive neuroscience is to determine the time-course with which sensory inputs from different modalities are transformed and integrated into coherent conceptual representations. With respect to processing objects, it has been suggested that auditory and visual object features are analyzed within hierarchically structured sensory processing streams from sensory-specific cortex to superior and inferior temporal cortex, and integrated in antero-medial temporal regions (Simmons & Barsalou, 2003; Taylor et al., 2005). To determine the time-course with which different processing components occur, we carried out an 128-channel EEG study using the same auditory-visual stimuli as used in Taylor et al. (2005)’s erFMRI study. Subjects performed a one-back same/different cross-modal identity task on auditory, visual, and cross-modal (audio-visual) stimuli. We found an amplitude enhancement of early visual (P1) and auditory (N1, P2) ERP components in the cross-modal condition compared to each unimodal condition, and compared to the summed (auditory and visual) condition. Between 0-200ms from stimulus presentation, multiple dipole localisation revealed significant, and recurring, cross-modal interactions in the posterior and anterior sources including the left/right occipital temporal, left/right superior temporal, left/right anterior-ventral temporal, and left inferior frontal cortices. In the left anterior temporal source, cross-modal interactions started from 160ms and were modulated by semantic factors, such as stimulus congruency (congruent vs. incongruent), suggesting that conceptual semantic processes occur early in the object recognition process. These data suggest that the recurrent interplay between bottom-up (sensory-specific cortex) and top-down processes (frontal, antero-temporal cortex) may underpin cross-modal object recognition.