We study how the representation of real-world object images is transformed along the ventral visual pathway. In early visual areas, the representation is somewhat like an image, laid out in retinal coordinates. In the high-level object representation in inferior temporal (IT) cortex, the representation is highly specialised for our natural visual eenvironment, emphasises behaviourally important categorical divisions like animate/inanimate and face/body and is less sensitive to changes in appearance of the same object when viewed from a different angle or at a different distance.

We view the representation at each stage as a neuronal population code whose representational geometry determines which images are represented as similar and which as dissimilar. We view neuronal computation as the transformation of this representational similarity structure across stages of processing. Our goal then is to characterise the representation at each stage and to find computational models (largely inspired by computer vision) that mimic the representational similarity structure in each brain region

We study the neurobiological bases of individual differences in personality and behaviour, which might be reflected in cognitive-level object representations in temporal and frontal regions. In the first project we will investigate how particular objects are represented in individual brains. We aim to provide a proof of concept and methodology for relating individual differences at the behavioural level to individual brain representations of mental content in cortical activity patterns.

We perform some of the key fMRI experiments on object vision in humans and nonhuman primates, using identical stimulus sets and closely matched tasks. We will then match up representationally analogous regions between human and nonhuman primates and determine which computational models best explain the empirical data for each functional region. This project links the human and monkey literatures and promises results that will help decide what questions require nonhuman primate studies (e.g. invasive cell-recording studies), what nonhuman-primate studies might be replaced with human experiments (e.g. using high-resolution fMRI and pattern-information analyses), and which nonhuman-primate results are likely to generalise to the human, thus enabling us to make better use of nonhuman-primate studies in the future. This project benefits from collaboration with John Duncan’s programme.

The ultimate goal of object vision research is to produce a computational model that can process natural visual stimuli and perform the recognition and inference tasks that human vision is capable of, while at same time explaining neuronal population activity and behavioural measures (such as reaction times and errors). We take models from computational neuroscience and computer vision and test them with brain-activity data. Our tests involve presenting the stimuli used in fMRI experiments to the computational models and comparing their internal representations to brain representations in different regions with representational similarity analysis.

This line of research develops the methodological basis of the neuroscientific studies. A key technique is representational similarity analysis (RSA). In RSA, the brain activity pattern associated with each stimulus is viewed as a “representation” of the stimulus in a neuronal population code. The dissimilarity matrix between the activity patterns elicited by a set of stimuli is used to characterise the geometry of the representational space. The representational dissimilarity matrix reflects to what extent stimulus differences are reflected or abstracted from in the population code. We develop statistical methods for comparing representations (by correlating dissimilarity matrices) between brains and models (to test if the models can account for the brain representations), between different brain regions (to understand the transformation of representational similarity across stages of processing), and between species (to understand homologies and functional analogies).

Our key techniques of pattern-information and representational similarity analysis rely on the reflection of neuronal population codes in fMRI hemodynamic patterns. High-resolution fMRI promises access to more fine-grained neuronal pattern information. However, increasing resolution in fMRI lowers the functional contrast-to-noise ratio. Moreover, it has been suggested that even standard-resolution fMRI (3-mm isotropic voxels) might reveal columnar-grain neuronal pattern information. Even at lower resolutions, fMRI voxel patterns might reflect sub-voxel-scale columnar patterns through complex spatiotemporal filtering (Kriegeskorte et al., 2009). This project has the pragmatic aim of determining the best acquisition scheme for pattern-information fMRI (Formisano & Kriegeskorte, 2012).

Deep convolutional neural net model explains IT object representational space in man and monkey

Faciotopy: The occipital face area might be a face-feature map with face-like topology

Object representations match closely between man and monkey

The primate-IT object representation is inherently categorical

Face identity may be represented in anterior IT, rather than in FFA

Ventral object representations can arise from the dorsal motion-flowfield representation

Information, not activation

Searchlight mapping

Representational similarity analysis

Voxel + venules = multipronged sensor?

Circular analysis

Topologically correct cortex segmentation

Nikolaus Kriegeskorte, Programme leader
Milena Vurro, Lab manager

Katherine Storrs, MRC core-funded postdoctoral researcher
Marieke Mur, Postdoctoral researcher, British Academy Fellow
Vassilis Pelekanos, Postdoctoral researcher, ERC Proof of Concept Grant
Tim Kietzmann, Visiting postdoctoral researcher, DynaVision – Representational Dynamics in Vision –  personal page
Jasper van den Bosch, Postdoctoral researcher, ERC Proof of Concept Grant
Rob Mok, Postdoctoral researcher, ERC Proof of Concept Grant
Xiaokang Liu, Programmer, ERC Proof of Concept Grant
Andrew Bell, senior invesigator (50% position), MRCccordion]

Jonathan O'Keeffe, PhD student, Wellcome Trust
Patrick McClure, PhD student
Courtney Spoerer, PhD student
Johannes Mehrer, PhD student
Lynn Sörensen, Master Student

The programme benefits from collaborations with a wide range of local, national, and international partners. These collaborations involve, on the one hand, common interests in visual perception and, on the other hand, application of RSA across perception, cognition, and action, and across species. Within the CBU, the programme has collaborations with PLs Henson, Duncan, Calder, Rowe, Carlyon, Hauk, Norris, Holmes, Pulvermueller, and Anderson, and postdocs Carota, Mitchell, Mur, and Wimber. In Cambridge, we collaborate with William Marslen-Wilson and Simon Baron-Cohen, and their groups. National and international collaborators include Jörn Diedrichsen at the University College London, Thomas Carlson at University of Maryland in the USA, Jack Gallant at Berkeley, Alexandra Woolgar at Macquarie University in Australia, David Feinberg at Berkeley, Leslie Ungerleider and Peter Bandettini at the National Institute of Mental Health, and Adam Anderson at the University of Toronto.

Matching categorical object representations in inferior temporal cortex of man and monkey
Kriegeskorte N, Mur M, Ruff D, Kiani R, Bodurka J, Esteky H, Tanaka K, Bandettini P. (2008). Neuron 60(6): 1126-41.

Representational similarity analysis – connecting the branches of systems neuroscience
Kriegeskorte N, Mur M and Bandettini PA (2008) Frontiers in Systems Neuroscience. doi:10.3389/neuro.06.004.2008.

Information-based functional brain mapping
Kriegeskorte N, Goebel R, Bandettini P. (2006) PNAS 103: 3863-3868.

Visual Population Codes – Toward a Common Multivariate Framework for Cell Recording and Functional Imaging
Kriegeskorte N, Kreiman G (Eds) MIT Press 2011

An emerging consensus for open evaluation: 18 visions for the future of scientific publishing
Kriegeskorte N, Walther A, Deca D. Front Comput Neurosci. 2012;6:94.

Open evaluation: a vision for entirely transparent post-publication peer review and rating for science
Kriegeskorte N. Front Comput Neurosci. 2012;6:79.

Blog  Future of scientific publishing