Object vision and population code representation

FP7-gen-RGB jauneObject vision and population code representation

The objective of this programme is to understand the computational and neural mechanisms of visual object perception in health and disease and their variation across individuals.

Object recognition is effortless for humans and animals, and yet it is one of the unsolved problems of artificial intelligence. Our central challenge is to understand the transformation of representations along the ventral visual stream, the level at which natural categories and semantic dimensions are represented and the computational mechanisms that enable the brain to perform object recognition so swiftly and more reliably than current computer vision systems.

We study representations of objects, places, and faces and higher-level semantic content in healthy volunteers and also in selected clinical populations, starting with autism. Our main methodology is high-resolution functional magnetic resonance imaging combined with pattern-information analyses.

 

jugment MDS & RDM_screen

Projects

Human ventral-stream object representations

We study how the representation of real-world object images is transformed along the ventral visual pathway. In early visual areas, the representation is somewhat like an image, laid out in retinal coordinates. In the high-level object representation in inferior temporal (IT) cortex, the representation is highly specialised for our natural visual eenvironment, emphasises behaviourally important categorical divisions like animate/inanimate and face/body and is less sensitive to changes in appearance of the same object when viewed from a different angle or at a different distance.

We view the representation at each stage as a neuronal population code whose representational geometry determines which images are represented as similar and which as dissimilar. We view neuronal computation as the transformation of this representational similarity structure across stages of processing. Our goal then is to characterise the representation at each stage and to find computational models (largely inspired by computer vision) that mimic the representational similarity structure in each brain region.

Individual differences in object representational space: Do they reflect disorders and personality?

We study the neurobiological bases of individual differences in personality and behaviour, which might be reflected in cognitive-level object representations in temporal and frontal regions. In the first project we will investigate how particular objects are represented in individual brains. We aim to provide a proof of concept and methodology for relating individual differences at the behavioural level to individual brain representations of mental content in cortical activity patterns.

Nonhuman-primate object representations and their functional analogies to the human

We perform some of the key fMRI experiments on object vision in humans and nonhuman primates, using identical stimulus sets and closely matched tasks. We will then match up representationally analogous regions between human and nonhuman primates and determine which computational models best explain the empirical data for each functional region. This project links the human and monkey literatures and promises results that will help decide what questions require nonhuman primate studies (e.g. invasive cell-recording studies), what nonhuman-primate studies might be replaced with human experiments (e.g. using high-resolution fMRI and pattern-information analyses), and which nonhuman-primate results are likely to generalise to the human, thus enabling us to make better use of nonhuman-primate studies in the future. This project benefits from collaboration with John Duncan’s programme.

Testing computational models of object recognition

The ultimate goal of object vision research is to produce a computational model that can process natural visual stimuli and perform the recognition and inference tasks that human vision is capable of, while at same time explaining neuronal population activity and behavioural measures (such as reaction times and errors). We take models from computational neuroscience and computer vision and test them with brain-activity data. Our tests involve presenting the stimuli used in fMRI experiments to the computational models and comparing their internal representations to brain representations in different regions with representational similarity analysis.

Representational similarity analysis

This line of research develops the methodological basis of the neuroscientific studies. A key technique is representational similarity analysis (RSA). In RSA, the brain activity pattern associated with each stimulus is viewed as a “representation” of the stimulus in a neuronal population code. The dissimilarity matrix between the activity patterns elicited by a set of stimuli is used to characterise the geometry of the representational space. The representational dissimilarity matrix reflects to what extent stimulus differences are reflected or abstracted from in the population code. We develop statistical methods for comparing representations (by correlating dissimilarity matrices) between brains and models (to test if the models can account for the brain representations), between different brain regions (to understand the transformation of representational similarity across stages of processing), and between species (to understand homologies and functional analogies).

Optimising pattern-information FMRI

Our key techniques of pattern-information and representational similarity analysis rely on the reflection of neuronal population codes in fMRI hemodynamic patterns. High-resolution fMRI promises access to more fine-grained neuronal pattern information. However, increasing resolution in fMRI lowers the functional contrast-to-noise ratio. Moreover, it has been suggested that even standard-resolution fMRI (3-mm isotropic voxels) might reveal columnar-grain neuronal pattern information. Even at lower resolutions, fMRI voxel patterns might reflect sub-voxel-scale columnar patterns through complex spatiotemporal filtering (Kriegeskorte et al., 2009). This project has the pragmatic aim of determining the best acquisition scheme for pattern-information fMRI (Formisano & Kriegeskorte, 2012).

 

Findings

 

Research team

Senior staff and postdocs


Katherine Storrs, MRC core-funded postdoctoral researcher
Marieke Mur, Postdoctoral researcher, British Academy Fellow
Vassilis Pelekanos, Postdoctoral researcher, ERC Proof of Concept Grant
Tim Kietzmann, Visiting postdoctoral researcher, DynaVision – Representational Dynamics in Vision –  personal page
Jasper van den Bosch, Postdoctoral researcher, ERC Proof of Concept Grant
Rob Mok, Postdoctoral researcher, ERC Proof of Concept Grant
Xiaokang Liu, Programmer, ERC Proof of Concept Grant
Andrew Bellsenior invesigator (50% position), MRC

Students


Jonathan O’Keeffe, PhD student, Wellcome Trust
Patrick McClure, PhD student
Courtney Spoerer, PhD student
Johannes Mehrer, PhD student
Lynn Sörensen, Master Student

Previous members

Linda Henriksson, Visiting postdoctoral researcher, Aalto University Fellow
Marta Correia, MRC staff scientist
Yara van Someren, MA student
Olivier Joly, Postdoctoral researcher, ERC
Arjen Alink, Postdoctoral researcher, British Academy Fellow
Hamed Nili, PhD student, MRC
Alexander Walther, PhD student, Gates Foundation
Seyed Mahdi Khaligh-Razavi, PhD student, Cambridge International Scholarship Scheme
Ian Charest, MRC core-funded postdoctoral Researcher
Johan Carlin, Postdoctoral researcher, British Academy Fellow

Collaborations

The programme benefits from collaborations with a wide range of local, national, and international partners. These collaborations involve, on the one hand, common interests in visual perception and, on the other hand, application of RSA across perception, cognition, and action, and across species. Within the CBU, the programme has collaborations with PLs Henson, Duncan, Calder, Rowe, Carlyon, Hauk, Norris, Holmes, Pulvermueller, and Anderson, and postdocs Carota, Mitchell, Mur, and Wimber. In Cambridge, we collaborate with William Marslen-Wilson and Simon Baron-Cohen, and their groups. National and international collaborators include Jörn Diedrichsen at the University College London, Thomas Carlson at University of Maryland in the USA, Jack Gallant at Berkeley, Alexandra Woolgar at Macquarie University in Australia, David Feinberg at Berkeley, Leslie Ungerleider and Peter Bandettini at the National Institute of Mental Health, and Adam Anderson at the University of Toronto.

 

Publications

Full list of publications

Three key publications

Matching categorical object representations in inferior temporal cortex of man and monkey
Kriegeskorte N, Mur M, Ruff D, Kiani R, Bodurka J, Esteky H, Tanaka K, Bandettini P. (2008). Neuron 60(6): 1126-41.

Representational similarity analysis – connecting the branches of systems neuroscience
Kriegeskorte N, Mur M and Bandettini PA (2008) Frontiers in Systems Neuroscience. doi:10.3389/neuro.06.004.2008.

Information-based functional brain mapping
Kriegeskorte N, Goebel R, Bandettini P. (2006) PNAS 103: 3863-3868.

Book

Visual Population Codes – Toward a Common Multivariate Framework for Cell Recording and Functional Imaging
Kriegeskorte N, Kreiman G (Eds) MIT Press 2011