Welcome to the

Vision and Perception Science Lab Homepage

Neural Models of Visual Information Processing, Computational Vision and Learning


Prof. Heiko Neumann

Institute of Neural Information Processing
Faculty of Engineering and Computer Sciences
Ulm University
Oberer Eselsberg
D-89069 Ulm
Germany
Tel. +49 (0)731 50-24158
Fax +49 (0)731 50-24156
email: heiko.neumann@uni-ulm.de

General research overview ( Vision Science @ Ulm University)

The research of our group focuses on the investigation of mechanisms and the underlying structure of visual information processing in biological and technical systems (biological and machine vision) as well as their adaptation to changing environments (visual learning). The topics of our investigation primarily focus on mechanisms of visual information processing (see details in section Research interests and facilities below). The neural computational mechanisms that we use for the modeling of generating complex visual behavior and adaptation also transfer to related processes such as auditory processing and the control of (invertebrate) motor pattern generation.

Our research program utilizes an integrative approach based on the analysis of empirical data derived from psychophysics, neuroscience and imaging studies, and the mathematical and computational investigation of the underlying neural processes. We conduct psychophysical experiments in our computational perception lab where we utilize 2D and 3D Computer Graphics methods for synthetic image generation to create test stimuli and sequences. Such data aims to simulate complex visual displays and animations, e.g., for spatio-temporal grouping, surface perception, or spatial navigation in various environments. We also conduct experiments using head and eye tracking facilities. Eye tracking particularly enables us to record eye movement traces and length of fixations deployed to scenic objects. The results of such experimental investigations (i) further guide our modeling to gain insights into the computational mechanisms of brain function and (ii) advances application oriented investigations into, e.g., attention processes for feature selection.

The results of neural modeling gain new insights into the computational mechanisms underlying complex brain function and steer the development of new approaches and mechanisms for computational vision, image processing and (space-variant) active vision. These developments contribute new methods to several application areas such as, e.g., vision in perceptual and attentive man-computer interfaces, biometric systems, automotive technologies, medical image analysis and recognition, and visually-guided robotics. The development of future emergent technology in human-computer interaction (HCI) aims at developing advanced mechanisms endowing computers with more human-like capabilities and performance. In order to provide a cross-disciplinary forum of joint research activities we have founded an interdisciplinary Competence Center Perception and Interactive Technology (PIT) (together with the Institutes of Media Computing and Information Technology, respectively, at Ulm University). Here, researchers and technology are brought together from various scientific disciplines such as, e.g., computer science and engineering sciences, medical, psychological and neurosciences.

A brief summary of our overall research statement and recent research topics of our group can be found on the Vision Science @ Ulm University poster.



Research interests and facilities

Modeling mechanisms of static surface perception
Surfaces of environmental objects create structure in the ambient light patterns that enable visually guided interaction with the environment, such as, e.g., avoiding obstacles, grasping objects, or locating and identifying members in a social group. In order to develop a coherent model of core mechanisms and their interaction we investigate, a,ong others, the neural mechanisms that underlie the detection of features, the grouping of extended boundaries, or texture segregation, and the regularization of such low- and mid-level vision processes. We particularly pursue an approach that highlights the importance of feedforward and feed-back connections in neural network architecture.

Modeling mechanisms of motion perception
The neural mechanisms underlying motion segregation and integration still remain unclear to a large extent. Local motion estimates often remain ambiguous (aperture problem) when lacking form features, such as corners or junctions. Even in the presence of such features, local motion estimates may be erroneous if they were generated by mutual spatial occlusions. We have developed a neural model of visual motion inetgration based that robustly disambiguates locally conflicting cues. Building upon these local motion estimates large-field optical flow patterns are extracted to generate distributed representations of neural activation that, in turn, can be used to control complex visual tasks such as 3D egomotion.

Neurodynamical modeling and perceptual psychophysics
We study the computational processes underlying perceptual and cognitive mechanisms at different levels of abstraction, namely the dynamic behavior of ensembles of neurons with their spatio-temporal firing patterns and as well as the dynamics at the system level represented by the mean activities of model neurons (firing rates). The mathematical modeling and computer simulations of such models help to understand the mechanisms and functions underlying visually guided behavior in biological systems. These results are further utilized to design new bioinspired and neuromorphic technology to enhance, e.g., robotic and human-computer interface technology. We also conduct psychophysical experiments in our perception laboratory to investigate behavior in human perception and cognition. The data is again used to steer the modeling process.

Modeling mechanisms of perceptual learning
We investigate mechanisms underlying the perceptual learning of mechanisms for complex feature extraction as well as velocity extraction. Furthermore, we will assess the influence of response modulation through learning by utilizing large-scale neural network modelling. The mechanisms considered here are biologically realistic learning approaches, namely, Hebbian learning implemented by spike-timing dependent plasticity (STDP). The results of these computational investigations in turn have profound clinical impact on investigations for the improvement of perceptual capabilities in rehabilitation.

Modeling attention, visual search and studying eye movements
We investigate the role and neural mechanisms of spatial attention in the visual perception of static and moving patterns. Within the framework of neural architecture of feedforward and feed-back processing, we model the generation of response patterns that are consistent with the biased competition hypothesis of attention. We also employ eye tracking studies to monitor the active motor processes involved in the overt gaze shifting to deploy focussed attention to different locations in a scene. These facilities also provide a technology to provide a service for companies, external customers and other research groups to conduct selected experiments in usability and human-computer interaction.

Modeling detection of view direction and guided attention
The visual detection of human heads and the associated estimation of head pose and view direction provide core mechanisms to enable the assessment of visual communication processes between members of a group. Based on experimental evidence we investigate the underlying processes of a more specialized neural system that is devoted to analyzing faces and their component. The goal is to identify some necessary specializations in the processing to acquire task-relevant data, such as, e.g., head orientation and eye gaze.

Space-variant vision
An important characteristic of the primate visual system is the space-variant mapping of the visual field onto the cortex (with a small central region of highest resolution (fovea) and decreasing resolution towards the periphery). This, in turn, necessitates eye movements allowing rapid deployment of the high-resolution fovea to interesting regions of the environment. Space-variant active vision is investigated for fixation, smooth pursuit as well as investigating the advantages of reduced sampling at the same time keeping a wide field of view.

Applications - Computer vision, image processing, and pattern recognition
We transfer biologically inspired mechanisms and models to algorithms that can be applied in computer vision scenarios, as well as image processing, and pattern recognition and classification approaches in various domains. The main pplication domains are in, e.g., the medical, automotive, and human-computer interaction research areas.


Current research projects and collaborations

The computer as dialogue companion – Perception and interaction in multi-user scenarios
(Funding: State of Baden-Wuerttemberg)
The project aims at developing novel mechanisms and principles of systems architectures for intelligent human-computer interaction and evaluate these by using psycho-physical and psycho-biological test methods. We aim at the creation of advanced user interfaces with extended perceptual and interactive capabilities utilizing adaptive mechanisms and their ability to learn. Such systems should be capable of analyzing the spatio-temporal and user-specific context of interaction, possibly with multiple partners.
(Link: http://www.informatik.uni-ulm.de/pit)

Sensor-based situation estimation and adaptive human-automotive interaction in multi-vehicle scenarios
(Funding: State of Baden-Wuerttemberg)
The project aims at developing methods to evaluate the status, complexity and potential danger of the current traffic situation on the basis of context representations. Intentions and plans of other traffic participants will be considered by integrating information gained from inter-vehicle communication regarding the status of other vehicles. Information and behavioural strategies for the driver-vehicle interaction will be derived and adapted accordingly. The adaptive speech-based driver-vehicle interface will be triggered by the complexity of the current traffic situation as well as the status and attentiveness of the driver.
(Link: http://www.informatik.uni-ulm.de/pit)

Neural Decision-Making in Motion (Decisions-in Motion)
(Funding: European Union, EU FP6 IST Cognitive Systems integrated project, no. 02 71 98)
The research goal of the project "DECISIONS-IN-MOTION" is to describe the neural mechanisms used to guide behaviour in complex visual scenes, in which the living (or animated) agent is in motion and navigates to avoid stationary and/or moving objects. During reporting period P1 we have explored motion-based image segmentation in the visual cortex, and we have begun to derive neural models that explicitly make use of a hierarchy of sensory areas (low-, mid-, high-level visual areas) to extract meaningful information about the location and motion of objects in the environment. One objective of the project is to use the outputs of these units for sensory-based decision-making. This process will weight these inputs and relations between these inputs based on utility functions. The resulting cognitive architecture will be tested in an autonomous robot navigating in complex visual environments to determine the efficiency of the image motion segmentation and goal-directed adaptive behaviour.
(Link: http://www.decisionsinmotion.org/)


Members - Publications - Teaching

Vision group members (Visionaries)

Publications (per year)

Teaching - Course program


Former vision group members