Adaptive Visual Attention Based Object Recognition

Rebecca Fay, Ulrich Kaufmann, Heiner Markert, Günther Palm

When performing tasks in complex environments robots are likely to encounter objects they have not seen before and consequently cannot identify. Thus the ability to learn novel objects during run time is an essential skill for advanced mobile service robots. Another helpful skill is the ability to track known and unknown objects since changes in the visual scene are very common due to motion of the robot and of possible objects of interest. Moreover, knowledge about the position of an already localised or classified object reduces the necessity of recalculations for every new image. We present a multi-stage visual object recognition system that localises and identifies objects using an adaptive colour-based visual attention control algorithm and hierarchical neural networks for object recognition and is able to track the localised objects as well as to learn novel objects during run-time. The approach is evaluated in a test scenario where a robot is located in front of a table with different kinds of fruit and other simple objects on it. The robot has to localise and identify these objects as well as to perform a set of object manipulating tasks such as grasping, showing or moving specified objects. The experiments conducted showed encouraging results. New objects can be learnt with reasonable classification rates and in adequate time. The tracking of the objects allows for advanced object classification even on slower computers because classification is not exerted for every image.

Keywords:
Selective visual attention, object tracking, object recognition, hierarchical neural networks, adaptive incremental online learning.