Covariant and invariant deep networks

When we humans interpret the visual information that reaches our eyes, our ability to maintain a stable perception of the environment, although the visual stimuli vary substantially on the retina due to geometric transformations and illumination variations, is essential for us to function robustly in a dynamic world.This project aims at developing new generically useful methodologies (i) for defining deep networks that handle natural image transformations by provable covariance and invariance properties and (ii) for handling temporal signals and spatio-temporal video using time-causal and time-recursive image operations, with the aim of designing more robust and computationally efficient as well as training efficient methods for visual recognition and video analysis.The project is organized into three tasks of (A) Scale-covariant and scale-invariant deep networks (Years 1-3), (B) Handling of temporal scales in time-dependent signals and video (Years 1-2) and (C) Time-causal and time-recursive video analysis (Years 3-4), with subgoals regarding theoretical analysis, algorithmic implementation and experimental evaluation as outlined in the project plan.The theories and methodologies developed in this way are intended as (i) general purpose primitives for building perceptual, cognitive or robotic vision systems or other tasks related to automated interpretation of visual information and (ii) computational tools for understanding possibilities for visual perception.