Computer vision via Unlimited Sensing Framework

Digital devices play a crucial role in our lives and are foundational to the modern society. Since we live in an analogue world, there is a need for Analogue-to-Digital Converts (ADCs) to convert analogue information into its digital form. When implementing conventional ADCs in practice, system designers have to consider the trade-off between digital resolution (DRes) and dynamic range (DR), and it is impossible to optimize for both simultaneously.

The newly introduced Unlimited Sensing Framework (USF) allows to optimize for both at the same time by applying zero-centred modulo non-linearity in analogue domain before digitizing the input signal.

Most research on USF has focused on signal reconstruction, but information processing directly in the measurement domain remains largely unexplored. This project aims to investigate fundamental computer vision tasks such as image classification, object detection, image segmentation, optical flow on measurements acquired using USF pipeline. These tasks are well-studied for conventional acquisition but are not explored for non-linear USF acquisition.

The project will be mostly realized by simulations, which will involve designing new algorithms performing tasks on 2D modulo measurements.

Modulo non-linearity breaks the structure of the 2D signal at the folding locations by effectively adding more edges. The challenge in designing new algorithms is to distinguish edges inherent to data from the edges created by the modulo operation. The novelty lies in designing algorithms that operate directly in the modulo measurements domain, bypassing the need for signal reconstruction. Designed algorithms will involve both traditional and machine-learning-based approaches.

The application potential is broad, since many handheld devices are equipped with cameras. For example, USF-enabled cameras and relevant algorithms could improve AR game experience in the presence of bright reflections, locating people in the photo gallery or performing online search based on photos. Other application areas include autonomous driving, manufacturing facilities quality control, space applications for docking satellites and many others.

The project lies at the intersection of Digital Signal Processing, Non-Linear Systems, Image and vision computing.