Recovering a high-resolution (HR) image from a low resolution one is a classical problem in computer vision for which many algorithms have been developed to date. Most notably, methodologies using sparse coding: these techniques have achieved current state-of-the-art results, but suffer from long execution times, which makes them less attractive for real-time applications.
The HR reconstruction and recovery of images finds its applications in nearly all domains involving image acquisitions, such as pattern recognition, texture analysis, computer vision for medical devices, astronomy and so on. The fundamental challenge in HR recovery is that a solution, when it exists, is not unique. That is, an infinite number of mappings might exist between a low resolution image to its high resolution counterpart. Thus, the recovery problem is an under-determined inverse challenge.
High resolution image reconstruction with deep CNNs
The strategy to handle such ill-posed inverse problems is to place strong constraints on the prior information needed for recovery (e.g, the non-linear inverse transformations).
The prior information is usually learned by example-based methods, which take advantage of similarities within the image. To bypass such constraints, the parameters of the non-linear mapping from low to high resolution can be learned from a set of example images in the context of deep convolutional neural networks (CNNs). The typical architectures of a CNN for HR image recovery consist of very few layers (typically three) and a sequence of two convolutions, which are analogous to applying a sequence of non-linear filters to the low resolution image.
The training set for the CNN is particularly easy to generate, as it consists of a set of high resolution (ground truth) images, blurred and down-scaled. The CNN then learns the weights and biases of the convolutional layers using (stochastic) gradient descent, with the goal of minimizing the error between the transformed low resolution image (and/or overlapping patches of it) to its ground truth high resolution.
The use of deep CNN in non-linear denoising and restoration has rapidly replaced the use of bottom-up methodologies, due to its superiority in execution and overall quality of results. As a result, deep learning is today the key technique also in these super-resolution problems. The non-linear nature of the HR restoration problem makes deep CNN especially attractive for such task, which can learn by example based on the optimal (in some sense) transformation from low resolution to high resolution.
RSIP Vision builds tailor-made algorithmic solutions to our clients’ computer vision and image processing problems. Many of the first pre-processing operations performed on image involve increasing the overall quality of the image. For this end, RSIP Vision’s engineering employs state-of-the-art algorithmic solutions to be integrated into the solution that we ship to our clients. To learn more about RSIP Activities, please visit our project page. To discuss the best solutions for your project, contact our experts now.
