Super-resolution

Since the very beginning of computer-based imaging, there has been a lot of work dedicated to digitally improve image quality. Thanks to the growing popularity of Artificial Intelligence, super-resolution (SR) technologies have become more prominent. These technologies can be used in several areas and industries, from movies and photos to video games and even medical imagery. Let’s dive into the fascinating world of super-resolution and all the concepts involved!

The challenge: Create high definition images from low quality pictures

The main challenge behind super-resolution is being able to turn low quality pictures into high definition images. This increase in quality involves not only the pixel count, but also some extra details such as texture quality and anti-aliasing among other things. Several techniques and approaches are used to achieve the results and the main challenge today is to make it as accessible and optimized as possible.

What is Super-resolution (SR)?

Super-resolution, also referred to as “upscale,” “upsize,” “up-convert,” or “uprez,” is a term for a set of imaging techniques aiming to turn a low resolution image or video into something with a higher resolution. The topic was originally mentioned in the mid 80’s and the term “Super-resolution” itself appeared at around 1990¹. However, this idea has gained more popularity over the years and has become particularly useful today. Although many approaches have existed, they roughly share the same basis consisting in regenerating the missing pixels out of a low-resolution image thanks to special algorithms. One way to find the missing information is to use multiple images and therefore try to extract details from every image in a sequence to reconstruct the other frames. This usually results in an image which dimensions are larger than the original’s. This approach can be taken a step further by incorporating Artificial Intelligence concepts. This allows to develop advanced functions based on Machine Learning. The program then gets “trained” with no actual user input. It receives several low-quality and high-quality images. It then upscales those low-quality images and compares the results with the original high-quality versions. The program then learns from its mistakes and is eventually able to generate automatically the missing pixels from virtually any image. There is also another math-based approach: interpolation. This approach relies more on approximations based on a set of data. It allows to work on a single image instead of many, and is often used in photography. It uses relatively complex math concepts allowing to calculate the missing values based on tendencies drawn from the actual data we have.

One technology applicable in multiple industries

What makes super-resolution so popular today is thanks to its versatility:

• In video games, image manipulation has been omnipresent for many decades already with techniques such as anti-aliasing, which consists in making lines smoother. With the release of new image-rendering technologies and ultra high resolution monitors becoming more and more accessible, SR techniques help game developers lower the hardware requirements of their video games. Nvidia’s DLSS is a great example of this: “DLSS leverages a deep neural network to extract multidimensional features of the rendered scene and intelligently combine details from multiple frames to construct a high-quality final image.” Nvidia basically takes super-resolution one step further by incorporating Artificial Intelligence and other advanced technologies to deliver high quality graphics at a lower cost hardware-wised.
• In photography, super-resolution techniques are already applied in several ways that can be observed daily. They are used to reduce noise for instance by taking multiple shots with different exposures then merging these shots into a perfectly balanced image. This technology is often referred to as “HDR.” All modern photo editing software are equipped with a lot of similar features and rely heavily on these techniques. This also includes the ability to add or remove existing items on a picture or recolor an image.
• As for videos, modern televisions and playback devices such as Blu Ray players use SR techniques a lot. This allows them to turn play in 4K movies that weren’t actually recorded in 4K. In this case, both interpolation and image sampling techniques are used.

How businesses can benefit from super-resolution techniques

While super-resolution techniques are nowadays ubiquitous without people actually noticing it, they can be used in a smart by individuals but also companies to improve their offers and potentially reduce their costs. The most straightforward use cases are for people and companies posting photos of their products online. In this case, these techniques allow to increase the resolution of these shots and even adding some extra effects to make banal pictures look very professional. There are quite a few tools in the market able to perform such tasks, includingPixellena.

Super-resolution is definitely not another buzz word but a passionating topic. It has a lot of uses in our daily lives that aren’t limited to the most complex use cases like the ones based on machine learning. It is getting more and more accessible as many devices incorporate these concepts natively and a lot of tools are released and they keep getting better. As a consumer with no particular knowledge in advanced technology topics, now is a great time to take advantage of all these technologies, and this is one of many features incorporated in ShimmerCat.

¹ M. Irani and S. Peleg, Super resolution from image sequences, in 10th International Conference on Pattern Recognition, vol. 2, Jun. 1990, pp. 115120.