Enhance with super-resolution

If the original image needs to be improved to create a better master image from which the optimization happens, the API can enhance it using super-resolution.

This feature is useful if you want to produce up-scaled versions of images starting from a lower resolution image which is the best you have available.

Image enhancement

Image enhancement is the procedure of improving the quality and the information content of original data before processing. In a rough overview, the enhancement in Pixellena consists of two steps:

Step 1: Upscaling using super-resolution

The enhancement algorithm is Super Resolution GAN (SRGAN). The motive of this architecture is to recover finer textures from the image when we upscale it so that it’s quality cannot be compromised. There are other methods such as Bilinear Interpolation that can be used to perform this task but they suffer from image information loss and smoothing.

The algorithm is trained on the DIVK image dataset, with 4x up-scaling.

Step 2: Color correction and noise removal

After the upscaling, a color correction step and a noise removal steps are applied to the up-scaled image.

What parameters we use?

The input parameter for the master step is either identity or srgan-dt-x4.

The default value is identity, which basically means Do not change the image, use the original value. This is the algorithm to use if you are starting from a high-resolution image and want to shrink it down to some manageable size.

We will show here how to perform upscaling with the option srgan-dt-x4, take a look to the figure below that we are going to use in the example. It’s a jpg image named costume__200w.jpg with dimensions 200x200 pixels.

Image for upscaling with Pixellena super-resolution

You can use the SDK to do request to our API. Let’s show examples about how you can use super-resolution to enhance your images.

Enhancement basic example

We will show you how you can send a request to the API that uses super-resolution to enhance an image to 4x its size.

super_resolution_basic.js

import { LUX_API, Toilmore } from '@shimmercat/toilmore-sdk';
import fs from "fs";
import { Readable } from "stream";

let toilmore = new Toilmore(
    {
        'api_config': LUX_API,
        'api_token': 'xxxx-xxxx',   // <-- Use a valid API token here.
        'domain': 'domain.com'      // <-- Use a domain connected to the token here. 
    });

let adjustments = {
  	"master": {
    	"algorithm-name": "srgan-dt-x4"
  }
}

let force_processing = false;

let result_promise =  toilmore.optimize_with_precursor(
    "costume__200w.jpg",
    "prjpg",
    force_processing,
    adjustments
);

result_promise.then((result) => {
        // `result` can contain a short status string or null if the image
        // could not be optimized.
        store_optimized_image(result);
    },
    (err) => {
        console.log('Optimized images is not ready yet. Waiting 10 seconds...');
        // Let's wait a bit
        setTimeout(() => {
            store_optimized_image(result);
        }, 10);
    }
);

function store_optimized_image(result) {
    let w = fs.createWriteStream("./costume__enhanced.webp");
    result.pipe(w);
}

You should put the image file costume__200w.jpg and the super_resolution_basic.js script in the same directory, install the SDK as explained in the SDK tutorial, and then run:


node super_resolution_basic.js

Once you get the optimized image you will get the upscaled costume__enhanced.webp version of this image, and the resulting image is 800x800 pixels, see below.

Upscaled image using Pixellena super-resolution

We basically enhanced the image to 4x its size, and optimized it to the same format jpg. You can see at our lux API doc, that you can get optimized images for any of the precursor_name: webp0, prjpg, pngzo, jp2o0, avifo0.

Updated: