Encoding to final format using quality measures

Pixellena allows you to perform image format conversion and optimization, while preserving image quality.

This is useful if you want to take a quality-based approach rather than a compression-based approach when it comes to image optimization.

Output image formats

There are two types of image formats that Pixellena can generate: (1) lossy or (2) lossless.

For lossless formats, we use the best encoder available to us, but the resulting image is faithful to the output from the shifter pixel-by-pixel. It’s more complicated for lossy formats, because lossy encoders can generate more or less distorted images depending on quality parameters passed to it.

Therefore, when generating a lossy image, the encoder uses an image similarity index - a quality measure - to measure the distortions induced by the encoding process, and via a binary search, it selects the quality settings that generate the smallest image that it’s above the quality threshold (a number between 0.25-1.00 where 1.00 is original quality).

What parameters we use?

There are three input parameters for the encoder step:

• quality-measure: The quality measure to use.
• qual-threshold: The quality threshold. The encoder will try to produce the smallest image whose quality (as defined by the quality measure) is above this quality threshold.
• max_overhead: Defines the maximum size of the resulting image as the value of this parameter times the size of the original image.

Note: The API will produce a deny status response if the resulting image is bigger than the original image size times the value of the max_overhead parameter.

Quality measures

The possible quality measures to use when encoding an image to its final format are:

• default: Let’s our platform select a default measure

• perceptual-sc: This is a simple neural-network approach that tries to mimic the human capability to notice image compression artifacts. Best used when transcoding thumbnail-sized JPEG images to WEBP

• ssim: Structural Similarity Index: This is a pretty traditional measure. More information about it at the Wikipedia page and in the original paper

• fsim-c: Phase congruency image similarity measure. This is one of the top-ranked algorithms when it comes to perceptual quality, see more info here

• asymmetric-fsim-c: This is a modification of fsim-c that gives less weight to compression artifacts in uniform-color backgrounds

Quality threshold

The quality threshold is used to define the lower bound for the quality of the ouput image in relation to the input image.

The minimum value of the threshold is 0.25 and the maximum 1.

• For qual-threshold == 1 the resulting image will have the same quality - as defined by the quality measure - and the smallest size possible.

If the threshold is smaller than 1, then the encoder will try to produce the smallest image whose quality - as defined by the quality measure - is above this quality threshold.

The max_overhead parameter

The max_overhead is a number that specifies the maximum for how much the original file size can be increased, as a proportion of the image’s original size. The default value is 0.99, minimum 0.1, and maximum 10.

The workings of the parameter can be best explained using an example:

• Say that you have an input image original.png of size 100 kB
• With the default value of max_overhead = 0.99 the maximum size of the encoded output image will be 100*0.99 = 99 kB. This means that if the output image is larger than 99 kB, the API will return a deny status
• If you edit the value and have for example max_overhead = 2.0, then the API will return the image if the resulting size is smaller than 100*2.0 = 200 kB

The rationale behind this parameter is that when serving an image for visualization in a browser, a smaller file size is (almost) always preferred.

Encoder basic example

We will show you how you can send a request to the API that uses the encoder with a specific quality measure, quality threshold and value for the max_overhead parameter

import { LUX_API, Toilmore } from '@shimmercat/toilmore-sdk';
import fs from "fs";
import { Readable } from "stream";

let toilmore = new Toilmore(
    {
        'api_config': LUX_API,
        'api_token': 'xxxx-xxxx',   // <-- Use a valid API token here.
        'domain': 'domain.com'      // <-- Use a domain connected to the token here. 
    });

let adjustments = {
    "encoder": {
        "quality-measure": "fsim-c",
        "qual-threshold": 0.90,
        "max_overhead": 3
    }
}

let force_processing = false;

let result_promise =  toilmore.optimize_with_precursor(
    "hoodie1.png",
    "webp0",
    force_processing,
    adjustments
);

result_promise.then((result) => {
        // `result` can contain a short status string or null if the image
        // could not be optimized.
        store_optimized_image(result);
    },
    (err) => {
        console.log('Optimized images is not ready yet. Waiting 10 seconds...');
        // Let's wait a bit
        setTimeout(() => {
            store_optimized_image(result);
        }, 10);
    }
);

function store_optimized_image(result) {
    let w = fs.createWriteStream("./hoodie1_90.webp");
    result.pipe(w);
}

You should put the image file hoodie1.png (original size 1.1 MB) and the script encoder_basic.js in the same directory, install the SDK as explained in the SDK tutorial, and then run:

node encoder_basic.js

Once you get the optimized image you will get the hoodie1_95.webp version of this image. In the figure below we have run the script for different values of "qual-threshold", ranging from 0.90 to 0.99.

Figure 2: Encoding of hoodie1.png using quality-measure fsim-c, and qual-threshold ranging from 0.90 to 0.99. Note that the original image file size is 1.1 MB

qual-threshold: 90 Size: 20 kB

qual-threshold: 92 Size: 20 kB

qual-threshold: 94 Size: 52 kB

qual-threshold: 96 Size: 74 kB

qual-threshold: 98 Size: 100 kB

qual-threshold: 99 Size: 230 kB