It’s not easy to dynamically scale and crop images of different sizes or aspect ratios. Typically, this is resolved in one of two ways: manipulate images by hand (slow, high quality, expensive), or indiscriminately crop-to-fit from the middle-out (fast, low quality, cheap).
Smart Thumbnail is a microservice developers can use to programmatically manipulate images at scale while ensuring that every image is perfectly cropped.
Since web services and social networks can’t employ hundreds of people to manually crop and resize images, they rely on “dumb” algorithms to generate these thumbnails. This is why we experience thumbnails throughout the web of decapitated people, mountains without peaks, or subjects missing from the frame entirely.
Here are two examples of Smart Thumbnail fixing images poorly cropped images:
What is Smart Thumbnail
Smart Thumbnail is a microservice that can automatically resize and crop images with the most relevant parts framed and retained.
This microservice takes advantage of both facial recognition with OpenCV, as well as a deep convolutional network for saliency predication to “learn” where in an image to scale and crop.
For example, the most relevant part of an image with people would probably revolve around their faces.
Smart Thumbnail will use OpenCV to recognize the faces in the image and then crops the image to your desired thumbnail size with the face in the frame. No more decapitated people.
But what if image doesn’t contain people? Good question!
Smart Thumbnail will recognize that, and then use saliency via SalNet, an implementation of the “Shallow and Deep Convolutional Networks for Saliency Prediction” research.
(The technical explanation goes like this: By default, Smart Thumbnail takes the average of the centroid of the largest face it detects, if one is detected, as well as the centroid of saliency in the image to determine where to crop. If no face is detected, Smart Thumbnail will use the centroid of saliency to figure out what the most important part of the image is, and ensure it’s retained when scaled and cropped.)
Saliency represents the most prominent part of an image, like a person in a painting. The visual representation of saliency could best be seen as a heatmap like this:
Whether or not your image contains people, places, or things, Smart Thumbnail can intelligently crop and resize your images into thumbnails.
Why You Need Smart Thumbnail
By using Smart Thumbnail, you ensure that thumbnails on your site are always perfectly cropped, even if the most salient object isn’t already centered.
When you leverage a microservice like Smart Thumbnail, you can scale your ability to manipulate images on your site programmatically, and avoid embarrassing thumbnails like the examples above.
Or, just look at this real life examples of a poorly cropped image:
Low quality thumbnails like this erode trust and undermine professionalism.
How to Use Smart Thumbnail
There are a few ways to leverage Smart Thumbnail in your apps and services. The API requires an array with an input URL, output URL, thumbnail width, and thumbnail height.
The input requires either a URL or an Algorithmia Data URI. Your output will be the binary representation of the image, unless you specify a destination for a file when making the API call. This will be saved as a PNG.
The height and width are the desired output size of the image.
To start making calls, you’ll need a free API key from Algorithmia.
Below is the sample input/output. We’ll be using this image for reference:
import Algorithmia input = [ "https://hd.unsplash.com/photo-1466840787022-48e0ec048c8a", "data://.algo/temp/test.png", 500, 500 ] client = Algorithmia.client('API KEY HERE') algo = client.algo('opencv/SmartThumbnail/2.1.1') print algo.pipe(input)
That was easy. Now you have a solution for dynamically generating thumbnails next time you’re working on a web development project, app, or service.
Give a try and let us know what you think @Algorithmia.