Your website publishes thousands of articles each day. Your writers create stories, embed images, and tag them for SEO purposes. It’s your job to share them out on social media… but you’re struggling to keep up with the volume.

After coming up with a snappy tagline, you still have to select the best image and crop it to different sizes for Facebook, Twitter, LinkedIn, and all the other networks. Using a batch image-cropper might remove something important from the photo — like Elon Musk’s face, or half of the car being featured — so you put in a lot of time cropping and resizing by hand.

What if you had an automated way of handling the image picking and cropping process? Well, there’s now an algorithm for that. Today we’ll talk about how we’ve managed to bring together many different algorithms into a single ensemble that can intelligently select, crop, and score images for social media sharing.

How does it work?

Let’s look at the things you already have:

  • A blog post or news article that you’re ready to share
  • Optionally, some tags the author added for SEO purposes
  • A few images, preselected by the atricle’s creator

To use SocialMediaImageRecommender, you simply provide it with the article’s text and/or SEO tags, and at least 2 images you want it to use. You also tell it what image size (width & height) you’d like it to produce.

Once it has this information, the algorithm fires up a bunch of other tools automatically for you:

  1. StanfordNLP/SentenceSplit and nlp/LDA extract tags from the article’s text, combining them with any SEO tags you’ve manually specified.
  2. translation/GoogleTranslate detects the language of the content, translating it to English if needed.
  3. media/ContentAwareResize intelligently resizes the images to the required dimensions, keeping important objects (faces, etc) in the picture while cropping out unimportant background space
  4. deeplearning/InceptionNet inspects the images and extracts tags which describe the content
  5. nlp/Word2Vec calculates the similarity score between each image’s extracted tags and the provided text / SEO tags

The algorithm then returns a list of images back to you, ranked by their scores, and smart-resized to the dimensions you need. With one click, you know which of your images is best to use in your social post, and have a copy of it in the sizes you need.


The longer your article or blog post is, the better the algorithm will perform. This will give the algorithm more text tags to work with so that it can make better decisions about how well each image relates to the post.

Text length won’t significantly affect the running time of your algorithm, but adding more images will. We recommend increasing algorithm runtime if you’re going to provide more than 5-10 images.

Code Sample

Here we’re using a sample food article. We’ll also use the images that come with the article to come up with the best image for sharing on Facebook.

import Algorithmia
client = Algorithmia.client("YOUR_API_KEY")
input = {
  "text": "Not long ago, I watched a woman set a carton of Land O’ Lakes Fat-Free Half-and-Half on the conveyor belt at a supermarket. Can I ask you why you’re buying fat-free half-and-half? I said. Half-and-half is defined by its fat content: about 10 percent, more than milk, less than cream. Because it’s fat-free? she responded. Do you know what they replace the fat with? I asked. Hmm, she said, then lifted the carton and read the second ingredient on the label after skim milk: Corn syrup. She frowned at me. Then she set the carton back on the conveyor belt to be scanned along with the rest of her groceries. The woman apparently hadn’t even thought to ask herself that question but had instead accepted the common belief that fat, an essential part of our diet, should be avoided whenever possible. Then again, why should she question it, given that we allow food companies, advertisers and food researchers to do our thinking for us? In the 1970s, no one questioned whether eggs really were the heart-attack risk nutritionists warned us about. Now, of course, eggs have become such a cherished food that many people raise their own laying hens. Such examples of food confusion and misinformation abound. This country will never have a healthy food supply, said Harry Balzer, an NPD Group analyst and a gleeful cynic when it comes to the American food shopper. Never. Because the moment something becomes popular, someone will find a reason why it’s not healthy. Here, Balzer used the most dangerous term of all: healthy. We are told by everyone, from doctors and nutritionists to food magazines and newspapers, to eat healthy food. We take for granted that a kale salad is healthy and that a Big Mac with fries is not. I submit to you that our beloved kale salads are not healthy. And we are confusing ourselves by believing that they are. They are not healthy; they are nutritious. They may be delicious when prepared well, and the kale itself, while in the ground, may have been a healthy crop. But the kale on your plate is not healthy, and to describe it as such obscures what is most important about that kale salad: that it’s packed with nutrients your body needs. But this is not strictly about nomenclature. If all you ate was kale, you would become sick. Nomenclature rather shows us where to begin.  ‘Healthy’ is a bankrupt word, Roxanne Sukol, preventive medicine specialist at the Cleveland Clinic, medical director of its Wellness Enterprise and a nutrition autodidact (They didn’t teach us anything about nutrition in medical school), told me as we strolled the aisles of a grocery store. Our food isn’t healthy. We are healthy. Our food is nutritious. I’m all about the words. Words are the key to giving people the tools they need to figure out what to eat. Everyone’s so confused. Last March, the Food and Drug Administration sent the nut-bar maker Kind a letter saying their use of the word healthy on their packaging was a violation (too much fat in the almonds). Kind responded with a citizens’ petition asking the FDA to reevaluate its definition of the word. If I may rephrase the doctor’s words: Our food is not healthy; we will be healthy if we eat nutritious food. Words matter. And those that we apply to food matter more than ever. Kraft cheese slices cannot be called cheese but must be labeled cheese food or a cheese product. Pringles cannot be called chips but rather crisps. Yet packaged foods can be labeled natural or all-natural — what exactly is the difference between the two, anyway? — with little regulation. Here is a word we think we understand: protein. Protein is good, yes? Builds strong muscles, has positive health connotations. That’s why protein shakes are a multibillion-dollar business. Pork cracklings do not have positive health connotations because we think of them as having a high fat content. But pork cracklings are little more than strips of fried pig skin. Skin is one of the many forms of connective tissue in all animal bodies and is composed almost entirely of protein, typically undergirded by a layer of fat. When these strips of pig skin are fried, most of the fat is rendered out and the connective tissue puffs, resulting in a delectable, crunchy, salty crackling. I therefore recommend them to you as a protein snack during your on-the-go day. Given the infinitely malleable language of food, it’s no wonder American food shoppers are confused. What is mechanically separated meat, a standard ingredient in the turkey bacon and chicken sausages popularized because of our low-fat love? Do you know what that is? a grocery store owner asked me. They basically put poultry carcasses in a giant salad spinner. Whatever winds up on the walls of the spinner in addition to meat — bits of cartilage (protein!), nerves (I have enough of my own, thank you), vessels, bone fragments — is scraped off and added to the mixing bowl. Mechanically separated meat engages our imagination only when someone attaches new words to it, such as pink slime. Refined is another critical food word. Generally, refined means elegant and cultured in appearance, manner or taste, or with impurities removed. Yet that is what food companies have been calling wheat from which the germ and bran have been removed, leaving what is in effect pure starch, devoid of the fiber, oils, iron and vitamins that make wheat nutritious. That’s not refined, Sukol said, that’s stripped. Flour stripped of the nutrition that makes it valuable to our bodies but reduces shelf life. Because it has been stripped, we must enrich it. Enriched. Fortified. Good, yes? To make rich, to make strong. Food companies added the iron they took out during the refining process, but not enough of what we need. Refined flour — this resulted in B vitamin and iron deficiencies, Sukol said, so they added vitamins and iron. And what do they call that? Enriched and fortified. But they forgot to add folate, vitamin B9, until the 1990s. What we don’t know, Sukol said, is how those additions, not to mention the diglycerides and sulphates, combined with the lack of fiber, will affect our metabolism in the long run. So far, she said, it has resulted in diabetes and metabolic syndrome. We will be healthy if we eat nutritious food. Our food is either nutritious or not. We are healthy or we are not. If we eat nutritious food, we may enhance what health we possess. This is not a judgment on what you choose to eat. If you hunger for a cheese product grilled between bread that’s been stripped of its nutrition, along with a bowl of Campbell’s tomato soup (made with tomato paste, corn syrup and potassium chloride), fine. It was one of my favorite childhood meals. Just be aware. Buy fat-free half-and-half if that’s what you like; just know what it is you’re putting in your body and why. Because, and this is the judgment call, fat isn’t bad; stupid is bad. And until we have better information and clearer shared language defining our food, smart choices will be ever harder to make. Ruhlman is the author of many books, including Ruhlman’s How to Roast and the recent collection of novellas In Short Measure. He blogs and can be reached at He will join Wednesday’s Free Range chat at noon at Correction: A previous version of this article mistakenly said that refined wheat has had the endosperm and bran removed. In fact, the germ and bran have been removed, and the endosperm remains.",
  "images": [
  "dimension": {
    "height": 630,
    "width": 1200
algo = client.algo("web/SocialMediaImageRecommender/0.1.3").set_options(timeout=300, stdout=True)
res = algo.pipe(input)
print res.result

Image Recommendations:

  "recommendations": [
      "original_image": "",
      "score": 26.883455970727177,
      "social_image": "data://.algo/media/ContentAwareResize/temp/04c2c56e-d65f-452c-bc3d-144f27dd5511.png"
      "original_image": "",
      "score": 26.62695843199379,
      "social_image": "data://.algo/media/ContentAwareResize/temp/199f556c-1010-4822-af79-454b93dbd2e2.png"
      "original_image": "",
      "score": 24.925008693776423,
      "social_image": "data://.algo/media/ContentAwareResize/temp/4b951169-fdbd-4920-919f-0ddcf5d0abd9.png"
      "original_image": "",
      "score": 24.743368895267537,
      "social_image": "data://.algo/media/ContentAwareResize/temp/46ba2132-ecf3-4c80-b6d0-c46e7bce2a2a.png"

To find the recommended images, go to your Hosted Data at Algorithmia, and then look for the media/ContentAwareResize folder within the Algorithm data section. The output files will appear in that folder.


Social Media Image Recommender is an algorithm that utilizes the power of multiple deep learning algorithms to recommend the best image for sharing on social media. It uses an ensemble of algorithms to extract topic keywords from text, classify the given images, and return the most relevant image for social media sharing. Plus, it automatically works on multiple languages too!

The magic and heavy lifting is handled by the Algorithmia API, so you can just focus on developing your app without worrying about scalability issues.

Please let us know what you think @algorithmia on Twitter!

A. Besir Kurtulmus