deeplearning

deeplearning / SalNet / 0.2.2

README.md

1. Introduction


The salience (also called saliency) of an item – be it an object, a person, a pixel, etc. – is the state or quality by which it stands out relative to its neighbors. Saliency detection is considered to be a key attentional mechanism that facilitates learning and survival by enabling organisms to focus their limited perceptual and cognitive resources on the most pertinent subset of the available sensory data.

This deep learning algorithm automatically detects salients for you in a given image.

Input:

  • (Required) Image Data API Url, Web (http/https) Url, binary image or a base64 encoded image.
  • (Optional) Output image location. (key="location")
  • (Optional) Saliency matrix save location. (key="saliencyLocation")

Output:

  • Image output location.
  • Saliency matrix output location.

Note: The first call to this algorithm will take a bit longer than sequential calls to due algorithm initialization. All following calls will be significantly faster.

Note: The saliency matrix is a pixel-by-pixel salience representation of the image. Each pixel can have a value between 0-255, in which higher numbers correspond to higher salience. The matrix is saved as a list of lists, as in each child list represents rows of pixels in the image.

Saliency Matrix Example:

[
  [0, 0, 0, ..., 167, 177, 220],
  [0, 0, 0, ..., 156, 177, 210],
  .
  .
  .
  [111, 107, 101, ..., 0, 0, 0],
  [102, 129, 105, ..., 0, 0, 0],
]

2. Examples

Example 1.

  • Parameter 1: Data API Url
{
    "image": "data://deeplearning/example_data/mona_lisa.jpg"
}

Output

{
  "output": "data://.algo/temp/mona_lisa.png"
}

Example 2.

  • Parameter 1: HTTPS Url
{
    "image": "https://s3.amazonaws.com/algorithmia-assets/algo_desc_images/deeplearning_SalNet/mona_lisa.jpg"
}

Output:

{
  "output": "data://.algo/temp/mona_lisa.png"
}

Example 3.

  • Parameter 1: Base64 image
{
    "image": "data:image/png;base64....",
}

Output;

{
  "output": "data://.algo/temp/output.png"
}

Example 4.

  • Parameter 1: HTTPS Url
  • Parameter 2: Output file save location.
{
  "image": "data://deeplearning/example_data/mona_lisa.jpg",
  "location": "data://.algo/temp/test42.png"
}

Output:

{
  "output": "data://.algo/temp/test42.png"
}

3. Credits

For more information, please refer to https://github.com/imatge-upc/saliency-2016-cvpr or Junting Pan, Kevin McGuinness, Elisa Sayrol, Noel O'Connor, Xavier Giro-i-Nieto (2016). Shallow and Deep Convolutional Networks for Saliency Prediction. arXiv preprint arXiv:1603.00845v1.

Demo image were retrieved from Wikipedia under Public https://en.wikipedia.org/wiki/File:Mona_Lisa_(copy,_Hermitage).jpg