Introduction

Face Recognition is a state-of-the-art deep learning algorithm that can train on human faces and recognize them later.

The novelty of this algorithm comes from the fact that it can train on as little as 10 images per person to achieve accurate predictions. Training time is also relatively fast, as it generally trains under a few minutes.

Additionally, you can share your trained classifiers and easily add/remove images from your classifier by using the actions defined in the API.

Actions

1. Add Images

Description: Immediately re-trains a new model by using the new images with the old ones. Model is trained and kept in the given name space.

Input:

Parameter 1 (required): add_images action.
Parameter 2 (required): Custom name space for saving model.
Parameter 3 (required): A list of images with a url and person.
Parameter 4 (optional): Data collection name to save classifier files. (default = FaceRecognition)

{
  "action": "add_images",
  "name_space": <name_space>,
	"data_collection": <data_collection>,
  "images": [
    {
      "url": <url_1>,
      "person": <person>
    }, {
      "url": <url_2>,
      "person": <person>
    }, {
      "url": <url_3>,
      "person": <person>
    }, {
      "url": <url_4>,
      "person": <person>
    }, {
      "url": <url_5>,
      "person": <person>
    }, {
      "url": <url_6>,
      "person": <person>
    }, {
      "url": <url_7>,
      "person": <person>
    }, {
      "url": <url_8>,
      "person": <person>
    }, {
      "url": <url_9>,
      "person": <person>
    }, {
      "url": <url_10>,
      "person": <person>
    }
  ]
}

Output:

{
	"result": "success"
}

2. Remove Images

Description: Removes images and immediately trains a new model based on deleted images. Model is trained and kept in the given name space.

Input:

Parameter 1 (required): remove_images action.
Parameter 2 (required): Custom name space for saving model.
Parameter 3 (required): A list of images with a url.
Parameter 4 (optional): Data collection name to save classifier files. (default = FaceRecognition)

{
  "action": "remove_images",
  "name_space": <name_space>,
	"data_collection": <data_collection>,
  "images": [
    {
      "url": <url_1>
    }, {
      "url": <url_2>
    }, {
      "url": <url_3>
    }
  ]
}

Output:

{
	"result": "success"
}

3. List Images

Description: Lists all images used for training the latest model. Model is loaded from the given name space.

Input

Parameter 1 (required): list_images action.
Parameter 2 (required): Custom name space for loading images from.
Parameter 3 (optional): Data collection name to save classifier files. (default = FaceRecognition)

{
  "action": "list_images",
	"data_collection": <data_collection>,
  "name_space": <name_space>
}

Output:

{
  "result": "success",
  "images": [
    {
      "url": <url_1>,
      "person": <person_1>
    }, {
      "url": <url_2>,
      "person": <person_1>
    }, {
      "url": <url_3>,
      "person": <person_1>
    }, {
      "url": <url_4>,
      "person": <person_2>
    }, {
      "url": <url_5>,
      "person": <person_2>
    }, {
      "url": <url_6>,
      "person": <person_2>
    }, {
      "url": <url_7>,
      "person": <person_3>
    }, {
      "url": <url_8>,
      "person": <person_3>
    }, {
      "url": <url_9>,
      "person": <person_3>
    }, {
      "url": <url_10>,
      "person": <person_4>
    }, {
      "url": <url_11>,
      "person": <person_4>
    }, {
      "url": <url_12>,
      "person": <person_4>
    }
  ]
}

4. List People

Description: Lists all people who are identified in the given images. Images are loaded from the given name space.

Input:

Parameter 1 (required): list_people action.
Parameter 2 (required): Custom name space for list people from.
Parameter 3 (optional): Data collection name to save classifier files. (default = FaceRecognition)

{
  "action": "list_people",
	"data_collection": <data_collection>,
  "name_space": <name_space>
}

Output:

{
  "result": "success",
  "people": [
    <person_1>,
    <person_2>,
    <person_3>,
    <person_4>,
    <person_5>
  ]
}

5. Remove Person/People

Description: Removes given person/people and re-trains a new model based on deleted images of given person/people. Model is trained and kept in the given name space.

Input:

Parameter 1 (required): remove_person action.
Parameter 2 (required): Custom name space for removing people from.
Parameter 3 (required): Person name.
Parameter 4 (optional): Data collection name to save classifier files. (default = FaceRecognition)

{
	"action": "remove_person",
	"data_collection": <data_collection>,
	"name_space": <name_space>,
	"person": <person>
}

Output:

{
	"result": "success"
}

Input:

Parameter 1 (required): remove_people action.
Parameter 2 (required): Custom name space for removing people from.
Parameter 3 (required): People names.
Parameter 4 (optional): Data collection name to save classifier files. (default = FaceRecognition)

{
	"action": "remove_people",
	"data_collection": <data_collection>,
	"name_space": <name_space>,
	"people": [
		<person_1>,
		<person_2>,
		<person_3>
	]
}

Output:

{
	"result": "success"
}

6. List Name Spaces

Description: Lists all available name spaces.

Input:

Parameter 1 (required): list_name_spaces action.
Parameter 2 (optional): Data collection name to save classifier files. (default = FaceRecognition)

{
	"action": "list_name_spaces",
	"data_collection": <data_collection>
}

Output:

{
	"result": "success",
	"name_spaces": [
		<name_space_name_1>,
		<name_space_name_2>,
		<name_space_name_3>,
		<name_space_name_4>
	]
}

7. Remove Name Space

Description: Remove a name space, and all related data.

Input:

Parameter 1 (required): remove_name_space action.
Parameter 2 (required): Name space to be removed.
Parameter 3 (optional): Data collection name to save classifier files. (default = FaceRecognition)

{
	"action": "remove_name_space",
	"data_collection": <data_collection>,
	"name_space": <name_space>
}

Output:

{
	"result": "success"
}

8. Predict

Description: Tries to predict the person/people in the given images using a trained model. Pulls the model from the given name space.

Input:

Parameter 1 (required): A list of images with urls.
Parameter 2 (required): Custom name space for pulling model for prediction.
Parameter 3 (optional): Data collection name to save classifier files. (default = FaceRecognition)

{
	"action": "predict",
	"data_collection": <data_collection>,
	"name_space": <name_space>,
	"images": [
		{
			"url": <url_1>
		}, {
			"url": <url_2>
		}, {
			"url": <url_3>
		}
	]
}

Output:

{
	"result": "success",
	"images": [
		{
			"url": <url_1>,
			"predictions": [
				{
					"person": <person_1>,
					"bb": {
						"bottom": xx,
						"top": xx,
						"left": xx,
						"right": xx
					}
				}, {
					"person": <person_2>,
					"bb": {
						"bottom": xx,
						"top": xx,
						"left": xx,
						"right": xx
					}
				}
			]
		}
	]
}

Example

In this example we're going to train a face recognition classifier on celebrity images. We find at least 10 images per celebrity, and upload them to the dataAPI CV/FACE_RECOGNITION_EXAMPLE_DATA collection. We also use the celebrities name space for saving our trained model. We need to add at least 2 people for the classifier to work.

Step 1. Training

Parameter 1 (required): add_images action for adding at least 2 different people.
Parameter 2 (required): celebrities name space for saving trained model.
Parameter 3 (required): Celebrity training images. 10 per celebrity, 50 in total.
Parameter 4 (optional): Data collection name to save classifier files. (default = FaceRecognition)

{
  "action": "add_images",
	"data_collection": "MyFaceClassifiers",
  "name_space": "celebrities",
  "images": [
    {
      "url": "data://cv/face_recognition_example_data/jim_carrey_01.jpg",
      "person": "jim_carrey"
    },{
      "url": "data://cv/face_recognition_example_data/jim_carrey_02.jpg",
      "person": "jim_carrey"
    },{
      "url": "data://cv/face_recognition_example_data/jim_carrey_03.jpg",
      "person": "jim_carrey"
    },{
      "url": "data://cv/face_recognition_example_data/jim_carrey_04.jpg",
      "person": "jim_carrey"
    },{
      "url": "data://cv/face_recognition_example_data/jim_carrey_05.jpg",
      "person": "jim_carrey"
    },{
      "url": "data://cv/face_recognition_example_data/jim_carrey_06.jpg",
      "person": "jim_carrey"
    },{
      "url": "data://cv/face_recognition_example_data/jim_carrey_07.jpg",
      "person": "jim_carrey"
    },{
      "url": "data://cv/face_recognition_example_data/jim_carrey_08.jpg",
      "person": "jim_carrey"
    },{
      "url": "data://cv/face_recognition_example_data/jim_carrey_09.jpg",
      "person": "jim_carrey"
    },{
      "url": "data://cv/face_recognition_example_data/jim_carrey_10.jpg",
      "person": "jim_carrey"
    },{
      "url": "data://cv/face_recognition_example_data/jim_caviezel_01.jpg",
      "person": "jim_caviezel"
    },{
      "url": "data://cv/face_recognition_example_data/jim_caviezel_02.jpg",
      "person": "jim_caviezel"
    },{
      "url": "data://cv/face_recognition_example_data/jim_caviezel_03.jpg",
      "person": "jim_caviezel"
    },{
      "url": "data://cv/face_recognition_example_data/jim_caviezel_04.jpg",
      "person": "jim_caviezel"
    },{
      "url": "data://cv/face_recognition_example_data/jim_caviezel_05.jpg",
      "person": "jim_caviezel"
    },{
      "url": "data://cv/face_recognition_example_data/jim_caviezel_06.jpg",
      "person": "jim_caviezel"
    },{
      "url": "data://cv/face_recognition_example_data/jim_caviezel_07.jpg",
      "person": "jim_caviezel"
    },{
      "url": "data://cv/face_recognition_example_data/jim_caviezel_08.jpg",
      "person": "jim_caviezel"
    },{
      "url": "data://cv/face_recognition_example_data/jim_caviezel_09.jpg",
      "person": "jim_caviezel"
    },{
      "url": "data://cv/face_recognition_example_data/jim_caviezel_10.jpg",
      "person": "jim_caviezel"
    },{
      "url": "data://cv/face_recognition_example_data/michael_emerson_01.jpg",
      "person": "michael_emerson"
    },{
      "url": "data://cv/face_recognition_example_data/michael_emerson_02.jpg",
      "person": "michael_emerson"
    },{
      "url": "data://cv/face_recognition_example_data/michael_emerson_03.jpg",
      "person": "michael_emerson"
    },{
      "url": "data://cv/face_recognition_example_data/michael_emerson_04.jpg",
      "person": "michael_emerson"
    },{
      "url": "data://cv/face_recognition_example_data/michael_emerson_05.jpg",
      "person": "michael_emerson"
    },{
      "url": "data://cv/face_recognition_example_data/michael_emerson_06.jpg",
      "person": "michael_emerson"
    },{
      "url": "data://cv/face_recognition_example_data/michael_emerson_07.jpg",
      "person": "michael_emerson"
    },{
      "url": "data://cv/face_recognition_example_data/michael_emerson_08.jpg",
      "person": "michael_emerson"
    },{
      "url": "data://cv/face_recognition_example_data/michael_emerson_09.jpg",
      "person": "michael_emerson"
    },{
      "url": "data://cv/face_recognition_example_data/michael_emerson_10.jpg",
      "person": "michael_emerson"
    },{
      "url": "data://cv/face_recognition_example_data/amy_acker_01.jpg",
      "person": "amy_acker"
    },{
      "url": "data://cv/face_recognition_example_data/amy_acker_02.jpg",
      "person": "amy_acker"
    },{
      "url": "data://cv/face_recognition_example_data/amy_acker_03.jpg",
      "person": "amy_acker"
    },{
      "url": "data://cv/face_recognition_example_data/amy_acker_04.jpg",
      "person": "amy_acker"
    },{
      "url": "data://cv/face_recognition_example_data/amy_acker_05.jpg",
      "person": "amy_acker"
    },{
      "url": "data://cv/face_recognition_example_data/amy_acker_06.jpg",
      "person": "amy_acker"
    },{
      "url": "data://cv/face_recognition_example_data/amy_acker_07.jpg",
      "person": "amy_acker"
    },{
      "url": "data://cv/face_recognition_example_data/amy_acker_08.jpg",
      "person": "amy_acker"
    },{
      "url": "data://cv/face_recognition_example_data/amy_acker_09.jpg",
      "person": "amy_acker"
    },{
      "url": "data://cv/face_recognition_example_data/amy_acker_10.jpg",
      "person": "amy_acker"
    },{
      "url": "data://cv/face_recognition_example_data/sarah_shahi_01.jpg",
      "person": "sarah_shahi"
    },{
      "url": "data://cv/face_recognition_example_data/sarah_shahi_02.jpg",
      "person": "sarah_shahi"
    },{
      "url": "data://cv/face_recognition_example_data/sarah_shahi_03.jpg",
      "person": "sarah_shahi"
    },{
      "url": "data://cv/face_recognition_example_data/sarah_shahi_04.jpg",
      "person": "sarah_shahi"
    },{
      "url": "data://cv/face_recognition_example_data/sarah_shahi_05.jpg",
      "person": "sarah_shahi"
    },{
      "url": "data://cv/face_recognition_example_data/sarah_shahi_06.jpg",
      "person": "sarah_shahi"
    },{
      "url": "data://cv/face_recognition_example_data/sarah_shahi_07.jpg",
      "person": "sarah_shahi"
    },{
      "url": "data://cv/face_recognition_example_data/sarah_shahi_08.jpg",
      "person": "sarah_shahi"
    },{
      "url": "data://cv/face_recognition_example_data/sarah_shahi_09.jpg",
      "person": "sarah_shahi"
    },{
      "url": "data://cv/face_recognition_example_data/sarah_shahi_10.jpg",
      "person": "sarah_shahi"
    },{
      "url": "data://cv/face_recognition_example_data/kevin_chapman_01.jpg",
      "person": "kevin_chapman"
    },{
      "url": "data://cv/face_recognition_example_data/kevin_chapman_02.jpg",
      "person": "kevin_chapman"
    },{
      "url": "data://cv/face_recognition_example_data/kevin_chapman_03.jpg",
      "person": "kevin_chapman"
    },{
      "url": "data://cv/face_recognition_example_data/kevin_chapman_04.jpg",
      "person": "kevin_chapman"
    },{
      "url": "data://cv/face_recognition_example_data/kevin_chapman_05.jpg",
      "person": "kevin_chapman"
    },{
      "url": "data://cv/face_recognition_example_data/kevin_chapman_06.jpg",
      "person": "kevin_chapman"
    },{
      "url": "data://cv/face_recognition_example_data/kevin_chapman_07.jpg",
      "person": "kevin_chapman"
    },{
      "url": "data://cv/face_recognition_example_data/kevin_chapman_08.jpg",
      "person": "kevin_chapman"
    },{
      "url": "data://cv/face_recognition_example_data/kevin_chapman_09.jpg",
      "person": "kevin_chapman"
    },{
      "url": "data://cv/face_recognition_example_data/kevin_chapman_10.jpg",
      "person": "kevin_chapman"
    }
  ]
}

Output:

{
  "result": "success"
}

Step 2. Prediction of Images

Parameter 1 (required): Name space to get model from for prediction.
Parameter 2 (required): predict action.
Parameter 3 (required): Celebrity images for prediction.
Parameter 4 (optional): Data collection name to save classifier files. (default = FaceRecognition)

{
  "name_space": "celebrities",
	"data_collection": "MyFaceClassifiers",
  "action": "predict",
  "images": [
    {
      "url": "data://cv/face_recognition_example_data/jim_carrey_11.jpg"
    },{
      "url": "data://cv/face_recognition_example_data/jim_caviezel_11.jpg"
    },{
      "url": "data://cv/face_recognition_example_data/michael_emerson_11.jpg"
    },{
      "url": "data://cv/face_recognition_example_data/amy_acker_11.jpg"
    },{
      "url": "data://cv/face_recognition_example_data/sarah_shahi_11.jpg"
    },{
      "url": "data://cv/face_recognition_example_data/kevin_chapman_11.jpg"
    }
  ]
}

Output:

{
  "images": [
    {
      "predictions": [
        {
          "bb": {
            "bottom": 196,
            "left": 196,
            "right": 325,
            "top": 67
          },
          "confidence": 0.5740970666573434,
          "person": "jim_carrey"
        }
      ],
      "url": "data://cv/face_recognition_example_data/jim_carrey_11.jpg"
    },
    {
      "predictions": [
        {
          "bb": {
            "bottom": 188,
            "left": 254,
            "right": 328,
            "top": 113
          },
          "confidence": 0.8355480780767119,
          "person": "jim_caviezel"
        }
      ],
      "url": "data://cv/face_recognition_example_data/jim_caviezel_11.jpg"
    },
    {
      "predictions": [
        {
          "bb": {
            "bottom": 236,
            "left": 46,
            "right": 201,
            "top": 81
          },
          "confidence": 0.5752365863904597,
          "person": "michael_emerson"
        }
      ],
      "url": "data://cv/face_recognition_example_data/michael_emerson_11.jpg"
    },
    {
      "predictions": [
        {
          "bb": {
            "bottom": 188,
            "left": 303,
            "right": 378,
            "top": 113
          },
          "confidence": 0.7898999826380547,
          "person": "amy_acker"
        }
      ],
      "url": "data://cv/face_recognition_example_data/amy_acker_11.jpg"
    },
    {
      "predictions": [
        {
          "bb": {
            "bottom": 339,
            "left": 270,
            "right": 425,
            "top": 184
          },
          "confidence": 0.806686600634139,
          "person": "sarah_shahi"
        }
      ],
      "url": "data://cv/face_recognition_example_data/sarah_shahi_11.jpg"
    },
    {
      "predictions": [
        {
          "bb": {
            "bottom": 415,
            "left": 241,
            "right": 464,
            "top": 192
          },
          "confidence": 0.8151349714680911,
          "person": "kevin_chapman"
        }
      ],
      "url": "data://cv/face_recognition_example_data/kevin_chapman_11.jpg"
    }
  ],
  "result": "success"
}

Step 3. Prediction with Visualized Image

Parameter 1 (required): Name space to get model from for prediction.
Parameter 2 (required): predict action.
Parameter 3 (required): Celebrity cast photo for visualized prediction.
Parameter 4 (required): Visualized celebrity cast photo save location.
Parameter 5 (optional): Data collection name to save classifier files. (default = FaceRecognition)

{
  "name_space": "celebrities",
	"data_collection": "MyFaceClassifiers",
  "action": "predict",
  "images": [
    {
      "url": "data://cv/face_recognition_example_data/poi_cast.jpg",
      "output": "data://.algo/cv/FaceRecognition/temp/poi_cast_visualized.png"
    }
  ]
}

Output:

{
  "images": [
    {
      "output": "data://.algo/cv/FaceRecognition/temp/poi_cast_visualized.png",
      "predictions": [
        {
          "bb": {
            "bottom": 315,
            "left": 633,
            "right": 723,
            "top": 226
          },
          "confidence": 0.6931293290507438,
          "output": "data://.algo/cv/FaceRecognition/temp/poi_cast_visualized.png",
          "person": "kevin_chapman"
        },
        {
          "bb": {
            "bottom": 308,
            "left": 543,
            "right": 605,
            "top": 246
          },
          "confidence": 0.7273244502357664,
          "output": "data://.algo/cv/FaceRecognition/temp/poi_cast_visualized.png",
          "person": "sarah_shahi"
        },
        {
          "bb": {
            "bottom": 279,
            "left": 337,
            "right": 411,
            "top": 204
          },
          "confidence": 0.7004215725611026,
          "output": "data://.algo/cv/FaceRecognition/temp/poi_cast_visualized.png",
          "person": "amy_acker"
        },
        {
          "bb": {
            "bottom": 285,
            "left": 225,
            "right": 315,
            "top": 196
          },
          "confidence": 0.7960040928686051,
          "output": "data://.algo/cv/FaceRecognition/temp/poi_cast_visualized.png",
          "person": "michael_emerson"
        },
        {
          "bb": {
            "bottom": 304,
            "left": 453,
            "right": 527,
            "top": 229
          },
          "confidence": 0.7856644416423558,
          "output": "data://.algo/cv/FaceRecognition/temp/poi_cast_visualized.png",
          "person": "jim_caviezel"
        }
      ],
      "url": "data://cv/face_recognition_example_data/poi_cast.jpg"
    }
  ],
  "result": "success"
}

Frequently Asked Questions

FAQ 1: How many images do I need to train a classifier?

Answer: In most cases, training on 10 images per person seems should yield decent results. If you don't think your classifier is accurate enough though, you can add more images until desired results are obtained. You also need to add images for at least 2 people for the classifier to work.

FAQ 2: What is the correct way to do training and predicting?

Answer: There are two categories of actions:

First category is composed of write/update/delete actions where you train a new classifier each time you call these actions. These actions must be done in serial (one after the other) not in parallel. Otherwise a race condition might happen, and you might lose training data.

First category actions are: add_images, remove_images, remove_person, remove_people and remove_name_space.

Second category is read actions. These actions only read and return information from the latest available trained classifier. You may make as many parallel calls for read actions.

Second category actions are: list_name_spaces, list_images, list_people, predict.

FAQ 3: Can I share my trained classifier with other users?

Answer: Yes you can. Other users can use your trained classifier for prediction. They cannot access it for any other action.

All of your data is stored under your preferred data_collection (default is FaceRecognition) and name_space.

Before sharing anything, just make sure that your data collection has the correct permissions (has public access).

Other users can access your trained classifier in the following:

{
	"data_collection": <your_username>/<your_data_collection>,
	"name_space": <your_namespace>,
	"action": "predict",
	"images": [
		{
			"url": <image_url>
		}
	]
}

FAQ 4: What is allowed in name spaces and data collections?

Answer: Currently only letters and numbers are allowed in naming name_space's and data_collection's.

FAQ 5: Where can I find my trained classifier/model?

Answer: The last trained classifiers are kept inside of your data_collection with your given name_space. You have access up to 10 previously trained classifiers in here.

Your classifiers are saved in your preferred data_collection. (default = FaceRecognition)

The classifiers are named is the following schema: <name_space>_<time_stamp>.state

FAQ 6: What kind of URL's are supported?

Answer: Since this algorithm utilizes the util/SmartImageDownloader algorithm, it both supports http(s):// and data:// URL's.

Credits

This algorithm was based on the work in OpenFace: A general-purpose face recognition library with mobile applications and OpenFace Project

Example images retrieved from:

Jim Carrey [1], [2], [3], [4], [5], [6], [7], [8] [9], [10], [11]
Jim Caviezel [1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11]
Michael Emerson [1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11]
Amy Acker [1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11]
Sarah Shahi [1], [2], [3], [4], [5], [6], [7], [8], [9], [10]. [11]
Kevin Chapman [1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11]
Demo photo [1]

cv / FaceRecognition / 0.2.2

Comments

Loading comments