media / VideoMetadataExtraction / 0.6.0
Video Metadata Extraction
This algorithm uses an image processing algorithm to extract metadata from each frame of a video, and returns a json file containing each frames timestamp and the resulting metadata.
note: This algorithm takes longer than most other algorithms as it recursively calls an image algorithm over each frame of a video file, make sure to add a large custom timeout to your algorithm call before using videos longer than 3 minutes.
note: This algorithm is royalty free, but the image processing algorithms it can call might not be, check your account's credit balance before running this algorithm on a custom video.
Changelog
0.6.0 - August 1st, 2018:
- Added a early termination watchdog thread that detects processing jobs that are forecasted to take longer than the maximum timeout of 50 minutes.
- various bug fixes that should improve stability and early termination in the event of a detected failure.
0.5.0 - May 23, 2017:
- Added Smart Video Downloader support to resolve a compressed gzip request problem.
- with Smart Video Downloader added,
input_file
may now point to any web hosted URL, including youtube, vimeo, etc.
Input
input_file: String,
output_file: String,
algorithm: String,
advanced_input: Json,
fps: Double,
image_compression: Int,
video_compression: Int
- input_file - (required) - The input url of the video file, can be any standard video container format (mp4, mkv, avi, etc), can be an http/https url or a data connector uri(
data://
,s3://
,dropbox://
, etc). - output_file - (required) - The output url of the json file, must be a data connector uri (
data://
,s3://
,dropbox://
, etc). - algorithm - (required) - The image processing algorithmia algorithm uri, if no default mode is found for that algorithm, advanced_input must be defined.
- advanced_input - (optional) - if you have advanced requirements or want to use an algorithm with no default parameters, See Advanced Input.
- fps - (optional) - If you have a desired fps sampling rate, input it here. defaults to the input video's fps.
- image_compression - (optional) - if you want to improve performance of processing, it's possible to compress each frame using the jpeg compression algorithm, the value provided is the associated compression ratio - from
0
(uncompressed) to100
(most compressed) default jpeg compression is20
. - video_compression - (optional) - by default, the output video file is raw and uncompressed, if you desire to compress your output video using the libx264 codec, provide a compression ratio value for this element, from
0
(uncompressed) to `100 (most compressed)
Advanced Input
If advanced_input is found, this algorithm will parse it as a json object; finding and replacing keywords relating to uploading and downloading files in both batch and single form with paths that the Video Metadata Extraction algorithm uses.
The keywords are:
$SINGLE_INPUT
- whenever you see a single image input path for an algorithm, this keyword will work
$BATCH_INPUT
- similar to single input keyword, but works for arrays of input files (Nuditydetectioni2v for example)
Example:
IllustrationTagger algorithm default input
{
"image": "https://upload.wikimedia.org/wikipedia/commons/thumb/9/9a/Trudeaujpg.jpg/348px-Trudeaujpg.jpg",
"tags": ["1boy", "male", "sky", "water", "safe"]
}
Video Metadata Extraction with advanced Illustration Tagger input
{
"input_file": "data://path/to/file.mp4",
"output_file": "data://save/data.json,
"algorithm": "algo://deeplearning/IllustrationTagger/0.2.3",
"advanced_input": {
"image": "$SINGLE_INPUT",
"tags": ["1boy", "male", "sky", "water", "safe"]
},
"fps": 12.5
}
Output
{
"output_file": String
}
- output_file - the complete algorithmia URI for where the output json file has been uploaded to, it's format can be seen below.
formatting
The resulting output json file has the following format:
{
"frame_data":[
{
"timestamp":Double,
"data":Json
} ...
]
}
- frame_data - the main json key, contains an array of json objects containing timestamps and data.
- timestamp - the timestamp of the frame in seconds from the start (frame 0 is 0.0, etc).
- data - the resulting output json data from the image processing algorithm for that particular frame, can be any json parsable format.
for example, output from nuditydetectioni2v:
{
"frame_data":[
{
"data":{
"confidence":1,
"nude":false
},
"timestamp":0.0
},
{
"data":{
"confidence":1,
"nude":false
},
"timestamp":0.0666446023989338
},
{
"data":{
"confidence":1,
"nude":false
},
"timestamp":0.1332892047978676
},
{
"data":{
"confidence":1,
"nude":false
},
"timestamp":0.1999338071968014
}
]
}
Default Options
This algorithm has default parameters set up for a number of algorithms, this list will grow as new image altering algorithms are released:
- Nudity Detection - uses version
0.2.4
, in batch mode (sfw/NudityDetectioni2v) - illustration Tagger - uses version
0.2.3
, using default tags and threashold (deeplearning/SalNet)
Examples
Nuditydetectioni2v default input
{
"input_file":"data://media/videos/kenny_test.mp4",
"output_file":"data://.algo/temp/kenny_test_nudity.json",
"algorithm":"algo://sfw/nuditydetectioni2v"
}
Places365 Classifier advanced input
{
"input_file":"data://media/videos/kenny_test.mp4",
"output_file":"data://.algo/temp/kenny_test_places.json",
"algorithm":"algo://deeplearning/places365classifier/0.1.9",
"advanced_input":{
"image":"$SINGLE_INPUT",
"numResults": 15
}
}
IllustrationTagger default input with fps
{
"input_file":"data://media/videos/kenny_test.mp4",
"output_file":"data://.algo/temp/kenny_illustrated.json",
"algorithm":"algo://deeplearning/IllustrationTagger",
"fps": 14.24
}
Credits
All video processing is handled by FFMpeg
, which is a fantastic video/image manipulation tool that can be found here