media

media / VideoAlgorithms / 0.2.5

README.md

words to the wise: video files are highly compressed objects, when you convert a video into still images the file size can increase dramatically. File transfer times may bottleneck your application if you aren't careful!

Table of Contents

Introduction

This package has a collection of video processing algorithms for manipulating video using ffmpeg. Useful as a building block to construct video processing pipelines with other algorithms such as Nudity Detection or Emotion Detection.

API

Since this package contains 4 separate algorithms in this package, we have 4 inputs. However they all share the same output structure to simplify things!

Split into subvideos

This algorithm splits any length/format video into smaller subvideos losslessly on keyframes. This algorithm can also change the video container format; avi, mp4, ogg, mkv are all supported, and many more.

very useful for massively parallel video -> image process app -> video pipeline apps.

protip: this algorithm is most effective on large videos with reasonably spaced keyframes.

{  
   "format":"SplitToSubVideos",
   "data":{  
      "input": SplittingInputs,
      "output":Outputs
   }
}
  • format - (required) - This text switch determines which algorithm to call from the VideoAlgorithms package, SplitToSubVideos tells the package to expect to split a video into subvideos.
  • data - (required) - genetic data container for all entrypoints, contains an input and an output, the input depends on if it's a split operation or combine, whereas output is general to all algorithms.

Split into frames

This algorithm splits any length/format video into still images at a particular fps, the image format output can be any standard image format such as png, jpg, gif, tiff, bmp. Extremely useful for performing image processing / classification operations on video files.

{  
   "format":"SplitToFrames",
   "data":{  
      "input":SplittingInputs,
      "output":Outputs,
      "fps":Double
   }
}
  • format - (required) - This text switch determines which algorithm to call from the VideoAlgorithms package, SplitToFrames tells the package to expect to split a video into subvideos.
  • data - (required) - genetic data container for all entrypoints, contains an input and an output, the input depends on if it's a split operation or combine, whereas output is general to all algorithms.
  • fps - (required) - sets the frames per second to capture and extract. The more you capture the less choppy the recombined operation will be, but the larger your image batches will be. don't set the fps target above the video's native fps, you'll have problems.

Combine from Subvideos

This algorithm concatenates previously split subvideos (aka video chunks split on keyframes) back together based on the filenames, if the subvideos contain an audio stream then the audio is also concatenated. Video container formats are changable in this algorithm, however major changes (like avi to ogg) may lead to a broken video.

{  
   "format":"CombineFromSubvideos",
   "data":{  
      "input":CombineInputs,
      "output":Outputs,
   }
}
  • format - (required) - This text switch determines which algorithm to call from the VideoAlgorithms package, CombineFromSubvideos tells the package to expect to combine subvideos to from a single video file.
  • data - (required) - genetic data container for all entrypoints, contains an input and an output, the input depends on if it's a split operation or combine, whereas output is general to all algorithms.

Combine from Frames

This algorithm concatenates images/frames from a previously split video file back together.

Encoding a new video file is time consuming an requires alot of compute power, try to split up the work across multiple threads by using the Split into subvideos method first.

{  
   "format":"CombineFromFrames",
   "data":{  
      "input":CombineInputs,
      "output":Outputs,
      "fps":Double
   }
}
  • format - (required) - This text switch determines which algorithm to call from the VideoAlgorithms package, CombineFromFrames tells the package to expect to encode a new video from still images.
  • data - (required) - genetic data container for all entrypoints, contains an input and an output, the input depends on if it's a split operation or combine, whereas output is general to all algorithms.
  • fps - (required) - Sets the frames per second for encoding the new video, if this algorithm is used in an operation sequence with Split To Frames make sure the fps parameters match, or your video duration will be altered.

Outputs

{  
   "uuid":String,
   "files":List[String]
}
  • uuid - The unique identifier for this operation, very useful for passing output data between different processing steps, this should be piped into the inputs of combine operations such as combine from frames and combine from subvideos.
  • files - The output file(s) path, if you selected zippedOutput then the output will be a tar.gz file, otherwise it will return a list of output objects.

Splitting Inputs

Each split operation has the same generic input parameters:

{  
   "inputVideoUrl":String"
}

  • inputVideoUrl - (required) The input video url for the algorithm, can be a file hosted in any of the regular algorithmia datasources like data://, dropbox:// and S3://, or from a web HTTP source.

Combine Inputs

Each combine operation has the same generic input parameters:

{  
   "uuids":List[String],
   "inputCollections":List[String]
}
  • uuids - (required) As a combine operation's inputs may come from multiple independent splitting operations (such as split to frames, where each video is a separate subvideo), there may be multiple UUIDs that must be injested into a combine operation.
  • inputCollections - (required) The algorithm will search the provided algorithmia data URIs for the above uuids, if any are found they are downloaded and pre-processed for concatenation. An input collection can be from a data://, dropbox:// or s3:// source.

pass the UUIDs from the previous split operation(s) into the called combine operation, otherwise the algorithm won't combine properly!

Examples

Split to subvideos

input

{  
   "format":"SplitToSubvideos",
   "data":{  
      "input":{  
         "inputVideoUrl":"data://pixvana/examples/Baseball_Stadium_Fly_Over.mp4"
      },
      "output":{  
         "collection":"data://.algo/temp",
         "prefix":"testing123",
         "extension":"avi",
         "zippedOutput":false
      }
   }
}

output

{  
   "uuid":"9aeff5e41dd446d88df5ee1b882f8bce",
   "files":[  
      "data://.algo/temp/testing123-00002-9aeff5e41dd446d88df5ee1b882f8bce.avi",
      "data://.algo/temp/testing123-00010-9aeff5e41dd446d88df5ee1b882f8bce.avi",
      "data://.algo/temp/testing123-00008-9aeff5e41dd446d88df5ee1b882f8bce.avi",
      "data://.algo/temp/testing123-00007-9aeff5e41dd446d88df5ee1b882f8bce.avi",
      "data://.algo/temp/testing123-00003-9aeff5e41dd446d88df5ee1b882f8bce.avi",
      "data://.algo/temp/testing123-00013-9aeff5e41dd446d88df5ee1b882f8bce.avi",
      "data://.algo/temp/testing123-00001-9aeff5e41dd446d88df5ee1b882f8bce.avi",
      "data://.algo/temp/testing123-00006-9aeff5e41dd446d88df5ee1b882f8bce.avi",
      "data://.algo/temp/testing123-00009-9aeff5e41dd446d88df5ee1b882f8bce.avi",
      "data://.algo/temp/testing123-00004-9aeff5e41dd446d88df5ee1b882f8bce.avi",
      "data://.algo/temp/testing123-00011-9aeff5e41dd446d88df5ee1b882f8bce.avi",
      "data://.algo/temp/testing123-00012-9aeff5e41dd446d88df5ee1b882f8bce.avi",
      "data://.algo/temp/testing123-00014-9aeff5e41dd446d88df5ee1b882f8bce.avi",
      "data://.algo/temp/testing123-00005-9aeff5e41dd446d88df5ee1b882f8bce.avi",
      "data://.algo/temp/testing123-00000-9aeff5e41dd446d88df5ee1b882f8bce.avi"
   ]
}

Combine from subvideos

input

{  
   "format":"CombineFromSubvideos",
   "data":{  
      "input":{  
         "uuids":[  
            "56ee0efa7afe42e1b94af9efb1776e4c"
         ],
         "inputCollections":[  
            "data://.algo/temp"
         ]
      },
      "output":{  
         "collection":"data://.algo/perm",
         "prefix":"baseballGame",
         "extension":"mkv",
         "zippedOutput":false
      }
   }
}

output

{  
   "uuid":"967707f036dd43e19bb5d9e433b9d2ea",
   "files":[  
      "data://.algo/perm/baseballGame-967707f036dd43e19bb5d9e433b9d2ea.mkv"
   ]
}

Split to frames

input

{  
   "format":"SplitToFrames",
   "data":{  
      "input":{  
         "inputVideoUrl":"data://.algo/pixvana/VideoAlgorithms/temp/baseballSplits-00003-56ee0efa7afe42e1b94af9efb1776e4c.avi"
      },
      "output":{  
         "collection":"data://.algo/temp",
         "prefix":"baseballStills",
         "extension":"jpg",
         "zippedOutput":true
      },
      "fps":20
   }
}

output

{  
   "uuid":"15b164fbfdb6439b8e7099f8e5127fe6",
   "files":[  
      "data://.algo/temp/15b164fbfdb6439b8e7099f8e5127fe6.tar.gz"
   ]
}

Combine from frames

input

{  
   "format":"CombineFromFrames",
   "data":{  
      "input":{  
         "uuids":[  
            "15b164fbfdb6439b8e7099f8e5127fe6"
         ],
         "inputCollections":[  
            "data://.algo/temp"
         ]
      },
      "output":{  
         "collection":"data://.algo/perm",
         "prefix":"baseballGame",
         "extension":"mkv",
         "zippedOutput":false
      },
      "fps":20
   }
}

output

{  
   "uuid":"1d1fcc025beb4a9ca6dd85af820cd113",
   "files":[  
      "data://.algo/perm/baseballGame-from_chunk-00003-1d1fcc025beb4a9ca6dd85af820cd113.mkv"
   ]
}

Credits

All video processing is handled by FFMpeg, which is a fantastic video/image manipulation tool that can be found here