magicanded

magicanded / gethtml / 0.2.18

gethtml - Get HTML of javascript based web page

1 Credit Royalty API Calls - 425
gethtml load given web page, scroll down a number of times and wait until javascript loaded, then save HTML.
Language
Java
Permissions
The Algorithm Platform License is the set of terms that are stated in the Software License section of the Algorithmia Application Developer and API License Agreement. It is intended to allow users to reserve as many rights as possible without limiting Algorithmia's ability to run it as a service. Learn More
This is necessary for algorithms that rely on external services, however it also implies that this algorithm is able to send your input data outside of the Algorithmia platform.

Run an Example

Input
Output
"{\"isError\":false,\"errorMessage\":\"Getting HTML is completed successfully!\",\"errorCode\":\"Success\",\"results\":{\"outputFile\":\"data://.algo/magicanded/gethtml/temp/eb613b46426c4e529e1f89c26b5dc35f.html\",\"injectJsFileUrl\":\"https://algorithmia.com/v1/data/magicanded%2Fpublic%2Fgethtml-inject.js\",\"errorStream\":\"\",\"outputFileUrl\":\"https://algorithmia.com/v1/data/.algo%2Fmagicanded%2Fgethtml%2Ftemp%2Feb613b46426c4e529e1f89c26b5dc35f.html\",\"scroll\":5,\"injectJsFile\":\"data://magicanded/public/gethtml-inject.js\",\"outputStream\":\"Failure upon successfully waiting for global dependencies.\\n\\n ReferenceError: Can\\u0027t find variable: jQuery\\n\\n  https://www.huffingtonpost.com/entry/apple-new-iphone-x_us_59b809f9e4b027c149e2dbe0:107\\nFailure upon successfully waiting for global dependencies.\\n\\n ReferenceError: Can\\u0027t find variable: jQuery\\n\\n  https://www.huffingtonpost.com/entry/apple-new-iphone-x_us_59b809f9e4b027c149e2dbe0:107\\nFailure upon successfully waiting for global dependencies.\\n\\n ReferenceError: Can\\u0027t find variable: jQuery\\n\\n  https://www.huffingtonpost.com/entry/apple-new-iphone-x_us_59b809f9e4b027c149e2dbe0:107\\n\",\"cookiesFile\":\"data://.algo/magicanded/gethtml/temp/cf98f88de4e04cb08c1bfc30375de688.cookies\",\"cookiesFileUrl\":\"https://algorithmia.com/v1/data/.algo%2Fmagicanded%2Fgethtml%2Ftemp%2Fcf98f88de4e04cb08c1bfc30375de688.cookies\"}}"

Install and Use

Install

Install the Algorithmia CLI client by running:

curl -sSLf https://algorithmia.com/install.sh | sh

Then authenticate by running:

algo auth
# Enter API Key: YOUR_API_KEY
CLI Install Docs

Use



algo run magicanded/gethtml/0.2.18 -d '{
  "url": "https://www.huffingtonpost.com/entry/apple-new-iphone-x_us_59b809f9e4b027c149e2dbe0",
  "scroll": 5,
  "injectJsFile": "data://magicanded/public/gethtml-inject.js"
}' --timeout 300
  
CLI Docs