Site Map

No algorithm description given

Table of Contents Introduction Examples Credit Introduction Starting from a given url, crawls pages within the same domain.  Returns a graph representing the link structure of the crawled site. Input: (Required): Website URL. (Optional): Depth of search. (default = 2) (Optional) : Save result to given file location instead of listing in output. Output returns an empty object when this parameter is given. Useful when output larger than the 10 MB limit. Output: A list of URLs extracted from the given URL. (with same domain name.) Examples Example 1. Parameter 1: Algorithmia website URL. ["https://algorithmia.com", 1] Output: {
 "http://algorithmia.com": [
 "http://developers.algorithmia.com",
 "https://algorithmia.com/terms",
 "https://algorithmia.com/algorithms/TimeSeries/OutlierDetection",
 "https://algorithmia.com/algorithms/opencv/FaceDetection",
 .....
 "https://algorithmia.com/about",
 "https://algorithmia.com/algodev",
 "https://algorithmia.com/algorithms/util/Url2Text",
 "https://algorithmia.com/privacy"
 ]
} Example 2. Parameter 1: Archive.org website URL. ["http://archive.org", 1] Output: {
 "http://archive.org": [
 "https://archive.org/details/library_of_congress",
 "https://archive.org/web/",
 "https://archive.org/details/microfilm",
 "https://blog.archive.org/category/announcements/",
 .....
 "https://archive.org/details/television",
 "https://archive.org/details/internetarcade",
 "https://archive.org/details/animationandcartoons",
 "https://archive.org/details/netlabels"
 ]
} Credit Algorithm was built using the JSOUP library.

Tags
(no tags)

Cost Breakdown

0 cr
royalty per call
1 cr
usage per second
avg duration
This algorithm has permission to call other algorithms which may incur separate royalty and usage costs.

Cost Calculator

API call duration (sec)
×
API calls
=
Estimated cost
per calls
for large volume discounts
For additional details on how pricing works, see Algorithmia pricing.

Internet access

This algorithm has Internet access. This is necessary for algorithms that rely on external services, however it also implies that this algorithm is able to send your input data outside of the Algorithmia platform.


Calls other algorithms

This algorithm has permission to call other algorithms. This allows an algorithm to compose sophisticated functionality using other algorithms as building blocks, however it also carries the potential of incurring additional royalty and usage costs from any algorithm that it calls.


To understand more about how algorithm permissions work, see the permissions documentation.

1. Type your input

2. See the result

Running algorithm...

3. Use this algorithm

curl -X POST -d '{{input | formatInput:"curl"}}' -H 'Content-Type: application/json' -H 'Authorization: Simple YOUR_API_KEY' https://api.algorithmia.com/v1/algo/web/SiteMap/0.1.7
View cURL Docs
algo auth
# Enter API Key: YOUR_API_KEY
algo run algo://web/SiteMap/0.1.7 -d '{{input | formatInput:"cli"}}'
View CLI Docs
import com.algorithmia.*;
import com.algorithmia.algo.*;

String input = "{{input | formatInput:"java"}}";
AlgorithmiaClient client = Algorithmia.client("YOUR_API_KEY");
Algorithm algo = client.algo("algo://web/SiteMap/0.1.7");
AlgoResponse result = algo.pipeJson(input);
System.out.println(result.asJsonString());
View Java Docs
import com.algorithmia._
import com.algorithmia.algo._

val input = {{input | formatInput:"scala"}}
val client = Algorithmia.client("YOUR_API_KEY")
val algo = client.algo("algo://web/SiteMap/0.1.7")
val result = algo.pipeJson(input)
System.out.println(result.asJsonString)
View Scala Docs
var input = {{input | formatInput:"javascript"}};
Algorithmia.client("YOUR_API_KEY")
           .algo("algo://web/SiteMap/0.1.7")
           .pipe(input)
           .then(function(output) {
             console.log(output);
           });
View Javascript Docs
var input = {{input | formatInput:"javascript"}};
Algorithmia.client("YOUR_API_KEY")
           .algo("algo://web/SiteMap/0.1.7")
           .pipe(input)
           .then(function(response) {
             console.log(response.get());
           });
View NodeJS Docs
import Algorithmia

input = {{input | formatInput:"python"}}
client = Algorithmia.client('YOUR_API_KEY')
algo = client.algo('web/SiteMap/0.1.7')
print algo.pipe(input)
View Python Docs
library(algorithmia)

input <- {{input | formatInput:"r"}}
client <- getAlgorithmiaClient("YOUR_API_KEY")
algo <- client$algo("web/SiteMap/0.1.7")
result <- algo$pipe(input)$result
print(result)
View R Docs
require 'algorithmia'

input = {{input | formatInput:"ruby"}}
client = Algorithmia.client('YOUR_API_KEY')
algo = client.algo('web/SiteMap/0.1.7')
puts algo.pipe(input).result
View Ruby Docs
use algorithmia::*;

let input = {{input | formatInput:"rust"}};
let client = Algorithmia::client("YOUR_API_KEY");
let algo = client.algo('web/SiteMap/0.1.7');
let response = algo.pipe(input);
View Rust Docs
Discussion
  • {{comment.username}}