Tokenizer

Tokenizes given text into an array of strings

Algorithmia Platform License · Internet Access

This algorithm has Internet access.

This is necessary for algorithms that rely on external services, however it also implies that this algorithm is able to send your input data outside of the Algorithmia platform.

Run an Example

[
  "This",
  "is",
  "a",
  "sentence",
  "to",
  "tokenize",
  ".",
  "This",
  "sentence",
  "is",
  "here",
  "for",
  "the",
  "same",
  "reason",
  "."
]

Install & Use

Use

curl -X POST -d '"This is a sentence to tokenize. This sentence is here for the same reason."' -H 'Content-Type: application/json' -H 'Authorization: Simple YOUR_API_KEY' https://api.algorithmia.com/v1/algo/ApacheOpenNLP/Tokenizer/0.1.0