nlp

nlp / AnalyzeTweets / 0.1.13

README.md

0. TL;DR

This is a recipe for Analyzing Tweets. It combines twitter/RetrieveTweetsWithKeywordnlp/SocialSentimentAnalysis, and nlp/LDA into a micro-service.

1. Introduction

Recipes are plug-n-play utilities that solve a specific problem in a novel way. These micro-services are comprised of one or more algorithms that allow users to quickly and easily get value from Algorithmia. They’re modular, and should be thought of as mini-products with obvious value.

This recipe is composed in the following way:

  1. The recipe is given a keyword by the user and retrieves relevant tweets from twitter using twitter/RetrieveTweetsWithKeyword.
  2. These relevant tweets are analyzed, labelled and sorted by nlp/SocialSentimentAnalysis.
  3. The top 20% (positive) tweets and the bottom 20% (negative) tweets are used to extract positive and negative topics using nlp/LDA.
  4. The corresponding positive & negative topics and tweets are returned.

Input:

  • (Required): The query keyword/string
  • (Required): Number of tweets
  • (Required): Twitter API authentication keys

Output

  • A list of positive topics (nlp/LDA output)
  • A list of negative topics (nlp/LDA output)
  • A list of all tweets
  • A list of positive tweets
  • A list of negative tweets

2. Query

The query keyword/string: The keyword that you're searching for on Twitter. (key = "query")

Example of a query keyword/string:

{
    "query": "algorithmia"
}

Number of tweets: The number of tweets you want to return from the search. (key = "numTweets")

Example of number of tweets:

{
    "numTweets": 250
}

3. Authentication

Twitter API authentication keys: The API keys that are necessary to access Twitter API service. You can get yours here(key = "auth")

{
    "auth": {
        "app_key": "xxxxxxx",
        "app_secret": "xxxxxxx",
        "oauth_token": "xxxxxxx",
        "oauth_token_secret": "xxxxxxx"
    }
}

4. Output

A list of positive topics: A list of topics extracted from the positive tweets using nlp/LDA. (key = "posLDA")

Example of a list of positive topics:
{
    "posLDA": [{
        "good": 5,
        "blizzard": 9,
        "service": 7,
        "awesome": 6,
        "comcastcareers": 6,
        "comcast": 43,
        "pretty": 6,
        "weekend": 6
    },
    ...,
    {
        "great": 16,
        "outdoor": 9,
        "ads": 9,
        "show": 10,
        "nra": 15,
        "american": 9,
        "comcast": 23,
        "internet": 8
    }]
}

A list of negative topics: A list of topics extracted from the negative tweets using nlp/LDA. (key = "negLDA")

Example of a list of negative topics:

{
    "negLDA": [{
        "https://t.co": 6,
        "cable": 5,
        "comcastcares": 4,
        "stuallard": 4,
        "sportsnet": 5,
        "comcast": 43,
        "internet": 6,
        "hate": 9
    },
    ...,
    {
        "premiere": 6,
        "nra": 7,
        "gun": 11,
        "remove": 7,
        "prejudice": 6,
        "zombies": 6,
        "images": 7,
        "l.a": 6
    }]
}

A list of all tweets: All of the tweets that returned from the search query. (key = "allTweets")

Example of a list of all tweets:

{
    "allTweets": [
        {
            "text": "Here is the link to out of control on iTunes! It would mean a lot if you guys got it! Thanks! https://t.co/FUns7SLwnv",
            "created_at": "Mon Jan 25 15:30:36 +0000 2016",
            "tweet_url": "https://twitter.com/statuses/691644344814559232",
            "overall_sentiment": 0.5826,
            "positive_sentiment": 0.159,
            "neutral_sentiment": 0.841,
            "negative_sentiment": 0
        },
        ...,
        {
            "text": "Apple 5se rumored to use A9/M9 chips https://t.co/3O8P2ftTpJ #tech #gadgets",
            "created_at": "Mon Jan 25 15:30:27 +0000 2016",
            "tweet_url": "https://twitter.com/statuses/691644307577540609",
            "overall_sentiment": 0,
            "positive_sentiment": 0,
            "neutral_sentiment": 1,
            "negative_sentiment": 0
        }
    ]
}

A list of positive tweets: The most positive tweets from the twitter search query. (key = "posTweets")

Example of a list of positive tweets:

{
    "posTweets": [
        {
            "text": "Need some music this morning, #AmazonPrime free music thx! AMZN Mobile LLC - Amazon Music with Prime Music - https://t.co/frF0j5szpV #iTunes",
            "created_at": "Mon Jan 25 15:30:37 +0000 2016",
            "tweet_url": "https://twitter.com/statuses/691644349805785088",
            "overall_sentiment": 0.7777,
            "positive_sentiment": 0.328,
            "neutral_sentiment": 0.672,
            "negative_sentiment": 0
        },
        ...,
        {
            "text": "This cool new #app helps #PowerAfrica track #energy deals in #Africa: https://t.co/2sZEfKKzZQ Download @AppStore https://t.co/a24GSQnCkg",
            "created_at": "Mon Jan 25 15:30:32 +0000 2016",
            "tweet_url": "https://twitter.com/statuses/691644327836192768",
            "overall_sentiment": 0.5994,
            "positive_sentiment": 0.274,
            "neutral_sentiment": 0.726,
            "negative_sentiment": 0
        }
    ]
}

A list of negative tweets: The most negative tweets from the twitter search query. (key = "newTweets")

Example of a list of negative tweets:

{
    "negTweets": [
        {
            "text": "RT @TotalTrafficCIN: Closed due to accident in #Cincinnati on Kellogg Ave at Apple Hl Rd #traffic https://t.co/oaARUhWVVr",
            "created_at": "Mon Jan 25 15:30:36 +0000 2016",
            "tweet_url": "https://twitter.com/statuses/691644345544527873",
            "overall_sentiment": -0.4767,
            "positive_sentiment": 0,
            "neutral_sentiment": 0.838,
            "negative_sentiment": 0.162
        },
        ...,
        {
            "text": "@FSBThamesValley hi, not quite sure what is wrong as from iPad it says blocked yet i can follow u from my pc obviously Apple issue.",
            "created_at": "Mon Jan 25 15:30:33 +0000 2016",
            "tweet_url": "https://twitter.com/statuses/691644333779505152",
            "overall_sentiment": -0.749,
            "positive_sentiment": 0,
            "neutral_sentiment": 0.73,
            "negative_sentiment": 0.27
        }
    ]
}

5. Example

Example 1:

  • Parameter 1: A query keyword/string
  • Parameter 2: Number of tweets
  • Parameter 3: Twitter API authentication keys
{
    "query": "google",
    "numTweets": 250,
    "auth": {
        "app_key": "xxxxxxx",
        "app_secret": "xxxxxxx",
        "oauth_token": "xxxxxxx",
        "oauth_token_secret": "xxxxxxx"
    }
}