A simple Python app that can give you valuable insights into just about any topic you can think of
[caption id="attachment_23434" align="aligncenter" width="620"]
Photo by camilo jimenez on Unsplash[/caption]
With about 6,000 tweets being sent out per second and 500 million posted per day, the average person could not even imagine trying to parse out all this data. Unless that average person is reading this article: then you might venture to try. Now we won’t be checking every tweet, but rather tweets of a bespoke nature. We can extract tweets that mention a specific keyword or phrase and with that information measure how the ‘twitter-verse’ feels about said topic or phrase.
This tutorial will be very noob-friendly, in fact, it won’t require any prior coding experience to run and install. You just need a laptop and Python 3.8 — I’ll explain the rest. If you’ve absolutely never touched Python, the most difficult part will be setting up your IDE with Python 3.8 and installing the required dependencies. This project uses much of the same code as my previous article, however while that project was primarily focused on presenting and testing an idea, the final product of this project is an executable .py application.
Again, we’ll be using Python 3.8 for this project so if you’re using a previous version, you’ll need to set up a new virtual environment with 3.8 or update your current environment. Check your Python version with:
from platform import python_version
print(python_version())
Installing Dependencies
To start out, as with all coding projects, let us install some dependencies:
import pandas as pd
import numpy as np
import csv
import snscrape.modules.twitter as sntwitter
from vaderSentiment.vaderSentiment import SentimentIntensityAnalyzer
import datetime as dt
import time
If you receive an error installing any of these packages, just pip3 install them (for example pip3 install snscrape) or google how to install them.
We initialize our date-time objects so that we pull data starting from the beginning of the previous day. This means that we aren’t specifically extracting tweets from the last 24 hours, rather we are extracting tweets from the start of the previous day and all tweets posted thus far today.
# Generating datetime objects
from datetime import datetime, timedelta
now = datetime.now()
now = now.strftime('%Y-%m-%d')
yesterday = datetime.now() - timedelta(days = 1)
yesterday = yesterday.strftime('%Y-%m-%d')
Prompting the user to input a keyword:
keyword = input('Enter a topic or keyword, please:')
Twitter Scraping
Scraping twitter and writing a unique CSV file titled with the keyword we chose in our user input and the date we run the code.
Running sentiment-bot.py in Mac Terminal[/caption]