Location-based social media monitoring

Beacon Hill, Sunday night, 10:50 p.m.: Sitting at my kitchen table, I heard a series of pops followed immediately by the sound of sirens.  “Were those gunshots or fireworks?  Should I be worried?  And are the sirens related to the pops I heard?”  

My first reaction was to search for possibly related posts on Twitter while looking for a live audio feed of Boston police scanners.  Instead, I remembered reading about the location-based social media search services that aggregated posts from across several platforms, and I tried the first one I could quickly get a free trial for: Echosec.

Instead of searching across several social media platforms separately, Echosec allowed me to search for all geotagged posts in an area of my choosing and within specified date ranges.  My story has a simple ending — I found on Echosec that neighbors on reddit posted that it was definitely fireworks, which was later confirmed by the police through the live feed.  

Nothing became of this tiny story, but imagine the uses for location-based social media monitoring services in situations with more impact and higher stakes.  

Using Echosec (and other similar services) for discovery and identification

Google “location-based social media monitoring,” and you’ll find pages of lists suggesting various services, most of which appear to be enterprise services.  While many of these services appear to primarily serve police departments, security companies, and marketing departments of large businesses, over the last couple years, journalists have also used these tools to assist their reporting.  For example:

  • At NBC 5 in Chicago, a producer used Geofeedia to quickly find photos of people who were hiding inside a building after an employee shot his boss.  Based on these photos, the station was able to identify potential sources.
  • A social media editor for The Associated Press used SAM to identify students at a South Dakota high school where a shooting was foiled, which led to a reporter being able to conduct an interview to confirm details seen on social media.

In more general cases, these tools can also be used to get a sense for people’s reactions to news and events across the board, not only to identify sources and images.  Broadly, using geolocated social media search tools as several benefits over simply searching on Twitter.

  • Aggregating data from many social media platforms saves time in pressing situations.
  • Aggregation also provides more comprehensive coverage, especially as different social media platforms are prominent in different areas of the world.
  • Searches can be more location-specific and time-specific than most apps allow within their own search function.

Drawbacks to these services

However, there are two major hurdles these services have to overcome to gain more mainstream traction:

  1. They’re relatively costly.  At the lower end, Echosec costs $129 per user per month, and as of 2012, the much more powerful Geofeedia’s preliminary pricing was $1,450 per month for five users.  (And as I searched through lists of services that were only a couple years old, I found that free versions don’t seem to last long in the marketplace, or if they still exist, are not well supported.)  Either the prices have to come down, or the services have to become much, much better than they currently are in order to make the price tag worth it for newsrooms that are satisfied searching on their own.
  2. The vast majority of social media posts are not geolocated. While the percentage varies by platform (Instagram, for example, tends to have “a lot more [geolocated posts] than Facebook, Youtube, or other platforms”), a Knight Lab sample of 200,000 tweets run in 2015 found less than 0.4% were geocoded.  This means that while you can get a sample of tweets that are geolocated, you do have to make sure not to rely on these tools too much — you could miss an important non-geocoded post that does not turn up in your searches.

That said, for many reporting purposes, simply knowing how to strategically search on popular social media sites is enough.  For journalists without access to these fancier aggregated geolocation search tools, old-fashioned hashtag-hunting and keyword-monitoring may be sufficient.

The potential

A common accusation recently is that the “mainstream media” has lost touch with the average American.  One way to gain easy access to some representation of those viewpoints (although we do then get into the issue of comment rage and trolls — which we’ll sidestep for now) is to see what everyone is saying across various social media channels and be able to check for location-based trends.  After all, the Internet is supposed to be the great equalizer — according to a 2016 Pew Research study, 87% of Americans use the internet.  That percentage will only grow.

Going forward, I do think location-based social media monitoring tools have the potential to become even more powerful as a way to explore the public conversation and identify trends, or simply to get the “pulse” of the public.  

Creativity and collaboration –aspects of future quality work

Even though the speed of processing information is extremely relevant in today’s sea of news, I believe that tools which help us be more creative and collaborative play an extremely important role for criticality and depth of what we perceive. In particular, working in interdisciplinary teams and viewing a topic from multiple perspectives seem like a crucial quality for any future work and meaningful storytelling.

For this reason I found the app GroupMap very interesting for managing creative and productive group flow. The tool helps us improve group brainstorming and creative processes but it also enables easier team decision making.

To facilitate a discussion, one can create its own map or choose from a list of templates and customize them to group’s needs. I found particularly useful templates for Charts where a group of people can visualize everyone’s opinions and thus ease decision making. For instance, there is an Important vs Urgent map and Effort vs Impact chart where, through dot voting, a group can set priorities and organize their time accordingly. In a way, this tool can help us quickly work together and visualize subjective opinions which can have powerful implications for group’s quality of work and outcomes.

#illgowithyou transgender bathroom laws

My action is related to this story about North Carolina’s shameful transgender law. I wanted to give allies an idea of one thing they might do. So I created a YouTube video in Keynote with some really bad motion graphics.

LGBT rights are the civil rights of my generation (although I could make a good argument that neither people of color nor women have achieved full participation in American society), but often people who want to be allies don’t know what to do. This is a very practical step to take.

The idea for motion graphics came from one of the blogs Ethan recommended in this week’s readings (although I’m not sure which blog now). Here’s the inspiration video from Linda Dong.

More on restroom access rights here from Lambda Legal. The “I’ll Go With You” ally site is here.

A Community Tackles Diversity

For my four hour challenge, I decided to kill two birds with one stone by covering an event using the tool Audacity — a platform I’ve been wanting to try out for a while. Knowing I would need as much of that time as possible for editing, I stayed close to home for this assignment to speak to my fellow students and colleagues about ongoing efforts to promote an inclusive community at The Fletcher School.




I am not a designer. At all.

Luckily, Canva allows me to compensate for what I lack in design sense. It’s a fairly easy to use tool (even if it is restrictive).

Canva is a graphic design tool that uses a drag and drop UI to allow you to create anything from blog graphics to posters. The essence of it is that you choose a theme, add elements to that theme (like a grid structure, lines, icons and charts) to create an infographic.

Pros: It’s easy to use and it’s sleek. The icons are, for the most part, designed well and you can add your own images to build on what Canva provides.

Cons: If you’re looking to represent percentages that aren’t quartiles in a chart, good luck. Canva, for the most part, provides quartile percentages for its graphs (so stats have to be 25%, 50%, 75% or 100%) and the bar graph sizing isn’t great — you essentially have to guesstimate the proportion of the bars you use.

If you’re willing to forego a perfectly accurate data representation (for the bar graphs) and can live with using a pie / circle chart for quartile percentages only, Canva is a useful tool to display information beautifully. You can even be creative and forego the typical pie chart to display stats in a more innovative format — Canva has several templates that provide decent inspiration like this one:

Screen Shot 2016-02-17 at 4.56.15 PM

If you have any questions about using Canva or how I created my graphics for my media diary, let’s chat!

SQL (the quick and dirty way)

Navigating SQL is daunting at first, but completely doable, even if you have little programming experience. I’ll walk through how I ran SQL queries against my Chrome browsing data and the tools I used to do it.

Finding the SQL database

First, make sure you know where the SQL Database is in your computer, and make sure you have the appropriate permissions to access it. For example, when I was accessing my Chrome browsing data, I needed to get to this location on my computer:
~/Library/Application\ Support/Google/Chrome/Default/History

Where “~” means my home directory and “History” is the name of the SQL database file.
If this is daunting, don’t worry! This is what you’d type into a command line. If you don’t know what a command line is, either Google it or come talk to me — I’d be happy to walk you through it. *Note, if you’re looking to access your Chrome history database, make sure you’ve quit out of your Chrome browser. I learned this the hard way.

Notably, the Chrome history database is SQLite and not SQL. For our intents and purposes, this is fine. Functionally, they’re similar in usage; SQLite is a subset of SQL. Just be sure you know what type of database file you’re working with before you start.

How to access and browse your database

So you have your database file. How do you access it and browse through it? The tool I use is sqlitebrowser, made specifically for SQLite. You can open a database file or even create your own. Once you open your database, it looks something like this:

Screen Shot 2016-02-17 at 4.33.18 PM

You can browse through the data row by row, view the structure of the database and execute SQL commands. The “table” dropdown refers to all the different tables in a database; for the Chrome history example, there’s a table for downloads, a table for URLs, and a table source for visits.

It’s great that you can view all this information, but you’re probably also looking to make some meaning from this. To extract rows that are relevant, you’ll want to write a SQL query. I’m not going to go into the details of writing SQL queries here, but I do recommend W3school’s tutorial for the quick version which should be good for most basic queries.

In general, your queries will follow a structure that’s something like this:
select *
from "urls"
where "last_visit_time" > 13099253131722513
and "url" like "%facebook%"
or "url" like "%twitter"
or "url" like "%github%"
or "url" like "%linkedin%"

“select” refers to which columns you’d like to select from the table (I just choose to display all columns by default), “from” refers to which table you’re using and “where” acts like a conditional — if x is true for row i , then include row i in results. You’ll notice my use of “%” for matching strings — these are wildcards (and is easiest to Google as needed).

Below is what’s returned when I run the SQL query I wrote above:

Screen Shot 2016-02-17 at 4.39.52 PM

You can also group together results (like I could’ve grouped by “url” to see how many of each type of URL i visited) and sort by a column.

If you’re new to programming, this probably seems overwhelming, but I definitely think reading the tutorial and just playing around with some SQL queries will help you get the hang of it. I learned basic SQL by having a test database and running queries to figure out what did and didn’t work, as well as how syntax works.

If you have any questions about the post, about databases or about SQL, please reach out! I’d be happy to chat. 🙂

Tools from yesterday’s discussion

Alexis from FOLD here—thanks for the great discussion yesterday!

Here are some of the tools and resources mentioned during my presentation:

  • Storify
  • Genius
  • Embeddable Context Cards by Vox
  • Sketch (alternative to Adobe Illustrator, also allows you to create interactive prototypes. $99, but cheaper with a student discount)
  • Balsamiq (another prototyping tool, draws interface elements in a “sketchy” style, which can be really useful for getting honest early-stage feedback)
  • Meteor.js (Javascript framework we used to build FOLD)

If you’d like to give FOLD a try for one of your assignments this semester, head over to fold.cm to make an account. You can log in with e-mail or with Twitter (signing up with Twitter allows you to embed tweets in your stories).

We’ll be releasing lots of new features over the next month. Please let me know if you have any questions or feedback! You can reach me at alexis [at] fold.cm, or on Twitter: @alexishope / @readFOLD.



Can we use Peer-to-Peer transfer technologies to upload videos from mobiles?

It was during an impromptu assignment back in India few years ago that I first found this problem. I could shoot a video while covering an event, but sending it down to my editor using mobile data connection proved tough. My chance of being a pioneer in my bureau and earning some ‘brownie’ points with my editor were soundly dashed. To cover-up my frustration, I blamed poor data connectivity and was under the impression that it was a problem we face only in India. I was proved wrong when I came to the US in 2014. Anyone with a T-Mobile connection in Medford Campus of Tufts University would attest to the fact that ‘no signal’ is not a beast that troubles souls selectively.

Pun apart, my understanding about the limitation that I was facing in uploading videos through data connectivity of a mobile was again challenged when I took a video of my three-minute pitch and tried to upload the file on to my Box Folder, I could not send a 100 MB file through my mail. This time I was on Harvard Wi-Fi and viola, my mobile crashed again and again. I was forced to concede defeat and transfer the files onto my computer and well, rest is ‘going to be’ history.

These experiences taught me something interesting, for a person to shoot a video and upload it onto a website without using any of the apps being provided by the likes of YouTube or Facebook, could face a serious problem as both the strength of data connectivity and the ability of a handset to handle large file transfers can decide whether a data transfer can be made successfully in the first place.

With the definition of journalism changing fast and live streaming and quick video uploads becoming a norm in journalism, the ability of a journalist to not just shoot a video but also to upload it becomes a key prerequisite in their trade. But this ability can be severely compromised if he or she is working in a place with spotty data or Wi-Fi connectivity.

Is there a possible answer?

One of the issue we face in huge file transfers is that in the event of a disruption, the entire transfer fails. I think this could be addressed with the possibility of breaking down the file into manageable packets and transferring it them in sequence, my like peer-to-peer torrent transfer.

Peer-to-peer transfer technologies are also coming of age. For example Terranet is testing its Mesh technology to connect mobile devices without the necessity of having a data connection. If these incredible peer-to-peer transfer technologies are harnessed, I feel that we can create a process that can be used by journalists in creating and distributing multimedia files even from the remote places. While I could not find a product that does this function yet, I guess we can explore the possibilities of tackling this issue this semester.