Harnessing the power of historical Twitter data

Thursday, 1 September 2016

Want to understand consumer reaction to the launch of the iPhone 6 when it came out 2 years ago? Looking to understand the evolution of public opinion in Brazil about the Olympics since Rio won the bid back in 2009? Need to analyze the last 3 years of marketing campaigns around Black Friday? The full archive of public Twitter data has 10 years of conversations around almost any topic you can imagine. By analyzing this incredibly rich data set, businesses can uncover key insights to make better decisions.

Through Gnip, our enterprise API platform, we offer a range of historical data solutions that provide complete access to this rich archive. Historical PowerTrack provides batch access, and our Search solutions provide instant access to the complete archive.

Here are a few of the ways that we see businesses capturing the power of this data.

Develop benchmarking
Monitoring real-time conversations and trends around specific topics on Twitter has never been easier with PowerTrack. However, you or one of your customers might suddenly become interested in a new topic. By unlocking the power of historical Twitter data, you can go back and analyze the topic over the course of history and compare it against the new data coming in. This enables you to spot seasonal trends, measure reactions to previous product launches, and spot key inflection points around the popularity of specific topics.

Analyze unexpected events
A powerful attribute of our Search products are their ability to instantly provide you with insights on recent unexpected events that your real-time rules were not set up to capture. Given the live nature of Twitter, it’s difficult to adjust your real-time data capture rules as events are breaking on the platform. Should you want to dive deeper into what happened, a simple and effective solution is to use Search to backfill the data you need.

Create powerful product demos
Using Search during the sales process can help you close new business by demonstrating the power of your product using real, relevant data about the customer you are pitching.

Enable customer exploration
Discovering new topics in real-time can be challenging; your PowerTrack rules have been set up to explicitly filter out noise. With Search, you can perform unstructured discovery by applying new operators to the full Twitter archive, instantly analyzing the results, and refining your query as you begin to discover new signals in the data. Once you find a new topic worth monitoring, you can apply the exact same ruleset to your real-time stream with the confidence that no noise will be introduced.

Create a rich onboarding experience
During the customer onboarding process, you can use our Search APIs to preview the data returned by a customer’s new rules. Once the customer is happy with the data, you can run a Historical PowerTrack job to pull in large volumes of data matching the rules, all the way back to the first Tweet in 2006. This enables you to provide your customers with a valuable and interactive experience right out of the gate.

Identify signal
Search provides the fastest, easiest way to get up and running consuming full-fidelity Twitter data. You can filter out the noise and extract only the historical data you need by querying using our standard PowerTrack operators.

An often overlooked use case of this is testing out your PowerTrack rules on historical data before applying them to your realtime streams or a large-volume Historical PowerTrack job. Search makes it simple by giving you immediate, visible feedback around the combination of PowerTrack operators that you have entered, allowing you to tinker and experiment with rules in real time. Since complete Tweets are returned, it is easy to screen whether you are using the right keywords, geo boundaries, or other operators needed to get your expected results.

Save on data costs
Measuring the signal of Tweets returned by your PowerTrack rules is not the only preview functionality available via Search. The counts endpoint of the Search APIs enable you to estimate the volume and velocity of Tweets returned by your PowerTrack rules, should you apply them to your real-time stream. This enables you to catch and amend any issues with your rules that would result in you collecting too many or too few Tweets from your real-time stream.

These are just a handful of use cases and examples that are made possible with access to historical Twitter data. Ultimately, the possibilities are almost endless. For more information about our full suite of historical products, including pricing, contact [email protected].