Making the Viz

The SOTU visualization took about four weeks to design and develop, and we already had a corpus of data to work with: the tweets that happened during the 2013 SOTU address and the text of the speech itself from last year.

It started off looking something like this, which showed the volume of tweets per state, minute by minute on Twitter:

For every minute, we would extract a key quote from the address and resurface the most mentioned keywords on Twitter. Each dot represents the volume of tweets during a given minute for a given state. At the right the image captures an image of the president during the address with a key quote and also we can see most mentioned keywords on Twitter during this minute (the blue labels). The visualization didn’t show any interesting trends, besides illustrating that Washington D.C. has a high level of tweets during the address. This is normalized by Twitter users on each location.

We wanted to add some more text analysis, and this came in with version two:

For this second version, we annotated each paragraph of the 2013 address with its timestamp gathered tweets mentioning #sotu in that window. Then we analyzed the keywords in those tweets to get topics out of them. For example, if tweets mentioned “war,” “army,” “afghanistan,” then we would tag those tweets as belonging to the topic “defense”; and so on. The state-by-state analysis now is condensed in the margin. The extra space is used to show the actual transcript.

We started to get closer to the final result with the next iteration, where we played with a timeline on top of the transcript that would show the volume of tweets around each topic during the address. First with a simple timeline:

Then with the more aesthetically pleasing streamgraph:

The streamgraph visualization created using the JavaScript InfoVis Toolkit, made by Nicolas, was used as the index for the transcript. If you were interested in what happened during a peak in tweet volume around the topic “defense,” then you can click there and you’ll be taken to the paragraph where that happened. The margin visualizations served as a detailed analysis for each paragraph, showing the real-time conversational aspect of the platform. The map on the right hand side (created with d3) would show which subjects were heavily discussed in which states, all normalised by Twitter users there.

The Test

So, we had the viz, now we needed to populate it.

As soon as the President started talking, we started annotating the speech with timestamps so that we could match the text to the viz, plus the new dataset surfaced some bugs that we had to quickly fix.

We were up until 2:30am but we had it made and working so that the next day you could explore the data with the speech. Yes, you tweet what you think—now we have another way to show that.

Follow @TwitterData to catch more of these viz as we make them.