Features:
A Bot to Find the Source of Serendipity
Jacqui Maher reveals the secrets of @NYTPlusContext
“Here’s a secret: You can make gravy”
“not in the face but in the intestine”
Just before Thanksgiving last year, a new novelty Twitter account gained notice in our newsroom. @NYTMinusContext, promising “All Tweets Verbatim From New York Times Content. Not Affiliated with New York Times.” tweeted fragments from Times articles that you might not think twice about while reading in article format. Isolated, though, these phrases, like the ones I picked above, can be absurd, surprising, and delightful.
I found myself copying and pasting many of these tweets into the search box on nytimes.com, wanting to know how on earth expressions like “a scrum of Euro-kissing and plastic surgery Expressionism” flowed into our coverage.
You could say I was hooked from the very first tweet:
“wrote one wife, as translated by the Assyriologist Klaas Veenhof”
Whose wife? What did she write? In what language was it translated from? And what’s an Assyriologist?
One night my colleague James Robinson and I were talking about the account when we had an idea: is it possible to add the context back to these quotes without ruining the serendipity of @NYTMinusContext?
That was when a little robot named @NYTPlusContext was born.
How We Made It
The @NYTMinusContext Twitter bot to which @NYTPlusContext responds.
@NYTPlusContext is a well-meaning automated Twitter account that runs every few minutes, grabbing any new tweets from @NYTMinusContext. I initially had it query The New York Times using the regular site search. It quickly became apparent that the person—or algorithm—sourcing these quotes was spelunking into the archives as well as recent content, though—the site search skews results towards current news. Switching to our internal full-text article search API led to far more accurate results. Once the bot finds a match for the quote, it tweets the headline and a link to the original article @NYTMinusContext.
@NYTMinusContext Happiness Is a Warm iPhone http://t.co/mRZXpsezzs
— NYT Plus Context (@NYTPlusContext) February 26, 2014Making this bot was really simple. After setting up OAuth credentials to interact with Twitter programmatically, I just had to write a few lines of Ruby thanks to libraries like the Twitter, curl and oj RubyGems.
require 'twitter'
require 'curb'
require 'oj'
class TwitterBot
attr_accessor :client, :last_tweet_id
def initialize
self.client = Twitter::REST::Client.new do |config|
config.consumer_key = "CONSUMER_KEY"
config.consumer_secret = "CONSUMER_SECRET"
config.access_token = "ACCESS_TOKEN"
config.access_token_secret = "ACCESS_TOKEN_SECRET"
end
end
def internal_search_for(term)
json_data = Curl.get("https://our.internal.search/api").body_str
data = Oj.load(json_data)
result = data['results'].first
end
def add_context
tweets = client.user_timeline("NYTMinusContext", :since_id => last_tweet_id)
if tweets && tweets.last
puts "found #{tweets.size} tweets"
self.last_tweet_id = tweets.last.id
store_last_tweet
tweets.each do |tweet|
if result = internal_search_for(tweet.text)
nyt_tweet = "@NYTMinusContext #{result[:headline]} #{result[:url]}"
client.update(nyt_tweet, :in_reply_to_status_id => tweet.id)
end
end
end
end
end
TwitterBot.new.add_context
Former OpenNews fellow Brian Abelson showed up while I was writing the code above. James and Brian quickly got to work creating the profile for @NYTPlusContext, relocating the man and his sign from a grassy field to our building’s lobby and selecting the right Archie comic for an avatar.
We agreed to reply directly to each tweet, limiting views to users who follow both accounts, instead of trying to work around that restriction. Fans of whimsy, we have no desire to take it away from internet when we find it.
Credits
-
Jacqui Maher
Are we there yet? Is that a knife? Why does Barbie want a shrimp? What’s an arvo? And other things I ask Australians.
