Source

Jump to site navigation Search this site

Features:

A Bot to Find the Source of Serendipity

Jacqui Maher reveals the secrets of @NYTPlusContext

By Jacqui Maher

Posted on:

“Here’s a secret: You can make gravy”

“at least mildly fascist”

“not in the face but in the intestine”

“no, no, relax your mouth”

Just before Thanksgiving last year, a new novelty Twitter account gained notice in our newsroom. @NYTMinusContext, promising “All Tweets Verbatim From New York Times Content. Not Affiliated with New York Times.” tweeted fragments from Times articles that you might not think twice about while reading in article format. Isolated, though, these phrases, like the ones I picked above, can be absurd, surprising, and delightful.

I found myself copying and pasting many of these tweets into the search box on nytimes.com, wanting to know how on earth expressions like “a scrum of Euro-kissing and plastic surgery Expressionism” flowed into our coverage.

You could say I was hooked from the very first tweet:

“wrote one wife, as translated by the Assyriologist Klaas Veenhof”

Whose wife? What did she write? In what language was it translated from? And what’s an Assyriologist?

Ask Etsy, apparently.

One night my colleague James Robinson and I were talking about the account when we had an idea: is it possible to add the context back to these quotes without ruining the serendipity of @NYTMinusContext?

That was when a little robot named @NYTPlusContext was born.

How We Made It

The @NYTMinusContext Twitter bot to which @NYTPlusContext responds.

@NYTPlusContext is a well-meaning automated Twitter account that runs every few minutes, grabbing any new tweets from @NYTMinusContext. I initially had it query The New York Times using the regular site search. It quickly became apparent that the person—or algorithm—sourcing these quotes was spelunking into the archives as well as recent content, though—the site search skews results towards current news. Switching to our internal full-text article search API led to far more accurate results. Once the bot finds a match for the quote, it tweets the headline and a link to the original article @NYTMinusContext.

Making this bot was really simple. After setting up OAuth credentials to interact with Twitter programmatically, I just had to write a few lines of Ruby thanks to libraries like the Twitter, curl and oj RubyGems.

require 'twitter'
require 'curb'
require 'oj'

class TwitterBot
  attr_accessor :client, :last_tweet_id
  
  def initialize
    self.client = Twitter::REST::Client.new do |config|
      config.consumer_key        = "CONSUMER_KEY"
      config.consumer_secret     = "CONSUMER_SECRET"
      config.access_token        = "ACCESS_TOKEN"
      config.access_token_secret = "ACCESS_TOKEN_SECRET"
    end
  end

  def internal_search_for(term)
    json_data = Curl.get("https://our.internal.search/api").body_str  
    data = Oj.load(json_data)
    result = data['results'].first
  end
  
  def add_context
    tweets = client.user_timeline("NYTMinusContext", :since_id => last_tweet_id)

    if tweets && tweets.last
      puts "found #{tweets.size} tweets"
      self.last_tweet_id = tweets.last.id
      store_last_tweet

      tweets.each do |tweet|
        if result = internal_search_for(tweet.text)
          nyt_tweet = "@NYTMinusContext #{result[:headline]} #{result[:url]}"
          client.update(nyt_tweet, :in_reply_to_status_id => tweet.id)
        end
      end
    end
  end  
end

TwitterBot.new.add_context

Former OpenNews fellow Brian Abelson showed up while I was writing the code above. James and Brian quickly got to work creating the profile for @NYTPlusContext, relocating the man and his sign from a grassy field to our building’s lobby and selecting the right Archie comic for an avatar.

We agreed to reply directly to each tweet, limiting views to users who follow both accounts, instead of trying to work around that restriction. Fans of whimsy, we have no desire to take it away from internet when we find it.

People

Organizations

Credits

Recently

Current page