Under the Hood

Can you say more about the mini-CMS features that the rig enables for simple interactives and visualizations?

AH: Each mini-project can be linked to a Google Spreadsheet. We’ve used Google Spreadsheets as a kind of low-fi CMS for our larger projects, and it’s worked pretty well for us. It allows us to separate text from code, so an editor can enter or copyedit content while I set up the rest of the project — and there’s no worry of a DIV tag accidentally getting nicked or anything like that.

It’s also great for populating basic data tables. There are only so many TR/TD tags I can stand to write! (There’s an example of the template markup at the bottom of this post about responsive tables on our team blog.)

And you can use multiple sheets in your Google Spreadsheet, each serving a different purpose. One sheet might be a data table that you loop over, and another sheet might be more one-off chunks of text, like a headline or credit.

Did any interesting challenges arise on the dev side as you built out the rig?

Chris Groskopf: This project was a fairly natural evolution of our existing app-template, so we had already solved most of the problems. Probably the two most interesting pieces from a technical perspective are the responsive iframes javascript library (that we have since factored out as pym.js) and the little bit of Python magic that allows each graphic to have its own configuration. Without going into too much detail on the latter, our rendering infrastructure will dynamically import a file called graphic_config.py if it exists and then use the contents of that file to configure the graphic template. In particular, we use this to specify a unique Google spreadsheet for each graphic that uses the copytext part of the rig.

AH: I think another challenge, just workflow-wise, was to keep this as lightweight as possible for the user. For one, most potential users of this rig on our team are beginning-to-intermediate-level coders (including those of us who make graphics and a couple photo editors). Certainly, it takes time to get comfortable with the command line, but this can’t be horribly complicated. Second, a lot of the work that we do with dailygraphics is on-deadline stuff, and we need to be able to publish (and republish last-minute edits) with minimal overhead. I think we’ve achieved that.

Also, since this rig is used almost daily, we find all kinds of interesting edge cases (or just plain bugs) that inform changes to the overall rig, or to components like copytext. So we’re constantly improving on this, which is great.

As a sidenote, the two-repo setup that keeps the “machinery” open while protecting NPR’s copyrighted or embargoed content is kind of ingenious. How did you end up with this solution?

CG: This is something we’ve struggled with frequently at NPR. We want to open source everything, but for perfectly legitimate reasons our lawyers don’t want us implying that the source code license applies to the content (text, images, etc.) by having them in the same repository. Previously, this caused us to develop both copytext and the assets rig, both of which are now part of dailygraphics.

For dailygraphics the problem was compounded because the source code effectively is the content in pretty much every case. Moreover, we’re building charts and tables, which are also frequently things we don’t want to slip out before we’re ready. To that end we looked at a variety of ways of segmenting the repository. Initially I had thought we would use a git submodule, but ultimately it ended up being far simpler for other users of the rig if we just set up two repositories and handled the connection between them with configuration. This also has the ancillary benefit of making it very easy for new users to get started.

Getting Set Up

If I’m in a newsroom that hasn’t yet adopted something like the App Template, what kind of set-up am I getting into if I want to start using this? What kind of skills do I need?

CG: In general, if you have the skill to build a web graphic with HTML, CSS and JavaScript, then you probably have the skills you need to use this rig. Beyond web programming fundamentals, you’re going to need an Amazon Web Services account so that you can use S3, which is where the rig deploys the graphics. You’ll also need enough command line familiarity to setup your S3 credentials, checkout the project from Github and install the requirements. If you want to use our copytext component for managing text, you’ll need to have a Google Drive account.

That’s really it. We go out of our way to work with easily accessible technologies, so if you’re doing any web development then there is a very good chance you already know everything you need to know in order to use it.

Is there anything especially tricky to keep an eye on here as a potential user? Common blunders?

AH: I think we’ve ironed out any big trip-ups over the past few months of using it. The main one for me since we’ve split up the repo between dailygraphics machine and actual graphics is managing my terminal tabs — committing to GitHub from one tab, but doing everything else from another tab. But that’s fairly minor.

Change the Code, Change the Culture

One of the things that’s most interesting about a lot of the code coming out of NPR these days is that it’s sort of a carrier wave for culture and good practices—in this case, for treating even small projects in a clean, standard way. How much of that is conscious? And what kind of impact are you seeing from this approach?

CG: It’s very, very conscious. I take the craft of software development very seriously and I believe that there is an intrinsic danger in treating small things as unworthy of being done well. If all of our projects are important then they all deserve to be done thoughtfully. If they aren’t important then we shouldn’t do them. I spoke about my ardent desire for code quality in my NICAR 2014 lightning talk this year. As I have said to several people who have asked me about that talk, I saw that as a first salvo in a mission to insist on quality software development practices in our community. This year especially, I’ve tried to ensure we are good role models for how to code cleanly, write good documentation, and provide transparency into our methods. Fortunately, everyone on the Visuals team shares an amazing commitment to quality and I think we’ve all seen the tangible benefits of working this way. All that documentation we write is for us too. And all those blog posts we write clarify our thinking and provide fodder for the next iterations. Everything we do is about reducing our own internal dependencies, including turning our code out into the world for everyone else to poke holes in.

Thanks, Alyson and Chris!