I finished a project with a home-built table that was about 16GB, some 60 million rows by 110ish fields. It was…big. Sometimes it was painful. Mostly, though, it worked out, and it got us what I think is a damned good story. Anyway, I think it was Ben Welsh who’d observed something like: We have some good tools to work with Big Data, but not great tools for data that’s not quite so big. I ran into that situation.

Welcome! This is the first of an occasional series of posts I’ll be writing here about how news organizations around the world are going about the same things differently. The plan is to focus on the kind of work that doesn’t fit neatly into our CMSes, or that need custom design or code. I’ve been doing this work for seven years and remain to be fascinated by the varying approaches to the same subject matter. When news breaks, (or perhaps when news is planned 12 months ahead) how do we in media make the same things, differently?

Data can be impersonal, especially large datasets with thousands or even millions of records. The fact that most data of this magnitude is calculated by machines is, however, a sharp contrast to the ultimate goal of examining it in the first place—to find human trends and patterns behind the numbers.

The mini-series Mars, that aired on the National Geographic Channel in November 2016, imagined what it would be like to live on Mars in the near future. For the interactive narrative Rewind the Red Planet, we endeavored to show Mars as it was before it was a red desert, back to a time when liquid water may have run freely, between three and four billion years ago. We wanted to allow readers to see ancient Mars in its entirety from a planetary scale, how it may have featured a vast northern ocean, or may have had water trapped in expansive glaciers.

A new Mac OS X desktop app, autoEdit, creates automatic transcription from a video or audio file. The user can then make text selections and export those selections as a video sequence, in the editing software of their choice.

Data journalism training in Berlin, plus a bunch of upcoming meetups around the world.

OpenNews wants to help you get to events, plus meetups later this month.

In researching Settling for Misconduct, we had to account for details from hundreds of county and federal court filings, identify thousands of officers named in civil complaints and tally hundreds of millions of dollars in monetary awards. We also needed thorough reporting to connect issues of police misconduct to fiscal accountability. And oh yeah – we had to have a slick web app to present the data to the public.

Slow start to the new year for events, but that means more time for fellowship applications.

Call for pitches, looking ahead to 2017.

As a Knight-Mozilla fellow, I wanted to do some type of research during my fellowship that could benefit the news community. During my 10-month fellowship in Berlin in 2016, I spent about eight months researching, collecting data and interviewing reporters, editors, managers, and directors about their on-boarding and off-boarding processes.

Working in the news cycle often leaves little time to design different iterations of story ideas, and even less time to test them with an audience. But as an author or designer—and as a media outlet which provides a public service, like WNYC at New York Public Radio—we have a responsibility to know how the public might understand and engage with our stories. The reason to make time for some kind of usability testing is because it makes our work better and increases its impact on the audience we serve.

We wanted to get a better idea of where President-elect Donald Trump gets his information. So we analyzed everything he has tweeted since he launched his campaign to take a look at the links he has shared and the news sources they came from. But first, we had to get the tweets.

Lessons learned from trying to create one chart with as many applications, libraries, and programming languages as possible.

This week, eleven contributors gathered with us in Washington, D.C. to work on a new resource—a playbook for open-sourcing newsroom code. Together we hoped to tackle a question that’s come up again and again: how to help more newsrooms produce open-source projects, so that everyone can spend more time on great journalism instead of re-creating common tools, tech, and datasets from scratch.

Cleverly named meetup from Journocoders, and a scant few other events and deadlines before the end of the year.

The Los Angeles Times created a new publishing system by focusing on the needs of editors and reporters, supporting great journalism with better tools.

Deadlines this week for the JSK fellowship and IFF conference, plus some upcoming events.

We can take advantage of our human ability to analyze natural language and use really simple techniques to assist and amaze our users. Here are a couple of ways to use these techniques in your own projects.

Session ideas are due soon for the Internet Freedom Festival, plus a bunch of other upcoming grant and fellowship deadlines.