In October 2022, we published the largest-ever investigation into Toronto’s Vision Zero program, a multimillion-dollar road safety initiative started in 2016 that has been widely considered a failure by road safety advocates. Our goal at The Local—an award-winning Toronto-based magazine covering social issues and urban health in the city—was to figure out how and why things were going wrong, and, if possible, quantify the scale of the problem. The city was facing a municipal election that month, and we wanted to scrutinize what our elected leaders had achieved in the last four years.

What we found was significant inequity in the way the policy was being enacted. Toronto’s Vision Zero operations rely on local city councillors to make decisions about their wards, and our investigation found massive disparities between how frequently downtown councillors and their counterparts from lower-income and racialized inner suburbs bring forward and enact motions on road safety infrastructure.

The project we thought we’d do

About a year ago, The Local’s editor-in-chief, Tai Huynh, suggested we do an investigation into road safety in school zones across Toronto. The city’s open data portal publishes detailed data from speeding monitors in school zones across the city, and we thought we’d map out the worst spots, talk to some parents and teachers, do some data visualizations. All-in-all a pretty straightforward investigation. But when we found out about the Data-Driven Reporting Project, we thought, “Why not apply and expand the scope of the project? What’s stopping us from being really ambitious?” (The DDRP is a partnership between Medill and the Google News Initiative that helps journalists work on investigative data projects; you can check it out here.)

Road safety in Toronto is notoriously bad, and Vision Zero is referred to as “Zero Vision” by pedestrian and cyclist advocates in the city. But there hadn’t yet been a comprehensive investigation pinpointing why, or quantifying the problem. Our expanded plan included filing FOIAs for requests for road safety work, to see the scale and pace at which improvements were—or rather, weren’t—taking place across Toronto, and doing a comparative analysis of which parts of the city were the most unsafe.

What we hadn’t prepared for is the fact that the city of Toronto keeps essentially no records on city councillors’ requests for road safety improvements.

The project we did

City councillors have two avenues for requesting road safety improvements from the city’s transportation department: through community council meetings and through the city’s 311 portal. That 311 system is the same one the public uses to make requests, and records don’t really indicate whether a request was made by, say, a concerned parent or a councillor’s office. The FOIA officer told me it would take thousands of hours and wading through tens of thousands of documents to divide them up. And even then, the accuracy would be low, at best.

We also couldn’t just compare the road safety improvements made in each ward overall because the city doesn’t have an accurate, up-to-date list of all the road safety improvements made over the last four years. I hope I’m painting a vivid enough picture of Toronto’s data transparency and records-keeping infrastructure.

After this first blow destroyed our plans, it became clear that the only reliable way to find out how active city councillors are in implementing road safety infrastructure is through the city’s council meeting portal, which lists the details of every council motion. Then came the second obstacle: While council documents seem relatively similar at first look, motions aren’t catalogued in a uniform way that code could scrape and sort. Many meeting documents don’t even list the name of the councillor who put forward a motion, much less the details behind it. I realized, with a hint of overwhelming dread, that quantifying these motions would require a human eye. But it was also the only shot we had to put any numbers to the city council’s record on Vision Zero.

This is where we were about four months from our planned publication date. From there, the process was actually relatively simple, though incredibly tedious. I drafted a list of 30 keywords on road safety infrastructure, pulling heavily from the Vision Zero website: phrases like “speed hump,” “signal” or “traffic control signal,” “stop sign,” “crosswalk,” and more. The full list of motions containing those keywords proved to be way too much to analyze within our time constraints, so I picked the four keywords with the highest number of motions associated with them. That left us with about 800 unique motions from the 2018–22 term.

I then searched for each keyword in the city council’s database of meetings, read through the attached report and minutes (which, mercifully, were usually quite brief), and noted specific details from each corresponding motion. My manual dataset tracked which councillor put forward the motion, for which ward, the type of request, whether or not it was carried, and who voted against it. The process took months of slow chipping away, and revealed essentially what we had hypothesized: that councillors in downtown wards were taking a far more active approach to implementing road safety improvements than their peers in the inner suburbs, areas of the city with poorer transit infrastructure and a greater dependence on car use, and which are home to a greater proportion of lower-income and non-white residents. For good measure, I did an overview of how frequently all 30 of our original keywords came up in community council meetings, where these decisions are made, and found that the downtown council mentioned them more often than all of the suburban councils combined.

When we applied for the DDRP award, we imagined that a significant amount of the resources would go toward FOIA requests and analyses. Instead, given the lack of information to request, we expanded my hours on the project for the heavily manual data assembly. The DDRP award also helped fund the few FOIAs we did request, along with data visualization tools, and accompanying aerial photography. It gave us security to know we could tackle something so ambitious.

When I think of data journalism, I hadn’t quite envisioned a project this manual—but how else are we supposed to quantify a problem that hasn’t been organized or analyzed in any comprehensive way? I did receive responses to some of my FOIAs, including those related to the number of requests for safety improvements made by members of the public. Those matched our findings on the councillor level, indicating that residents of wealthier and whiter neighbourhoods were 1.5 times more likely to request road safety improvements than their lower-income counterparts.

I also did the more straightforward analysis of speeding in school zones—the idea that had begun this massive journey—and spoke to policy experts, parents, school workers, and people affected by traffic collisions. While we had initially hoped to do a more complex mapping component to this project, our cramped timeline and a rush of elections-related projects meant we had to scale down that ambition significantly.

Simultaneous to this project, our team was also working on another massive project writing fact-checked election bios for everyone running for municipal office, which put an immense amount of pressure on our time. Here’s a tip: Even though it all turned out great in the end, maybe don’t do that.

What we’ll be thinking about next time

Looking back to the start of our investigation, we couldn’t have predicted how intensive a process it would be—though even if we had, I don’t think it would’ve stopped us. This project has laid the groundwork for us to investigate issues at city hall for which no comprehensive records are maintained, and reduces our reliance on the FOI process for stories like this.

Personally, I got so deep into the data that I didn’t see its use, for our readers, as a means to an end rather than an end in itself.

For anyone thinking of conducting a similar project to our Vision Zero analysis, the most significant resources you’ll need are, rather unsurprisingly, time and money. Start a project like this six months or a year in advance, and chip away at it slowly. Don’t be afraid to be ambitious, but give yourself enough runway to do it sustainably.

One thing we wish we’d been able to do is publish the project even a couple of weeks earlier—it was released just a few days before the election—so that readers could spend more time with the information and use it to hold their councillors to account on road safety. Personally, I got so deep into the data that I didn’t see its use, for our readers, as a means to an end rather than an end in itself.

There are also some technical hurdles to be careful of, particularly when it comes to duplication—we had to be careful to avoid counting the same motion twice if it appeared under multiple keyword searches, and watch for motion follow-ups that were given a separate unique identifier.

But most importantly, this project exemplified how abysmal the policy tracking, data, and transparency infrastructure is in Toronto. We’d like to apply the same approach to other policy issues in the future, and we’re now trying to figure out what issues could be best served by this painstaking process. We’d also like to see if there’s a way to do the analysis using machine learning, something we’re able to learn about from fellow organizations that were awarded DDRP funds. Working on this dataset made me realize how much more transparent political bodies could be if they formatted and organized their motions in a way that could be scraped and analyzed with code, rather than having to be manually picked apart, understood, and categorized.

Perhaps there’s a way to encourage that uniformity and transparency, to systematize it for the benefit of the politician, the journalist, and members of the public—or maybe that’s something to daydream about the next time you find yourself reading through hundreds of city council motions.

• Share on Twitter
• Share on Facebook