I was at a digital journalism conference when Apple released iOS6 and set off a firestorm of criticism over their custom built mapping application, so perhaps it was inevitable that I would connect these things. In fact, I have written before about how journalists can be the “information cartographers” of the digital age, mapping the ecosystem of news and helping us find our way. However, as I have been reading up on how Apple built its maps I think there are some important lessons for journalists who are thinking about data and community in important new ways.
Late last week Matt Marshall over at Venture Beat pointed me to a blog post by Mike Dobson, who was Rand McNally’s chief cartographer for almost 15 years, that has quickly become one of the most cited sources on Apple’s mapping problem. Dobson had actually predicted a lot of the problems Apple is now having earlier this summer when the company announced it was building its own mapping application.
For Dobson, a key source of Apples problems come from a lack of quality control in regards to the underlying data that is powering their maps. Specifically, Dobson argues, Apple lacked the human element that has always been a part of map making. “Perhaps the most egregious error,” writes Dobson, “is that Apple’s team relied on quality control by algorithm and not a process partially vetted by informed human analysis.” He continues later in the post:
“While the mathematics of mapping appear relatively straight forward, I can assure you that if you take the informed human observer who possesses local and cartographic knowledge out of the equation that you will produce exactly what Apple has produced – A failed system.”
The day after Dobson wrote his post I was sitting in a session at the Online News Association on measuring the impact of journalism led by Greg Linch. Linch has argued previously that we need to get beyond basic web metrics (pageviews, unique visitors, time on site) to measure the value of journalism. From his original post:
“For journalism, the goal should be to add more meaning to the information we use to measure our work. […] The quantities of metrics increase because the works of journalism they describe are meaningful. Or, put another way, impactful. So, what if we measured journalism by its impact?”
But how concretely do we define impact and then measure it? That was the guiding question of his panel, but was probably too big a conversation for the confines of a brief conference session. The session featured a demo of Sparkwise, a dashboard for measuring impact designed specifically for media makers, and representatives from other data tracking applications, like Chartbeat, spoke up in the Q&A session. However, in the end, I think two largely unaddressed tensions stopped up from moving past fairly standard ideas about what and how we are measuring success in journalism.
- Quantitative versus Qualitative: Quantitative data is incredibly hip right now. You can slice it and dice it, create infographics with it, compare across organizations and sectors. But the impact journalism has on our audience cannot be summed up in clean, easily measurable numbers. We need to develop better tools for collecting, measuring, and tracking the subjective and qualitative feedback we get that illustrates our impact on the world. We need stories as well as stats. (*Hat tip to Jonathan Stray who I chatted with a bit about this topic. Read some of his thoughts on how journalism “works” and what it “does.”)
- Online versus Offline: As more and more of our work is reaching our community online it is tempting to adopt online measurement tools to track our impact. However, regardless of where our journalism is consumed, we hope it’ll also have impact in the larger world – our city streets, the halls of power, in our schools and around dinner tables. We need to grapple with how we measure our impact on and offline. In his presentation Linch pointed to impact measurement models from international development, but I think we could also look to community organizing for other models.
At the heart of both these tensions is how we engage our communities. I worry about an over-reliance on abstract digital data points for measuring impact just as news organizations begin to take seriously what it means to engage with their communities in meaningful ways. In making their new mapping application Apple relied too much on data sets, without testing that data against the real world. Journalists shouldn’t make the same mistake.
Map makers have a word for this: Ground Truthing. One of the best definitions of ground truthing comes from Terry Tempest Williams who describes it as “The use of a ground survey to confirm findings of aerial image or to calibrate quantitative aerial observations; validation and verification techniques used on the ground to support maps; walking the ground to see for oneself if what has been told is true; near-surface discoveries.” Indeed, ground truthing shares many qualities with journalism.
If we are going to measure our impact in the world we have to get out in our world. We have to join community conversations, we have to be better listeners, and we have to ask different kind of questions. Our journalism shouldn’t be seen as something that happens to the community, but rather with the community. More than ever, as the ground beneath journalism shifts and our newsrooms adapt, we need to be testing our assumptions and ground truthing our data.
According to Dobson, the cartographer, community is also the key to how Apple will solve its mapping mishap. “Apple does not have enough qualified people to fix this problem,” Dobson writes. “Apple needs to get active in crowdsourcing. They must find a way to harness local knowledge and invite their users to supply local information, or at least lead them to the local knowledge that is relevant.” All of our dashboards and metrics are meaningless if they are not infused with intelligence and feedback from our local community.
This is work intensive, but it is critical. In a post on measuring civic impact for journalists Ethan Zuckerman points out, “What we measure, we become. If we measure only how many people view, like or tweet, but not how many people learn more, act or engage, we run the risk of serving only the market and forsaking our civic responsibilities, whether we’re editing a newspaper or writing a blog.” As such, what we measure and how we measure it should be part of the conversation we have with our neighbors, not just in our newsrooms.