One of the many fields experiencing transformational change in this digital revolution is that of the journalist. As citizens, we struggle with the information firehose. We work to teach our students how to filter information, how to assess the reliability of sources, how to efficiently locate the really useful stuff. But journalists — good journalists — have to do this with much more data much more quickly than we mere mortals. If they miss something, they get scooped by the competition. If they trust an uncredible source, they could end up in court as part of a libel suit, or find themselves issuing an embarrassing public mei culpa. For Wired magazine’s April story about anthrax, the paper archive of documented sources for the story is 17.5 inches high and weighs more than 36 pounds. Even writing this blog post, I feel like I should tell you that this information comes from page 18 of said April issue of Wired. I’d document it properly if I could remember how to document print sources.

With the recent events in Egypt, Libya, Bahrain, and Japan, we’ve seen an unprecedented use of social media to report what’s actually happening on the ground. The Twitter feeds, in particular, of participants and eye-witnesses give us access to primary sources that we’ve never had before.

At the same time, though, it’s also easy and profitable to fabricate. Rumors can spread like wildfire. Users can exaggerate or falsify to promote a particular political purpose. And the anonymity required to give voice to the oppressed also protects those with a hidden agenda who are trying to obscure the truth.

Andy Carvin has become a pioneer in navigating the Twitter minefield. As a Senior Strategist at National Public Radio, he has been using Twitter to establish contacts and produce reports from the most volatile locations in this politically active year.

Last week, Carvin spoke with This Week in Google about his work. Thanks to the TWiT network’s use of Creative Commons licensing, I can extract the 18-minute segment addressing Andy’s work and post it here.

Talking toTWiG’s Leo Laporte, Jeff Jarvis, Gina Trapani, and Google’s Chris Messina, Carvin discusses his work, including the process for determining the reliability of his sources. He also gives away a few secrets on how to find eye-witnesses by doing Twitter searches after a major event (warning: there’s some mild profanity).

Most of this boils down to analyzing the network. We tell our kids that they’re judged by the company they keep. This is true online as well. My blog is made credible by the people who link to it. My Twitter account is reliable because of the people who follow it. If they know what they’re talking about, and they’re listening to me, then I probably know what I’m talking about too.

At this point, all of this is done by hand. There aren’t any great tools to look at the network as a whole and figure out where the reliable sources are. We’re getting close with some of the marvelous analytics tools being developed (see Gephi, ThinkUp, and Snapp for examples).

As journalists stuggle to remain relevant, they’re finding significant value in their ability to sift through the information, identify trends, provide analysis, and make connections. That’s certainly worth my NPR membership.

Photo credit: David Warlick
Audio Credit:  This Week in Google