Philip Meyer, the pioneering journalist who covered the 1967 Detroit riots using computer-assisted reporting, wrote Precision Journalism. Agustin Armendariz, a reporter from California Watch, analyzes data for investigative reporting.

By Frank Bi
Voices

Californians know that living in the Golden State means going to sleep knowing a catastrophic earthquake might wake them up.

Located in the state with the most earthquakes in the continental United States, the 810-mile San Andreas fault line has been known historically for its potential to be violent and deadly.

Using a variety of databases, California Watch, an investigative reporting group based in Berkeley, showed in an exposé earlier this year that roughly 20,000 building projects in California public schools were not certified to be earthquake resistant.

“Who knew?” said Corey G. Johnson, an education reporter for California Watch. “I sure didn’t.”

Gradual advancements in technology have not only made computer-assisted reporting faster and easier for reporters, but also become an increasingly essential investigative tool for reporters, many of whom are swimming in an abundance of publicly available data.

“At its most basic level, [computer-assisted reporting] is dealing with electronic information,” said Agustin Armendariz, a reporter who specializes in data analysis at California Watch. But he says computer-assisted reporting in today’s newsroom has become more sophisticated as computer programs are now able to analyze large amounts of data quickly. The California Watch investigation used several applications that have only been available in the last five years.

Johnson, whose byline appeared on the earthquake investigation, worked with a team of about 40 reporters on a multimedia package that included information for parents, a downloadable coloring book and an interactive map.

The map shows fault lines in each county, while a street-level view allows the user the ability to read each school’s status in complying with the state’s certification requirements.

“If you were trying to look at individual maps for 10,000 schools, it would take forever,” Armendariz said. “But because we can use computers to do a lot of the grunt work for us, we can start tying together things in the order of 10,000 schools and hundreds and hundreds of maps.”

Detroit is the birthplace of computer-assisted reporting. Many credit the first example of this new wave of journalism to a Pulitzer Prize-winning investigation into the 1967 Detroit riots.

Philip Meyer pitched the idea to Detroit Free Press editors of using a computer to crunch survey data on 437 individuals. He used an early statistical software on an IBM 360/40, a mainframe computer now mainly shown in museums. Meyer was able to dig underneath the data and create a profile of a rioter.

The results were shocking at the time as many preconceived beliefs about the rioters were shattered. Many assumed those who participated in the riots were not educated and economically poor, but Meyer found neither economic status or level of education had any correlation with riot participants.

“What I did was fairly straightforward,” said Meyer, now a professor emeritus at the University of North Carolina at Chapel Hill. He admits he could have done everything by hand without a computer, but it would have taken longer than the three weeks the newspaper had budgeted.

Today, the same type of investigation Meyer conducted could be done much faster, depending on the scope of the data crunching.

The California Watch investigation took over two years to produce, said Armendariz, who added that thanks to an open-source web-application framework called Django, his team was able to overlay several maps with the data in due time.

With data ranging from fault lines provided by the U.S. Geological Survey, building data from the State Architect’s Office, maps of seismic hazards from the Department of Conservation, in addition to the listing of public schools provided by the Department of Education, the web application “weaved it all together,” Armendariz said.

However, a bulk of the time was still used to “clean” the information, standardizing the data so they could be processed easier.

The database the Department of Education provided was not up to date and had several schools inputted insufficiently, resulting in an extra effort by the team to verify the schools’ location and authenticity. They had more than five full spreadsheets of data, with nearly 15,000 lines.

Instead of going through each one by hand, the team used tagging that matched the address of a school to the topography and seismic maps to speed up the process and avoid human error in correcting each line by hand. The effort still took one year, Armendariz said.

“You can’t really over emphasize how important it is to be able to automate this stuff with computers,” Armendariz said.

Some of the newest technology used in computer-assisted reporting have been created by journalists.

At Duke University’s Sanford School of Public Policy, former Washington Post reporter Sarah Cohen recently helped create TimeFlow, a chronology tool.

“As a reporter, one of the most difficult things to do is to work with a timeline while you’re covering the story, like a long court case,” Cohen said.

Currently a professor at Duke University in Durham, N.C., Cohen says she and others are working on testing several reporting tools, including DocumentCloud, an application that provides relevant information about the people and places mentioned in documents uploaded by the reporter.

Tools like DocumentCloud, TimeFlow and Django are often free and available readily online, but their use depends on the resources and manpower a media outlet is willing to devote to a project.

“In an actual newspaper, I don’t think this gets done unless you spend a lot of your own time doing it,” Armendariz said. “Not that there aren’t staff on small newspapers that couldn’t do this, it’s whether or not that publication feels that they have the mandate to put the time in.”

Time is a luxury many news organizations cannot afford to give, said Johnson, whose story was initially supposed to have a quick turnaround, but became a much larger project after the data were analyzed.

Years from now, the same investigation will take much less time, predicts Armendariz, who said algorithms will make it a lot easier to match and pull things together.

Aron Pilhofer, editor of Interactive News at The New York Times and a co-founder of DocumentCloud, says that in the future, text mining will be the next big thing under the umbrella term of computer-assisted reporting.

Also known as text analysis, text mining is the uncovering of previously unknown information by using artificial intelligence to extract data from different documents and databases.

“The goal is to take away some of the mind-numbingly dull redundant work and help people work quickly so it can free up manpower for other reporting,” Cohen said.

Although computers are needed to help reporters better process and understand what the story is behind electronic databases, these technologically savvy journalists stress old-fashioned investigative reporting is still essential.

“It started with the shoe leather, just the basic detective work,” Johnson said. “The analysis done by my guys helped us focus our reporting and who we questioned to find out more.

“But without the data, we would have never been able to know the full scope and breadth of what was going on.”

Follow Frank Bi @frankiebi.

By Kyle Kim
Voices

Detroit Free Press senior managing editor Jeff Taylor leads a Monday morning online news meeting. The Free Press and the Detroit News made changes from the seven-day home delivery model two years ago.

Two years after Detroit’s largest newspapers reduced their seven-day home deliveries, officials from the organizations say the drastic change has helped them stay afloat in an industry where flagship newspapers around the country are sinking.

The Detroit News and the Detroit Free Press scaled back delivery to just three days a week — Thursdays, Fridays and Sundays — in March 2009, as other newsrooms cut reporters to bail news organizations out of financial crisis.

On days when issues were not delivered to front porches and mailboxes, readers would have to go to newsstands and other retail locations or go online for the e-edition, a replica of that day’s print edition in PDF format.

A year later, the papers began offering daily home delivery through commercial paper carriers if readers opted to pay for a separate subscription plan.

The idea was that if the money saved from cutting down printing and distribution exceeded the losses from advertising and circulation revenue, it could prevent deep cuts to newsroom staff, according to Detroit Media Partnership officials.

“It was a very calculated risk, but it was a huge risk nonetheless,” said Joyce Jenereaux, president of Detroit Media Partnership, which jointly operates the business side of the competing, independent newspapers.

A major factor in the decision was forecasting that the papers were able to retain 77 percent of revenue, while trimming expensive printing and distribution costs during the rest of the week.

But the transition was not completely smooth.

“Of course, we had complaints, and, of course, there are people that still prefer to hold a newspaper in their hands,” said Free Press editor and publisher Paul Anger.

Detroit News union representative and metro reporter Santiago Esparza said many employees didn’t buy into the partnership’s reasons for the business change in 2009 and still fear one or both papers may close its doors in the near future.

On top of weeklong furloughs in 2010, bargaining unit employees for the two dailies also took pay cuts and health insurance concessions.

“People are miserable,” said Free Press union representative Jocelyn Faniel-Heard. “You got people doing more work for less pay and benefits. It gets tiring for folks.”

Both Anger and Detroit News editor and publisher Jon Wolman acknowledge their newsrooms have not been entirely immune to staff cuts, but they credit the changes to home delivery for keeping layoffs to a minimum and original reporting a priority. Both are delivering news across innovative platforms, such as mobile phone applications and e-readers for an increasing digital audience.

The Free Press had a 39 percent web traffic increase this year, compared with March 2009, according to a circulation report, while the Detroit News saw a 13 percent rise during the same time.

Officials caution, however, that the business model is not a one-size-fits-all solution for newspapers.

“The reality is, it works,” said Rich Harshbarger, Detroit Media Partnership vice president of consumer marketing and communications. “It might not work for every market, and it might not work for the industry, but it’s right for Detroit.”

Follow Kyle Kim @kyleykim