Open Journalism & the Open Web

Not that I don't already respect journalists, but reading AIDS in the Heartland by Banazynski made me respect an aspect of journalism which I don't often think about: the deeply human, personal storytelling, which lets us as readers connect with a world very different from our own.

My personal reaction to the story itself aside, what struck me most about the piece was how interdisciplinary an endeavor it must have been to create. This is a story that mixes medical science, regional politics, rural folk-ways, radical activism, and gay culture (to name a few) all into one coherent narrative. You could even say that despite the "old media" long-form text format, this piece is actually a collage of datapoints and vignettes from many different perspectives.

Compiling and managing all that information must be a huge task, and I can imagine two main stages. The writer first has to learn enough about various and sundry specialized fields to be able to give readers a competent account of the relevant highlights (for example: what AZT is and what it does in 1 paragraph). They then have to take all of this information presented in diverse forms and formats and distill it into a single written story.

I wonder if the first part, the learning, is transforming due to the web. Quick, easy access to specialized information is a given on the web. Perhaps more importantly, useful introductory information on virtually any subject is easily available especially in places like Wikipedia and specialized online messageboard communities. This makes it much easier for anybody to become a "15-minute expert" on a given subject.

The second part cries out for discussions of multimedia, mashup, non-linear formats for the presentation of a story. I'll be honest, I did not read every word of AIDS in the Heartland. At points I found myself scanning, picking out interesting sections and drilling into them for more detail. This is the default mode of information absorption on the web, and I wonder how a piece like this could be presented in a way that preserves its literary long-form majesty, but also acknowledges the reality of how people like me are going to interact with the story.

“Everybody’s a reporter”
My takeaway is: How can existing news sites take advantage of this trend? More people are reporting – blogs, tweets, video – from the site of everything from shootings to Selectmen meetings. When New England Patriots quarterback Tom Brady was in a car wreck a week ago, The Boston Globe/boston.com used a picture on page 1 from a bystander. (Who doesn’t have a camera in their pocket these days?)

More reporters (or commentators) means a broader base of information for readers. Hopefully that translates into a more transparent society.

On the flip side, as more people report, the competition for readers’ eyeballs becomes more intense, which leads to more aggressive reporting, which leads to… what? It will be interesting to see which types of reporting/reporters predominate in the coming years.

”A fundamental way newspaper sites need to change.”
Everything Holovaty says makes sense. I do data nearly every day, and understand well how it can be used to enhance storytelling. It kills me how little data is used in the paper or online. For instance, I do a simple chart called “Snapshot” once a week that shows, for instance, avg taxes for towns in Greater Boston. The chart is popular online, for a day or two, then disappears. But the information could be recycled – or “repurposed”, as Adrian put it – endlessly in stories, pictures etc. There should be links every town a specific town is mentioned (or at least when taxes are mentioned) to the chart. That’s a simple example, but there are tons of data that could be constantly used to help flesh out people’s understandings of the complex world we live in.

“The journalist as programmer”
Two years ago I was speaking with Aron Pilhofer at a conf at MIT and we both agreed that if we could do college over again, we’d at least minor in computer science. This report does nothing but confirm my own feelings. I want to improve my programming skills. This is a trend that’s been building for two decades.

I am part of a generation of journalists who grew up having to traditional media as a reference. Or so we had to for some time. It's strange, but we attends a special period where the media are competing without any geographical boundaries, and where the loyalty of readers, the possibility of building what we call "reading contract" is questioned every day. While readint the four articles recommended by the course, I got four or five articles more. I'm not sure where they came from, but they were there. I do not know if I will ever find them when I try to remember where they were, I don't know where will I archive the materials that this course is offering to me. While looking for information for an article, beyond the conventional access to sources (interviews, reports), we get saturated information from local, regional and international uncertain sources. Many times, in the net, I honestly do not know how I got the information I was seeking, and often get information that was not looking. The possibility of working with programmers, or think as a programmer in a journalistic content, is a way to build new types of reading contracts also with digital readers: the own look. While Twitter seems to be breaking the way we relate with the news, journalists have to think about new ways to get readers to stay in front of our stories. Undoubtedly, the creation of digital tools and special crews to think content for the online world will be a long task against the inertia of traditional media, especially media peripherals such as Página/12, which I work in Argentina . However, my main concern is how can we think of new content without losing our valuable time to think for ourselves.

Adrian Holovaty and Stijn Debrouwere both have the same fundamental question and issue: What is the best way to portray information to our readers and why do news organizations believe there is only one way to do it?

In Holovaty’s 2006 piece, “A fundamental way newspaper sites need to change,” he writes that news organizations need to stop the “story-centric” worldview, or that the only way to provide content is by collecting information and putting it into a story.

During our weekly meeting, my editor asks, “What stories are you working on?” Usually, I don’t think twice about that question. But after reading Holovaty’s piece, I think that perhaps my editor should ask, “What information are you seeking and how do you plan on sharing that information once you get it?”

Here is an example: There’s been a ton of written stories about
bank failures, but for me, The Wall Street Journal provides the information best by this regularly-updated interactive graphic.

Four years later, Debrouwere writes in “We’re in the information business” that most news organizations still have that “story-centric” worldview. He also points out that news organizations need to develop a structure, a content management system, that makes it natural for news organizations to repurpose content for a variety of platforms and consumer needs.

The key problem, Debrouwere writes, is that most content management systems are really web publishing tool. It simply takes a news story and puts it on the web, with no considerations of other new ways to present that content.

This piece, simply affirms why we need to have this course. There is a clear disconnect between content providers (journalists) and content producers (programmers). We have a real opportunity to work together to provide models that will better meet the goal in Holovaty's piece: "...more concern for important, focused information that is useful to people's lives and helps them understand the world."

***
On a separate note, regarding Matt's earlier post (see above) about minoring in computer science, when I began college a decade ago, my mother encouraged me to double major in computer science noting my interest in technology. I brushed her off, saying "I need to focus on my journalism career." Now I wish that I took her advice!

>Perhaps it's time for some MYSQL study.

Surely not. With websites like GeoCommons (http://geocommons.com/) you don't need to learn any database details. My only caveat is make sure you are not giving rights to your information when you use online sites. For example, when a mashup is created with Google Maps, Google has the right to use that data forever without payment (I heard this from a trusted source, but have not read the licenses myself.

I'll be honest. I was bored by the AIDS article after the first ten pages. At 22 pages, it seemed predictable. Predictable is the wrong concept. Perhaps it was that the writing style or tempo didn't change? I wanted to see that picture of them when they were happy so that I could connect to them. I'm not sensitive by nature and so probably not the author's targeted demographic.

I'd be more interested to read about the contrast between two sets of couple - one gay and one not - and how their experiences shaped their reactions and situations.

I don't know if my comments are going to help this forum or are relevant to the class. Let me know if I head off into tangential areas. Or simply don't reply. I won't be insulted.

The idea that most grabbed me, from all the four "journalist" articles, was Debrouwere's suggestion for a journalistic Markdown-like language, one that would finally capture the historical databases journalists slowly build over the months and years, helping spot trends that might otherwise go unnoted.

That piece and Holovaty's were obviously tackling a very complex, cumbersome problem, and while both were inspirational I'm not sure either really illuminated how we put the semantic into the semantic web.

Take one example: "A college graduate has a home state, a home town, a degree, a major and graduation year." This person also may or may not have parents, a job, debt, an original graduate year, a second degree, an arrest records, or millions of other data points that any given news article may or may not touch upon. And those data points can be incredibly useful, in fact critical, down the line if the paper decides to put together a LilSis or Muckity.com, for example.

If you're just putting the data away semantically for the sake of doing it, I worry you're creating busy work on an already taxed team, particularly since so many databases are started and then shortly abandoned, or else the methodology of how they are created is improved, thus breaking any sort of valid historical analysis that once could have been made.

I guess I'm just extra skeptical, because MuckRock.com has recently started juggling salary and budget data sets, and even those are so inconsistent across agencies as to be incomparable, so I wonder how we're possibly supposed to make these future datasets as anything but neat gimmicks without some serious re-working of core journalist practices.

Those questions aside (and they're definitely questions, not criticisms!), the pieces were all fantastic and thought provoking.

I read Holovaty's digital prescription for newspapers, and the Chicago Reader item on the expanding definition of who exactly is a journalist.

Both pieces were about how media is changing, and about the resulting social friction. (Incidentally: They also embody that change, with a rich set of reader comments adding context, inquiry, clarification and further intellectual depth.)

However, I feel that the sense of social conflict identified by Holovaty -- of intransigence on the part of the journalists -- has turned out to be fairly fleeting. The news industry has been trashed, and with so many jobs gone, all bets are off. Folks are taking up the medium with more of a sense of appetite and opportunity -- and no doubt urgency.

On the other hand, the social conflict around "who is a journalist" is underplayed in the Chicago Reader piece. Remember that Feinstein and Schumer's federal shield law is being modified in the wake of the WikiLeaks affair to include only "traditional news-gathering activities," according to the New York Times. That discussion is far from over.

The real point of Holovaty's piece is the "atomization" and remixing of discrete, characteristic data that appear in newspaper articles ... to some degree, it seems like something of an technological inevitability. The comments pointed out that this might ultimately not require a human actor, just a good script that can scrape articles.

Others pointed out that this sort of interconnectivity between datums is exactly the point of the semantic Web, though whether newspaper websites embrace the open, semantic Web or do it all on an walled or tiered in-house system remains to be seen.

Data-linked story structure is a powerful narrative tool, but it's only in an open-Web context that the structure will give readers true agency.

"Everybody’s a Reporter" discusses how in Chicago they are redefining who is considered to be a report. The past definition of this that was very narrow and limited to the few well established new outlets. Nowadays with the advent of the internet and explosion of media outlets there is a strong need to modernize the definition of what a journalist is to be more inclusive. While I think the steps taken in Chicago are good I think it is important that there still be some limits in place. The requirement to provide proof that someone is connected to a media outlet should provide that check.

"A fundamental way newspaper sites need to change" lays the direction that the author believes news paper web sites need to go. It is important to remember most newspapers have a long history of being just that, papers. Not to long ago simply being on the web was a huge step forward for many. If more papers pushed hard to do “repurposing” I think that would go a long ways towards making the news accessible to more people. While the point of “structured data” and not posting everything as a “news article” has some merit I think it also has some drawbacks. I wouldn’t want to start reading news articles that were simply assembled based on a handful of data pieces. To be effective news needs to be so much more then just a handful of data points put together. Maybe it is just semantics but it feels like the story still needs to be the foundation and then added layers of data could be available for those that want to learn more or dig deeper.

Adrian Holovaty, in clear, concise language, presents how today’s media establishments refuse to see the wealth of information contained in the data that make up stories. His call for this change in his “A Fundamental Way Newspaper Sites Need to Change” asks media managers/reporters to find ways to collect that data and repurpose it in a way that gives readers actual information instead of a daily text lecture about what is going on in the world.

Four years later, Stijn Debrouwere, weighs in, discussing how in four years since Holovaty’s call to arms, there’s been no large movement on the structured data front by the media. But instead of offering a concrete program or method as to how to how to loosen the bottleneck, goes on at length at how the media needs to get even more micro about the data and find ways to repurpose it.

In other words, no one has come up with an easy, CMS answer to how any one in a newsroom could capture this data and repurpose it. And no, Caspio isn’t that plan no matter how much some newsroom managers say it is.
So what’s missing here, is the unwritten third essay with “the plan.”

Not the “plan” that saves newsrooms (the old how-do-we-make-news profitable plan that every one waits around for someone else to come up with the answer we can copy like someone’s 10th grade geometry homework). I’m talking about a yet, unwritten “plan” that gives us some sort of template, an add-on that incorporates in the publishing tools already installed in newsrooms to allow anyone in a newsroom – from the lowliest agate clerk to the highest-ranking editor – to both mark and cull these objects in a story for repurposing now or later.

This lack of incorporation of Excel and Access into a publishing system is what is keeping newsrooms from being more efficient. No one has come up with that. That’s the essay I want to read. While I’ll always read the countless essays in which another academic infocrat proclaims that MSM is “reluctant to change” (as if this industry is the only one where that’s a problem), what I want now is a real plan, a real template that does more than identify the problem.

Burt Herman comes closest to making the sort of publishing/culling tool a reality with Storify. He’s taken an old AP verb and made it new it again. He’s done what many newsrooms need: a tool that places the placing the culling process into a publishing template that anyone can use.

It's not that newsrooms don't want to change and don't want to repurpose the massive amounts of data that comes at them in the reporting process. It's that they have a profound lack of efficient tools that can be utilized quickly. The first programmer to fix the efficiency problem, where marking and culling the data is as easy as writing the story that provides context in one program (no whipping out of Excel or a database manager is needed) will be the one credited with truly bringing publishing into a new era.

After plowing through everything on the journalist reading list there are roughly one zillion threads I want to pursue with the group and my colleagues. One thing I want to put right up top, however, is the issue of passion.

I was struck by this nugget from "The Journalist as Programmer":

"The department sought and hired people with a passion for journalism, an interest in telling an important story and the ability to work across departments. These are not necessarily traits that automatically come with someone from a computer science or software development background."

Hell, these are not necessarily traits that automatically come with someone from a journalism background. I'm of a mind that the world doesn't need more people who call themselves journalists--it needs more people who are passionate about journalism. Thinking of it this way it becomes easier to bring hacks and hackers together and call what they do journalism rather than some clunky hybrid like "programmer journalist."

If you broaden things a bit to include people with a passion for storytelling, all of the resistance and issues of arrogance discussed in "A fundamental way newspaper sites need to change" and "We're in the information business" can be beat, so long as we are talking about storytelling by any means necessary.

Reading Holovaty and Debrouwere, I thought of the bins and bins of printed data I have in my basement--information I collected while reporting stories for newspapers and alt-weeklies before I was giving any thought to story forms beyond the "blob of text."

The information in those bins is what I didn't use in dozens of news stories and investigative pieces. I keep them to remember exactly the kinds of information that I wasted when I did stories the old-fashioned way and to remind myself to always be thinking of new ways for using that kind of information. In some cases, I am actually going back to that data in my new job with MinnPost and building something new from it.

I'll stop there, though it's not for any lack of a burning desire to go on and on and on...

Something that strikes me about this piece is the polemic employed. It's less common for inflamed or emotional polemic in journalistic pieces that are liberal, which is why someone like Michael Moore is noted as standing out in recent documentary film-making. Emotional and inflammatory polemic tends to come in the shape of illiberal rabble-rousing, by the likes of Fox News and Rush Limbaugh. Indeed, the thing that strikes me about "AIDS In The Heartland" is exactly this very emotional polemic running through a story tale structure.

This brings me to my next point. Is this a good thing to do? This is crass generalisation, but bear with me; part of the theory behind the success of right-wing campaigns, such as the Tea Party movement, comes from the fact that a lot of them are run with an impassioned emotional narrative with scarce regards for facts and figures. In contrast, a lot of liberal viewpoints are illustrated rationally with frequent recourse to statistics (e.g. this is why a disadvantaged poor family would do better under us, with the figures to prove it). As sad as this is, a lot of people tend to be persuaded by the irrational narrative, even if it's against their best interests. But would "AIDS In The Heartland" really benefit from being reduced to an article about the facts and figures of people living with AIDS? Well, it wouldn't obviously. This is why the author has given the article that title; to show that AIDS isn't some metropolitan liberal disease, but can strike in the heartland of rural America. This is a hook to people who might not otherwise read it, rather than just something that might look like preaching to the converted.

I'm imagining that, as a web developer, reading "AIDS In The Heartland" is to do with ascertaining how useful social media is in promoting news stories and journalistic pieces. On one extreme, do we really need a human factor in reporting some stories? As was illustrated in the second lecture, you can actually assemble a sports story with a mix of open (sport) data and clever news feeds. You effectively make the human sports journalist redundant. It doesn't really surprise me that sports journalism *could* be as reductive as this. However, does this mean we can translate any news story into atomistic data? The other extreme is to figure in only the human factor with no access to data. Then you're likely to get either introspective pieces, which would either produce the journalistic equivalent of navel-gazing, daily gonzo journalism (which would eventually just kill the journalist) or the equivalent of an extreme version of Fox News (some people would say there's no such thing as an "extreme version of Fox News"!).

The conclusion? Journalism by it's very nature is human, so the storytelling narrative is vital to get other people to actually read it for the emotional content. But in the current world of expanding open data, journalism that doesn't tap the Open Web will seem increasingly poor and irrelevant. Journalism doesn't need to abandon its human heart either; the richness of the data should enhance the richness of deeper storytelling. Despite the fact that "AIDS In The Heartland" is a great example of heartstring-tugging observational documentary and confessional interview (which you can agree with or not), there's still enough journalistic integrity in hanging real facts* on the tapestry, albeit sparingly.

* "He has his blood tested to determine his white blood cell count his body’s natural defense system. It often is below 1,000; a healthy person’s count would be closer to 5,000".

I presently work in a shop that has been in survival mode for the past three or four years, which is not really a new concept for newsrooms across the country. It is a challenge every day to do more with less but when you don't have the tools or people around to implement the ideas brought up in the readings, it is discouraging to see your effort withering on the vine.

My paper still is behind the curve of being able to develop these data-driven reports. Our web presence is less than compelling, though it does look nice. It leaves me with the "So what" that is supposed to come next. Being able to collect good data, write interesting stories and then have it all saved into a newsroom system that can find new and innovative ways to present the information should be the goal of journalists. News organizations that can achieve this goal will flourish (find a way to make money), while those that don't will be shuttered.

Even though I'm a newspaper guy, newspaper websites face increasing competition from television news websites as more put their content online. TV news websites can horn in on the news space newspapers once held almost exclusively but it is harder for newspapers to do the reverse. You can get headlines and stories from both but most newspaper websites aren't going to have the volume of video.

Newspapers can reclaim their news gathering superiority by capitalizing on their in-house talent to produce interesting content that can be delivered in print or electronically. But there has to be that commitment on the part of newspapers to delivering the "So what does it mean to me" aspect to their readers.

I would argue that the easiest way to win an argument is to establish a pattern that illustrates "the issue" your trying to prove. Often times when I gather information for news stories I try to identify any patterns because I find my work can be that more useful and compelling when there is a pattern identified. For example, if there is a car accident at intersection X, I'd be interested in reporting the news but then also look at other reports to see if there is a pattern there, have other accidents happened there and why?

In "A fundamental way newspaper sites need to change" Adrian Holovaty argues newspapers should have a system in which readers can access raw data and facts alongside the print story. Although Holovaty wrote this piece in 2006 parts of his proposal are still innovative. Allowing readers to identify patterns for themselves can help build strong emotional connections to both newspapers and the stories they print. In "Everybody's a Reporter" Fullerton writes about the importance of press passes but just as important is access to data, having a system where readers can access raw facts like the one Holovaty writes about can also make a tremendous difference for bloggers and independent writers trying to access historical data to write more comprehensive stories.

Response to: Holovaty piece, and Royal piece.

It was interesting to read Holovaty's torch-bearing 2006 piece, and then to read the formalized case study from earlier this year; the former is short and sweet, but it hits upon most (if not all) of the areas discussed in the latter: platform vs. content, foundational news culture vs. evolving news culture, and user experience.

Looking at my own opening paragraph here, it's telling that I naturally use the adversarial "vs." to describe the concepts being explored in these pieces. For most veteran journalists, the rules for finding, refining and constructing a story have been, until recently, absolute. But now, journos must re-evaluate that process and adapt in order to survive; the ones who can't get beyond their aversion, will not.

One concept that I responded to was User Experience: Holovaty acknowledges this by emphasizing that a "big blob of text" has value, and should not be wholly replaced by data; Royal hits upon this by describing how the NYT team builds its interactives with the user experience at the forefront: they limit instructions, avoid excessive manipulation, etc. This is key, as the fundamentals of a "news story," i.e. inverted pyramid, nut graph, etc., need parallel conventions in the technological realm.

Here's my question: What are the new rules in hacker journalism that guide the structure of the final product? (THEN: Inverted Pyramid; NOW:?)

Good Stuff!

Sol