LOEX of the West presentation, 2008

Peer Review 2.0: Tomorrow’s Scholarship for Today’s Students

LOEX of the West, Las Vegas

Anne-Marie Deitering & Kate Gronemyer


Five Web 2.0 themes — from the ACRL Instruction Section’s Current Issues Discussion Forum, Research Instruction in a Web 2.0 World (Annual, 2006).


{Edit: These didn’t make it into the presentation, but they are examples of some discussions on the web over the last year that started our thinking on this topic.}

danah boyd – open-access is the future: boycott locked-down academic journals

danah boyd – Viewing American Class Divisions through Facebook and MySpace

danah boyd – editing a special issue of JCMC






Current Anthropology

Expressive Processing: An Experiment in Blog-Based Peer Review – Noah Waldrip Fruin on Grand Text Auto

Cognitive Daily blog

ScienceBlogs – The World’s Largest Conversation about Science

BPR3: Bloggers for Peer-Reviewed Research Reporting — Icons


UsefulChem Wiki

Radiology Wiki

Open Notebook Science Using Blogs and Wikis (Jean-Claude Bradley, at Nature Precedings)

Rrresearch Blog

“The Academic Manuscript” — Wicked Anomie: Sociology Run Amok

Welcome to Nature Precedings


Barbara Fister points to this article in the Chronicle:  Certifying Online Research (Gary Olson), about the challenges of evaluating online publications.  See also Barbara’s post at ACRLog: Peer (to Peer) Review.

good interface + bad metadata =

well, not really bad metadata.  More like the wrong metadata.

Dipity lets you build interactive timelines. You can pull in all kinds of information sources — video, text, images — and display them in a nice, linear timeline. The interface is easy to navigate. Each item in the timeline can be viewed within the timeline, and each item has a handy “next event” button to make navigation easier.

Sample Dipity timeline about the Beatles

Anyway, I have some ideas for how to use this interface and if I ever get time to try them, I’ll post about it more.  But that’s the thing – putting this stuff together does take time.  If you choose the browse timelines option on the dipity site – there are a LOT of timelines with no events, with 2 events, with 4 events.  I’d imagine a fair number of them will never be finished.

So the other day I noticed this – Time Tube – a mashup that puts the Dipity timeline interface together with YouTube.  Same cool interface, but you just have to keyword search a topic and your timeline will be populated with YouTube videos a few minutes later.  It’s created a fair amount of buzz over the last couple of days, most of it positive because it’s fun to use and some of the timelines are pretty interesting.

But – the timelines are based on the date the video was uploaded.  If all you want is a nice browsing interface this is okay – as another way to display the results of a YouTube keyword search.  But as a way of visualizing information, if what you want is to add some kind of meaning or context to the videos, it’s only useful for a very narrow set of topics.

Compare this “Beatles” timeline to the one above -

TimeTube for \"The Beatles\" - 1 year span

It doesn’t look too bad, but there’s no real meaning there.  Not even a spike when Across the Universe was released.

TimeTube lets you pick a longer timespan – here’s fifty years.  This really shows the limitations.

TimeTube for \"The Beatles\" - 50 year span

Everything clustered in the middle because YouTube didn’t exist until a short time ago.  The fact that they let you open out your range to 20, 50, 100 years suggests that the upload date might not always be the way these things are generated?  Or maybe it’s just a holdover from the original Dipity interface.  The timelines created are dynamic and there’s no way to save them.  There’s no account to create so you can’t find timeline buddies either.

Where this is useful now is for topics, like “olympic torch protests” where people upload their videos right away after an event happens.  Or to track the zeitgeist when something emerges out of nowhere to become the next big thing.  Or as a fun browsable interface for a YouTube keyword search.

Digital content for free! (semi-free) (or something)

So, from Encyclopedia Britannica there’s now Webshare – making it easy for “web publishers” (which means – bloggers?) to access premium encyclopedia content, and to share that content with users.  From the project site:

a limited program that enables people who regularly publish content on the Internet—bloggers, webmasters, and writers for the Web—to obtain free subscriptions to Britannica Online, which includes the entire Encyclopaedia Britannica and thousands of additional articles as well as access to other reference databases, links to valuable Web sites selected by our editors, and more.

It looks like I can search the online site for free, and I can use links or widgets to share the information I find with others – even if that information would normally have a price tag.  Now, one reason for this is clear – as Kathryn Greenhill has already explained - Britannica wants more links because more links = more better in a Google-dominated world.

And clearly, Britannica needs to be more relevant – the realities of online writing are that you’ll pick what’s good and linkable over what’s great and not.  Jason Griffey at Pattern Recognition looks at Webshare in this light, and makes a strong argument that the project doesn’t go far enough down the open and sharing road.

So this is interesting, as a model for digital content sharing, but I’m also interested in how well the site  works as an online reference source.  And in particular as an online reference source that can support an process-based model of research, based on broad exploration.   A few years ago now – I think it was probably 2005 – we were revising a set of information literacy assignments that are embedded in OSU’s beginning composition course.  These assignments were, and are, intended to introduce just such a research model.

For a variety of reasons we decided that a lot of students needed to explore their topics in reference-type sources.  At the time, we had access to the online Britannica in my library and we thought “maybe we’ll send them to Wikipedia and to the published encyclopedia.”  After a few test searches using our students’ keywords we realized that this plan was doomed to fail.  The oddest and most specific searches were successful in Wikipedia, but even common terms like “biodiesel” were unsuccessful in Britannica.  More than that, the hypertext-rich articles in Wikipedia encouraged further and broader exploration, while the traditional entries in Britannica were clearly intended to stand alone.

I describe this whole process in more detail in a chapter in this book, and (with Sara Jameson) in this article, but the upshot is that we decided to send the students to Wikipedia alone, and we haven’t turned back from that plan.

So have things gotten better since 2005?

I searched on “biodiesel” which is my default sample search, which is a very popular argument paper topic among OSU undergraduates, and which yielded exactly no results in the online Britannica 3 years ago.  This time, we did a little better.  There seems to be a topic on biodiesel, but while it looks like there should be an associated article, the interface just churned and churned without loading one.  When it finally stopped, there was a notation indicating that biodiesel is a fuel, which isn’t much, but it is an improvement over last time.  There is also a note saying that this topic is “being discussed” on four external websites:  Biodieselnow, the National Biodiesel Board, Willie Nelson’s Biodiesel, and a biography of Rudolf Diesel.  There are also links to pages on Rudolf Diesel and Willie Nelson.

So – better, yes, but not especially good.  A student could do some exploration of this topic here, but there’s nothing to give them a sense of the discourse, to suggest entry points and keywords into that discourse – no article to help make them better searchers on the topic and only a limited range of places to look further.

And even beyond this, I’d like to extend Jason Griffey’s conclusion a bit -

In all, this is the right direction for Britannica to be going if they hope to ever be relevant in the 21st century, but they haven’t gone far enough. You need some serious added value at this point to compete.

I think his suggestions are spot-on, and I could see using Britannica for things as he describes it.  But for this particular assignment — even if all these suggestions came to pass I’m not sure any source created in a non-dynamic way could add the value I’m looking for.   As I’ve worked with students on this assignment, I’ve come to think that the very things that can make Wikipedia scary to some educators also make it an ideal resource to illustrate some of the important epistemological themes in academic writing:  the idea that knowledge is something constructed, or the idea that some problems are “wicked problems” with a whole lot of plausible solutions.

That’s a serious value-add for me that is only possible because of the community of people who co-create Wikipedia and because of the transparency inherent in their creation process.

What I haven’t had time to say about the Cult of the Amateur

was said pretty well yesterday at Daily Kos.

Keen tends to claim that the participatory web is destroying traditional media at great cost to our culture.  I’ve always thought that the mainstream media has done a great deal to destroy itself.  And I don’t think I can say it better than this:

The media — newspapers, radio, and television — is not made up of reporters running on a sparkling field of journalistic integrity.  Those reporters are instead embedded in a machine intended to do the one thing that Mr. Keen sets as the mark of professionalism — make money.  And the way the media has chosen to make money over the last few decades is, perversely, by devaluing their own product.

I’m not just annoyed by Keen in interviews and panels.  I think his completely uncritical acceptance of the traditional corporate media as a guarantor of quality is destructive to the very discourse he claims to embrace.  “Debate” about the participatory web that is sparked by arguments like Keen’s tends to look like this:

Andew Keen: But the problem is that gatekeepers — the agents, editors, recording engineers — these are the very engineers of talent. Web 2.0′s distintermediated media unstitches the ecosystem that has historically nurtured talent. Web 2.0 misunderstands and romanticizes talent. It’s not about the individual — it’s about the media ecosystem. Writers are only as good as their agents and editors. Movie directors are only as good as their studios and producers. These professional intermediaries are the arbiters of good taste and critical judgment.

David Weinberger: Actually, I’d suggested you take a look at the Top 40 songs. Of course you’re within your rights to cite the New York Times best-sellers list instead, but that’s indicative of the problem with your method. Are you seriously maintaining that pop culture off line is represented by six good books on the New York Times hardcover non-fiction list? Why do you find it so awkward to acknowledge the obvious point that the gatekeepers of commercial publishing and production — the producers of TV shows, magazines, pop music, movies, books — are usually driven not by high cultural standards, but by the need to reach a broad audience? Do I need to remind you that “The Secret” is likely ultimately to outsell all six of those worthy books combined?

Full-Text: Keen vs. Weinberger (WSJ)

Weinberger, or whoever is engaging with Keen-ish arguments can sit there taking home-run swings at Keen’s blind approval of the “media ecosystem” — making the earth-shattering argument that the mainstream media wants to reach a broad audience and make money.  I want to see what Weinberger says when really pushed about the limits and value of the participatory web.  Keen, regrettably given how much attention the media gives him, never provides that push.

And in libraries, the same thing goes on when Michael Gorman writes on these topics – because he has the same kind of un-critical acceptance of traditional scholarly methods as Keen does of mainstream media producers.  We need serious discussion about the implications of the read/write web for scholarly knowledge production, and that can only happen if we turn the same critical eye on traditional practices as we do on the new.  But as long as one can engage with Gorman by saying “peer review isn’t perfect” – that real discussion doesn’t have to happen.

(For an example of what I mean by “real discussion” – the March issue of First Monday is a good start)

(And let me say that I have a lot more sympathy for Gorman than anyone who would make the claim that movie studios these days, minor subsidiaries of corporate conglomerations as they are, have a clearer picture of quality than directors)

I haven’t read the comments on this piece – I don’t usually see Daily Kos because there’s too much discussion there and I know I won’t resist the comment threads even though I can’t keep up – so thanks to Copyfight for the pointer.

digital stories/ digital study spaces, following-up

The word “follow” suddenly looks really strange to me.

A few quick follow-ups from last week’s posts:

The Ryerson student who was threatened with expulsion for administering a study group in Facebook will not be expelled.  There will be some fallout from this episode that sticks with him, but he was not found guilty of 147 counts of academic dishonesty.  There’s been a lot of talk around my corners of the web about this, with probably more people than not coming down on the side that the kid was cheating.  But nothing I’ve seen has really pushed me off my original position.

I agree that the invitation to the group (which, I believe, this particular student did not write) was badly worded.  But until someone can show me that the students were actually getting grades for work and learning they did not do – I don’t see the cheating.  And I still think that because this study group was virtual, if cheating had happened, the evidence would be there to collect and display.

The first experiment in digital storytelling from Penguin is up at wetellstories.co.uk.  The story, The 21 Steps, is a thriller by Charles Cumming, and the platform for the telling is Google Maps.  I’m only partway through -but I’m having fun so far.  If you didn’t see Adrian Hon’s comment on my previous post about this project (ooh! alliteration) — he’s written a post about this kind of storytelling that gets into some really interesting stuff about how the medium affected the creative process.

(I believe you might find a rabbit hole on the main project page as well)

Oscar doesn’t get YouTube?

CinemaTech pointed this out last year, and again today — the Academy’s attitude towards YouTube is pretty messed up. And I think they’re blowing a terrific opportunity here. Like the Royal Family, the Academy of Motion Picture Arts and Sciences has exactly the kind of content that works on YouTube. They should have the key clips up immediately, just like SuperBowl commercials. But they don’t have any clips from last night up and there’s way less stuff from other people than you’d think — suggesting that they are being pulled soon after they go up.

And why is it that people who want to hang on so tightly to this content are usually so bad at doing anything interesting with it themselves? What I was going to write was that Oscar should have his own YouTube channel – but as it turns out, he does. So now what I’m going to write is that Oscar should have a good YouTube channel.

This is just a weird hodgepodge of clips — some are cool moments someone might actually want to revisit, but most are … not. Like – why do they have Sidney Poitier accepting his honorary Oscar, but not his real one? Seriously, which award is more meaningful? Why do they have nothing about the glitz and glamour – nothing even suggesting that there is a red carpet?

With the video resources AMPAS has – resources no one else has – they could really build a fascinating channel connecting the show and the history of movies and Hollywood and celebrity, but instead it really looks like they just grabbed a handful of stuff that was readily available. Maybe it’s just me just me but it looks like the Academy has something to learn about “show don’t tell” — almost half of these videos are talking heads telling us why the Oscars are important. I honestly think watching the great big movie stars and all those other creative people at the Oscars over the years would tell that story better for me.

It’s insane — if AMPAS understood what YouTube can do in terms of raising interest and awareness and buzz they could be using their archives to build a lot of interest, but with this they’re just reinforcing the idea that they’re (the awards and the organization) increasingly out of touch.

ETA — I’m thinking about what Caleb was talking about last week – when people get information they don’t stop asking questions – instead they get new questions. A kick-ass Oscar channel on YouTube doesn’t mean people will stop being interested in the Oscars – it’ll generate new reasons why they’re interested.

But – to be fair, my favorite moment of last night is up there and has stayed up long enough to get 142,222 views.

ETA — It’s gone now.  I don’t know when it went away, but the IP police have had their say.