So back in April, gg at Skulls in the Stars challenged science bloggers across the disciplines to read and research some classic article in their discipline, and then write a blog post about it.  The results are in, and they’re awesome.  Not just fascinating - this is a potential time suck (with none of the guilt I feel wasting time with old sports clips on YouTube - I mean, it’s reading about science.  Important science!) - but also a really intriguing way to think about introducing a lot of overlapping ideas about scholarship to students.

One - we all know that context is one of the hardest things to figure out when you’re taking your first steps into understanding a new topic or discipline. Which things to read, what do they mean, why were they important, why are they still important  - answers to these questions aren’t immediately apparent to an outsider and scholarship written for other experts takes a lot of the keys to unlocking this discourse for granted.  Each of these posts lifts some disciplinary curtain aside, telling us what to read and why - in language written not for experts but for smart, motivated people who don’t already have that contextual knowledge.

And by showing the significance of a work in a discourse, these bloggers also (in both text and subtext) show us something about what discourse is and how it works in science or scholarship or research.  My hands-down most favorite entry in this series is from the person who issued the initial challenge - the Gallery of Failed Atomic Models - and this entry really gets at what I’m talking about here.  From gg:

It is often said that history is “written by the victors”. While this statement is usually referring to the winners of a military or political conflict, a similar effect occurs in the history of science. Physics textbooks, for instance, often describe the development of a theory in a highly abbreviated manner, omitting many of the false starts and wrong turns that were taken before the correct answer was found. While this is perfectly understandable in a textbook (it is rather inefficient to teach students all of the wrong answers before teaching them the right answer), it can lead to an inaccurate and somewhat sterile view of how science actually works.

And that might be my favorite piece of this project - the view of how science actually works that you get from these articles is anything but sterile.  They’re planning a second go-round of this project, which will be hosted here in about a month.  I’m marking my calendar.  Well not literally.  But I’m glad this will be an ongoing thing.

There’s another version of the first set of posts up at A Blog Around the Clock - organized chronologically, with some great excerpts highlighting what makes each post good.

Thanks to Cognitive Daily for the pointer.

I would really like to respond to this call for papers, and since abstracts aren’t due for several weeks I’m using it as  a reason to do some reading and re-reading.  Right now, it’s A Pedagogy for Liberation, a dialogue between Ira Shor and Paulo Freire.  This isn’t the most famous Freire, that’s undoubtedly Pedagogy of the Oppressed, but it’s one of my favorites because it is a dialogue — and they talk about the benefits of that format in language that’s very compelling - all about co-creating meaning:

Dialogue belongs to the nature of human beings, as beings of communication.  Dialogue seals the act of knowing, which is never individual, even though it has its individual dimension (p. 3-4).

And since I initially described this space as a place where I might do some pre-writing, and that concept is entirely tied up in the idea that doing that pre-writing in a place that is not my own head might be useful and valuable in a way that internal reflection is not — I’m going to indulge in putting some of the ideas this re-read is sparking down here.

I got through the first two chapters last night (and for the record, this book is very short, and very readable).  And I have mostly been thinking since about the question of motivation and what it means for libraries.  Freire and Shor agree that motivation has to be located in the here and the now of learning - not in some future benefit or some future activity.  Freire says, “I never, never could understand the process of motivation outside of practice, before practice (5).”  Shor echoes this with, “I’d emphasize that motivation has to be inside the action of study itself, inside the students’ recognition of the importance of knowing to them (6).”

I find this really compelling.  I also think is something I need to think about a lot more in terms of library instruction because much of the motivation we provide to students in library instruction sessions is “learn this and you’ll see the benefits at some later time.”  We deal with that in a limited and imperfect way by requiring that students have a research assignment that we can teach to, but that just moves the point at which the motivation kicks in a little closer.  It doesn’t actually put it in the here and now.

There’s a scene in Dazed and Confused where my favorite character Cynthia says “God, don’t you ever feel like everything we do and everything we’ve been taught is just to service the future?”  That line is why Cynthia is my favorite character and that line is what Freire and Shor are talking about here.  And that line really describes some of my anxiety about library instruction sessions, particularly those of the one-shot variety in basic skills courses that are themselves presented to students as disconnected from the “real” work they will be doing in the disciplines.

This puts the motivation two steps away, right?  Learn these basic skills so that you can perform well in later classes and you want to perform well in those later classes so that you can get a good job. Can we really blame students for feeling like nothing they do in school matters now, and can we blame them for resisting when they can’t see a direct line between the thing you’re teaching and that elusive “good job” goal down the line?  We need to give them something better, and I’m not sure what.

Or maybe I should say I’m not sure how.  I do think I have a sense of the what. I think we all have a sense of the what.  We teach this stuff because we find it intrinsically fulfilling, after all.   I talked about this briefly in the gaming post the other day, and I also talked around this concept today over at ⌘f — there is motivation to be found in research and learning.  Those processes are compelling and even fun.  But I don’t feel like I get there very often in my interactions with students - they may get there by themselves later because of something we did, but that’s not quite the same thing.

Freire and Shor argue that part of the process of finding this here and now motivation is not trying to do it alone.  In other words, by watching and listening to students and seeing what they are really doing, what they are really interested in, and what they are really motivated by, you can co-create a learning experience that will be compelling and motivating to all of the learners in the room - students and teachers alike.  I think there’s something in that for library intructors.

This means creating environments where students feel comfortable enough to act authentically and to show their true motivations.  The one-shot library session?  Probably not.  Maybe in the hands of a better or a different teacher than I am it could be, but I’ve never mastered the art of immediate (within 50 minutes anyway) relationship-building that would require.  But as librarians, we’re not limited to the classroom - we also have our libraries. And out in the library, I think, we might get some of the answers we need, if we’re willing to listen.

Last week, Female Science Professor wrote a lovely pair of posts about scholars and scholarship, what it feels like when your work has an impact on someone and what it feels like to meet the people who have influenced you in that particular undefinable way where it’s hard to even express what they’ve meant to you. I shared one, saved the other and generally felt very good about being a small part of this world where rock star crushes on ideas and the people who share them are understood and embraced.

Way to ruin everything, Inside Higher Ed.

Okay, not really. But seriously, it’s a lot harder to feel like a rock star because someone has read and used your work if, as Malcolm Wright and J. Scott Armstrong suggest, they probably didn’t read it and if they did, they probably read it wrong.

That might be a little bit strong, but not by much. So what does it mean when a published, peer-reviewed article in a real life journal kicks off its final, concluding paragraph with this sentence - Authors should read the papers they cite.

!

This isn’t a library tutorial aimed at fifth-graders writing their first research paper, after all. This is a paper talking about what professional scholars, people responsible for the continued development of knowledge in disciplines, should do. It can’t mean anything good. Here’s the original article:

Article at Interfaces - requires subscription

Article at Dr. Anderson’s faculty page - does NOT require a subscription - (opens in PDF)

Nutshell - Dr. Anderson wrote one of the more impact-heavy articles in his discipline, and the only article that analyzes and explains how to correct for non-response bias in mail surveys (that’s bias caused by people who do not respond at all to the survey). By analyzing 1. how often research based on mail surveys includes a citation to this article, and 2. how often the later researchers seem to interpret and apply the original article correctly the authors conclude that many, many researchers are not reading all of the relevant literature. More disturbingly, many, many researchers aren’t even reading all of the articles they themselves cite.

Now, on one level this isn’t a shocker - anyone who has read moderately deeply in any body of literature has probably looked at at least one bloated literature review and said “hey - this person probably didn’t really read all of these books and articles.” This article suggests that it’s more complex than just lit-review padding, that scholarly authors also mis-cite and mis-use the resources they use to support the methods they use and the conclusions they draw.

Working on the assumption that if your research uses a mail survey, you should at least be considering the possibility of nonresponse bias, they found that:

…far less than one in a thousand mail surveys consider evidence-based findings related to nonresponse bias. This has occurred even though the paper was published in 1977 and has been available in full text on the Internet for many years.

Working on the further assumption that someone who makes a claim about nonresponse bias, and who reads, understands and cites an article that outlines a particular method for correcting nonresponse bias to support that claim, will follow the method outlined in the article they cited, the authors conclude that many authors are either not reading or are not understanding the articles they cite:

The net result is that whereas evidence-based procedures for dealing with nonresponse bias have been available since 1977, they are properly applied only about once every 50 times that they are mentioned, and they are mentioned in only about one out of every 80 academic mail surveys.

Most of the research that seriously digs into how well researchers use the sources they cite has come out of the sciences, particularly the medical sciences. This is one of the first articles I’ve seen dealing with the social sciences, and I think it’s worth reading more closely because this very rough and brief summary doesn’t really do justice to the issues it raises. But right now, I want to turn to the authors’ conclusions because I think they get at some of the things we’ve been talking about around here about how new technologies and the read/write web might have an impact on scholarship.

The first two outline author responsibilities:

• First - read the sources you cite. I think we can take that as a given - a bare-minimum practice not a best practice.
• Secondly, “authors should use the verification of citations procedure.” Here they’re calling for authors to contact all of the researchers whose work they want to cite to make sure that they’re citing it correctly. I’m going to come back to this one.

The second two put some of the burden on the journals:

• Journals should require authors to attest that they have in fact read the work they cite and that they have performed due diligence to make sure their citations are correct. That seems a sad, largely symbolic, but not unreasonable precaution.
• Finally, journals should provide easily accessible webspaces for other people to post additional work and additional research that is relevant to research that has been published in the journal. Going to come back to this one too because I think it’s actually related to the one above.

Basically - both of these recommendations suggest that more communication and more transparency would be more better for knowledge creation. And what is the read/write web about if not communication and transparency, networking and openness?

Some of the commenters on the IHE article expressed, shall we say, polite skepticism that an author should be obligated to contact every person they cite before citing them. These concerns were also raised by one of the formal comment pieces attached to the Interfaces article. And I have to say I agree with these concerns for a few reasons. Anderson made the claim more than once that he does this as an author, with good results, and that the process is not too onerous. But that doesn’t really address the question of how onerous it would be for a prolific or influential author to have to field all of those requests.

And I’ll also admit to having some author is dead reactions to this. What if I contact Author A and say I’m planning to use your work in this way and they say “well I didn’t intend it to be used in that way so you shouldn’t.” Does that really mean I shouldn’t? Really? It’s hard to see this kind of thing not devolving quickly into something that actually hinders the development of new knowledge because it hinders new researchers’ ability to push at and find new connections in work that has come before.

But not to throw everything out with this bathwater - the idea that more and better and faster communication between scholars (more and better and faster than can be provided within journals and the citation-as-communication tradition) makes better scholarly conversations and better scholarship - that’s something I think we need to hold on to. Anderson points out how talking to the researcher who really knows the area described in the thing you’re citing can point you to other, less cited but more useful resources - how they can expand your knowledge of the field you’re talking about:

We checked with Franke to verify that we cited his work correctly. He directed us to a broader literature, and noted that Franke (1980) provided a longer and more technically sophisticated criticism; this later paper has been cited in the ISI Citation Index just nine times as of August 2006.

This is an area where the transparency, speed and networking aspects of the emerging web might have a real impact on the quality of scholarship even if there are no material changes in the practice of producing journal articles. I might not be sure about the idea of making this communication a part of citation verification but it should be a part of knowledge creation. And it’s tied as well with the final recommendation - that journals should provide webspaces for some, not all but some, of this communication to happen.

The types of conversations between similarly interested scholars that Anderson is describing is nothing new - the emerging web offers some opportunities for those conversations to move off the backchannel. Or maybe it’s the idea that it’s still a backchannel, but the back channel being visible is interesting. Whether the journal has its own backchannel for errors, additions, omissions and new ideas to be posted, or whether that backchannel exists on blogs, in online knoweldge communities, or networking spaces doesn’t matter so much as it can exist. We certainly have the technology.

And the journal Interfaces itself I think provides a suggestion as to why this kind of addtional discourse and conversation is valuable. You may have noticed that what looks like a fifteen page article is really an eight page article with six pages of response pieces, followed by an authors’ response. The responses challenge parts of the original article, and enrich other parts with additional information and examples. They illustrate the collaborative nature of knowledge production in the disciplines in a way that citations alone cannot. I couldn’t find anything on the journal’s website about this practice - if it’s a regular thing, how responses are solicited, or more. These responses are a spot of openness in a fairly closed publication.

And that as well points to the last point to make here because this is far too long already - I don’t think we have to change everything to fix the problems raised here - and I don’t think if we did change everything it would fix all of the problems raised here. There’s that scene in Bull Durham where Eppie Calvin gets his guitar taken away because he won’t get the lyrics right. And that’s the connection between FemaleScienceProfessor and Anderson and Wight — who can feel like a rock star if they’re singing your songs but getting them wrong?

There will always be Eppie Calvins out there inside and outside of academia -for them, women are wooly because of the stress. But injecting just some openness, making some communication visible - won’t stop Eppie Calvin, but might keep the next person from replicating his mistakes. And that’s a good thing.

So I wrote something today, but I didn’t write it here. I wrote it over here. And even better, Caleb and Rachel wrote stuff over there too. Stuff that I’m even more excited about than what I wrote.

Go! Read!

(cross-posted at Notes and Links)

2008 ALA Annual Conference
Rachel Bridgewater, Reed College
Anne-Marie Deitering, OSU Libraries
Karen Munro, University of Oregon

Links to our examples, and many more resources to browse can be found at our Library a la Carte page: Library Instruction 2.0

Web pages, CMS tools, LMS tools

LibGuides (SpringShare)

Library a la Carte (Oregon State University)

Haiku (web-based LMS, free service is limited)

NetVibes

PageFlakes

Viviti (still in private beta)

Widgetize-able tools and applications

Sprout - use Sprout to create widgets out of RSS feeds and more

del.icio.us

• Karen’s del.icio.us links
• Anne-Marie’s del.icio.us links
• Rachel’s del.icio.us links

CiteULike

MeeboMe

• Karen’s post comparing chat widgets: Web-based chat widget face-off

LibraryThing

SlideShare

• SlideShare tutorial on adding audio to slideshare

YouTube’s Embeddable Custom Player (you must be signed into YouTube)

VodPod (to include videos from providers other than YouTube)

Resources for Keeping Up

Google Reader

• Official Google Reader Blog

Wikipedia’s page on RSS aggregators - lots to choose from!

Infodoodads

Information Aesthetics

Jane’s E-Learning Pick of the Day

Read/Write Web

Blackboard blogs

I have definitely hit that “what am I forgetting before ALA” mode where it is not a matter of if I forget anything, but rather how important the thing I forget will turn out to be. I am deep in the throes of preparing to present this pre-conference workshop with these awesome people while at the same time I try to make sure all loose ends are tied up here before I go.

So! I blog! Because I really want to just say a few words about one of the presentations I saw at LOEX of the West. Late, I know, but it’s sparked a really fascinating email conversation between several of my colleagues here at OSU and I want to write a few things about it before I forget them.

So the presentation was this one - A Portal to Student Learning, in which Nicholas Schiller of WSU-Vancouver argued that perhaps video games and gaming are not interesting to instruction librarians because we can make games that are more fun and engaging than traditional instruction sessions.  Instead, they should be interesting to us because the people who design games put their considerable skills, talent, time and resources to work to, essentially, teach a group of players how a system works, how to navigate that system and how to get what they need to solve problems and achieve goals within that system.

Apologies to Nicholas for that very brief and rough paraphrase but even brief and rough — it sounds a lot like what research is?

One of the overarching points here, and one that came up as well when Rachel and I talked about Alternate Reality Games at the last Online NW, is that good games and good game environments are really, really hard to do.  There are people who spend all of their professional time, every day, creating these games and environments and sometimes even they fail.  Librarians have other jobs being librarians and do we really have time to create the types of games that will be engaging, that will contain within them whatever it is that makes success within the game environment an end in itself to players?

One of my co-workers pointed out that it is hard to design a game to teach people how to research effectively because everyone’s research process is different, everyone’s goals are different, and people’s goals shift and change even as they engage in their own research process.  And that’s certainly true -if we expect their motivation to play and do well at a game to be external to the game - I want to do well at a game because of what it will get me outside the game — then I think that’s probably not the way to get engaged in a game.  But a game about information literacy skills that has within it enough motivation that people want to succeed at it for the game’s sake alone - I might be a little too cynical to be able to picture that.

But what I liked so much about Nicholas’ presentation was that he showed a way to think about this that doesn’t require us to design games that meet our users’ idiosyncratic and deeply individual needs.  It doesn’t require us to have the technical skills to develop games that will be engaging and effective.  It requires us to understand that when people are playing games they are learning, about systems and environments.  In effect, the game gives them what they need to teach themselves the rules of the game, including where those rules can be bent or broken.

And I think that’s a really exciting way to think about our interfaces, our tools and our systems.  Because they have rules too.  The ways that game designers use feedback, scaffolding, and other techniques to help the user teach themselves by doing — that seems to have direct applicability to how we can think about our systems and the tools that give our users access to those systems.  This might be a deeper and better way of thinking about visual search than I’ve been doing here for a while now.  I suspect that it is.

Because where I’m not cynical at all, I’m probably downright Pollyanna-ish, is in the idea that research brings with it its own rewards.  One reason I’m so resistant to the idea that we need to staple another motivation (winning a game) on top of learning research skills is that research itself is fun, adventurous, creative, surprising — and even competitive.  Haven’t we all felt like we won, somehow, when we made that breakthrough, found that thing that showed us where our project was going to go so that all of a sudden we could see it all the way through to the end?

I’m not sure I can describe it better than Caleb did here - talking about games and research and the fun.  I think he’s right - that libraries are very well suited to that kind of learning.  But our systems don’t always keep up.  So thanks Nick, for suggesting some ways that maybe they can.

Over at Peer-to-Peer, the Nature blog for peer reviewers, Maxine Clark points out that it is Nature Precedings’ first birthday (or one-year anniversary) today. Her post does a great job of pointing out how the web-based peer review part of this experiment has developed. I especially like her example of constructive review “Nature Precedings style.”

She also points to these charts by Santosh Patnaik, using the openness of Nature Precedings to mine and analyze some data about how use of the site has developed over time. I found it interesting that while manuscript submissions have skyrocketed, the site hasn’t really developed as a hot place to put up conference papers or presentations. I would imagine that to those immersed in these disciplines, the reasons for this are clear — concerns about putting stuff up too early? Concerns about hurting future publication chances by making stuff freely available? I don’t know - but I do think it’s interesting to think about.

Dr. Patnaik estimates that the 500th paper will go up some time in the next few weeks - not bad for a year’s (collective, collaborative) effort.

This is the paper Kate and I submitted along with our LOTW presentation, rendered into this gorgeous tagcloud by Wordle, a new tagcloud generator I saw today on Information Aesthetics.  I love tagclouds anyway - but this one lets you play with layout, fonts and colors in a way I’ve never seen before.  You can upload or copy and paste any text, or hook it into a del.icio.us account as a new way to see those tags.  So much fun.

Kate and I are still buzzing from the great conversation we had with the people who came to our session at LOEX of the West. It’s always an amazing and kind of surreal experience when you find out that other people are excited by the same ideas you are.

And it seems that other people really are. Almost the second we stopped talking, we started finding other people who were. All over the web.

At ACRLog, Barbara Fister brings up the issue of promotion and tenure, and how many committees find it difficult to evaluate the significance of publications that don’t fit into the traditional scholarly formats — particularly when they are trying to evaluate the impact of scholars from other disciplines. These ideas are strongly connected to the ideas about distributing professional rewards, and we really just got started talking about the question of expertise, and evaluating work outside your discipline at the end there - good to see and think more about it.

Dorothea Salo talks about the differences between informal writing on the participatory web (like blog posts) and scholarly journal writing. She brings up one benefit to scholarly journals that we only hinted at - the way that the lengthy give and take between author and editor in the traditional publication process can make an individual article better. Not bring it up to some objective standard of quality, but make it better than it was. She also talks about something we did spend a lot of time talking about - the archive of knowledge, or the scholarly record. But she goes a lot further than we did talking about the role academic libraries play in that process.

Then today, I saw Tenured Radical’s discussion of the Social Science Research Network. She’s asking why historians aren’t participating in this project, and looking at some of the implications of that lack of participation. The SSRN is a digital archive that has as its goal the rapid dissemination of research in the social sciences. It includes an abstract database (of scholarly working papers and forthcoming papers) and an e-library of downloadable papers. These resources are available to registered members for free; there are also entry points into some proprietary database holdings, for a fee.

Tenured Radical highlights one of the reasons we think it’s so important that we are all having these conversations - not to replace traditional forms of publication, but to make them accessible. Not to encourage scholars to write for the public instead of for each other, but to leverage technological change in ways that can keep that scholarly discourse available to those who want to find it –

An insistence that the only good work has been heavily vetted through our current refereeing practices may be a mistake, much as soliciting the criticisms of others does contribute to producing good work (although it doesn’t always, I’m afraid, as cases where flawed research has slipped through to publication or to a prize demonstrates.) In its current form, it may be a fetish that is doing us more harm than good, and may be something that our professional associations need to review to take advantage of an atmosphere of intellectual vigor offered by electronic and other forms of mass publication.

Peer Review 2.0: Tomorrow’s Scholarship for Today’s Students

LOEX of the West, Las Vegas

Anne-Marie Deitering & Kate Gronemyer

WEB 2.0 BACKGROUND

Five Web 2.0 themes — from the ACRL Instruction Section’s Current Issues Discussion Forum, Research Instruction in a Web 2.0 World (Annual, 2006).

DANAH BOYD EXAMPLES

{Edit: These didn’t make it into the presentation, but they are examples of some discussions on the web over the last year that started our thinking on this topic.}

danah boyd - open-access is the future: boycott locked-down academic journals

• good response from Ann Galloway - Boycott? I think not. (Purse Lip Square Jaw)
• response thread on AIR-L (Association of Internet Researchers)

danah boyd - Viewing American Class Divisions through Facebook and MySpace

• Response from Andrew Keating - ‘Digital Divide’ meets Web 2.0? Ridiculous and Poorly Written Article
• Short discussion at TeleRead
• BBC article - “Social Sites Reveal Class Divide”

danah boyd - editing a special issue of JCMC

NORMAL SCIENCE & INNOVATIONS

• Thomas Kuhn, The Structure of Scientific Revolutions
• The Paradigm Concept — Stanford Encyclopedia of Philosophy
• Coping with Peer Rejection - Nature

PEER REVIEW, QUALITY CONTOL & FRAUD

• Fiona Godlee (et al), “Effect on the Quality of Peer Review of Blinding Reviewers and Asking them to Sign their Reports,” JAMA, 280 (1998)
• Scientific Misconduct: Bell Labs Fires Physicist Found Guilty of Forging Data, Science (4 October 2002)

DISTRIBUTING PROFESSIONAL REWARDS

• 2006 Report of the MLA Task Force on Evaluation Scholarship for Tenure and Promotion (opens in PDF)
• This is Scholarship (video). Catherine C. Braun & Kenneth L. Gilbert, KAIROS, volume 12:3
• Piled Higher and Deeper (webcomic)

WHAT IF WE IGNORE NEW MODELS?

• Michael Gorman, Jabberwiki: The Educational Response, Part II

NEW MODELS

Current Anthropology

Expressive Processing: An Experiment in Blog-Based Peer Review - Noah Waldrip Fruin on Grand Text Auto

Cognitive Daily blog

ScienceBlogs - The World’s Largest Conversation about Science

BPR3: Bloggers for Peer-Reviewed Research Reporting — Icons

CiteULike

UsefulChem Wiki

Radiology Wiki

Open Notebook Science Using Blogs and Wikis (Jean-Claude Bradley, at Nature Precedings)

Rrresearch Blog

“The Academic Manuscript” — Wicked Anomie: Sociology Run Amok

Welcome to Nature Precedings

EDITED TO ADD:

Barbara Fister points to this article in the Chronicle:  Certifying Online Research (Gary Olson), about the challenges of evaluating online publications.  See also Barbara’s post at ACRLog: Peer (to Peer) Review.