tutorials a la carte

So the thing about DIY tutorials is that even though the technology bar required to build them is pretty low, there are some back-end logistical issues that have to be navigated.  When I decided I wanted to try these with some of my distance learning classes, I had to figure out how to create a locally hosted blog, and we ended up doing it in a way that hadn’t been tried before (using Drupal) so we had to go through some rounds of figuring out settings and styling.

Don’t get me wrong, this wasn’t a bad thing to go through for me, or for the library.  And when colleagues wanted a similar tool, we were that much further down the road and had some good reasons to try a different tool.  Still, when the goal is “I want to get a tutorial up” these kinds of barriers aren’t necessarily what you want to go through.

I realize that a local option wasn’t the only option – if all I cared about was putting up a blog for tutorials quick and fast there are obviously tons of options out there.  But that wasn’t the only problem I wanted to solve.  I didn’t just want to make tutorials myself, I wanted to do something that would make it easy for all of us to create tutorials and other instructional aids.

The issue of how to share the burden of creating instructional materials has been around for a long time.  Learning objects repositories like MERLOT offer lots of options for teachers – the ability to borrow online instructional modules, and the ability to comment on and rate those tools.  Shared tutorial projects like ANTS let librarians share tutorials, collaborate on prioritizing and creating tutorials, and also provide a social space to talk about tutorial-related issues.  Here, locally, we have  CLIP (Collaborative Library Instruction Project) which takes a good step in the right direction – making source files available and shareable, letting librarians share some but not all of the work of tutorial-building.

CLIP is just getting off the ground, so I don’t know what will happen there.  But here’s the thing.  Whenever I have talked to anyone involved with projects like these, or heard them speaking about the larger issues, they say the same thing – getting participation from educators is really, really hard.  They spend a lot of time gathering the content for the repositories themselves, they have a core group of committed people sharing, but getting the concept of sharing — of taking the time to participate in these types of projects and communities – to be part of the normal workflow for teachers is super difficult.

My own experience with the Library Instruction Wiki (now offline due to basement server room flood) left me feeling pessimistic about that project’s stop reinventing the wheel tagline.  I have come to believe some things – some things that got me thinking that instead of sharing the final product of tutorials, the way to go is to figure out ways groups of teaching librarians can share a process:

1. Teachers like reinventing the wheel.

This is something Jean Caspers said to me once, and its stuck with me because I think it’s essentially true.  Not that teachers don’t want to borrow and adapt and take advantage of other people’s cool ideas and good work, but to really feel comfortable going into a classroom and teaching a group of students something, a lot of us need to feel like we’ve made the stuff we’re teaching ours.  And the only way to do that is to adapt, and reshape, and refine.  So we don’t want to just point to other people’s handouts and tutorials (sometimes we do, but go with it).  We want to make them ours.

And an important part of the cognitive process that a lot of us go through preparing to teach is preparing the materials.  I know that when I create a course page for a class, I’m thinking about how I am going to present the material in class, and about how I am going to transition from one topic to the next.  And when I make a tutorial for an online class, I want to tailor it to their assignment, and the process of putting it together helps me clarify what they need to know/do for that assignment.

So, it’s not a bad thing that teachers want to reinvent the wheel, not at all.  And I think it’s a need teachers have that should be considered by anyone trying to help them work collaboratively.

2. The hard part of developing tutorials isn’t technical, it’s in the content.  And that’s the hardest piece to share.

One piece of this is practical – try teaching anyone how to do anything research-related on your campus and see how long it takes before there is some local quirk that you need or want to explain.  It doesn’t take long.  There’s a reason why the tutorials we’re most likely to share are on topics like plagiarism or citations — things where we are all working on basically the same standards and rules, defined outside of our local institutions.

One piece of this is related to the above point.  We want the content to be tailored to the students’ needs – we have strong opinions about how to do research and about how to teach it and about what students need to know about it.  I don’t know, I haven’t talked to many people who say things like “I would totally borrow X’s tutorial, but it’s built in Camtasia and I would rather it be in Captivate. The people I talk to are more likely to forgo borrowing a tutorial because they don’t like something about it — they’d like to change the way it explains a concept, or to add just one more piece.

3.  There is one sure-fire way to make the technical part hard – that is to tell everyone they have to use the same tool at every stage of the process.

Now this may seem to undercut the whole process-sharing thing I mentioned above but bear with me – I’m really talking about all of the component parts of the final product.  If you tell everyone that they have to use the same tool to build webpages, that’s going to leave you with a few people upset yes.  You’ll also have a lot of people that don’t really care.

But if you tell them that they have to use the same thing to take screenshots, the same thing to do screencasts, the same thing to create word clouds, or to display bookmarks, or to push useful links — then you’re going to start getting the kind of resistance that makes people decide not to create the thing at all.  This is especially true if people have been left on their own to figure out their own best way to do those things before.

So, to get to the point already…

So, Hannah and I were working on redoing our big tutorial – and one of the problems we wanted to address was the bottleneck that occurs when only a few people can edit or make changes to a tool.  We also wanted to de-Blackboard our beginning composition assignments, and make them more lightweight and dynamic.  Thinking to kill two birds with one stone, we decided to look at content-management-izing our tutorial building process.

The brain trust behind Library a la Carte (an open-source, lightweight CMS for building course pages and subject guides) works down the hall from us, so we had a place to go with this problem.  We were initially open to a variety of approaches (including Drupal and WordPress) for building these tutorials, but we ended up deciding that extending LALC to include a tutorial-building function made the most sense.

We launched it barely-alpha, with the fall term beginning composition students.  We found a bunch of things that wanted fixing, but even in this sub-optimal situation, there was an awful lot of it that worked well.   Now it is in barely-beta, and with about 40 sections of beginning composition using it, the reports we have had about it not working for the students have been in the single digits.

This is what the tutorials look like:

(Here’s a link if you want to look at the whole tutorial.  This one is cool because it has modules that feature: images, cartoons, videos and text included.  You have to log in to see the quizzes, but not to see the rest)

There’s a few things I really like about it -

First, it allows our teaching librarians to share at a pretty granular (modular) level – I can borrow Hannah’s catalog-using screencast, and put it into one tutorial that is really about how distance students can have books delivered to them by mail, and also into another tutorial that is really about the serendipitous process of browsing for books on the shelves.

And even at that granular level, it lets us borrow-and-then-tweak — keeping things right in most teachers’ comfort zone.

Using an incredibly scientific data gathering method (n=1; n=me) I have determined that instruction librarians just may be more likely to borrow from each other and to remember to share with each other in this format.

Secondly, it solves the problem of where everything is going to live.  Because it is integrated with our subject and class guides, the system lets us create tutorials and then automatically puts them on the website, with no pesky decisionmaking steps beyond what to call the thing.  Which admittedly, can stymie me for a while, but…

Using the same data collection method (this time n=me and Hannah) I have determined that with this tool available, we are more likely to include tutorials and learning modules in the things we do for our classes, whether we meet with them face-to-face or not.

It also lets the librarian pull in content from elsewhere, so they can use any method they are comfortable with to create the content initially.  If I want to store my photos on flickr, and edit with Picnik, that works fine. But so does uploading the photos that I edited with Photoshop.  If I want to embed a video I edited with iMovie, I do the same thing I would do to drop in a screencast video I created with Camtasia.

Editing a video module:

It also lets us easily use dynamic content (which sometimes breaks the styling but, live and learn) so if I want to embed a delicious linkroll to recommend links to three different classes of students, I can, and then of course I can update all 3 class pages at the same time when I update my Delicious.  Or if I want to do the same thing by embedding a Twitter feed, no problem.

Again, because this is integrated with our class pages system, we’ve already made a lot of decisions about where to share some of our resources.  But even though we have a flickr archive, lots of us don’t use that for building the modules.  And even though we have a YouTube channel, the videos there can be created in lots of ways.

I have no idea at this point if anyone else is going to start using this, or if it will mostly be used to update our main tutorial and in beginning composition.  But I do like the concept of process-sharing, and I think this might be a way that idea makes sense.

what I’ve been doing instead of blogging

I have no idea why I am feeling compelled to put this up here, except that it is what I have been writing instead of blogging.  Not that I’ve been spending the actual minutes writing it, because I limited the writing time carefully, but it is where the mental energy that would usually produce a blog post has been going for the last couple of weeks.  The hours when I’d usually be thinking about something I have read and getting worked up enough to write about it have gone to thinking about how to finish this teaching philosophy statement.

It starts here –

Learning can be hard, it can be exhilarating, it can be scary, and it can be transformative changing the way the learner understands the world.  The most meaningful experiences I have had in my education, as both a teacher and a student, have happened when a new thought, a new idea or a new understanding has that transformative effect.

…human learning presupposes a specific social nature and a process by which
children grow into the intellectual life of those around them.
(Lev Vygotsky, Mind and Society)

Teaching college-level research inherently means teaching students that research and learning are social processes.  Scholars do their research and communicate their findings according to practices and conventions defined by disciplinary communities and practice networks.  Teaching information literacy works best when the learning students do is grounded in these larger conversations.  This means the most effective teaching happens when I am working closely with faculty, to connect the research activities students do to the broader learning going on in the class.  For example, in Oregon State University’s beginning composition classes students engage in information literacy activities throughout the term, that connect information literacy skills to every stage of the writing process.

I believe that the best learning is a personal act of meaning-making where new information is integrated with existing mental models to create new knowledge.  Within this context, information literacy is not an end in itself.  Instead, it is the thing that gives students the cognitive capacity make that meaning for themselves.  As such, I believe that the best way I can teach students about information literacy is to introduce them to the necessary concepts, skills and ideas in an environment and context where they can immediately apply them within a larger process of learning and meaning-making.

While the meaning we make out of new ideas and information is deeply personal, the learning that supports that meaning-making is still social and collaborative.  My own ideas about teaching and learning have been strongly influenced by constructivist philosopher Lev Vygotsky, who frequently focuses on this connection. Vygotsky’s work emphasizes the interplay between teacher and student.  His description of learning as that which we can do with help, reflects the teacher’s expertise and body of experience without devaluing the knowledge, understanding and body of experiences the student brings to the process.  This is an especially useful and important way to think about learning for me, as an academic librarian.  My teaching is most effective when my knowledge and expertise about the research process combines with the student’s own expertise and experience with their topic area.

Liberating education consists in acts of cognition, not transferals of information.
(Paolo Freire, Pedagogy of the Oppressed)

When that connection happens, between teacher and student, we are both engaged in the learning process.  I believe that one of my most important responsibilities as a teacher is creating an environment where that kind of learning can happen.

As a teaching librarian, I think one of the most important things I can do is to model an appreciative, curious, recursive research process.  Scholars do not know all of the answers in advance when they do their research; the research projects that result in deep learning are frequently those where no easy answer exists to find.  It is in the process of constructing an answer out of the information that is out there, a process that is often messy and chaotic, that that deep learning happens.  I have come to believe that avoiding that messiness and chaos when I am teaching does my students a disservice.  Instead, using my time with the students to show them how to navigate the research process, to troubleshoot when problems arise, and to keep their minds open enough to recognize opportunities when they find them is a more effective, though stressful, way for me to teach.

This includes strategies as simple as using new, untested topics to demonstrate research strategies, trying keywords without knowing what they will find.  It includes developing activities that focus on broad exploration, and teaching students how to browse a topic before searching for specific sources.  For example, in OSU’s advanced composition class the students and I spend half of our class session browsing obviously biased online news and commentary sites before switching our focus to published sources.

This also includes an emphasis in the classroom on active learning activities that allow students to connect new skills and concepts introduced in class to their own topics of inquiry.   It is when the idea or skill I have just demonstrated works to further their knowledge of their topic that they learn it best.  Building in time for them to make and reflect on those connections is crucial.

One of the biggest challenges I face as a teaching librarian is that most of the direct contact I have with students happens in the context of a single-shot, guest lecture appearance in someone else’s class.  It can be very difficult to push beyond the idea of teaching as an act of transferal in that context, because students frequently make the crucial connections between your teaching and their learning after they leave that guest-lecture session.  After several years in this environment, I still feel that I have not mastered this type of teaching.

The big deal is, that it’s part of my job to make sure that you don’t grow up stupid
…it’s bad for the world.
(Tami Taylor, Friday Night Lights)

I will continue to work on becoming effective within that one-shot context, and developing new ways to teach beyond that context because I believe that what I teach, what we teach as academic librarians, is important.  It’s good for the world.   Information literate learners can take control of their own learning, and continue learning throughout their lives.

Students who understand what evidence is, and how other people use it to further particular agendas are powerful.  Students who can find, understand, evaluate and use evidence themselves are even more powerful.  When people graduate from college without those skills and without mastering those concepts, it’s bad for the world.  As a teaching librarian I get to focus my time and energy on helping students develop their power, and making the world a better place.

and ends here.

So there it is.  For all I like reflecting on stuff, I find this kind of writing excruciatingly mentally difficult, and have to strictly limit how much time I give it.

In the end, the only way I could was to focus on a few of the things that have really sparked me to think about teaching in ways that have stuck with me – even the quotation  from Mrs. Coach on Friday Night Lights (which I agonized about because the word “stupid” seemed too unlike me to put in a teaching philosophy statement because I would never say that. But Tami Taylor would so I couldn’t change it either, and there’s no room in said statement to explain the context where it is not about that, but actually is exactly about what I wanted it to be about.

And of course it’s not finished.  That’s why this kind of writing is so difficult for me.  I never feel happy with it, and it never feels done.  I suspect that is the main reason I feel compelled to post it – because sharing it takes it out of my head.

on tag clouds and teaching

Inspired in part by a conversation from the Information Literacy Summit last week, and in part by this post from the awesome David Silver, we made tag clouds in my class this morning.  U-Engage is a first-year experience class, providing an introduction to what a research university is in general and to OSU in particular.  There are 100 students in the section I teach (actually, there are somewhere around 90) and the room is a classic, old-school lecture hall.

The size and the room make the course difficult in terms of planning engaging activities for the lecture time (the assignments and recitations are a different story – I think they’ve had lots of interactive activities there).  This has been compounded by the fact that most of the course content in the middle of the term has been delivered by guest lecturers.  I’m co-teaching the class and both of us have been feeling that we really wanted the last few weeks to be interactive and engaging because we have been kind of on the sidelines for so long.

My co-teacher had a brainstorm for last week’s activity, but we still didn’t have anything ready for this week – the final week before their group project presentations.  The textbook wasn’t inspiring.  There is some great information about the value of reflection (which has been a theme throughout the course), and on goal-setting but that stuff didn’t lend itself well to the kind of activity we had in mind.  Reflection and goal-setting kind of needs to be personal to be meaningful the way these things are presented in the text, and we wanted something social and collaborative.

So at the IL Summit my husband Shaun talked about how he has an assignment requiring students to pull keywords out of the readings they do.  It was fascinating to hear about that because his students’ difficulties finding keywords in many ways mirrors the difficulties we see students having in the library when it comes to choosing keywords.  But because he is working from a known thing “what are the keywords that the author uses that capture the key ideas in THIS TEXT” he has an advantage we don’t – it seems like that would work way better as a first step than what we frequently have to do – “try and predict what words authors will use in this discourse that is entirely or almost entirely unfamiliar to you.”

In his presentation, Shaun mentioned that he is still working on ways to make the larger class discussion about the keywords work better, and I thought about David’s post above.  But it wasn’t until I was working at the reference desk on Saturday that it all came together in my head – collaboration, social, keywords and tagclouds.

So here’s what we did – I demonstrated Wordle because, like David, I expected that most of the class wouldn’t know what a tag was and by extension, wouldn’t know what a tag cloud was.  I wanted to get two ideas across:  the idea that the tags needed to be single words or at most two-word phrases, and the idea that the tags would display larger if they were used more often.

(I also thought that Wordle would be of interest to some of the class, that it might come in useful for its own sake in their final projects, and that some of them might find it a useful study tool).

Then we asked them to come up with 5 keywords that described their first term/ transition to college and write a reflective paragraph about that.  Once that was done, we go to do the social part.  We had them get in groups of 4-5 and create a tag cloud out of all of their keywords.   This is why it was important to me to make sure the concept of “more use = bigger text” was clear.  To do that right, they would need to talk about their keywords, what they had in common, and if they maybe chose slightly different words to get at the same concept.  These small groups also had to come to consensus on five “group” keywords.

Looking at the group keywords after the fact, I was surprised that a lot of the groups seemed to put together their five not by choosing the five most “popular” keywords (the ones used by most people) but instead that they tried to choose one of each group members’ keywords.  I think this makes sense when the assignment is as personal as this – if they were trying to choose keywords that would reflect the meaning of another person’s text, I would expect this might be different.  In addition, some groups used keywords for their “group” keywords that didn’t appear on any of the individual lists.

ETA – we took pictures -

uengage1

Then, each group had to send one member to the chalkboard to come up with a group tagcloud, using the chalk that was available.  On one level, this would ideally take a while as the groups compared their five words and decided how best to represent them.  In practice, it didn’t work out that way, but I think that the way it did work out was better.  As it turns out, there was very little overlap in terms – in fact, there was less overlap in the group terms than there was in the individual terms in one sense.  So while there were a few terms that should probably have been bigger because lots of groups had them, there weren’t many like that.

ETA – more pictures -

uengage31

Secondly, even though there were 14 or 15 people up at the board making the tagcloud (remember – there were about 75 people in class, and the small groups were only 4-5) that meant there were still WAY more people than that still sitting in the lecture hall seats.  This means that every minute the representatives spent figuring out the “right” way to make a tagcloud was a minute the vast majority of the class had to be dis-engaged.

So instead, people talked some and made some words bigger because they were repeated.  But they also made some words bigger because they were the words that the smaller group had felt were the most important.  And there’s a validity to that, and a meaning to that, that I don’t think our original plan had captured.

ETA – last photo –

uengage21

All in all, I was pretty happy with how the exercise turned out.   I think this kind of exercise could have been especially effective earlier in the term, before the students knew each other as well as they do now – to humanize the 100-student classroom environment.  And on that note, I think this kind of exercise would work really, really well in library instruction sessions as well.  Combined with Shaun’s idea above about pulling keywords from a text, or perhaps using keywords generated another way – it’s a safe, collaborative way to talk about the connections between ideas and the terms we use to describe those ideas.

And isn’t that the big picture philosophy behind keyword searching – I mean, isn’t that the fun part?

Finally, I have to say that we could have just had each group representative put their terms into wordle – but I don’t think that would have worked as well.  I think the physicality of the chalkboard and the actual social/cognitive act of having to do themselves what the computer does for us was important in making this engaging.  In this case, the sheer number of people involved and the limited time we had meant that some got more out of it than others – and the chalkboard session had more of a free-for-all feel to it than a Deep Thoughts feel.

This also seems like a great way to connect class work/ reading with library session work.  I think we all feel like the “teach how to drive the databases” one shot feels disconnected from the learning that goes on in class in a way we don’t like.  We talked after the Summit about building in some keyword exercises in our beginning composition classes, at the start of the term in the non-researched papers, and using those keyword exercises to get at the critical reading piece that Shaun talked about with his assignment.

I think that’s a great idea on its own, but I also think that then building from keywords there to keywords while exploring ideas and finding your own sources might draw a connection between the idea of critical reading, writing and research — all of the pieces of the course.   With the tagcloud exercise, it’s easy to think of ways to do that in the disciplines as well – a pre-library session assignment identifying keywords from a class text, a library session tag cloud of that text and another of the student keywords.  And so on and so on and so on, leading up again to student-generated tag clouds representing ideas about research and suggesting pathways they can use to research about ideas.

showing not telling

I’ve been working today on a presentation for next week – not a typical conference or workshop presentation, but a presentation for a class I’m teaching. I thought I was thinking of it more like a presentation than like teaching because the environment I’m teaching in here is so different than the environment I’m used to – 100 students, lecture-style classroom – very different than the 20 students/ hands-on workshop that I’m more familiar with.

Today, though, I’ve been realizing that I’m thinking of it more like a presentation as much because of the content I have to cover as anything else. AND – because I’m thinking in terms of ‘coverage’ – which is not only something I don’t usually have to do, but also a focus I usually find myself arguing against. So I had to think about it – why am I thinking about coverage now?  Why am I thinking about the stuff I want to introduce more than the stuff I want students to be able to do when they’re done?

On Monday, I need to talk about college student development theory.  Part of this is to supplement their textbook’s super-brief description of Chickering’s seven vectors of college student development, because all of the 700 students (our section + six others) are supposed to apply that model their midterm paper. Part of this is to introduce the concept of theories and models, what they means for academic discourse, and what it means to use theory critically. The following Monday, I need to talk about critical thinking. For fifteen minutes.

So, the thing is, there really isn’t anything concrete that I want them to be able to do – or, there’s nothing specific that I want them all to be able to do. I don’t really want to break down critical thinking, or what theory is, and have them take the first step in this class – which is the only way I can think about active learning in this case.  That might be a limitation on my part, but there it is.

Because a huge piece of these topics, a huge issue that I actually do want to communicate to the students, is that no one can make these students (or anyone) take theory seriously and apply it to practice.  No one can make them think critically – even if they have all of the skills and understandings necessary to do those things they can still choose not to do them and there is nothing that I, or anyone else, can do to change that.

Take this paper assignment.  Yes, we can require them to read about Chickering’s seven vectors, we can require them to understand them well enough to write about them.  We can even require them to write a paper about their own connection to that theory — but we can’t make them do it for real.  We can’t force them to reflect.  We can’t require them to be honest in their analysis and application.  They can make up every specific example from their own lives and there is truly nothing that we can do about it.  And, to be sure, doing so successfully would exhibit some high-level critical thinking and creativity skills, but it wouldn’t get at everything we want to get at here.

On at least one level, this is what the content itself is all about.  College student development theory is, in a very real way, describing the process by which students get to the point where they can do those things not just because they can, but because they want to.

So, what I’m thinking is that the issue isn’t how to figure out active learning exercises for teaching these topics.  On some level, the papers and projects in the class are the active learning experiences – and they are active learning experiences that give the students a lot of freedom to do these things in a way that makes sense for them.

No, what I’m thinking is that the best thing to do is to acknowledge up front that no one can make them use this stuff.  Correction – no one can make them meaningfully use this stuff.  Acknowledge that up front, and keep the focus of the presentation on why it’s useful to me.  Why I like it, how I use it, how other scholars use it and why it’s useful.  After all, if we’re going to ask them to be honest and reflective, it seems only fair to provide them with a little of that in my own presentation.

Oddly, I’m fairly comfortable with this focus, even though it means I’ll be talking for quite a while on Monday.  Of course, I have plenty of places where I can ask them questions and check in with them, but still it’ll be a lot of me-talking.  And I’m sure part of the class will never engage with what I say, no matter how many cool pictures and analogies I use to supplement.

All of this has me thinking as well about library instruction.  In a lot of cases, we do teach the stuff that we can (kind of) require people to do – we can require people to find articles using databases.  No matter how resistant they are, if they go through the steps and the exercises, they’ll learn something about how to do that.  It’s easy to come up with concrete, “what I want them to be able to do,” learning outcomes for that stuff.

But there are also these bigger picture questions we would like them to start thinking about, and start applying to their own lives, their own practice, and their own world view about how knowledge is communicated and valued.  And I realized, in those segments I do the same thing I’m doing here – I’ve stopped telling students that the quality of the information they’ll find in databases is inherently better, for example.  Or that the quality of peer-reviewed articles is inherently better.  Generally, I approach the “why you should use databases” piece as a demonstration, and from the perspective of “here some reasons why I use these tools or sources.  Here’s why I think they’re cool.”  It seems to work.  I’m certainly more comfortable with that approach, and they seem to be open to hearing the message in this way.

Which gets me rethinking some of these ideas about coverage.  Certainly, there are lots of situations where we value coverage over learning.  The whole idea that if I mention 10 topics in an hour, they’ll somehow learn about 10 topics in an hour is bad and has to go.  But maybe the flip side is also unbalanced – if we try too hard to make everything about active meaning-making for them, do we leave ourselves out of the equation too much?  If learning is social, and I think that it is, constructing meaning together is a crucial part of that.  We know that when it comes to group work, and to peer groups.   But maybe showing why and how we care, why and how we really use the stuff we want them to use – isn’t the same as telling so much as another way of constructing meaning together?

Teaching undergraduates about peer review – how and why, and did I mention how?

Lately I’ve noticed a number of different conversations I’ve been having coalescing around the question of evaluation – how can students evaluate the information they find. Some of the conversations have been versions of your normal standard “information on the web can be bad” and aren’t very interesting, but more of them have been about the much more interesting and much trickier question of — how do students evaluate scholarly information they find on the web when they are neither content experts (like their classroom teachers are) nor format/scholarly communication experts (like librarians are).

Which is why the title of this post jumped out and hit me over the head when I saw it today: Can you tell a good article from a bad based on the abstract and title alone?

(the post is a couple of months old and had quite a bit of discussion in the science blogs, but I haven’t seen much about it in library discussions)

So – what do you think? Can you? I sure can. And can’t. I mean, it depends, right? But when students are looking at something like this — that’s kind of what we’re asking them to do.

typical result list - ebsco

And the thing about the story linked above is that is also shows that the default we sometimes turn to – peer review – isn’t good enough. A lot of the comments on this post and on these related posts at P.Z. Myers and the Nature blogs focus on the suggestion that this paper is written from a creationist/ intelligent design perspective and the implications of this for peer review –

  • The potential that an author can choose/target politically friendly reviewers for a paper
  • The suggestion that this paper’s publications might allow an affirmative answer to the question “can you find one peer reviewed article supporting intelligent design” – and what that might mean for science.

The article was retracted by the journal, not because of its politics but because of plagiairism. Which is also something one would hope would be caught by the peer review process. It seems like it would be the least we should expect.

So on the one hand, you have the science blogs – you have someone reading the title and abstract for this article, seeing some red flags, using the dynamic web to point them out. This generates discussion, which spreads to other dynamic sites and eventually results in the article in question being pulled down. On the other hand, you have the peer reviewers, working in isolation, who didn’t seem to catch any of the red flags. On one level, it reads like a fairly straightforward Web 2.0 Makes Good story.

But on another level, what does this mean for students, especially undergraduate students? Here’s the sentence that raised the red flags for most of these scholars:

These data are presented with other novel proteomics evidence to disprove the endosymbiotic hypothesis of mitochondrial evolution that is replaced in this work by a more realistic alternative.

I can’t say this raises the same questions for me. “Novel… evidence” might be a little odd, and “a more realistic alternative” is an interesting turn of phrase. But the thing is, you have to know something about the “endosymbiotic hypothesis” to be able to contextualize, or criticize, the idea expressed here. How many students are going to have the content knowledge to do either of those things? And the other thing is – if this had become the one peer reviewed article supporting intelligent design, there’s a really good chance that even my beginning composition students would come across it.

I don’t have any really good answers for how to help students make sense of this – except I don’t think librarians and composition instructors can do this alone. And I don’t think we can make any decent stab at figuring out an answer to this question without engaging with the question of what the participatory web means for scholarship – and engaging with the related question of what the limitations of traditional peer review are as well.

And this is where the “wisdom of crowds” vs. “cult of the amateur” story that gets played out so much in the popular media really fails us. Because if this story shows anything, it shows that we still need experts to help us evaluate, contextualize and make sense of information. And at the same time it shows that trusting those experts blindly doesn’t work out so well. Adding the transparency of the participatory web to the opaque processes of traditional scholarly publication – I think part of the answer is in that grey area somewhere.

A long post at the Bench Marks blog examines the question of Why Web 2.0 is failing in Biology. It would make this too crazy long to engage with everything there today, but I do want to pull out a bit from the end. After talking about how life scientists aren’t reading or contributing content to blogs, he does look at the end at who is reading science blogs and what that might mean.

Two of the groups he pulls out are really relevant here I think — science journalists and non-scientists. If blogging is a good way to get scientific ideas out there to a more general public — people who aren’t reading the scholarly journals or going to the conferences — then they’re a way that that general public can get access to the kind of experts who can help them make sense of the research literature. More on this later, maybe.

Full disclosure – some of this thinking is to prepare for this presentation.