discovery and creation and… lies!

I’ve never really understood the whole pirate thing. Talk like a pirate day can come and go without my noticing, and despite the presence of Johnny Depp, I didn’t make it through the whole Pirates of the Caribbean trilogy.

So even if I had seen the mentions of the Last American Pirate hoax on the blogs I read all the time, I’m not sure that I would have bothered to follow the links. But maybe I would have. This story does combine two of my favorite things – scholarly uses of social media and history. Still, amidst holiday preparations and Oregon-style snowapocalypses I totally missed the initial stories on the topic.

Which is relevant in that I’m not a disgruntled blog reader feeling taken in. I was not personally hurt in any way by the deliberate historical hoax created by the students of History 389 at George Mason University last term.

And yet.  I keep thinking about it and I’m not sure I can really articulate why.

So, quick recap.  Professor Mills Kelly of eduwired.org fame taught a class on historical hoaxes last term.  Early in the term, he gave advance notice that his class would be perpetuating their own historical hoax.  The class created a fake story about the search for the Last American Pirate, a guy named Edward Owens.  The search was chronicled by fake student Jane on this fake blog, discussed in these fake interviews on YouTube, and finally reported as fact in this fake Wikipedia article.  Some people were taken in by said hoax, most notably a pop culture blogger at USA Today.  Kelly reportedly pulled the plug on the hoax when some of his real-world colleagues were taken in and the whole thing was revealed in the December 19th  Chronicle of Higher Education in an article only found behind the paywall.

So why do I keep thinking about it?  There has been a fair amount of discussion about it, some I really like.  Some talking about things I really don’t care about.  There are some people that love the experiment.  I’m not really moved by any of those arguments.   They seem to be mostly focused on the idea that kids today can’t get into traditional historical research, so this is a good, creative alternative.

The criticisms i find most compelling are found here, where Michael Feldstein explains why vandalising Wikipedia for the sake of a lesson is uncool and here in the comments on Dr. Kelly’s reveal post.  Commenter Martha, in particular, talks about the impact of this kind of project on trust networks.  Given that trust networks are, I think, a crucial part of meaningful information evaluation on the social web and thus a tool any information literate student should know how to use in this context, an assignment that deliberately devalues and damages those networks strikes me as problematic, even if there is some small benefit on the cautionary tale scale.

But that’s not what I keep thinking about except in a tangential way.  No, what’s got me thinking is what this project means for teaching information literacy and research — first in terms of the evaluation skills that are an overt, intended outcome articulated in the syllabus but also, and more deeply, in terms of research itself – why we do it and why we want students to do it.   These are, I suspect, related, but I’m not sure how.  Maybe if I write about it they’ll come together.  Maybe this will be in two parts.

Dr. Kelly says at the top that he is hoping for an information-literacy, information evaluation benefit to this assignment.

I’m hoping that this will mean that my students dig in and do some excellent historical research. I’m also hoping that they’ll learn a number of technical skills, will learn to work in a group, and will develop greater “information literacy” as we like to call it here. And, of course, I’m hoping they’ll have fun.

Specifically (from the syllabus – opens in PDF):

I do have some specific learning goals for this course. I hope that you’ll improve your research and analytical skills and that you’ll become a much better consumer of historical information. I hope you’ll become more skeptical without becoming too skeptical for your own good. I hope you’ll learn some new skills in the digital realm that can translate to other courses you take or to your eventual career. And, I hope you’ll be at least a little sneakier than you were before you started the course.

So the quick issue I have with this is that I just don’t see where the information literacy skills here translate into what most students need in their real work with online information sources.  Increasingly, I just think that a focus on deliberate hoaxes isn’t a very good way to teach students how to evaluate information.

Now I get that the work done to create the hoax might give the students in this class a greater appreciation for stuff that could make them more information literate, and that knowing specifically what they did to create a fake site might give them some stuff to look for in other sites, but I don’t really see the larger benefit here beyond the reminder that stuff on the Internet can be fake and I honestly don’t think that our students don’t know that full well already.

Because here’s the first thing – helping students learn that there is stuff on the wild, wild web that was put there just to trick them,  to punk them or to prank them – well, there’s not a lot of value in that.  The punker or the pranker will either be really good at it, in which case all of the abstract stuff we might teach them about how to identify bad information won’t help them because the good pranker isn’t going to do any of that stuff.  Or, and this is more likely, the prank won’t be all that good.  And our students – I really think they’re very able to identify the obvious crap that exists online.

They don’t need help identifying stuff that is fake or wrong just for the sake of being fake or wrong because there’s not a ton of stuff like that out there.  Honestly, our ability to identify stuff that exists for no other reason than to trick us is not a real-world problem that keeps me up at night. Most people who put fake or wrong or misleading information out there on the Internet have an agenda beyond April Fool’s – they’re trying to do more than trick us and what our students need is help identifying those agendas. They need help identifying the information that isn’t flat out lies, but that is a particular kind of truth.

There’s not a lot of historical information TO evaluate on the pieces of this hoax that are available to the public – the blog talks a a lot (I mean, a LOT) about how painful and difficult research in archives and mircofilm collections is – but the details about the sources themselves are pretty light.  Most sources are presented as transcripts  (“once I found the articles, there was no way to get a copy of them, apparently the machine is broken, so I had to transcribe them by hand,” that kind of thing).  The main thing that is presented as a digitized image is a will, not found in any archive or collection that could be investigated further – it is from the private attic-type collection of one of Edward Owens’ “descendants.”

Very clever.

No, what we have to consider here if we are evaluating information is not the quality of the historical sources in question (for the most part).  We don’t have the information to evaluate most of the fake sources, and beyond that – most historical sources in the world aren’t on blogs or YouTube so the skills that would help us evaluate them there wouldn’t necessarily translate to evaluating sources in archives.  What we really have to evaluate here are the classic foci of Internet evaluation: the authority of the scholar/author  herself and the nature of the digital tools used to present that scholarship.  And here is where I think it is useful to return to the criticisms mentioned above  – the tools we need to use to filter the social web are different than the tools of historical scholarship – and this project made those tools less useful for the rest of us.

Yes, we should remember that our trust networks and Wikipedia pages aren’t infallible.  Treating them as if they are is dumb and dangerous, of course.  But not starting from the assumption that someone is willing to do all this work just to fake you out? That’s not unreasonable.  Creating a hoax like this just for its own  sake, after all, is not more fun than the work it takes to do it is not fun.  This one took an entire class of students working for a whole term with the great big huge carrot of the GRADE as motivation, after all.  When someone, or a class of someones, does deliberately put false information out there – and I’m not talking here about the fake historical documents, but the fake blog posts and tweets and comments and pointers – it makes it harder for all of us to use the skills that really do help us navigate and evaluate the social web.

I think it’s pretty significant that outside of the USA Today blogger, most of the people who got excited about this story – excited enough to blog about it – weren’t excited because of the history beyond the “that’s kind of cool” level.  The excitement was about how “Jane” leveraged social media tools to present her research broadly:

This undergraduate took her research to the next level by framing the experience on her blog, full with images and details from her Library of Congress research, video interviews with scholars and her visit to Owens house, her bibliography, along with a link to the Wikipedia page she created for this little known local pirate.

Or stated more directly, after the reveal:

But I want to concentrate on something else. Amidst all the fiction, alternate and virtual realities, hoaxes and pranks, one thing jumps out at me as utterly real, wholly genuine, honest. Read Jim’s post on this when he first came across the project. Here is passion and excitement, a celebration of what a student might be able to achieve with the tools now available, given the right puzzle to work on and a supportive network and intellectual environment.

And I agree with all of this in theory, but in terms of this specific hoax there is still something missing to me, and it’s an important something.  It’s research – and inquiry – and discovery.

I know I am only seeing a tiny portion of what is going on in this classroom – and from the syllabus just the idea that one of the goals of the class is to show that hoaxes can themselves be the topic of serious historical research, just like wars or elections, is something I find fairly awesome.  I have no idea how the process of discovery was inculcated in the other projects the students did.  All I have is the public pieces of the course – the blogs, the videos, and the rest.

And that’s a piece of this discussion that shouldn’t be missed.  By putting this material up on the real web, on the public web, by consciously trying to get people to access and engage with this material the question of what kind of learning experience does this material provide for those of us NOT in the class is a valid one.   Is our learning experience supposed to be related to information literacy as well?  To history? Or is it just a clever, creative prank?

Because here’s the next thing – I don’t think that there is much of a learning experience for the rest of us in this project – at least not in terms of information literacy.

Don’t get me wrong, I value creation and creativity.  I value world-building and imagination.  And I don’t think those things are separate from academic research.  There is definitely creativity and imagination in scholarly inquiry, in looking at sources and seeing what might have been or what could be and re-searching based on that new potential meaning.  Watching a class of students using the social web to extend and communicate such a learning process would itself be valuable in that information literacy context.

And I think there’s room in that picture for fiction as well – in telling a story that you know in your bones to be a kind of truth even though you can’t prove it, at least not in a way that would be recognized as proof, epistemologically speaking.  I think there are truths and stories and voices that can only be captured with fiction.  So it’s not the made up or false part that gives me pause.

But in the case of this project, as it is laid out for us to see — the public pieces of this class project combine to celebrate what a truly information-literate student can do to take control of their own learning – but all the time that information literacy only exists on the surface.

This is why I have problems thinking about the pirate hoax as a great new way to talk about or teach information literacy. Because beyond the fact that I don’t think hoaxes are a great way to teach evaluation, I’m also not sure they are a great way to talk about research and scholarly creativity. At its heart, I think information literacy is inherently linked to inquiry, and discovery.   It’s about the ability to learn from information – not just to find the sources worth learning from but to use that new information to change the way you understand things, and change the way you approach the next question.

“Jane” talks endlessly about the physical pain she feels as a result of days of looking at microfilm:

But, I have no idea how I am functioning right now…I can barely look at the screen without wanting to throw up, my eyes are in so much pain.

And she goes on about how frustrating it is not to find that evidence in the documents that will prove that her pirate existed:

After my failed trip to the town, I was really discouraged. I found out enough information to keep me going, but nothing really substantial. I have not gotten any closer to figuring out a name, and my trips to the library that last four hours at a time to look through the microfilm (I’m convinced I’m causing permanent damage to my eyes), have yielded absolutely no results.

But she never talks about that other kind of pain and frustration that comes with research and learning – one of the big things that makes research hard – feeling stupid, or having to question what you thought you knew before.   That’s what I mean when I say “Jane’s” process is all surface-level.  She never finds anything in her research that leads her in a new direction. She finds additional things she can use on the path she’s already on, but that’s not the same.

In the end, it is a lucky break that brings Jane’s process to a close.  The lucky break isn’t the issue — the real issue is that at the end of the research process described in the blog she finds exactly the single document perfect right source she had been looking for from the start.  The perfect right source she imagined might exist that would answer the narrow question she formulated before she even know much about her topic at all.  That’s not how research usually works.  You could argue that that’s not how good research ever works.

And that’s the last and main thing.  At no point does Jane really engage with something that leads her to change her mind about anything, to reevaluate her process, to go back over the same ground with a new understanding or a new set of questions.  It’s needle in the haystack searching she does – she has to be creative to find different ways into the haystacks but at the same time she’s not going into the haystacks to find out what’s there.  She’s going in to look for that one needle that she thinks/hopes must be there.

And yes, I get that she’s pretend, but the fictional process the real class came up with does suggest that historical research is difficult and tedious and one doesn’t make the great discovery by engaging with sources in an open-minded way.   If the class had been engaged in a discovery-based research process I would hope that that would have come through in their fictional avatar’s narrative.  It doesn’t.  There is no doubt that this group of students were truly engaged – playing with history, creating a new world and the characters to fill it.

I can’t find it now, but when I was reading about this project earlier I was struck by the description of how the topic was selected in the first place – all of the considerations were practical – not too well known, not too likely to inspire a lawsuit if the hoax was discovered, and so on.  The reasons for piracy were practical as well – a topic of broad popular interest, local, not likely to be something anyone would already be an expert on, etc.  They didn’t talk about discovering the space in the historical record for their hoax to exist, they talked about creating it.

And if it’s mainly about creativity, about the class’ engagement around creating this alternate reality, around engaging with each other, and about engaging with others on the social web, then I’m not sure I see the value in making it a hoax.  Except that that was the topic of the rest of the class to which we were not privy.  If the skills they were learning were about creativity and world-building it seems like the resulting project could have taken the form of an ARG or a similar project where those creative muscles could be flexed in the service of creating a world for the rest of us to play in, too.

showing not telling

I’ve been working today on a presentation for next week – not a typical conference or workshop presentation, but a presentation for a class I’m teaching. I thought I was thinking of it more like a presentation than like teaching because the environment I’m teaching in here is so different than the environment I’m used to – 100 students, lecture-style classroom – very different than the 20 students/ hands-on workshop that I’m more familiar with.

Today, though, I’ve been realizing that I’m thinking of it more like a presentation as much because of the content I have to cover as anything else. AND – because I’m thinking in terms of ‘coverage’ – which is not only something I don’t usually have to do, but also a focus I usually find myself arguing against. So I had to think about it – why am I thinking about coverage now?  Why am I thinking about the stuff I want to introduce more than the stuff I want students to be able to do when they’re done?

On Monday, I need to talk about college student development theory.  Part of this is to supplement their textbook’s super-brief description of Chickering’s seven vectors of college student development, because all of the 700 students (our section + six others) are supposed to apply that model their midterm paper. Part of this is to introduce the concept of theories and models, what they means for academic discourse, and what it means to use theory critically. The following Monday, I need to talk about critical thinking. For fifteen minutes.

So, the thing is, there really isn’t anything concrete that I want them to be able to do – or, there’s nothing specific that I want them all to be able to do. I don’t really want to break down critical thinking, or what theory is, and have them take the first step in this class – which is the only way I can think about active learning in this case.  That might be a limitation on my part, but there it is.

Because a huge piece of these topics, a huge issue that I actually do want to communicate to the students, is that no one can make these students (or anyone) take theory seriously and apply it to practice.  No one can make them think critically – even if they have all of the skills and understandings necessary to do those things they can still choose not to do them and there is nothing that I, or anyone else, can do to change that.

Take this paper assignment.  Yes, we can require them to read about Chickering’s seven vectors, we can require them to understand them well enough to write about them.  We can even require them to write a paper about their own connection to that theory — but we can’t make them do it for real.  We can’t force them to reflect.  We can’t require them to be honest in their analysis and application.  They can make up every specific example from their own lives and there is truly nothing that we can do about it.  And, to be sure, doing so successfully would exhibit some high-level critical thinking and creativity skills, but it wouldn’t get at everything we want to get at here.

On at least one level, this is what the content itself is all about.  College student development theory is, in a very real way, describing the process by which students get to the point where they can do those things not just because they can, but because they want to.

So, what I’m thinking is that the issue isn’t how to figure out active learning exercises for teaching these topics.  On some level, the papers and projects in the class are the active learning experiences – and they are active learning experiences that give the students a lot of freedom to do these things in a way that makes sense for them.

No, what I’m thinking is that the best thing to do is to acknowledge up front that no one can make them use this stuff.  Correction – no one can make them meaningfully use this stuff.  Acknowledge that up front, and keep the focus of the presentation on why it’s useful to me.  Why I like it, how I use it, how other scholars use it and why it’s useful.  After all, if we’re going to ask them to be honest and reflective, it seems only fair to provide them with a little of that in my own presentation.

Oddly, I’m fairly comfortable with this focus, even though it means I’ll be talking for quite a while on Monday.  Of course, I have plenty of places where I can ask them questions and check in with them, but still it’ll be a lot of me-talking.  And I’m sure part of the class will never engage with what I say, no matter how many cool pictures and analogies I use to supplement.

All of this has me thinking as well about library instruction.  In a lot of cases, we do teach the stuff that we can (kind of) require people to do – we can require people to find articles using databases.  No matter how resistant they are, if they go through the steps and the exercises, they’ll learn something about how to do that.  It’s easy to come up with concrete, “what I want them to be able to do,” learning outcomes for that stuff.

But there are also these bigger picture questions we would like them to start thinking about, and start applying to their own lives, their own practice, and their own world view about how knowledge is communicated and valued.  And I realized, in those segments I do the same thing I’m doing here – I’ve stopped telling students that the quality of the information they’ll find in databases is inherently better, for example.  Or that the quality of peer-reviewed articles is inherently better.  Generally, I approach the “why you should use databases” piece as a demonstration, and from the perspective of “here some reasons why I use these tools or sources.  Here’s why I think they’re cool.”  It seems to work.  I’m certainly more comfortable with that approach, and they seem to be open to hearing the message in this way.

Which gets me rethinking some of these ideas about coverage.  Certainly, there are lots of situations where we value coverage over learning.  The whole idea that if I mention 10 topics in an hour, they’ll somehow learn about 10 topics in an hour is bad and has to go.  But maybe the flip side is also unbalanced – if we try too hard to make everything about active meaning-making for them, do we leave ourselves out of the equation too much?  If learning is social, and I think that it is, constructing meaning together is a crucial part of that.  We know that when it comes to group work, and to peer groups.   But maybe showing why and how we care, why and how we really use the stuff we want them to use – isn’t the same as telling so much as another way of constructing meaning together?

ARGs in academia, or, gaming for health

via ARGnet – researchers at Indiana got a big ($185K) grant from the Robert Wood Johnson foundation to study how digital, interactive games can improve students’ health.  They’re studying this by creating a Lee Sheldon-designed alternate reality game called Skeleton Chase, “to help college freshmen develop healthy habits for life.”

While the game will contain the same kinds of mysterious, creepy elements found in lots of ARGs, there’s not much secrecy about the game’s existence.  Given the legal and ethical restrictions involved when a college does research on, or provides services to, its own students it’s not too surprising that TINAG would be one of the first things to go.

In any event, new Hoosiers will find announcements about the grant and game (and associated research) on the webpage describing faculty research projects, on the Department of Telecommunications webpage, and in the local media.

It also looks like the traditional “rabbit hole” mechanism for allowing players to come across and ARG won’t be in play here.  The designers are, obviously, not disclosing a lot of plot points, but in an early press release about the grant they explained some of the game’s structure:

Sheldon is designing The Skeleton Chase, which for eight weeks will pit 30 teams of three students each against each other as they solve an “undisclosed” mystery and learn about nutrition, stress management, physical activity and other aspects of a healthy lifestyle along the way…

…Students participating in the study live in the Fitness and Wellness Living Learning Center, one of seven specially themed environments in IU Bloomington residence halls. Johnston said Residential Programs and Services is supporting a pilot project designed to examine the impact of participation in the Fitness and Wellness Living Learning Center on health and well-being within the college student population.

One of the things that is interesting about this to me is that research and assessment about the learning impact of this game is built into the project.  The researchers in this case, like principal investigator Jeanne Johnston, are focusing on whether participants’ health and wellness related habits change by looking at physical activity, health and wellness outcomes at the start and the finish of the school year in question.

They also plan to look a little more deeply at the gaming experience and what people like about it.  Researcher Anne Massey (Lilly Faculty Fellow for Information Systems) developed a “a psychological attractiveness metric and procedure to assess not only the strengths and weaknesses of design elements embedded in The Skeleton Chase, but assess other games as well.”

I hope this research also parses out some of the experiential aspects of this kind of “alternate learning environment” – something a little different, or at least not pretending to be, an alternate reality.

critically thinking about comment threads

So this study, the one from Science suggesting that gender isn’t such a useful variable when trying to predict if an individual will be good or math at not – is all over my feeds and my del.icio.us network. And it’s got me thinking about critical thinking, perception, and the really big thing we’re trying to support with our talk and our teaching about information literacy.

So the study in question basically looked at NCLB data from a lot of states, looking at how students performed on the math sections by gender. The differences they found were statistically insignificant at every level, from primary to secondary grades. They concluded that,

for grades 2 to 11, the general population no longer shows a gender difference in math skills….There is evidence of slightly greater male variability in scores, although the causes remain unexplained. Gender differences in math performance, even among high scorers, are insufficient to explain lopsided gender patterns in participation in some STEM fields.

So what does this have to do with critical thinking? The study itself isn’t really what I’m interested in here so much as the reaction to it. Because one of the things critical thinking is about is how we react when we come across information that challenges what we thought to be true. And for every math teacher that reacted to this study with a “duh” there are a lot of people around there who have some ingrained assumptions about how boys are better at math and girls are better at reading.

One of the most common narratives about boys and girls and math goes like this – boys and girls show similar aptitude so long as the math is easy. But when it gets complex, boys are better. That’s the line that used to explain why girls stopped taking math in high school, and now it’s used to explain why they don’t do math as much in college. So it’s not like I was surprised to see that that a whole bunch of commenters go Right There.

But the thing is – the study’s authors deal with this. They talk about the complexity question (they used a different data set to get at that) and they talk about the SAT scores thing. It’s not buried – it’s a whole section with a heading and everything.

And we can’t blame bad science reporting or Science’s paywall on this – the posts or linked stories mention the complexity question because the authors didn’t just mention it in the study – they emphasized it. This is a super-short article, and they spend some of their very limited time to say that our NCLB tests kind of suck – they don’t test for much, at least not for what they should be testing for. I mean really – that topic is their big finish, the last line -

An unexpected finding was that state assessments designed to meet NCLB requirements fail to test complex problem-solving of the kind needed for success in STEM careers, a lacuna that should be fixed.

Now I’m not saying that these commenters should automatically buy the analysis presented, but they should notice it. They should engage with it. Not to do so suggests, well, a lack of a disposition to think critically.

In the very late 80′s the APA engaged in a Delphi project to define critical thinking in a way that would be useful for higher education and for educational assessment. A panel of experts on critical thinking instruction, assessment and theory was convened and together they developed an influential consensus* of an ideal critical thinker as -

1. Someone who can think critically, has a set of skills, including : interpretation, analysis, evaluation, inference, explanation and self-evaluation. This skill dimension is an essential part of critical thinking.

2. Someone with the disposition to use those skills, to learn. Critical thinkers are sensitive about their own biases. They are open-minded. They are inquisitive, questioning people. They have an eagerness for knowledge and learning.

(An aside – the Delphi Method of research that grounded this project is pretty cool itself if you’re geeky like me)

Some definitive examples of lacking the disposition to think critically can be found at ABC News’ coverage of the gender/math study (ETA – in the comments, not ABC’s report) — there’s this:

The fact remains, boys tend to do better in math than girls. And there’s no shame in that. Just like girls tend to do better in languages.I wonder who skewed these figures?

And there’s this:

That doesn’t make any sense. There is no rational reason for this gap to disappear. It is a fact that men are better then women at certain tasks and worse then them at others. I think that the disappearance of this gap speaks more to our educators doing a better job of “teaching the tests” then to students actually understanding the material better.

In other words, “I read this thing. It contradicts what I believe. So I will simply restate my previously held beliefs and perhaps suggest a conspiracy.”

Now, these people obviously aren’t worth engaging with – I mean, they’re commenting on a story at ABC News dot com, and they’re not doing so especially well. But the thing is – I’ve read things just like this from my students before.
We have them write a bunch of stuff about the things they encounter in their early exploratory research stages and so we get a lot of information about how they’re reacting to the ideas they encounter. Sometimes their reaction is exactly this – “this article says X which is wrong because I believe Y.”

And that’s not a slam on my students – learning how to think critically, and developing the disposition to think critically is something that we should expect people to do in the college years. But that aspect of it – that willingness to examine your own biases and to accept new information that challenges your absolute world view as potentially valid – that’s the critical thinking big leagues. It’s not easy stuff. Not for anyone.

And what the many, many online discussions of this study have got me thinking about is how many different ways that one can resist thinking critically – the discussions on Slashdot and at the Chronicle, for example, are at an obviously higher level than the one at ABC News – they’re discussions, for one. And the arguments raised are more complex, and mostly subtler than “nuh uh.” But I think there’s still a lot of knee-jerk refusals to consider information that challenges worldviews, mental models, belief structures or whatever you want to call all of that cognitive and affective and mental baggage we bring along with us when we encounter new information going on in those comment threads.

With some others, Paul Facione (the guy who wrote the executive summary for the Delphi project) talks more thoroughly about the disposition to think critically** and in particular this article talks about what we might expect from new college students. There’s a lot of good stuff here but I’m going to engage in some super-simplistic summary and say that the authors show that college students are positively disposed to think critically in many ways – but the one that hangs them up some is this “truth-seeking” aspect.

I’m not in love with the phrase “truth-seeking” here but I’m fine with what they mean by that phrase – someone with a positive disposition towards truth seeking is “eager to seek the best knowledge in a given context, courageous about asking questions and honest and objective about pursuing inquiry if the findings do not support one’s self-interests or one’s preconceived opinions.”

Just as interesting is the related finding – that, for the most part, these students were rewarded more in their first year of college for showing positive dispositions along other scales (most notably “analyticity” or the ability to evaluate and create reasoned arguments) than for truth-seeking. That piece feels true to me, at least so far as my experiences with argument papers and comm 111 speeches extends.

While we encourage students to choose a topic they want to learn about, not just one they feel strongly about (and in this our composition faculty are taking a different tack than the one in most of the books I’ve seen)m many students still choose to write on topics they “already know.” Sometimes it is clear from the start that they feel so strongly about their chosen topic that they will not be able to learn from their research process. And, of course, some of them can craft beautifully-reasoned arguments without ever really engaging with sources in a way that leaves them open to changing their minds on a topic. I know I’ve done it.

We do focus on their argument-building ability more than their truth-seeking, and perhaps that isn’t where they need the most help to become critical thinkers. Over the years, we have added some dimensions of the latter into their work, asking them to reflect on their own biases and preconceptions, for example, but I suspect we could do more. Something to think about – hopefully critically and open-mindedly.

___________

*Facione, Peter A. (1990). Executive Summary, “The Delphi Report” (opens in PDF

**Facione, P.A., Sanchez (Giancarlo), C.A., Facione, N.C. & Gainen, J. (1995) The disposition toward critical thinking. Journal of General Education, 44:1, 1-25.

pointing out those giants, there with the shoulders

So back in April, gg at Skulls in the Stars challenged science bloggers across the disciplines to read and research some classic article in their discipline, and then write a blog post about it.  The results are in, and they’re awesome.  Not just fascinating – this is a potential time suck (with none of the guilt I feel wasting time with old sports clips on YouTube – I mean, it’s reading about science.  Important science!) – but also a really intriguing way to think about introducing a lot of overlapping ideas about scholarship to students.

One – we all know that context is one of the hardest things to figure out when you’re taking your first steps into understanding a new topic or discipline. Which things to read, what do they mean, why were they important, why are they still important  – answers to these questions aren’t immediately apparent to an outsider and scholarship written for other experts takes a lot of the keys to unlocking this discourse for granted.  Each of these posts lifts some disciplinary curtain aside, telling us what to read and why – in language written not for experts but for smart, motivated people who don’t already have that contextual knowledge.

And by showing the significance of a work in a discourse, these bloggers also (in both text and subtext) show us something about what discourse is and how it works in science or scholarship or research.  My hands-down most favorite entry in this series is from the person who issued the initial challenge – the Gallery of Failed Atomic Models – and this entry really gets at what I’m talking about here.  From gg:

It is often said that history is “written by the victors”. While this statement is usually referring to the winners of a military or political conflict, a similar effect occurs in the history of science. Physics textbooks, for instance, often describe the development of a theory in a highly abbreviated manner, omitting many of the false starts and wrong turns that were taken before the correct answer was found. While this is perfectly understandable in a textbook (it is rather inefficient to teach students all of the wrong answers before teaching them the right answer), it can lead to an inaccurate and somewhat sterile view of how science actually works.

And that might be my favorite piece of this project – the view of how science actually works that you get from these articles is anything but sterile.  They’re planning a second go-round of this project, which will be hosted here in about a month.  I’m marking my calendar.  Well not literally.  But I’m glad this will be an ongoing thing.

There’s another version of the first set of posts up at A Blog Around the Clock – organized chronologically, with some great excerpts highlighting what makes each post good.

Thanks to Cognitive Daily for the pointer.

liberation and library instruction – part 1 of ?

WorldCat recordI would really like to respond to this call for papers, and since abstracts aren’t due for several weeks I’m using it as  a reason to do some reading and re-reading.  Right now, it’s A Pedagogy for Liberation, a dialogue between Ira Shor and Paulo Freire.  This isn’t the most famous Freire, that’s undoubtedly Pedagogy of the Oppressed, but it’s one of my favorites because it is a dialogue — and they talk about the benefits of that format in language that’s very compelling – all about co-creating meaning:

Dialogue belongs to the nature of human beings, as beings of communication.  Dialogue seals the act of knowing, which is never individual, even though it has its individual dimension (p. 3-4).

And since I initially described this space as a place where I might do some pre-writing, and that concept is entirely tied up in the idea that doing that pre-writing in a place that is not my own head might be useful and valuable in a way that internal reflection is not — I’m going to indulge in putting some of the ideas this re-read is sparking down here.

I got through the first two chapters last night (and for the record, this book is very short, and very readable).  And I have mostly been thinking since about the question of motivation and what it means for libraries.  Freire and Shor agree that motivation has to be located in the here and the now of learning – not in some future benefit or some future activity.  Freire says, “I never, never could understand the process of motivation outside of practice, before practice (5).”  Shor echoes this with, “I’d emphasize that motivation has to be inside the action of study itself, inside the students’ recognition of the importance of knowing to them (6).”

I find this really compelling.  I also think is something I need to think about a lot more in terms of library instruction because much of the motivation we provide to students in library instruction sessions is “learn this and you’ll see the benefits at some later time.”  We deal with that in a limited and imperfect way by requiring that students have a research assignment that we can teach to, but that just moves the point at which the motivation kicks in a little closer.  It doesn’t actually put it in the here and now.

There’s a scene in Dazed and Confused where my favorite character Cynthia says “God, don’t you ever feel like everything we do and everything we’ve been taught is just to service the future?”  That line is why Cynthia is my favorite character and that line is what Freire and Shor are talking about here.  And that line really describes some of my anxiety about library instruction sessions, particularly those of the one-shot variety in basic skills courses that are themselves presented to students as disconnected from the “real” work they will be doing in the disciplines.

This puts the motivation two steps away, right?  Learn these basic skills so that you can perform well in later classes and you want to perform well in those later classes so that you can get a good job. Can we really blame students for feeling like nothing they do in school matters now, and can we blame them for resisting when they can’t see a direct line between the thing you’re teaching and that elusive “good job” goal down the line?  We need to give them something better, and I’m not sure what.

Or maybe I should say I’m not sure how.  I do think I have a sense of the what. I think we all have a sense of the what.  We teach this stuff because we find it intrinsically fulfilling, after all.   I talked about this briefly in the gaming post the other day, and I also talked around this concept today over at ⌘f — there is motivation to be found in research and learning.  Those processes are compelling and even fun.  But I don’t feel like I get there very often in my interactions with students – they may get there by themselves later because of something we did, but that’s not quite the same thing.

Freire and Shor argue that part of the process of finding this here and now motivation is not trying to do it alone.  In other words, by watching and listening to students and seeing what they are really doing, what they are really interested in, and what they are really motivated by, you can co-create a learning experience that will be compelling and motivating to all of the learners in the room – students and teachers alike.  I think there’s something in that for library intructors.

This means creating environments where students feel comfortable enough to act authentically and to show their true motivations.  The one-shot library session?  Probably not.  Maybe in the hands of a better or a different teacher than I am it could be, but I’ve never mastered the art of immediate (within 50 minutes anyway) relationship-building that would require.  But as librarians, we’re not limited to the classroom – we also have our libraries. And out in the library, I think, we might get some of the answers we need, if we’re willing to listen.

Games, systems and a LOTW shout-out

I have definitely hit that “what am I forgetting before ALA” mode where it is not a matter of if I forget anything, but rather how important the thing I forget will turn out to be. I am deep in the throes of preparing to present this pre-conference workshop with these awesome people while at the same time I try to make sure all loose ends are tied up here before I go.

So! I blog! Because I really want to just say a few words about one of the presentations I saw at LOEX of the West. Late, I know, but it’s sparked a really fascinating email conversation between several of my colleagues here at OSU and I want to write a few things about it before I forget them.

So the presentation was this one – A Portal to Student Learning, in which Nicholas Schiller of WSU-Vancouver argued that perhaps video games and gaming are not interesting to instruction librarians because we can make games that are more fun and engaging than traditional instruction sessions.  Instead, they should be interesting to us because the people who design games put their considerable skills, talent, time and resources to work to, essentially, teach a group of players how a system works, how to navigate that system and how to get what they need to solve problems and achieve goals within that system.

Apologies to Nicholas for that very brief and rough paraphrase but even brief and rough — it sounds a lot like what research is?

One of the overarching points here, and one that came up as well when Rachel and I talked about Alternate Reality Games at the last Online NW, is that good games and good game environments are really, really hard to do.  There are people who spend all of their professional time, every day, creating these games and environments and sometimes even they fail.  Librarians have other jobs being librarians and do we really have time to create the types of games that will be engaging, that will contain within them whatever it is that makes success within the game environment an end in itself to players?

One of my co-workers pointed out that it is hard to design a game to teach people how to research effectively because everyone’s research process is different, everyone’s goals are different, and people’s goals shift and change even as they engage in their own research process.  And that’s certainly true -if we expect their motivation to play and do well at a game to be external to the game – I want to do well at a game because of what it will get me outside the game — then I think that’s probably not the way to get engaged in a game.  But a game about information literacy skills that has within it enough motivation that people want to succeed at it for the game’s sake alone – I might be a little too cynical to be able to picture that.

But what I liked so much about Nicholas’ presentation was that he showed a way to think about this that doesn’t require us to design games that meet our users’ idiosyncratic and deeply individual needs.  It doesn’t require us to have the technical skills to develop games that will be engaging and effective.  It requires us to understand that when people are playing games they are learning, about systems and environments.  In effect, the game gives them what they need to teach themselves the rules of the game, including where those rules can be bent or broken.

And I think that’s a really exciting way to think about our interfaces, our tools and our systems.  Because they have rules too.  The ways that game designers use feedback, scaffolding, and other techniques to help the user teach themselves by doing — that seems to have direct applicability to how we can think about our systems and the tools that give our users access to those systems.  This might be a deeper and better way of thinking about visual search than I’ve been doing here for a while now.  I suspect that it is.

Because where I’m not cynical at all, I’m probably downright Pollyanna-ish, is in the idea that research brings with it its own rewards.  One reason I’m so resistant to the idea that we need to staple another motivation (winning a game) on top of learning research skills is that research itself is fun, adventurous, creative, surprising — and even competitive.  Haven’t we all felt like we won, somehow, when we made that breakthrough, found that thing that showed us where our project was going to go so that all of a sudden we could see it all the way through to the end?

I’m not sure I can describe it better than Caleb did here – talking about games and research and the fun.  I think he’s right – that libraries are very well suited to that kind of learning.  But our systems don’t always keep up.  So thanks Nick, for suggesting some ways that maybe they can.

dude, that’s so punk rock

So my Facebook friends, and my other friends, and the people in the cubicles next to me, and, well, anyone who has ever heard me speak knows that I’m not a big fan of the Blackboard learning management system. Despite having some good interactions over email with Karen Gage and the group of people responsible for the “2.0″ side of Blackboard development, my actual experiences with Blackboard have been confusing, frustrating, clunky and … more frequent than I would like to remember.

So this post yesterday at the Chronicle’s Wired Campus made me smile when I scanned the headline and saved it to my del.icio.us account – I’m not just a disgruntled librarian who lacks the patience to make Blackboard sing. I’m totally punk rock. This morning, however, I realized I needed to go back and read more than the headline – this edupunk idea is apparently a whole thing that resonates with a bunch of people.

This post says “Enter EDUPUNK” and future posts will continue to develop the idea and what it means. I’ll admit I loved this post because it details how the idea developed out of connections made between frustration with something Blackboard was doing, reading a really great novel and an awesome bar conversation — a process I can totally relate to. I loved this statement:

The insanely irresponsible advertising for BlackBoard 8 suggests that Academic Suite release 8.0 will “enhance critical thinking skills” and “improve classroom performance.” What LMS can do this? What Web 2.0 tool can do this? This is total bullshit, how can they make such an irresponsible claim? These things are not done by technology, but rather people thinking and working together.

OMG yes. But more than that – the connections drawn go on to talk about the implications of the specific, corporate environment Blackboard creates and is created by — “And this move by BlackBoard to commodify the labor of others is exactly the problem with the idea that educational technology “is about the technology.”

On his other blog, term-coiner Jim Groom provides an image of himself as the edupunk poster boy. There’s a little badge on the bavatuesdays blog now – leading here — edupunk.org (nothing there yet but an anthem)

Leslie Madsen Brooks at BlogHer provides a much better rundown of the conversation that’s been going on on the blogs.

Generally speaking, when I’ve talked about my problems with Blackboard I have focused on the closed nature of the LMS and how I think that closed off, password-protected, walled garden e-learning environment doesn’t work. Of course on one level I’m talking about the way that Blackboard just doesn’t work very well and there’s no way to go in and fix it. It’s a closed shop and a closed shop that doesn’t seem all that interested in creating something that works well.

But the real problem with the passwords and the walled gardens isn’t really about how it affects me as a teacher. I usually end up talking about how those things affect students as learners. I think that learning how to learn on the web and learn from other people on the web, is an essential part of what it means to be able to learn at all today. And that means learning how to learn in public. It means learning how to find the learning communities that will help you get where you need to go, and learning how to participate in those learning communities — contributing to the shared knowledge as well as consuming it. My del.icio.us network is hands-down one of the most important learning networks I have and it has been for a long time. And a big part of why it is so important is the way that it pushes things I wouldn’t otherwise find across my path. In that community, I make connections between things that I would never have made otherwise.

When Blackboard introduced it’s del.icio.us clone Scholar, Karen Gage provided me with a password so that I could check it out. Even though my interactions with her were really positive, my reactions to the product were not. The big advantage over del.icio,us seemed to be the ability to tag items by course name/number. I needed the special password in the first place because the only way to use it in my own BB environment was for our Blackboard administrator to do a system-wide implementation. And at that time, you could bring your bookmarks into Scholar, but you couldn’t export them out.

Scholar to me became the perfect metaphor of the BB LMS — sort everything into courses, don’t even consider that the best learning comes sometimes from drawing connections between learning experiences, don’t consider customization or user control of their own environment and once you leave school – you won’t need that knowledge base you developed while you were there anymore. Yeah, I know that Scholar might have improved in the year since I checked it out. But at the end of the day, I don’t think that matters.

Even if Scholar worked perfectly and even if it did some awesome things that del.icio.us could never do — I don’t think I would think it a substitute because of the walled garden thing. Our students don’t need to learn how to learn from a pre-selected, safe group of peers. They need to learn how to function on the wild, wide open web. They should be learning that in college. E-learning is something they will be doing for the rest of their lives. They should be learning how to do that in college. Closed off LMS’s don’t give them that experience, and they never will.

Alex Reid takes a rhetorical look at the term, and decides that it might be a case of trying too hard. And I think he might be right. I like his concise articulation of the question –

Still, I think there’s an interesting question here about how pedagogues position themselves in relation to institutionally-approved technologies and in the marketplace and commons of the larger techno-mediascape.

But I’ll also admit to liking the edupunk term. My friend Matt Cibula wrote this essay way back in the nascent days of the participatory web when we were very young but already nostalgic. Matt was a year ahead of me at Canby High School and while we never talked about the Clash while we were in Canby, this essay explains why they were important to some of us who were there better than I ever could. I honestly never expected to have a reason to link to it here – so bonus!

Teaching undergraduates about peer review – how and why, and did I mention how?

Lately I’ve noticed a number of different conversations I’ve been having coalescing around the question of evaluation – how can students evaluate the information they find. Some of the conversations have been versions of your normal standard “information on the web can be bad” and aren’t very interesting, but more of them have been about the much more interesting and much trickier question of — how do students evaluate scholarly information they find on the web when they are neither content experts (like their classroom teachers are) nor format/scholarly communication experts (like librarians are).

Which is why the title of this post jumped out and hit me over the head when I saw it today: Can you tell a good article from a bad based on the abstract and title alone?

(the post is a couple of months old and had quite a bit of discussion in the science blogs, but I haven’t seen much about it in library discussions)

So – what do you think? Can you? I sure can. And can’t. I mean, it depends, right? But when students are looking at something like this — that’s kind of what we’re asking them to do.

typical result list - ebsco

And the thing about the story linked above is that is also shows that the default we sometimes turn to – peer review – isn’t good enough. A lot of the comments on this post and on these related posts at P.Z. Myers and the Nature blogs focus on the suggestion that this paper is written from a creationist/ intelligent design perspective and the implications of this for peer review –

  • The potential that an author can choose/target politically friendly reviewers for a paper
  • The suggestion that this paper’s publications might allow an affirmative answer to the question “can you find one peer reviewed article supporting intelligent design” – and what that might mean for science.

The article was retracted by the journal, not because of its politics but because of plagiairism. Which is also something one would hope would be caught by the peer review process. It seems like it would be the least we should expect.

So on the one hand, you have the science blogs – you have someone reading the title and abstract for this article, seeing some red flags, using the dynamic web to point them out. This generates discussion, which spreads to other dynamic sites and eventually results in the article in question being pulled down. On the other hand, you have the peer reviewers, working in isolation, who didn’t seem to catch any of the red flags. On one level, it reads like a fairly straightforward Web 2.0 Makes Good story.

But on another level, what does this mean for students, especially undergraduate students? Here’s the sentence that raised the red flags for most of these scholars:

These data are presented with other novel proteomics evidence to disprove the endosymbiotic hypothesis of mitochondrial evolution that is replaced in this work by a more realistic alternative.

I can’t say this raises the same questions for me. “Novel… evidence” might be a little odd, and “a more realistic alternative” is an interesting turn of phrase. But the thing is, you have to know something about the “endosymbiotic hypothesis” to be able to contextualize, or criticize, the idea expressed here. How many students are going to have the content knowledge to do either of those things? And the other thing is – if this had become the one peer reviewed article supporting intelligent design, there’s a really good chance that even my beginning composition students would come across it.

I don’t have any really good answers for how to help students make sense of this – except I don’t think librarians and composition instructors can do this alone. And I don’t think we can make any decent stab at figuring out an answer to this question without engaging with the question of what the participatory web means for scholarship – and engaging with the related question of what the limitations of traditional peer review are as well.

And this is where the “wisdom of crowds” vs. “cult of the amateur” story that gets played out so much in the popular media really fails us. Because if this story shows anything, it shows that we still need experts to help us evaluate, contextualize and make sense of information. And at the same time it shows that trusting those experts blindly doesn’t work out so well. Adding the transparency of the participatory web to the opaque processes of traditional scholarly publication – I think part of the answer is in that grey area somewhere.

A long post at the Bench Marks blog examines the question of Why Web 2.0 is failing in Biology. It would make this too crazy long to engage with everything there today, but I do want to pull out a bit from the end. After talking about how life scientists aren’t reading or contributing content to blogs, he does look at the end at who is reading science blogs and what that might mean.

Two of the groups he pulls out are really relevant here I think — science journalists and non-scientists. If blogging is a good way to get scientific ideas out there to a more general public — people who aren’t reading the scholarly journals or going to the conferences — then they’re a way that that general public can get access to the kind of experts who can help them make sense of the research literature. More on this later, maybe.

Full disclosure – some of this thinking is to prepare for this presentation.