2008 July « info-fetishist

thinking about discourse when you don’t know what it looks like

July 30th, 2008 § 7 Comments

We have 3 online assignments that all beginning composition students complete before coming to the library for a library session. They’re designed to encourage a broad exploration of a topic, to support a research process where students develop an argument/thesis out of their reading instead of doing that at the outset, and to encourage students to try a variety of keywords and other searches on their topic.

The first two are working pretty well. The broad exploration piece I’m particularly happy with, and the developing a thesis out of research, well, there’s a lot of history and baggage to overcome there and I’m pretty confident that we’re overcoming some of that. Given that most research texts counsel narrowing a topic before starting research, and many students learn a paper-writing process in high school that explicitly tells them to develop their thesis statement first – we’re doing all right.

What I’m struggling with now is with assignment 2 – we ask students there to look at their topic and to think about how the keywords that will be successful when they search the popular media might be different than the keywords that will be successful in the scholarly literature. We are failing miserably with this assignment, I think.

We have a variety of examples to show them – and the examples work. They’re good examples, clear and when I talk to students they understand the concept we’re trying to get across. What they can’t do is apply it to their own topics. And I’m completely blocked about how to break this assignment down further, or reframe it in such a way that they can be successful on it. You see, I think the problem isn’t with understanding the concept that the language might be different in these contexts – the problem is that we’re asking students to predict something about one of those contexts (the scholarly discourse) when there’s a good chance they’ve never actually seen it.

We ask them to generate a list of potential keywords out of Wikipedia and they do a great job with this – they come up with lots of keywords that would work really, really well in scholarly journals. But when we ask them “what keywords would scholars use” they don’t list those – and how could they? Without ever reading scholarly journal articles, how could they predict how scholars frame arguments, how they articulate arguments – or even how narrow and precise some of those scholarly arguments actually are? They can’t – what we’re asking them to do is not reasonable.

So what would be reasonable? I’m really not sure. I can’t figure out how to introduce the concept AND give the students the info they need to be successful – on their topics (which can be anything) in a relatively unmediated online assignment.

In an ideal world, we’d probably be thinking more seriously about Barbara’s suggestion that this kind of thing is really being introduced too early when we introduce it in FYC (first-year composition) and that we could scaffold this learning much better. Our world isn’t ideal – our surveys suggest our students don’t do as much writing and presenting as students at comparator institutions do. And our required composition sequence isn’t a sequence at all – it’s one 10 week class. But for some of our students it’s the only writing from sources they’ll do for a while – maybe even until their Writing Intensive Course. We feel a lot of pressure to introduce the concept of academic writing at this point, particularly because it is so new to most of our students.

So I’ll keep working.

critically thinking about comment threads

July 25th, 2008 § 6 Comments

So this study, the one from Science suggesting that gender isn’t such a useful variable when trying to predict if an individual will be good or math at not – is all over my feeds and my del.icio.us network. And it’s got me thinking about critical thinking, perception, and the really big thing we’re trying to support with our talk and our teaching about information literacy.

So the study in question basically looked at NCLB data from a lot of states, looking at how students performed on the math sections by gender. The differences they found were statistically insignificant at every level, from primary to secondary grades. They concluded that,

for grades 2 to 11, the general population no longer shows a gender difference in math skills….There is evidence of slightly greater male variability in scores, although the causes remain unexplained. Gender differences in math performance, even among high scorers, are insufficient to explain lopsided gender patterns in participation in some STEM fields.

So what does this have to do with critical thinking? The study itself isn’t really what I’m interested in here so much as the reaction to it. Because one of the things critical thinking is about is how we react when we come across information that challenges what we thought to be true. And for every math teacher that reacted to this study with a “duh” there are a lot of people around there who have some ingrained assumptions about how boys are better at math and girls are better at reading.

One of the most common narratives about boys and girls and math goes like this – boys and girls show similar aptitude so long as the math is easy. But when it gets complex, boys are better. That’s the line that used to explain why girls stopped taking math in high school, and now it’s used to explain why they don’t do math as much in college. So it’s not like I was surprised to see that that a whole bunch of commenters go Right There.

But the thing is – the study’s authors deal with this. They talk about the complexity question (they used a different data set to get at that) and they talk about the SAT scores thing. It’s not buried – it’s a whole section with a heading and everything.

And we can’t blame bad science reporting or Science’s paywall on this – the posts or linked stories mention the complexity question because the authors didn’t just mention it in the study – they emphasized it. This is a super-short article, and they spend some of their very limited time to say that our NCLB tests kind of suck – they don’t test for much, at least not for what they should be testing for. I mean really – that topic is their big finish, the last line -

An unexpected finding was that state assessments designed to meet NCLB requirements fail to test complex problem-solving of the kind needed for success in STEM careers, a lacuna that should be fixed.

Now I’m not saying that these commenters should automatically buy the analysis presented, but they should notice it. They should engage with it. Not to do so suggests, well, a lack of a disposition to think critically.

In the very late 80′s the APA engaged in a Delphi project to define critical thinking in a way that would be useful for higher education and for educational assessment. A panel of experts on critical thinking instruction, assessment and theory was convened and together they developed an influential consensus* of an ideal critical thinker as -

1. Someone who can think critically, has a set of skills, including : interpretation, analysis, evaluation, inference, explanation and self-evaluation. This skill dimension is an essential part of critical thinking.

2. Someone with the disposition to use those skills, to learn. Critical thinkers are sensitive about their own biases. They are open-minded. They are inquisitive, questioning people. They have an eagerness for knowledge and learning.

(An aside – the Delphi Method of research that grounded this project is pretty cool itself if you’re geeky like me)

Some definitive examples of lacking the disposition to think critically can be found at ABC News’ coverage of the gender/math study (ETA – in the comments, not ABC’s report) — there’s this:

The fact remains, boys tend to do better in math than girls. And there’s no shame in that. Just like girls tend to do better in languages.I wonder who skewed these figures?

And there’s this:

That doesn’t make any sense. There is no rational reason for this gap to disappear. It is a fact that men are better then women at certain tasks and worse then them at others. I think that the disappearance of this gap speaks more to our educators doing a better job of “teaching the tests” then to students actually understanding the material better.

In other words, “I read this thing. It contradicts what I believe. So I will simply restate my previously held beliefs and perhaps suggest a conspiracy.”

Now, these people obviously aren’t worth engaging with – I mean, they’re commenting on a story at ABC News dot com, and they’re not doing so especially well. But the thing is – I’ve read things just like this from my students before.
We have them write a bunch of stuff about the things they encounter in their early exploratory research stages and so we get a lot of information about how they’re reacting to the ideas they encounter. Sometimes their reaction is exactly this – “this article says X which is wrong because I believe Y.”

And that’s not a slam on my students – learning how to think critically, and developing the disposition to think critically is something that we should expect people to do in the college years. But that aspect of it – that willingness to examine your own biases and to accept new information that challenges your absolute world view as potentially valid – that’s the critical thinking big leagues. It’s not easy stuff. Not for anyone.

And what the many, many online discussions of this study have got me thinking about is how many different ways that one can resist thinking critically – the discussions on Slashdot and at the Chronicle, for example, are at an obviously higher level than the one at ABC News – they’re discussions, for one. And the arguments raised are more complex, and mostly subtler than “nuh uh.” But I think there’s still a lot of knee-jerk refusals to consider information that challenges worldviews, mental models, belief structures or whatever you want to call all of that cognitive and affective and mental baggage we bring along with us when we encounter new information going on in those comment threads.

With some others, Paul Facione (the guy who wrote the executive summary for the Delphi project) talks more thoroughly about the disposition to think critically** and in particular this article talks about what we might expect from new college students. There’s a lot of good stuff here but I’m going to engage in some super-simplistic summary and say that the authors show that college students are positively disposed to think critically in many ways – but the one that hangs them up some is this “truth-seeking” aspect.

I’m not in love with the phrase “truth-seeking” here but I’m fine with what they mean by that phrase – someone with a positive disposition towards truth seeking is “eager to seek the best knowledge in a given context, courageous about asking questions and honest and objective about pursuing inquiry if the findings do not support one’s self-interests or one’s preconceived opinions.”

Just as interesting is the related finding – that, for the most part, these students were rewarded more in their first year of college for showing positive dispositions along other scales (most notably “analyticity” or the ability to evaluate and create reasoned arguments) than for truth-seeking. That piece feels true to me, at least so far as my experiences with argument papers and comm 111 speeches extends.

While we encourage students to choose a topic they want to learn about, not just one they feel strongly about (and in this our composition faculty are taking a different tack than the one in most of the books I’ve seen)m many students still choose to write on topics they “already know.” Sometimes it is clear from the start that they feel so strongly about their chosen topic that they will not be able to learn from their research process. And, of course, some of them can craft beautifully-reasoned arguments without ever really engaging with sources in a way that leaves them open to changing their minds on a topic. I know I’ve done it.

We do focus on their argument-building ability more than their truth-seeking, and perhaps that isn’t where they need the most help to become critical thinkers. Over the years, we have added some dimensions of the latter into their work, asking them to reflect on their own biases and preconceptions, for example, but I suspect we could do more. Something to think about – hopefully critically and open-mindedly.

___________

*Facione, Peter A. (1990). Executive Summary, “The Delphi Report” (opens in PDF

**Facione, P.A., Sanchez (Giancarlo), C.A., Facione, N.C. & Gainen, J. (1995) The disposition toward critical thinking. Journal of General Education, 44:1, 1-25.

pointing out those giants, there with the shoulders

July 16th, 2008 § Leave a Comment

So back in April, gg at Skulls in the Stars challenged science bloggers across the disciplines to read and research some classic article in their discipline, and then write a blog post about it. The results are in, and they’re awesome. Not just fascinating – this is a potential time suck (with none of the guilt I feel wasting time with old sports clips on YouTube – I mean, it’s reading about science. Important science!) – but also a really intriguing way to think about introducing a lot of overlapping ideas about scholarship to students.

One – we all know that context is one of the hardest things to figure out when you’re taking your first steps into understanding a new topic or discipline. Which things to read, what do they mean, why were they important, why are they still important – answers to these questions aren’t immediately apparent to an outsider and scholarship written for other experts takes a lot of the keys to unlocking this discourse for granted. Each of these posts lifts some disciplinary curtain aside, telling us what to read and why – in language written not for experts but for smart, motivated people who don’t already have that contextual knowledge.

And by showing the significance of a work in a discourse, these bloggers also (in both text and subtext) show us something about what discourse is and how it works in science or scholarship or research. My hands-down most favorite entry in this series is from the person who issued the initial challenge – the Gallery of Failed Atomic Models – and this entry really gets at what I’m talking about here. From gg:

It is often said that history is “written by the victors”. While this statement is usually referring to the winners of a military or political conflict, a similar effect occurs in the history of science. Physics textbooks, for instance, often describe the development of a theory in a highly abbreviated manner, omitting many of the false starts and wrong turns that were taken before the correct answer was found. While this is perfectly understandable in a textbook (it is rather inefficient to teach students all of the wrong answers before teaching them the right answer), it can lead to an inaccurate and somewhat sterile view of how science actually works.

And that might be my favorite piece of this project – the view of how science actually works that you get from these articles is anything but sterile. They’re planning a second go-round of this project, which will be hosted here in about a month. I’m marking my calendar. Well not literally. But I’m glad this will be an ongoing thing.

There’s another version of the first set of posts up at A Blog Around the Clock – organized chronologically, with some great excerpts highlighting what makes each post good.

Thanks to Cognitive Daily for the pointer.

liberation and library instruction – part 1 of ?

July 10th, 2008 § 1 Comment

WorldCat record I would really like to respond to this call for papers, and since abstracts aren’t due for several weeks I’m using it as a reason to do some reading and re-reading. Right now, it’s A Pedagogy for Liberation, a dialogue between Ira Shor and Paulo Freire. This isn’t the most famous Freire, that’s undoubtedly Pedagogy of the Oppressed, but it’s one of my favorites because it is a dialogue — and they talk about the benefits of that format in language that’s very compelling – all about co-creating meaning:

Dialogue belongs to the nature of human beings, as beings of communication. Dialogue seals the act of knowing, which is never individual, even though it has its individual dimension (p. 3-4).

And since I initially described this space as a place where I might do some pre-writing, and that concept is entirely tied up in the idea that doing that pre-writing in a place that is not my own head might be useful and valuable in a way that internal reflection is not — I’m going to indulge in putting some of the ideas this re-read is sparking down here.

I got through the first two chapters last night (and for the record, this book is very short, and very readable). And I have mostly been thinking since about the question of motivation and what it means for libraries. Freire and Shor agree that motivation has to be located in the here and the now of learning – not in some future benefit or some future activity. Freire says, “I never, never could understand the process of motivation outside of practice, before practice (5).” Shor echoes this with, “I’d emphasize that motivation has to be inside the action of study itself, inside the students’ recognition of the importance of knowing to them (6).”

I find this really compelling. I also think is something I need to think about a lot more in terms of library instruction because much of the motivation we provide to students in library instruction sessions is “learn this and you’ll see the benefits at some later time.” We deal with that in a limited and imperfect way by requiring that students have a research assignment that we can teach to, but that just moves the point at which the motivation kicks in a little closer. It doesn’t actually put it in the here and now.

There’s a scene in Dazed and Confused where my favorite character Cynthia says “God, don’t you ever feel like everything we do and everything we’ve been taught is just to service the future?” That line is why Cynthia is my favorite character and that line is what Freire and Shor are talking about here. And that line really describes some of my anxiety about library instruction sessions, particularly those of the one-shot variety in basic skills courses that are themselves presented to students as disconnected from the “real” work they will be doing in the disciplines.

This puts the motivation two steps away, right? Learn these basic skills so that you can perform well in later classes and you want to perform well in those later classes so that you can get a good job. Can we really blame students for feeling like nothing they do in school matters now, and can we blame them for resisting when they can’t see a direct line between the thing you’re teaching and that elusive “good job” goal down the line? We need to give them something better, and I’m not sure what.

Or maybe I should say I’m not sure how. I do think I have a sense of the what. I think we all have a sense of the what. We teach this stuff because we find it intrinsically fulfilling, after all. I talked about this briefly in the gaming post the other day, and I also talked around this concept today over at ⌘f — there is motivation to be found in research and learning. Those processes are compelling and even fun. But I don’t feel like I get there very often in my interactions with students – they may get there by themselves later because of something we did, but that’s not quite the same thing.

Freire and Shor argue that part of the process of finding this here and now motivation is not trying to do it alone. In other words, by watching and listening to students and seeing what they are really doing, what they are really interested in, and what they are really motivated by, you can co-create a learning experience that will be compelling and motivating to all of the learners in the room – students and teachers alike. I think there’s something in that for library intructors.

This means creating environments where students feel comfortable enough to act authentically and to show their true motivations. The one-shot library session? Probably not. Maybe in the hands of a better or a different teacher than I am it could be, but I’ve never mastered the art of immediate (within 50 minutes anyway) relationship-building that would require. But as librarians, we’re not limited to the classroom – we also have our libraries. And out in the library, I think, we might get some of the answers we need, if we’re willing to listen.

Why we should read it before we cite it — no, really!

July 8th, 2008 § 5 Comments

Last week, Female Science Professor wrote a lovely pair of posts about scholars and scholarship, what it feels like when your work has an impact on someone and what it feels like to meet the people who have influenced you in that particular undefinable way where it’s hard to even express what they’ve meant to you. I shared one, saved the other and generally felt very good about being a small part of this world where rock star crushes on ideas and the people who share them are understood and embraced.

Way to ruin everything, Inside Higher Ed.

Okay, not really. But seriously, it’s a lot harder to feel like a rock star because someone has read and used your work if, as Malcolm Wright and J. Scott Armstrong suggest, they probably didn’t read it and if they did, they probably read it wrong.

That might be a little bit strong, but not by much. So what does it mean when a published, peer-reviewed article in a real life journal kicks off its final, concluding paragraph with this sentence – Authors should read the papers they cite.

This isn’t a library tutorial aimed at fifth-graders writing their first research paper, after all. This is a paper talking about what professional scholars, people responsible for the continued development of knowledge in disciplines, should do. It can’t mean anything good. Here’s the original article:

Article at Interfaces – requires subscription

Article at Dr. Anderson’s faculty page – does NOT require a subscription – (opens in PDF)

Nutshell – Dr. Anderson wrote one of the more impact-heavy articles in his discipline, and the only article that analyzes and explains how to correct for non-response bias in mail surveys (that’s bias caused by people who do not respond at all to the survey). By analyzing 1. how often research based on mail surveys includes a citation to this article, and 2. how often the later researchers seem to interpret and apply the original article correctly the authors conclude that many, many researchers are not reading all of the relevant literature. More disturbingly, many, many researchers aren’t even reading all of the articles they themselves cite.

Now, on one level this isn’t a shocker – anyone who has read moderately deeply in any body of literature has probably looked at at least one bloated literature review and said “hey – this person probably didn’t really read all of these books and articles.” This article suggests that it’s more complex than just lit-review padding, that scholarly authors also mis-cite and mis-use the resources they use to support the methods they use and the conclusions they draw.

Working on the assumption that if your research uses a mail survey, you should at least be considering the possibility of nonresponse bias, they found that:

…far less than one in a thousand mail surveys consider evidence-based findings related to nonresponse bias. This has occurred even though the paper was published in 1977 and has been available in full text on the Internet for many years.

Working on the further assumption that someone who makes a claim about nonresponse bias, and who reads, understands and cites an article that outlines a particular method for correcting nonresponse bias to support that claim, will follow the method outlined in the article they cited, the authors conclude that many authors are either not reading or are not understanding the articles they cite:

The net result is that whereas evidence-based procedures for dealing with nonresponse bias have been available since 1977, they are properly applied only about once every 50 times that they are mentioned, and they are mentioned in only about one out of every 80 academic mail surveys.

Most of the research that seriously digs into how well researchers use the sources they cite has come out of the sciences, particularly the medical sciences. This is one of the first articles I’ve seen dealing with the social sciences, and I think it’s worth reading more closely because this very rough and brief summary doesn’t really do justice to the issues it raises. But right now, I want to turn to the authors’ conclusions because I think they get at some of the things we’ve been talking about around here about how new technologies and the read/write web might have an impact on scholarship.

The first two outline author responsibilities:

First – read the sources you cite. I think we can take that as a given – a bare-minimum practice not a best practice.
Secondly, “authors should use the verification of citations procedure.” Here they’re calling for authors to contact all of the researchers whose work they want to cite to make sure that they’re citing it correctly. I’m going to come back to this one.

The second two put some of the burden on the journals:

Journals should require authors to attest that they have in fact read the work they cite and that they have performed due diligence to make sure their citations are correct. That seems a sad, largely symbolic, but not unreasonable precaution.
Finally, journals should provide easily accessible webspaces for other people to post additional work and additional research that is relevant to research that has been published in the journal. Going to come back to this one too because I think it’s actually related to the one above.

Basically – both of these recommendations suggest that more communication and more transparency would be more better for knowledge creation. And what is the read/write web about if not communication and transparency, networking and openness?

Some of the commenters on the IHE article expressed, shall we say, polite skepticism that an author should be obligated to contact every person they cite before citing them. These concerns were also raised by one of the formal comment pieces attached to the Interfaces article. And I have to say I agree with these concerns for a few reasons. Anderson made the claim more than once that he does this as an author, with good results, and that the process is not too onerous. But that doesn’t really address the question of how onerous it would be for a prolific or influential author to have to field all of those requests.

And I’ll also admit to having some author is dead reactions to this. What if I contact Author A and say I’m planning to use your work in this way and they say “well I didn’t intend it to be used in that way so you shouldn’t.” Does that really mean I shouldn’t? Really? It’s hard to see this kind of thing not devolving quickly into something that actually hinders the development of new knowledge because it hinders new researchers’ ability to push at and find new connections in work that has come before.

But not to throw everything out with this bathwater – the idea that more and better and faster communication between scholars (more and better and faster than can be provided within journals and the citation-as-communication tradition) makes better scholarly conversations and better scholarship – that’s something I think we need to hold on to. Anderson points out how talking to the researcher who really knows the area described in the thing you’re citing can point you to other, less cited but more useful resources – how they can expand your knowledge of the field you’re talking about:

We checked with Franke to verify that we cited his work correctly. He directed us to a broader literature, and noted that Franke (1980) provided a longer and more technically sophisticated criticism; this later paper has been cited in the ISI Citation Index just nine times as of August 2006.

This is an area where the transparency, speed and networking aspects of the emerging web might have a real impact on the quality of scholarship even if there are no material changes in the practice of producing journal articles. I might not be sure about the idea of making this communication a part of citation verification but it should be a part of knowledge creation. And it’s tied as well with the final recommendation – that journals should provide webspaces for some, not all but some, of this communication to happen.

The types of conversations between similarly interested scholars that Anderson is describing is nothing new – the emerging web offers some opportunities for those conversations to move off the backchannel. Or maybe it’s the idea that it’s still a backchannel, but the back channel being visible is interesting. Whether the journal has its own backchannel for errors, additions, omissions and new ideas to be posted, or whether that backchannel exists on blogs, in online knoweldge communities, or networking spaces doesn’t matter so much as it can exist. We certainly have the technology.

And the journal Interfaces itself I think provides a suggestion as to why this kind of addtional discourse and conversation is valuable. You may have noticed that what looks like a fifteen page article is really an eight page article with six pages of response pieces, followed by an authors’ response. The responses challenge parts of the original article, and enrich other parts with additional information and examples. They illustrate the collaborative nature of knowledge production in the disciplines in a way that citations alone cannot. I couldn’t find anything on the journal’s website about this practice – if it’s a regular thing, how responses are solicited, or more. These responses are a spot of openness in a fairly closed publication.

And that as well points to the last point to make here because this is far too long already – I don’t think we have to change everything to fix the problems raised here – and I don’t think if we did change everything it would fix all of the problems raised here. There’s that scene in Bull Durham where Eppie Calvin gets his guitar taken away because he won’t get the lyrics right. And that’s the connection between FemaleScienceProfessor and Anderson and Wight — who can feel like a rock star if they’re singing your songs but getting them wrong?

There will always be Eppie Calvins out there inside and outside of academia -for them, women are wooly because of the stress. But injecting just some openness, making some communication visible – won’t stop Eppie Calvin, but might keep the next person from replicating his mistakes. And that’s a good thing.

hey! look over there!

July 3rd, 2008 § Leave a Comment

So I wrote something today, but I didn’t write it here. I wrote it over here. And even better, Caleb and Rachel wrote stuff over there too. Stuff that I’m even more excited about than what I wrote.

Go! Read!

Where Am I?

Follow “info-fetishist”