So this study, the one from Science suggesting that gender isn’t such a useful variable when trying to predict if an individual will be good or math at not – is all over my feeds and my del.icio.us network. And it’s got me thinking about critical thinking, perception, and the really big thing we’re trying to support with our talk and our teaching about information literacy.
So the study in question basically looked at NCLB data from a lot of states, looking at how students performed on the math sections by gender. The differences they found were statistically insignificant at every level, from primary to secondary grades. They concluded that,
for grades 2 to 11, the general population no longer shows a gender difference in math skills….There is evidence of slightly greater male variability in scores, although the causes remain unexplained. Gender differences in math performance, even among high scorers, are insufficient to explain lopsided gender patterns in participation in some STEM fields.
So what does this have to do with critical thinking? The study itself isn’t really what I’m interested in here so much as the reaction to it. Because one of the things critical thinking is about is how we react when we come across information that challenges what we thought to be true. And for every math teacher that reacted to this study with a “duh” there are a lot of people around there who have some ingrained assumptions about how boys are better at math and girls are better at reading.
One of the most common narratives about boys and girls and math goes like this – boys and girls show similar aptitude so long as the math is easy. But when it gets complex, boys are better. That’s the line that used to explain why girls stopped taking math in high school, and now it’s used to explain why they don’t do math as much in college. So it’s not like I was surprised to see that that a whole bunch of commenters go Right There.
But the thing is – the study’s authors deal with this. They talk about the complexity question (they used a different data set to get at that) and they talk about the SAT scores thing. It’s not buried – it’s a whole section with a heading and everything.
And we can’t blame bad science reporting or Science’s paywall on this – the posts or linked stories mention the complexity question because the authors didn’t just mention it in the study – they emphasized it. This is a super-short article, and they spend some of their very limited time to say that our NCLB tests kind of suck – they don’t test for much, at least not for what they should be testing for. I mean really – that topic is their big finish, the last line –
An unexpected finding was that state assessments designed to meet NCLB requirements fail to test complex problem-solving of the kind needed for success in STEM careers, a lacuna that should be fixed.
Now I’m not saying that these commenters should automatically buy the analysis presented, but they should notice it. They should engage with it. Not to do so suggests, well, a lack of a disposition to think critically.
In the very late 80’s the APA engaged in a Delphi project to define critical thinking in a way that would be useful for higher education and for educational assessment. A panel of experts on critical thinking instruction, assessment and theory was convened and together they developed an influential consensus* of an ideal critical thinker as –
1. Someone who can think critically, has a set of skills, including : interpretation, analysis, evaluation, inference, explanation and self-evaluation. This skill dimension is an essential part of critical thinking.
2. Someone with the disposition to use those skills, to learn. Critical thinkers are sensitive about their own biases. They are open-minded. They are inquisitive, questioning people. They have an eagerness for knowledge and learning.
(An aside – the Delphi Method of research that grounded this project is pretty cool itself if you’re geeky like me)
Some definitive examples of lacking the disposition to think critically can be found at ABC News’ coverage of the gender/math study (ETA – in the comments, not ABC’s report) — there’s this:
The fact remains, boys tend to do better in math than girls. And there’s no shame in that. Just like girls tend to do better in languages.I wonder who skewed these figures?
And there’s this:
That doesn’t make any sense. There is no rational reason for this gap to disappear. It is a fact that men are better then women at certain tasks and worse then them at others. I think that the disappearance of this gap speaks more to our educators doing a better job of “teaching the tests” then to students actually understanding the material better.
In other words, “I read this thing. It contradicts what I believe. So I will simply restate my previously held beliefs and perhaps suggest a conspiracy.”
Now, these people obviously aren’t worth engaging with – I mean, they’re commenting on a story at ABC News dot com, and they’re not doing so especially well. But the thing is – I’ve read things just like this from my students before.
We have them write a bunch of stuff about the things they encounter in their early exploratory research stages and so we get a lot of information about how they’re reacting to the ideas they encounter. Sometimes their reaction is exactly this – “this article says X which is wrong because I believe Y.”
And that’s not a slam on my students – learning how to think critically, and developing the disposition to think critically is something that we should expect people to do in the college years. But that aspect of it – that willingness to examine your own biases and to accept new information that challenges your absolute world view as potentially valid – that’s the critical thinking big leagues. It’s not easy stuff. Not for anyone.
And what the many, many online discussions of this study have got me thinking about is how many different ways that one can resist thinking critically – the discussions on Slashdot and at the Chronicle, for example, are at an obviously higher level than the one at ABC News – they’re discussions, for one. And the arguments raised are more complex, and mostly subtler than “nuh uh.” But I think there’s still a lot of knee-jerk refusals to consider information that challenges worldviews, mental models, belief structures or whatever you want to call all of that cognitive and affective and mental baggage we bring along with us when we encounter new information going on in those comment threads.
With some others, Paul Facione (the guy who wrote the executive summary for the Delphi project) talks more thoroughly about the disposition to think critically** and in particular this article talks about what we might expect from new college students. There’s a lot of good stuff here but I’m going to engage in some super-simplistic summary and say that the authors show that college students are positively disposed to think critically in many ways – but the one that hangs them up some is this “truth-seeking” aspect.
I’m not in love with the phrase “truth-seeking” here but I’m fine with what they mean by that phrase – someone with a positive disposition towards truth seeking is “eager to seek the best knowledge in a given context, courageous about asking questions and honest and objective about pursuing inquiry if the findings do not support one’s self-interests or one’s preconceived opinions.”
Just as interesting is the related finding – that, for the most part, these students were rewarded more in their first year of college for showing positive dispositions along other scales (most notably “analyticity” or the ability to evaluate and create reasoned arguments) than for truth-seeking. That piece feels true to me, at least so far as my experiences with argument papers and comm 111 speeches extends.
While we encourage students to choose a topic they want to learn about, not just one they feel strongly about (and in this our composition faculty are taking a different tack than the one in most of the books I’ve seen)m many students still choose to write on topics they “already know.” Sometimes it is clear from the start that they feel so strongly about their chosen topic that they will not be able to learn from their research process. And, of course, some of them can craft beautifully-reasoned arguments without ever really engaging with sources in a way that leaves them open to changing their minds on a topic. I know I’ve done it.
We do focus on their argument-building ability more than their truth-seeking, and perhaps that isn’t where they need the most help to become critical thinkers. Over the years, we have added some dimensions of the latter into their work, asking them to reflect on their own biases and preconceptions, for example, but I suspect we could do more. Something to think about – hopefully critically and open-mindedly.
___________
*Facione, Peter A. (1990). Executive Summary, “The Delphi Report” (opens in PDF
**Facione, P.A., Sanchez (Giancarlo), C.A., Facione, N.C. & Gainen, J. (1995) The disposition toward critical thinking. Journal of General Education, 44:1, 1-25.
info-fetishist 
This is fascinating. I’m going to share it with a group working on our first term seminars since they are looking at concrete issues with critical thinking, and the idea that we offer rewards for many dispositions but not for truth seeking is fascinating. Points for how many times you read something and changed your mind!
Seriously …. you’ve given me an interesting idea….. hmmm….
Meanwhile, just fyi – the Facione link doesn’t seem to work today.
Interesting posting. The gender prejudice about math ability is obviously just that. I had the opportunity to work with a Fields Medal winner (the mathematical equivalent of a Nobel Prize) many years ago, and I feel fairly strongly that there are no tests short of success in the university that would adequately identify why he was so great a mathematician.
There is an affect aspect of critical thinking. You have to care enough about an issue to spend the time and effort to critically evaluate what other people have written about that subject. One of the most powerful motivators for critical thinking is an intuition that what is written is wrong, or the cognitive dissonance when what is written conflicts with what one believes based on previous reading and analysis.
Scientists know this. The good ones get very emotional about publications that challenge the conventional dogma of their fields. The scientific system depends on the challenges of peers to validate assertions made in the literature, as well as replication of experimental results. The problem, as Kuhn points out, is that the leaders of a paradigm frequently never accept a revolutionary finding.
So I at least appreciate the person who is reluctant to accept newly advanced views that conflict with their existing views, and who follow their (often inarticultate) intuition in driving their critical thinking.
This is really interesting on a number of levels. People’s reactions to this information, while not shocking, are still somewhat disturbing to me. I especially enjoy your summary: ““I read this thing. It contradicts what I believe. So I will simply restate my previously held beliefs and perhaps suggest a conspiracy.”” I think I want this printed on a T-shirt…perhaps I will wear it while teaching WR121.
“The Bible says it. I believe it.”
That’s what we’re up against. Which I take more metaphorically, having known my fair share of doctrinaire fellow lefties. “Well, according to Marx….”
Great post.
The linked report can be found here
There’s a lot of cynicism out there about the idea that evidence can be used to form the basis of one’s beliefs. That cynicism so permeates our political system and the way it’s examined in the press that it has made it hard to believe truth is out there, or that it matters when the winner is the one who is the best manipulator. The analysis of a political debate often focuses exclusively on presentation, which valorizes argumentation totally over evidence. Nobody ever says “Candidate Kafka’s statistics on poverty totally blew the competition out of the water.” It’s “Candidate Kafka appeared angry, while the opposition candidate really bonded with the audience.”
Gee shucks, folks. Don’t want to make you uncomfortable or nothing.
Barbara – thanks for fixing the link. I actually went in and fixed the *other* one and figured that’s the one you meant – it didn’t occur to me that I entirely failed on both of them! I hate linking to pdf’s.
I hope we get to hear your idea some time. I’m very intrigued :-) Because I think this idea of truth-seeking is really important, but it IS hard to figure out a non-silly way to reward for it, isn’t it? Because you can go into a project with an awesome disposition towards truth seeking and still not end up showing any major outward signs of changing your mind (though I would imagine that your *thinking* should still change even if the stance you take doesn’t).
And I entirely agree that our culture and our media’s treatment of political discourse is entirely related to this – I started to head down that road but stopped because it’s a whole post of its own. But we seem to be in a place where refining one’s position based on new information is often treated as a sign of moral weakness – it’s enough to make me feel really pessimistic.
____
Hi John! Thanks for commenting. I’m so glad you brought up Kuhn because that’s a really important distinction. I’m actually kind of a geek about how strongly I feel about the importance of normal science, and the overuse of the term “paradigm shift.” I absolutely agree that paradigms shift slowly and more than that I think they *should* – but that’s not inconsistent with a belief that our students (and our culture) needs to be more open to letting new information influence their positions and beliefs. Scientists have a toolkit in their minds that they can use to evaluate and consider new information and even though this very toolkit might keep them from considering the truly revolutionary, it’s very useful in considering the non-revolutionary but still new. Using that toolkit requires you to actually look at the claims being made, the method and the conclusions — that’s very different than throwing out “well the truth about this is actually X” when you haven’t even read the article that says it’s Y.
I think most of us can point to more than one experience in college where we read something or heard something that really pushed us to change the way we looked at the world – those experiences are part of why we all ended up staying in college long after the B.A. or B.S., right? Some of my students don’t have the skill set or the disposition to have those experiences and that means, I think, that they’re missing a lot of what learning is about.
____
Julie – we could wear matching t-shirts! That would be very entertaining! Great blog, btw – I didn’t know you were out there but now I’ll be reading. I’m interested to hear about your thesis, Wharton is a favorite of mine but I’m a total dilettante.
____
Hi Kevin – OMG between my high school and my grad school experiences I know what you mean on *both* sides of that. I have to tell you about one of my most memorable grad school experiences – it was such a caricature of a discussion. One historiography class with a bunch of historians claiming that theory didn’t matter and that they weren’t affected by it (right). And a group of – I think lit crit people? Anyway, they were all Marxists. Like real Das Kapital-thumpers.
(Seriously, this same group ran the alternative newspaper at the school I was at, and over the course of the year their numbers got smaller and smaller as people were squeezed out for not being doctrinaire enough. First the Trotskyists left, then the Leninists. We used to joke that at the end of the year the only ones left standing would be the Stalinists.
I think there’s still evidence somewhere on the web of the time this group chose to devote an ENTIRE issue of the paper to arguing against something Shaun said about anarchism.)
The class was an education in itself and gave me a lot of chances to really refine my thinking about why theory mattered – but I just got so frustrated when we couldn’t take the discussion about new ideas to new places because we kept getting dragged back to Marx. It came to a head one day when this one woman wouldn’t let us talk about E.P. Thompson – and that’s a guy with some Marxist street cred, right? She just kept saying “But what Thompson isn’t considering is this thing Marx said about religion.” Finally someone (not me) exploded with “OMG Thompson *considered* it and rejected it. There’s a difference. ” So yeah, absolutely fundamentalisms are a huge part of what we’re struggling against.