Know when to fold ’em

Today I am teaching what might be my last session of beginning composition this term.

(Wow, I initially typed that “beginning compassion” – that’s a typo that could spark it’s own blog post right there).

Yes, I just checked, and this is my last one of the term.  I’m teaching today with a regular member of the writing faculty, who has probably done more than any other person currently on the writing faculty to maintain and cultivate the information literacy component of our beginning composition course here.

We revise constantly – both because our personalities push us to do so and because external pressures (like record enrollment for each of the last few years) make it necessary.  Most of the time, we’re tweaking, but in this last year or so we’ve really attacked the problem of what we should be teaching in this course more intentionally and aggressively than we have since probably 2005 or so.  And I wanted to talk about the process because in some ways we’ve moved in a full circle this year – from a deep, intense focus on teaching students about the peer-reviewed article (in an authentic and useful way) to de-emphasizing that part of the discourse and looking for other places where it might be taught more meaningfully.

Now, this isn’t anything that hasn’t come up a million times already this year.  Barbara has talked about it brilliantly many times; Meredith brought it up just a little while ago.  It’s not even new here – Kate and I wrote about this issue just last year in the context of other courses (outside the paywall, get it while its free).

So factors we have to consider in our FYC:

  1. The # of sections per term has gone up from about 25 when I started at OSU, to 40-45 now.
  2. The sections are taught, independently, by GTA’s, adjuncts and a few full-time faculty
  3. There is a common curriculum everyone is required to use, including assignments (portfolios), revision, and texts.
  4. As is the case on many campuses, our FYC course has a service component to it – meaning the idea that the rest of campus is relying on FYC to provide some basic instruction in academic writing.  This includes an expectation that students will learn what peer reviewed articles and library databases are.

So, we started from a question of what could we (where we = librarians) teach most meaningfully in the 50 minutes or so we had with students. Given #1, our ability to continue to teach in all of the sections of FYC can’t be taken for granted anymore.  If we want to continue, and we do, we really have to get a handle on what it is that that contact does that other ways of teaching (and teaching in other courses) can’t do.

Given #2 and #4 above, the question of “what do you need to know about finding, reading, understanding and using peer-reviewed articles” seemed like it might fit the bill.  As a requirement, the peer-reviewed article isn’t going away.  And as librarians, particularly librarians who teach FYC students every term we actually felt like we were in a better position to talk about this discourse than the TA’s, who are 1. often themselves brand new to OSU, 2. unevenly prepared (depending on their own college experiences) to teach about peer review and journals and 3. focused on a part of the scholarly literature, English and humanities, that most students are NOT going to use in their FYC essays.

So.

Here’s the thing – in the databases we see unbundled articles pulled together by our keywords in a list organized by relevance – by our keyword matches.  Everything about that “is this peer-reviewed” question, however, assumes a knowledge of the discourse that produces these articles.  The way those articles are written, formatted, contextualized and, yes, quality-controlled is all about the discourse.

“Is this peer-reviewed” shouldn’t even be the question — and I think it’s a question that  confuses.  It implies “Is this good?” “Is this high-quality?”   “Is this some kind of generic definition of ‘scholarly’?”

To make sense, though, to really reflect how peer review works, the question should be “does this journal use peer review?”  We wanted to talk about peer review as a method of quality control, to focus on the ways that peer review reinforces the expectations of the discipline.  That matched the rhet/comp focus of the course, it allowed the TA’s to talk about authorship, audience and message, concepts they were focusing on throughout the rest of the course.

So we designed a set of activities, including in-class activities, tutorials, follow-up activities and rubrics, focused on getting students to connect the article to the journal, the journal to the idea of peer-review and to understand the kinds of standards that the peer-reviewers use to decide which articles should be published.

We dropped some of the more mechanical “how to find it” pieces from our teaching and moved those to online help.  We moved most of the teaching on finding and using books to the TA’s, who were more comfortable with that discourse than they were with peer-reviewed articles.  We piloted these ideas in a few sections (about seven, selected by TA opt-in).

And it worked well.  The IL curriculum was well integrated with the rest of the course, and the classes felt meaningful.  We had some trouble covering what we wanted to cover in the classes, particularly the 50-minute sections, but there was some general idea that it was better than what we had been doing before.

(It was better)

But the little problems we noticed with the pilot became big problems when we expanded from the hand-picked, opt-in sections to all of the sections.  The TA’s who were teaching in the pilot were really engaged and invested with the curriculum and prepped the students about as well as they could possibly be prepped.  Without that level of investment, the gaps between where the students really are with their needs and their understanding of academic, source-based writing, became so very clear.  I had one class where the instructor had been so very successful communicating the “you MUST use peer-reviewed articles” message that the students were highly, highly motivated to get it right.  We spent almost 15 minutes of a 50 minute class on the “what is a journal” question.  Teaching FYC session became a stressful race against the clock and sometimes I felt like they left more confused than they were coming in, and not in a good way.

Let’s be clear. I’m not saying these students weren’t smart (they were) or that they weren’t trying (they definitely were) or that they’re weren’t serious about what we were doing (they very definitely really were).  I know that there are people out there who read these types of arguments and say “aren’t these students in college?” or “how could you not do this in high school?”

And all I have to say to that is, whatever.  You keep telling yourself that all of your high school students totally get everything about peer review, totally know what they’re going to find in scientific articles that graduate students would have to read twice, and can totally navigate the unwritten rules of scholarly/expert communities.  When I see people making those arguments, I’ll admit, my assumption is that they don’t really understand peer-review either.

Everything about the way that the scholarly literature is organized is based on the journal, the discipline and the scholarly community that connects those two things.  Expecting first-years to get that from the outside is ridiculous.  Expecting first years to get that because they’re taught about it by graduate students who are just becoming conversant with their own discourse community is ridiculous.  And expecting first years to get that because they spend 50 minutes with a librarian is ridiculous.

And I’m not sure I’d be able to say that with such confidence if we hadn’t tried to do it – and to do it as right as we could.  To do it in an authentic, meaningful way that could be built upon in later courses when they start doing real work in a discipline.  To do it in a way where self-directed learning experiences are connected to group activities, reflective activities, hands-on exploration, and feedback.  To do it with teaching librarians who teach this course every term, who participate in professional development activities and who understand the students and the learning goals.

So today, I taught the course as we tweaked it over winter break – to de-emphasize the scholarly source.  It’s still a requirement, but we don’t focus on it any more than we focus on news or book sources.  We talk about it as “a way to find out what the research is” and not more than that.  The session focus has returned to exploration and thinking about the topic — which lets us tie everything to their assignment (which is itself interesting, and something I should probably give its own post):  focused on reflection and analysis and presenting themselves as academic writers.

I’m not particularly happy with it at this point – it was a tweak, not a fix.  We haven’t figured out cool ways to teach this stuff.  But still, the pressure and stress of the session was gone, as was the sense that the students left feeling less like “I can do this.”  What to teach – we’re a lot closer there.

If timing is everything

then Zotero’s standalone beta isn’t worth mentioning.  I’m in the throes of course revisions for the class I’ve been building around Zotero and I am not even sure what the final project is going to be this year (more on that later) so do I really have time to decide whether I want to teach my students to use the standalone or stick with original flavor?

It doesn’t feel like I do, that’s for sure.  But like Mark Sample at Profhacker said today, I’ve been working with it for a couple of days and it is working really well – stable, easy and not in Firefox.  Plus also, he’s right about the standalone having a better icon.

So, which to teach?  I think I’m coming down on the side of the standalone.   I don’t have very many 19 year olds browser zealots in my classes, but those I do aren’t Firefox devotees.  There are almost always 1-2 who want to use Chrome or Safari.  And since none of my students (if past experience is any guide) will have existing Zotero libraries to consider, or existing Zotero workflows to un-learn, I think we might just work with the standalone.

And yes, that means building in time to re-do some previous work.

So, why am I changing the final assignment? 

Well, I have some reasons.  (The following is heavily cribbed from an assessment report I sent to the chair of the department & thanks to her for sparking me to think about and write it)

  • One is logistical – the faculty of the School of Writing, Literature and Film (formerly known as the Department of English) at OSU is not quite big enough to support individual projects for all 40-50 students. Not to mention that a number of faculty members are very busy working on the transition to the new model. At the same time, I don’t want to overburden individual  faculty members which precludes me from letting students to choose their own faculty member to focus on.  This means that the challenges students face with the assignment are very different depending on the faculty member they draw, and their learning is sometimes affected more by their topic than by their own motivation or effort.
  • The second reason is more important.  One of my students pointed out in the course evaluation that the process I was teaching — asking students to search comprehensively on a topic (to find everything their faculty member has published) before they evaluate and decide which sources to include on the final bibliography — doesn’t reflect what they need to do for almost all of the research that they will ever do.  There are only a few contexts where people are asked to search in this manner (the literature review for a dissertation would be one example) which meant that this assignment was emphasizing the wrong skills.

I should say that I think one reason that student was able to make such an insightful observation was that I was more successful communicating the process aspects of what I was trying to do this term — but that fact, that I want to provide students with a way to reflect on research and writing as intertwined processes – is exactly why I need to change to something that will be more authentic for them.

I need to shift to topics that will allow them to follow a more exploratory process, but that’s not the part I am struggling with.  I am struggling with – what do I want this final project to look like?  The person who taught this course before me had the students do research to create a “critical edition” of a favorite novel.  I was in on some of the early brainstorms about that assignment, and I think it worked out well for her.  So I am thinking of returning to that – maybe have the final project be an introduction to their “critical edition” where they analyze and cite the sources they want to include?  We’ll see.

After all, this is a process too.

Previous posts on this topic:

Zotero Group Bibliography Assignment (10/2010)

Zotero Assignment Update (11/2010)

Zotero Assignment Revisions (12/2010)

Zotero assignment revisions

So, in the end the Zotero assignment worked very well on the Zotero side, and less well on the information literacy side.  So I’m spending this week revising it and designing some new activities.  A few quick takeaways:

The assignment was trying to do too much.  It was the main way to assess:

  • Students’ ability to recognize different source types and explain where the fit into the scholarly process.
  • Students’ ability to track down those different source types.
  • Students’ understanding of what the scholarly and creative output of their department (and by extension the scope of intellectual activity within their discipline).
  • Students’ ability to use research tools to organize and manage their sources.

Way too much.  Illustrated mainly by the fact that there were a few students to managed to do all of those things in their work.  That made it very clear what others were missing and made me want to figure out a way for all students to be able to get to where the few did in this class.

So here’s the thing – the first two outcomes up there were the problem, not the technology or logistics of syncing libraries and the like.  The bibliography project should really be about the 3rd and 4th outcomes.  The collaborative nature of the bibliography (and ability to see the breadth of what our faculty produces) was lost on students who had to work to hard to meet all of the format requirements that were in place to measure the first two outcomes.  All of the format requirements I put in to meet the first two outcomes took away from the authenticity of the experience, and of the evaluation and contextualization I had hoped the students would be able to do.

So this term, I am planning to get at those first two outcomes in different ways, and then make some changes to the bibliography assignment:

  1. drop the number of sources required in the annotated bibliography from 5 to 3.
  2. increase the emphasis on evaluation (and multiple methods of evaluation) in the annotations.
  3. change the workflow a bit – have students create a broad, pre-evaluated body of resources in a personal library and then have them select their 3 sources from that larger pool, annotate them and add them to the collaborative bibliography.
  4. build in a required conference so that I talk directly to each student about the process fairly early on.
  5. drop the format requirements altogether and allow students to add any 3 resources they want (while increasing their responsibility to justify those choices in multiple ways in their annotations).
  6. push the due date for the sources up a week, add a week between the final sources due date and the final reflection due date, and target and focus the scope final reflection essay significantly.

(Big hat tip to my students.  Many of these changes were also articulated by them when I asked them to help – in some cases their input was what really allowed me to put my finger on the problems).

What about the tech?

In the end, syncing did cause problems for a few, and Zotero hurdles did cause problems for a few.  Students who were, for whatever reason, not able to spend a focused amount of time at some point earlier in the term learning the mechanics of Zotero found it very challenging to manage finding sources and figuring out Zotero in the context of a last-minute scramble.

I had thought that my students would have to do the bulk of their Zotero work at home because of having to re-download and sync Zotero every time in the classroom.  MY Zotero library was still very difficult to sync in the classroom (I assume the hugeness is a factor) but the students rarely had to wait for more than 2-3 minutes.  Clearly, I can and should rely a lot more on classroom time as a place where students can be working with Zotero.

Most students were very positive about Zotero.  A few found it cumbersome.  There was a clear pattern though that I found interesting, but troubling in that there is nothing I can do with it.  The pattern was this — those students who had reason to use Zotero for real, for a real research project, during the term were much, much clearer in their evaluation of its value.  And by extension, I believe that they are the ones most likely to keep using it.

My class is a 1-credit class.  I can’t assign an authentic student-y scholarly research project that would take that little work.  But whether or not they have reason to use it in another class is nothing I can control.  It’s troubling because it points to a deeper issue about this class’ place within the major – issues we all know about but aren’t sure how to fix.

Yes, we did write that up

Finally!

Kate and I finally got an article related to our LOEX of the West presentation (from 2008!) finished and published.  This peer-reviewed article delay had nothing to do with publishing cycles and everything to do with writing process.  But it’s available (in pre-print) now, and I pretty much like it.

Beyond Peer-Reviewed Articles: Using Blogs to Enrich Students’ Understanding of Scholarly Work

Critical Literacy for Research – Sort of Peer-Reviewed Friday

Unexpectedly it’s Peer Reviewed Friday.  Well sort of.  Harvard Educational Review is a student-run journal, with an editorial board made up of graduate students deciding which articles get published.

I was teaching a class in our small classroom – where I never teach – so I went up early to make sure that I still knew how to work the tech.  It’s on the 5th floor, where the L’s are shelved, so I was flipping through the Fall 2009 issue while I waited for them to show up.  This article caught my eye — well worth reading, both for the content/ideas and because it is very enjoyably written.

Harouni, Houman (Fall 2009). High School Research and Critical Literacy: Social Studies with and Despite Wikipedia. Harvard Educational Review, 79:3. 473-493.

It’s a reflective, case-study type description of the author’s experiences reworking his research assignments in high school social studies classes. There’s a ton here to talk about – the specific exercises he developed and describes, the way the piece works as an example of critical reflective practice — but mainly I want to unpack this bit, which I think is the central theme of the work:

If students do not engage in the process of research inside the classroom, then it is natural for them to view the assignment in a results-oriented manner — the only manifestation of their work being their final paper and presentation.  It is not surprising then, that they are willing to quickly accept the most easily accessible and seemingly accurate information that satisfies the assignment and spares them the anxiety of questioning their data.  And when their final products did not meet my expectations, the students responded not by rethinking the research process itself but by simply attempting to adjust the product in light of what they perceived to be personal preferences. (476-77)

(emphasis mine)

Basically, the narrative he lays out says that his research projects had been unsuccessful for a while, but it wasn’t until he noticed his students’ heavy and consistent reliance on Wikipedia as a source that he started digging into why, what that meant, what he really wanted to teach, and what he really wanted students to learn.  And he changed stuff based on those reflections.

Harouni’s thinking about information literacy (which he calls “critical literacy for research”) was initially sparked by students who were not evaluating sources or showing any sign of curiosity as they researched, but it was further sparked when his first attempts at addressing student gaps didn’t work, sparked by students who were trying, and failing, to evaluate texts they weren’t yet ready to evaluate.

Along the way, he talks about the limitations of a checklist, or “algorithmic” approach to evaluation — limitation he discovered when he reflected on what his students actually did when he tried to use that approach in his classroom:

Two observations confirmed the shallowness of the learning experience created through the exercise: first, the students did not apply their learning unless I asked them to do so; second, they remained dependent on the list of rules and questions to guide their inquiry. (480)

In other words, they could do the thing he asked them to do (apply the checklist to information sources) but it didn’t affect their actual practice as researchers, nor did it change how they viewed the information they were getting from Wikipedia.

And also why it is important to help students understand the openness and dynamism of Wikipedia, but that that itself is not enough:  “knowledge of the uncertainties of a source does not automatically translate into an awareness of one’s relationship with the information (477).”

This piece is, I think, essential at getting at what I think is the real value of his insights and experience — many of our students want to find certainty in their research processes.  They want to know that a source is good or bad.  Wikipedia bans feed that.  Checklists feed that too, especially when they are not taught as an initial step in an evaluation process, but as the process itself.  What we really want students to be able to do when they research is to manage uncertainty — to say I know this is uncertain and I can figure out what it means for me as I try to answer my real, important, and complex question.

Harouni’s process his is an excellent reminder of how teachers want clarity too – and how they have to be willing to embrace uncertainty themselves if they are to guide students through a process of authentic inquiry:

In teaching critical literacy for research, I have had to separate research from its dry, academic context and consider it as an everyday practice of becoming informed about issues that have an impact on students’ lives.  I must value not answers but instead questions that represent the continued renewal of the search.  I must value uncertainty and admit complexity in the study of all things. (490)

In this, he knocks on the door of a question that I frequently have as an instruction librarian (one which I think many instruction librarians have — how much can I really accomplish as a teacher on my own).  If the classroom instructor – the person creates, assigns, explains, and evaluates the research assignment isn’t actively engaged with the students’ research process – are there limits to what I can do?  I do think there are.  I don’t think those limits means that I should do nothing, far from it – but I do think those limits affect what I think I should be trying to accomplish on my own and affect the other ways I should be thinking about furthering my goals for students, inquiry and learning.

At the end of the day, one of Harouni’s basic assumptions is that it is part of his job as a social studies teacher to foster inquiry and curiosity in his students, “[f]or two semesters, research projects remained a part of my curriculum — not because they were wonderful learning experiences, but because I could not justify, to myself, a social studies class that did not work to improve the way students navigated the ocean of available information (474-5).”  In other words, he believes that teaching information literacy is an essential part of what he does.   And that is key.  You can’t have that perspective and also value coverage – of content information – above all else.  It’s one or the other.  (is it?  Yeah, I think it is).

Every faculty member isn’t going to have that idea of what their job is.  And every librarian isn’t either – but I think maybe for instruction librarian it should be.  It is true that rules and clarity make coverage easier.  There was a question on ILI-L yesterday from someone (responding to an ongoing discussion about teaching web evaluation)  asking “how do you even have time to talk about web evaluation when you have to cover all this other stuff.”

Rules make it easier to “cover” web evaluation.  Faculty want us to “cover” lots of different tools.  WE want to “cover” lots of different tools.

(N.B. I am not suggesting that everyone who engaged in the “web evaluation” discussion just “covers” it and doesn’t teach it.  Nor am I suggesting that the people who worry about covering what the faculty want them to cover are only interested in coverage.  I do think though that the pressure to “cover” is as true for us as it is for people in the disciplines and these discussions spark reminders of that)

But if we want students to think about research as a process, if we want research to BE a learning process, then we have to engage in teaching the process.  And that’s extra hard for us – we can’t do that in the one-shot by ourselves.  And we can’t do it if we’re worried about coverage — about covering everything the library has to offer.  And I’m not just saying that about “we can’t teach everything about the library in a one-shot” — I think we all know that.  I think I am saying that it can’t be about that at all – that the point has to be about the process, about authenticity, about this –

I now understand that whatever research strategies students use in their day-to-day lives, which no doubt will vary depending on who the learners are, must be investigated and taken into account by their teacher.  Neither this goal nor the goal of improving these strategies can be attained unless students have time to engage in research while they are in the classroom.  And inviting students to the computer lab and remaining attentive to their interaction with online sources is as important as accompanying students to the library. (490)

And maybe this means not worrying about teaching research as a recursive learning process in the one-shot.  Maybe this means rethinking what and where we teach and maybe it’s work with faculty that gets at that overarching goal.  I don’t know.  I do know, though, that I have some great ideas for rethinking my credit class next term.

Classroom activities to promote critical literacy for research:

1. A (relatively innocuous) vandalism example demonstrated in class.  He didn’t change the content of pages, just the accompanying photo to illustrate the process of editing.

2. Students work in pairs to evaluate a Wikipedia article on a topic they know a lot about (for example, one student used the article about her former high school). Through this exercise he was able to teach about:  skepticism & its place in the research process, identifying controversial claims in a text, citations and footnotes, and verifying claims by checking outside sources.

3. Judging a book by its first sentence. He brought in 5 history textbooks, showed the covers and provided the first sentence.  Then he asked students to describe what they could figure out about the book from that first sentence.  With this exercise he was able to teach: authorial bias or point of view; finding the author’s voice.

4. Research beyond the first sentence.  When they tried to apply these critical skills to the texts they found in their research projects, though, they still had trouble because they didn’t know enough about the stuff they were researching.  So he looked for a way through this problem. Enter Wikipedia.  He provided a list of pages identified by Wikipedia editors as biased or lacking a neutral point of view, and asked the students to choose an article on a somewhat familiar topic and write a brief essay, with specific references to the text, with suggestions for improving the piece to meet the Wikipedia’s neutrality standard.

5. Contributing as an author.  Similar to other projects like this, it was one option for his students as a final project.  Interesting in that he collaboratively developed the assignment and rubric with interested students.

Zotero assignment update

So the first mini-deadline on the Zotero assignment has come and gone, and I’m pretty happy with the results so far.  They’re not very impressive to look at, but when you compare what is actually happening with what I thought could happen, I think we are well on our way to getting this done.

For the first section, which has 21 students:

  • 11 successfully added a scholarly source to their Zotero library AND successfully synced to the group library.  Another one got the sync to work, but what got saved isn’t in very good shape yet.  Three more are waiting on ILL to decide which article they want to save to the bibliography.
  • Of those 11, 6 have added an original annotation and tags.

There are a few who added something in another format (and I’m not sure if that is a result of still not knowing how to find a scholarly article for their person, or if it is a matter of the best sources authored by their person not being scholarly articles)  I’ll find out more about that in class this week.

In the second, which has 24 students registered:

  • 13 successfully added a source to their Zotero library AND successfully synced.  Another one did the sync okay, but what got added was wonky.  There is one person who has added two things.  There is also an example article that I added still in there.
  • And there is a weird article from the medical literature that is still mysterious.  The author doesn’t share a last name with one of our target authors, so I am thinking maybe it was left in one of the classroom computers’ Zotero libraries and accidentally got dragged into our group library?
  • Nine have added original annotations.
  • Another handful are waiting on their articles from ILL.

Most of these have wonky notes/ attachments from the databases, and some need some of their metadata cleared up.  Batting 500+, though, was more than I expected at this point.  Why?  A few reasons, actually –

  1. First, these students have never used Zotero before at all.  Most of them have never used any kind of Firefox plugin.  That whole process of downloading and installing Firefox, then the plugin, was conceptually something new.  I expected this to be a hurdle in and of itself, before we even got to the the group library and syncing piece of the puzzle.  And it was, for sure, for some.  But not for most – most got themselves set up with Firefox no problem, and got the plug in working just fine.
  2. I want to be really clear here – it’s not that I thought these students weren’t intelligent enough to do this nor did I think it was really hard – I just thought it was going to be new and made more difficult by the fact that I asked them to do most of this new thing on their own on their own computers.  I did this mostly because I wasn’t at all certain that syncing the classroom computers to the Zotero group library would work with any kind of reliability.  So it comes down to –  I thought that showing them in class and then asking them to do the work at home was not necessarily setting them up for success (for all that that is how homework usually works).
  3. I really didn’t give them much instruction on how to do this at all.  We went over Zotero on the first day of class, and then I asked them to test different features of it along the way.  But here’s the thing – most of them didn’t do that along the way stuff because I wasn’t grading it and it wasn’t on the syllabus.  It was mostly a “please do this for your own good” thing and wasn’t at the top of anybody’s priority list.  So that .500+ batting average comes from students figuring stuff out with the tutorial I provided and what they could find in their notes and on the Zotero website.
  4. Some of the problems that have happened are undoubtedly not about Zotero at all, but are about navigating library systems and databases and the difficulties that come up during the process of finding scholarly articles — those are the primary reason for this class, after all!
  5. The syncing with the classroom computers is working really well – or at least it has for the last two sessions.  I have to tell you that I was worried about this with good reason.  Every time I have attempted to show this in the classroom, the sync has churned and churned and churned without any end (or any sync) in sight.  So when the students were having no trouble syncing the Zotero libraries in the classroom to their group accounts in class two weeks ago and again last week, I was shocked.  But what this means is that this week we can treat the classroom like a lab and troubleshoot most of the remaining problems together.

Onward!