And now, on to Peer-Reviewed Monday, part two but still not Monday.
Mesmer-Magnus, J., & Viswesvaran, C. (2010). The role of pre-training interventions in learning: A meta-analysis and integrative review☆ Human Resource Management Review, 20 (4), 261-282 DOI: 10.1016/j.hrmr.2010.05.001
As I said earlier this week, this was started by a link to this article, a meta-analysis trying to dig deeper into the questions: which of the pre-practice interventions examined in the Cannon-Bowers, et al study are most effective? For what type of learning outcomes? And under what conditions?
The first part of the paper reviews what each of the pre-training interventions are, and presents hypotheses about what the research will reveal about their effectiveness.
METHOD
They reviewed 159 studies, reported in 128 manuscripts. For this work, they considered only studies that met all of the following conditions:
- they involved the administration of a pre-training intervention
- the study population included adult learners
- the intervention was part of a training program
- the study measured at least one learning outcome
- the study provided enough information to compute effect sizes.
The studies were coded for: the type of pre-practice intervention; the type of learning outcome; the method of training delivery; and the content of the training.
The codes for pre-practice intervention were drawn from Cannon-Bowers, et al: attentional advice, metacognitive strategies, advance organizers, goal orientation, and preparatory information.
The codes for learning outcomes were drawn from the Kraiger, et al (1993) taxonomy:
- Cognitive learning (can be assessed at 3 stages: verbal knowledge, knowledge organization and cognitive strategies)
- Skill-based learning (also assessed at 3 stages: skill acquisition, skill complication, and skill automaticity)
- Affective learning (attitudinal outcomes, self-efficacy outcomes and disposition outcomes)
Training methods coded were very relevant to information literacy instruction: traditional classroom; self-directed or distance learning or simulations, such as role-plays or virtual reality.
Training content was coded as: intellectual, interpersonal, task-related or attitude.
RESULTS & DISCUSSION — so, what does the research say:
For attentional advice — this was one that I was able to immediately think of one-shot related applications for, so it was particularly interesting to me that medium to large positive effects were found for both skill-based and cognitive outcomes, with the largest gains found for skill-based outcomes — given that so much of what is taught in one-shots is skill-based, intended to promote success on particular assignments. These effects are strongest when general, not specific, advice is given.
Metacognitive strategies —
The authors identified two main forms of meta-cognitive strategies that were studied: strategies that involved the learner asking why questions, and strategies where the learner was prompted to think aloud during learning activities.
The research shows that meta-cognitive strategies seem to promote all levels of cognitive and skill-based learning. Why-based strategies had more consistent effects for all levels of cognitive learning, which supports the authors’ initial hypothesis — but think-aloud strategies do a better job of supporting skill-based outcomes, which does not.
Advance organizers —
Positive results were found for these for both cognitive and skill-based outcomes. Of particular note for instruction librarians is this finding: “stronger results were found for graphic organizers than text-based ones across all levels of skill-based outcomes.”
Goal orientation —
When compared with situations were no overt goal was provided to the learners, goal orientations seem to support all types of learning: cognitive, skill-based and affective, with the strongest effects (just by a little bit) in the affective domain.
The authors also hypothesized that mastery goals would be better than performance goals. The findings suggest this hypothesis is true for skill-based learning and for affective learning. They were not able to test it for cognitive learning. They did find something odd with regards to affective learning – when they compared performance goals and mastery goals separately against no-goal situations, then performance goals showed greater effects. But when they compared mastery goals and performance goals, stronger effects were found for mastery goals.
Preparatory information —
This showed positive effects for skill-based and affective learning, but they weren’t able to test it for cognitive learning outcomes.
SO WHAT ELSE COULD HAVE AN EFFECT?
The training conditions and content were coded to see if those things had an effect on which pre-practice interventions were most effective. Of particular interest to me were the finding that stronger effects for cognitive learning were found for advance organizers paired with self-directed training (e.g. tutorials) than for traditional classrooms or simulations. (Of course, it’s important to remember that those showed positive effects too).
RESULTS BY TYPE OF OUTCOME
This turned out to be the most interesting way to think about it for me, so I’m going to include all of these probably at a certain level of length…
For skill-based outcomes, broken down – the strategies that work best seem to be:
- skill acquisition: mastery goals & graphic advance organizers.
- skill compilation: think-aloud meta-cognitive strategies, attentional advice and goals.
- skill automaticity: graphic organizers and pre-training goals.
This seems to suggest pretty strongly that librarians should find a way to communicate goals to students prior to the one-shot. Obviously, the best way to do this would probably be via the classroom faculty member, which is why this also makes me think about the implicit message in the goals we do send to students – most specifically, I mean the implicit message sent by requirements like “find one of these, two of these, three of these and use them in your paper. It does seem like this could be considered a performance goal more than a mastery goal and even if the main impact on students is added stress to perform, is that stress that is serving any purpose or should it be eliminated?
For cognitive outcomes, also broken down – these strategies emerged from the literature:
- verbal knowledge: specific attentional advice, why-based meta-cognitive strategies, and graphic advance organizers had the largest effect.
- knowledge organization: general attentional advice and think-aloud metacognitive strategies
- development of cognitive strategies: why-based strategies and attentional advice.
This is interesting, of course, because while we know that teaching on this cognitive-outcome level is pretty hard in 50 minutes, a lot of the topics we’re asked to address in the one shot are really asking students to perform in that domain. Ideas like information ethics, intellectual honestly, scholarly communication, identifying a good research article – these all require more than a set of skills, but also require a way of thinking. So in this area, I am thinking okay, we can’t teach this in 50 minutes, but if we can prep them in advance, maybe we have a better chance of getting to something meaningful in that time.
For affective outcomes —
- Overall, a pre-training goal orientation and attentional advice were most effective in this domain.
These might not seem relevant in the one-shot, but they really are. We’re talking in many cases about teaching them something with the hope that they’ll use it later, when they really get to that stage of their research process, their confidence and self-efficacy is clearly relevant, as is their disposition to believe that you’re teaching them something valuable! In fact, I think this might be as worth or more worth focusing on that cognitive outcomes. So that makes these findings particularly interesting:
- post training self-efficacy AND disposition toward future use of the training material were most influence when a performance goal orientation was used.
- Attentional advice, mastery goals and preparatory information are also promising here.
info-fetishist 