The Active Class

Since my last post on the Flipped Classroom, I’ve stumbled across enough particularly good resources on a similar topic to merit a follow-up post.  The idea behind the Flipped Classroom is that classtime is better spent in helping students to apply ideas (e.g., working problems, doing labs, or in other words making sense of the content) rather than in the traditional lecture content-delivery mode.  So, students watch mini recorded lectures at home to get the content, and then spend class time applying the ideas, with the teacher as a coach.  You can see more about this technique on the previous posts, or at Learning4Mastery. In this post, I will talk about ways to help students use pre-lecture time to adequately prepare for class — whether you’re using a flipped classroom model or not — and the research behind some of those techniques.

Have student reading habits changed?

It’s a common complaint:  Students don’t read the book before class.  It’s probably equally true in the humanities, but my main experience is in the sciences.  Science textbooks are dense, full of extraneous diagrams and pictures, and it’s a real challenge for an introductory student to muddle their way through all that information to try to extract useful information from it.  So most don’t bother — they go to class to see what content the professor thinks is important, and then use the textbook to help them to do the homework and guide their exam studying.  But this constrains us to use class in content-delivery mode: If students don’t know the first thing about Newton’s Laws, then how can we do anything in class other than tell them about Newton’s Laws?

Do they read?

Some of my colleagues at CU Boulder studied how students use textbooks in introductory physics (Podolefsky and Finkelstein, “The Perceived Value of Textbooks: Students and instructors may not see eye to eye.” The Physics Teacher, 44, 6, 2006),     Noah Podolefsky, one of the study’s authors, summarizes it below (as quoted from a physics teacher listserv):

In a nutshell, what we found was that few students read the book before class, more student did read (but still not many). However, there was no correlation between reading habits and final grade.  We interviewed students and basically found that they had different strategies for
reading – some read straight through, some read in a non-linear way (going back and forth), some just read the summary. Some students didn’t use the book because they scoured the internet for resources that made more sense to them. We couldn’t find any consistent patterns that related reading habits to performance in the class.

From their data, he claims, it was not clear that encouraging students to read the textbook would have actually helped them.  There’s no correlation between reading the book and doing well in the class, and students are reading in so many different ways that it’s hard to say anymore what “reading” means.  Noah postulates:

I can speculate as to why textbooks are not read, and perhaps not that useful, which is that they aren’t very well designed tools for learning physics. They’re pretty good for re-learning physics if you already have a good framework (i.e., you’re an upper level physics major or grad). But they don’t match very well how new comers (intro students) learn.

I think that this is why we, as academics, get so frustrated when students don’t read.  This is how we learn a new topic — by reading a journal article or a book.  But we’re experts, and we can make sense of the information as it’s presented in the book.  But students are novices, and need more guidance.

So how can we provide that guidance?

One thing that some faculty have tried are multimedia modules to help guide your students in their pre-class preparation.  For example, the University of Illinois has created a suite of multimedia modules, about 10 minutes long, which each guide students through some of the main ideas in the text for a particular topic.   Students watch the videos before class, and take a short quiz on their content to encourage participation. Below are a few such resources that are available online, but please let me know of others that you’re aware of:

1. Physics: Multimedia Modules; 20-minute lessons with pictures and audio from the University of Illinois Urbana-Champagne.  Published work on effectiveness here.
2. Various science: Hippocampus. Short lessons on various topics from the Monterey Institute of Technology, including some recommended ones on physics.
3. Chemistry & Physics: Georgia Public Broadcasting.  Videos on science, recommended by a high school teacher.
4. Chemical Engineering: LearnChMe screencasts from CU-Boulder. A richly developed suite of materials on engineering topics.
5. Various science: Learning4Mastery website by Bergman and Sams covers high school chemistry, physics, earth science, astronomy, calculus, and biology, though their style is idiosyncratic and less easily incorporated into another class

text

Some other ways to guide students in their preparation are:

1. Skip the multimedia part and just record your own lectures (using, for example, Panopto).  See some examples of this in physics here and here.
2. Use pre-existing lectures such as MIT Open Courseware, or other lectures available on iTunesU.
3. Ask students a pre-lecture quiz, to encourage and guide the reading, or simply ask them what was confusing or what they don’t understand.  This can also guide you as the instructor as to what students are struggling with.  This is called Just in Time Teaching, or JiTT. The quiz can be multiple choice and graded in your course management system to reduce grading burden.
4. Have students write a brief summary of the reading, and a question that they have about the reading.

text

The research.

Of course, the big question is, does this stuff work? The answer is probably, as always, “it depends.”  There are few, if any, plug-and-play solutions in education.  How an instructor uses these resources, and coordinates them with the class time, is essential.  That said, here are the results of a few studies.

The UIUC multimedia modules have been studied for several years.  One way to look at the effectiveness is to look at a particular topic, and show students either the multimedia modules, or let them read the traditional textbook.  When they did this (Am. J. Phys, 2009), students did better on a subsequent test on their learning of that topic than with the text-based presentation alone.  That’s not too surprising, since using multiple modes of presentation is typically better than only one mode.  The UIUC folks have also used the multimedia modules in several courses — students watch the modules, and then take a short quiz on their understanding before class.  In another publication (Phys. Rev. ST, 2010), they found that students overall performed better on these “preflight” questions than did students in traditional lectures.  However, they have also reported that students don’t do much better, if at all, on course exams (Am. J. Phys, 2010).  Their interpretation of these results are that students are masters of efficiency.  If they’re aiming for a “B”,  then they’re going to get that “B” with as little work as possible.  So, by guiding students, the modules might have helped them to be more efficient in their studying practices.

Another study in Biology (Lents and Cifuentes, Web-based learning enhancements, J. College Sci. Teach., Nov/Dec 2009), some lecture attendance was replaced with video lectures that consisted of the visual of a powerpoint slide presentation enhanced with audio voiceover.  They found no effect (negative or positive) on student learning from this substitution.  While these authors were aiming to reduce student time-on-task for their largely commuter college, this does suggest the next step — having students engage in video-based learning at home and using lecture time for additional engagement — could be beneficial.

So, it certainly doesn’t seem to hurt to add some sort of pre-class preparation, and if you find some way to guide your students through the topic in a way that is more suited to novice learners than a dense textbook — it could help free up some of your class time to do more in-depth learning.

Assessment is a stressful subject — for teachers and students alike.  How do you get through grading all those tests?  How do you know if your test really measured your students’ learning?  How do you know if they got it?

I recently attended a talk by Andy Rundquist of Hamline University where he introduced the audience to the idea of standards-based assessment — and showed us how he’s using some innovative technology to measure his students’ achievement of those standards.  In standards-based assessment, you don’t just take the students’ overall average on several tests and midterms to get their final grade — rather, you test to see if they’ve achieved mastery of some of the core concepts that you have set as the “standards” of the course.  Typically those standards take the form of a learning goal or learning objective.  Can they derive the centripetal force on a planet rotating around the sun?  Can they write a computer program to add together simple numbers?  Etc.

But the trick is that, now, how do you know if a student has achieved mastery of that standard?  A typical paper test doesn’t seem as useful anymore.  That tests whether they can fill out the test, but not necessarily true mastery.  Hence, I think that Andy’s technological fix can be potentially useful to many of us instructors.

What he did was to use pencasts and screencasts to see whether students had achieved mastery of the goal.  This way, he could watch the student, and hear their thinking, and get a very complete picture of just how well they grasped the material.  Think of it as an oral exam, but one that the teacher can conduct from the comfort of their own home, and one that causes the student a bit less stress (and allows him/her to think about, and correct, their mistakes more easily).

Pencasts

These were done using the Livescribe pen and software.  The Livescribe pen records audio as it keeps track of where the pen is and what it’s doing.  The result?  A play-by-play audio recording with accompanying digital transcription of the student work.  We use these in research, to watch student thinking, and we use them in notetaking so that we have complete audio to go along with our rough notes.  But I’d never thought of using them as an assessment tool.  Andy pointed out that he can jump around to the “tricky bits” of a problem as he watches, so he can see whether they properly explain what they do at that point.  He can see the entire worked problem, too, so can see at a glance if it’s correct — but what he’s really listening for is whether they can explain why they did what they did.  I love it.

Screencasts

A screencast is just what you’d think — you can watch the student’s screen as they talk.  This is most important if you’re assessing them on something that’s not written, such as a computer program.  He used Jing, but there are a variety of tools out there.  Just like the pencasts, he was listening for student reasoning and thinking.

Of course, if you have a large lecture, this is harder to do.  He conjectures that you might be able to have students engage in peer assessment — in fact, during the oral exams that accompanied this approach, he had the class discuss the merits and shortfalls of each student’s work, coming to a consensus on what grade the student deserved.  He managed to frame this as a communal, not competitive, activity.  So, with the right framing and clear rubrics, this could work — and could really benefit students too!

Homework is a key learning opportunity for students — it’s where they spend most of their time on your course out of class, and it’s typically the only place where they spend time on their own, puzzling out the ideas presented in lecture.

So, how do you create homework that helps bring students to a deeper understanding of the material, targeting their specific needs?

Some techniques are pedagogical — such as Just in Time Teaching, where you frequently quiz students on their understanding of the topics, adjusting your instruction and gaining deep insight into their common difficulties.  This can help you properly target the homework to the class.

But some solutions could be technological.  I’ve been really excited about ALEKS (Assessment and LEarning in Knowledge Spaces) since one of our chemistry instructors enthusiastically told me how much she likes it.  ALEKS provides individualized learning through “adaptive questioning” — or, questions that change as you go along.  If you’ve taken the computerized GRE, you know what this is like — if the questions start getting really easy, you know you’re in trouble, because that means it’s trying to adapt the questions to your level.

But unlike the GRE, ALEKS isn’t trying to assess student understanding to assign a grade or a score — rather, ALEKS offers targeted instruction to the student on the topics that he/she is ready for.  For those learning theorists among you, those would be the topics in that student’s zone of proximal development. And what’s most interesting is that it doesn’t use multiple choice very much — it uses open-ended tools, such as input into graphs.  The teacher gets a report indicating the students’ aptitude in a variety of topics.

Here is a very nice outline of ALEKS, complete with screenshots.

ALEKS can help both with placement and with learning — in learning mode, the student gets practice problems and explanations.  Once the student has demonstrated mastery of the topic, then ALEKS moves on to new material.  It seems that this would be very appropriate to use with the standards based grading that I wrote about in my last post.

Ways I’ve seen ALEKS used:

• By institutions, to place students in the appropriate course
• By homeschoolers, as an instructional tool
• By students, as a tutor
• By instructors, for homework and formative assessment

They have a variety of course offerings, many in K12, but in higher ed they have many different products in math (e.g., pre-algebra, trigonometry, and various prep courses), business, statistics for the behavioral sciences, and science (mainly chemistry, plus math prep for college physics).  It’s not free — last I saw it cost $20 per student per month, though there are some bulk discounts.  Though, as ALEKS points out, it’s cheaper than a human tutor, and does provide individualized feedback.  I’m particularly  happy to see that it’s research based, though I admit I’m not familiar with the theory that supports it, and I don’t see information on whether it’s research tested (i.e., does it do what it purports to do) rather than just based on reasonable theory — though this article suggests that they are doing good work in that regard.

For those of you needing a free solution — there is Diagnoser, which isn’t quite the same, but offers research-based testing to help teachers determine their students difficulties and misconceptions and offer suggestions on addressing those difficulties in class.

Image by Mosborne01 on Wikimedia Commons.

We have our classroom spies.  And they have sent us their report in a forum that will probably not be seen by the students that they were observing: In a study released last week in the American Journal of Physics, two authors report on over 300 recorded student conversations during clicker questions within 3 introductory astronomy classrooms.  And the results are fascinating.

When we write clicker questions, we typically choose some topic or concept that we think will be difficult for students.  We devise answer choices to capture common pitfalls we think that they’ll fall into.  We expect them to read the question, and discuss amongst themselves — using critical thinking, logic and reasoning to decide between the different choices, until settling on the best choice.

James and Willoughby, however, found that this happened less than half the time (38%).  The rest of the time, students were doing one of the following

• Falling into pitfalls that didn’t lead to productive conversation (38%) – such as deferring to a confident student, ending the conversation before fully discussing the answer choices, not talking about the reasoning behind answer choices, or just not having a productive conversation to begin with.  Note that this was more of a problem in classrooms where students were heavily graded on getting the correct answer!
• Giving some answer that didn’t really represent their thinking (26%) -- like using another student’s answer (even when the student doesn’t agree), guessing, or looking for clues in the way the question was phrased
• Bringing up ideas that weren’t included in the existing answer choices (12%) – like gaps in their fundamental science knowledge, or bringing in irrelevant ideas

This is big news, and gives us insight into what is really happening during peer instruction in a way that we could only have guessed at.  To me, what this highlights is the need for two things during the implementation of a clicker question — things that I’ve always advocated for, but now I have some hard data to back me up:

1. The instructor should circulate and listen to student conversation. By listening in, the instructor can get some sense of what students are discussing, ask Socratic questions to spur productive conversation, and see how the question might be revised to more accurately capture student thinking.  If the instructor is alone in a large lecture room and can’t cover the whole room, consider using graduate TA’s or undergraduate learning assistants to help circulate, facilitate and listen.
2. Facilitate a whole-class debriefing conversation at the end of the peer discussion, and discuss the reasoning behind the right answer (and why the wrong answers are wrong) — even if there appears to be consensus. If nothing else, this study highlights that students very very very often give the right answer for the wrong (or confused) reasoning.  Having a discussion about the question, listening to multiple student reasoning, and clearly indicating why the instructor favors the right answer and rejects the others, is critical.
3. Provide credit for incentive, but not high-stakes. At Colorado we use Mazur’s suggested method, where we give participation credit for clicking in, but extra credit (which counts against the exam scores) for getting the right answer.

James and Willoughby offer some additional suggestions:

1. Encourage students to share their ideas that do not match the question answers listed, during whole-class discussion or via written feedback at the end of class
2. Add “none of the above” as a common answer choice
3. Ask students to rate their degree of confidence in their answer
4. Ask series of questions, each focusing on one link in a logical chain, to more clearly highlight where students are having difficulties
5. More clearly guide student interactions during clicker questions (e.g., assess all answer choices, generate your own answer choices if necessary, make note of questions and confusions, ask for help from other students and instructors if you don’t know how to start your conversation).

Hey, I just wanted readers to know about my two upcoming webinars on clicker use. One is on how to teach faculty how to use clickers effectively — basically, some tips on effective professional development. That’s coming up next Tuesday! The second is a repeat of an earlier webinar — how to use clickers effectively. Details below!

Teaching Faculty about Effective Clicker Use

Time: Tuesday, January 18, 1pm Eastern Standard Time. Register.
Note: Want the recording? You’ll get a download link after the session if you register.

Geared specifically for those involved in faculty development and support (e.g., instructional technologists, faculty excellence programs, or other faculty professional developers), this webinar will cover best practices in helping faculty to use clickers to enhance their teaching. The webinar presenter has been creating faculty professional development materials around clicker use for years, and will share tips and techniques — many based on research — for helping faculty to see the potential power of this technology and learn to implement it effectively. Webinar components will include: (1) best practices in clicker use, (2) resources available for faculty learning to use clickers, (3) research-based techniques for faculty development around clickers, and (4) working with faculty resistance and alleviating frustration. HIghly recommended: Watch “Make Clickers Work for You” webinar recording at http://theactiveclass.com/speaking-events/ prior to this webinar, and/or the video “How to use clickers effectively” at http://STEMvideos.colorado.edu.

Make Clickers Work for You

Wednesday, Feb 16th (1pm EST). Register.
In this interactive webinar, we’ll explore tips and ideas for incorporating clickers into your particular class. Clickers offer a powerful way to increasing student engagement and improve learning. At the University of Colorado, we have transformed our classrooms by using clickers to promote peer instruction. We’ll show research results on the most effective use of clickers, and discuss common challenges. In particular, we’ll focus on the attributes of “great” clicker questions, discuss example questions, and share ideas on facilitating effective wrap-up discussions once all the votes are in.

I am a Science Teaching Fellow at the University of Colorado at Boulder, and have been working with faculty and K12 instructors for the past several years on effective use of clickers. See my other workshops, and the videos we’ve produced on clickers in the classroom.

** I’m giving a free webinar, Tuesday November 9th at 1pm EDT. “Writing Great Clicker Questions” is geared toward college science faculty, but all are welcome!  **

I’ve been giving workshops for faculty and K-12 teachers on the effective use of clickers (personal response systems) for the past few years.  By “effective,” I mean using research-based techniques that help students learn.  Across the board, the studies show that clickers are best used to promote student discussion (peer instruction) — clickers can be a way to increase student engagement by getting students to wrestle with challenging questions.

The most enjoyable part of the workshop is when we discuss the challenges to using clickers this way.  Now that I’ve gotten a chance to stand on my soapbox and say how I’m suggesting clickers be used, I’m turning it back over to the teachers to bring up the realities of the classroom.  This is where we really get into the meat of the topic.

Over and over, the main thing that teachers bring up as a major challenge is “writing questions,” or “finding time to write questions.”  We’re not used to seeing challenging questions that are used to help students learn, rather than just to check to see where they’re at.  So, where do we start in writing them?

I’ve recently started to break down the question-writing process into three main areas:

1. Mechanics
2. Depth
3. Goals

Mechanics refers to the wording of the question.  Is it clear and jargon-free?  Are the distractors (the “wrong” answers) tempting?

Depth indicates whether the question requires high-level cognitive processes, or whether it’s relatively shallow.  I find Bloom’s Taxonomy to be a helpful rubric for considering the depth of a question, and the question stems and verbs associated with each level of Bloom’s are very useful in brainstorming questions.

Goals addresses the fact that there are various goals for clicker questions.  Are you surveying students?  Trying to figure out how much they know before your lesson?  Giving them a chance to apply a concept that was just discussed?  Testing what they learned at the end of a lecture?  I really like Ian Beatty’s take on this — and the below outline is taken from one of his short papers written for teachers (TEFA Note #2, Sept 2007).

I am interested to hear what others make of this kind of breakdown of different aspects of clicker questions.

One other item to note is that we’ve compiled some clicker question collections (mostly for STEM education at the college level) on our STEMclickers website.

Want more? Come to the free webinar, Tuesday November 9th at 1pm EDT. “Writing Great Clicker Questions” is geared to college science faculty, but all are welcome.

Derek Bruff (clicker guru extraordinaire) has shared some nice resources from a webinar that he did recently for the Center for the Integration of Research, Teaching, and Learning (CIRTL) CIRTLcast series.

You can see the whole 60-minute webinar at the archives, but the cool folks at CIRTL also edited it to a nice short 10-minute presentation on YouTube, embedded below.

What I really enjoyed about this presentation was how he tied the use of clickers to what we know about how people learn, such as giving them a chance to reflect on their own knowledge, or creating a “time for telling,” where students are eager and ready to hear an expert explanation, or reducing cognitive load. He really promotes the use of clickers for formative assessment, which is just what we’ve been promoting at the University of Colorado.

Sign up for my upcoming “Make Clickers Work for You” webinar to learn how to write great clicker questions and facilitative effective discussions.

The field of education is abuzz—and has been for a while—about formative assessment. You can’t go to a conference or pick up an educational journal without reading about how formative assessment promises to change education. Sometimes, when I’m listening to people talk about their formative assessment projects, I raise my hand to ask, “So once you get the results from your assessments, then what do you do?” This really is the million-dollar question and one that is not easy to answer. If there was an obvious next step, everyone would be taking it.

Assessment is formative if you do something with the results—not simply assign a grade, but change the course of your instruction (or, if you are a student, the course of your studying).  In other words, a meaningful formative assessment is not just an information-gathering exercise but an opportunity to identify the learning needs of the student and make adjustments to better meet those needs (Wiliam, 2006). And yet, while researchers have demonstrated that well-executed formative assessments can improve student achievement and understanding, practical constraints tend to limit the use of formative assessments in the classroom.

To begin with, assessment has not historically been linked to instruction, and teachers are not required as part of their education to know a great deal about it. What’s more, when teachers reach the classroom, there is little information available to help them implement a formative assessment plan. Finally, there is the question of the time, resources, and expertise needed to use formative assessment. Stiggins (2004) remarks that day-to-day and even minute-to-minute decisions have a great impact on learning, and that assessment in particular should be used almost constantly—to “diagnose student needs, allocate time, design and implement instructional interventions, judge student work, and assign grades” (p. 26). But this is a tall order, especially for teachers who have not been formally trained to use formative assessment.

Given these challenges, what can we do to start implementing good formative assessment practices? Targeted professional development for teachers is clearly one way we can increase teacher knowledge of good assessment practices. Ideally, administrators should create opportunities for instructors to learn more about good assessment practices and make sure they have the time, resources, and infrastructure to act on the information they receive. But we also need to consider how to actually manage the use of formative assessment in the classroom—and this is not always straightforward. Here are some points that might be useful as a starting point:
1.    Develop strategies for implementing and applying good assessment practices.
2.    Take action to understand how to use assessment in a formative way.
3.    Help students see the benefits of formative assessment. For example, show them how formative assessment reveals gaps in their understanding and helps them focus their study efforts.
4.    Learn to use assessment information diagnostically to determine the course of instruction and learning, and to deal with learning difficulties.

In the meantime, I welcome the input, comments, and questions of teachers or administrators who are currently in the trenches and thinking about formative assessment and how best to implement it in the classroom.

Stiggins, R. (2004). New assessment beliefs for a new school mission. Phi Delta Kappan, 86(1), 22-27.
Wiliam, D. (2006). Formative assessment: Getting the focus right. Educational Assessment, 11 (3 & 4), 283-289.