Browsed by
Category: Peer Instruction

Using IF-AT as Part of Exam Review

Using IF-AT as Part of Exam Review

blog if-at image

One of the highlights of the recent ICTCM conference was Eric Mazur’s keynote address about “Assessment For Learning.” He mentioned an assessment technique known as IF-AT (Immediate Feedback Assessment Technique) that reminded me a great deal of the review strategy I have been using in my intermediate algebra course. (Here’s a blog on that review strategy.) I thought it was a great idea to try, so immediately postponed my elementary algebra exams on systems of two linear equations in two unknowns by one day to give this a try. I added a second review day so I can introduce this strategy to my students over a two day period.

The idea is that students work on a series of problems individually. After half of the class period has ended, students submit their individual work and form groups of four students. They then discuss their answers as a team and submit a team answer to the first problem. If they get it right the first time they get full credit. If they get it wrong they can select a second answer to submit for 1/2 credit. They can even try it a third time if needed for 1/4 credit. I saw a video where the problems were in a  multiple choice format with 4 possible answers, and the teams were given a scratch off card. A star was displayed on the right answer – if students see the star they know they are right, if they see a blank space that counts as an incorrect attempt.

You can see a step-by-step demonstration of how the IF-AT works
on this web page posted by Epstein Educational Enterprises.

I am going to use a Learning Catalytics Team-Based Assessment to put my spin on this process. On day 1, students will work individually on 8 problems for 30 minutes. Some problems will be conceptual, some will be systems to solve, and there will be two word problems. They will submit their answers as they work. I will then launch the team portion of the assessment. Students will form their own groups of 4, with one person responsible for entering their team answers. I have decided to give teams only two attempts on each problem. A correct answer on the first attempt will receive full credit and a correct answer on the second attempt will receive half credit.

The individual portion will make up half of the score, with the team portion making up the other half of the score. I will be counting the score as an in-class activity in my flipped classroom model. I expect students to take a little time to get used to working with Learning Catalytics, so I have tried to select problems that they will be able to answer in the given time limit. I expect day 2 of the review will be smoother. My students work so well together, and I expect to see their bonding pay off in this review.

Later this week I will let you know how it goes.

Do you have any experience using IF-AT in the classroom? Do you use it for testing? I’m curious how you address students who have testing accommodations through the testing office. Let me know by leaving a comment, reaching me through the contact page on my website, or reaching out to me on Twitter (@georgewoodbury).


Learning Catalytics- #ICTCM17

Learning Catalytics- #ICTCM17

This Saturday I will be speaking at ICTCM about how I use Learning Catalytics in my Statistics and Algebra courses. 

Collecting Homework 

I started slowly in my Statistics courses, using Learning Catalytics to collect “written” homework. I often give written assignments to supplement MyStatLab exercises, and Learning Catalytics allows me to collect certain problems or parts of certain problems. The answers are automatically graded and scores are transferred to my grade book in MyStatLab. This strategy encourages students to do the homework and to be on time. Students, if you wish, can have conversations about their strategies or answers. As the results come in I can address common errors or misconceptions. 

Reviewing for Exams

I found Learning Catalytics to be helpful for reviewing for exams. For example, while reviewing for an inferential exam I can post a problem and ask students to tell me which hypothesis test is the appropriate one to use. The same can be done for reviews on probability distributions, descriptive statistics, … I can ask conceptual questions or problems requiring calculations. 

I can use these results to get a real time read on how my students are doing with their preparation, and determine which concepts to address in detail. 

Flipping the Classroom/Peer Instruction

Here is where the real classroom power lies. When I flipped my Statistics class, I used Learning Catalytics to make the class sessions more interactive and engaging. I post a question and ask students to submit an answer. Then I either ask students to explain their answers to the class, discuss their answers in small groups, or I offer some insights of my own. At that point I allow students to change their answers if they wish. 

This approach has turned my class into a conversation with my students, or a conversation among my students, which is more effective than the traditional “top down” lecture.  

If you have any questions or comments about Learning Catalytics, flipped classrooms, Interactive Statistics, or anything else in this blog, please leave me a comment or reach out to me on Twitter @georgewoodbury. 

ICTCM here I come …

Bootstrap – Matched Pairs

Bootstrap – Matched Pairs

This week I began with a bootstrap project for a paired-difference/matched-pairs scenario.

Download a pdf of the Project Here

One of my goals is to get students working with data they have collected, so I had students collect prices for 25 identical items at two stores. We used this for one of the investigations.

Investigation 1

A researched was investigating whether sons are taller than their fathers. My students were provided with 13 matched pairs. I had them find the difference for each pair (d = father’s height – son’s height). They had to determine whether to expect differences that were positive or differences that were negative. This is an important skill when setting up the alternative hypothesis, and I was happy with how my students understood what type of differences to expect.

We applied the bootstrap method to the sample differences, and the results are shown below.


Since the interval from the 2.5th percentile to the 97.5th percentile contained 0, we were unable to conclude that there is a difference between the heights of fathers and their sons. We had a great opportunity to discuss the implication of 0 being contained in the interval – and my students were able to understand that if a difference of 0 is in the interval then it is possible that there is no difference between the two groups.

Investigation 2

Here my students used their data from the two stores in an effort to support the claim that prices at Store A are lower than they are at Store B. We had some interesting results, including groups that reached 3 different conclusions when comparing Walmart and Target (Walmart is cheaper, Walmart is more expensive, no difference).

Follow Up

The following day in class I included a Learning Catalytics session where students were given various scenarios and intervals and asked for the appropriate conclusion. They proved that they retained their understanding, and several students displayed that by sharing their reasoning with the class after the question was closed for responses.

I am looking forward to the day where we cover the formal (p-value) hypothesis test for paired differences to see how well they understand the big picture.

Book Review: Teaching with Classroom Response Systems by Derek Bruff

Book Review: Teaching with Classroom Response Systems by Derek Bruff

Just finished reading this book by Derek Bruff (@derekbruff on Twitter), so I thought I’d share what I wrote on Goodreads. (By the way, if you’d like to be reading buddies, here’s my Goodreads profile page.)

Here goes …


Although some of the technology has really changed since this book was published, I would strongly recommend this book to anyone who plans to incorporate classroom response systems into their teaching. Bruff clearly lays out the pros and cons of different strategies of incorporation, grading schemes, question types, revealing correct answers, …

He does a great job introducing Peer Instruction, and I have been using that strategy in my classroom with great success. I feel that more of my students understand more of the material at this point of the semester than in previous semesters. (I am moving on to Mazur’s Peer Instruction book to go into greater depth on the strategy.)

I also love the concept of Agile Teaching. I love the uncertainty of not knowing which way the class will go, while remaining confident that I can adapt to what I am seeing from my students. I am teaching the same class back-to-back, and have had to focus on different topics with each class. It’s a lot of fun, and it fits in with my belief that we teach our students instead of teaching the material. I now walk into class each day thinking “I Am An Agile Teacher” and I feel so empowered. I should probably put that on a t-shirt.

Note: I am using Learning Catalytics as my classroom response system in my Intro Stats classes, and I am about to start using Plickers in my Elementary Algebra classes.

Learning Catalytics Questions

Learning Catalytics Questions

There is a growing pool of questions available inside Learning Catalytics. Some have been generated by the publisher, others have been generated by the community of instructors using Learning Catalytics. For my first Flipped Classroom I wrote my own questions (sample vs population, descriptive vs inferential, levels of data, …) and it was very easy to do. However, for day 2, I used 7 questions created by the publisher/author and the community of instructors and those questions were outstanding.

The feature that allows you to search for questions works really well. I chose the subject (Statistics > Introductory Statistics) and then refined my search by typing in the author’s name (Sullivan) or by entering tags (explanatory variable, designed experiment, …).

If you are using Learning Catalytics I would encourage you to share your questions by simply checking the box in each question that allows other instructors to copy them. I am sharing all of my questions – look for them under the community option by typing in my last name (Woodbury).

Day 2 of Flipped Classroom/Peer Instruction in Statistics

Day 2 of Flipped Classroom/Peer Instruction in Statistics

OK, this was a good day. No. It was a great day!

I cut down the review of the home content to 10 minutes, and was happy to see so many students participating in the student-driven review by offering their own explanations and definitions. Students know to come to class prepared and looking to participate. I can see where this could turn into a situation where only a handful of students participate though, and will seek to eliminate this pre-review and jump more quickly into the Peer Instruction portion of the class.

I also cut down the number of questions I asked by half. We did not feel so rushed, and it gave more time for deeper discussion. I guess one of my weaknesses is trying to cram too much into my class sessions (and exams too). I will constantly ask myself if I have too many questions, and if can I make due with fewer.

After making our way through a series of experiments where students had to determine the explanatory and response variables I stopped the class to make sure my students understood why I am using the approach. I asked them how confident they were about knowing the difference between an observational study and a designed experiment as well as how confident they were in their ability to identify explanatory and response variables. They were highly confident. Then I asked them if they would have felt as confident learning the material in the opposite order – with me giving the definitions and some examples followed by them cementing their understanding working at home. I saw the lights come on for many students. They understood that their learning was much greater in this system, and I think I have a great deal of buy-in from them now.

Tomorrow we turn our attention to an activity involving sampling techniques instead of a Learning Catalytics assessment. Although we won’t be using our “clickers” I will try to encourage the same level of discussion that I witnessed today.

Statistics – My New Approach

Statistics – My New Approach

This semester I started a new approach in my Statistics classes.

I wanted to

  • focus more on conceptual understanding
  • make class time more engaging
  • introduce inference much earlier in the course – including the use of bootstrapping, simulation, and resampling
  • cover nonparametric options for certain hypothesis tests

The first strategy I decided to employ was the flipped classroom, incorporating peer instruction.

We are using the eText (Interactive Statistics) that I co-authored with Michael Sullivan, and the Interactive Reading Assignments are perfect for flipping the classroom. In these assignments students read a little/watch a little/do a little – students are reading text, watching conceptual videos and example videos, and answering questions that feed directly into their grade book. Students must complete the Interactive Reading Assignment before coming to class the next day. (There is a standard homework assignment that students must complete the following night.)

I am using Learning Catalytics to help with the peer instruction part of the class. Learning Catalytics is like a powerful version of a clicker – students submit answers through their smart phone, tablet, or computer. (The class is inside a computer lab, so most students are using a computer.) Learning Catalytics supports so many types of questions, but at this time I am only using multiple-choice and numerical questions. I post a question and give students time to answer it themselves, then I give them time to discuss their answers with their classmates and potentially change their answers. When most or all of the class has the right answer discussion may not be necessary.

Today I spent 20 minutes going over key concepts from the previous night’s reading assignment. I asked students to define the concepts, to explain the difference between related concepts, and provide examples. I restricted myself to only chiming in when I felt that more detail or insight would help. Most students were highly engaged and many students contributed to the discussion.

I next spent about 10 minutes helping students log into the computers and find the Learning Catalytics session.

Finally, that left 20 minutes for the 15 questions in Learning Catalytics. That was a little rushed in places. Tomorrow I plan on 10-15 minutes of discussion about the previous night’s Interactive Reading Assignment, leaving 35-40 minutes for fewer questions. This should allow for an excellent peer instruction session, followed by a time to review what we have learned.

Students were engaged with the questions and worked well with each other. My favorite moment occurred when only about one-third of the students initially got a question correct. Vigorous debate ensued, and in a short while many of the students had the right answer. I asked a student who switched their answer to explain why, and I noticed that several students really gained understanding from that student.

In future posts I will discuss my plans to include inference early in the course (starting in week 2), and the different techniques we will be learning in one of the 20 classroom projects we will be working on.

– George

I am a math instructor at College of Sequoias in Visalia, CA and the author of an algebra textbook and co-author of Interactive Statistics with Michael Sullivan (both with Pearson).