Browsed by
Category: statistics

Case Study of Interactive Statistics in an Online Class

Case Study of Interactive Statistics in an Online Class

I’m really proud of this latest study involving our Interactive Statistics (I am a co-author with Michael Sullivan) by Sam Bazzi at Henry Ford College. I saw Sam present his results at ICTCM and was really impressed. I encourage you to check out his case study.

Read the Study Here

This reinforces the fact that there is not a better product to use in an online statistics course: students persisted at higher rates and their test scores improved as the semester progressed. Sam took a lot of time and effort to set this course up, and according to his students it really paid off.

How It Works

The overall idea behind Interactive Statistics is for students to read a little, watch a little, and do a little as they make their way through the section.

  • Concepts are presented through text and video, and reinforced through applets.
  • Each example has 3 associated video solutions: by hand, by StatCrunch, and by calculator.
  • Examples are followed by exercises that students complete. Scores are incorporated into the student’s grade book immediately.

My Online Class

My online students do an IRA (Interactive Reading Assignment) for each section to learn the material, then follow up with a traditional homework assignment. In addition to the guided notebook that is available inside Interactive Statistics, I provide my students with Pointers for each section, and Guides for each IRA and HW assignment – check them out on my website here. The IRA can replace the “lecture” that traditional students get. My students come to campus for an in-person midterm exam and final exam.

Not Just For Online Classes

I use Interactive Statistics for my face-to-face classes as well. I use it to flip my classroom.

  • Students complete the IRA for the section before it is discussed in class.
  • Most classes begin with a Learning Catalytics session to determine the level of understanding and to identify any misconceptions.
  • Many classes incorporate collaborative engaging problem solving during the class session.
  • I no longer “lecture” – we have a student driven discussion instead.

This has allowed me to develop inferential intuition through simulations early in the semester, and incorporate alternative randomization tests and nonparametric tests later in the semester. I feel like my students have a greater understanding of statistics, and I am having more fun in the classroom than ever.

Any Questions?

If you’d like to talk about how to use Interactive Statistics in your class, or how to flip your statistics class, please leave a comment or reach out to me on Twitter or through the contact page on my web site.


Flipping a Two Hour Class – Intro Stats

Flipping a Two Hour Class – Intro Stats

This semester I am teaching a short term Intro Stats class, and I have found it more challenging to flip this class. The class meets 4 days a week for 2 hours a day. This can be difficult because I typically have two main concepts to cover, and students have trouble preparing for a second topic until they get a chance to work on the first concept in class. I will share some of the strategies I have used.

The Best Days

I have found that the best days are those which I have a concept that can extend to the entire two hours. For example, today I covered the two mean test using independent samples. We started by having a discussion about comparing the two mean test to the paired difference test that we covered yesterday. Students then worked through a few tests in their groups. Once I felt they had the two mean test under control I pivoted to the nonparametric Mann Whitney test, the test we use when the necessary conditions for the two mean test are not met. I was able to introduce this concept with a brief 10 minute mini-lecture, and followed up with another group activity with four tests to work through – some two mean & some Mann Whitney. Students got a chance to learn when to use each technique, and I felt confident that they understood both tests.

Making it Work

I have had to be flexible with my traditional approach. For example, I often cover binomial probabilities on one day and follow up with Poisson probabilities the next day. I think asking students to work on a Flip assignment on Poisson probabilities before we discuss binomial probabilities is a tall order.
First Day
So, for the first day students worked a Flip assignment on binomial probabilities before class, and in class the first hour was devoted to a Learning Catalytics assignment and a problem solving session. At that point I could have given a short 20 minute lecture on Poisson probabilities followed by more problem solving. Instead, we spent the second hour on a project introducing the concept of a one proportion test using the binomial distribution. (That is 4 chapters before we formally cover hypothesis testing.)
Second Day
For the second day, students worked on a Flip assignment on Poisson probabilities before class. We spent the first hour doing a team-based Learning Catalytics session followed by some problem solving with the Poisson distribution. For the second hour students did more problem solving on a mixture of general discrete probability distributions, the binomial distribution, and the Poisson distribution.

In a typical class that meets an hour per day this might have taken 3 days, but it took 4 hours of in-class time. This has happened a lot, and I have had to be real careful in terms of how I plan the schedule for this class. Switching from unit exams to a midterm/final approach has bought me a few days. I have learned to be more efficient with other topics.

Stacking Concepts

There are some pairs of topics that can be handled with two flip assignments on the same night. For example, sampling and sampling techniques are covered in two sections in our textbook and I typically spend two days on this material. I was able to give a combined flip assignment on sampling. In class we worked on a Learning Catalytics assignment, followed by an activity in which students got to experiment with the various methods.

Other places where this worked included qualitative and quantitative graphs, and measures of central tendency and dispersion.

Mid-Class Flip

One strategy I did not employ, but holds great promise, is using a mid-class flip assignment. The idea is that I could give students a flip assignment on one topic and begin the class with a group activity, then follow up with a 15-20 minute flip activity for that day’s second topic.

If technology is not available, that flip assignment could be as simple as a guided reading assignment. An open-ended problem solving assignment from the next section could be given. In a smart classroom, videos could be played for the entire class. There are many options.

Once that mid-class flip assignment is done the class could move on to a group activity or a Learning Catalytics assessment.


I feel I will be better prepared for the next time I flip a 2-hour class. I think the real key is to stop doing things the way I have always done them and really leverage the advantages of the flipped classroom.

I have also flipped my elementary algebra classes this semester, and will share about those in a later blog.

If you have any questions, please leave a comment or reach out to me on Twitter or through the contact page on my web site.


ICTCM 2017 

ICTCM 2017 

Had a great time at ICTCM 2017 in Chicago. And I learned a lot. Between Maria Andersen’s opening day keynote and Eric Mazur’s Saturday sessions I feel that I am making good progress in some areas, but there is still room for improvement. My “lecture time” is very active and engaging, but I feel like I need to turn it up a bit and allow my students more time to experiment and discover. I need to use assessment FOR learning. 

I heard from many instructors who are using our (Sullivan/Woodbury) Interactive Statistics and having great success. It gives students more responsibility for learning, and has changed the ways that the class is taught. We have quite an ambitious plan for 2.0, and cannot wait to share the details. 

Finally, it was great to catch up with so many friends and well respected colleagues. I learn so much from you all and you motivate me to be my best. I am thankful to have such a terrific personal learning network. 

Next week I will summarize the sessions I attended and the trends I noticed. 

– George 

Learning Catalytics- #ICTCM17

Learning Catalytics- #ICTCM17

This Saturday I will be speaking at ICTCM about how I use Learning Catalytics in my Statistics and Algebra courses. 

Collecting Homework 

I started slowly in my Statistics courses, using Learning Catalytics to collect “written” homework. I often give written assignments to supplement MyStatLab exercises, and Learning Catalytics allows me to collect certain problems or parts of certain problems. The answers are automatically graded and scores are transferred to my grade book in MyStatLab. This strategy encourages students to do the homework and to be on time. Students, if you wish, can have conversations about their strategies or answers. As the results come in I can address common errors or misconceptions. 

Reviewing for Exams

I found Learning Catalytics to be helpful for reviewing for exams. For example, while reviewing for an inferential exam I can post a problem and ask students to tell me which hypothesis test is the appropriate one to use. The same can be done for reviews on probability distributions, descriptive statistics, … I can ask conceptual questions or problems requiring calculations. 

I can use these results to get a real time read on how my students are doing with their preparation, and determine which concepts to address in detail. 

Flipping the Classroom/Peer Instruction

Here is where the real classroom power lies. When I flipped my Statistics class, I used Learning Catalytics to make the class sessions more interactive and engaging. I post a question and ask students to submit an answer. Then I either ask students to explain their answers to the class, discuss their answers in small groups, or I offer some insights of my own. At that point I allow students to change their answers if they wish. 

This approach has turned my class into a conversation with my students, or a conversation among my students, which is more effective than the traditional “top down” lecture.  

If you have any questions or comments about Learning Catalytics, flipped classrooms, Interactive Statistics, or anything else in this blog, please leave me a comment or reach out to me on Twitter @georgewoodbury. 

ICTCM here I come …

Flipped Classroom Materials for Statistics

Flipped Classroom Materials for Statistics

Last semester I flipped my Statistics classroom, and was really happy with the results. I have put together some pages explaining exactly how I flipped the classroom, with links/descriptions of documents that I used along with a calendar showing how I we covered the material.

You can find it all at

I will be adding more to the site as I make my way through a second semester of flipping that classroom – this time in a short-term (8 week) semester.

– George

Building an Early Inferential Approach into the Calendar

Building an Early Inferential Approach into the Calendar

I have had a few questions about how I am managing to work all of these early inferential projects into my Intro Stats course.

1) Switching from Chapter Exams to a Midterm and a Final

In the first 7 chapters of our textbook I used to give 4 exams. That means that I would use 4 days for exams and approximately 6 days for review. I have 4 days built into my calendar for review (2 days) and the midterm exam (2 days). That is a net gain of 6 days in the first half of the course.

I have 8 project days scheduled in the first half of the course so that only puts me two days behind, but I have been able to avoid spending more than one day on any section so there are unofficial gains there.

I have checked in with two colleagues and I am one day behind one of them and even with the other.

In the second half of the course I will apply the days saved from chapter tests to cover alternatives to the traditional hypothesis tests, including simulations and non-parametric tests.

2) I Will Not Have To Introduce Hypothesis Testing in the Second Half of the Course

I typically spend 4 days to cover the first hypothesis test (the 1-proportion test), but I should be able to jump right in and cover that test in one day.

My Calendar for Weeks 1-5

Here is the schedule I have followed to this point. I have put the project days in bold.

Date Topic
15-Aug Day 1 Syllabus etc.
16-Aug 1.1 Intro to Stats
17-Aug 1.2 Observational Studies, Experiments
18-Aug 1.3/1.4 Sampling Techniques
22-Aug 1.6 Experimental Design
23-Aug 2.1 Qualitative Graphs
24-Aug Project 1: Simulation for 1-Proportion
25-Aug Project 2: Randomization Test for 2-Proportions
29-Aug 2.2 Quantitative Graphs
30-Aug 3.1 Measures of Center
31-Aug Project 3: Bootstrap Method for Estimating a Mean or Median
1-Sep Project 4: Using Simulation for a Population Mean
5-Sep holiday
6-Sep Project 5: Using Bootstrap Method for a Paired Difference Test
7-Sep 3.2 Measures of Dispersion
8-Sep 3.4 Quartiles
12-Sep 3.5 5-Number Summary and Boxplots
13-Sep Project 6: Randomization Test for Two Means
14-Sep 4.1 Correlation
15-Sep Project 7: Hypothesis Test for Correlation


Randomization Test for Two Means

Randomization Test for Two Means

This semester I have incorporated a three-pronged strategy in my intro statistics classes:

  1. Flip the classroom – having students learn material at home
  2. Make the classroom more engaging – using more group activities and Learning Catalytics sessions
  3. Focus on early inferential statistics early and often

This week began with students learning about the 5-number summary and how to create a boxplot. On Monday we did an activity where students compared two samples of quantitative data in an effort to determine whether the population means were different. That was a natural lead-in to the randomization test for two means, which is an inferential concept that students can handle early in the semester.

Download the Two Mean Randomization project (pdf) here

This project began with the same data as Monday’s activity, and we found that approximately 5% of the trials produced a mean difference at least as extreme as the observed difference between the samples. Most students were slightly below 5% and were able to conclude that there was a difference between the two population means. A few students (and myself) came in at 5% or slightly higher and were not able to conclude that the population means were different (it was plausible that the two population means were equal). This led to a great discussion of how our results varied and the implications of that.

2_mean_randomizationI followed up with two other investigations. The first compared Overall Quality ratings from for mathematics instructors and English instructors. The second compared Overall Quality ratings for instructors at my college and nearby Fresno State. Students gathered all of the data before showing up for class.

Now that we have discussed many inferential concepts we will write up our first formal hypothesis test tomorrow when trying to determine whether a linear relation exists between two variables. Look for that blog post later this week.

Comparing Two Samples (Quantitative)

Comparing Two Samples (Quantitative)

My students are wrapping up the part of the course where we cover descriptive statistics. I gave them two sets of data (test scores from two different versions of the same exam) and they spent the day in class computing sample statistics and creating graphs for each sample. Their overall goal was to analyze their results and determine whether there was a significant difference between the two versions or not.

Download a pdf of this activity

Students compared measures of central tendency and the 5-number summaries and I asked them to share their observations. They went on to compute measures of dispersion and then we talked about whether the dispersion of each sample was similar. Finally they created histograms, pie charts, and boxplots and we discussed what they felt the graphs were telling them.

We had another great opportunity to discuss the fact that a perceived difference may not be significant unless we can determine whether the observed difference (the means were 4.8 points apart) would be unusual through some sort of repeated sampling.

This leads into our sixth project of the semester where we will use the randomization test for two means to determine whether the observed difference was significant. We will use StatCrunch for this test, although there are many other tools out there that can be used. My students will then move on to apply this test to two sets of data they collected. I will blog about the outcomes of that project in my next post.

Bootstrap – Matched Pairs

Bootstrap – Matched Pairs

This week I began with a bootstrap project for a paired-difference/matched-pairs scenario.

Download a pdf of the Project Here

One of my goals is to get students working with data they have collected, so I had students collect prices for 25 identical items at two stores. We used this for one of the investigations.

Investigation 1

A researched was investigating whether sons are taller than their fathers. My students were provided with 13 matched pairs. I had them find the difference for each pair (d = father’s height – son’s height). They had to determine whether to expect differences that were positive or differences that were negative. This is an important skill when setting up the alternative hypothesis, and I was happy with how my students understood what type of differences to expect.

We applied the bootstrap method to the sample differences, and the results are shown below.


Since the interval from the 2.5th percentile to the 97.5th percentile contained 0, we were unable to conclude that there is a difference between the heights of fathers and their sons. We had a great opportunity to discuss the implication of 0 being contained in the interval – and my students were able to understand that if a difference of 0 is in the interval then it is possible that there is no difference between the two groups.

Investigation 2

Here my students used their data from the two stores in an effort to support the claim that prices at Store A are lower than they are at Store B. We had some interesting results, including groups that reached 3 different conclusions when comparing Walmart and Target (Walmart is cheaper, Walmart is more expensive, no difference).

Follow Up

The following day in class I included a Learning Catalytics session where students were given various scenarios and intervals and asked for the appropriate conclusion. They proved that they retained their understanding, and several students displayed that by sharing their reasoning with the class after the question was closed for responses.

I am looking forward to the day where we cover the formal (p-value) hypothesis test for paired differences to see how well they understand the big picture.

Bootstrap Method – Estimating a Population Mean

Bootstrap Method – Estimating a Population Mean

Last week we did our third project that focuses on introducing inferential statistics earlier in the semester.

Download the Activity (pdf) Here

The bootstrap method repeatedly samples from a sample (with replacement) to help develop an interval estimate of any population parameter. For example, if there is a sample of 10 numerical values we select 10 values (with replacement) and compute the mean of that sample. We then repeat that process for a total of 1000 samples. We can then use the 2.5th and 97.5th percentiles to bootstrap a 95% confidence interval estimate for the population mean.

Investigation 1

Here’s the first example we walked through together, bootstrapping a 95% interval estimate:
A manager of a fast food restaurant devises a new drive-through system that he believes will decrease wait time from the time an order is placed to the time the order is received.  He initiates the new system at his restaurant and measures the wait time for 10 randomly selected orders.  The wait times, in seconds, are provided below.

108.5 67.4 58.0 75.9 65.1
80.4 95.5 86.3 70.9 72.0

Use the bootstrap method to create a 95% confidence interval for the mean wait time for the new system.

Here are the StatCrunch results for bootstrapping 1000 samples.

project3_aThe interval was 69.65, 87.54.

I followed up with an inferential question:
The manufacturer of the system claims that the mean wait time for all customers should be approximately 80 seconds.
Is this value contained inside the 95% interval?
Is the manufacturer’s claim plausible or is it unlikely to be true? Explain your decision.

Investigation 2

I followed up with a second investigation involving the sale prices of beachfront condos, and an inferential question where the claimed population mean fell outside of the interval.

Investigation 3

How old are students at our college?

I sampled the ages of 126 of my students and we applied the bootstrap method to this sample. I first had the students make their own claim about the mean (and median) age of all students at our college, and they then evaluated their claims using the bootstrapped estimation intervals.

I followed up by giving them the true mean and median age from our college’s information office, and both of these values were outside the estimation intervals. We then had a great chance to discuss the ways my sample could be biased and how the conclusions based on intervals from this sample were unreliable.

All in all, it was a great learning experience and I felt we took one giant step towards understanding the big picture in intro statistics.