Skip to content

#97 Some good news

In one of my classes last week, I realized that I was dutifully following a piece of advice that I gave in a blog post more than one year ago.  Because I don’t always heed my own advice, I decided that this was noteworthy enough to warrant its own blog post.

In post #33 (here), I implored statistics teachers to use data, examples, activities, and assignments that reveal human progress.  I provided many examples in that post and its follow-up (here), involving things such as increasing life expectancy and decreasing poverty rates around the world over the past few decades.  Ironically, the global pandemic began to disrupt all of our lives shortly after those posts appeared.

I still believe that it’s worthwhile to make students aware of data that reveal good news and human progress.  I recently read some news reports that inspired me to do this.  I asked students in my Statistical Communication class* to read a pre-peer-review article about an experimental vaccine for malaria (here).  This vaccine for malaria has the potential for a huge, positive impact on human welfare, particularly among children in Africa**. 

* See post #94 (here) for another activity that I used in this class.

** The World Health Organization estimates that more than 400,000 people died of malaria in 2019, with African children the most vulnerable group (here).

How did I try to ask good questions of my students about the malaria vaccine article?  To begin with, I gave them an online, auto-graded quiz consisting of ten questions.  In fact, I think you’ll get a good sense of the article just by reading my quiz questions and knowing that the correct answer is the always the first option presented here:

  1. What is the World Health Organization’s goal for the efficacy of a malaria vaccine by the year 2030?  [Options: 75%, 50%, 90%, 95%, 99%]
  2. In what country was this study conducted?  [Options: Burkina-Faso, Kenya, United Kingdom, United States, India, Sweden]
  3. Was this an observational study or a randomized experiment?  [Options: randomized experiment; observational study]
  4. How many treatment groups were used?  [Options: three, two, four, five, six]
  5. What ages were the subjects in this study, when they were randomized into one of the treatment groups?  [Options: 5-17 months, 5-17 years, 16 years and older]
  6. How many subjects were enrolled and received at least one vaccination?  [Answer: 450]
  7. Which group was NOT blinded as to which subjects received which treatment?  [Options: pharmacists, participants, participants’ families, local study team]
  8. Which of the following was NOT studied as a possible confounder?  [Options: race, gender, age group, bed net use]
  9. Which of the following comes closest to the percentage of participants with adequate bed net use?  [Options: 85%, 95%, 75%, 50%, 25%]
  10. What was the vaccine efficacy of the high-dose treatment after one year?  [Answer: 77%]

Notice that the first and last of these ten questions reveal the great promise of this vaccine.

One purpose of these quiz questions is to guide students in their reading.  I don’t mind if they look at the quiz questions in advance and if they refer to the quiz questions while they are reading.  Of course, I hope that my students read the full article and don’t treat this as a scavenger hunt to find the answers to my questions.  But I hope that my questions point students to some of the most important take-away points from the article. 

Another purpose of the quiz is to prepare students for our in-class discussion.  I admit that I do not feel very confident with leading class discussions, particularly via zoom.  Two pieces of advice that I give myself here are to ask good questions* and give students time to discuss their answers in small groups first. 

* You knew this was coming.

Because this class is about statistical communication rather than statistical concepts or methods, I posed this question to guide the discussion: What elements of the study should appear in a news article for the general public about this study?  In addition to providing a list of these elements, I also asked students to classify each element that they proposed into one of three categories: (1) essential, (2) helpful but not essential, (3) unnecessary.

Before I assigned students to breakout rooms to discuss this in small groups, I decided that I should first provide a few examples of what I mean by elements of the study.  I gave these three:

  • where the study was conducted
  • how many subjects participated
  • affiliations of the researchers. 

After a brief discussion, I used a zoom poll for students to vote on how they wanted to classify each of these three elements.  A large majority voted that the first two items are essential.  On the third item, the vote was about evenly split between the “helpful” and “unnecessary” categories.  I chimed in that there were so many co-authors on the study that I would include at most the affiliation of the lead author in a news report.

Then I assigned students to discuss this question in breakout rooms with 3-4 students each.  After fifteen minutes, we reconvened as a full class to discuss what they had come up with.  I typed their suggestions into a file that I projected to their screens during class and then posted after class.  For each element that they mentioned, we conducted a zoom poll to vote on whether the item was essential, helpful, or unnecessary.  When the discussion began to lose steam, I consulted my own list that I had created before class.  In both sections of the course, a few items on my list had not been mentioned by the students, so I offered them for consideration. 

Some of the items that my students identified as essential included:

  • background information about severity and consequences of malaria
  • selection criteria, including ages of the participants
  • when the study was conducted
  • use of random assignment, blindness
  • treatments used, including for the control group
  • response variable studied (whether or not the person developed malaria)
  • percentages in each group who developed malaria
  • statistical significance of the results

Some elements that were identified as helpful but not essential include:

  • charts or graphs of the results
  • ethics permissions that were obtained
  • adverse reactions that were studied
  • potential confounding variables that were studied
  • next steps to be taken

For the last five minutes of class, I asked a different question: What are some criteria for evaluating whether a news article provides a good report for the general public about this study?  I suggested that the article should include all elements that we considered to be essential.  Some suggestions from my students included:

  • captures interest at the outset
  • makes case for importance
  • appears visually appealing
  • uses appropriate language for general audience
  • includes link to the source article
  • states appropriate conclusions (not over- or under-stated) from study
  • includes some, but not too many, specific statistics from the study
  • interprets values correctly
  • does not contain errors

That covers questions that I asked before and during class.  Here’s the assignment that students are working on before our next class session: Find a news article about the malaria vaccine study that we discussed in class.  Include a link to this article with your report.  Identify which essential and helpful elements (as we identified in class) are included, and not included in the article.  Then write a paragraph analyzing how well the article summarizes and presents the malaria vaccine study for the general public.

This malaria vaccine article could provide an worthwhile example in an introductory statistics course, as well.  For example, you could discuss experimental design issues such as random assignment and double-blindness.  The example also lends itself to analyzing categorical data, both descriptively and with a chi-square test to compare success proportions among three groups.

I’m fairly pleased with how this particular session of my Statistical Communication course went, for several reasons.  Sometimes I feel guilty that I often present examples from before many of my students were born, so I’m delighted to focus on an article that made the news just a few weeks ago.  I am also very happy to introduce students to a research study that holds promise for considerable progress about human welfare and global health.  It’s very gratifying to show students that science and statistics can contribute to such hopeful developments. It’s also just plain fun to share such good news!

#96 Fostering collaborative learning online

This guest post has been contributed by Anelise Sabbag.  You can contact her at

Anelise Sabbag is a colleague of mine in the Statistics Department at Cal Poly – San Luis Obispo.  She earned her Ph.D. in the field of statistics education at the University of Minnesota.  Anelise regularly teaches our course for prospective teachers, as well as many other statistics courses, and she also conducts research into how students learn statistics.  She began teaching online, and studying how students learn effectively online, well before the pandemic forced many of us to teach remotely.  I am delighted that Anelise agreed to write this guest blog post about fostering collaborative learning in online courses.

Now that we have been teaching online for a few terms, some of you might be starting to consider the possibility of teaching online occasionally in the future. Maybe you have realized that online teaching is not as bad as you feared. One of the most challenging aspects is facilitating interactions among students’ online, which you may not have had time to think about as you were forced into online teaching during a pandemic. But maybe you have time to consider this now that you are a bit more used to teaching online. So, the question that I address in this blog post is: How can you encourage student-to-student interactions in your online asynchronous course?

This can be done in a variety of ways, through forum discussions and wiki assignments. But I would like to share with you another simple way to do this: a google doc file with a few good statistics questions and a collaborative structure. The “good questions” part seems to be pretty well covered in this blog already, so let’s explore a bit the idea of creating a collaborative structure in an assignment.

The idea of a collaborative structure is based on cooperative learning theory. The idea is to create an assignment that requires students to work in groups but also establishes positive interdependence among the students to encourage them to work together effectively. A collaborative structure starts with a common goal. That famous expression about “sinking or swimming together” is what we want students to understand. The success of the group is tied to the success of each student in the group. A common goal could be to require students to provide an answer to each of the good statistical questions you selected and inform the students that their responses will be graded as a group. The number of questions is really up to you. In my course, I ask about 7-8 questions per assignment, but because of time constraints I end up grading 3-4 of the questions. But having a collaborative assignment with one good question might be enough in your first attempt, until you figure out how much interesting information you can derive from these assignments. Then you might add more questions for your next iteration. But starting slowly always helps, right?

Here is an example of what I am talking about: In post #19 (here), Allan addressed the concepts of sampling bias and random sampling. The activity covered in this post asks students to read the Gettysburg Address and circle ten words as a representative sample from this passage. For each word in the sample, students are asked to record how many letters are in the word, calculate the average number of letters per word in the sample, and plot their sample average on a dotplot on the board, along with the sample averages of their classmates. Students then reflect on what they see in the dotplot and eventually conclude (or we hope they do) that this sampling method is biased. After this part of the activity, Allan suggests asking this question to students: Would asking people to circle twenty words (rather than ten) eliminate, or at least reduce, the sampling bias? I asked a very similar question in my class, and here is the final group answer for one of the groups:

This looks like a good answer, right?! It seems students are recognizing that the issue is really the sampling method and so increasing the sample size would not help. So, the students in this group were able to accomplish the shared goal of providing a strong group answer to this question.

But don’t you wish you could see the students’ thought process and interactions that led to this answer? Of course you do! At this point we have no clue about whether students worked together to get to this final answer. It could be that one student just wrote this answer and didn’t even discuss it with their group members. So, the collaborative structure of the assignment is not reflected in this final group answer. We need more!

The answer I showed above is the last step of a three-step process in my collaborative assignments. Now I will tell you what happens before that. We need to create a structure in the assignment that encourages students to get to that final group answer through collaboration and discussion. We could ask them to get together on Zoom and discuss (remember that the structure here is an online asynchronous class). But we might have several issues with that approach.  First, students need to find a time that they can all get together at the same time. That could already be tricky depending on the number of students in the group. Another issue we might face is student participation in this online discussion. Some students might not be willing to participate, perhaps because they are shy or maybe because they do not care. (Yeah, it is hard to admit that some students, especially in a general education course, do not really care about our beloved introductory statistics course.) So, if there is a way for students to do less work in the course, some of them will unfortunately choose that route. The issue then becomes individual accountability. Even though students are working together, we still want to hold them accountable for their individual work. So why not ask students to answer the questions individually first and then share their individual response with their group? Here are the initial answers that eventually led to the final group answer above:

Now we can see that the three students were not all on the right track initially. Student A, like many students in introductory statistics courses, might be thinking that a larger sample size leads to a representative sample. This student is most likely ignoring whether the sampling method is biased or not. Student B has the best answer compared to the other two answers. She does recognize that the issue is still the biased sampling method used. Her first comment, though, makes me want to ask a follow up question: In what way do you think increasing the sample size “might help a little”? I am hoping that this student would talk about precision, which would be great. Though she might talk about accuracy, which would not make me so happy because that is an incorrect connection (larger sample size leads to more accuracy when estimating the parameter of interest) that many students make. I think that Student C is moving toward this mistake when she says that the larger sample size would make the average more exact. This student then contradicts this first idea at the end of her sentence when stating that the results would be similar. If I could, I would ask her: How can the average be more exact and the results still be similar? By the way, I never did ask these questions as I am not “involved” in this part of the assignment. At this point, students only share the google doc file with their group members.

While asking students to provide initial answers might help students to be individually accountable, we are still stuck with the issue of students needing to find a time that they can all get together at the same time. So, why not make the discussion asynchronous? Even better, why not make this a written discussion? In this way you could get more insight into students’ thought processes. Using a google doc file can make this very straightforward. Students start by posting their initial answers in the file and sharing with their group members. Then they can have some sort of discussion about these answers to come up with a final group answer. Here is the discussion that the students had to get to final answer above:

This discussion is not as thoughtful as you might want, but it’s still interesting. The student who gave the right answer earlier is now helping the other students to understand what they might have missed. Of course, this is not the same help that a professor would give to a student, but you still see some interesting interactions. Student A, who was the one displaying worst understanding in the initial answer, seems to now be paying more attention to the sampling method, which she failed to do before. Student C also seems to be on the right track now, but this student could be just re-stating Student B’s answers. Or maybe this student finally understood that increasing the sample size will not account for the bias in the sampling method.

Of course, you are not always going to see thoughtful and encouraging discussions. Ideally, we hope that once all initial answers are provided, students will read and compare their answers, identify and correct mistakes so that they can finally end with a final group answer that is correct. The amount of interaction you see might depend on how specific your instructions are. You could require a minimum number of words in the discussion section or that every student should provide at least two contributions to the discussion.

Sometimes you might see a disappointing discussion, because students are not able to identify which answers are correct/incorrect as they are still unclear about the material. So, while students are completing this collaborative assignment, I release video(s) to help them. Please note that the statistical questions I use in these collaborative assignments are part of activities that students are required to complete individually first (and submit for grading based on completion). Usually, the activity consists of 30-40 questions, but the collaborative assignment is composed of only 7-8 of those questions. So, the videos that I release to students refer to the most important aspects of the activity and might include examples of typical mistakes students usually make. Students report that these videos are very helpful resources for them to complete these collaborative assignments. In addition to these videos, I also have “question forums” through which students can post any questions they might have about the assignments they are completing. Sometimes these also become helpful resources to students (see example below).

One important aspect in these assignments is to provide questions that could inspire discussion, or questions for which you think students might struggle and reach different answers. But even if students end up with similar and correct answers, you might still get some good interactions. For instance, a basic question that many of us are already asking our students is to calculate a p-value and show their work. In the example below, students noticed that they used the same correct inputs in the simulation but ended up with slightly different p-values. They were a bit stuck on this and looked at the “question forum” to get some help, where they noticed that some students already posted a question about this. At the end maybe they were able to recognize the role of random chance in their simulations.

Here is another example of a collaborative assignment using one of the questions that Allan suggests in post #41 (here). When students are conducting a simulation-based analysis, we want to make sure that they know what they are doing and are not just blindly following instructions. So, we can simply ask: What does each of the 1000 dots represent? I asked a similar question in an activity that guides students in examining whether dogs understand human cues (this activity is from the textbook used in my course: Introduction to Statistical Investigations by Tintle et al). To test this, an experimenter performed some sort of gesture (pointing, bowing, looking) toward one of two cups. The researchers then saw whether the dog would go to the cup that was indicated. Students are presented with the performance of a dog named Harley, who was tested 10 times and chose the correct cup for 9 of those times. One of the earlier questions in the activity shows an example of a student (Julia) flipping a coin 10 times to simulate the process of Harley randomly choosing between the cups. For each set of 10 flips, Julia records the number of heads. She repeats this process 11 times and creates the following dotplot:

Students are then asked: Using the context of the problem, explain what a dot on the plot represents.

In this example, we see students reflecting on their own answers and also going back to the question that was asked and the information (dotplot) provided to them. Students were able to help each other differentiate between the proportion and count of success and also bring in the context of the problem to their final answer, which none of them did initially. This is an example of a group containing only two students (the third group member dropped the class). In my experience, these collaborative assignments work better in groups of 3 (which would also help with the amount of time grading), but this group was mostly successful throughout the quarter.

In my course, students are randomly put into groups at the beginning of the quarter. The first week of class is focused heavily on getting to know each other, clarifying expectation of work individually and as a group, and creating an individual and group schedule for the remainder of the quarter. These first week assignments are essential in an online course! I believe we must help students recognize that their success in an online course is tied to their organization and schedule. I also believe that these beginning “organize yourself and your group” assignments set the stage for helpful and constructive group interactions. And I believe this is why I usually do not have “group problems” in my course, and students end up working with their group through the whole quarter. The more we help the group to get to know each other and work together, the more they will help each other and learn together. To encourage positive group interactions, you could even offer them a group reward. For instance, if all students in the group do well on the individual homework at the end of the week, then everyone in the group will receive an extra credit point.  

I hope this post can give you some ideas of how to provide your online students with opportunities to work together. I have high hopes that you will be pleasantly surprised with what you will see.

#95 Independence day, part 1

One of my favorite class sessions when I teach probability is the day that we study independent events.  This post will feature questions that I pose to students on this topic, which (as usual) appear in italics.

I introduce conditional probability and independence with some real data that Beth Chance and I collected in 1998, when the America Film Institute unveiled a list of what they considered the top 100 American films (see the list here).  Beth and I tallied up which films we had seen and produced the following table of counts:

Suppose that one of these 100 films is selected at random, meaning that each of the 100 films is equally likely to be selected.

  • a) What is the probability that Beth has seen the film?
  • b) Given the partial information that Allan has seen the film, what is the updated (conditional) probability that Beth has seen it?
  • c) Does learning that Allan has seen the randomly selected film change the probability that Beth has seen it?  In which direction?  Why might this make sense?
  • d) Repeat this analysis based on the following table (of made-up data) for two other people, Cho and Dwayne:

These probabilities are Pr(B) = 59/100 = 0.59 and Pr(B|A) = 42/48 = 0.875.  Learning that Allan has seen the film increases the probability that Beth has seen it considerably.  On the other hand, learning that Cho has seen the film does not change the probability that Dwayne has seen it: Pr(D) = 60/100 = 0.60 and Pr(D|C) = 42/70 =0.60.

  • e) In which case (Allan-Beth) or (Cho-Dwayne) would it make sense to say that the events are independent?

This question is my attempt to lead students to define the term independent events for themselves, without simply copying what I say or reading what the textbook says.  Dwayne’s having seen the film is independent of Cho’s having seen it, because the probability that Dwayne has seen the film does not change upon learning that Cho has seen it.  But Beth’s having seen the film is not independent of Allan’s having seen it, because her probability changes in light of that partial information about the film.

  • f) Based on these data, would you still say that (Allan having seen the film) and (Beth having seen the film) are dependent events, even if they never saw any films together and perhaps did not even know each other?

This question points to a fairly challenging idea for students to grasp.  Probabilistic dependence does not require a literal or physical connection between the events.  In this case, even if Allan and Beth did not know each other, being a similar age or having similar tastes could explain the substantial overlap in which films they have seen.  Similarly, Cho and Dwayne might have watched some films together, but the data reveal that their movie-watching habits are probabilistically independent.

My next example gives students more practice with identifying independent and dependent events, in the most generic context imaginable: rolling a pair of fair, six-sided dice.  Let’s assume that one die is green and the other red, so we can tell them apart. 

Consider these four events: A = {green die lands on 6}, B = {red die lands on 5}, C = {sum equals 11}, D = {sum equals 7}.  For each pair of events, determine whether or not the events are independent.  Justify your answers with appropriate probability calculations.

Here is the sample space of 36 equally likely outcomes:

We know that A and B are independent events, because we assume that rolling two fair dice means to roll them independently, so the outcome for one die has no effect on the outcome for the other.  The calculations are Pr(A) = 1/6, Pr(B) = 1/6, Pr(A|B) = 6/36 = 1/6, and Pr(B|A) = 6/36 = 1/6.

It makes sense that A and C are not independent, because learning that the green die lands on 6 increases the chance that the sum equals 11.  The calculations are: P(C) = 2/36 = 1/18 and Pr(C|A) = 6/36 = 1/6.  From the other perspective, Pr(A) = 1/6 and Pr(A|C) = 1/2, because learning that the sum is 11 leaves a 50-50 chance for whether the green die landed on 6 or 5.  The events B and C are also dependent, for the same reason and with the same probabilities.

Some students are surprised to work out the probabilities and find that A and D are independent.  We can calculate Pr(D) = 6/36 = 1/6 and Pr(D|A) = 1/6.  This conditional probability comes from restricting our attention to the last row of the sample space, where the green die lands on 6.  Even though the outcome of the green die is certainly relevant to what the sum turns out to be, the sum has a 1/6 chance of equaling 7 no matter what number the green die lands on.  Similarly, B and D are also independent.

The events C and D also make an interesting case.  Some students find the correct answer to be obvious, while others struggle to understand the correct answer after it’s explained to them.  I like to offer this hint: If you learn that the sum equals 7, how likely is it that the sum equals 11?  I want them to say that the sum certainly does not equal 11 if the sum equals 7.  Then I follow up with: So, does learning that the sum equals 7 change the probability that the sum equals 11?  Yes, the probability that the sum equals 11 becomes zero!*  These events C and D are definitely not independent, because Pr(D) = 2/36 but Pr(D|C) = 0, which is quite different from 2/36.

* Be careful not to read this as zero-factorial**.

** I never get tired of this joke.

Next I show students that if E and F are independent events, then Pr(E and F) = Pr(E) × Pr(F).  Then I provide an example in which we assume that events are independent and calculate additional probabilities based on that assumption.

Suppose that you have applied to two internship programs E and F.  Based on your research about how competitive the programs are and how strong your application is, you believe that you have a 60% chance of being accepted for program E and an 80% chance of being accepted for program F.  Assume that your acceptance into one program is independent of your acceptance into the other program.

  • a) What is the probability that you will be accepted by both programs?
  • b) What is the probability that you will be accepted by at least one of the two programs?  Show two different ways to calculate this.

Part (a) is as simple as they come: Pr(E and F) = Pr(E) × Pr(F) = 0.6 × 0.8 = 0.48*.  For part (b), we could use the addition rule: Pr(E or F) = Pr(E) + Pr(F) – Pr(E and F) = 0.6 + 0.8 – 0.48 = 0.92.  We could also use the complement rule and the multiplication rule for independent events, because complements of independent events are also independent: Pr(E or F) = 1 – Pr(not E and not F) = 1 – Pr(not E) × Pr(not F) = 1 – 0.4 × 0.2 = 0.92.  I like to specify that students should solve this in two different ways.  I go on to encourage them to develop a habit of looking for multiple ways to solve probability problems in general. Students could also solve this by producing a probability table:

* You probably noticed that I was a bit lax with notation here.  I am using E to denote the event that you are accepted into program E.  Depending on the student audience, I might or might not emphasize this point.

Next I mention that the multiplication rule generalizes to any number of independent events.  Then I ask: Now suppose that you also apply to programs G and H, for which you believe your probabilities of acceptance are 0.7 and 0.2, respectively.  Continue to assume that all acceptance decisions are independent of all others.

  • c) What is the probability that you will be accepted by all four programs?  Is this pretty unlikely?
  • d) What is the probability that you will be accepted by at least one of the four programs?  Is this very likely?

Again part (c) is quite straightforward: Pr(E and F and G and H) = Pr(E) × Pr(F) × Pr(G) × Pr(H) = 0.6 × 0.8 × 0.7 × 0.2 = 0.0672.  This is pretty unlikely, less than a 7% chance, largely because of applying to very competitive program H.  Part (d) provides much better news: Pr(E or F or G or H) = 1 – Pr(not E and not F and not G and not H) = 1 – Pr(not E) × Pr(not F) × Pr(not G) × Pr(not H) = 1 – 0.4 × 0.2 × 0.3 × 0.8 = 0.9808.  You have a very good chance, better than 98%, of being accepted into at least one program.

  • e) Explain why the assumption of independence is probably not reasonable in this situation.

Even though the people who administer these scholarship programs would not be comparing notes on applicants or colluding in any way, learning that you were accepted into one program probably increases the probability that you’ll be accepted by another, because they probably have similar criteria and standards.  It’s plausible to believe that learning that you were accepted by one school makes it more likely that you’ll be accepted by the other, as compared to your uncertainty before learning about your acceptance to the first school.  This means that the calculations we’ve done should not be taken too seriously, because they relied completely on the assumption of independence.

Next I ask students to consider a context in which independence is much more reasonable to assume and justify:

Suppose that every day you play a lottery game in which a three-digit number is randomly selected.  Your probability of winning for each day is 1/1000.

  • a) Is it reasonable to assume that whether you win or lose is independent from day to day?  Explain.
  • b) Determine the probability that you win at least once in a 7-day week.  Report your answer with five decimal places.  Also explain why this probability is not exactly equal to 7/1000.
  • c) Determine the probability that you win at least once in a 365-day year.
  • d) Suppose that your friend says that because there are only 1000 three-digit numbers, you’re guaranteed to win once if you play for 1000 days.  How would you respond?
  • e) Express the probability of winning at least once as a function of the number of days that you play.  Also produce a graph of this function, from 1 to 3652 days (about 10 years).  Describe the function’s behavior.
  • f) For how many days would you have to play in order to have at least a 90% chance of winning at least once?  How many years is this?
  • g) Suppose that the lottery game costs $1 to play and pays $500 when you win.  If you were to play for that many days (your answer to the previous part), is it likely that you would end up with more or less money than you started with?

Because the three-digit lottery number is selected at random each day, whether or not you win on any given day does not affect the probability of winning on any other day, so your results are independent from day to day.

We will use the complement rule and the multiplication rule for independent events throughout this example: Pr(win at least once) = 1 – Pr(lose every day) = 1 – (0.999)n, where n represents the number of days.  For a 7-day week in part (b), this produces Pr(win at least once) = 1 – (0.999)7 ≈ 0.00698.  Notice that this is very slightly less than 7/1000, which is what we would get if we added 0.001 to itself for the seven days.  Adding these probabilities does not (quite) work because the events are not mutually exclusive, because it’s possible that you could win on more than one day.  But it’s extremely unlikely that you would win on more than one day, so this probability is quite close to 0.007.  I specifically asked students to report five decimal places in their answer just to see that the probability is not exactly 0.007*.

* I like to refer to this as a James Bond probability.

For the 365-day year in part (c), we find: Pr(win at least once) = 1 – (0.999)365 ≈ 0.306.  The friend’s argument in part (d) about being guaranteed to win if you play for 1000 days is not legitimate, because it’s certainly possible that you would lose on all 1000 days.  In fact, that unhappy prospect is not terribly unlikely: Pr(win at least once) = 1 – (0.999)1000 ≈ 0.632 is greater than one-half but much closer to one-half than to one!*

* Feel free to read this as one-factorial.

Here’s the graph requested in part (e) for the function Pr(win at least once) = 1 – (0.999)n:

This function is increasing, of course, because your probability of winning at least once increases as the number of days increases.  But the graph is concave down, meaning that the rate of increase gradually decreases as time goes on.  The probability of winning at least once reaches 0.974 after 10 years of playing every day.

Part (f) asks us to solve the inequality 1 – (0.999)n ≥ 0.9.  We can see from the graph that the number of days n needs to be between 2000 and 2500.  Examining the spreadsheet in which I performed the calculations and produced the graph reveals that we need n ≥ 2302 days in order to have at least a 90% chance of winning at least once.  This is equivalent to 2302/365.25 ≈ 6.3 years.  If you’d like your students to work with logarithms, you could ask them to solve the inequality analytically.  Taking the log of both sides of (0.999)n ≤ 0.1 and solving, remembering to flip the inequality when diving by a negative number, gives: n ≥ log(0.1) / log(0.999) ≈ 2301.434 days.

I included part (g) just to make sure that students realize that winning at least once does not mean coming out ahead of where you started financially.  At this point of the course, we have not yet studied random variables and expected values, but I give students a preview of coming attractions by showing this graph of the expected value of your net winnings as a function of the number of days that you play*:

* The expected value of net winnings for one day is (-1)(0.999) + (500)(.001) = -0.499, so the expected value of net winnings after n days is -0.499 × n.

I still have not gotten to my favorite example for independence day, but this post is already long enough.  That example will have to wait for part 2 of this post, which will not be independent of this first part in any sense of the word.

#94 Non-random words

I am teaching a course called Statistical Communication during this spring quarter*.  This course aspires to help Statistics majors, most of whom are completing their second year, to improve their written communication skills with regard to statistical ideas and analyses.  These students have taken at least two statistics courses, and most have taken several more than that.  This class meets synchronously but remotely, with a total of fifty students across two sections.  I am teaching this course for the first time and have never taught a similar course.

* Cal Poly is on the quarter system, so we teach three ten-week (plus finals week) terms per year, not including summer.  Now that we have reached the second half of April, we are three weeks into the Spring term.  We only have one week between quarters, during which we must finish grading for Winter term and then get ready for the Spring term.  Because that’s not much time to prepare for a new course that is very different from any I’ve taught before, I warned my students on day one that I’ll be winging it, figuring out what happens in each class session as we go along.

This post describes a recent class session for this course.  As always, questions that I posed to students appear in italics.

The handout that I prepared for the class meeting bore the same title as this post: Non-random words.  I introduced students to the topic as follows:

One of the challenges in communicating well about statistics and data is that many terms that describe statistical concepts also have meanings in common, everyday conversation.  For some terms, the statistical and everyday meanings match up very well, so the common use can help with understanding the statistical meaning.  But for other terms, the everyday meaning is different enough to provide a hindrance to understanding the statistical meaning.

a) Join a breakout room, and prepare a list of statistical terms that also have common, everyday meanings.  Also think about the statistical and everyday meanings.  Then try to classify each term with regard to how closely the everyday meaning matches the statistical meaning.

I gave students about 12 minutes for this discussion, with 4-5 students in each breakout room.  Before I opened the breakout rooms, I provided an example by pointing to the title of the handout.  I suggested that random is a prime example of a word for which its meanings in everyday conversation are not completely aligned with statistical uses of the word. 

b) We will reconvene as a full class to compile this list and discuss how closely the meanings align.

Some words that my students suggested include: normal, uniform, mean, range, distribution, correlation, significant, confident, independent, risk, odds, chance, effect, control, interaction, block, confounding, sample, population, parameter, factor, response, model, residual, error.

Before class, I had generated my own list.  After giving my students about ten minutes to suggest their words, I looked at my list and found several that had not been mentioned yet: bias, expected, variance, association, statistic, tendency, likelihood, skew.

Next I asked students which words are the most problematic, in that the everyday usage hinders understanding of the statistical meaning.  Some words that students put in this category include: normal, odds, independent, significant, power, control, block.

Our discussion of questions (a) and (b) took more than half of the 50-minute class session.  For the rest of the time, I turned our discussion to an in-depth discussion of the word random:

c) On a scale of 0 – 10, how important would you say the word “random” is in statistics?

I asked students to respond to this question in the zoom chat window*.  All of their responses were on the high side, ranging from 7 to 10, with a majority at 9 or 10.

* I wish that I had thought ahead to prepare this as a zoom poll question.  I think almost all students would have responded to a poll question, whereas only about ten students in each section responded in the chat.

d) What do you think “random” means in everyday (or even slang) usage?

Some common responses were to say surprising, unusual, and unlikely.  Other synonyms offered were odd and weird.  Slightly longer responses included out-of-the-ordinary and out-of-context.  For example, if someone says that a “random” thing happened to them today, they probably mean that it was an unusual, out-of-the-ordinary, occurrence.

A second type of response referred to being haphazard or unpredictable, lacking a pattern or plan.

e) Look up some definitions of “random” in an online dictionary.

I wanted my students to think first for themselves about everyday meanings of “random.”  But then I figured that I should take advantage of knowing that the students are all online during class.  Some dictionary definitions that they provided include:

  • Unknown, unspecified
  • Without method or conscious decision
  • Lacking definite purpose or plan

f) In what ways is “random” used in statistics?

I intended to spend a good bit of time on this question.  Because most students take this course about halfway through their undergraduate career, it provides a good opportunity to review some of the most important topics that they should have learned.  Prior to the class meeting, I had four aspects of “random” in mind for this discussion:

  • Random sampling aims to select a representative sample from a population, so findings about the sample can be generalized to the population.
  • Random assignment tends to produce similar treatment groups, to enable cause-and-effect conclusions if the treatment groups reveal a significant difference in the response.
  • Random selection applies to situations such as choosing a representative from a group of people or dealing out cards in a game.
  • Random variables can model various real-world phenomena, such as waiting time at a fast-food restaurant or number of transactions at an automatic teller machine.

I don’t think I’m very good at leading class discussions, in part because I often have a specific endpoint in mind as I did with this question.  Sometimes I even confess to my students that I’d like them to read my mind, even though I know that’s completely unfair to ask.  In this case my students read my mind quite well and suggested a variation on each of these four aspects of the word “random.”

g) Would you say that the everyday usage of “random” is a help or a hindrance when trying to communicate statistical uses of the word?

Once again, I wish that I had prepared this as a poll question in advance.  Instead I asked students to reply in the chat window, and most of them were reluctant to do that.  Those who volunteered an answer voted for hindrance, which is the response I was hoping for. 

I proposed to students that there is substantial irony here.  In everyday usage random means having no method or plan.  But random sampling and random assignment are very specific methods that require a lot of planning to implement.  Similarly, a random variable provides a very specific, predictable pattern for the long-run behavior of what it’s modeling, even though the outcome of a specific instance is unpredictable.

I try to think a lot about what kind of assessments to provide after a class session like this.  In this case, I made two small assignments and am contemplating a third, more substantial one.

I’ve mentioned many times that I give lots of quizzes in my courses.  Following this class session, I gave my students a very easy quiz.  I simply asked them to select their five favorite words that illustrate the distinction between everyday and statistical meanings.  I realize that this quiz amounted to giving free points for showing up to class, paying a modest amount of attention, and taking a few minutes to respond in Canvas.  But I hope students gave a little reflection as they answered, and I enjoyed reading their responses to see which words most resonated with them.

I also created a discussion in Canvas in which I asked students: Describe an example that uses a word with a specific statistical meaning in a way that carries a different meaning than the statistical one.  I’m thinking of the words that we discussed and listed in connection with the “Non-random Words” handout. Be very clear about which word(s) you are referring to.  Also describe what you perceive to be the intended meaning of the word(s).  If you found the example online, include a link.  I kicked off this discussion with an example that had appeared in my inbox just that morning: I received an email message inviting me to attend a webinar titled “A Conversation on Power, Structural Racism, and Perceptions of Normality in STEM Through a Lens of Critical Race Theory.”  The statistical words used in non-statistical ways are power and normality.  In this context, power refers to authority or control over others, and normality refers to what is typical or expected.  Here is a link to the webinar announcement.

I am also considering asking students to write an essay with these instructions: Select one of the words that we identified in class.  Write an essay of 250-400 words in which you describe how the statistical meaning of the word compares to the everyday meaning.  Mention similarities as well as differences, if there are similarities.  Provide at least one example to explain the word’s meaning in statistics.  Write as if to a relative of yours who is well-educated and intellectually curious but has not specialized in a STEM field and has never taken a statistics course.  Be sure to cite any references that you use (e.g., dictionary, textbook, wikipedia, …)

I have not given this assignment yet, because I am trying to balance students’ workload (and my grading load) with other assignments.  I am also debating whether I should ask them to select from a small list of words that I provide, such as: normal, bias, error, power, independent, expectation.

I realize that few readers of this blog are teaching a course called Statistical Communication.  I suspect that you might be thinking: What does this have to do with teaching introductory statistics*? 

* Even though I italicized this question for emphasis, this one is directed at myself and perhaps you, rather than students.

Many words have slightly or substantially different meanings in statistics than in everyday conversation, which can present a hurdle for introductory students to overcome.  I think we can help students by highlighting such discrepancies, as with the word random.  By pointing out that such words have a particular meaning in statistics that differs from what students might expect*, we can help them to concentrate on the statistical meanings that we’d like them to learn.  Also, even though few courses have the word “communication” in their title, many introductory courses have an explicit or implicit learning objective to help students learn to communicate effectively with data.

* By all means, do not expect the statistical meaning of the word expect to mean what your students might expect.  See post #18, titled What do you expect?, here.

#93 Twenty-one questions about USCOTS ’21

Registration for the 2021 U.S. Conference on Teaching Statistics (USCOTS) opens today.  I’m so excited about this that I will devote this blog post to answering 21 questions that you may have* about this conference**.

* You may not even have realized that you have these questions until you read them.

** I also wrote a bit about USCOTS in a meandering and autobiographical post #76, titled Strolling into serendipity, here.

1. Where can I register?  Follow the link here.

2. How much is the registration fee?  $25.  If this would constitute a hardship, you can receive a full waiver.

3. When is it?  The conference runs from June 28 – July 1.  Sessions will run from approximately 11:30am – 5:30pm Eastern time (U.S.) on each day.  Pre-conference workshops begin on June 24.

4. Where is it?  USCOTS will be held virtually for the first time this year, so it’s happening wherever you and your internet connection happen to be at the time.

5. Why should I attend USCOTS?  (Thanks for asking.  I really should have started there, shouldn’t I?)  Many statistics conferences include sessions on teaching, and many teaching conferences include sessions on statistics, but USCOTS is devoted entirely to the challenge of teaching statistics well.  If you teach statistics at the undergraduate or high school level, you will find sessions that are relevant to your everyday work in every time slot.  Our goal is for every session to include both practical advice and thought-provoking ideas, and also to present them in an engaging, perhaps even fun, manner.  If you’ve never attended USCOTS, we welcome you and hope that you’ll meet some new friends.  If you have attended USCOTS, we welcome you back to renew acquaintances.  We hope that you’ll be inspired to improve your teaching of statistics.

6. What is the conference theme?  Expanding opportunities.

7. Can you say more about that?  We encourage presenters and attendees to interpret this theme broadly, but we primarily have two questions in mind:

  • How can we (teachers of statistics and others involved with statistics education) increase participation and achievement in studying statistics by students from underrepresented groups?
  • How can we better encourage and support students and colleagues who are beginning or contemplating careers in statistics education?

8. What kind of sessions are planned?  Each of the four days will feature a keynote presentation and interactive breakout sessions.  We’ll also have “posters and beyond” presentations, “birds-of-a-feather” discussions, and exhibitor demonstration sessions.  New this year will be a speed mentoring session.  Another highlight will be an awards presentation ceremony.  Speaking of highlights, I almost forgot to mention my own favorite: Opening and closing sessions will feature lively five-minute presentations on the conference theme.  You can see the conference program here.

9. Who are some of the presenters?  The keynote speaker for Monday is Rebecca Nugent from Carnegie Mellon.  She will discuss how the emerging field of data science can expand opportunities for students who have been under-represented in statistics.  Tuesday’s keynote presentation will be a panel discussion about expanding horizons and fostering diversity, with panelists Felicia Simpson, Jacqueline Hughes-Oliver, Jamylle Carter, Prince Afriyie, and Samuel Echevarria-Cruz.  On Wednesday Catherine D’Ignazio and Lauren Klein will discuss theme from their book Data Feminism.  Alana Unfried from California State University – Monterey Bay will give Thursday’s keynote presentation.  She will discuss the advantages of a co-requisite model that enables students needing remediation to enter directly into an introductory statistics course.

10. What are some of the workshop topics?  These topics include community-engaged learning, data visualization, data science, Bayesian statistics, R tidyverse, games, multivariable thinking, and statistical literacy.  You can see the list of pre-conference workshops here.

11. How about some of the breakout session topics?  These topics include data science, social justice, gamification, communication skills, oral assessments, computational thinking, data visualization, community building, educational fun, and data ethics.  You can find the list of breakout sessions here.

12. What platforms will the conference use?  The primary platform will be zoom.  You can attend sessions simply by following zoom links.  We’ll make frequent use of breakout rooms, polls, and chat within zoom to increase engagement.  We will also use to replicate an in-person experience more closely.

13. Will the conference be interactive and engaging?  That’s our goal.  I think this is more challenging with a virtual conference than with an in-person one, but we’ll do our best.  Of course, interactivity and engagement depend on participants being willing* to interact and engage.

* I hope eager!

14. Can I still submit a proposal to present at the conference?  Yes.  Proposals for “posters and beyond” sessions are due by April 22 (here).  Proposals to lead a birds-of-a-feather discussion are due by May 31 (here).

15. How can I earn a free registration?  Participate in the SPARKS video challenge.  This asks for a very short (10-20 seconds) video clip that can be used in teaching statistics.  You can see examples and submit your entry here.

16. Do you have a social media hashtag in mind?  Yes, please use #USCOTS21.

17. Would you like me to spread the word to colleagues and friends?  Yes, absolutely!

18. Do I have to attend every minute of every session of the conference?  No.  (Whew, I’m glad to have a chance to introduce some variability to that long string of “yes” answers that I have been giving.)  Feel free to tune in when you can and step away when you need to.  As you would expect, I think it would be ideal if you can block out several hours of uninterrupted time for each day of the conference, but of course I realize that your circumstances may not allow that.

19. Can I see what has happened in previous USCOTS conferences?  I can resume my “yes” answers again.  See the links for “previous years” on the right side of the main conference page here.

20. Do you happen to have a one-minute video with a musical invitation to attend USCOTS that I could watch and point others to?  Yes*!  Thanks to the creativity and talents of Larry Lesser and Mary McLellan, please enjoy the video here.

* Wow, what a great question; it’s like you were reading my mind!

21. Please remind me: how can I register?  Just follow the link here.

#92 What can you do?

Teachers are often asked: What can you do with …?  For example, many students and prospective students have asked me: What can you do with a degree in statistics? 

I used to find it very challenging to answer this question well.  One reason is that I have never had a job other than college professor.  Don’t get me wrong: I love my job, and I would make the same choice again, without a second thought, if I were starting over.  But my career has not provided me with much first-hand experience for answering that question.

I eventually came up with an answer that I really liked.  I came to give this answer every time I heard the question.  I still give the same answer now.  In fact, I like this answer so much that I put it on the back of my business cards. 

My answer is:  There you can find  the alumni updates section of our department newsletter*.  I am referring to the Department of Statistics at Cal Poly – San Luis Obispo.  We have had a bachelor’s degree program in statistics since the mid-1970s, and we are very proud of our alums.

* You can also find previous editions of the newsletter here and here.

Why do I like this answer so much?  Let me count the ways:

  1. This answer relies on other people’s words, not mine.  Because I do not have much relevant first-hand experience for addressing this question, I am very happy to refer to others’ experiences.
  2. These people have an undergraduate degree in statistics and are out in the “real world.” Most are outside of academia, applying what they’ve learned.
  3. Our alums have experienced diverse work experiences.  Many work very closely with data and statistics on a daily basis, but others’ careers are only tangentially related to data, if at all.  Some are not using their academic background in statistics at all, which I think is valuable for demonstrating that what you study as an undergraduate does not dictate what you have to do with the rest of your life.
  4. Needless to say, these are real people with real lives, including families and hobbies and interests that are not related to statistics at all.  I think it’s nice for current and prospective students to see that these folks have families, weddings (some to fellow alums of our program), children, pets, hobbies, (pre-pandemic) travel adventures, and more.
  5. This answer fits on the back of a business card.

Communicating with our alums to solicit these updates was one of my favorite tasks when I recently served as department chair for six years.  In fact, I enjoyed this activity so much that I volunteered to continue after I completed my terms from my chair.  I am very proud that so many of our alums take the time to respond with an update; 73 responded for the most recent edition, and even more replied for the two previous editions.

A big part of my enjoyment is that I taught many of these students, so of course it’s fun for me to hear from them and learn about what they’re up to, both professionally and personally.  I realize that you do not know these Cal Poly alums personally*, but I’m hoping that you might enjoy reading about the kinds of careers that people with undergraduate degrees in statistics can pursue.  I will provide a brief summary in this post, but I highly recommend that you follow the links above to read their words directly for yourself**.

* Unless you are one of my Cal Poly colleagues, or perhaps even one of the Cal Poly alums who contributed an update

** You’ll find that the updates are spread across many pages, arranged by graduating class year.  Click on links at the bottom of the pages to see more updates.

Many of the job titles for these alums include the terms data scientist or data analyst.  Some other terms include data quality analyst, research analyst, risk consultant, actuary, software engineer, SAS programmer, or R programmer.

The industries in which these alumni work run the gamut, including banking, insurance, financial services, health care, fashion, marketing, medicine, pharmaceuticals, biotechnology, social media, gaming, entertainment, education, and more.

Some alums are pursuing or have completed graduate degrees, in fields such as statistics, biostatistics, public health, data science, business analytics, computer science, computational science, psychology, and education.

A few of the alums almost apologize for not using statistical methods in their daily work.  But they generally say that learning how to think about data and solve problems has served them well.  For example, Alex wrote that our year-long sequence in mathematical statistics “taught me to think ‘why’ instead of just ‘how.’”  Cisco contributed that “the most important thing that I learned from statistics and still use is the thought process to take big generic problems and turn them into manageable steps toward improvement.”

That summary was brief, as promised, but very dry.  Like I said, I’d prefer that you read the alums’ words rather than mine (again, here and here and here). Rather than delete my dry summary, let me instead try to add some life by highlighting a few specific updates. I hope these might help to persuade you to read them all :

  1. Maddie works as a financial data analyst for a solar energy company during the week.  On weekends she works at a residential care facility for adolescent girls with anxiety disorders.
  2. Jianyi started by working for a non-profit organization while launching her own cake-baking business.  Now she works as a production manager and data analyst for a company that designs lighting accessories.
  3. Alicia taught at an all-girls Catholic high school in Sacramento and now teaches statistics and calculus at Sacramento State University.  She also writes and performs comedy sketches, is writing a screenplay, and writes a blog here.
  4. Upneet moved to a city in which probability plays a large role in the economy: Las Vegas.  She works as an analyst at the Venetian/Palazzo Hotel and Casino in Las Vegas.
  5. Caiti has held positions as a data scientist for two companies that I suspect you have heard of: The Gap, Inc. and Google.
  6. David started his career as an engineer for Disney.  Now he is co-founder of an e-sports social media start-up company.
  7. Hunter earned his Ph.D. in Statistics and returned to Cal Poly as a faculty member in our department.  He has recently earned tenure, and he has also co-authored a blog on teaching data science (here).
  8. Chris heads up the data effort for a video game start-up company in Berlin.  He has helped the video game industry to become more data-driven, implementing more sophisticated methods and technologies.
  9. Emily taught AP Statistics for a decade before becoming Mathematics Coordinator for the Merced County Office of Education.  One of her initiatives involves developing a data science course to offer high school students an additional mathematics pathway to college readiness*.
  10. Kendall also taught AP Statistics for a decade, until he recently bought a coffee farm on the Big Island of Hawaii, where he also works on a dive boat.

* This is far from her most impressive accomplishment, but Emily wrote a guest post for this blog (here).

What can you do with a degree in statistics?  The American Statistical Association has some great materials for answering this question, including their This is Statistics project (see here and here).

For students attending or considering Cal Poly, I like my answer of pointing them to alumni updates (once again, for the final time, see here and here and here).  I hope that this answer might also be a reasonable one for you to offer to your students.  Even better, you could reach out to your own former students and compile their updates.

I have greatly enjoyed using our department newsletter as a vehicle for keeping in touch with alums.  I focus a lot of my teaching effort on preparing handouts and activities, developing and grading assessments*.  These alumni updates provide me with a reminder that the most important part of teaching is helping students to learn and prepare for their careers and lives.

* Remember: Ask good questions.

Because this post has extolled the virtues of reading words other than my own, I will conclude with advice and encouragement from Jose, who graduated from Cal Poly with a degree in Statistics in 1993: Think about what’s fulfilling for the soul and not the bank account….  These are exciting times for statisticians and anyone analytically inclined. Predicting the future with confidence and with limited data was never more important and exciting.

#91 Still more final exam questions

In my previous post (here), I discussed two open-ended questions that I asked on a recent final exam.  Now I will discuss eight auto-graded, multiple-choice questions that I asked on that final exam.  As with last week’s question, my goal here is to assess students’ big-picture understanding of fundamental ideas rather than their ability to apply specific procedures.  As always, you can judge how well you think these questions achieve that goal.  Also as always, questions that I pose to students appear in italics.

1. Suppose that Cal Poly’s Alumni Office wants to collect sample data to investigate whether Cal Poly graduates from the College of Business differ from Cal Poly graduates from the College of Engineering with regard to average annual salary.

a) What are the observational units?  [Options: Cal Poly graduates; Annual salaries; Colleges]

b) What is the response variable?  [Options: Annual salary; Which college the person graduated from; Whether or not the average annual salary differs between graduates of the two colleges]

c) Should you advise the alumni office to use random sampling to collect the data?  [Options: Yes, no]

d) Should you advise the alumni office to use random assignment to collect the data?  [Options: No, yes]

e) What is the alternative hypothesis to be tested?  [Options: That the population mean salaries are different between the two colleges; That the population mean salaries are the same between the two colleges; That the sample mean salaries are different between the two colleges; That the sample mean salaries are the same between the two colleges]

This question covers a lot of basics: observational units and variables, random sampling and random assignment, parameters and statistics.  I think this provides a good example of emphasizing the big picture rather than specific details.  The toughest question is part d), because many students instinctively believe that random assignment is a good thing that should be used as much as possible.  But it’s not feasible to randomly assign students to major in a particular college, and it’s certainly not possible to randomly assign college graduates to have majored in a particular college in retrospect.

2. Suppose that a student collects sample data on how long (in seconds) customers wait to be served at two fast-food restaurants.  Based on the sample data, the student calculates a 95% confidence interval for the difference in population mean wait times to be (-20.4, -6.2).  What can you conclude about the corresponding p-value for testing whether the two restaurants have different population mean wait times?  [Options: Smaller than 0.05; Smaller than 0.01; Larger than 0.05; Larger than 0.10; Impossible to say from this confidence interval]

I could have asked students to calculate a confidence interval for a difference in population means.  But this question tries to assess a big-picture idea: how a confidence interval relates to a hypothesis test.  Because the confidence interval does not include zero, the sample data provide substantial evidence that the population mean wait times differ between the two restaurants.  How much evidence?  Well, this is a 95% confidence interval, so the difference must be significant at the analogous 5% significance level.  This means that the (two-sided) p-value must be less than 0.05.

I’m not usually a fan of including options such as “impossible to say.”  But that’s going to be the correct answer for the next question, so I realized that I should occasionally include this as an incorrect option.

3. The following output comes from a multiple regression model for predicting a car’s overall MPG (miles per gallon) rating from its weight and cargo volume:

If you were to use the same data to fit a regression model for predicting a car’s MPG rating based on only its cargo volume, what (if anything) can you say about whether cargo volume would be a statistically significant predictor?  [Options: Impossible to say from this output; Yes; No]

This question tries to assess a big-picture idea with multiple regression. The result of a t-test for an individual predictor variable only pertains to the set of predictor variables used in that model.  This output reveals that cargo volume is not a helpful predictor of MPG rating when used in conjunction with weight.  But cargo volume may or may not be a useful predictor of MPG rating on its own.

4. Suppose that you select a random sample of people and ask for their political viewpoint and whether or not they support a particular policy proposal.  Suppose that 60% of liberals support the proposal, compared to 35% of moderates and 25% of conservatives.  For which sample size will this result provide stronger evidence that the three political groups do not have the same population proportions who support the proposal?  [Options: Sample size of 200 for each group; Sample size of 20 for each group; The strength of evidence will be the same for both of these sample sizes.]

Students may have recognized this as a situation calling for a chi-square test, because we’re comparing proportions across three groups.  But this question is assessing a more fundamental idea about the impact of sample size on strength of evidence.  Students needed only to realize that, all else being the same, larger sample sizes produce stronger evidence of a difference among the groups.

5. Suppose that you select a random sample of 50 Cal Poly students majoring in Business, 50 majoring in Engineering, and 50 majoring in Liberal Arts.  You ask them to report how many hours they study in a typical week.  You calculate the average responses to be 25.6 hours in Business, 32.2 hours in Engineering, and 21.8 hours in Liberal Arts.  For which standard deviation will this result provide stronger evidence that the three majors do not have the same population mean study time?  [Options: Standard deviation of 4.0 hours in each group; Standard deviation of 8.0 hours in each group; The strength of evidence will be the same for both of these standard deviations.]

This question is very much like the previous one.  Now the response variable (self-reported study time) is numerical rather than categorical, so we are comparing means rather than proportions, and ANOVA is the relevant procedure.  This question asks about the role of within-group variability, without using that term.  Students should recognize that, all else being equal, less within-group variability provides stronger evidence of a difference among the groups.

6. Why do we not usually use 99.99% confidence intervals?  [Options: The high confidence level generally produces very wide intervals; The technical conditions are much harder to satisfy with such a high confidence level; The calculations become quite time-consuming with such a high confidence level; The high confidence level generally produces very narrow intervals.]

This question addresses a very basic and fundamental issue about confidence intervals.  I believe that if a student cannot answer this correctly, then they are misunderstanding something important about confidence intervals.  In the past, I have asked this as a free-response question, and I have asked students to limit their response to a single sentence.  I’m not very satisfied with the options that I presented here, so I’m not sure that this question works well as multiple-choice.

7. The United States has about 255 million adult residents.  Which of the following comes closest to the sample size needed to estimate the proportion of American adults who traveled more than one mile from their home yesterday with a margin-of-error of plus-or-minus 2 percentage points?  [Options: 255; 2550; 25,500; 255,000; 2,550,000]

I ask a variation of this question on almost every final exam that I give.  I presented a very similar version in post #21 (here).  Just to mix things up a bit, I changed this version to refer to adult Americans rather than all Americans.  Mostly for fun, I used options that all begin with the same digits 255, so the question asks about order of magnitude.  Many students mistakenly believe that the necessary sample size is larger than the correct response of 2550.  Students could perform a calculation to determine this answer, but I have in mind that they should remember that many class examples of real surveys had sample sizes in the range of 1000-1500 people and produced margins-of-error close to 3 percentage points*.

* I suspect that you have noticed that this is the first question for which the correct answer was not the first option given.  Of course, students see the options in random order determined by the learning management system (LMS).  I find it convenient to enter the options into the LMS with the correct answer first, so I thought I would do the same in this post.  I altered that for question #7 just to keep you on your toes.

8. Which of the following procedures would you use to investigate whether Cal Poly students tend to prefer milk chocolate or dark chocolate when offered a choice between the two?  [Options: One-sample z-test for a proportion; Two-sample z-test for comparing proportions; One-sample t-test for a mean; Paired t-test for comparing means; Chi-square test for two-way tables]

I included several questions of this “which procedure would you use” form on my final exam.  I especially like this one, the context for which I borrowed from Beth Chance.  This scenario is the most basic one of all, and the very first inference setting that I present to students: testing a 50/50 hypothesis about a binary categorical variable.  Some students mistakenly believe that this is a two-sample comparison.  The “between the two” language at the end of the question probably contributes to this confusion.  I used that wording on purpose to see whether some students would mistakenly conclude that this suggests a comparison between two groups rather than a comparison between two categories of a single variable.

Last week I received a comment asking whether I worry that my students might read my blog posts to discover some exam questions that I like to ask, along with discussion about the answers.  I have to admit that I do not worry about that at all.  If my students are motivated enough to read this blog, I’ll be delighted.

I promise that next week’s blog post address something other than exam questions.  I always feel like writing about exam questions is somewhat lazy on my part, but I’ve invested so much time in writing and grading these questions that it’s very helpful to double-dip by using them in blog posts as well.  The Spring term at Cal Poly begins today, so I’m hoping that will inspire some new ideas for blog posts.

#90 Two more final exam questions

As they prepare for a final exam, I always advise my students to try to focus on the big picture rather than small details.  I’m pretty sure that they find this advice to be unsatisfying, perhaps worthless.  I don’t think they know what I mean when I say to focus on the big picture.  I also admit that this is much easier said than done. 

I just gave a final exam to my students, as the Winter quarter has now ended at Cal Poly*.  I think I asked some final exam questions that succeed at focusing on the big picture.  I will present and discuss two such free-response questions here.  As always, questions that I pose to students appear in italics.

* Well, perhaps I should clarify that the Winter quarter has ended for students, but it continues for faculty like me who still have final exams to grade and course grades to assign.

My students were randomly assigned to receive one or the other of these two versions:

1a. Suppose that a friend of yours says that they were reading about confidence intervals, and they encountered the symbols x-bar and mu (μ).  How would you respond if they ask: What’s the difference between what these symbols represent, and what does that have to do with confidence intervals?

1b. Suppose that a friend of yours says that they were reading about confidence intervals, and they encountered the symbols p-hat and pi (π).  How would you respond if they ask: What’s the difference between what these symbols represent, and what does that have to do with confidence intervals?

My goal here was to assess whether students could provide a big-picture overview of the distinction between parameter and statistic, along with explaining how that distinction relates to the topic of confidence intervals.  I’m fairly pleased with how this question turned out.

Before I continue, let me say that students were allowed to use their notes and my handouts on this exam.  This is not a new policy of mine related to the pandemic and remote teaching; I have used open-notes exams for a long time.  It’s also possible, of course, that some students also performed google searches during my unproctored final exam.

As I’m sure you can imagine, many students copied sentences directly from their noted or my handouts into their response.  As you can also imagine, this question was not a routine one to grade.  The grading went fairly smoothly, though, once I settled on the four things that I would look for:

  1. that  p-hat/x-bar represents sample proportion/mean;
  2. that pi/mu represents population proportion/mean;
  3. that the goal of a confidence interval is to estimating the unknown value of pi/mu with a high level of confidence;
  4. that the confidence interval uses p-hat/x-bar as its midpoint and then extends a certain amount on either side of that midpoint.

Each of these four aspects was worth one point.  The first two of these should have been easy points.  Most students earned these points successfully, but some did not.  For example, one student wrote that p-hat represents a population proportion and pi represents a population mean.

For the third component, I awarded a half-point for conveying the idea that a confidence interval estimates the value of pi/mu.  The word “estimates” was not needed for this half-point.  Many students earned this half-point with fairly loose language such as “the confidence interval is for mu.”  The other half-point was for communicating the idea that the value of the parameter is unknown, or estimated with a high level of confidence.  This half-point proved elusive for many students.

Students could earn a half-point for the fourth component by saying that the confidence interval is calculated from the value of the statistic.  The response needed to mention the midpoint, which most responses failed to do, in order to earn full credit.

I had also wanted to insist upon a fifth aspect for full credit.  I had hoped that strong responses would say something about “proportion of the sample having a characteristic of interest” or “sample mean value for the variable of interest.”  But very few responses included something along these lines, so I decided against requiring it.

I was skeptical about whether this question would provide helpful information about students’ understanding, but I decided that it worked well.  Grading the question was not easy, but I think the four aspects described above provided a good rubric.  When I use a variation of this question again, I might explicitly say not to use formulas as part of the response, and I also might say that responses should be limited to 3-5 sentences.

Here is one of six versions of another question on my students’ final exam:

2a. Suppose that the manager of a Walmart store collects data on the following variables for a random sample of transactions/receipts at the store:

  • Total amount spent
  • Number of items purchased
  • Day of week
  • Time of day (morning, afternoon, evening)
  • Payment type (credit card, cash, other)

a) State a research question that could be addressed by applying analysis of variance (ANOVA) to (some of) the data. 

b) State an additional variable for which data could be collected, and classify it as categorical or numerical.

Two other versions presented similar scenarios followed by the same questions (a) and (b):

2b. Suppose that a restaurant manager collects data on the following variables for a random sample of parties who dine at the restaurant:

  • Total amount spent on meal
  • Time of day (breakfast, lunch, dinner)
  • Day of week
  • Amount spent on drinks
  • Number of people in the party
  • Number of children (younger than age 18) in the party

2c. Suppose that a hotel manager collects data on the following variables for a random sample of customers’ stays at the hotel:

  • Number of people staying in the room
  • Distance from their home
  • Total amount spent at the hotel during the stay
  • Type of reservation (online, telephone, none)
  • Day of week on which stay began

The other three versions arose by repeating the same scenarios and variables, but with simple linear regression replacing ANOVA as the procedure in in part (a).

I often give my students practice with identifying which procedure is the relevant one to address a particular research question.  In fact, we spent the last day of class this term doing nothing else, as we discussed 15 questions for which my students were to identify the appropriate analysis procedure.  I always tell my students that the key to identifying the correct procedure is to identify the variables and their types.

This final exam question asks students to do the opposite: state a research question for which a particular procedure would be appropriate.  The same key applies here.  For example, students needed to realize that ANOVA applies when the explanatory variable is categorical and the response variable is numerical.  With that in mind, a reasonable answer for part (a) of version 2a is: “do Walmart customers tend to spend different amounts on their transaction, on average, depending on whether they shop in the morning, afternoon, or evening?”

Coming up with a research question is often challenging for students.  I made it easier this time by presenting many variables to them.  I suspect that part (a) of this question would have been substantially harder if students had needed to think of variables for themselves. 

Part (b) is meant to be fairly easy, but some students struggle with the ideas of observational units and variables despite my emphasizing those ideas frequently.  Two common, correct answers for the Walmart scenario have been:

  • the amount of time spent in the store prior to completing the transaction, which is numerical
  • whether the transaction was completed with a cashier or self-service, which is categorical

This question was worth four points, two points for each part.  Students generally did very well on this question.  I graded fairly strictly; incorrect responses received zero points.  For example, an answer of “does payment type help to predict total amount spent?” for the regression version of the question earned zero points, because the explanatory variable given is categorical, not numerical.  Examples of incorrect responses for part (b) often followed from mis-understanding the observational units, such as “how many customers shopped at Walmart that day?” and “what part of the country was the Walmart located in?”

For essentially correct responses with poor or unclear wording, I deducted a half-point.  For example, some students answered the regression version of part (a) with: “what is the correlation between number of items and total amount paid?”  I deducted a half-point for this response, on the grounds that there’s a lot more to regression than calculating the value of a single statistic.  I also deducted a half-point for using causal language inappropriately, for example by answering the ANOVA version of part (a) with: “does type of payment affect total amount spent?”

In hindsight, I wish that I had worded these questions a bit more clearly myself.  I should have been more clear that responses to part (a) were to be based only on the variables that I presented.  Part (b) could have been more clear by specifying that the variable proposed needed to be based on the same observational units as the ones presented.

I provide my students with practice questions before midterm exams but not for the final exam, mostly because I try to keep final exam questions secure.  But I might consider providing these questions to students before the final exam in the future, to help them understand my advice about focusing on the big picture.  The drawback is that I’ll then have to come up with new and better questions to use on the final exam.

#89 An exam question

It’s hard to imagine a more boring title for a blog post, isn’t it?  I’m going to present an open-ended, five-part exam question that I used in the past week.  I will describe my thought process behind writing and grading the question, and I will discuss what I learned from common student responses.  I think the question turned out to be quite revealing, so I hope that this post will turn out to be less boring that its title and first paragraph.

This was my third exam of the term.  I was not entirely pleased with how the first two exams worked out.  In hindsight the first exam was too hard, the second one too easy.  I was really hoping for a Goldilocks result (just right) for the third exam.  It can be quite challenging to write and grade exams, and assessments in general, that distinguish between students with a very thorough understanding of fundamental ideas from those with a modest level of understanding.

The topic of this exam question is multiple regression.  I do not teach this topic very often, so I have not developed a large bank of questions that I like to pose.  Also, I am less aware of common student misunderstandings than I am with more introductory topics.  I spent a lot of time writing this exam, and now I am taking a break from grading it* to write this post.

* In post #66 (here), I proposed that the first step of grading exams is: Procrastinate!

This question is based on the same dataset about prices for pre-owned Prius cars that I described in post #86 (here).  My students should have been familiar with the dataset from the assignment that I described in that post.  But in that assignment I asked students to predict a car’s price based on a single variable: its age in years, or its number of miles driven.  For this exam question, I asked students to consider a multiple regression model for predicting price from both age and miles.

The exam question presented students with output but did not provide the datafile or ask students to analyze the data themselves.  Students were allowed to use their notes on the exam.  Here’s the background and output for the question:

Consider the following output from a multiple regression model that uses both age (in years) and number of miles driven to predict price (in dollars), based on a sample of 32 pre-owned Prius cars advertised for sale in February of 2021:

Now I will present and discuss one part of the question at a time.  The entire question was worth 10 points, on a 40-point exam.  Each of the five parts was worth 2 points.

a) Write out the regression equation for predicting price from the two predictor variables.

This is as basic as it gets, right?  I would not quite consider this part as free points, but I intended this to provide two easy points to students who simply learned how to read computer output well enough to express a regression equation.  The correct answer is: predicted price = 22,076.66 – 0.0619 × miles – 579.21 × age

Most, but not all, of my students earned full credit for this part.  The most common error surprised me a bit: neglecting to include the left-hand side of the equation.  Several students only wrote: 22,076.66 – 0.0619 × miles – 579.21 × age.  I don’t like to be a stickler for mathematical notation, but omitting the response variable strikes me as failing to communicate that the goal of this regression analysis is to predict price.  I deducted a half-point for this error.

A few students wrote: The regression equation = 22,076.66 – 0.0619 × miles – 579.21 × age.  I also deducted a half-point for this, because of the missing response variable.  But I did give full credit to responses that included a colon rather than an equal sign: predicted price: 22,076.66 – 0.0619 × miles – 579.21 × age.

I usually insist on using the word predicted or a carat (“hat”) symbol with the response variable, but this time I did not deduct a half-point for omitting that.

b) Identify and interpret the value of the residual standard error.

Almost all students identified the correct value in the output, the root mean square error value of 1841.708.  A few students mistakenly answered the standard error of the intercept term, 651.9749.

I was looking for an interpretation along the lines of: A typical predicted price from this model differs from the actual price of a Prius in this sample by about $1841.71.  I realize that “typical” is a vague word, but using a more precise word like “average” is not technically correct.  I did award full credit to students who used “average” or “on average” in their response, though.

This question makes me worry that I am rewarding students simply for copying phrases from their notes without thinking.  (In fact, one student expressed the interpretation in terms of age and number of bidders for an auction of grandfather clocks, which was one of the examples we had worked through in class.)  But I hope that students demonstrate some understanding by selecting the correct interpretation and also by revising the generic interpretation to fit the context.

Some students mistakenly said that the residual standard error is a typical amount by which the regression line deviates from predicted values.  I did not penalize them for referring to a line instead of a plane, but I did deduct a half-point for not talking about deviations from the actual prices.

A few students did not use the measurement units (dollars) in their interpretation.  I only deducted a half-point once if they also failed to mention dollars in their response to part (c). 

I did not ask for an interpretation of R2 in this question, only because I asked for that on the previous exam that included simple linear regression among its topics.

c) Interpret what the value -579.2092 means in this context.

This is the coefficient of the age variable.  I was looking for students to say something like: The predicted price of a Prius decreases by about $579.21 for each additional year of age on the car, after accounting for number of miles driven. Another version is: Among pre-owned Prius cars with the same number of miles, we predict the price to decrease by about $579.21 for each additional year of age.

Many students neglected to include the caveat about accounting for the number of miles driven.  This is the key difference between interpreting coefficients with multiple versus simple regression.  Such responses earned 1 of 2 points.  Some gave a more generic interpretation that mentioned accounting for all other variables.  I deducted a half-point for this, on the grounds that this response did not describe context fully. 

A few students did not include direction (decreased) in their interpretation, and some did not express the “for each additional year of age” part of the interpretation clearly.  Each of these errors earned a half-point deduction.

d) JMP produced the following under “Mean Confidence Interval” with a setting of 95%, for input values of 5 and 50,000: ($15,321, $16,854).  Interpret what this interval means.

I really wrestled with how to word this question.  My main goal was to assess whether a student can distinguish between a confidence interval for a mean, as opposed to a prediction interval for an individual observation.  I worried that I was giving too much away by using the word mean in my statement about the output.  But I couldn’t figure out how else to identify which confidence interval I was providing.

I need not have worried.  Many of my students interpreted this interval in terms of the price of an individual car.  Such a response earned 1 of 2 points, if the other components of the response were correct.  Of course, I don’t know whether such responses indicated a lack of understanding or simply poor communication by omitting the word mean or average.  Needless to say, there’s a big difference between an average and an individual value.  I regret that so many of my students failed to answer this part correctly, but this is a big idea that is worthy of assessing, so I’m glad that I asked the question.

When writing this part of the question, I also struggled with how to express that the confidence interval was generated for 5-year-old cars with 50,000 miles.  At the last minute, I decided to make that wording more vague by simply referring to input values of 5 and 50,000.  I figured that I could reasonably expect students to realize that 5 referred to age in years, and 50,000 pertained to miles. 

I’m glad that I made this change, because it revealed that some of my students did not understand that these were inputs for two different predictor variables.  A few responses talked about cars with between 5 and 50,000 miles.

I was surprised by a somewhat common error in which students did not refer to the input values at all. Several responses interpreted the interval as estimating the population mean price of all pre-owned Prius cars listed for sale online in February 2021, with no regard for the car’s age or number of miles.

A few students made clear that they thought the interpretation applied to a sample mean rather than a population mean.  I only deducted a half-point for this error, if the rest of the interpretation was fine, because they at least recognized that the interval estimates a mean rather than an individual value.

e) How would you respond to someone who says: “Age and miles must be related to each other, because older cars have been driven for more miles than newer cars.  Therefore, it’s not necessary or helpful to include both age and miles together in a model for predicting price.”

This is, by far, my favorite part of this question.  I think this gets addresses a very important aspect of multiple regression analysis: investigating whether including an additional variable is worthwhile to include in a model.

I wanted students to notice that individual t-tests for both predictor variables produce very large (in absolute value) test statistics and therefore very small p-values: t = -5.77 and t = -4.83 for miles and age, respectively, with p-values considerably less than 0.0001.  Those test results reveal that each variable is helpful to include in the model, even in the presence of the other.  Even though age and miles may very well be positively correlated, the individual t-tests reveal that both variables are worth including in the model for predicting a car’s price.

Again I struggled mightily with how to word this part of the question.  In particular, I debated with myself about whether to prompt students to refer to output in their response.  As you can see, I decided against including that, and I’m glad that I did. 

Many students did not refer to output at all.  I think it’s telling that these students opted to rely on their own impressions of the context rather than look at what the data revealed.  In the words of David Moore: Data beat anecdotes.  I’m pleased that I asked this question in a way that assessed whether students would look to data, rather than their own opinions or suppositions, to answer the question.  I graded this fairly harshly: Students who did not refer to output could only earn 0.5/2 points for this part. 

Some students used this part of the question to remind me of some things I had said in class.  For example, several repeated my comment that fitting regression models is an inexact science, and a few cited George Box’s famous saying: All models are wrong; some models are useful.  I’m glad that this saying made enough of an impression that some students wanted to write it on an exam, but I wish that they had instead looked at the data, as reflected in the output provided.

I suspect that few students have any idea how much time and thought goes into writing and grading exam questions.  Speaking of which, I need to get back to grading the second question on this exam.  I’ll spare you a 2000-word analysis* of that one.

* Actually, in case you are keeping track,, I believe that this post fell just short of containing 2000 words, until this sentence put it over the top.

#88 It’s about time, part 2

In last week’s post (here), I presented some examples and questions through which I introduce my students to time series data.  I left off with a bit of a cliff-hanger, as I presented the following graph of national average prices, by month, for a gallon of unleaded gasoline:

As you can see, this series begins in January of 1981, during my first year of college*, and concludes in January of 2021, the first year of college for most of my current students. 

* I mentioned last time that January of 1981 feels like a frighteningly long time ago.  I’m sorry to report that this feeling has not subsided in the past week.  In fact, January of 1981 feels even longer ago this week than it did last week.

The national average price of a gallon of unleaded gasoline increased from $1.298 in January of 1981 to $2.326 forty years later.  I ask my students: Calculate the percentage increase.  This works out to be a (2.326 – 1.298) / 1.298 × 100% ≈ 79.2% increase.  Then I ask: Does gasoline really cost this much more now than in my first year of college?  This is where last week’s post ended.

The answer I am seeking for that question is: Not really.  To which I respond with: Why not?  Because a dollar was worth more, in terms of what it could buy, back in 1981 than it is worth now.  This realization leads into the topic of converting from current to constant dollars, also known as adjusting for inflation.

I have to admit that this is one of my favorite topics to teach.  I truly feel like I’m teaching my students a valuable skill when I show them how to adjust and compare monetary values at different points in time.  Why do I feel somewhat guilty about enjoying this topic?  Because it’s not really statistical.  I have learned to assuage my guilt with three thoughts:

  1. Analyzing time series data about prices that cover multiple years requires adjusting for inflation to make meaningful comparisons.
  2. The U.S. Bureau of Labor Statistics (BLS) uses lots of statistical methods, primarily associated with sampling, to determine the Consumer Price Index (CPI) on which such adjustments depend.
  3. If I’m helping my students to learn an idea and skill that are valuable, interesting, and even fun, who cares about whether it’s labeled as math or statistics or something else?

The Consumer Price Index (CPI) is based on prices for items that most people buy on a regular basis, gathered from urban areas around the U.S.  Monthly values of the CPI back to January of 1913 can be found from the BLS site here*.  The key idea for converting a monetary value from one time to its equivalent value at another time is to multiply by the ratio of the CPI values: value at time B = value at time A × (CPI at time B) / (CPI at time A).

* I provide a link to an Excel file containing these monthly CPI values at the end of this post.

Convert the $1.298 average national price of a gallon of gasoline in January 1981 into the equivalent price in January 2021.  The value of the CPI was 87.0 in January 1981 and 261.582 in January 2021.  This conversion is: $1.298 × (261.582/87.0) ≈ $1.298 × 3.007 ≈ $3.903. 

Interpret what the converted value means.  In terms of the buying power of today’s (well, January 2021’s) dollar, the amount of $1.298 in January of 1981 is equivalent to $3.903 now. 

In which month – January 1981 or January 2021 – did it really cost more for a gallon of unleaded gasoline, after adjusting for inflation?  Explain your answer.  The January 2021 average price of $2.326 is considerably less than the January 1981 price after its conversion to the equivalent of January 2021 dollars.  So, gasoline actually cost more, in terms of the buying power of currency at the time, in January 1981 than in January 2021.

Calculate the percentage difference of the two prices, after adjusting for inflation, using January 1981 as the baseline.  Also write a sentence interpreting this value.  This calculation is:(2.326 – 3.903) / 3.903 × 100% ≈ -40.4%.  In terms of constant dollars, the January 2021 price is about 40.4% less than the January 1981 price.

Next I ask students: Convert the entire time series of gasoline prices into constant dollars as of January 2021.  Produce a graph of these converted prices, along with the original prices.  Comment on what the graph reveals about how the price of gasoline has changed over these four decades. 

This conversion is conceptually straight-forward: We simply need to apply the same calculation as for January 1981 to all 481 months in the series.  This task requires using software and is well-suited for a spreadsheet package such as Excel*.

* The data on gasoline prices can be found in an Excel file at the end of this post.

On the left is the formula for using Excel to perform the conversion of the January 1981 price into constant dollar as of January 2021.  On the right is the result of entering that formula:

Filling that formula down for the entire column produces the following results at the bottom:

I often encourage students to ask questions of themselves to check their work.  A good example is: Does the adjusted price for January 2021 make sense?  Yes!  Because we’re converting all prices to constant dollars as of January 2021, the price should (and does) stay the same for that month.

Produce graphs of these two series for easy comparison. This results in:

Describe what the graph reveals. Whereas the original series (in blue) shows a fairly constant price for gasoline through the 1980s and 1990s, the series of converted prices (in orange) shows that the inflation-adjusted prices decreased in these decades.  Both series reveal an increase in the price of gasoline in the first decade of the 2000s, aside from a fairly dramatic price drop in 2008.  Notice that the two series converge in the 2010s, because much less of an adjustment for inflation is needed as the time gets closer to the present.

I also like to ask students to use CPI values to calculate and compare inflation rates by decade.  First: Starting with the 1910s and ending with the 2010s, which decade do you predict to have the lowest rate of inflation?  Which do you predict to have the highest rate of inflation?  Then I give them the following table and ask: Calculate the inflation rate for each decade.  If they need a hint: Calculate the percentage change in the CPI for each decade.

Here are the inflation rates for these decades:

Describe how the inflation rate has varied over the past ten decades.  The 1920s and 1930s experienced negative inflation.  Inflation surged in the 1940s and then slowed in the 1950s and 1960s.  Inflation exploded in the 1970s, as the CPI more than doubled in that decade.  Since then, the inflation rate has decreased more and more with each passing decade.

Now for some exam questions that I have asked on this topic:

My annual salary when I began my career as a college professor in September of 1989 (when the CPI was 125.00) was $27,000.  If my salary kept pace with inflation but otherwise did not increase, what would my salary be today (as of January 2021)?  This calculation is straight-forward: $27,000 × (261.582 / 125.0) ≈ $27,000 × 2.093 ≈ $56,501.71.

In the television series The Rockford Files, private investigator Jim Rockford charged $200/day for his services in the year 1975 (when the CPI was 53.8).  In the novel P is for Peril by Sue Grafton, detective Kinsey Millhone charged $400/day for her services in the year 1986 (in which the CPI was 109.6).  After adjusting for inflation, who charged more for their daily fee – Rockford or Millhone?  Justify your answer with a sentence accompanied by appropriate calculations.

This question is more challenging than the previous one, because it does not specifically ask students to perform a particular price adjustment.  Students need to decide for themselves what to calculate to answer this question.  Several reasonable options are available.  Students could convert Rockford’s fee into constant 1986 dollars, or they could convert Millhone’s fee into constant 1975 dollars.  A third option is to convert both fees into constant dollars for some other time, such as January 2021.  This gives $200 × (261.582 / 53.8) ≈ $972.42 for Rockford’s fee in constant January 2021 dollars, compared to $400 × (261.582 / 109.6) ≈ $954.68 for Millhone’s fee.  These are remarkably similar, but Rockford’s fee is slightly larger than Millhone’s after converting to comparable dollars.

Finally, mostly for fun but also to award a point for paying minimal attention, I sometimes ask: What does CPI stand for?  [Options: Consumer price index, Capital product inflation, Cats project integrity]

I always find adjusting for inflation to be a fun topic to teach, worthwhile for students to learn.  I take advantage of a brief unit on time series to sneak this topic into my course.  I also enjoy the opportunity to give students practice with basic spreadsheet skills.  I hope their quantitative and computational skills will help them to earn starting salaries that exceed $27,000 from 1989, even after adjusting for inflation.

P.S. Files containing data on CPI values and gasoline prices can be accessed from the links below: