#97 Some good news
In one of my classes last week, I realized that I was dutifully following a piece of advice that I gave in a blog post more than one year ago. Because I don’t always heed my own advice, I decided that this was noteworthy enough to warrant its own blog post.
In post #33 (here), I implored statistics teachers to use data, examples, activities, and assignments that reveal human progress. I provided many examples in that post and its follow-up (here), involving things such as increasing life expectancy and decreasing poverty rates around the world over the past few decades. Ironically, the global pandemic began to disrupt all of our lives shortly after those posts appeared.
I still believe that it’s worthwhile to make students aware of data that reveal good news and human progress. I recently read some news reports that inspired me to do this. I asked students in my Statistical Communication class* to read a pre-peer-review article about an experimental vaccine for malaria (here). This vaccine for malaria has the potential for a huge, positive impact on human welfare, particularly among children in Africa**.
* See post #94 (here) for another activity that I used in this class.
** The World Health Organization estimates that more than 400,000 people died of malaria in 2019, with African children the most vulnerable group (here).
How did I try to ask good questions of my students about the malaria vaccine article? To begin with, I gave them an online, auto-graded quiz consisting of ten questions. In fact, I think you’ll get a good sense of the article just by reading my quiz questions and knowing that the correct answer is the always the first option presented here:
- What is the World Health Organization’s goal for the efficacy of a malaria vaccine by the year 2030? [Options: 75%, 50%, 90%, 95%, 99%]
- In what country was this study conducted? [Options: Burkina-Faso, Kenya, United Kingdom, United States, India, Sweden]
- Was this an observational study or a randomized experiment? [Options: randomized experiment; observational study]
- How many treatment groups were used? [Options: three, two, four, five, six]
- What ages were the subjects in this study, when they were randomized into one of the treatment groups? [Options: 5-17 months, 5-17 years, 16 years and older]
- How many subjects were enrolled and received at least one vaccination? [Answer: 450]
- Which group was NOT blinded as to which subjects received which treatment? [Options: pharmacists, participants, participants’ families, local study team]
- Which of the following was NOT studied as a possible confounder? [Options: race, gender, age group, bed net use]
- Which of the following comes closest to the percentage of participants with adequate bed net use? [Options: 85%, 95%, 75%, 50%, 25%]
- What was the vaccine efficacy of the high-dose treatment after one year? [Answer: 77%]
Notice that the first and last of these ten questions reveal the great promise of this vaccine.
One purpose of these quiz questions is to guide students in their reading. I don’t mind if they look at the quiz questions in advance and if they refer to the quiz questions while they are reading. Of course, I hope that my students read the full article and don’t treat this as a scavenger hunt to find the answers to my questions. But I hope that my questions point students to some of the most important take-away points from the article.
—
Another purpose of the quiz is to prepare students for our in-class discussion. I admit that I do not feel very confident with leading class discussions, particularly via zoom. Two pieces of advice that I give myself here are to ask good questions* and give students time to discuss their answers in small groups first.
* You knew this was coming.
Because this class is about statistical communication rather than statistical concepts or methods, I posed this question to guide the discussion: What elements of the study should appear in a news article for the general public about this study? In addition to providing a list of these elements, I also asked students to classify each element that they proposed into one of three categories: (1) essential, (2) helpful but not essential, (3) unnecessary.
Before I assigned students to breakout rooms to discuss this in small groups, I decided that I should first provide a few examples of what I mean by elements of the study. I gave these three:
- where the study was conducted
- how many subjects participated
- affiliations of the researchers.
After a brief discussion, I used a zoom poll for students to vote on how they wanted to classify each of these three elements. A large majority voted that the first two items are essential. On the third item, the vote was about evenly split between the “helpful” and “unnecessary” categories. I chimed in that there were so many co-authors on the study that I would include at most the affiliation of the lead author in a news report.
Then I assigned students to discuss this question in breakout rooms with 3-4 students each. After fifteen minutes, we reconvened as a full class to discuss what they had come up with. I typed their suggestions into a file that I projected to their screens during class and then posted after class. For each element that they mentioned, we conducted a zoom poll to vote on whether the item was essential, helpful, or unnecessary. When the discussion began to lose steam, I consulted my own list that I had created before class. In both sections of the course, a few items on my list had not been mentioned by the students, so I offered them for consideration.
Some of the items that my students identified as essential included:
- background information about severity and consequences of malaria
- selection criteria, including ages of the participants
- when the study was conducted
- use of random assignment, blindness
- treatments used, including for the control group
- response variable studied (whether or not the person developed malaria)
- percentages in each group who developed malaria
- statistical significance of the results
Some elements that were identified as helpful but not essential include:
- charts or graphs of the results
- ethics permissions that were obtained
- adverse reactions that were studied
- potential confounding variables that were studied
- next steps to be taken
For the last five minutes of class, I asked a different question: What are some criteria for evaluating whether a news article provides a good report for the general public about this study? I suggested that the article should include all elements that we considered to be essential. Some suggestions from my students included:
- captures interest at the outset
- makes case for importance
- appears visually appealing
- uses appropriate language for general audience
- includes link to the source article
- states appropriate conclusions (not over- or under-stated) from study
- includes some, but not too many, specific statistics from the study
- interprets values correctly
- does not contain errors
That covers questions that I asked before and during class. Here’s the assignment that students are working on before our next class session: Find a news article about the malaria vaccine study that we discussed in class. Include a link to this article with your report. Identify which essential and helpful elements (as we identified in class) are included, and not included in the article. Then write a paragraph analyzing how well the article summarizes and presents the malaria vaccine study for the general public.
This malaria vaccine article could provide an worthwhile example in an introductory statistics course, as well. For example, you could discuss experimental design issues such as random assignment and double-blindness. The example also lends itself to analyzing categorical data, both descriptively and with a chi-square test to compare success proportions among three groups.
I’m fairly pleased with how this particular session of my Statistical Communication course went, for several reasons. Sometimes I feel guilty that I often present examples from before many of my students were born, so I’m delighted to focus on an article that made the news just a few weeks ago. I am also very happy to introduce students to a research study that holds promise for considerable progress about human welfare and global health. It’s very gratifying to show students that science and statistics can contribute to such hopeful developments. It’s also just plain fun to share such good news!
Ask Good Questions 