#75 More final exam questions

I gave my first asynchronous online final exam this past week.  I find writing online exams to be much more time-consuming and stressful than writing good, old-fashioned in-person exams*.  I’ve identified five aspects of writing online exams that take considerable time and effort:

1. Writing with good multiple-choice questions and answer options;
2. Creating multiple versions of most questions in an effort to reduce cheating;
3. Thinking of questions where googling does not provide much of an advantage;
4. Entering all of the questions into the format required by the learning management system;
5. Double- and triple- and quadruple-checking everything**

* I’m finding it hard to remember the days of photocopying exams and handing them to students on paper.

** I became obsessed with this last one, because typos and other errors are so much more problematic now than they used to be.  I may not remember photocopying, but I fondly recall the good old days when a student would point out a mistake and I simply had to say: Excuse me, class, please look on the board to see a correction for part c) of question #3.  I really stressed and lost sleep about this.  And somehow I still managed to mess up!  I’m embarrassed to report that despite my efforts, students found an error on both the Wednesday and Friday versions of my final exams.  I was especially grateful to the student who started the exam at 7am on Wednesday and let me know about the error as soon as she finished, so I was able to make the correction before most students began the exam.

Now I’m in the throes of grading.  You may know that when it comes to grading, I enjoy procrastination*.  But the timeline is tight because grades are due on Tuesday.  Without further preamble, I will now discuss some of the multiple-choice questions that I asked my students on this exam.  I will provide answers at the end.

* See post #66, First step of grading exams, here.

---

1. Suppose that you want to investigate whether Cal Poly students tend to watch more movies than Cal Poly faculty.  Would you collect data to investigate this question using random sampling, random assignment, or both? [Options: A) Random sampling only; B) Random assignment only; C) Both random sampling and random assignment]

I like this question because I try to emphasize the distinction between random sampling and random assignment.  This is a meant to be an easy question.  Students should realize that it’s not reasonable to randomly assign people to the roles of faculty or student.

2. Suppose that the nine current members of the U.S. Supreme Court are still the same nine members of the Supreme Court two years from now. Indicate how the following values will change from now until then (two years from now). a) Mean of ages; b) Standard deviation of ages; c) Median of ages; d) Inter-quartile range of ages [Options: A) Increase; B) Decrease; C) Remain the same]

This is also intended as an easy question.  The mean and median will increase by two years.  But as measures of variability, the standard deviation and inter-quartile range will not change when everyone becomes two years older.

3. a) Which would be larger – the mean weight of 10 randomly selected people, or the mean weight of 1000 randomly selected cats (ordinary domestic housecats)?  b) Which would be larger – the standard deviation of the weights of 1000 randomly selected people, or the standard deviation of the weights of 10 randomly selected cats (ordinary domestic housecats)? [Options: A) Cats; B) People]

I have written about this question before*.  Part (b) is very challenging for students.  Unfortunately, many students come to believe that a larger sample size produces a smaller standard deviation, without realizing that this result applies to the variability of a sample statistic, such as a sample mean, not to variability in the original measurements, such as weights of people and cats.

* See post #16, Questions about cats, here.

4. Suppose that a fair coin is flipped 10 times.  Which is more likely – that the flips result in 5 heads and 5 tails, or that the flips result in 6 of one outcome and 4 of the other? [Options: A) 5 of each; B) 6-4 split; C) These are equally likely.]

Students could answer this by calculating the relevant binomial probabilities.  But they might also realize the key point that a 6-4 split can happen in two different ways.  Even though a particular 6-4 split is less likely than a 5-5 result, a 6-4 split in either direction is more likely than a 5-5 result.  These probabilities turn out to be 0.246 for obtaining 5 heads and 5 tails, 0.410 for achieving a 6-4 split.

5. Suppose that Chiara has a 10% chance of making an error when she conducts a test. If she conducts 10 independent tests, which of the following is closest to the probability that she makes at least one error? [Options: A) 0.10; B) 0.25; C) 0.50; D) 0.65; E) 0.99]

I intend for students to perform the calculation: Pr(at least one error) = 1 – Pr(no errors) = 1 – (0.9)¹⁰ ≈ 0.651.  I chose options far enough apart that some students might use their intuition to determine the correct answer, if they realize that making at least one error would be more likely than not without being extremely likely.

---

6. The United States has about 330 million residents.  Suppose that you want to estimate the proportion of Americans who wore socks yesterday to within a margin-of-error of 3.5 percentage points with 95% confidence.  Which of the following is closest to the number of people that you would need to randomly sample? [Options: A) 30; B) 1000; C) 30,000; D) 1,000,000]

I also discussed this question, which I ask on every final exam, in post #21 here.  Influenced by the 330 million number, many students mistakenly believe that a sample size of 1 million, or at least 30 thousand, people is required.

7. Suppose that Carlos, Dwayne, and Elsa select separate and independent random samples of 50 Cal Poly students each.  They ask each student in the sample how much sleep they got last night, in minutes.  Then they calculate the average amount of sleep for the students in their sample.  How likely is it that Carlos, Dwayne, and Elsa obtain the same value for their sample average? [Options: A) This is very likely. B) There’s about a 50% chance of this. C) There’s a 1 in 3 chance of this. D) This is very unlikely.]

This question addresses the concept of sampling variability, which is even more fundamental than that of sampling distribution.  This is meant to be an easy question that students can answer based on their intuition or by remembering what we discovered when simulating the drawing of random samples with an applet such as this one (here) that randomly samples words from the Gettysburg Address.

8. Suppose that Yasmin and Jade want to select a random sample of San Luis Obispo county residents and ask each person whether or not they spent Thanksgiving in their own home.  Suppose also that Yasmin wants to estimate the population proportion to within ± 0.04 with 95% confidence, and Jade wants to estimate the population proportion to within ± 0.02 with 95% confidence.  Who would need to use a larger sample size?  (You need not calculate any sample sizes to answer this question.)  [Options: A) Jade; B) Yasmin; C) They would both need the same sample size.]

Here is another question for which students could spend a good bit of time performing calculations, but they’re better served by thinking this through.  They need only realize that obtaining a smaller margin-of-error requires a larger sample size.

9. Suppose that you conduct a hypothesis test about a population mean and calculate the t-test statistic to equal 0.68.  Which of the following is the best interpretation of this value?  [Options: A) If the null hypothesis were true, the probability would be 0.68 of obtaining a sample mean as far as observed from the hypothesized value of the population mean. B) The probability is 0.68 that the null hypothesis is true. C) The sample mean is 0.68 standard errors greater than the hypothesized value of the population mean. D) The sample mean is equal to 0.68 times the standard error.]

Students’ ability to interpret the value of a test statistic is worth assessing.  You no doubt realize that I purposefully chose a value less than 1 for the t-test statistic here, partly to see whether students might confuse the interpretation of a test statistic and a p-value.

10. Suppose that you take a random sample of 100 books from a large library.  For each of the following questions, indicate the appropriate inference procedure. a) How old, on average, is a book from this library? b) Are 75% of books in this library less than 20 years old? c) What percentage of books in this library contain fewer than 300 pages? d) How many pages, on average, are contained in a book from this library? e) What percentage of books in this library have been borrowed at least once in the past 10 years? [Options: A) z-interval for proportion; B) z-test for proportion; C) t-interval for mean; D) t-test for mean]

This series of questions is very similar to the questions that I discussed in last week’s post (A sneaky quiz, here), so my students should have expected questions of this type.  I think these questions are a bit harder than the ones I presented in class and on that quiz, though.  Parts (b) and (c) involve a categorical variable, but students might be tempted to think of a numerical variable because the context also refers to a book’s age and number of pages.

---

I’m selfishly glad that the time I invested into writing multiple-choice questions for my final exam has now served double-duty by providing me with the basis for this blog post.  But I really do need to get back to grading the open-ended questions …

P.S. The correct answers are: 1. A; 2. A, C, A, C; 3. B, B; 4. B; 5. D; 6. B; 7. D; 8. A; 9. C; 10. C, B, A, C, A.