Since being forced to administer exams on the LMS I have been including questions using the "select all that apply" option. Some students--and not just the worst performing ones who benefit most from guessing--have expressed strong objections to multiple answer questions as opposed to standard multiple choice questions. Thoughts?
Quote from: Anon1787 on November 09, 2020, 09:34:39 PM
Since being forced to administer exams on the LMS I have been including questions using the "select all that apply" option. Some students--and not just the worst performing ones who benefit most from guessing--have expressed strong objections to multiple answer questions as opposed to standard multiple choice questions. Thoughts?
For the most part, I'd tell them to suck it up. (Nicely, and professionally, of course.)
However, if the LMS grades 'all-or-nothing', I'm more sympathetic to complaints. My LMS does partial credit - gives credit for correct selections and takes credit away for incorrect selections. I wouldn't use this type of question if partial credit weren't an option.
I use these questions but I manually grade them on anything but homework (homework they can just hit "try again"). The computer does all or nothing when grading them.
They tend to be worth about half as many points as there are answer selections and I remove one point (bottoming out at 0) for each incorrectly togged one. So if there are 4 answers, 2 of which are correct, it's a 2 point question. You get 0 for leaving it blank, 0 for choosing all, and 1/2 if you choose one of the two answers that apply but not the other.
Quote from: kiana on November 10, 2020, 04:31:09 AM
I use these questions but I manually grade them on anything but homework (homework they can just hit "try again"). The computer does all or nothing when grading them.
They tend to be worth about half as many points as there are answer selections and I remove one point (bottoming out at 0) for each incorrectly togged one. So if there are 4 answers, 2 of which are correct, it's a 2 point question. You get 0 for leaving it blank, 0 for choosing all, and 1/2 if you choose one of the two answers that apply but not the other.
I used to do one point for each option toggled correctly. A question with five options (I often included none of the above and used it not infrequently) would be worth five points and students could see that upfront. However, one has to toggle everything correctly to get full credit. An incorrect answer toggled to selected is a point lost as is a correct answer not selected. By selecting everything, one generally got about half correct, but a 50% is not a passing grade.
Per the other thread, eventually I went to having an explanation for all options that were selected and then a selected answer with an incorrect explanation was only half a point--to get the whole point, a student had to give a correct explanation. Blindly guessing then yielded about 25%, which is really not passing.
My guess as to why they don't like it is that they have honed their strategies for regular multiple choice (eliminating answers, etc.) through SAT prep courses and so forth, and don't like being forced to actually learn the material.
From me, such comments only earn students a mini-lecture on the science of learning and memory and the purpose of testing, and an offer to discuss their study strategies for the next exam.
It's a valid and commonly used assessment measure. If you want to use it, it's totally fine.
Otherwise, the only meaningful feedback I can think to offer is to make sure that for any assessment measure that you use, that you clearly articulate the instructions to your students on how to correctly complete and submit the assessment. For assessments operated within an LMS system, extra troubleshooting on the professor's end is often needed to ensure that things work correctly.
Quote from: Anon1787 on November 09, 2020, 09:34:39 PM
Since being forced to administer exams on the LMS I have been including questions using the "select all that apply" option. Some students--and not just the worst performing ones who benefit most from guessing--have expressed strong objections to multiple answer questions as opposed to standard multiple choice questions. Thoughts?
I beg of you to send the following email:
Students have which of the following valid options if they object to multiple-answer questions:
A. Continue guessing at answers in their usual manner
B. Learn the course material to gain confidence in their answers
C. Hold-in their breaths indefinitely until the professor changes the test format
D. Participate in Zoom study sessions and chapter reviews
E. Offer the professor $100 per question to change the format of test questions
F. A, B, and C
G. A, B, and D
H. C and E
I. A, B, C, D, and E
J. None of the aboveOkay, I think I'm going to have to double-dog dare you to send the email. :)
Chuckle.
^Like.
M.
Quote from: Anon1787 on November 09, 2020, 09:34:39 PM
Since being forced to administer exams on the LMS I have been including questions using the "select all that apply" option. Some students--and not just the worst performing ones who benefit most from guessing--have expressed strong objections to multiple answer questions as opposed to standard multiple choice questions. Thoughts?
I've used those question types and the "partial credit" option in the CMS. I like Polly's idea of 5 options = it's worth 5 points, 1 for each correctly toggled response. I've seen colleagues try this in a really messy way where the penalty for guessing can give you a negative score. Or where each correct response is worth 2/3 of a point so you get scores like 76.6666666%
Figure out the logistics behind the scoring first, make them clear to the students, and use them. Do not tell them how many choices are correct or students will waste time trying to figure out if it's worth guessing or not (or if there is no penalty for incorrect answers, they will just pick all of the choices).
It's totally fine, just harder.
Like Polly, I give one point per correct answer.
If you use these, I strongly suggest grading them yourself. Canvas has these but the penalty for choosing an incorrect answer is worth as much as the bonus for choosing a correct answer. So if you choose one correct and one incorrect, you earn no points.
I stopped using them and instead started including standard one choice MC with way more than 5 options, along the lines of what fishprof posted above.
I have been using the partial credit option on the LMS and the instructions to every quiz/test mentions that fact. The grade grubber has generously given me the option to replace multiple answer questions with short answer or multiple choice questions. But if there are good pedagogical reasons for not using multiple answer questions, I am content to stop using them.
I'm sorely tempted to send fishbrains' email.
Quote from: Anon1787 on November 10, 2020, 01:01:59 PM
I have been using the partial credit option on the LMS and the instructions to every quiz/test mentions that fact. The grade grubber has generously given me the option to replace multiple answer questions with short answer or multiple choice questions. But if there are good pedagogical reasons for not using multiple answer questions, I am content to stop using them.
I'm sorely tempted to send fishbrains' email.
I had an entire class (OK, it was probably several loud, pushy students, but it felt like the whole class) give me significant push back on multiple answer. I was using Polly's grading method. So, the next test, I converted them all of the multiple answer to sets of true/false questions.
So, instead of a multiple answer question like this:
What are the correct descriptions of the cat on my desk (choose all that apply):
A. The cat is sleeping
B. The cat is black.
C. The cat is laying on top of my grading pile
D. The cat responds to her name.
(A, B, and C are correct)
I wrote sets of true false questions:
This set of questions is about the cat on my desk:
The cat on my desk is sleeping: T/F
The cat on my desk is black: T/F
The cat on my desk is laying on top of my grading pile: T/F
The cat on my desk response to her name: T/F
I'm pretty sure they still thought I was mean, but nobody complained about the true-false questions.
There must be some way to characterize those differences....that's really interesting.
Is it the Piagetian thing of being able to do one thing at a time (i.e., get each T/F distinction separately) but not combining them (i.e., several descriptions may apply together rather than just singly to the same situation?)...so, additive but not multiplicative thinking, maybe?
There's something like that going on, it seems like....very insightful that you were able to convert the question like that and then compare the responses.
M.
Quote from: OneMoreYear on November 10, 2020, 02:09:31 PM
Quote from: Anon1787 on November 10, 2020, 01:01:59 PM
I have been using the partial credit option on the LMS and the instructions to every quiz/test mentions that fact. The grade grubber has generously given me the option to replace multiple answer questions with short answer or multiple choice questions. But if there are good pedagogical reasons for not using multiple answer questions, I am content to stop using them.
I'm sorely tempted to send fishbrains' email.
I had an entire class (OK, it was probably several loud, pushy students, but it felt like the whole class) give me significant push back on multiple answer. I was using Polly's grading method. So, the next test, I converted them all of the multiple answer to sets of true/false questions.
So, instead of a multiple answer question like this:
What are the correct descriptions of the cat on my desk (choose all that apply):
A. The cat is sleeping
B. The cat is black.
C. The cat is laying on top of my grading pile
D. The cat responds to her name.
(A, B, and C are correct)
I wrote sets of true false questions:
This set of questions is about the cat on my desk:
The cat on my desk is sleeping: T/F
The cat on my desk is black: T/F
The cat on my desk is laying on top of my grading pile: T/F
The cat on my desk response to her name: T/F
I'm pretty sure they still thought I was mean, but nobody complained about the true-false questions.
The mathematically inclined will quickly realize that the multiple-choice question with five non-exclusive answers is really five true-false questions on the same subject. That should be fine as long as the instructor realizes that is what they are doing and structure the grading accordingly.
Fishbrains has a completely different model, with only one of the ten answers being the correct one.
Quote from: Hibush on November 10, 2020, 03:26:43 PM
Quote from: OneMoreYear on November 10, 2020, 02:09:31 PM
Quote from: Anon1787 on November 10, 2020, 01:01:59 PM
I have been using the partial credit option on the LMS and the instructions to every quiz/test mentions that fact. The grade grubber has generously given me the option to replace multiple answer questions with short answer or multiple choice questions. But if there are good pedagogical reasons for not using multiple answer questions, I am content to stop using them.
I'm sorely tempted to send fishbrains' email.
I had an entire class (OK, it was probably several loud, pushy students, but it felt like the whole class) give me significant push back on multiple answer. I was using Polly's grading method. So, the next test, I converted them all of the multiple answer to sets of true/false questions.
So, instead of a multiple answer question like this:
What are the correct descriptions of the cat on my desk (choose all that apply):
A. The cat is sleeping
B. The cat is black.
C. The cat is laying on top of my grading pile
D. The cat responds to her name.
(A, B, and C are correct)
I wrote sets of true false questions:
This set of questions is about the cat on my desk:
The cat on my desk is sleeping: T/F
The cat on my desk is black: T/F
The cat on my desk is laying on top of my grading pile: T/F
The cat on my desk response to her name: T/F
I'm pretty sure they still thought I was mean, but nobody complained about the true-false questions.
The mathematically inclined will quickly realize that the multiple-choice question with five non-exclusive answers is really five true-false questions on the same subject. That should be fine as long as the instructor realizes that is what they are doing and structure the grading accordingly.
Fishbrains has a completely different model, with only one of the ten answers being the correct one.
Yes, in the way I was grading it, it was the same (multiple answer vs sets of t/f). The students complained less, the grades improved (slightly), and my chair thought I was "responsive to student feedback," despite the fact that my student eval scores did not significantly raise because no one wanted to be taking my class.
I do think it was because I was forcing them to think about each statement individually, rather than as a group, which, for these students, was probably the correct scaffolding.
Quote from: sprout on November 09, 2020, 11:32:10 PM
Quote from: Anon1787 on November 09, 2020, 09:34:39 PM
Since being forced to administer exams on the LMS I have been including questions using the "select all that apply" option. Some students--and not just the worst performing ones who benefit most from guessing--have expressed strong objections to multiple answer questions as opposed to standard multiple choice questions. Thoughts?
For the most part, I'd tell them to suck it up. (Nicely, and professionally, of course.)
However, if the LMS grades 'all-or-nothing', I'm more sympathetic to complaints. My LMS does partial credit - gives credit for correct selections and takes credit away for incorrect selections. I wouldn't use this type of question if partial credit weren't an option.
This. It's not the same, if say, out of a list of languages from which they're asked to choose all the Romance languages, they select all the right ones save Catalan or if they select only Mandarin and Japanese.
Test takes are rational.
Assume five possible answers, without all of the above and none of the above.
Without "Select all that apply", the test taker has a 20% chance of getting the right answer.
With "Select all that apply", the probability of guessing correctly is tiny.
I have used the "select all that apply" option in Canvas for formative quizzes that students can take multiple times, with only the highest grade counting.
In general, I prefer to use "which of the following statements is not true" rather than traditional multiple choice questions. That way, if there are five prompts, the student is reading and hopefully reinforcing four correct statements and at most one incorrect one, rather than vice versa (which is rehearsing bad information).
If you want to rehearse only good information in a given question, you can throw in a "none of the above is not true" option on occasion.
I also made students pick T/F and then explain their answers, half credit for correct option and half credit for good logic in the explanation. Trying-hard students appreciated the possibility of partial credit for a good explanation on a confusing-to-them statement. Gaming students tended to be angry about guessing resulting in poor grades.
I've never written these kinds of questions, but for what it's worth, I like the suggestion that one answer option = one point, and I like the true/false suggestion as a way to help students feel better about being tested on basically the same information. On my own exams, I sometimes ask students to pick the best answer, so they know there's only one correct option but multiple options could seem correct in one way or another and they have to distinguish between them. For instance, the question might be something like "what is above people's heads," with answer options including "umbrellas" (kind of correct, because they are indeed sometimes above people's heads) and "the sky" (more correct, because it's always above people's heads). Or the question might be "what is on my desk" with answer options including "a water bottle" (kind of correct, because there is indeed a water bottle on my desk) and "a water bottle and a computer" (more correct, because both of those things are on my desk). If students are really revolting against multiple answer questions, this might be another format to test depth of knowledge that might not provoke such a strong negative response - or at least, my students haven't complained about this format.
I think that students are probably upset if picking an incorrect answer on the "select all that apply" can lower their score. Changing the questions to all True/False means that they have a 50% chance of a correct guess on each one, typically with no penalty.
Quote from: the_geneticist on November 12, 2020, 08:36:52 AM
I think that students are probably upset if picking an incorrect answer on the "select all that apply" can lower their score. Changing the questions to all True/False means that they have a 50% chance of a correct guess on each one, typically with no penalty.
Most people would want to correct that, otherwise by default everyone passes. (It's easy to correct any M/C, including T/F, so that the score is unbiased by guesses.)